방해요소가 포함된 RGB-D 센서 기반 깊이 이미지에서 조선 조립공정의 수동 용접자세 인식

Abstract

It is a significant process to predict man-hours, a key factor in ship production planning, as accurately as possible. The proportion of welding work in total man-hours required for ship construction has been perceived to be large, and welding man-hours are greatly affected by working posture. Continuous research has been conducted to identify the posture in the welding operation by utilizing the relationship between man-hours and working posture. However, the results that quickly and conveniently reflect the effect of the welding posture on man-hours are not available. An important factor in recognizing working posture is that the measuring process does not interfere with the field workers, and the results of the working posture should be immediately available. Although studies on posture recognition based on depth image analysis are being positively reviewed, welding operation, unlike the other operations in the field, has difficulties in image interpretation because an external obstacle caused by arcs exists, and the depth image is consequently distorted. Therefore, for correct image analysis, any obstacle element must be removed in advance. This study proposes a method to acquire work postures using a low-cost RGB-D camera and recognize the welding position through image analysis. It searches for and removes obstacles that appear as depth holes in the depth image, and then restores the removed part to the desired state using the depth image's surrounding depth information. Following the establishment of the criteria for classifying the welding positions, the joint body coordinates are extracted from the restored image in order to identify the correct welding position. A Convolution Neural Network is used to train the established criteria, and the corresponding welding position is determined. The restored image showed significantly improved recognition accuracy than the image distorted by depth holes. The proposed method acquires, analyzes, and automates the recognition of welding positions in real-time. The principles used in the method can be applied to the welding field and all work areas where image interpretation is difficult due to obstacles.