A study on single enhancement and stereo vision for underwater imaging

Abstract

In this study, single enhancement and stereo vision technologies for underwater imaging were investigated. As unmanned automatic technologies are actively adopted for underwater works such as underwater construction and exploration, visualization of underwater situations is getting important for unmanned automation. An underwater stereo camera was developed for underwater imaging. Underwater stereo camera takes a optical image pair simultaneously with multiple camera lenses more than two and generates RGB-D point clouds expressing the exterior surface of the interesting objects. In contrast with the atmospheric environment, the underwater images are deteriorated with color casting and low visibility owing to the wavelength-dependent attenuation of visible light. Therefore, image enhancement is essential for underwater optical imaging to compensate the color casting and low visibility. Three underwater single enhancement methods, image fusion, cycle-consistent generative adversarial network (CycleGAN) and underwater generative adversarial network (UGAN) were considered. Image fusion is a three-step image processing of white balancing, enhancing the contrast and edge by contrast limited adaptive histogram equalization and unsharp masking principle methods, and fusing the enhanced images. Two GANs learns the specific features from datasets of clean and underwater images. First, CycleGAN which is constructed 9-residual network generator and patch GAN discriminator networks was trained with two image groups of 5000 clean images and 5000 underwater distorted images. And then, the paired training image datasets were generated by the trained CycleGAN. UGAN which is constructed U-Net generator and patch GAN discriminator was trained with the 5000 clean and underwater image pairs. The performances of three single enhancement methods were evaluated over 150 underwater images in terms of underwater images quality measurement (UIQM) and underwater color image quality evaluation (UCIQE). All single enhancement methods improved the colorness and the visibility of the underwater images with higher UIQM and UCIQE. CycleGAN resulted in the highest UIQM and UCIQE among the single enhancement methods. In this study, two stereo imaging systems were developed with a waterproof commercial stereo camera (ZED 2, STEREOLABS, France) and two underwater optical cameras (Eagle IPZ/4000, OTAQ, UK). Geometric calibration was performed to estimate the transformation matrices between two camera imaging coordinates of each stereo imaging systems. Using the camera parameters estimated by geometric calibration, the reprojected points from the image coordinate to world coordinate were well-matched to the real positions. Underwater stereo vision performs single enhancement on two time-synced underwater images, rectification, disparity map estimation and reconstruction of 3D point clouds. Rectification aligns two underwater images to place their pixels on the same y-axis based on epipolar constraint condition. Disparity map is estimated from the corresponding pixels on two images by semi-global matching algorithm. RGB-D 3D point clouds are reconstructed from the image coordinate and the estimated disparity values through the calibrated reprojection matrix. The reconstructed point clouds were evaluated in terms of the accuracy of the depth estimation and intensity recovery degree of the red color channel which is most attenuated by water. According to comparison on mean red intensities over the reconstructed point clouds corresponding to red plates of RGB phantom, it is confirmed that single enhancement is essential for underwater stereo imaging. However, single enhancement did not affect the accuracy on depth estimation. All three single enhancements improved qualitatively and quantitatively the underwater images, and CycleGAN trained with the unpaired clean and underwater image datasets achieved the best performance among single enhancement methods. For underwater stereo vision, single enhancement is an essential step to compensate the color of the reconstructed RGB-D point clouds, but it does not affect the accuracy on depth estimation.