Send to

Choose Destination
Comput Assist Surg (Abingdon). 2019 May 31:1-7. doi: 10.1080/24699322.2018.1560082. [Epub ahead of print]

Unsupervised binocular depth prediction network for laparoscopic surgery.

Xu K1,2, Chen Z1, Jia F2,3.

Author information

a School of Computer Science and Information Security , Guilin University of Electronic Technology , Guilin , China.
b Research Lab for Medical Imaging and Digital Surgery , Shenzhen Institutes of Advanced Technology, Chinese Academy of Sciences , Shenzhen , China.
c Shenzhen Key Laboratory of Minimally Invasive Surgical Robotics and System , Shenzhen Institutes of Advanced Technology, Chinese Academy of Sciences , Shenzhen , China.


Minimally invasive surgery (MIS) is characterized by less trauma, shorter recovery time, and lower postoperative infection rate. The two-dimensional (2D) laparoscopic imaging lacks depth perception and does not provide quantitative depth information, thereby limiting precise and complex surgical operations. Three-dimensional (3D) laparoscopic imaging provides surgeons depth perception. This study aims to 3D reconstruction of the surgical scene based on the disparity map generated by the depth estimation algorithm. An unsupervised learning autoencoder method was proposed to calculate the accurate disparity with a 101-layer residual convolutional network. The loss function included three parts: left-right consistency loss, structure similarity loss, and reconstruction error loss, the combination can improve reconstruction accuracy and robustness. The method was validated on a Hamlyn Center Laparoscopic/Endoscopic Video Dataset. The structural similarity index (SSIM) is 0.8349 ± 0.0523 and the peak signal-to-noise ratio (PSNR) is 14.4957 ± 1.9676. The depth prediction network has high accuracy and robustness. The average time to produce each disparity map is about 16 ms. The experimental result shows that the proposed depth estimation method can offer dense disparity map, and can meet surgical real-time requirement. Future work will focus on network structure optimization and loss function design, transfer learning to improve the robustness and accuracy further.


3D reconstruction; Depth estimation; laparoscopic surgery; unsupervised learning

Supplemental Content

Loading ...
Support Center