A CNN Regression Approach for Real-Time 2D/3D Registration

IEEE Trans Med Imaging. 2016 May;35(5):1352-1363. doi: 10.1109/TMI.2016.2521800. Epub 2016 Jan 26.

Abstract

In this paper, we present a Convolutional Neural Network (CNN) regression approach to address the two major limitations of existing intensity-based 2-D/3-D registration technology: 1) slow computation and 2) small capture range. Different from optimization-based methods, which iteratively optimize the transformation parameters over a scalar-valued metric function representing the quality of the registration, the proposed method exploits the information embedded in the appearances of the digitally reconstructed radiograph and X-ray images, and employs CNN regressors to directly estimate the transformation parameters. An automatic feature extraction step is introduced to calculate 3-D pose-indexed features that are sensitive to the variables to be regressed while robust to other factors. The CNN regressors are then trained for local zones and applied in a hierarchical manner to break down the complex regression task into multiple simpler sub-tasks that can be learned separately. Weight sharing is furthermore employed in the CNN regression model to reduce the memory footprint. The proposed approach has been quantitatively evaluated on 3 potential clinical applications, demonstrating its significant advantage in providing highly accurate real-time 2-D/3-D registration with a significantly enlarged capture range when compared to intensity-based methods.

MeSH terms

  • Arthroplasty, Replacement, Knee
  • Echocardiography, Transesophageal
  • Humans
  • Imaging, Three-Dimensional / methods
  • Knee Joint / diagnostic imaging
  • Machine Learning
  • Neural Networks, Computer*
  • Radiography / methods*
  • Regression Analysis