Probabilistic sparse matching for robust 3D/3D fusion in minimally invasive surgery

Joint estimation of depth and motion from a monocular endoscopy image sequence using a multi-loss rebalancing network

Building an in vivo three-dimensional (3D) surface model from a monocular endoscopy is an effective technology to improve the intuitiveness and precision of clinical laparoscopic surgery. This paper proposes a multi-loss rebalancing-based method for joint estimation of depth and motion from a monocular endoscopy image sequence. The feature descriptors are used to provide monitoring signals for the depth estimation network and motion estimation network. The epipolar constraints of the sequence frame is considered in the neighborhood spatial information by depth estimation network to enhance the accuracy of depth estimation. The reprojection information of depth estimation is used to reconstruct the camera motion by motion estimation network with a multi-view relative pose fusion mechanism. The relative response loss, feature consistency loss, and epipolar consistency loss function are defined to improve the robustness and accuracy of the proposed unsupervised learning-based method. Evaluations are implemented on public datasets. The error of motion estimation in three scenes decreased by 42.1%,53.6%, and 50.2%, respectively. And the average error of 3D reconstruction is 6.456 ± 1.798mm. This demonstrates its capability to generate reliable depth estimation and trajectory reconstruction results for endoscopy images and meaningful applications in clinical.

Registration of Magnetic Resonance Tomography (MRT) Data with a Low Frequency Adaption of Fourier-Mellin-SOFT (LF-FMS)

Fourier-Mellin-SOFT (FMS) is a rigid 3D registration method, which allows the robust registration of 3 degrees-of-freedom (dof) rotation, 1-dof scale, and 3-dof translation between scans on discrete grids. FMS is based on a spectral decomposition of these 7-dof. This complete spectral representation of the input data enables an adaption to certain frequency ranges. This special property is used here to focus on relevant mutual 3D information between bone structures with a Low Frequency adaptation of FMS (LF-FMS), that is, it is utilized for matching and concurrently determining corresponding transformation parameters. This process is applied on a set of Magnetic Resonance Tomography (MRT) data representing the hand region, in particular the carpal bone area, in a sequence of different hand positions. This data set is available for different probands, which allows a comparison of resulting parameter plots and furthermore matching in between bone structures.

An Overview on Image Registration Techniques for Cardiac Diagnosis and Treatment

Image registration has been used for a wide variety of tasks within cardiovascular imaging. This study aims to provide an overview of the existing image registration methods to assist researchers and impart valuable resource for studying the existing methods or developing new methods and evaluation strategies for cardiac image registration. For the cardiac diagnosis and treatment strategy, image registration and fusion can provide complementary information to the physician by using the integrated image from these two modalities. This review also contains a description of various imaging techniques to provide an appreciation of the problems associated with implementing image registration, particularly for cardiac pathology intervention and treatments.