Automatic landmark detection and mapping for 2D/3D registration with BoneNet

The 3D musculoskeletal motion of animals is of interest for various biological studies and can be derived from X-ray fluoroscopy acquisitions by means of image matching or manual landmark annotation and mapping. While the image matching method requires a robust similarity measure (intensity-based) or an expensive computation (tomographic reconstruction-based), the manual annotation method depends on the experience of operators. In this paper, we tackle these challenges by a strategic approach that consists of two building blocks: an automated 3D landmark extraction technique and a deep neural network for 2D landmarks detection. For 3D landmark extraction, we propose a technique based on the shortest voxel coordinate variance to extract the 3D landmarks from the 3D tomographic reconstruction of an object. For 2D landmark detection, we propose a customized ResNet18-based neural network, BoneNet, to automatically detect geometrical landmarks on X-ray fluoroscopy images. With a deeper network architecture in comparison to the original ResNet18 model, BoneNet can extract and propagate feature vectors for accurate 2D landmark inference. The 3D poses of the animal are then reconstructed by aligning the extracted 2D landmarks from X-ray radiographs and the corresponding 3D landmarks in a 3D object reference model. Our proposed method is validated on X-ray images, simulated from a real piglet hindlimb 3D computed tomography scan and does not require manual annotation of landmark positions. The simulation results show that BoneNet is able to accurately detect the 2D landmarks in simulated, noisy 2D X-ray images, resulting in promising rigid and articulated parameter estimations.

First clinical experience with a novel, mobile cone-beam CT system for treatment quality assurance in brachytherapy

Background and purpose

On-site cone-beam computed tomography (CBCT) has gained in importance in adaptive brachytherapy during recent years. Besides treatment planning, there is increased need particularly for image-guidance during interventional procedures and for image-guided treatment quality assurance (QA). For this purpose, an innovative CBCT device was rolled out at our hospital as the first site worldwide. We present the first clinical images and experiences.

Materials and methods

The novel CBCT system is constructed of a 121 cm diameter ring gantry, and features a 43.2 × 43.2 cm² flat-panel detector, wireless remote-control via tablet-PC, and battery-powered maneuverability. Within the first months of clinical operation, we performed CBCT-based treatment QA for a total of 26 patients (8 with breast, 16 with cervix, and 2 with vaginal cancer). CBCT scans were analyzed regarding potential movements of implanted applicators in-situ during the brachytherapy course.

Results

With the presented device, treatment QA was feasible for the majority of patients. The CBCT scans of breast patients showed sufficient contrast between implanted catheters and tissue. For gynecologic patients, a distinct visualization of applicators was achieved in general. However, reasonable differentiations of organic soft tissues were not feasible.

Conclusion

The CBCT system allowed basic treatment QA measures for breast and gynecologic patients. For image-guidance during interventional brachytherapy procedures, the current image quality is not adequate. Substantial performance enhancements are required for intraoperative image-guidance.

Deformable registration of chest CT images using a 3D convolutional neural network based on unsupervised learning

Purpose

The deformable registration of 3D chest computed tomography (CT) images is one of the most important tasks in the field of medical image registration. However, the nonlinear deformation and large‐scale displacement of lung tissues caused by respiratory motion cause great challenges in the deformable registration of 3D lung CT images.

Materials and methods

We proposed an end‐to‐end fast registration method based on unsupervised learning, optimized the classic U‐Net, and added inception modules between skip connections. The inception module attempts to capture and merge information at different spatial scales to generate a high‐precision dense displacement vector field. To solve the problem of voxel folding in flexible registration, we put the Jacobian regularization term into the loss function to directly penalize the singularity of the displacement field during training to ensure a smooth displacement vector field. In the stage of data preprocessing, we segmented the lung fields to eliminate the interference of irrelevant information in the network during training. The existing publicly available datasets cannot implement model training. To alleviate the problem of overfitting caused by limited data resources being available, we proposed a data augmentation method based on the 3D‐TPS (3D thin plate spline) transform to expand the training data.

Results

Compared with the experimental results obtained by using the VoxelMorph deep learning method and registration packages, such as ANTs and Elastix, we achieved a competitive target registration error of 2.09 mm, an optimal Dice score of 0.987, and almost no folding voxels. Additionally, the proposed method was much faster than the traditional methods.

Conclusions

In this study, we have shown that the proposed method was efficient in 3D chest image registration. The promising results demonstrated that our method showed strong robustness in the deformable registration of 3D chest CT images.

Feasibility study of navigated endoscopy for the placement of high dose rate brachytherapy applicators in the esophagus and lung

PURPOSE

To evaluate the electromagnetic (EM) tracking of endoscopes and applicators as a method of positioning a high dose rate (HDR) luminal applicator.

METHOD

An anatomical phantom consisting of a rigid trachea and flexible esophagus was used to compare applicator placement measurements using EM tracking vs the traditional method using two-dimensional (2D) fluoroscopy and surface skin markers. The phantom included a tumor in the esophagus and several pairs of optically visible points inside the lumen that were used to simulate proximal and distal ends of tumors of varying lengths. The esophagus tumor and lung points were visible on a computed tomography (CT) image of the phantom, which was used as ground truth for the measurements. The EM tracking system was registered to the CT image using fiducial markers. A flexible endoscope was tracked using the EM system and the locations of the proximal and distal ends of the tumor identified and this position recorded. An EM-tracked applicator was then inserted and positioned relative to the tumor markings. The applicator path was mapped using the EM tracking. The gross tumor length (GTL) and the distance between the first dwell position and distal edge of tumor (offset) were measured using the EM tracking and 2D fluoroscopy methods and compared to the same measurements on the CT image.

RESULTS

The errors in GTL using EM tracking were on average -0.5 ± 1.7 mm and 0.7 ± 3.6 mm for esophagus and lung measurements, similar to errors measured using the 2D fluoroscopy method of -0.9 ± 1.2 mm and 3.4 ± 4.4 mm. Offset measurements were slightly larger while using EM tracking relative to the fluoroscopy method but these were not statistically significant.

CONCLUSIONS

Electromagnetic tracking for placement of lumen applicators is feasible and accurate. Tracking of the endoscope that is used to identify the proximal and distal ends of the tumor and of the applicator during insertion generates accurate three-dimensional measurements of the applicator path, GTL and offset. Guiding the placement of intraluminal applicators using EM navigation is potentially attractive for cases with complex insertions, such as those with nonlinear paths or multiple applicator insertions.