Visualization of Deformable Image Registration Quality Using Local Image Dissimilarity

Enhancing unsupervised learning in medical image registration through scale-aware context aggregation

Deformable image registration (DIR) is essential for medical image analysis, facilitating the establishment of dense correspondences between images to analyze complex deformations. Traditional registration algorithms often require significant computational resources due to iterative optimization, while deep learning approaches face challenges in managing diverse deformation complexities and task requirements. We introduce ScaMorph, an unsupervised learning model for DIR that employs scale-aware context aggregation, integrating multiscale mixed convolution with lightweight multiscale context fusion. This model effectively combines convolutional networks and vision transformers, addressing various registration tasks. We also present diffeomorphic variants of ScaMorph to maintain topological deformations. Extensive experiments on 3D medical images across five applications-atlas-to-patient and inter-patient brain magnetic resonance imaging (MRI) registration, inter-modal brain MRI registration, inter-patient liver computed tomography (CT) registration as well as inter-modal abdomen MRI-CT registration-demonstrate that our model significantly outperforms existing methods, highlighting its effectiveness and broader implications for enhancing medical image registration techniques.

Estimating medical image registration error and confidence: A taxonomy and scoping review

Given that image registration is a fundamental and ubiquitous task in both clinical and research domains of the medical field, errors in registration can have serious consequences. Since such errors can mislead clinicians during image-guided therapies or bias the results of a downstream analysis, methods to estimate registration error are becoming more popular. To give structure to this new heterogenous field we developed a taxonomy and performed a scoping review of methods that quantitatively and automatically provide a dense estimation of registration error. The taxonomy breaks down error estimation methods into Approach (Image- or Transformation-based), Framework (Machine Learning or Direct) and Measurement (error or confidence) components. Following the PRISMA guidelines for scoping reviews, the 570 records found were reduced to twenty studies that met inclusion criteria, which were then reviewed according to the proposed taxonomy. Trends in the field, advantages and disadvantages of the methods, and potential sources of bias are also discussed. We provide suggestions for best practices and identify areas of future research.

Rapid Quality Assessment of Nonrigid Image Registration Based on Supervised Learning

When preprocedural images are overlaid on intraprocedural images, interventional procedures benefit in that more structures are revealed in intraprocedural imaging. However, image artifacts, respiratory motion, and challenging scenarios could limit the accuracy of multimodality image registration necessary before image overlay. Ensuring the accuracy of registration during interventional procedures is therefore critically important. The goal of this study was to develop a novel framework that has the ability to assess the quality (i.e., accuracy) of nonrigid multimodality image registration accurately in near real time. We constructed a solution using registration quality metrics that can be computed rapidly and combined to form a single binary assessment of image registration quality as either successful or poor. Based on expert-generated quality metrics as ground truth, we used a supervised learning method to train and test this system on existing clinical data. Using the trained quality classifier, the proposed framework identified successful image registration cases with an accuracy of 81.5%. The current implementation produced the classification result in 5.5 s, fast enough for typical interventional radiology procedures. Using supervised learning, we have shown that the described framework could enable a clinician to obtain confirmation or caution of registration results during clinical procedures.

Principles of Visualization in Radiation Oncology

BACKGROUND

Medical visualization employs elements from computer graphics to create meaningful, interactive visual representations of medical data, and it has become an influential field of research for many advanced applications like radiation oncology, among others. Visual representations employ the user's cognitive capabilities to support and accelerate diagnostic, planning, and quality assurance workflows based on involved patient data.

SUMMARY

This article discusses the basic underlying principles of visualization in the application domain of radiation oncology. The main visualization strategies, such as slice-based representations and surface and volume rendering are presented. Interaction topics, i.e., the combination of visualization and automated analysis methods, are also discussed. Key Messages: Slice-based representations are a common approach in radiation oncology, while volume visualization also has a long-standing history in the field. Perception within both representations can benefit further from advanced approaches, such as image fusion and multivolume or hybrid rendering. While traditional slice-based and volume representations keep evolving, the dimensionality and complexity of medical data are also increasing. To address this, visual analytics strategies are valuable, particularly for cohort or uncertainty visualization. Interactive visual analytics approaches represent a new opportunity to integrate knowledgeable experts and their cognitive abilities in exploratory processes which cannot be conducted by solely automatized methods.

High-order Feature Learning for Multi-atlas based Label Fusion: Application to Brain Segmentation with MRI

Multi-atlas based segmentation methods have shown their effectiveness in brain regions-of-interesting (ROIs) segmentation, by propagating labels from multiple atlases to a target image based on the similarity between patches in the target image and multiple atlas images. Most of the existing multiatlas based methods use image intensity features to calculate the similarity between a pair of image patches for label fusion. In particular, using only low-level image intensity features cannot adequately characterize the complex appearance patterns (e.g., the high-order relationship between voxels within a patch) of brain magnetic resonance (MR) images. To address this issue, this paper develops a high-order feature learning framework for multi-atlas based label fusion, where high-order features of image patches are extracted and fused for segmenting ROIs of structural brain MR images. Specifically, an unsupervised feature learning method (i.e., means-covariances restricted Boltzmann machine, mcRBM) is employed to learn high-order features (i.e., mean and covariance features) of patches in brain MR images. Then, a group-fused sparsity dictionary learning method is proposed to jointly calculate the voting weights for label fusion, based on the learned high-order and the original image intensity features. The proposed method is compared with several state-of-the-art label fusion methods on ADNI, NIREP and LONI-LPBA40 datasets. The Dice ratio achieved by our method is 88:30%, 88:83%, 79:54% and 81:02% on left and right hippocampus on the ADNI, NIREP and LONI-LPBA40 datasets, respectively, while the best Dice ratio yielded by the other methods are 86:51%, 87:39%, 78:48% and 79:65% on three datasets, respectively.

Quantitative error prediction of medical image registration using regression forests

Predicting registration error can be useful for evaluation of registration procedures, which is important for the adoption of registration techniques in the clinic. In addition, quantitative error prediction can be helpful in improving the registration quality. The task of predicting registration error is demanding due to the lack of a ground truth in medical images. This paper proposes a new automatic method to predict the registration error in a quantitative manner, and is applied to chest CT scans. A random regression forest is utilized to predict the registration error locally. The forest is built with features related to the transformation model and features related to the dissimilarity after registration. The forest is trained and tested using manually annotated corresponding points between pairs of chest CT scans in two experiments: SPREAD (trained and tested on SPREAD) and inter-database (including three databases SPREAD, DIR-Lab-4DCT and DIR-Lab-COPDgene). The results show that the mean absolute errors of regression are 1.07  ±  1.86 and 1.76  ±  2.59 mm for the SPREAD and inter-database experiment, respectively. The overall accuracy of classification in three classes (correct, poor and wrong registration) is 90.7% and 75.4%, for SPREAD and inter-database respectively. The good performance of the proposed method enables important applications such as automatic quality control in large-scale image analysis.