VoxelMorph: A Learning Framework for Deformable Medical Image Registration

Deep Learning-based Aligned Strain from Cine Cardiac MRI for Detection of Fibrotic Myocardial Tissue in Patients with Duchenne Muscular Dystrophy

Purpose To develop a deep learning (DL) model that derives aligned strain values from cine (noncontrast) cardiac MRI and evaluate performance of these values to predict myocardial fibrosis in patients with Duchenne muscular dystrophy (DMD). Materials and Methods This retrospective study included 139 male patients with DMD who underwent cardiac MRI at a single center between February 2018 and April 2023. A DL pipeline was developed to detect five key frames throughout the cardiac cycle and respective dense deformation fields, allowing for phase-specific strain analysis across patients and from one key frame to the next. Effectiveness of these strain values in identifying abnormal deformations associated with fibrotic segments was evaluated in 57 patients (mean age [± SD], 15.2 years ± 3.1), and reproducibility was assessed in 82 patients by comparing the study method with existing feature-tracking and DL-based methods. Statistical analysis compared strain values using t tests, mixed models, and more than 2000 machine learning models; accuracy, F1 score, sensitivity, and specificity are reported. Results DL-based aligned strain identified five times more differences (29 vs five; P < .01) between fibrotic and nonfibrotic segments compared with traditional strain values and identified abnormal diastolic deformation patterns often missed with traditional methods. In addition, aligned strain values enhanced performance of predictive models for myocardial fibrosis detection, improving specificity by 40%, overall accuracy by 17%, and accuracy in patients with preserved ejection fraction by 61%. Conclusion The proposed aligned strain technique enables motion-based detection of myocardial dysfunction at noncontrast cardiac MRI, facilitating detailed interpatient strain analysis and allowing precise tracking of disease progression in DMD. Keywords: Pediatrics, Image Postprocessing, Heart, Cardiac, Convolutional Neural Network (CNN) Duchenne Muscular Dystrophy Supplemental material is available for this article. © RSNA, 2025.

FetDTIAlign: A deep learning framework for affine and deformable registration of fetal brain dMRI

Diffusion MRI (dMRI) offers unique insights into the microstructure of fetal brain tissue in utero. Longitudinal and cross-sectional studies of fetal dMRI have the potential to reveal subtle but crucial changes associated with normal and abnormal neurodevelopment. However, these studies depend on precise spatial alignment of data across scans and subjects, which is particularly challenging in fetal imaging due to the low data quality, rapid brain development, and limited anatomical landmarks for accurate registration. Existing registration methods, primarily developed for superior-quality adult data, are not well-suited for addressing these complexities. To bridge this gap, we introduce FetDTIAlign, a deep learning approach tailored to fetal brain dMRI, enabling accurate affine and deformable registration. FetDTIAlign integrates a novel dual-encoder architecture and iterative feature-based inference, effectively minimizing the impact of noise and low resolution to achieve accurate alignment. Additionally, it strategically employs different network configurations and domain-specific image features at each registration stage, addressing the unique challenges of affine and deformable registration, enhancing both robustness and accuracy. We validated FetDTIAlign on a dataset covering gestational ages centered between 23 and 36 weeks, encompassing 60 white matter tracts. For all age groups, FetDTIAlign consistently showed superior anatomical correspondence and the best visual alignment in both affine and deformable registration, outperforming two classical optimization-based methods and a deep learning-based pipeline. Further validation on external data from the Developing Human Connectome Project demonstrated the generalizability of our method to data collected with different acquisition protocols. Our results show the feasibility of using deep learning for fetal brain dMRI registration, providing a more accurate and reliable alternative to classical techniques. By enabling precise cross-subject and tract-specific analyses, FetDTIAlign paves the way for new discoveries in early brain development. The code is available at https://gitlab.com/blibli/fetdtialign.

MBSS-T1: Model-based subject-specific self-supervised motion correction for robust cardiac T1 mapping

Cardiac T1 mapping is a valuable quantitative MRI technique for diagnosing diffuse myocardial diseases. Traditional methods, relying on breath-hold sequences and cardiac triggering based on an ECG signal, face challenges with patient compliance, limiting their effectiveness. Image registration can enable motion-robust cardiac T1 mapping, but inherent intensity differences between time points pose a challenge. We present MBSS-T1, a subject-specific self-supervised model for motion correction in cardiac T1 mapping. Physical constraints, implemented through a loss function comparing synthesized and motion-corrected images, enforce signal decay behavior, while anatomical constraints, applied via a Dice loss, ensure realistic deformations. The unique combination of these constraints results in motion-robust cardiac T1 mapping along the longitudinal relaxation axis. In a 5-fold experiment on a public dataset of 210 patients (STONE sequence) and an internal dataset of 19 patients (MOLLI sequence), MBSS-T1 outperformed baseline deep-learning registration methods. It achieved superior model fitting quality (R2: 0.975 vs. 0.941, 0.946 for STONE; 0.987 vs. 0.982, 0.965 for MOLLI free-breathing; 0.994 vs. 0.993, 0.991 for MOLLI breath-hold), anatomical alignment (Dice: 0.89 vs. 0.84, 0.88 for STONE; 0.963 vs. 0.919, 0.851 for MOLLI free-breathing; 0.954 vs. 0.924, 0.871 for MOLLI breath-hold), and visual quality (4.33 vs. 3.38, 3.66 for STONE; 4.1 vs. 3.5, 3.28 for MOLLI free-breathing; 3.79 vs. 3.15, 2.84 for MOLLI breath-hold). MBSS-T1 enables motion-robust T1 mapping for broader patient populations, overcoming challenges such as suboptimal compliance, and facilitates free-breathing cardiac T1 mapping without requiring large annotated datasets. Our code is available at https://github.com/TechnionComputationalMRILab/MBSS-T1.

MedIAnomaly: A comparative study of anomaly detection in medical images

Anomaly detection (AD) aims at detecting abnormal samples that deviate from the expected normal patterns. Generally, it can be trained merely on normal data, without a requirement for abnormal samples, and thereby plays an important role in rare disease recognition and health screening in the medical domain. Despite the emergence of numerous methods for medical AD, the lack of a fair and comprehensive evaluation causes ambiguous conclusions and hinders the development of this field. To address this problem, this paper builds a benchmark with unified comparison. Seven medical datasets with five image modalities, including chest X-rays, brain MRIs, retinal fundus images, dermatoscopic images, and histopathology images, are curated for extensive evaluation. Thirty typical AD methods, including reconstruction and self-supervised learning-based methods, are involved in comparison of image-level anomaly classification and pixel-level anomaly segmentation. Furthermore, for the first time, we systematically investigate the effect of key components in existing methods, revealing unresolved challenges and potential future directions. The datasets and code are available at https://github.com/caiyu6666/MedIAnomaly.

Guided ultrasound acquisition for nonrigid image registration using reinforcement learning

We propose a guided registration method for spatially aligning a fixed preoperative image and untracked ultrasound image slices. We exploit the unique interactive and spatially heterogeneous nature of this application to develop a registration algorithm that interactively suggests and acquires ultrasound images at optimised locations (with respect to registration performance). Our framework is based on two trainable functions: (1) a deep hyper-network-based registration function, which is generalisable over varying location and deformation, and adaptable at test-time; (2) a reinforcement learning function for producing test-time estimates of image acquisition locations and adapted deformation regularisation (the latter is required due to varying acquisition locations). We evaluate our proposed method with real preoperative patient data, and simulated intraoperative data with variable field-of-view. In addition to simulation of intraoperative data, we simulate global alignment based on previous work for efficient training, and investigate probe-level guidance towards an improved deformable registration. The evaluation in a simulated environment shows statistically significant improvements in overall registration performance across a variety of metrics for our proposed method, compared to registration without acquisition guidance or adaptable deformation regularisation, and to commonly used classical iterative methods and learning-based registration. For the first time, efficacy of proactive image acquisition is demonstrated in a simulated surgical interventional registration, in contrast to most existing work addressing registration post-data-acquisition, one of the reasons we argue may have led to previously under-constrained nonrigid registration in such applications. Code: https://github.com/s-sd/rl_guided_registration.

Lightweight cross-resolution coarse-to-fine network for efficient deformable medical image registration

BACKGROUND

Accurate and efficient deformable medical image registration is crucial in medical image analysis. While recent deep learning-based registration methods have achieved state-of-the-art accuracy, they often suffer from extensive network parameters and slow inference times, leading to inefficiency. Efforts to reduce model size and input resolution can improve computational efficiency but frequently result in suboptimal accuracy.

PURPOSE

To address the trade-off between high accuracy and efficiency, we propose a Lightweight Cross-Resolution Coarse-to-Fine registration framework, termed LightCRCF.

METHODS

Our method is built on an ultra-lightweight U-Net architecture with only 0.1 million parameters, offering remarkable efficiency. To mitigate accuracy degradation resulting from fewer parameters while preserving the lightweight nature of the networks, LightCRCF introduces three key innovations as follows: (1) selecting an efficient cross-resolution coarse-to-fine (C2F) registration strategy and integrating it into the lightweight network to progressively decompose the deformation fields into multiresolution subfields to capture fine-grained deformations; (2) a Texture-aware Reparameterization (TaRep) module that integrates Sobel and Laplacian operators to extract rich textural information; (3) a Group-flow Reparameterization (GfRep) module that captures diverse deformation modes by decomposing the deformation field into multiple groups. Furthermore, we introduce a structural reparameterization technique that enhances training accuracy through multibranch structures of the TaRep and GfRep modules, while maintaining efficient inference by equivalently transforming these multibranch structures into single-branch standard convolutions.

RESULTS

We evaluate LightCRCF against various methods on the three public MRI datasets (LPBA, OASIS, and ACDC) and one CT dataset (abdomen CT). Following the previous data division methods, the LPBA dataset comprises 30 training image pairs and nine testing image pairs. For the OASIS dataset, the training, validation, and testing data consist of 1275, 110, and 660 image pairs, respectively. Similarly, for the ACDC dataset, the training, validation, and testing data include 180, 20, and 100 image pairs, respectively. For intersubject registration of the abdomen CT dataset, there are 380 training pairs, six validation pairs, and 42 testing pairs. Compared to state-of-the-art C2F methods, LightCRCF achieves comparable accuracy scores (DSC, HD95, and MSE), while demonstrating significantly superior performance across all efficiency metrics (Params, VRAM, FLOPs, and inference time). Relative to efficiency-first approaches, LightCRCF significantly outperforms these methods in accuracy metrics.

CONCLUSIONS

Our LightCRCF method offers a favorable trade-off between accuracy and efficiency, maintaining high accuracy while achieving superior efficiency, thereby highlighting its potential for clinical applications. The code will be available at https://github.com/PerceptionComputingLab/LightCRCF.

An end-to-end neural network for 4D cardiac CT reconstruction using single-beat scans

Objective.Motion artifacts remain a significant challenge in cardiac CT imaging, often impairing the accurate detection and diagnosis of cardiac diseases. These artifacts result from involuntary cardiac motion, and traditional mitigation methods typically rely on retrospective rescans, which increase radiation exposure and are less effective for patients with irregular heart rhythms. In this study, we proposed a deep learning-based end-to-end reconstruction framework for dynamic cardiac imaging using single-beat rapid CT scanning.Approach.The method used common cardiac CT projections and applied a sliding-window approach to divide the projection data into overlapping short-scan intervals centered on specific cardiac phases. Each short-scan interval was first reconstructed and then processed through a denoising network, before being fed into the registration module to compute deformation vector fields (DVFs). These DVFs were then used to perform motion-compensated reconstruction. The denoising and registration networks were trained end-to-end by minimizing the difference between the reconstructed images and ground truth in a supervised manner. The model was trained using simulated projection data from 30 real patients and validated on simulated datasets from different institutions and XCAT-generated continuous phantoms.Main results.Experimental results showed that the proposed method effectively reduced motion artifacts and restored key anatomical structures such as coronary arteries. On high-resolution test cases, SSIM improved from 0.7234 to 0.7795, peak signal-to-noise ratio from 35.40 to 37.58, and root mean square error (HU) decreased from 63.98 to 49.28. Additional evaluations showed consistent improvements in segmentation accuracy, with Dice similarity coefficient scores for the left ventricle, coronary arteries, and calcified plaques increasing from 0.8025, 0.7347, and 0.5966 to 0.9614, 0.8811, and 0.7774, respectively.Significance.By relying solely on single-cycle scan data and placing no explicit restrictions on heart rate, the method demonstrated strong potential for generalizability and wider clinical applications.

DSMR: Dual-Stream Networks with Refinement Module for Unsupervised Multi-modal Image Registration

 Multi-modal medical image registration aims to align images from different modalities to establish spatial correspondences. Although deep learning-based methods have shown great potential, the lack of explicit reference relations makes unsupervised multi-modal registration still a challenging task. In this paper, we propose a novel unsupervised dual-stream multi-modal registration framework (DSMR), which combines a dual-stream registration network with a refinement module. Unlike existing methods that treat multi-modal registration as a uni-modal problem using a translation network, DSMR leverages the moving, fixed and translated images to generate two deformation fields. Specifically, we first utilize a translation network to convert a moving image into a translated image similar to a fixed image. Then, we employ the dual-stream registration network to compute two deformation fields respectively: the initial deformation field generated from the fixed image and the moving image, and the translated deformation field generated from the translated image and the fixed image. The translated deformation field acts as a pseudo-ground truth to refine the initial deformation field and mitigate issues such as artificial features introduced by translation. Finally, we use the refinement module to enhance the deformation field by integrating registration errors and contextual information. Extensive experimental results show that our DSMR achieves exceptional performance, demonstrating its strong generalization in learning the spatial relationships between images from unsupervised modalities. The source code of this work is available at https://github.com/raylihaut/DSMR .

Two-stage color fundus image registration via Keypoint Refinement and Confidence-Guided Estimation

Color fundus images are widely used for diagnosing diseases such as Glaucoma, Cataracts, and Diabetic Retinopathy. The registration of color fundus images is crucial for assessing changes in fundus appearance to determine disease progression. In this paper, a novel two-stage framework is proposed for conducting end-to-end color fundus image registration without requiring any training or annotation. In the first stage, a pre-trained SuperPoint and SuperGlue network are used to obtain matching pairs, which are then refined based on their slopes. In the second stage, Confidence-Guided Transformation Matrix Estimation (CGTME) is proposed to estimate the final perspective transformation matrix. Specifically, a variant of 4-point algorithm, namely CG 4-point algorithm, is designed to adjust the contribution of matched points in estimating the perspective transformation matrix based on the confidence of SuperGlue. Then, we select the matched points with high confidence for the final estimation of transformation matrix. Experimental results show that our proposed algorithm can improve the registration performance effectively.

Shared subcortical arousal systems across sensory modalities during transient modulation of attention

Subcortical arousal systems are known to play a key role in controlling sustained changes in attention and conscious awareness. Recent studies indicate that these systems have a major influence on short-term dynamic modulation of visual attention, but their role across sensory modalities is not fully understood. In this study, we investigated shared subcortical arousal systems across sensory modalities during transient changes in attention using block and event-related fMRI paradigms. We analyzed massive publicly available fMRI datasets collected while 1,561 participants performed visual, auditory, tactile, and taste perception tasks. Our analyses revealed a shared circuit of subcortical arousal systems exhibiting early transient increases in activity in midbrain reticular formation and central thalamus across perceptual modalities, as well as less consistent increases in pons, hypothalamus, basal forebrain, and basal ganglia. Identifying these networks is critical for understanding mechanisms of normal attention and consciousness and may help facilitate subcortical targeting for therapeutic neuromodulation.

Towards automatic US-MR fetal brain image registration with learning-based methods

Fetal brain imaging is essential for prenatal care, with ultrasound (US) and magnetic resonance imaging (MRI) providing complementary strengths. While MRI has superior soft tissue contrast, US offers portable and inexpensive screening of neurological abnormalities. Despite the great potential synergy of combined fetal brain US and MR imaging to enhance diagnostic accuracy, little effort has been made to integrate these modalities. An essential step towards this integration is accurate automatic spatial alignment, which is technically very challenging due to the inherent differences in contrast and modality-specific imaging artifacts. In this work, we present a novel atlas-assisted multi-task learning technique to address this problem. Instead of training the registration model solely with intra-subject US-MR image pairs, our approach enables the network to also learn from domain-specific image-to-atlas registration tasks. This leads to an end-to-end multi-task learning framework with superior registration performance. Our proposed method was validated using a dataset of same-day intra-subject 3D US-MR image pairs. The results show that our method outperforms conventional optimization-based methods and recent learning-based techniques for rigid image registration. Specifically, the average target registration error for our method is less than 4 mm, which is significantly better than existing methods. Extensive experiments have also shown that our method has a much wider capture range and is robust to brain abnormalities. Given these advantages over existing techniques, our method is more suitable for deployment in clinical workflows and may contribute to streamlined multimodal imaging pipelines for fetal brain assessment.

TRUSTED: The Paired 3D Transabdominal Ultrasound and CT Human Data for Kidney Segmentation and Registration Research

Inter-modal image registration (IMIR) and image segmentation with abdominal Ultrasound (US) data have many important clinical applications, including image-guided surgery, automatic organ measurement, and robotic navigation. However, research is severely limited by the lack of public datasets. We propose TRUSTED (the Tridimensional Renal Ultra Sound TomodEnsitometrie Dataset), comprising paired transabdominal 3DUS and CT kidney images from 48 human patients (96 kidneys), including segmentation, and anatomical landmark annotations by two experienced radiographers. Inter-rater segmentation agreement was over 93% (Dice score), and gold-standard segmentations were generated using the STAPLE algorithm. Seven anatomical landmarks were annotated, for IMIR systems development and evaluation. To validate the dataset's utility, 4 competitive Deep-Learning models for kidney segmentation were benchmarked, yielding average DICE scores from 79.63% to 90.09% for CT, and 70.51% to 80.70% for US images. Four IMIR methods were benchmarked, and Coherent Point Drift performed best with an average Target Registration Error of 4.47 mm and Dice score of 84.10%. The TRUSTED dataset may be used freely to develop and validate segmentation and IMIR methods.

Wavelet-Guided Multi-Scale ConvNeXt for Unsupervised Medical Image Registration

Medical image registration is essential in clinical practices such as surgical navigation and image-guided diagnosis. The Transformer architecture of TransMorph demonstrates better accuracy in non-rigid registration tasks. However, its weaker spatial locality priors necessitate large-scale training datasets and a heavy number of parameters, which conflict with the limited annotated data and real-time demands of clinical workflows. Moreover, traditional downsampling and upsampling always degrade high-frequency anatomical features such as tissue boundaries or small lesions. We proposed WaveMorph, a wavelet-guided multi-scale ConvNeXt method for unsupervised medical image registration. A novel multi-scale wavelet feature fusion downsampling module is proposed by integrating the ConvNeXt architecture with Haar wavelet lossless decomposition to extract and fuse features from eight frequency sub-images using multi-scale convolution kernels. Additionally, a lightweight dynamic upsampling module is introduced in the decoder to reconstruct fine-grained anatomical structures. WaveMorph integrates the inductive bias of CNNs with the advantages of Transformers, effectively mitigating topological distortions caused by spatial information loss while supporting real-time inference. In both atlas-to-patient (IXI) and inter-patient (OASIS) registration tasks, WaveMorph demonstrates state-of-the-art performance, achieving Dice scores of 0.779 ± 0.015 and 0.824 ± 0.021, respectively, and real-time inference (0.072 s/image), validating the effectiveness of our model in medical image registration.

A critical assessment of artificial intelligence in magnetic resonance imaging of cancer

Given the enormous output and pace of development of artificial intelligence (AI) methods in medical imaging, it can be challenging to identify the true success stories to determine the state-of-the-art of the field. This report seeks to provide the magnetic resonance imaging (MRI) community with an initial guide into the major areas in which the methods of AI are contributing to MRI in oncology. After a general introduction to artificial intelligence, we proceed to discuss the successes and current limitations of AI in MRI when used for image acquisition, reconstruction, registration, and segmentation, as well as its utility for assisting in diagnostic and prognostic settings. Within each section, we attempt to present a balanced summary by first presenting common techniques, state of readiness, current clinical needs, and barriers to practical deployment in the clinical setting. We conclude by presenting areas in which new advances must be realized to address questions regarding generalizability, quality assurance and control, and uncertainty quantification when applying MRI to cancer to maintain patient safety and practical utility.

Decomposing the effect of normal aging and Alzheimer's disease in brain morphological changes via learned aging templates

Alzheimer's disease (AD) subjects usually show more profound morphological changes with time compared to cognitively normal (CN) individuals. These changes are the combination of two major biological processes: normal aging and AD pathology. Investigating normal aging and residual morphological changes separately can increase our understanding of the disease. This paper proposes two scores, the aging score (AS) and the AD-specific score (ADS), whose purpose is to measure these two components of brain atrophy independently. For this, in the first step, we estimate the atrophy due to the normal aging of CN subjects by computing the expected deformation required to match imaging templates generated at different ages. We used a state-of-the-art generative deep learning model for generating such imaging templates. In the second step, we apply deep learning-based diffeomorphic registration to align the given image of a subject with a reference imaging template. Parametrization of this deformation field is then decomposed voxel-wise into their parallel and perpendicular components with respect to the parametrization of the expected atrophy of CN individuals in one year computed in the first step. AS and ADS are the normalized scores of these two components, respectively. We evaluated these two scores on the OASIS-3 dataset with 1,014 T1-weighted MRI scans. Of these, 326 scans were from CN subjects, and 688 scans were from subjects diagnosed with AD at various stages of clinical severity, as defined by clinical dementia rating (CDR) scores. Our results reveal that AD is marked by both disease-specific brain changes and an accelerated aging process. Such changes affect brain regions differently. Moreover, the proposed scores were sensitive to detect changes in the early stages of the disease, which is promising for its potential future use in clinical studies. Our code is freely available at https://github.com/Fjr9516/DBM_with_DL .

Deep learning-based estimation of respiration-induced deformation from surface motion: A proof-of-concept study on 4D thoracic image synthesis

BACKGROUND

Four-dimension computed tomography (4D-CT) provides important respiration-related information for thoracic radiotherapy. Its quality is challenged by various respiratory patterns. Its acquisition gives rise to the risk of higher radiation exposure. Based on a continuously estimated deformation, a 4D synthesis by warping a high-quality volumetric image is a possible solution.

PURPOSE

To propose a non-patient-specific cascaded ensemble model (CEM) to estimate respiration-induced thoracic tissue deformation from surface motion.

METHODS

The CEM is cascaded by three deep learning-based models. By inputting the surface motion, CEM outputs a deformation vector field (DVF) inside thorax. In our work, the surface motion was simulated using the body contours derived from 4D-CT. The CEM was trained on our private database including 62 4D-CT sets, and was tested on a public database encompassing 80 4D-CT sets. To evaluate CEM, we employed the model output DVF to generate a few series of synthesized CTs, and compared them with the ground truth. CEM was also compared with other published works.

RESULTS

CEM synthesized CT with an mRMSE (average root mean square error) of 61.06 ± 10.43HU (average ± standard deviation), an mSSIM (average structural similarity index measure) of 0.990 ± 0.004, and an mMAE (average mean absolute error) of 26.80 ± 5.65HU. Compared with other works, CEM showed the best result.

CONCLUSIONS

The results demonstrated the effectiveness of CEM on estimating tissue DVF inside thorax. CEM requires no patient-specific breathing data sampling and no additional training before treatment. It shows potential for broad applications.

UAE: Universal Anatomical Embedding on multi-modality medical images

Identifying anatomical structures (e.g., lesions or landmarks) is crucial for medical image analysis. Exemplar-based landmark detection methods are gaining attention as they allow the detection of arbitrary points during inference without needing annotated landmarks during training. These methods use self-supervised learning to create a discriminative voxel embedding and match corresponding landmarks via nearest-neighbor searches, showing promising results. However, current methods still face challenges in (1) differentiating voxels with similar appearance but different semantic meanings (e.g., two adjacent structures without clear borders); (2) matching voxels with similar semantics but markedly different appearance (e.g., the same vessel before and after contrast injection); and (3) cross-modality matching (e.g., CT-MRI landmark-based registration). To overcome these challenges, we propose a Unified framework for learning Anatomical Embeddings (UAE). UAE is designed to learn appearance, semantic, and cross-modality anatomical embeddings. Specifically, UAE incorporates three key innovations: (1) semantic embedding learning with prototypical contrastive loss; (2) a fixed-point-based matching strategy; and (3) an iterative approach for cross-modality embedding learning. We thoroughly evaluated UAE across intra- and inter-modality tasks, including one-shot landmark detection, lesion tracking on longitudinal CT scans, and CT-MRI affine/rigid registration with varying fields of view. Our results suggest that UAE outperforms state-of-the-art methods, offering a robust and versatile approach for landmark-based medical image analysis tasks. Code and trained models are available at: https://github.com/alibaba-damo-academy/self-supervised-anatomical-embedding-v2.

CT material decomposition with contrast agents: Single or multiple spectral photon-counting CT scans? A simulation study

PURPOSE

With the widespread introduction of dual energy computed tomography (DECT), applications utilizing the spectral information to perform material decomposition became available. Among these, a popular application is to decompose contrast-enhanced CT images into virtual non-contrast (VNC) or virtual non-iodine images and into iodine maps. In 2021, photon-counting CT (PCCT) was introduced, which is another spectral CT modality. It allows for scans with more than two different detected spectra. With these systems, it becomes possible to distinguish more than two materials. It is frequently proposed to administer more than one contrast agent, perform a single PCCT scan, and then calculate the VNC images and the contrast agent maps. This may not be optimal because the patient is injected with a material, only to have it computationally extracted again immediately afterwards by spectral CT. It may be better to do an unenhanced scan followed by one or more contrast-enhanced scans. The main argument for the spectral material decomposition is patient motion, which poses a significant challenge for approaches involving two or more temporally separated scans. In this work, we assume that we can correct for patient motion and thus are free to scan the patient more than once. Our goal is then to quantify the penalty for performing a single contrast-enhanced scan rather than a clever series of unenhanced and enhanced scans. In particular, we consider the impact on patient dose and image quality.

METHODS

We simulate CT scans of three differently sized phantoms containing various contrast agents. We do this for a variety of tube voltage settings, a variety of patient-specific prefilter (PSP) thicknesses and a variety of threshold settings of the photon-counting detector with up to four energy bins. The reconstructed bin images give the expectation values of soft tissue and of the contrast agents. Error propagation of projection noise into the images yields the image noise. Dose is quantified using the total CT dose index (CTDI) value of the scans. When combining multiple scans, we further consider all possible tube current (or dose) ratios between the scans. Material decomposition is done image-based in a statistical optimal way. Error propagation into the material-specific images yields the signal-to-noise ratio at unit dose (SNRD). The winning scan strategy is the one with the highest total SNRD, which is related to the SNRD of the material that has the lowest signal-to-noise ratio (SNR) among the materials to decompose into. We consider scan strategies with up to three scans and up to three materials (water W, contrast agent X and contrast agent Y).

RESULTS

In all cases, those scan strategies yield the best performance that combine differently enhanced scans, for example, W+WX, W+WXY, WX+WXY, W+WX+WY, with W denoting an unenhanced scan and WX, WY and WXY denoting X-, Y-, and X-Y-enhanced scans, respectively. The dose efficiency of scans with a single enhancement scheme, such as WX or WXY, is far lower. The dose penalty to pay for these single enhancement strategies is about two or greater. Our findings also apply to scans with a single energy bin and thus also to CT systems with conventional, energy-integrating detectors, that is, conventional DECT. Dual source CT (DSCT) scans are preferable over single source CT scans, also because one can use a PSP on the high Kilovolt spectrum to better separate the detected spectra. For the strategies and tasks considered here, it does not make sense to simultaneously scan with two different types of contrast agents. Iodine outperforms other high Z elements in nearly all cases.

CONCLUSIONS

Given the significant dose penalty when performing only one contrast-enhanced scan rather than a series of unenhanced and enhanced scans, one should consider avoiding the single-scan strategies. This requires to invest in the development of accurate registration algorithms that can compensate for patient and contrast agent motion between separate scans.

Enhanced DTCMR With Cascaded Alignment and Adaptive Diffusion

Diffusion tensor cardiovascular magnetic resonance (DTCMR) is the only non-invasive method for visualizing myocardial microstructure, but it is challenged by inconsistent breath-holds and imperfect cardiac triggering, causing in-plane shifts and through-plane warping with an inadequate tensor fitting. While rigid registration corrects in-plane shifts, deformable registration risks distorting the diffusion distribution, and selecting a reference frame among low SNR frames is challenging. Existing pairwise deep learning and iterative methods are unsuitable for DTCMR due to their inability to handle the drastic in-plane motion and disentangle the diffusion contrast distortion with through-plane motions on low SNR frames, which compromises the accuracy of clinical biomarker tensor estimation. Our study introduces a novel deep learning framework incorporating tensor information for groupwise deformable registration, effectively correcting intra-subject inter-frame motion. This framework features a cascaded registration branch for addressing in-plane and through-plane motions and a parallel branch for generating pseudo-frames with diffusion contrasts and template updates to guide registration with a refined loss function and denoising. We evaluated our method on four DTCMR-specific metrics using data from over 900 cases from 2012 to 2023. Our method outperformed three traditional and two deep learning-based methods, achieving reduced fitting errors, the lowest percentage of negative eigenvalues at 0.446%, the highest R2 of HA line profiles at 0.911, no negative Jacobian Determinant, and the shortest reference time of 0.06 seconds per case. In conclusion, our deep learning framework significantly improves DTCMR imaging by effectively correcting inter-frame motion and surpassing existing methods across multiple metrics, demonstrating substantial clinical potential.

IVIM-Morph: Motion-compensated quantitative Intra-voxel Incoherent Motion (IVIM) analysis for functional fetal lung maturity assessment from diffusion-weighted MRI data

Quantitative analysis of pseudo-diffusion in diffusion-weighted magnetic resonance imaging (DWI) data shows potential for assessing fetal lung maturation and generating valuable imaging biomarkers. Yet, the clinical utility of DWI data is hindered by unavoidable fetal motion during acquisition. We present IVIM-morph, a self-supervised deep neural network model for motion-corrected quantitative analysis of DWI data using the Intra-voxel Incoherent Motion (IVIM) model. IVIM-morph combines two sub-networks, a registration sub-network, and an IVIM model fitting sub-network, enabling simultaneous estimation of IVIM model parameters and motion. To promote physically plausible image registration, we introduce a biophysically informed loss function that effectively balances registration and model-fitting quality. We validated the efficacy of IVIM-morph by establishing a correlation between the predicted IVIM model parameters of the lung and gestational age (GA) using fetal DWI data of 39 subjects. Our approach was compared against six baseline methods: (1) no motion compensation, (2) affine registration of all DWI images to the initial image, (3) deformable registration of all DWI images to the initial image, (4) deformable registration of each DWI image to its preceding image in the sequence, (5) iterative deformable motion compensation combined with IVIM model parameter estimation, and (6) self-supervised deep-learning-based deformable registration. IVIM-morph exhibited a notably improved correlation with gestational age (GA) when performing in-vivo quantitative analysis of fetal lung DWI data during the canalicular phase. Specifically, over 2 test groups of cases, it achieved an Rf2 of 0.44 and 0.52, outperforming the values of 0.27 and 0.25, 0.25 and 0.00, 0.00 and 0.00, 0.38 and 0.00, and 0.07 and 0.14 obtained by other methods. IVIM-morph shows potential in developing valuable biomarkers for non-invasive assessment of fetal lung maturity with DWI data. Moreover, its adaptability opens the door to potential applications in other clinical contexts where motion compensation is essential for quantitative DWI analysis. The IVIM-morph code is readily available at: https://github.com/TechnionComputationalMRILab/qDWI-Morph.

Generation of short-term follow-up chest CT images using a latent diffusion model in COVID-19

PURPOSE

Despite a global decrease in the number of COVID-19 patients, early prediction of the clinical course for optimal patient care remains challenging. Recently, the usefulness of image generation for medical images has been investigated. This study aimed to generate short-term follow-up chest CT images using a latent diffusion model in patients with COVID-19.

MATERIALS AND METHODS

We retrospectively enrolled 505 patients with COVID-19 for whom the clinical parameters (patient background, clinical symptoms, and blood test results) upon admission were available and chest CT imaging was performed. Subject datasets (n = 505) were allocated for training (n = 403), and the remaining (n = 102) were reserved for evaluation. The image underwent variational autoencoder (VAE) encoding, resulting in latent vectors. The information consisting of initial clinical parameters and radiomic features were formatted as a table data encoder. Initial and follow-up latent vectors and the initial table data encoders were utilized for training the diffusion model. The evaluation data were used to generate prognostic images. Then, similarity of the prognostic images (generated images) and the follow-up images (real images) was evaluated by zero-mean normalized cross-correlation (ZNCC), peak signal-to-noise ratio (PSNR), and structural similarity (SSIM). Visual assessment was also performed using a numerical rating scale.

RESULTS

Prognostic chest CT images were generated using the diffusion model. Image similarity showed reasonable values of 0.973 ± 0.028 for the ZNCC, 24.48 ± 3.46 for the PSNR, and 0.844 ± 0.075 for the SSIM. Visual evaluation of the images by two pulmonologists and one radiologist yielded a reasonable mean score.

CONCLUSIONS

The similarity and validity of generated predictive images for the course of COVID-19-associated pneumonia using a diffusion model were reasonable. The generation of prognostic images may suggest potential utility for early prediction of the clinical course in COVID-19-associated pneumonia and other respiratory diseases.

Symmetric deformable registration of multimodal brain magnetic resonance images via appearance residuals

BACKGROUND AND OBJECTIVE

Deformable registration of multimodal brain magnetic resonance images presents significant challenges, primarily due to substantial structural variations between subjects and pronounced differences in appearance across imaging modalities.

METHODS

Here, we propose to symmetrically register images from two modalities based on appearance residuals from one modality to another. Computed with simple subtraction between modalities, the appearance residuals enhance structural details and form a common representation for simplifying multimodal deformable registration. The proposed framework consists of three serially connected modules: (i) an appearance residual module, which learns intensity residual maps between modalities with a cycle-consistent loss; (ii) a deformable registration module, which predicts deformations across subjects based on appearance residuals; and (iii) a deblurring module, which enhances the warped images to match the sharpness of the original images.

RESULTS

The proposed method, evaluated on two public datasets (HCP and LEMON), achieves the highest registration accuracy with topology preservation when compared with state-of-the-art methods.

CONCLUSIONS

Our residual space-guided registration framework, combined with GAN-based image enhancement, provides an effective solution to the challenges of multimodal deformable registration. By mitigating intensity distribution discrepancies and improving image quality, this approach improves registration accuracy and strengthens its potential for clinical application.

Supra-second tracking and live-cell karyotyping reveal principles of mitotic chromosome dynamics

Mitotic chromosome dynamics are essential for the three-dimensional organization of the genome during the cell cycle, but the spatiotemporal characteristics of this process remain unclear due to methodological challenges. While Hi-C methods capture interchromosomal contacts, they lack single-cell temporal dynamics, whereas microscopy struggles with bleaching and phototoxicity. Here, to overcome these limitations, we introduce Facilitated Segmentation and Tracking of Chromosomes in Mitosis Pipeline (FAST CHIMP), pairing time-lapse super-resolution microscopy with deep learning. FAST CHIMP tracked all human chromosomes with 8-s resolution from prophase to telophase, identified 15 out of 23 homologue pairs in single cells and compared chromosomal positioning between mother and daughter cells. It revealed a centrosome-motion-dependent flow that governs the mapping between chromosome locations at prophase and their metaphase plate position. In addition, FAST CHIMP measured supra-second dynamics of intra- and interchromosomal contacts. This tool adds a dynamic dimension to the study of chromatin behaviour in live cells, promising advances beyond the scope of existing methods.

DELR-Net: a network for 3D multimodal medical image registration in more lightweight application scenarios

PURPOSE

3D multimodal medical image deformable registration plays a significant role in medical image analysis and diagnosis. However, due to the substantial differences between images of different modalities, registration is challenging and requires high computational costs. Deep learning-based registration methods face these challenges. The primary aim of this paper is to design a 3D multimodal registration network that ensures high-quality registration results while reducing the number of parameters.

METHODS

This study designed a Dual-Encoder More Lightweight Registration Network (DELR-Net). DELR-Net is a low-complexity network that integrates Mamba and ConvNet. The State Space Sequence Module and the Dynamic Large Kernel block are used as the main components of the dual encoders, while the Dynamic Feature Fusion block is used as the main component of the decoder.

RESULTS

This study conducted experiments on 3D brain MR images and abdominal MR and CT images. Compared to existing registration methods, DELR-Net achieved better registration results while maintaining a lower number of parameters. Additionally, generalization experiments on other modalities showed that DELR-Net has superior generalization capabilities.

CONCLUSION

DELR-Net significantly improves the limitations of 3D multimodal medical image deformable registration, achieving better registration performance with fewer parameters.

Robust Fast Inter-Bin Image Registration for Undersampled Coronary MRI Based on a Learned Motion Prior

OBJECTIVE

To propose a 3D nonrigid registration method that accurately estimates the 3D displacement field from artifact-corrupted Coronary Magnetic Resonance Angiography (CMRA) images.

METHODS

We developed a novel registration framework for registration of artifact-corrupted images based on a 3D U-Net initializer and a deep unrolling network. By leveraging a supervised learning framework with training labels estimated from fully-sampled images, the unrolling network learns a task-specific motion prior which reduces motion estimation biases caused by undersampling artifacts in the source images. We evaluated the proposed method, UNROLL, against an iterative Free-Form Deformation (FFD) registration method and a recently proposed Respiratory Motion Estimation network (RespME-net) for 6-fold (in-distribution) and 11-fold (out-of-distribution) accelerated CMRA.

RESULTS

Compared to the baseline methods, UNROLL improved both the accuracy of motion estimation and the quality of motion-compensated CMRA reconstruction at 6-fold acceleration. Furthermore, even at 11-fold acceleration, which was not included during training, UNROLL still generated more accurate displacement fields than the baseline methods. The computational time of UNROLL for the whole 3D volume was only 2 seconds.

CONCLUSION

By incorporating a learned respiratory motion prior, the proposed method achieves highly accurate motion estimation despite the large acceleration.

SIGNIFICANCE

This work introduces a fast and accurate method to estimate the displacement field from low-quality source images. It has the potential to significantly improve the quality of motion-compensated reconstruction for highly accelerated 3D CMRA.

Encoding matching criteria for cross-domain deformable image registration

BACKGROUND

Most existing deep learning-based registration methods are trained on single-type images to address same-domain tasks, resulting in performance degradation when applied to new scenarios. Retraining a model for new scenarios requires extra time and data. Therefore, efficient and accurate solutions for cross-domain deformable registration are in demand.

PURPOSE

We argue that the tailor-made matching criteria in traditional registration methods is one of the main reason they are applicable in different domains. Motivated by this, we devise a registration-oriented encoder to model the matching criteria of image features and structural features, which is beneficial to boost registration accuracy and adaptability.

METHODS

Specifically, a general feature encoder (Encoder-G) is proposed to capture comprehensive medical image features, while a structural feature encoder (Encoder-S) is designed to encode the structural self-similarity into the global representation. Moreover, by updating Encoder-S using one-shot learning, our method can effectively adapt to different domains. The efficacy of our method is evaluated using MRI images from three different domains, including brain images (training/testing: 870/90 pairs), abdomen images (training/testing: 1406/90 pairs), and cardiac images (training/testing: 64770/870 pairs). The comparison methods include traditional method (SyN) and cutting-edge deep networks. The evaluation metrics contain dice similarity coefficient (DSC) and average symmetric surface distance (ASSD).

RESULTS

In the single-domain task, our method attains an average DSC of 68.9%/65.2%/72.8%, and ASSD of 9.75/3.82/1.30 mm on abdomen/cardiac/brain images, outperforming the second-best comparison methods by large margins. In the cross-domain task, without one-shot optimization, our method outperforms other deep networks in five out of six cross-domain scenarios and even surpasses symmetric image normalization method (SyN) in two scenarios. By conducting the one-shot optimization, our method successfully surpasses SyN in all six cross-domain scenarios.

CONCLUSIONS

Our method yields favorable results in the single-domain task while ensuring improved generalization and adaptation performance in the cross-domain task, showing its feasibility for the challenging cross-domain registration applications. The code is publicly available at https://github.com/JuliusWang-7/EncoderReg.

Deep Learning-based Quantitative CT Myocardial Perfusion Imaging and Risk Stratification of Coronary Artery Disease

Background Precise assessment of myocardial ischemia burden and cardiovascular risk stratification based on dynamic CT myocardial perfusion imaging (MPI) is lacking. Purpose To develop and validate a deep learning (DL) model for automated quantification of myocardial blood flow (MBF) and ischemic myocardial volume (IMV) percentage and to explore the prognostic value for major adverse cardiovascular events (MACE). Materials and Methods This multicenter study comprised three cohorts of patients with clinically indicated CT MPI and coronary CT angiography (CCTA). Cohorts 1 and 2 were retrospective cohorts (May 2021 to June 2023 and January 2018 to December 2022, respectively). Cohort 3 was prospectively included (November 2016 to December 2021). The DL model was developed in cohort 1 (training set: 211 patients, validation set: 57 patients, test set: 90 patients). The diagnostic performance of MBF derived from the DL model (MBFDL) for myocardial ischemia was evaluated in cohort 2 based on the area under the receiver operating characteristic curve (AUC). The prognostic value of the DL model-derived IMV percentage was assessed in cohort 3 using multivariable Cox regression analyses. Results Across three cohorts, 1108 patients (mean age: 61 years ± 12 [SD]; 667 men) were included. MBFDL showed excellent agreement with manual measurements in the test set (segment-level intraclass correlation coefficient = 0.928; 95% CI: 0.921, 0.935). MBFDL showed higher diagnostic performance (vessel-based AUC: 0.97) over CT-derived fractional flow reserve (FFR) (vessel-based AUC: 0.87; P = .006) and CCTA-derived diameter stenosis (vessel-based AUC: 0.79; P < .001) for hemodynamically significant lesions, compared with invasive FFR. Over a mean follow-up of 39 months, MACE occurred in 94 (14.2%) of 660 patients. IMV percentage was an independent predictor of MACE (hazard ratio = 1.12, P = .003), with incremental prognostic value (C index: 0.86; 95% CI: 0.84, 0.88) over conventional risk factors and CCTA parameters (C index: 0.84; 95% CI: 0.82, 0.86; P = .02). Conclusion A DL model enabled automated CT MBF quantification and accurate diagnosis of myocardial ischemia. DL model-derived IMV percentage was an independent predictor of MACE and mildly improved cardiovascular risk stratification. © RSNA, 2025 Supplemental material is available for this article. See also the editorial by Zhu and Xu in this issue.

CorrMorph: Unsupervised Deformable Brain MRI Registration Based on Correlation Mining

Deformable image registration, as a fundamental prerequisite for many medical image analysis tasks, has received considerable attention. However, existing methods suffer from two key issues: 1) single-stream methods that stack moving and fixed images as input are prone to interference from spatial misalignment and style discrepancy, while dual-stream methods that use fully parallel encoders face challenges in learning correlations between images. 2) CNN-based methods are difficult to capture the complex spatial correspondences between images, while Transformer-based methods lack the ability to capture local context information. Therefore, we propose an unsupervised deformable brain MRI registration network, CorrMorph, which achieves reasonable and accurate registration by mining correlations. Specifically, we design a match-fusion strategy that allows the independent extraction of shallow features from the moving and fixed images while capturing their correlations in deeper layers. Furthermore, we propose two novel modules. 1) Correlation Matching Module (CMM), which mines correlations between images to achieve effective feature matching, 2) Feature Transmission Module (FTM), which extracts important spatial features to achieve effective feature transmission. Extensive experiments are conducted on three brain MRI datasets, and the results indicate that our method achieves state-of-the-art performance, with an average improvement of 2.7% on DSC compared to the representative VoxelMorph.

Self-supervised 3D medical image segmentation by flow-guided mask propagation learning

Despite significant progress in 3D medical image segmentation using deep learning, manual annotation remains a labor-intensive bottleneck. Self-supervised mask propagation (SMP) methods have emerged to alleviate this challenge, allowing intra-volume segmentation with just a single slice annotation. However, the previous SMP methods often rely on 2D information and ignore volumetric contexts. While our previous work, called Vol2Flow, attempts to address this concern, it exhibits limitations, including not focusing enough on local (i.e., slice-pair) information, neglecting global information (i.e., volumetric contexts) in the objective function, and error accumulation during slice-to-slice reconstruction. This paper introduces Flow2Mask, a novel SMP method, developed to overcome the limitations of previous SMP approaches, particularly Vol2Flow. During training, Flow2Mask proposes the Local-to-Global (L2G) loss to learn inter-slice flow fields among all consecutive slices within a volume in an unsupervised manner. This dynamic loss is based on curriculum learning to gradually learn information within a volume from local to global contexts. Additionally, the Inter-Slice Smoothness (ISS) loss is introduced as a regularization term to encourage changes between the slices occur consistently and continuously. During inference, Flow2Mask leverages these 3D flow fields for inter-slice mask propagation in a 3D image, spreading annotation from a single annotated slice to the entire volume. Moreover, we propose an automatic strategy to select the most representative slice as initial annotation in the mask propagation process. Experimental evaluations on different abdominal datasets demonstrate that our proposed SMP method outperforms previous approaches and improves the overall mean DSC of Vol2Flow by +2.1%, +8.2%, and +4.0% for the Sliver, CHAOS, and 3D-IRCAD datasets, respectively. Furthermore, Flow2Mask even exhibits substantial improvements in weakly-supervised and self-supervised few-shot segmentation methods when applied as a mask completion tool. The code and model for Flow2Mask are available at https://github.com/AdelehBitarafan/Flow2Mask, providing a valuable contribution to the field of medical image segmentation.

Feature-targeted deep learning framework for pulmonary tumorous Cone-beam CT (CBCT) enhancement with multi-task customized perceptual loss and feature-guided CycleGAN

Thoracic Cone-beam computed tomography (CBCT) is routinely collected during image-guided radiation therapy (IGRT) to provide updated patient anatomy information for lung cancer treatments. However, CBCT images often suffer from streaking artifacts and noise caused by under-rate sampling projections and low-dose exposure, resulting in loss of lung anatomy which contains crucial pulmonary tumorous and functional information. While recent deep learning-based CBCT enhancement methods have shown promising results in suppressing artifacts, they have limited performance on preserving anatomical details containing crucial tumorous information due to lack of targeted guidance. To address this issue, we propose a novel feature-targeted deep learning framework which generates ultra-quality pulmonary imaging from CBCT of lung cancer patients via a multi-task customized feature-to-feature perceptual loss function and a feature-guided CycleGAN. The framework comprises two main components: a multi-task learning feature-selection network (MTFS-Net) for building up a customized feature-to-feature perceptual loss function (CFP-loss); and a feature-guided CycleGan network. Our experiments showed that the proposed framework can generate synthesized CT (sCT) images for the lung that achieved a high similarity to CT images, with an average SSIM index of 0.9747 and an average PSNR index of 38.5995 globally, and an average Pearman's coefficient of 0.8929 within the tumor region on multi-institutional datasets. The sCT images also achieved visually pleasing performance with effective artifacts suppression, noise reduction, and distinctive anatomical details preservation. Functional imaging tests further demonstrated the pulmonary texture correction performance of the sCT images, and the similarity of the functional imaging generated from sCT and CT images has reached an average DSC value of 0.9147, SCC value of 0.9615 and R value of 0.9661. Comparison experiments with pixel-to-pixel loss also showed that the proposed perceptual loss significantly enhances the performance of involved generative models. Our experiment results indicate that the proposed framework outperforms the state-of-the-art models for pulmonary CBCT enhancement. This framework holds great promise for generating high-quality pulmonary imaging from CBCT that is suitable for supporting further analysis of lung cancer treatment.

Real-time spatiotemporal optimization during imaging

High quality imaging is required for high quality medical care, especially in precision applications such as radiation therapy. Patient motion during image acquisition reduces image quality and is either accepted or dealt with retrospectively during image reconstruction. Here we formalize a general approach in which data acquisition is treated as a spatiotemporal optimization problem to solve in real time so that the acquired data has a specific structure that can be exploited during reconstruction. We provide results of the first-in-world clinical trial implementation of our spatiotemporal optimization approach, applied to respiratory correlated 4D cone beam computed tomography for lung cancer radiation therapy (NCT04070586, ethics approval 2019/ETH09968). Performing spatiotemporal optimization allowed us to maintain or improve image quality relative to the current clinical standard while reducing scan time by 63% and reducing scan radiation by 85%, improving clinical throughput and reducing the risk of secondary tumors. This result motivates application of the general spatiotemporal optimization approach to other types of patient motion such as cardiac signals and other modalities such as CT and MRI.

TCDE-Net: An unsupervised dual-encoder network for 3D brain medical image registration

Medical image registration is a critical task in aligning medical images from different time points, modalities, or individuals, essential for accurate diagnosis and treatment planning. Despite significant progress in deep learning-based registration methods, current approaches still face considerable challenges, such as insufficient capture of local details, difficulty in effectively modeling global contextual information, and limited robustness in handling complex deformations. These limitations hinder the precision of high-resolution registration, particularly when dealing with medical images with intricate structures. To address these issues, this paper presents a novel registration network (TCDE-Net), an unsupervised medical image registration method based on a dual-encoder architecture. The dual encoders complement each other in feature extraction, enabling the model to effectively handle large-scale nonlinear deformations and capture intricate local details, thereby enhancing registration accuracy. Additionally, the detail-enhancement attention module aids in restoring fine-grained features, improving the network's capability to address complex deformations such as those at gray-white matter boundaries. Experimental results on the OASIS, IXI, and Hammers-n30r95 3D brain MR dataset demonstrate that this method outperforms commonly used registration techniques across multiple evaluation metrics, achieving superior performance and robustness. Our code is available at https://github.com/muzidongxue/TCDE-Net.

Deformable image registration with strategic integration pyramid framework for brain MRI

Medical image registration plays a crucial role in medical imaging, with a wide range of clinical applications. In this context, brain MRI registration is commonly used in clinical practice for accurate diagnosis and treatment planning. In recent years, deep learning-based deformable registration methods have achieved remarkable results. However, existing methods have not been flexible and efficient in handling the feature relationships of anatomical structures at different levels when dealing with large deformations. To address this limitation, we propose a novel strategic integration registration network based on the pyramid structure. Our strategy mainly includes two aspects of integration: fusion of features at different scales, and integration of different neural network structures. Specifically, we design a CNN encoder and a Transformer decoder to efficiently extract and enhance both global and local features. Moreover, to overcome the error accumulation issue inherent in pyramid structures, we introduce progressive optimization iterations at the lowest scale for deformation field generation. This approach more efficiently handles the spatial relationships of images while improving accuracy. We conduct extensive evaluations across multiple brain MRI datasets, and experimental results show that our method outperforms other deep learning-based methods in terms of registration accuracy and robustness.

Adaptive Matching of High-Frequency Infrared Sea Surface Images Using a Phase-Consistency Model

The sea surface displays dynamic characteristics, such as waves and various formations. As a result, images of the sea surface usually have few stable feature points, with a background that is often complex and variable. Moreover, the sea surface undergoes significant changes due to variations in wind speed, lighting conditions, weather, and other environmental factors, resulting in considerable discrepancies between images. These variations present challenges for identification using traditional methods. This paper introduces an algorithm based on the phase-consistency model. We utilize image data collected from a specific maritime area with a high-frame-rate surface array infrared camera. By accurately detecting images with identical names, we focus on the subtle texture information of the sea surface and its rotational invariance, enhancing the accuracy and robustness of the matching algorithm. We begin by constructing a nonlinear scale space using a nonlinear diffusion method. Maximum and minimum moments are generated using an odd symmetric Log-Gabor filter within the two-dimensional phase-consistency model. Next, we identify extremum points in the anisotropic weighted moment space. We use the phase-consistency feature values as image gradient features and develop feature descriptors based on the Log-Gabor filter that are insensitive to scale and rotation. Finally, we employ Euclidean distance as the similarity measure for initial matching, align the feature descriptors, and remove false matches using the fast sample consensus (FSC) algorithm. Our findings indicate that the proposed algorithm significantly improves upon traditional feature-matching methods in overall efficacy. Specifically, the average number of matching points for long-wave infrared images is 1147, while for mid-wave infrared images, it increases to 8241. Additionally, the root mean square error (RMSE) fluctuations for both image types remain stable, averaging 1.5. The proposed algorithm also enhances the rotation invariance of image matching, achieving satisfactory results even at significant rotation angles.

Artificial intelligence in musculoskeletal applications: a primer for radiologists

As an umbrella term, artificial intelligence (AI) covers machine learning and deep learning. This review aimed to elaborate on these terms to act as a primer for radiologists to learn more about the algorithms commonly used in musculoskeletal radiology. It also aimed to familiarize them with the common practices and issues in the use of AI in this domain.

DeepCERES: A deep learning method for cerebellar lobule segmentation using ultra-high resolution multimodal MRI

This paper introduces a novel multimodal and high-resolution human brain cerebellum lobule segmentation method. Unlike current tools that operate at standard resolution (1 mm3) or using mono-modal data, the proposed method improves cerebellum lobule segmentation through the use of a multimodal and ultra-high resolution (0.125 mm3) training dataset. To develop the method, first, a database of semi-automatically labelled cerebellum lobules was created to train the proposed method with ultra-high resolution T1 and T2 MR images. Then, an ensemble of deep networks has been designed and developed, allowing the proposed method to excel in the complex cerebellum lobule segmentation task, improving precision while being memory efficient. Notably, our approach deviates from the traditional U-Net model by exploring alternative architectures. We have also integrated deep learning with classical machine learning methods incorporating a priori knowledge from multi-atlas segmentation which improved precision and robustness. Finally, a new online pipeline, named DeepCERES, has been developed to make available the proposed method to the scientific community requiring as input only a single T1 MR image at standard resolution.

Deep learning image registration for cardiac motion estimation in adult and fetal echocardiography via a focus on anatomic plausibility and texture quality of warped image

Temporal echocardiography image registration is important for cardiac motion estimation, myocardial strain assessments, and stroke volume quantifications. Deep learning image registration (DLIR) is a promising way to achieve consistent and accurate registration results with low computational time. DLIR seeks the image deformation that enables the moving image to be warped to match the fixed image. We propose that, during DLIR training, a greater focus on the warped moving image's anatomic plausibility and image texture can support robust results, and we show that it has sufficient robustness to be applied to both fetal and adult echocardiography. Our proposed framework includes (1) an anatomic shape-encoded constraint to preserve physiological myocardial and left ventricular anatomical topologies in the warped image, (2) a data-driven texture constraint to preserve good texture features in the warped image, and (3) a multi-scale training algorithm to improve accuracy. Our experiments demonstrate a strong correlation between the shape-encoded constraint and good anatomical topology and between the data-driven texture constraint and image textures. They improve different aspects of registration results in a non-overlapping way. We demonstrate that these methods can successfully register both fetal and adult echocardiography using our multi-demographic fetal dataset and the public CAMUS adult dataset, despite the inherent differences between adult and fetal echocardiography. Our approach also outperforms traditional non-DL gold standard registration approaches, including optical flow and Elastix, and could be translated into more accurate and precise clinical quantification of cardiac ejection fraction, demonstrating potential for clinical utility.

Predicting early stage lung cancer recurrence and survival from combined tumor motion amplitude and radiomics on free-breathing 4D-CT

BACKGROUND

Cancer control outcomes of lung cancer are hypothesized to be affected by several confounding factors, including tumor heterogeneity and patient history, which have been hypothesized to mitigate the dose delivery effectiveness when treated with radiation therapy. Providing an accurate predictive model to identify patients at risk would enable tailored follow-up strategies during treatment.

PURPOSE

Our goal is to demonstrate the added prognostic value of including tumor displacement amplitude in a predictive model that combines clinical features and computed tomography (CT) radiomics for 2-year recurrence and survival in non-small-cell lung cancer (NSCLC) patients treated with curative-intent stereotactic body radiation therapy.

METHODS

A cohort of 381 patients treated for primary lung cancer with radiotherapy was collected, each including a planning CT with a dosimetry plan, 4D-CT, and clinical information. From this cohort, 101 patients (26.5%) experienced cancer progression (locoregional/distant metastasis) or death within 2 years of the end of treatment. Imaging data was analyzed for radiomics features from the tumor segmented image, as well as tumor motion amplitude measured on 4D-CT. A random forest (RF) model was developed to predict the overall outcomes, which was compared to three other approaches - logistic regression, support vector machine, and convolutional neural networks.

RESULTS

A 6-fold cross-validation study yielded an area under the receiver operating characteristic curve of 72% for progression-free survival when combining clinical data with radiomics features and tumor motion using a RF model (72% sensitivity and 81% specificity). The combined model showed significant improvement compared to standard clinical data. Model performances for loco-regional recurrence and overall survival sub-outcomes were established at 73% and 70%, respectively. No comparative methods reached statistical significance in any data configuration.

CONCLUSIONS

Combined tumor respiratory motion and radiomics features from planning CT showed promising predictive value for 2-year tumor control and survival, indicating the potential need for improving motion management strategies in future studies using machine learning-based prognosis models.

EyeLiner: A Deep Learning Pipeline for Longitudinal Image Registration Using Fundus Landmarks

Objective

Detecting and measuring changes in longitudinal fundus imaging is key to monitoring disease progression in chronic ophthalmic diseases, such as glaucoma and macular degeneration. Clinicians assess changes in disease status by either independently reviewing or manually juxtaposing longitudinally acquired color fundus photos (CFPs). Distinguishing variations in image acquisition due to camera orientation, zoom, and exposure from true disease-related changes can be challenging. This makes manual image evaluation variable and subjective, potentially impacting clinical decision-making. We introduce our deep learning (DL) pipeline, "EyeLiner," for registering, or aligning, 2-dimensional CFPs. Improved alignment of longitudinal image pairs may compensate for differences that are due to camera orientation while preserving pathological changes.

Design

EyeLiner registers a "moving" image to a "fixed" image using a DL-based keypoint matching algorithm.

Participants

We evaluate EyeLiner on 3 longitudinal data sets: Fundus Image REgistration (FIRE), sequential images for glaucoma forecast (SIGF), and our internal glaucoma data set from the Colorado Ophthalmology Research Information System (CORIS).

Methods

Anatomical keypoints along the retinal blood vessels were detected from the moving and fixed images using a convolutional neural network and subsequently matched using a transformer-based algorithm. Finally, transformation parameters were learned using the corresponding keypoints.

Main Outcome Measures

We computed the mean distance (MD) between manually annotated keypoints from the fixed and the registered moving image. For comparison to existing state-of-the-art retinal registration approaches, we used the mean area under the curve (AUC) metric introduced in the FIRE data set study.

Results

EyeLiner effectively aligns longitudinal image pairs from FIRE, SIGF, and CORIS, as qualitatively evaluated through registration checkerboards and flicker animations. Quantitative results show that the MD decreased for this model after alignment from 321.32 to 3.74 pixels for FIRE, 9.86 to 2.03 pixels for CORIS, and 25.23 to 5.94 pixels for SIGF. We also obtained an AUC of 0.85, 0.94, and 0.84 on FIRE, CORIS, and SIGF, respectively, beating the current state-of-the-art SuperRetina (AUCFIRE = 0.76, AUCCORIS = 0.83, AUCSIGF = 0.74).

Conclusions

Our pipeline demonstrates improved alignment of image pairs in comparison to the current state-of-the-art methods on 3 separate data sets. We envision that this method will enable clinicians to align image pairs and better visualize changes in disease over time.

Financial Disclosures

Proprietary or commercial disclosure may be found in the Footnotes and Disclosures at the end of this article.

CGNet: A Correlation-Guided Registration Network for Unsupervised Deformable Image Registration

Deformable medical image registration plays a significant role in medical image analysis. With the advancement of deep neural networks, learning-based deformable registration methods have made great strides due to their ability to perform fast end-to-end registration and their competitive performance compared to traditional methods. However, these methods primarily improve registration performance by replacing specific layers of the encoder-decoder architecture designed for segmentation tasks with advanced network structures like Transformers, overlooking the crucial difference between these two tasks, which is feature matching. In this paper, we propose a novel correlation-guided registration network (CGNet) specifically designed for deformable medical image registration tasks, which achieves a reasonable and accurate registration through three main components: dual-stream encoder, correlation learning module, and coarse-to-fine decoder. Specifically, the dual-stream encoder is used to independently extract hierarchical features from a moving image and a fixed image. The correlation learning module is used to calculate correlation maps, enabling explicit feature matching between input image pairs. The coarse-to-fine decoder outputs deformation sub-fields for each decoding layer in a coarse-to-fine manner, facilitating accurate estimation of the final deformation field. Extensive experiments on four 3D brain MRI datasets show that the proposed method achieves state-of-the-art performance on three evaluation metrics compared to twelve learning-based registration methods, demonstrating the potential of our model for deformable medical image registration.

Deformable registration for nasopharyngeal carcinoma using adaptive mask and weight allocation strategy based CycleFCNs model

BACKGROUND

Deformable registration plays an important role in the accurate delineation of tumors. Most of the existing deep learning methods ignored two issues that can lead to inaccurate registration, including the limited field of view in MR scans and the different scanning angles that can exist between multimodal images. The purpose of this study is to improve the registration accuracy between CT and MR for nasopharyngeal carcinoma cases.

METHODS

269 cases were enrolled in the study, and 188 cases were designated for training, while a separate set of 81 cases was reserved for testing. Each case had a CT volume and a T1-MR volume. The treatment table was removed from their CT images. The CycleFCNs model was used for deformable registration, and two strategies including adaptive mask registration strategy and weight allocation strategy were adopted for training. Dice similarity coefficient, Hausdorff distance, precision, and recall were calculated for normal tissues of CT-MR image pairs, before and after the registration. Three deformable registration methods including RayStation, Elastix, and VoxelMorph were compared with the proposed method.

RESULTS

The registration results of RayStation and Elastix are essentially consistent. Upon employing the VoxelMorph model and the proposed method for registration, a clear trend of increased dice similarity coefficient and decreased hausdorff distance can be observed. It is noteworthy that for the temporomandibular joint, pituitary, optic nerve, and optic chiasma, the proposed method has improved the average dice similarity coefficient from 0.86 to 0.91, 0.87 to 0.93, 0.85 to 0.89, and 0.77 to 0.83, respectively, as compared to RayStation. Additionally, within the same anatomical structures, the average hausdorff distance has been decreased from 2.98 mm to 2.28 mm, 1.83 mm to 1.53 mm, 3.74 mm to 3.56 mm, and 5.94 mm to 5.87 mm. Compared to the original CycleFCNs model, the improved model has significantly enhanced the dice similarity coefficient of the brainstem, pituitary gland, and optic nerve (P < 0.001).

CONCLUSIONS

The proposed method significantly improved the registration accuracy for multi-modal medical images in NPC cases. These findings have important clinical implications, as increased registration accuracy can lead to more precise tumor segmentation, optimized treatment planning, and ultimately, improved patient outcomes.

Subject-Specific Mapping of Excess Manganese Accumulation in the Brain of Welders Using Magnetic Resonance Imaging Relaxometry

Chronic overexposure to manganese (Mn) can occur in occupational settings, such as welding, leading to increased Mn levels in the brain. Excess brain Mn accumulation may result in neurotoxicity, which is characterized by Parkinsonian-like symptoms including motor and cognitive dysfunctions. In this work, we demonstrate a novel methodology for personalized diagnosis and spatial characterization of abnormal Magnetic Resonance Imaging R1 (R1 = 1/T1) relaxation rates arising from excessive Mn accumulation in welders' brains. Utilizing voxel-wise population-derived norms based on a frequency age-matched non-exposed group (n = 25), we demonstrate the ability to conduct subject-specific assessments and mapping of Mn exposure using MRI relaxometry. Our results show elevated R1 in multiple brain regions in individual welders, but also extreme between-subject variability in Mn accumulation, debasing the concept that high exposures correlate with uniformly high Mn deposition in the brain. Consequently, the presented personalized methodology serves as a counterpart to group-based comparison, which allows for understanding the level of individual exposure and the toxicokinetics of Mn accumulation. This work lays a foundation for improved occupational health assessments and preventive measures against neurotoxic metal exposure.

M4S-Net: a motion-enhanced shape-aware semi-supervised network for echocardiography sequence segmentation

Sequence segmentation of echocardiograms is of great significance for the diagnosis and treatment of cardiovascular diseases. However, the low quality of ultrasound imaging and the complexity of cardiac motion pose great challenges to it. In addition, the difficulty and cost of labeling echocardiography sequences limit the performance of supervised learning methods. In this paper, we proposed a Motion-enhanced Shape-aware Semi-supervised Sequence Segmentation Network named M4S-Net. First, multi-level shape priors are used to enhance the model's shape representation capabilities, overcoming the low image quality and improving single-frame segmentation. Then, a motion-enhanced optimization module utilizes optical flows to assist segmentation in a geometric sense, which robustly responds to the complex motions and ensures the temporal consistency of sequence segmentation. A hybrid loss function is devised to maximize the effectiveness of each module and further improve the temporal stability of predicted masks. Furthermore, the parameter-sharing strategy allows it to perform sequence segmentation in a semi-supervised manner. Massive experiments on both public and in-house datasets show that M4S-Net outperforms the state-of-the-art methods in both spatial and temporal segmentation performance. A downstream apical rocking recognition task based on M4S-Net also achieves an AUC of 0.944, which significantly exceeds specialized physicians.

Enhancing unsupervised learning in medical image registration through scale-aware context aggregation

Deformable image registration (DIR) is essential for medical image analysis, facilitating the establishment of dense correspondences between images to analyze complex deformations. Traditional registration algorithms often require significant computational resources due to iterative optimization, while deep learning approaches face challenges in managing diverse deformation complexities and task requirements. We introduce ScaMorph, an unsupervised learning model for DIR that employs scale-aware context aggregation, integrating multiscale mixed convolution with lightweight multiscale context fusion. This model effectively combines convolutional networks and vision transformers, addressing various registration tasks. We also present diffeomorphic variants of ScaMorph to maintain topological deformations. Extensive experiments on 3D medical images across five applications-atlas-to-patient and inter-patient brain magnetic resonance imaging (MRI) registration, inter-modal brain MRI registration, inter-patient liver computed tomography (CT) registration as well as inter-modal abdomen MRI-CT registration-demonstrate that our model significantly outperforms existing methods, highlighting its effectiveness and broader implications for enhancing medical image registration techniques.

Boosting 2D brain image registration via priors from large model

BACKGROUND

Deformable medical image registration aims to align image pairs with local differences, improving the accuracy of medical analyses and assisting various diagnostic scenarios.

PURPOSE

We aim to overcome these challenges: Deep learning-based registration approaches have greatly enhanced registration speed and accuracy by continuously improving registration networks and processes. However, the lack of extensive medical datasets limits the complexity of registration models. Optimizing registration networks within a fixed dataset often leads to overfitting, hindering further accuracy improvements and reducing generalization capabilities.

METHODS

We explore the application of the foundational model DINOv2 to registration tasks, leveraging its prior knowledge to support learning-based unsupervised registration networks and overcome network bottlenecks to improve accuracy. We investigate three modes of DINOv2-assisted registration, including direct registration architecture, enhanced architecture, and refined architecture. Additionally, we study the applicability of three feature aggregation methods-convolutional interaction, direct fusion, and cross-attention-within the proposed DINOv2-based registration frameworks.

RESULTS

We conducted extensive experiments on the IXI and OASIS public datasets, demonstrating that the enhanced and refined architectures notably improve registration accuracy, reduce data dependency, and maintain strong generalization capabilities.

CONCLUSION

This study offers novel approaches for applying foundational models to deformable image registration tasks.

IConDiffNet: an unsupervised inverse-consistent diffeomorphic network for medical image registration

Objective.Deformable image registration (DIR) is critical in many medical imaging applications. Diffeomorphic transformations, which are smooth invertible mappings with smooth inverses preserve topological properties and are an anatomically plausible means of constraining the solution space in many settings. Traditional iterative optimization-based diffeomorphic DIR algorithms are computationally costly and are not able to consistently resolve large and complicated deformations in medical image registration. Convolutional neural network implementations can rapidly estimate the transformation in through a pre-trained model. However, the structure design of most neural networks for DIR fails to systematically enforce diffeomorphism and inverse consistency. In this paper, a novel unsupervised neural network structure is proposed to perform a fast, accurate, and inverse-consistent diffeomorphic DIR.Approach.This paper introduces a novel unsupervised inverse-consistent diffeomorphic registration network termed IConDiffNet, which incorporates an energy constraint that minimizes the total energy expended during the deformation process. The IConDiffNet architecture consists of two symmetric paths, each employing multiple recursive cascaded updating blocks (neural networks) to handle different virtual time steps parameterizing the path from the initial undeformed image to the final deformation. These blocks estimate velocities corresponding to specific time steps, generating a series of smooth time-dependent velocity vector fields. Simultaneously, the inverse transformations are estimated by corresponding blocks in the inverse path. By integrating these series of time-dependent velocity fields from both paths, optimal forward and inverse transformations are obtained, aligning the image pair in both directions.Main result.Our proposed method was evaluated on a three-dimensional inter-patient image registration task with a large-scale brain MRI image dataset containing 375 subjects. The proposed IConDiffNet achieves fast and accurate DIR with better DSC, lower Hausdorff distance metric, and lower total energy spent during the deformation in the test dataset compared to competing state-of-the-art deep-learning diffeomorphic DIR approaches. Visualization shows that IConDiffNet produces more complicated transformations that better align structures than the VoxelMorph-Diff, SYMNet, and ANTs-SyN methods.Significance.The proposed IConDiffNet represents an advancement in unsupervised deep-learning-based DIR approaches. By ensuring inverse consistency and diffeomorphic properties in the outcome transformations, IConDiffNet offers a pathway for improved registration accuracy, particularly in clinical settings where diffeomorphic properties are crucial. Furthermore, the generality of IConDiffNet's network structure supports direct extension to diverse 3D image registration challenges. This adaptability is facilitated by the flexibility of the objective function used in optimizing the network, which can be tailored to suit different registration tasks.

SurfNet: Reconstruction of Cortical Surfaces via Coupled Diffeomorphic Deformations

To achieve fast and accurate cortical surface reconstruction from brain magnetic resonance images (MRIs), we develop a method to jointly reconstruct the inner (white-gray matter interface), outer (pial), and midthickness surfaces, regularized by their interdependence. Rather than reconstructing these surfaces separately without taking into consideration their interdependence as in most existing methods, our method learns three diffeomorphic deformations jointly to optimize the midthickness surface to lie halfway between the inner and outer cortical surfaces and simultaneously deforms it inward and outward towards the inner and outer cortical surfaces, respectively. The surfaces are encouraged to have a spherical topology by regularization terms for non-negativeness of the cortical thickness and symmetric cycle-consistency of the coupled surface deformations. The coupled reconstruction of cortical surfaces also facilitates an accurate estimation of the cortical thickness based on the diffeomorphic deformation trajectory of each vertex on the surfaces. Validation experiments have demonstrated that our method achieves state-of-the-art cortical surface reconstruction performance in terms of accuracy and surface topological correctness on large-scale MRI datasets, including ADNI, HCP, and OASIS.

Neural network powered microscopic system for cataract surgery

Phacoemulsification with intraocular lens (IOL) implantation is a widely used effective treatment for cataracts. However, the surgical outcome relies heavily on precise operations with marked eye location and orientation, which ideally require a high-precision navigation system for complete guidance of surgical procedure. However, both research and current commercial surgical microscopes still face substantial challenges in handling various complex clinical scenarios. Here we propose a neural network-powered surgical microscopic system that can benefit from big data to address the unmet clinical need. In this system, we designed an end-to-end navigation network for real-time positioning and alignment of IOL and then built a computer-assisted surgical microscope with a complete imaging and display platform integrating the control software and algorithms for surgical planning and navigation. The network used an attention-based encoder-decoder architecture with an edge padding mechanism and an MLP layer for eye center localization, and combined siamese network, correlation filter, and spatial transformation network to track eye rotation. Using computer-aided annotation, we collected and labeled 100 clinical surgery videos from 100 patients, and proposed a data augmentation method to enhance the diversity of training. We further evaluated the navigation performance of the microscopic system on a human eye model.

The Developing Human Connectome Project: A fast deep learning-based pipeline for neonatal cortical surface reconstruction

The Developing Human Connectome Project (dHCP) aims to explore developmental patterns of the human brain during the perinatal period. An automated processing pipeline has been developed to extract high-quality cortical surfaces from structural brain magnetic resonance (MR) images for the dHCP neonatal dataset. However, the current implementation of the pipeline requires more than 6.5 h to process a single MRI scan, making it expensive for large-scale neuroimaging studies. In this paper, we propose a fast deep learning (DL) based pipeline for dHCP neonatal cortical surface reconstruction, incorporating DL-based brain extraction, cortical surface reconstruction and spherical projection, as well as GPU-accelerated cortical surface inflation and cortical feature estimation. We introduce a multiscale deformation network to learn diffeomorphic cortical surface reconstruction end-to-end from T2-weighted brain MRI. A fast unsupervised spherical mapping approach is integrated to minimize metric distortions between cortical surfaces and projected spheres. The entire workflow of our DL-based dHCP pipeline completes within only 24 s on a modern GPU, which is nearly 1000 times faster than the original dHCP pipeline. The qualitative assessment demonstrates that for 82.5% of the test samples, the cortical surfaces reconstructed by our DL-based pipeline achieve superior (54.2%) or equal (28.3%) surface quality compared to the original dHCP pipeline.

Toward Semantically-Consistent Deformable 2D-3D Registration for 3D Craniofacial Structure Estimation From a Single-View Lateral Cephalometric Radiograph

The deep neural networks combined with the statistical shape model have enabled efficient deformable 2D-3D registration and recovery of 3D anatomical structures from a single radiograph. However, the recovered volumetric image tends to lack the volumetric fidelity of fine-grained anatomical structures and explicit consideration of cross-dimensional semantic correspondence. In this paper, we introduce a simple but effective solution for semantically-consistent deformable 2D-3D registration and detailed volumetric image recovery by inferring a voxel-wise registration field between the cone-beam computed tomography and a single lateral cephalometric radiograph (LC). The key idea is to refine the initial statistical model-based registration field with craniofacial structural details and semantic consistency from the LC. Specifically, our framework employs a self-supervised scheme to learn a voxel-level refiner of registration fields to provide fine-grained craniofacial structural details and volumetric fidelity. We also present a weakly supervised semantic consistency measure for semantic correspondence, relieving the requirements of volumetric image collections and annotations. Experiments showcase that our method achieves deformable 2D-3D registration with performance gains over state-of-the-art registration and radiograph-based volumetric reconstruction methods. The source code is available at https://github.com/Jyk-122/SC-DREG.