US & MR/CT Image Fusion with Markerless Skin Registration: A Proof of Concept

This paper presents an innovative automatic fusion imaging system that combines 3D CT/MR images with real-time ultrasound acquisition. The system eliminates the need for external physical markers and complex training, making image fusion feasible for physicians with different experience levels. The integrated system involves a portable 3D camera for patient-specific surface acquisition, an electromagnetic tracking system, and US components. The fusion algorithm comprises two main parts: skin segmentation and rigid co-registration, both integrated into the US machine. The co-registration aligns the surface extracted from CT/MR images with the 3D surface acquired by the camera, facilitating rapid and effective fusion. Experimental tests in different settings, validate the system's accuracy, computational efficiency, noise robustness, and operator independence.

Convolutional LSTM model for cine image prediction of abdominal motion

Objective.In this study, we tackle the challenge of latency in magnetic resonance linear accelerator (MR-Linac) systems, which compromises target coverage accuracy in gated real-time radiotherapy. Our focus is on enhancing motion prediction precision in abdominal organs to address this issue. We developed a convolutional long short-term memory (convLSTM) model, utilizing 2D cine magnetic resonance (cine-MR) imaging for this purpose.Approach.Our model, featuring a sequence-to-one architecture with six input frames and one output frame, employs structural similarity index measure (SSIM) as loss function. Data was gathered from 17 cine-MRI datasets using the Philips Ingenia MR-sim system and an Elekta Unity MR-Linac equivalent sequence, focusing on regions of interest (ROIs) like the stomach, liver, pancreas, and kidney. The datasets varied in duration from 1 to 10 min.Main results.The study comprised three main phases: hyperparameter optimization, individual training, and transfer learning with or without fine-tuning. Hyperparameters were initially optimized to construct the most effective model. Then, the model was individually applied to each dataset to predict images four frames ahead (1.24-3.28 s). We evaluated the model's performance using metrics such as SSIM, normalized mean square error, normalized correlation coefficient, and peak signal-to-noise ratio, specifically for ROIs with target motion. The average SSIM values achieved were 0.54, 0.64, 0.77, and 0.66 for the stomach, liver, kidney, and pancreas, respectively. In the transfer learning phase with fine-tuning, the model showed improved SSIM values of 0.69 for the liver and 0.78 for the kidney, compared to 0.64 and 0.37 without fine-tuning.Significance. The study's significant contribution is demonstrating the convLSTM model's ability to accurately predict motion for multiple abdominal organs using a Unity-equivalent MR sequence. This advancement is key in mitigating latency issues in MR-Linac radiotherapy, potentially improving the precision and effectiveness of real-time treatment for abdominal cancers.

Real-time prediction of stomach motions based upon gastric contraction and breathing models

Objective.Precision radiation therapy requires managing motions of organs at risk that occur during treatment. While methods have been developed for real-time respiratory motion tracking, non-breathing intra-fractional variations (including gastric contractile motion) have seen little attention to date. The purpose of this study is to develop a cyclic gastric contractile motion prediction model to support real-time management during radiotherapy.Approach. The observed short-term reproducibility of gastric contractile motion permitted development of a prediction model that (1) extracts gastric contraction motion phases from few minutes of golden angle stack of stars scanning (at patient positioning), (2) estimate gastric phase of real-time sampled data acquired during treatment delivery to these reconstructed phases and (3) predicting future gastric phase by linear extrapolation using estimation results from step 2 to account for processing and system latency times. Model was evaluated on three parameters including training time window for step 1, number of spokes for real-time sampling data in step 2 and future prediction time. Mainresults. The model was tested on a population of 20 min data samples from 25 scans from 15 patients. The mean prediction error with 10 spokes and 2 min training was 0.3 ± 0.1 mm (0.1-0.7 mm) with 5.1 s future time, slowly rising to 0.6 ± 0.2 mm (0.2-1.1 mm) for 6.8 s future time and then increasing rapidly for longer forward predictions, for an average 3.6 ± 0.5 mm (2.8-4.7 mm) HD95 of gastric motion. Results showed that reducing of train time window (5-2 min) does not influence the prediction performance, while using 5 spokes increased prediction errors.Significance. The proposed gastric motion prediction model has sufficiently accurate prediction performance to allow for sub-millimeter accuracy while allowing sufficient time for data processing and machine interaction and shows the potential for clinical implementation to support stomach motion tracking during radiotherapy.

Free-breathing 3D stack-of-radial MRI quantification of liver fat and R2* in adults with fatty liver disease

PURPOSE

To investigate the agreement, intra-session repeatability, and inter-reader agreement of liver proton-density fat fraction (PDFF) and R₂* quantification using free-breathing 3D stack-of-radial MRI, with and without self-gated motion compensation, compared to reference breath-hold techniques in subjects with fatty liver disease (FLD).

METHODS

In this institutional review board-approved prospective study, thirty-eight adults with FLD and/or iron overload (24 male, 58 ± 12 years) were imaged at 3T using free-breathing stack-of-radial MRI, breath-hold 3D Cartesian MRI, and breath-hold single-voxel MR spectroscopy (SVS). Each sequence was acquired twice in random order. To assess agreement compared to reference breath-hold techniques, the dependency of liver PDFF and/or R₂* quantification on the sequence, radial sampling factor, and radial self-gating temporal resolution was assessed by calculating the Bayesian mean difference (MD_B) of the posteriors. Intra-session repeatability and inter-reader agreement (two independent readers) were assessed by the coefficient of repeatability (CR) and intraclass correlation coefficient (ICC), respectively.

RESULTS

Thirty-five participants (21 male, 57 ± 12 years) were included for analysis. Both free-breathing radial MRI techniques (with and without self-gating) achieved ICC ≥ 0.92 for quantifying PDFF and R₂*, and quantified PDFF with MD_B < 1.2% compared to breath-hold techniques. Free-breathing radial MRI required self-gating to accurately quantify R₂* (MD_B < 10s^-1 with self-gating; MD_B < 50s^-1 without self-gating). The radial sampling factor affected PDFF and R₂* quantification while the radial self-gating temporal resolution only affected R₂* quantification. Repeated self-gated free-breathing radial MRI scans achieved CR < 3% and CR < 27 s^-1 for PDFF and R₂*, respectively.

CONCLUSION

A free-breathing stack-of-radial MRI technique with self-gating demonstrated agreement, repeatability, and inter-reader agreement compared to reference breath-hold techniques for quantification of liver PDFF and R₂* in adults with FLD.

Selecting the Best Quantity and Variety of Surrogates for an Ensemble Model

Surrogate modeling techniques are widely used to replace the computationally expensive black-box functions in engineering. As a combination of individual surrogate models, an ensemble of surrogates is preferred due to its strong robustness. However, how to select the best quantity and variety of surrogates for an ensemble has always been a challenging task. In this work, five popular surrogate modeling techniques including polynomial response surface (PRS), radial basis functions (RBF), kriging (KRG), Gaussian process (GP) and linear shepard (SHEP) are considered as the basic surrogate models, resulting in twenty-six ensemble models by using a previously presented weights selection method. The best ensemble model is expected to be found by comparative studies on prediction accuracy and robustness. By testing eight mathematical problems and two engineering examples, we found that: (1) in general, using as many accurate surrogates as possible to construct ensemble models will improve the prediction performance and (2) ensemble models can be used as an insurance rather than offering significant improvements. Moreover, the ensemble of three surrogates PRS, RBF and KRG is preferred based on the prediction performance. The results provide engineering practitioners with guidance on the superior choice of the quantity and variety of surrogates for an ensemble.

Automatic needle tracking using Mask R-CNN for MRI-guided percutaneous interventions

PURPOSE

Accurate needle tracking provides essential information for MRI-guided percutaneous interventions. Passive needle tracking using MR images is challenged by variations of the needle-induced signal void feature in different situations. This work aimed to develop an automatic needle tracking algorithm for MRI-guided interventions based on the Mask Region Proposal-Based Convolutional Neural Network (R-CNN).

METHODS

Mask R-CNN was adapted and trained to segment the needle feature using 250 intra-procedural images from 85 MRI-guided prostate biopsy cases and 180 real-time images from MRI-guided needle insertion in ex vivo tissue. The segmentation masks were passed into the needle feature localization algorithm to extract the needle feature tip location and axis orientation. The proposed algorithm was tested using 208 intra-procedural images from 40 MRI-guided prostate biopsy cases, and 3 real-time MRI datasets in ex vivo tissue. The algorithm results were compared with human-annotated references.

RESULTS

In prostate datasets, the proposed algorithm achieved needle feature tip localization error with median Euclidean distance (dxy) of 0.71 mm and median difference in axis orientation angle (dθ) of 1.28°, respectively. In 3 real-time MRI datasets, the proposed algorithm achieved consistent dynamic needle feature tracking performance with processing time of 75 ms/image: (a) median dxy = 0.90 mm, median dθ = 1.53°; (b) median dxy = 1.31 mm, median dθ = 1.9°; (c) median dxy = 1.09 mm, median dθ = 0.91°.

CONCLUSIONS

The proposed algorithm using Mask R-CNN can accurately track the needle feature tip and axis on MR images from in vivo intra-procedural prostate biopsy cases and ex vivo real-time MRI experiments with a range of different conditions. The algorithm achieved pixel-level tracking accuracy in real time and has potential to assist MRI-guided percutaneous interventions.

Real-time 3T MRI-guided cardiovascular catheterization in a porcine model using a glass-fiber epoxy-based guidewire

PURPOSE

Real-time magnetic resonance imaging (MRI) is a promising alternative to X-ray fluoroscopy for guiding cardiovascular catheterization procedures. Major challenges, however, include the lack of guidewires that are compatible with the MRI environment, not susceptible to radiofrequency-induced heating, and reliably visualized. Preclinical evaluation of new guidewire designs has been conducted at 1.5T. Here we further evaluate the safety (device heating), device visualization, and procedural feasibility of 3T MRI-guided cardiovascular catheterization using a novel MRI-visible glass-fiber epoxy-based guidewire in phantoms and porcine models.

METHODS

To evaluate device safety, guidewire tip heating (GTH) was measured in phantom experiments with different combinations of catheters and guidewires. In vivo cardiovascular catheterization procedures were performed in both healthy (N = 5) and infarcted (N = 5) porcine models under real-time 3T MRI guidance using a glass-fiber epoxy-based guidewire. The times for each procedural step were recorded separately. Guidewire visualization was assessed by measuring the dimensions of the guidewire-induced signal void and contrast-to-noise ratio (CNR) between the guidewire tip signal void and the blood signal in real-time gradient-echo MRI (specific absorption rate [SAR] = 0.04 W/kg).

RESULTS

In the phantom experiments, GTH did not exceed 0.35°C when using the real-time gradient-echo sequence (SAR = 0.04 W/kg), demonstrating the safety of the glass-fiber epoxy-based guidewire at 3T. The catheter was successfully placed in the left ventricle (LV) under real-time MRI for all five healthy subjects and three out of five infarcted subjects. Signal void dimensions and CNR values showed consistent visualization of the glass-fiber epoxy-based guidewire in real-time MRI. The average time (minutes:seconds) for the catheterization procedure in all subjects was 4:32, although the procedure time varied depending on the subject's specific anatomy (standard deviation = 4:41).

CONCLUSIONS

Real-time 3T MRI-guided cardiovascular catheterization using a new MRI-visible glass-fiber epoxy-based guidewire is feasible in terms of visualization and guidewire navigation, and safe in terms of radiofrequency-induced guidewire tip heating.