Multimodality image fusion, moving forward

MedFusionGAN: multimodal medical image fusion using an unsupervised deep generative adversarial network

PURPOSE

This study proposed an end-to-end unsupervised medical fusion generative adversarial network, MedFusionGAN, to fuse computed tomography (CT) and high-resolution isotropic 3D T1-Gd Magnetic resonance imaging (MRI) image sequences to generate an image with CT bone structure and MRI soft tissue contrast to improve target delineation and to reduce the radiotherapy planning time.

METHODS

We used a publicly available multicenter medical dataset (GLIS-RT, 230 patients) from the Cancer Imaging Archive. To improve the models generalization, we consider different imaging protocols and patients with various brain tumor types, including metastases. The proposed MedFusionGAN consisted of one generator network and one discriminator network trained in an adversarial scenario. Content, style, and L1 losses were used for training the generator to preserve the texture and structure information of the MRI and CT images.

RESULTS

The MedFusionGAN successfully generates fused images with MRI soft-tissue and CT bone contrast. The results of the MedFusionGAN were quantitatively and qualitatively compared with seven traditional and eight deep learning (DL) state-of-the-art methods. Qualitatively, our method fused the source images with the highest spatial resolution without adding the image artifacts. We reported nine quantitative metrics to quantify the preservation of structural similarity, contrast, distortion level, and image edges in fused images. Our method outperformed both traditional and DL methods on six out of nine metrics. And it got the second performance rank for three and two quantitative metrics when compared with traditional and DL methods, respectively. To compare soft-tissue contrast, intensity profile along tumor and tumor contours of the fusion methods were evaluated. MedFusionGAN provides a more consistent, better intensity profile, and a better segmentation performance.

CONCLUSIONS

The proposed end-to-end unsupervised method successfully fused MRI and CT images. The fused image could improve targets and OARs delineation, which is an important aspect of radiotherapy treatment planning.

Comprehensive assessment of molecular function, tissue characterization, and hemodynamic performance by non-invasive hybrid imaging: Potential role of cardiac PETMR

Noninvasive cardiovascular imaging plays a key role in diagnosis and patient management including monitoring treatment efficacy. The usefulness of noninvasive cardiovascular imaging has been extensively studied and shown to have high diagnostic reliability and prognostic significance, while the nondiagnostic results frequently encountered with single imaging modality require complementary or alternative imaging techniques. Hybrid cardiac imaging was initially introduced to integrate anatomical and functional information to enhance the diagnostic performance, and lately employed as a strategy for comprehensive assessment of the underlying pathophysiology of diseases. More recently, the utility of computed tomography has grown in diversity, and emerged from being an exploratory technique allowing functional measurement such as stress dynamic perfusion. Cardiac magnetic resonance imaging (CMR) is widely accepted as a robust tool for evaluation of cardiac function, fibrosis, and edema, yielding high spatial resolution and soft-tissue contrast. However, the use of intravenous contrast materials is typically required for accurate diagnosis with these imaging modalities, despite the associated risk of renal toxicity. Nuclear cardiology, established as a molecular imaging technique, has advantages in visualization of the disease-specific biological process at cellular level using numerous probes without requiring contrast materials. Various imaging modalities should be appropriately used sequentially to assess concomitant disease and the progression over time. Therefore, simultaneous evaluation combining high spatial resolution and disease-specific imaging probe is a useful approach to identify the regional activity and the stage of the disease. Given the recent advance and potential of multiparametric CMR and novel nuclide tracers, hybrid positron emission tomography MR is becoming an ideal tool for disease-specific imaging.

Personalized Surgical Planning for Soft Tissue Sarcoma of the Popliteal Fossa with a Novel 3D Imaging Technique

OBJECTIVE

Soft tissue sarcomas (STSs) arising from the popliteal fossa pose surgical challenges due to their proximity to critical neurovascular structures. This study aimed to investigate whether a novel 3D imaging technique highlighting these key anatomical structures could facilitate preoperative planning and improve surgical outcomes in STS.

METHODS

This was a prospective, observational, pilot study. Between November 2019 and December 2020, 27 patients with STS of the popliteal fossa undergoing limb-sparing procedures were enrolled and assigned to either a control or intervention group. Control patients underwent traditional preoperative planning with separate computed tomography angiography, magnetic resonance imaging, and magnetic resonance hydrography. In the intervention group, 3D images were generated from these images, the tumor and skeletomuscular and neurovascular structures were revealed in three dimensions, and this was visualized on the surgeon's smartphone or computer. Primary endpoints were surgical margins and complications. Secondary endpoints included operative time, blood loss, serum C-reactive protein and interleukin-6, length of in-hospital stay, and limb function. Comparisons between groups were made using independent-sample t-tests for continuous data and the Mann-Whitney U and Fisher's exact tests for categorical data.

RESULTS

There was a lower but not significantly different inadvertent positive margin rate (1/15 vs. 3/12, P = 0.294), significantly shorter hospital stay (P = 0.049), and less numbers ≥75th percentile of operative time (P = 0.037) and blood loss (P = 0.024) in the intervention group. Differences in surgical complications, operative time, blood loss, C-reactive protein and interleukin-6 levels on the second postoperative day, and limb functional scores were statistically insignificant.

CONCLUSIONS

The novel 3D imaging technique facilitates complex preoperative planning and limb-salvage surgical procedures for patients with STS of the popliteal fossa, and this may affect how surgical planning is performed in the future.

[Research on three-dimensional printing technology based on three-dimensional multimodality imaging to assist the operation of malignant bone tumors of limbs]

OBJECTIVE

To explore the role and effectiveness of three-dimensional (3D) printing technology based on 3D multimodality imaging in surgical treatment of malignant bone tumors of limbs.

METHODS

The clinical data of 15 patients with malignant bone tumors of the limbs who met the selection criteria between January 2016 and January 2019 were retrospectively analyzed. There were 6 males and 9 females, with a median age of 34 years (range, 17-73 years). There were 5 cases of osteosarcoma, 3 cases of chondrosarcoma, 2 cases of Ewing sarcoma, 1 case of hemangiosarcoma, 1 case of ameloblastoma, and 3 cases of metastatic carcinoma. The tumors were located in the humerus in 5 cases, ulna in 2 cases, femur in 3 cases, and tibia in 5 cases. The disease duration was 2-8 months (median, 4 months). Preoperative 3D multimodality imaging was administered first, based on which computer-assisted preoperative planning was performed, 3D printed personalized special instruments and prostheses were designed, and in vitro simulation of surgery was conducted, successively. Two cases underwent knee arthroplasty, 2 had semi-shoulder arthroplasty, 2 had proximal ulna arthroplasty, and 9 had joint-preserving surgery. Surgical margins, operation time, intraoperative blood loss, surgical complications, Musculoskeletal Tumor Society (MSTS) score, and oncological outcome were collected and analyzed.

RESULTS

All 15 patients completed the operation according to the preoperative plan, and the surgical margins were all obtained wide resection margins. The operation time was 80-240 minutes, with a median of 150 minutes. The intraoperative blood loss was 100-400 mL, with a median of 200 mL. There was no significant limitation of limb function due to important blood vessels or nerves injury during operation. One case of superficial infection of the incision was cured after dressing change, and the incisions of the other patients healed by first intention. All patients were followed up 6-48 months, with a median of 24 months. Two of the patients died of lung metastasis at 6 and 24 months after operation, respectively. No local recurrence, prosthesis dislocation, or prosthesis loosening occurred during follow-up. At last follow-up, the MSTS score ranged from 23 to 30, with an average of 25.

CONCLUSION

3D printing tecnology, based on 3D multimodality imaging, facilitates precise resection and reconstruction for malignant bone tumors of limbs, resulting in improved oncological and functional outcome.

Preoperative Planning Using Three-Dimensional Multimodality Imaging for Soft Tissue Sarcoma of the Axilla: A Pilot Study

Axillary soft tissue sarcoma (STS) is challenging due to its proximity to vital neurovascular bundles. We conducted a prospective observational pilot study to explore whether 3D multimodality imaging (3DMMI) can improve preoperative planning for and surgical outcomes of patients with axillary STS. Twenty-one patients with STS (diameter > 5 cm) of the axilla were allocated, at their discretion, to either a control group undergoing traditional preoperative planning with separate computed tomography angiography, magnetic resonance imaging, and magnetic resonance neurography, or an intervention group where 3DMMI, digitally created based on these images, revealed the tumour and adjacent skeletomuscular and neurovascular structures in three dimensions. Primary outcome measures were surgical margins and surgical complications. Secondary outcomes included operative time, blood loss, serum C-reactive protein and interleukin-6, length of hospital stay, and limb function. The 3DMMI group had a lower, although not significantly different, inadvertent positive margin rate (1/12 vs. 3/9, p = 0.272), a significantly shorter operative time (p = 0.048), reduced blood loss (p = 0.038), and reduced length of hospital stay (p = 0.046). This endorses larger trials to improve complex surgical procedures and study how preoperative planning could be performed in the future.

Combination of Preoperative Multimodal Image Fusion and Intraoperative Dyna CT in Percutaneous Balloon Compression of Trigeminal Ganglion for Primary Trigeminal Neuralgia: Experience in 24 Patients

Objective

We retrospectively assessed the surgical results of PBC with preoperative multimodal image fusion and intraoperative Dyna Computed Tomography (CT) in 24 patients with primary trigeminal neuralgia (PTN) to explore a valuable aid for Percutaneous balloon compression (PBC).

Methods

We studied the data of 24 patients with PTN. All patients underwent PBC and were assessed with preoperative multimodal image fusion [computed tomography (CT) and magnetic resonance imaging (MRI)] and intraoperative Dyna CT in the Department of Neurosurgery of Zhuhai People’s Hospital between October 2020 and September 2021. Multimodal image fusion—three-dimensional (3D) reconstruction of CT and MRI data—was performed using 3D-Slicer software, and preoperative evaluation was performed according to the results of image fusion. Dyna CT was used to dynamically observe the position and shape of the metallic hollow introducer and Fogarty catheter and balloon during the operation to guide the operation in real time. We performed follow-up assessments each month and summarized the clinical characteristics, surgical effects, and complications in all patients.

Results

Surgery was successful for all patients; the patients reported immediate pain relief. Surgical complications included facial numbness in 24 patients (100%), mild masseter weakness in three (12.5%), herpes zoster in three (12.5%), and balloon rupture in one (4.2%). None of the patients had serious surgical complications. The mean follow-up time was 9.6 ± 2.7 months. During the follow-up period, 22 patients (91.7%) experienced no recurrence of pain, and two patients (8.3%) experienced recurrence of pain, of which one underwent secondary PBC surgery.

Conclusions

Preoperative multimodal image reconstruction can help fully evaluate PBC surgery, clarify the etiology, and predict the volume of contrast medium required during the operation. It provided important assistance for PBC treatment of trigeminal neuralgia patients when preoperative multimodal image fusion is combined with intraoperative Dyna CT.

3D reconstructions, 4D imaging and postprocessing with CT in musculoskeletal disorders: Past, present and future

Three-dimensional (3D) imaging and post processing are common tasks used daily in many disciplines. The purpose of this article is to review the new postprocessing tools available. Although 3D imaging can be applied to all anatomical regions and used with all imaging techniques, its most varied and relevant applications are found with computed tomography (CT) data in musculoskeletal imaging. These new applications include global illumination rendering (GIR), unfolded rib reformations, subtracted CT angiography for bone analysis, dynamic studies, temporal subtraction and image fusion. In all of these tasks, registration and segmentation are two basic processes that affect the quality of the results. GIR simulates the complete interaction of photons with the scanned object, providing photorealistic volume rendering. Reformations to unfold the rib cage allow more accurate and faster diagnosis of rib lesions. Dynamic CT can be applied to cinematic joint evaluations a well as to perfusion and angiographic studies. Finally, more traditional techniques, such as minimum intensity projection, might find new applications for bone evaluation with the advent of ultra-high-resolution CT scanners. These tools can be used synergistically to provide morphologic, topographic and functional information and increase the versatility of CT.