Combined unsupervised-supervised classification of multiparametric PET/MRI data: application to prostate cancer

Tackling the small data problem in medical image classification with artificial intelligence: a systematic review

Though medical imaging has seen a growing interest in AI research, training models require a large amount of data. In this domain, there are limited sets of data available as collecting new data is either not feasible or requires burdensome resources. Researchers are facing with the problem of small datasets and have to apply tricks to fight overfitting. 147 peer-reviewed articles were retrieved from PubMed, published in English, up until 31 July 2022 and articles were assessed by two independent reviewers. We followed the Preferred Reporting Items for Systematic reviews and Meta-Analyse (PRISMA) guidelines for the paper selection and 77 studies were regarded as eligible for the scope of this review. Adherence to reporting standards was assessed by using TRIPOD statement (transparent reporting of a multivariable prediction model for individual prognosis or diagnosis). To solve the small data issue transfer learning technique, basic data augmentation and generative adversarial network were applied in 75%, 69% and 14% of cases, respectively. More than 60% of the authors performed a binary classification given the data scarcity and the difficulty of the tasks. Concerning generalizability, only four studies explicitly stated an external validation of the developed model was carried out. Full access to all datasets and code was severely limited (unavailable in more than 80% of studies). Adherence to reporting standards was suboptimal (<50% adherence for 13 of 37 TRIPOD items). The goal of this review is to provide a comprehensive survey of recent advancements in dealing with small medical images samples size. Transparency and improve quality in publications as well as follow existing reporting standards are also supported.

A Clinical Role of PET-MRI in Prostate Cancer?

PET/MRI is a relevant application field for prostate cancer management, offering advantages in early diagnosis, staging, and therapy planning. Despite drawbacks such as higher costs, longer acquisition time, and the need for skilled personnel, the technical integration of PET and MRI provides valuable information for detecting primary tumors, identifying metastases, and characterizing the disease, leading to more accurate staging and personalized treatment strategies. However, PET/MRI adoption has been slow, but ongoing technological advancements and AI integration might overcome challenges and improve clinical utility. As precision medicine gains importance in oncology, PET/MRI's multiparametric data can tailor treatment plans to individual patients, providing a comprehensive assessment of tumor biology and aggressiveness for more effective therapeutic strategies.

Application of Artificial Intelligence in Oncologic Molecular PET-Imaging: A Narrative Review on Beyond [18F]F-FDG Tracers - Part I. PSMA, Choline, and DOTA Radiotracers

Artificial intelligence (AI) has evolved significantly in the past few decades. This thriving trend has also been seen in medicine in recent years, particularly in the field of imaging. Machine learning (ML), deep learning (DL), and their methods (eg, SVM, CNN), as well as radiomics, are the terminologies that have been introduced to this field and, to some extent, become familiar to the expert clinicians. PET is one of the modalities that has been enhanced via these state-of-the-art algorithms. This robust imaging technique further merged with anatomical modalities, such as computed tomography (CT) and magnetic resonance imaging (MRI), to provide reliable hybrid modalities, PET/CT and PET/MRI. Applying AI-based algorithms on the different components (PET, CT, and MRI) has resulted in promising results, maximizing the value of PET imaging. However, [18F]F-FDG, the most commonly utilized tracer in molecular imaging, has been mainly in the spotlight. Thus, we aimed to look into the less discussed tracers in this review, moving beyond [18F]F-FDG. The novel non-[18F]F-FDG agents also showed to be valuable in various clinical tasks, including lesion detection and tumor characterization, accurate delineation, and prognostic impact. Regarding prostate patients, PSMA-based models were highly accurate in determining tumoral lesions' location and delineating them, particularly within the prostate gland. However, they also could assess whole-body images to detect extra-prostatic lesions in a patient automatically. Considering the prognostic value of prostate-specific membrane antigen (PSMA) PET using AI, it could predict response to treatment and patient survival, which are crucial in patient management. Choline imaging, another non-[18F]F-FDG tracer, similarly showed acceptable results that may be of benefit in the clinic, though the current evidence is significantly more limited than PSMA. Lastly, different subtypes of DOTA ligands were found to be valuable. They could diagnose tumoral lesions in challenging sites and even predict histopathology grade, being a highly advantageous noninvasive tool. In conclusion, the current limited investigations have shown promising results, leading us to a bright future for AI in molecular imaging beyond [18F]F-FDG.

Contrastive Learning for Prediction of Alzheimer's Disease Using Brain 18F-FDG PET

Brain 18F-FDG PET images are commonly-known materials for effectively predicting Alzheimer's disease (AD). However, the data volume of PET is usually insufficient, which is unfavorable to train an accurate AD prediction networks. Furthermore, the PET image is noisy with low signal-to-noise ratio, and simultaneously the feature (metabolic abnormality) used for predicting AD in PET image is not always obvious. Therefore, a contrastive-based learning method is proposed to address the challenges of PET image inherently possessed. Firstly, the slices of 3D PET image are amplified by cropping the image of anchors (i.e., an augmented version of the same image) to generate extended training data. Meanwhile, contrastive loss is adopted to enlarge inter-class feature distances and reduce intra-class feature differences using subject fuzzy labels as supervised information. Secondly, we construct a double convolutional hybrid attention module to enhance the network to learn different perceptual domains where two convolutional layers with different convolutional kernels ($7\times 7$ and $5\times 5$) are constructed. Moreover, we recommend a diagnosis mechanism by analyzing the consistency of predicted result for PET slices alone with clinical neuropsychological assessment to achieve a better AD diagnosis. The experimental results show that the proposed method outperforms the state-of-the-arts for brain 18F-FDG PET images, and hence demonstrate the advantage of the method in effectively predicting AD.

Clinical application of AI-based PET images in oncological patients

Based on the advantages of revealing the functional status and molecular expression of tumor cells, positron emission tomography (PET) imaging has been performed in numerous types of malignant diseases for diagnosis and monitoring. However, insufficient image quality, the lack of a convincing evaluation tool and intra- and interobserver variation in human work are well-known limitations of nuclear medicine imaging and restrict its clinical application. Artificial intelligence (AI) has gained increasing interest in the field of medical imaging due to its powerful information collection and interpretation ability. The combination of AI and PET imaging potentially provides great assistance to physicians managing patients. Radiomics, an important branch of AI applied in medical imaging, can extract hundreds of abstract mathematical features of images for further analysis. In this review, an overview of the applications of AI in PET imaging is provided, focusing on image enhancement, tumor detection, response and prognosis prediction and correlation analyses with pathology or specific gene mutations in several types of tumors. Our aim is to describe recent clinical applications of AI-based PET imaging in malignant diseases and to focus on the description of possible future developments.

Radiomics and artificial intelligence in prostate cancer: new tools for molecular hybrid imaging and theragnostics

In prostate cancer (PCa), the use of new radiopharmaceuticals has improved the accuracy of diagnosis and staging, refined surveillance strategies, and introduced specific and personalized radioreceptor therapies. Nuclear medicine, therefore, holds great promise for improving the quality of life of PCa patients, through managing and processing a vast amount of molecular imaging data and beyond, using a multi-omics approach and improving patients’ risk-stratification for tailored medicine. Artificial intelligence (AI) and radiomics may allow clinicians to improve the overall efficiency and accuracy of using these “big data” in both the diagnostic and theragnostic field: from technical aspects (such as semi-automatization of tumor segmentation, image reconstruction, and interpretation) to clinical outcomes, improving a deeper understanding of the molecular environment of PCa, refining personalized treatment strategies, and increasing the ability to predict the outcome. This systematic review aims to describe the current literature on AI and radiomics applied to molecular imaging of prostate cancer.

A systematic review of the applications of Expert Systems (ES) and machine learning (ML) in clinical urology

BACKGROUND

Testing a hypothesis for 'factors-outcome effect' is a common quest, but standard statistical regression analysis tools are rendered ineffective by data contaminated with too many noisy variables. Expert Systems (ES) can provide an alternative methodology in analysing data to identify variables with the highest correlation to the outcome. By applying their effective machine learning (ML) abilities, significant research time and costs can be saved. The study aims to systematically review the applications of ES in urological research and their methodological models for effective multi-variate analysis. Their domains, development and validity will be identified.

METHODS

The PRISMA methodology was applied to formulate an effective method for data gathering and analysis. This study search included seven most relevant information sources: WEB OF SCIENCE, EMBASE, BIOSIS CITATION INDEX, SCOPUS, PUBMED, Google Scholar and MEDLINE. Eligible articles were included if they applied one of the known ML models for a clear urological research question involving multivariate analysis. Only articles with pertinent research methods in ES models were included. The analysed data included the system model, applications, input/output variables, target user, validation, and outcomes. Both ML models and the variable analysis were comparatively reported for each system.

RESULTS

The search identified n = 1087 articles from all databases and n = 712 were eligible for examination against inclusion criteria. A total of 168 systems were finally included and systematically analysed demonstrating a recent increase in uptake of ES in academic urology in particular artificial neural networks with 31 systems. Most of the systems were applied in urological oncology (prostate cancer = 15, bladder cancer = 13) where diagnostic, prognostic and survival predictor markers were investigated. Due to the heterogeneity of models and their statistical tests, a meta-analysis was not feasible.

CONCLUSION

ES utility offers an effective ML potential and their applications in research have demonstrated a valid model for multi-variate analysis. The complexity of their development can challenge their uptake in urological clinics whilst the limitation of the statistical tools in this domain has created a gap for further research studies. Integration of computer scientists in academic units has promoted the use of ES in clinical urological research.

A review of the application of machine learning in molecular imaging

Molecular imaging (MI) is a science that uses imaging methods to reflect the changes of molecular level in living state and conduct qualitative and quantitative studies on its biological behaviors in imaging. Optical molecular imaging (OMI) and nuclear medical imaging are two key research fields of MI. OMI technology refers to the optical information generated by the imaging target (such as tumors) due to drug intervention and other reasons. By collecting the optical information, researchers can track the motion trajectory of the imaging target at the molecular level. Owing to its high specificity and sensitivity, OMI has been widely used in preclinical research and clinical surgery. Nuclear medical imaging mainly detects ionizing radiation emitted by radioactive substances. It can provide molecular information for early diagnosis, effective treatment and basic research of diseases, which has become one of the frontiers and hot topics in the field of medicine in the world today. Both OMI and nuclear medical imaging technology require a lot of data processing and analysis. In recent years, artificial intelligence technology, especially neural network-based machine learning (ML) technology, has been widely used in MI because of its powerful data processing capability. It provides a feasible strategy to deal with large and complex data for the requirement of MI. In this review, we will focus on the applications of ML methods in OMI and nuclear medical imaging.

18F-choline PET/MRI on initial staging of prostate cancer. Impact on therapy approach

AIM

Evaluate the therapy impact of initial staging in patients diagnosed with prostate cancer by ¹⁸ F-choline PET/MRI hybrid technique.

MATERIAL

A prospective study which included 31 patients diagnosed with prostate cancer; Gleason > 7; mean PSA 13.6 ng/mL (range 6.3-20.6). PET/MRI studies were acquired simultaneously with hybrid equipment (SIGNA.3T, GE) following intravenous injection of 185 ± 18.5MBq of ¹⁸F-choline: - Early/prostate imaging: PET emission + multiparametric MR: DIXON-T1-T2-diffusion-gadolinium. - Late/whole-body imaging: PET emission + MR: DIXON-T1-T2-diffusion-STIR sequences. Images were visually evaluated. SUV & ADC & textures were also calculated. Treatment selection was based upon Oncology Committee consensus decision.

RESULTS

Procedure was well tolerated in all patients, and no artifacts were reported. MRI was superior in T staging in eight patients (25.8%) (Likert: 2-3), whereas PET increased MRI sensitivity in three patients (9.7%) (PIRADS: 3).

PROSTATE LESION LOCATION

Peripheral 91.4%, transitional 8.6%. SUVmax threshold: 2.95: sensitivity 92.9%, specificity 66.7%. No correlation SUV vs. ADC. Better distinction between stage T2 vs. T3 using the DiscrLin model with NG = 16 (AUC 0.7767 ± 0.3386). PET was superior to T2 in textures analysis (0.588 vs. 0.412). Seventeen patients (54.8%) were staged ≥ T3, with surgical treatment being contraindicated. Fifteen patients (48.4%) presented with extra-prostatic disease: 8/31 oligometastatic and 7/31 multiple metastasis. Therapy approach following PET/MRI was: radical treatment in 24/31 patients (77.4%), 14 radical prostatectomy and 10 MRI-guided radiotherapy; systemic treatment in 7/31 patients (22.6%).

CONCLUSION

¹⁸F-choline PET/MRI had a complementary role for the T staging, with a high detection rate for NM infiltration. PET/MRI findings allowed patients to be directed either to prostatectomy or MRI-guided radiotherapy, and thus avoiding radicaltreatment in 22.6% of patients.

Fully Automated Segmentation and Shape Analysis of the Thoracic Aorta in Non-contrast-enhanced Magnetic Resonance Images of the German National Cohort Study

PURPOSE

The purpose of this study was to develop and validate a deep learning-based framework for automated segmentation and vessel shape analysis on non-contrast-enhanced magnetic resonance (MR) data of the thoracic aorta within the German National Cohort (GNC) MR study.

MATERIALS AND METHODS

One hundred data sets acquired in the GNC MR study were included (56 men, average age 53 y [22 to 72 y]). All participants had undergone non-contrast-enhanced MR imaging of the thoracic vessels. Automated vessel segmentation of the thoracic aorta was performed using a Convolutional Neural Network in a supervised setting with manually annotated data sets as the ground truth. Seventy data sets were used for training; 30 data sets were used for quantitative and qualitative evaluation. Automated shape analysis based on centerline extraction from segmentation masks was performed to derive a diameter profile of the vessel. For comparison, 2 radiologists measured vessel diameters manually.

RESULTS

Overall, automated aortic segmentation was successful, providing good qualitative analyses with only minor irregularities in 29 of 30 data sets. One data set with severe MR artifacts led to inadequate automated segmentation results. The mean Dice score of automated vessel segmentation was 0.85. Automated aortic diameter measurements were similar to manual measurements (average difference -0.9 mm, limits of agreement: -5.4 to 3.9 mm), with minor deviations in the order of the interreader agreement between the 2 radiologists (average difference -0.5 mm, limits of agreement: -5.8 to 4.8 mm).

CONCLUSION

Automated segmentation and shape analysis of the thoracic aorta is feasible with high accuracy on non-contrast-enhanced MR imaging using the proposed deep learning approach.

PET/MRI in prostate cancer: a systematic review and meta-analysis

Aim

In recent years, the clinical availability of scanners for integrated positron emission tomography (PET) and magnetic resonance imaging (MRI) has enabled the practical potential of multimodal, combined metabolic-receptor, anatomical, and functional imaging to be explored. The present systematic review and meta-analysis summarize the diagnostic information provided by PET/MRI in patients with prostate cancer (PCa).

Materials and methods

A literature search was conducted in three different databases. The terms used were “choline” or “prostate-specific membrane antigen - PSMA” AND “prostate cancer” or “prostate” AND “PET/MRI” or “PET MRI” or “PET-MRI” or “positron emission tomography/magnetic resonance imaging.” All relevant records identified were combined, and the full texts were retrieved. Reports were excluded if (1) they did not consider hybrid PET/MRI; or (2) the sample size was < 10 patients; or (3) the raw data were not enough to enable the completion of a 2 × 2 contingency table.

Results

Fifty articles were eligible for systematic review, and 23 for meta-analysis. The pooled data concerned 2104 patients. Initial disease staging was the main indication for PET/MRI in 24 studies. Radiolabeled PSMA was the tracer most frequently used. In primary tumors, the pooled sensitivity for the patient-based analysis was 94.9%. At restaging, the pooled detection rate was 80.9% and was higher for radiolabeled PSMA than for choline (81.8% and 77.3%, respectively).

Conclusions

PET/MRI proved highly sensitive in detecting primary PCa, with a high detection rate for recurrent disease, particularly when radiolabeled PSMA was used.

Electronic supplementary material

The online version of this article (10.1007/s00259-020-05025-0) contains supplementary material, which is available to authorized users.

A Network-Based "Phenomics" Approach for Discovering Patient Subtypes From High-Throughput Cardiac Imaging Data

OBJECTIVES

The authors present a method that focuses on cohort matching algorithms for performing patient-to-patient comparisons along multiple echocardiographic parameters for predicting meaningful patient subgroups.

BACKGROUND

Recent efforts in collecting multiomics data open numerous opportunities for comprehensive integration of highly heterogenous data to classify a patient's cardiovascular state, eventually leading to tailored therapies.

METHODS

A total of 42 echocardiography features, including 2-dimensional and Doppler measurements, left ventricular (LV) and atrial speckle-tracking, and vector flow mapping data, were obtained in 297 patients. A similarity network was developed to delineate distinct patient phenotypes, and then neural network models were trained for discriminating the phenotypic presentations.

RESULTS

The patient similarity model identified 4 clusters (I to IV), with patients in each cluster showed distinctive clinical presentations based on American College of Cardiology/American Heart Association heart failure stage and the occurrence of short-term major adverse cardiac and cerebrovascular events. Compared with other clusters, cluster IV had a higher prevalence of stage C or D heart failure (78%; p < 0.001), New York Heart Association functional classes III or IV (61%; p < 0.001), and a higher incidence of major adverse cardiac and cerebrovascular events (p < 0.001). The neural network model showed robust prediction of patient clusters, with area under the receiver-operating characteristic curve ranging from 0.82 to 0.99 for the independent hold-out validation set.

CONCLUSIONS

Automated computational methods for phenotyping can be an effective strategy to fuse multidimensional parameters of LV structure and function. It can identify distinct cardiac phenogroups in terms of clinical characteristics, cardiac structure and function, hemodynamics, and outcomes.

Comparison of PET/MRI with multiparametric MRI in diagnosis of primary prostate cancer: A meta-analysis

OBJECTIVE

This meta-analysis aimed to compare the diagnostic performance of positron emission tomography (PET)/MRI using various radiotracers with multiparametric (mp) MRI for detection of primary prostate cancer (PCa).

METHODS

A systematic literature search up to January 2019 was performed to identify studies that evaluated the diagnostic value of PET/MRI and mpMRI for detection of PCa in the same patient cohorts and had sufficient data to construct 2 × 2 contingency tables for true-positive (TP), false-positive (FP), false-negative (FN), and true-negative (TN) results. The quality of each study was assessed using the Quality Assessment of Diagnostic Accuracy Studies-2 tool, and pooled sensitivity (SEN) and specificity (SPE) were calculated. Summary receiver operating characteristic (ROC) curves and area under the curves (AUCs) were used to compare the performances of PET/MRI and mpMRI.

RESULTS

We identified 9 eligible studies that included a total of 353 patients. PET/MRI had a SEN of 0.783 (95% CI, 0.758-0.807) and a SPE of 0.899 (95% CI, 0.879-0.917), and mpMRI had a SEN of 0.603 (95% CI, 0.574-0.631) and a SPE of 0.887 (95% CI, 0.866-0.906). PET/MRI had a higher AUC than mpMRI (0.9311, 95% CI, 0.8990-0.9632 vs. 0.8403, 95% CI, 0.7864-0.8942; P =  0.0036). There was no notable publication bias, but there was medium heterogeneity in outcomes. The meta-regression analysis showed the major potential cause of heterogeneity was the use of region-based rather than lesion-based analysis.

CONCLUSION

PET/MRI has very good diagnostic performance and outperforms mpMRI for the diagnosis of primary PCa.

Automatic classification of tissues on pelvic MRI based on relaxation times and support vector machine

Tissue segmentation and classification in MRI is a challenging task due to a lack of signal intensity standardization. MRI signal is dependent on the acquisition protocol, the coil profile, the scanner type, etc. While we can compute quantitative physical tissue properties independent of the hardware and the sequence parameters, it is still difficult to leverage these physical properties to segment and classify pelvic tissues. The proposed method integrates quantitative MRI values (T1 and T2 relaxation times and pure synthetic weighted images) and machine learning (Support Vector Machine (SVM)) to segment and classify tissues in the pelvic region, i.e.: fat, muscle, prostate, bone marrow, bladder, and air. Twenty-two men with a mean age of 30±14 years were included in this prospective study. The images were acquired with a 3 Tesla MRI scanner. An inversion recovery-prepared turbo spin echo sequence was used to obtain T1-weighted images at different inversion times with a TR of 14000 ms. A 32-echo spin echo sequence was used to obtain the T2-weighted images at different echo times with a TR of 5000 ms. T1 and T2 relaxation times, synthetic T1- and T2-weighted images and anatomical probabilistic maps were calculated and used as input features of a SVM for segmenting and classifying tissues within the pelvic region. The mean SVM classification accuracy across subjects was calculated for the different tissues: prostate (94.2%), fat (96.9%), muscle (95.8%), bone marrow (91%) and bladder (82.1%) indicating an excellent classification performance. However, the segmentation and classification for air (within the rectum) may not always be successful (mean SVM accuracy 47.5%) due to the lack of air data in the training and testing sets. Our findings suggest that SVM can reliably segment and classify tissues in the pelvic region.

PET/MRI Hybrid Systems

Over the last decade, the combination of PET and MRI in one system has proven to be highly successful in basic preclinical research, as well as in clinical research. Nowadays, PET/MRI systems are well established in preclinical imaging and are progressing into clinical applications to provide further insights into specific diseases, therapeutic assessments, and biological pathways. Certain challenges in terms of hardware had to be resolved concurrently with the development of new techniques to be able to reach the full potential of both combined techniques. This review provides an overview of these challenges and describes the opportunities that simultaneous PET/MRI systems can exploit in comparison with stand-alone or other combined hybrid systems. New approaches were developed for simultaneous PET/MRI systems to correct for attenuation of 511 keV photons because MRI does not provide direct information on gamma photon attenuation properties. Furthermore, new algorithms to correct for motion were developed, because MRI can accurately detect motion with high temporal resolution. The additional information gained by the MRI can be employed to correct for partial volume effects as well. The development of new detector designs in combination with fast-decaying scintillator crystal materials enabled time-of-flight detection and incorporation in the reconstruction algorithms. Furthermore, this review lists the currently commercially available systems both for preclinical and clinical imaging and provides an overview of applications in both fields. In this regard, special emphasis has been placed on data analysis and the potential for both modalities to evolve with advanced image analysis tools, such as cluster analysis and machine learning.