Artificial intelligence neuropathologist for glioma classification using deep learning on hematoxylin and eosin stained slide images and molecular markers

Dual-path neural network extracts tumor microenvironment information from whole slide images to predict molecular typing and prognosis of Glioma

BACKGROUND AND OBJECTIVE

Utilizing AI to mine tumor microenvironment information in whole slide images (WSIs) for glioma molecular subtype and prognosis prediction is significant for treatment. Existing weakly-supervised learning frameworks based on multi-instance learning have potential in WSIs analysis, but the large number of patches from WSIs challenges the effective extraction of key local patch and neighboring patch microenvironment info. Therefore, this paper aims to develop an automatic neural network that effectively extracts tumor microenvironment information from WSIs to predict molecular typing and prognosis of glioma.

METHODS

In this paper, we proposed a dual-path pathology analysis (DPPA) framework to enhance the analysis ability of WSIs for glioma diagnosis. Firstly, to mitigate the impact of redundant patches and enhance the integration of salient patch information within a multi-instance learning context, we propose a two-stage attention-based dynamic multi-instance learning network. In the network, two-stage attention and dynamic random sampling are designed to integrate diverse image patch information in pivotal regions adaptively. Secondly, to unearth the wealth of spatial context inherent in WSIs, we build a spatial relationship information quantification module. This module captures the spatial distribution of patches that encompass a variety of tissue structures, shedding light on the tumor microenvironment.

RESULTS

A large number of experiments on three datasets, two in-house and one public, totaling 1,795 WSIs demonstrate the encouraging performance of the DPPA, with mean area under curves of 0.94, 0.85, and 0.88 in predicting Isocitrate Dehydrogenase 1, Telomerase Reverse Tranase, and 1p/19q respectively, and a mean C-index of 0.82 in prognosis prediction. The proposed model can also group tumors in existing tumor subgroups into good and bad prognoses, with P < 0.05 on the Log-rank test.

CONCLUSIONS

The results of multi-center experiments demonstrate that the proposed DPPA surpasses the state-of-the-art models across multiple metrics. Through ablation experiments and survival analysis, the outstanding analytical ability of this model is further validated. Meanwhile, based on the work related to the interpretability of the model, the reliability and validity of the model have also been strongly confirmed. All source codes are released at: https://github.com/nzehang97/DPPA.

Multimodal Ensemble Fusion Deep Learning Using Histopathological Images and Clinical Data for Glioma Subtype Classification

Glioma is the most common malignant tumor of the central nervous system, and diffuse Glioma is classified as grades II-IV by world health organization (WHO). In the the cancer genome atlas (TCGA) Glioma dataset, grade II and III Gliomas are classified as low-grade glioma (LGG), and grade IV Gliomas as glioblastoma multiforme (GBM). In clinical practice, the survival and treatment process with Glioma patients depends on properly diagnosing the subtype. With this background, there has been much research on Glioma over the years. Among these researches, the origin and evolution of whole slide images (WSIs) have led to many attempts to support diagnosis by image analysis. On the other hand, due to the disease complexities of Glioma patients, multimodal analysis using various types of data rather than a single data set has been attracting attention. In our proposed method, multiple deep learning models are used to extract features from histopathology images, and the features of the obtained images are concatenated with those of the clinical data in a fusion approach. Then, we perform patch-level classification by machine learning (ML) using the concatenated features. Based on the performances of the deep learning models, we ensemble feature sets from top three models and perform further classifications. In the experiments with our proposed ensemble fusion AI (EFAI) approach using WSIs and clinical data of diffuse Glioma patients on TCGA dataset, the classification accuracy of the proposed multimodal ensemble fusion method is 0.936 with an area under the curve (AUC) value of 0.967 when tested on a balanced dataset of 240 GBM, 240 LGG patients. On an imbalanced dataset of 141 GBM, 242 LGG patients the proposed method obtained the accuracy of 0.936 and AUC of 0.967. Our proposed ensemble fusion approach significantly outperforms the classification using only histopathology images alone with deep learning models. Therefore, our approach can be used to support the diagnosis of Glioma patients and can lead to better diagnosis.

A Multicenter Study on Intraoperative Glioma Grading via Deep Learning on Cryosection Pathology

Intraoperative glioma grading remains a significant challenge primarily due to the diminished diagnostic attributable to the suboptimal quality of cryosectioned slides. Precise intraoperative diagnosis is instrumental in guiding the surgical strategy to balance resection extent and neurologic function preservation, thereby optimizing patient prognoses. This study developed a model for intraoperative glioma grading via deep learning on cryosectioned images, termed intraoperative glioma grading on cryosection (IGGC). The model was trained and validated on The Cancer Genome Atlas data sets and 1 cohort (ntrain = 1603 and nvalidate = 628), and tested on 5 cohorts (ntest = 213). The IGGC model achieved an area under the receiver operating characteristic curve value of 0.99 in differentiating between high-grade glioma and low-grade glioma, and an area under the receiver operating characteristic curve value of 0.96 in identifying grade 4 glioma. Integrated into the clinical workflow, the IGGC model-assisted pathologists of varying experience levels in reducing interobserver variability and enhancing diagnostic consistency. This integrated diagnostic model possesses the potential for clinical implementation, offering a time-efficient and highly accurate method for the 3-grade classification of adult-type diffuse gliomas based on intraoperative cryosectioned slides.

Brain tumour histopathology through the lens of deep learning: A systematic review

PROBLEM

Machine learning (ML)/Deep learning (DL) techniques have been evolving to solve more complex diseases, but it has been used relatively little in Glioblastoma (GBM) histopathological studies, which could benefit greatly due to the disease's complex pathogenesis.

AIM

Conduct a systematic review to investigate how ML/DL techniques have influenced the progression of brain tumour histopathological research, particularly in GBM.

METHODS

54 eligible studies were collected from the PubMed and ScienceDirect databases, and their information about the types of brain tumour/s used, types of -omics data used with histopathological data, origins of the data, types of ML/DL and its training and evaluation methodologies, and the ML/DL task it was set to perform in the study were extracted to inform us of trends in GBM-related ML/DL-based research.

RESULTS

Only 8 GBM-related studies in the eligible utilised ML/DL methodologies to gain deeper insights into GBM pathogenesis by contextualising histological data with -omics data. However, we report that these studies have been published more recently. The most popular ML/DL models used in GBM-related research are the SVM classifier and ResNet-based CNN architecture. Still, a considerable number of studies failed to state training and evaluative methodologies clearly.

CONCLUSION

There is a growing trend towards using ML/DL approaches to uncover relationships between biological and histopathological data to bring new insights into GBM, thus pushing GBM research forward. Much work still needs to be done to properly report the ML/DL methodologies to showcase the models' robustness and generalizability and ensure the models are reproducible.

Can we rely on machine learning algorithms as a trustworthy predictor for recurrence in high-grade glioma? A systematic review and meta-analysis

Early prediction of recurrence in high-grade glioma (HGG) is critical due to its aggressive nature and poor prognosis. Distinguishing true recurrence from treatment-related changes, such as radionecrosis, is a major diagnostic challenge. Machine learning (ML) offers a novel approach, leveraging advanced algorithms to analyze complex imaging data with high precision. A comprehensive search of PubMed, Embase, Scopus, Web of Science, and Google Scholar identified eligible studies. The sensitivity, specificity, accuracy, precision, F1 score, and the (area under the curve) AUC items were extracted from the included studies. After screening 1077 records, seven studies met the eligibility criteria for the systematic review, of which five were included in the meta-analysis. ML algorithm, particularly Support Vector Machines (SVM), demonstrated promising performance. A meta-analysis of five studies revealed a pooled sensitivity of 0.95 (95% CI: 0.84-0.99) and specificity of 0.80 (95% CI: 0.69-0.88). Additionally, the positive diagnostic likelihood ratio (DLR) was 4.75 (95% CI: 2.91-7.76), the negative DLR was 0.06 (95% CI: 0.02-0.21), and the diagnostic odds ratio was 80.97 (95% CI: 17.5-374.61). The diagnostic score was calculated as 4.39 (95% CI: 2.86-5.93), and the AUC was 0.86 (95% CI: 0.83-0.89). Subgroup analyses showed SVM-based models with higher sensitivity (0.98 vs. 0.87) and specificity (0.82 vs. 0.77) than non-SVM (p = 0.13). Comparing glioblastoma and Grade 3 tumors, sensitivities were 94 % vs. 97 %, and specificities were 79 % vs. 83 %, with no significant heterogeneity. These findings suggest that ML models, particularly SVM, offer promising diagnostic performance in distinguishing true tumor recurrence from treatment-related changes.

Weakly supervised deep learning-based classification for histopathology of gliomas: a single center experience

Multiple artificial intelligence systems have been created to facilitate accurate and prompt histopathological diagnosis of tumors using hematoxylin-eosin-stained slides. We aimed to investigate whether weakly supervised deep learning can aid in glioma diagnosis. We analyzed 472 whole slide images (WSIs) from 226 patients in West China Hospital (WCH) and 1604 WSIs from 880 patients in The Cancer Genome Atlas (TCGA). We utilized the OpenSlide library to load WSIs, segmented them into small patches using the DeepZoom module, and then normalized the color using the Reinhard method. A weakly supervised deep learning model was developed using ResNet-50 combined with an attention mechanism. We investigated the performance of the model by calculating area under the curve (AUC) in a ten-fold cross-validation setting. Heatmap visualizations showed the prediction mechanism of the model. The results were promising, with high AUC values for differentiating grades of astrocytomas, oligodendrogliomas, all gliomas, and glioma types in the TCGA dataset (0.9419, 0.8659, 0.9904, and 0.9298, respectively), and in the WCH cohort (0.9048, 0.7423, 0.9510, and 0.7098, respectively). The model demonstrated a strong ability to infer IDH status in the TCGA dataset (AUC = 0.9488). The weakly supervised deep learning model proved to be an effective and reliable tool for neuropathological diagnosis, making it an attractive auxiliary tool.

The emerging role of artificial intelligence in neuropathology: Where are we and where do we want to go?

The field of neuropathology, a subspecialty of pathology which studies the diseases affecting the nervous system, is experiencing significant changes due to advancements in artificial intelligence (AI). Traditionally reliant on histological methods and clinical correlations, neuropathology is now experiencing a revolution due to the development of AI technologies like machine learning (ML) and deep learning (DL). These technologies enhance diagnostic accuracy, optimize workflows, and enable personalized treatment strategies. AI algorithms excel at analyzing histopathological images, often revealing subtle morphological changes missed by conventional methods. For example, deep learning models applied to digital pathology can effectively differentiate tumor grades and detect rare pathologies, leading to earlier and more precise diagnoses. Progress in neuroimaging is another helpful tool of AI, as enhanced analysis of MRI and CT scans supports early detection of neurodegenerative diseases. By identifying biomarkers and progression patterns, AI aids in timely therapeutic interventions, potentially slowing disease progression. In molecular pathology, AI's ability to analyze complex genomic data helps uncover the genetic and molecular basis of neuropathological conditions, facilitating personalized treatment plans. AI-driven automation streamlines routine diagnostic tasks, allowing pathologists to focus on complex cases, especially in settings with limited resources. This review explores AI's integration into neuropathology, highlighting its current applications, benefits, challenges, and future directions.

Leveraging State-of-the-Art AI Algorithms in Personalized Oncology: From Transcriptomics to Treatment

BACKGROUND

Continuous breakthroughs in computational algorithms have positioned AI-based models as some of the most sophisticated technologies in the healthcare system. AI shows dynamic contributions in advancing various medical fields involving data interpretation and monitoring, imaging screening and diagnosis, and treatment response and survival prediction. Despite advances in clinical oncology, more effort must be employed to tailor therapeutic plans based on each patient's unique transcriptomic profile within the precision/personalized oncology frame. Furthermore, the standard analysis method is not compatible with the comprehensive deciphering of significant data streams, thus precluding the prediction of accurate treatment options.

METHODOLOGY

We proposed a novel approach that includes obtaining different tumour tissues and preparing RNA samples for comprehensive transcriptomic interpretation using specifically trained, programmed, and optimized AI-based models for extracting large data volumes, refining, and analyzing them. Next, the transcriptomic results will be scanned against an expansive drug library to predict the response of each target to the tested drugs. The obtained target-drug combination/s will be then validated using in vitro and in vivo experimental models. Finally, the best treatment combination option/s will be introduced to the patient. We also provided a comprehensive review discussing AI models' recent innovations and implementations to aid in molecular diagnosis and treatment planning.

RESULTS

The expected transcriptomic analysis generated by the AI-based algorithms will provide an inclusive genomic profile for each patient, containing statistical and bioinformatics analyses, identification of the dysregulated pathways, detection of the targeted genes, and recognition of molecular biomarkers. Subjecting these results to the prediction and pairing AI-based processes will result in statistical graphs presenting each target's likely response rate to various treatment options. Different in vitro and in vivo investigations will further validate the selection of the target drug/s pairs.

CONCLUSIONS

Leveraging AI models will provide more rigorous manipulation of large-scale datasets on specific cancer care paths. Such a strategy would shape treatment according to each patient's demand, thus fortifying the avenue of personalized/precision medicine. Undoubtedly, this will assist in improving the oncology domain and alleviate the burden of clinicians in the coming decade.

Deep Learning Classification and Quantification of Pejorative and Nonpejorative Architectures in Resected Hepatocellular Carcinoma from Digital Histopathologic Images

Liver resection is one of the best treatments for small hepatocellular carcinoma (HCC), but post-resection recurrence is frequent. Biotherapies have emerged as an efficient adjuvant treatment, making the identification of patients at high risk of recurrence critical. Microvascular invasion (mVI), poor differentiation, pejorative macrotrabecular architectures, and vessels encapsulating tumor clusters architectures are the most accurate histologic predictors of recurrence, but their evaluation is time-consuming and imperfect. Herein, a supervised deep learning-based approach with ResNet34 on 680 whole slide images (WSIs) from 107 liver resection specimens was used to build an algorithm for the identification and quantification of these pejorative architectures. This model achieved an accuracy of 0.864 at patch level and 0.823 at WSI level. To assess its robustness, it was validated on an external cohort of 29 HCCs from another hospital, with an accuracy of 0.787 at WSI level, affirming its generalization capabilities. Moreover, the largest connected areas of the pejorative architectures extracted from the model were positively correlated to the presence of mVI and the number of tumor emboli. These results suggest that the identification of pejorative architectures could be an efficient surrogate of mVI and have a strong predictive value for the risk of recurrence. This study is the first step in the construction of a composite predictive algorithm for early post-resection recurrence of HCC, including artificial intelligence-based features.

Revisiting the potential of regulated cell death in glioma treatment: a focus on autophagy-dependent cell death, anoikis, ferroptosis, cuproptosis, pyroptosis, immunogenic cell death, and the crosstalk between them

Gliomas are primary tumors that originate in the central nervous system. The conventional treatment options for gliomas typically encompass surgical resection and temozolomide (TMZ) chemotherapy. However, despite aggressive interventions, the median survival for glioma patients is merely about 14.6 months. Consequently, there is an urgent necessity to explore innovative therapeutic strategies for treating glioma. The foundational study of regulated cell death (RCD) can be traced back to Karl Vogt's seminal observations of cellular demise in toads, which were documented in 1842. In the past decade, the Nomenclature Committee on Cell Death (NCCD) has systematically classified and delineated various forms and mechanisms of cell death, synthesizing morphological, biochemical, and functional characteristics. Cell death primarily manifests in two forms: accidental cell death (ACD), which is caused by external factors such as physical, chemical, or mechanical disruptions; and RCD, a gene-directed intrinsic process that coordinates an orderly cellular demise in response to both physiological and pathological cues. Advancements in our understanding of RCD have shed light on the manipulation of cell death modulation - either through induction or suppression - as a potentially groundbreaking approach in oncology, holding significant promise. However, obstacles persist at the interface of research and clinical application, with significant impediments encountered in translating to therapeutic modalities. It is increasingly apparent that an integrative examination of the molecular underpinnings of cell death is imperative for advancing the field, particularly within the framework of inter-pathway functional synergy. In this review, we provide an overview of various forms of RCD, including autophagy-dependent cell death, anoikis, ferroptosis, cuproptosis, pyroptosis and immunogenic cell death. We summarize the latest advancements in understanding the molecular mechanisms that regulate RCD in glioma and explore the interconnections between different cell death processes. By comprehending these connections and developing targeted strategies, we have the potential to enhance glioma therapy through manipulation of RCD.

A graph-learning based model for automatic diagnosis of Sjögren's syndrome on digital pathological images: a multicentre cohort study

BACKGROUND

Sjögren's Syndrome (SS) is a rare chronic autoimmune disorder primarily affecting adult females, characterized by chronic inflammation and salivary and lacrimal gland dysfunction. It is often associated with systemic lupus erythematosus, rheumatoid arthritis and kidney disease, which can lead to increased mortality. Early diagnosis is critical, but traditional methods for diagnosing SS, mainly through histopathological evaluation of salivary gland tissue, have limitations.

METHODS

The study used 100 labial gland biopsy, creating whole-slide images (WSIs) for analysis. The proposed model, named Cell-tissue-graph-based pathological image analysis model (CTG-PAM) and based on graph theory, characterizes single-cell feature, cell-cell feature, and cell-tissue feature. Building upon these features, CTG-PAM achieves cellular-level classification, enabling lymphocyte recognition. Furthermore, it leverages connected component analysis techniques in the cell graph structure to perform SS diagnosis based on lymphocyte counts.

FINDINGS

CTG-PAM outperforms traditional deep learning methods in diagnosing SS. Its area under the receiver operating characteristic curve (AUC) is 1.0 for the internal validation dataset and 0.8035 for the external test dataset. This indicates high accuracy. The sensitivity of CTG-PAM for the external dataset is 98.21%, while the accuracy is 93.75%. In comparison, the sensitivity and accuracy for traditional deep learning methods (ResNet-50) are lower. The study also shows that CTG-PAM's diagnostic accuracy is closer to skilled pathologists compared to beginners.

INTERPRETATION

Our findings indicate that CTG-PAM is a reliable method for diagnosing SS. Additionally, CTG-PAM shows promise in enhancing the prognosis of SS patients and holds significant potential for the differential diagnosis of both non-neoplastic and neoplastic diseases. The AI model potentially extends its application to diagnosing immune cells in tumor microenvironments.

New vision of HookEfficientNet deep neural network: Intelligent histopathological recognition system of non-small cell lung cancer

BACKGROUND

Efficient and precise diagnosis of non-small cell lung cancer (NSCLC) is quite critical for subsequent targeted therapy and immunotherapy. Since the advent of whole slide images (WSIs), the transition from traditional histopathology to digital pathology has aroused the application of convolutional neural networks (CNNs) in histopathological recognition and diagnosis. HookNet can make full use of macroscopic and microscopic information for pathological diagnosis, but it cannot integrate other excellent CNN structures. The new version of HookEfficientNet is based on a combination of HookNet structure and EfficientNet that performs well in the recognition of general objects. Here, a high-precision artificial intelligence-guided histopathological recognition system was established by HookEfficientNet to provide a basis for the intelligent differential diagnosis of NSCLC.

METHODS

A total of 216 WSIs of lung adenocarcinoma (LUAD) and 192 WSIs of lung squamous cell carcinoma (LUSC) were recruited from the First Affiliated Hospital of Zhengzhou University. Deep learning methods based on HookEfficientNet, HookNet and EfficientNet B4-B6 were developed and compared with each other using area under the curve (AUC) and the Youden index. Temperature scaling was used to calibrate the heatmap and highlight the cancer region of interest. Four pathologists of different levels blindly reviewed 108 WSIs of LUAD and LUSC, and the diagnostic results were compared with the various deep learning models.

RESULTS

The HookEfficientNet model outperformed HookNet and EfficientNet B4-B6. After temperature scaling, the HookEfficientNet model achieved AUCs of 0.973, 0.980, and 0.989 and Youden index values of 0.863, 0.899, and 0.922 for LUAD, LUSC and normal lung tissue, respectively, in the testing set. The accuracy of the model was better than the average accuracy from experienced pathologists, and the model was superior to pathologists in the diagnosis of LUSC.

CONCLUSIONS

HookEfficientNet can effectively recognize LUAD and LUSC with performance superior to that of senior pathologists, especially for LUSC. The model has great potential to facilitate the application of deep learning-assisted histopathological diagnosis for LUAD and LUSC in the future.

Comprehensive quantitative radiogenomic evaluation reveals novel radiomic subtypes with distinct immune pattern in glioma

BACKGROUND

Accurate classification of gliomas is critical to the selection of immunotherapy, and MRI contains a large number of radiomic features that may suggest some prognostic relevant signals. We aim to predict new subtypes of gliomas using radiomic features and characterize their survival, immune, genomic profiles and drug response.

METHODS

We initially obtained 341 images of 36 patients from the CPTAC dataset for the development of deep learning models. Further 1812 images of 111 patients from TCGA_GBM and 152 images of 53 patients from TCGA_LGG were collected for testing and validation. A deep learning method based on Mask R-CNN was developed to identify new subtypes of glioma patients and compared the survival status, immune infiltration patterns, genomic signatures, specific drugs, and predictive models of different subtypes.

RESULTS

200 glioma patients (mean age, 33 years ± 19 [standard deviation]) were enrolled. The accuracy of the deep learning model for identifying tumor regions achieved 88.3 % (98/111) in the test set and 83 % (44/53) in the validation set. The sample was divided into two subtypes based on radiomic features showed different prognostic outcomes (hazard ratio, 2.70). According to the results of the immune infiltration analysis, the subtype with a poorer prognosis was defined as the immunosilencing radiomic (ISR) subtype (n = 43), and the other subtype was the immunoactivated radiomic (IAR) subtype (n = 53). Subtype-specific genomic signatures distinguished celllines into ISR celllines (n = 9) and control celllines (n = 13), and identified eight ISR-specific drugs, four of which were validated by the OCTAD database. Three machine learning-based classifiers showed that radiomic and genomic co-features better predicted the radiomic subtypes of gliomas.

CONCLUSIONS

These findings provide insights into how radiogenomic could identify specific subtypes that predict prognosis, immune and drug sensitivity in a non-invasive manner.

Bringing Artificial Intelligence (AI) into Environmental Toxicology Studies: A Perspective of AI-Enabled Zebrafish High-Throughput Screening

The booming development of artificial intelligence (AI) has brought excitement to many research fields that could benefit from its big data analysis capability for causative relationship establishment and knowledge generation. In toxicology studies using zebrafish, the microscopic images and videos that illustrate the developmental stages, phenotypic morphologies, and animal behaviors possess great potential to facilitate rapid hazard assessment and dissection of the toxicity mechanism of environmental pollutants. However, the traditional manual observation approach is both labor-intensive and time-consuming. In this Perspective, we aim to summarize the current AI-enabled image and video analysis tools to realize the full potential of AI. For image analysis, AI-based tools allow fast and objective determination of morphological features and extraction of quantitative information from images of various sorts. The advantages of providing accurate and reproducible results while avoiding human intervention play a critical role in speeding up the screening process. For video analysis, AI-based tools enable the tracking of dynamic changes in both microscopic cellular events and macroscopic animal behaviors. The subtle changes revealed by video analysis could serve as sensitive indicators of adverse outcomes. With AI-based toxicity analysis in its infancy, exciting developments and applications are expected to appear in the years to come.

Artificial intelligence in histopathological image analysis of central nervous system tumours: A systematic review

The convergence of digital pathology and artificial intelligence could assist histopathology image analysis by providing tools for rapid, automated morphological analysis. This systematic review explores the use of artificial intelligence for histopathological image analysis of digitised central nervous system (CNS) tumour slides. Comprehensive searches were conducted across EMBASE, Medline and the Cochrane Library up to June 2023 using relevant keywords. Sixty-eight suitable studies were identified and qualitatively analysed. The risk of bias was evaluated using the Prediction model Risk of Bias Assessment Tool (PROBAST) criteria. All the studies were retrospective and preclinical. Gliomas were the most frequently analysed tumour type. The majority of studies used convolutional neural networks or support vector machines, and the most common goal of the model was for tumour classification and/or grading from haematoxylin and eosin-stained slides. The majority of studies were conducted when legacy World Health Organisation (WHO) classifications were in place, which at the time relied predominantly on histological (morphological) features but have since been superseded by molecular advances. Overall, there was a high risk of bias in all studies analysed. Persistent issues included inadequate transparency in reporting the number of patients and/or images within the model development and testing cohorts, absence of external validation, and insufficient recognition of batch effects in multi-institutional datasets. Based on these findings, we outline practical recommendations for future work including a framework for clinical implementation, in particular, better informing the artificial intelligence community of the needs of the neuropathologist.

Machine learning-based pathomics signature of histology slides as a novel prognostic indicator in primary central nervous system lymphoma

PURPOSE

To develop and validate a pathomics signature for predicting the outcomes of Primary Central Nervous System Lymphoma (PCNSL).

METHODS

In this study, 132 whole-slide images (WSIs) of 114 patients with PCNSL were enrolled. Quantitative features of hematoxylin and eosin (H&E) stained slides were extracted using CellProfiler. A pathomics signature was established and validated. Cox regression analysis, receiver operating characteristic (ROC) curves, Calibration, decision curve analysis (DCA), and net reclassification improvement (NRI) were performed to assess the significance and performance.

RESULTS

In total, 802 features were extracted using a fully automated pipeline. Six machine-learning classifiers demonstrated high accuracy in distinguishing malignant neoplasms. The pathomics signature remained a significant factor of overall survival (OS) and progression-free survival (PFS) in the training cohort (OS: HR 7.423, p < 0.001; PFS: HR 2.143, p = 0.022) and independent validation cohort (OS: HR 4.204, p = 0.017; PFS: HR 3.243, p = 0.005). A significantly lower response rate to initial treatment was found in high Path-score group (19/35, 54.29%) as compared to patients in the low Path-score group (16/70, 22.86%; p < 0.001). The DCA and NRI analyses confirmed that the nomogram showed incremental performance compared with existing models. The ROC curve demonstrated a relatively sensitive and specific profile for the nomogram (1-, 2-, and 3-year AUC = 0.862, 0.932, and 0.927, respectively).

CONCLUSION

As a novel, non-invasive, and convenient approach, the newly developed pathomics signature is a powerful predictor of OS and PFS in PCNSL and might be a potential predictive indicator for therapeutic response.

Global trends and hotspots of ChatGPT in medical research: a bibliometric and visualized study

Objective

With the rapid advancement of Chat Generative Pre-Trained Transformer (ChatGPT) in medical research, our study aimed to identify global trends and focal points in this domain.

Method

All publications on ChatGPT in medical research were retrieved from the Web of Science Core Collection (WoSCC) by Clarivate Analytics from January 1, 2023, to January 31, 2024. The research trends and focal points were visualized and analyzed using VOSviewer and CiteSpace.

Results

A total of 1,239 publications were collected and analyzed. The USA contributed the largest number of publications (458, 37.145%) with the highest total citation frequencies (2,461) and the largest H-index. Harvard University contributed the highest number of publications (33) among all full-time institutions. The Cureus Journal of Medical Science published the most ChatGPT-related research (127, 10.30%). Additionally, Wiwanitkit V contributed the majority of publications in this field (20). "Artificial Intelligence (AI) and Machine Learning (ML)," "Education and Training," "Healthcare Applications," and "Data Analysis and Technology" emerged as the primary clusters of keywords. These areas are predicted to remain hotspots in future research in this field.

Conclusion

Overall, this study signifies the interdisciplinary nature of ChatGPT research in medicine, encompassing AI and ML technologies, education and training initiatives, diverse healthcare applications, and data analysis and technology advancements. These areas are expected to remain at the forefront of future research, driving continued innovation and progress in the field of ChatGPT in medical research.

A multicenter proof-of-concept study on deep learning-based intraoperative discrimination of primary central nervous system lymphoma

Accurate intraoperative differentiation of primary central nervous system lymphoma (PCNSL) remains pivotal in guiding neurosurgical decisions. However, distinguishing PCNSL from other lesions, notably glioma, through frozen sections challenges pathologists. Here we sought to develop and validate a deep learning model capable of precisely distinguishing PCNSL from non-PCNSL lesions, especially glioma, using hematoxylin and eosin (H&E)-stained frozen whole-slide images. Also, we compared its performance against pathologists of varying expertise. Additionally, a human-machine fusion approach integrated both model and pathologic diagnostics. In external cohorts, LGNet achieved AUROCs of 0.965 and 0.972 in distinguishing PCNSL from glioma and AUROCs of 0.981 and 0.993 in differentiating PCNSL from non-PCNSL lesions. Outperforming several pathologists, LGNet significantly improved diagnostic performance, further augmented to some extent by fusion approach. LGNet's proficiency in frozen section analysis and its synergy with pathologists indicate its valuable role in intraoperative diagnosis, particularly in discriminating PCNSL from glioma, alongside other lesions.

Computational Pathology for Prediction of Isocitrate Dehydrogenase Gene Mutation from Whole Slide Images in Adult Patients with Diffuse Glioma

Isocitrate dehydrogenase gene (IDH) mutation is one of the most important molecular markers of glioma. Accurate detection of IDH status is a crucial step for integrated diagnosis of adult-type diffuse gliomas. Herein, a clustering-based hybrid of a convolutional neural network and a vision transformer deep learning model was developed to detect IDH mutation status from annotation-free hematoxylin and eosin-stained whole slide pathologic images of 2275 adult patients with diffuse gliomas. For comparison, a pure convolutional neural network, a pure vision transformer, and a classic multiple-instance learning model were also assessed. The hybrid model achieved an area under the receiver operating characteristic curve of 0.973 in the validation set and 0.953 in the external test set, outperforming the other models. The hybrid model's ability in IDH detection between difficult subgroups with different IDH status but shared histologic features, achieving areas under the receiver operating characteristic curve ranging from 0.850 to 0.985 in validation and test sets. These data suggest that the proposed hybrid model has a potential to be used as a computational pathology tool for preliminary rapid detection of IDH mutation from whole slide images in adult patients with diffuse gliomas.

Study of radiochemotherapy decision-making for young high-risk low-grade glioma patients using a macroscopic and microscopic combined radiomics model

OBJECTIVES

As a few types of glioma, young high-risk low-grade gliomas (HRLGGs) have higher requirements for postoperative quality of life. Although adjuvant chemotherapy with delayed radiotherapy is the first treatment strategy for HRLGGs, not all HRLGGs benefit from it. Accurate assessment of chemosensitivity in HRLGGs is vital for making treatment choices. This study developed a multimodal fusion radiomics (MFR) model to support radiochemotherapy decision-making for HRLGGs.

METHODS

A MFR model combining macroscopic MRI and microscopic pathological images was proposed. Multiscale features including macroscopic tumor structure and microscopic histological layer and nuclear information were grabbed by unique paradigm, respectively. Then, these features were adaptively incorporated into the MFR model through attention mechanism to predict the chemosensitivity of temozolomide (TMZ) by means of objective response rate and progression free survival (PFS).

RESULTS

Macroscopic tumor texture complexity and microscopic nuclear size showed significant statistical differences (p < 0.001) between sensitivity and insensitivity groups. The MFR model achieved stable prediction results, with an area under the curve of 0.950 (95% CI: 0.942-0.958), sensitivity of 0.833 (95% CI: 0.780-0.848), specificity of 0.929 (95% CI: 0.914-0.936), positive predictive value of 0.833 (95% CI: 0.811-0.860), and negative predictive value of 0.929 (95% CI: 0.914-0.934). The predictive efficacy of MFR was significantly higher than that of the reported molecular markers (p < 0.001). MFR was also demonstrated to be a predictor of PFS.

CONCLUSIONS

A MFR model including radiomics and pathological features predicts accurately the response postoperative TMZ treatment.

CLINICAL RELEVANCE STATEMENT

Our MFR model could identify young high-risk low-grade glioma patients who can have the most benefit from postoperative upfront temozolomide (TMZ) treatment.

KEY POINTS

• Multimodal radiomics is proposed to support the radiochemotherapy of glioma. • Some macro and micro image markers related to tumor chemotherapy sensitivity are revealed. • The proposed model surpasses reported molecular markers, with a promising area under the curve (AUC) of 0.95.

Convolutional neural network-based magnetic resonance image differentiation of filum terminale ependymomas from schwannomas

PURPOSE

Preoperative diagnosis of filum terminale ependymomas (FTEs) versus schwannomas is difficult but essential for surgical planning and prognostic assessment. With the advancement of deep-learning approaches based on convolutional neural networks (CNNs), the aim of this study was to determine whether CNN-based interpretation of magnetic resonance (MR) images of these two tumours could be achieved.

METHODS

Contrast-enhanced MRI data from 50 patients with primary FTE and 50 schwannomas in the lumbosacral spinal canal were retrospectively collected and used as training and internal validation datasets. The diagnostic accuracy of MRI was determined by consistency with postoperative histopathological examination. T1-weighted (T1-WI), T2-weighted (T2-WI) and contrast-enhanced T1-weighted (CE-T1) MR images of the sagittal plane containing the tumour mass were selected for analysis. For each sequence, patient MRI data were randomly allocated to 5 groups that further underwent fivefold cross-validation to evaluate the diagnostic efficacy of the CNN models. An additional 34 pairs of cases were used as an external test dataset to validate the CNN classifiers.

RESULTS

After comparing multiple backbone CNN models, we developed a diagnostic system using Inception-v3. In the external test dataset, the per-examination combined sensitivities were 0.78 (0.71-0.84, 95% CI) based on T1-weighted images, 0.79 (0.72-0.84, 95% CI) for T2-weighted images, 0.88 (0.83-0.92, 95% CI) for CE-T1 images, and 0.88 (0.83-0.92, 95% CI) for all weighted images. The combined specificities were 0.72 based on T1-WI (0.66-0.78, 95% CI), 0.84 (0.78-0.89, 95% CI) based on T2-WI, 0.74 (0.67-0.80, 95% CI) for CE-T1, and 0.81 (0.76-0.86, 95% CI) for all weighted images. After all three MRI modalities were merged, the receiver operating characteristic (ROC) curve was calculated, and the area under the curve (AUC) was 0.93, with an accuracy of 0.87.

CONCLUSIONS

CNN based MRI analysis has the potential to accurately differentiate ependymomas from schwannomas in the lumbar segment.

Computational pathology-based weakly supervised prediction model for MGMT promoter methylation status in glioblastoma

Introduction

The methylation status of oxygen 6-methylguanine-DNA methyltransferase (MGMT) is closely related to the treatment and prognosis of glioblastoma. However, there are currently some challenges in detecting the methylation status of MGMT promoters. The hematoxylin and eosin (H&E)-stained histopathological slides have always been the gold standard for tumor diagnosis.

Methods

In this study, based on the TCGA database and H&E-stained Whole slide images (WSI) of Beijing Tiantan Hospital, we constructed a weakly supervised prediction model of MGMT promoter methylation status in glioblastoma by using two Transformer structure models.

Results

The accuracy scores of this model in the TCGA dataset and our independent dataset were 0.79 (AUC = 0.86) and 0.76 (AUC = 0.83), respectively.

Conclusion

The model demonstrates effective prediction of MGMT promoter methylation status in glioblastoma and exhibits some degree of generalization capability. At the same time, our study also shows that adding Patches automatic screening module to the computational pathology research framework of glioma can significantly improve the model effect.

Deep Learning: A Primer for Neurosurgeons

This chapter explores the transformative impact of deep learning (DL) on neurosurgery, elucidating its pivotal role in enhancing diagnostic performance, surgical planning, execution, and postoperative assessment. It delves into various deep learning architectures, including convolutional and recurrent neural networks, and their applications in analyzing neuroimaging data for brain tumors, spinal cord injuries, and other neurological conditions. The integration of DL in neurosurgical robotics and the potential for fully autonomous surgical procedures are discussed, highlighting advancements in surgical precision and patient outcomes. The chapter also examines the challenges of data privacy, quality, and interpretability that accompany the implementation of DL in neurosurgery. The potential for DL to revolutionize neurosurgical practices through improved diagnostics, patient-specific surgical planning, and the advent of intelligent surgical robots is underscored, promising a future where technology and healthcare converge to offer unprecedented solutions in neurosurgery.

Deep Multi-Dictionary Learning for Survival Prediction With Multi-Zoom Histopathological Whole Slide Images

Survival prediction based on histopathological whole slide images (WSIs) is of great significance for risk-benefit assessment and clinical decision. However, complex microenvironments and heterogeneous tissue structures in WSIs bring challenges to learning informative prognosis-related representations. Additionally, previous studies mainly focus on modeling using mono-scale WSIs, which commonly ignore useful subtle differences existed in multi-zoom WSIs. To this end, we propose a deep multi-dictionary learning framework for cancer survival prediction with multi-zoom histopathological WSIs. The framework can recognize and learn discriminative clusters (i.e., microenvironments) based on multi-scale deep representations for survival analysis. Specifically, we learn multi-scale features based on multi-zoom tiles from WSIs via stacked deep autoencoders network followed by grouping different microenvironments by cluster algorithm. Based on multi-scale deep features of clusters, a multi-dictionary learning method with a post-pruning strategy is devised to learn discriminative representations from selected prognosis-related clusters in a task-driven manner. Finally, a survival model (i.e., EN-Cox) is constructed to estimate the risk index of an individual patient. The proposed model is evaluated on three datasets derived from The Cancer Genome Atlas (TCGA), and the experimental results demonstrate that it outperforms several state-of-the-art survival analysis approaches.

A Short Review on the Impact of Artificial Intelligence in Diagnosis Diseases: Role of Radiomics In Neuro-Oncology

Artificial Intelligence (AI) is rapidly transforming various aspects of healthcare, including the field of diagnostics and treatment of diseases. This review article aimed to provide an in-depth analysis of the impact of AI, especially, radiomics in the diagnosis of neuro-oncology diseases. Indeed, it is a multidimensional task that requires the integration of clinical assessment, neuroimaging techniques, and emerging technologies like AI and radiomics. The advancements in these fields have the potential to revolutionize the accuracy, efficiency, and personalized approach to diagnosing neuro-oncology diseases, leading to improved patient outcomes and enhanced overall neurologic care. However, AI has some limitations, and ethical challenges should be addressed via future research.

Artificial intelligence in neuro-oncology

Artificial intelligence (AI) describes the application of computer algorithms to the solution of problems that have traditionally required human intelligence. Although formal work in AI has been slowly advancing for almost 70 years, developments in the last decade, and particularly in the last year, have led to an explosion of AI applications in multiple fields. Neuro-oncology has not escaped this trend. Given the expected integration of AI-based methods to neuro-oncology practice over the coming years, we set to provide an overview of existing technologies as they are applied to the neuropathology and neuroradiology of brain tumors. We highlight current benefits and limitations of these technologies and offer recommendations on how to appraise novel AI-tools as they undergo consideration for integration into clinical workflows.

Tumor Mutation Burden-Related Histopathologic Features for Predicting Overall Survival in Gliomas Using Graph Deep Learning

Tumor mutation burden (TMB) is a potential biomarker for evaluating the prognosis and response to immune checkpoint inhibitors, but its costly and time-consuming method of measurement limits its widespread application. This study aimed to identify the TMB-related histopathologic features from hematoxylin and eosin slides and explore their prognostic value in gliomas. TMB-related features were detected using a graph convolutional neural network from whole-slide images of patients from The Cancer Genome Atlas data set (619 patients), and the correlation between features and TMB was evaluated in an external validation set (237 patients). TMB-related features were used for predicting overall survival (OS) of patients to investigate whether these features have potential for prognostic prediction. Moreover, biological pathways underlying the prognostic value of the features were further explored. Histopathologic features derived from whole-slide images were significantly associated with patient TMB (P = 0.007 in the external validation set). TMB-related features showed excellent performance for OS prediction, and patients with lower-grade gliomas could be further stratified into different risk groups according to the features (P = 0.00013; hazard ratio, 4.004). Pathways involved in the cell cycle and execution of immune response were enriched in patients with higher OS risk. The TMB-related features could be used to estimate TMB and aid in prognostic risk stratification of patients with glioma with dysregulated biological pathways.

methylClass: an R package to construct DNA methylation-based classification models

DNA methylation profiling is a useful tool to increase the accuracy of a cancer diagnosis. However, a comprehensive R package specially for it is lacking. Hence, we developed the R package methylClass for methylation-based classification. Within it, we provide the eSVM (ensemble-based support vector machine) model to achieve much higher accuracy in methylation data classification than the popular random forest model and overcome the time-consuming problem of the traditional SVM. In addition, some novel feature selection methods are included in the package to improve the classification. Furthermore, because methylation data can be converted to other omics, such as copy number variation data, we also provide functions for multi-omics studies. The testing of this package on four datasets shows the accurate performance of our package, especially eSVM, which can be used in both methylation and multi-omics models and outperforms other methods in both cases. methylClass is available at: https://github.com/yuabrahamliu/methylClass.

A multi-center performance assessment for automated histopathological classification and grading of glioma using whole slide images

Accurate pathological classification and grading of gliomas is crucial in clinical diagnosis and treatment. The application of deep learning techniques holds promise for automated histological pathology diagnosis. In this study, we collected 733 whole slide images from four medical centers, of which 456 were used for model training, 150 for internal validation, and 127 for multi-center testing. The study includes 5 types of common gliomas. A subtask-guided multi-instance learning image-to-label training pipeline was employed. The pipeline leveraged "patch prompting" for the model to converge with reasonable computational cost. Experiments showed that an overall accuracy of 0.79 in the internal validation dataset. The performance on the multi-center testing dataset showed an overall accuracy to 0.73. The findings suggest a minor yet acceptable performance decrease in multi-center data, demonstrating the model's strong generalizability and establishing a robust foundation for future clinical applications.

A novel deep learning model for glioma epilepsy associated with the identification of human cytomegalovirus infection injuries based on head MR

Purpose

In this study, a deep learning model was established based on head MRI to predict a crucial evaluation parameter in the assessment of injuries resulting from human cytomegalovirus infection: the occurrence of glioma-related epilepsy. The relationship between glioma and epilepsy was investigated, which serves as a significant indicator of labor force impairment.

Methods

This study enrolled 142 glioma patients, including 127 from Shengjing Hospital of China Medical University, and 15 from the Second Affiliated Hospital of Dalian Medical University. T1 and T2 sequence images of patients' head MRIs were utilized to predict the occurrence of glioma-associated epilepsy. To validate the model's performance, the results of machine learning and deep learning models were compared. The machine learning model employed manually annotated texture features from tumor regions for modeling. On the other hand, the deep learning model utilized fused data consisting of tumor-containing T1 and T2 sequence images for modeling.

Results

The neural network based on MobileNet_v3 performed the best, achieving an accuracy of 86.96% on the validation set and 75.89% on the test set. The performance of this neural network model significantly surpassed all the machine learning models, both on the validation and test sets.

Conclusion

In this study, we have developed a neural network utilizing head MRI, which can predict the likelihood of glioma-associated epilepsy in untreated glioma patients based on T1 and T2 sequence images. This advancement provides forensic support for the assessment of injuries related to human cytomegalovirus infection.

Quantitative analysis of prion disease using an AI-powered digital pathology framework

Prion disease is a fatal neurodegenerative disorder characterized by accumulation of an abnormal prion protein (PrPSc) in the central nervous system. To identify PrPSc aggregates for diagnostic purposes, pathologists use immunohistochemical staining of prion protein antibodies on tissue samples. With digital pathology, artificial intelligence can now analyze stained slides. In this study, we developed an automated pipeline for the identification of PrPSc aggregates in tissue samples from the cerebellar and occipital cortex. To the best of our knowledge, this is the first framework to evaluate PrPSc deposition in digital images. We used two strategies: a deep learning segmentation approach using a vision transformer, and a machine learning classification approach with traditional classifiers. Our method was developed and tested on 64 whole slide images from 41 patients definitively diagnosed with prion disease. The results of our study demonstrated that our proposed framework can accurately classify WSIs from a blind test set. Moreover, it can quantify PrPSc distribution and localization throughout the brain. This could potentially be extended to evaluate protein expression in other neurodegenerative diseases like Alzheimer's and Parkinson's. Overall, our pipeline highlights the potential of AI-assisted pathology to provide valuable insights, leading to improved diagnostic accuracy and efficiency.

Neuropathologist-level integrated classification of adult-type diffuse gliomas using deep learning from whole-slide pathological images

Current diagnosis of glioma types requires combining both histological features and molecular characteristics, which is an expensive and time-consuming procedure. Determining the tumor types directly from whole-slide images (WSIs) is of great value for glioma diagnosis. This study presents an integrated diagnosis model for automatic classification of diffuse gliomas from annotation-free standard WSIs. Our model is developed on a training cohort (n = 1362) and a validation cohort (n = 340), and tested on an internal testing cohort (n = 289) and two external cohorts (n = 305 and 328, respectively). The model can learn imaging features containing both pathological morphology and underlying biological clues to achieve the integrated diagnosis. Our model achieves high performance with area under receiver operator curve all above 0.90 in classifying major tumor types, in identifying tumor grades within type, and especially in distinguishing tumor genotypes with shared histological features. This integrated diagnosis model has the potential to be used in clinical scenarios for automated and unbiased classification of adult-type diffuse gliomas.

Efficient diagnosis of IDH-mutant gliomas: 1p/19qNET assesses 1p/19q codeletion status using weakly-supervised learning

Accurate identification of molecular alterations in gliomas is crucial for their diagnosis and treatment. Although, fluorescence in situ hybridization (FISH) allows for the observation of diverse and heterogeneous alterations, it is inherently time-consuming and challenging due to the limitations of the molecular method. Here, we report the development of 1p/19qNET, an advanced deep-learning network designed to predict fold change values of 1p and 19q chromosomes and classify isocitrate dehydrogenase (IDH)-mutant gliomas from whole-slide images. We trained 1p/19qNET on next-generation sequencing data from a discovery set (DS) of 288 patients and utilized a weakly-supervised approach with slide-level labels to reduce bias and workload. We then performed validation on an independent validation set (IVS) comprising 385 samples from The Cancer Genome Atlas, a comprehensive cancer genomics resource. 1p/19qNET outperformed traditional FISH, achieving R2 values of 0.589 and 0.547 for the 1p and 19q arms, respectively. As an IDH-mutant glioma classifier, 1p/19qNET attained AUCs of 0.930 and 0.837 in the DS and IVS, respectively. The weakly-supervised nature of 1p/19qNET provides explainable heatmaps for the results. This study demonstrates the successful use of deep learning for precise determination of 1p/19q codeletion status and classification of IDH-mutant gliomas as astrocytoma or oligodendroglioma. 1p/19qNET offers comparable results to FISH and provides informative spatial information. This approach has broader applications in tumor classification.

Standardizing analysis of intra-tumoral heterogeneity with computational pathology

Many malignant cancers like glioblastoma are highly adaptive diseases that dynamically change their regional biology to survive and thrive under diverse microenvironmental and therapeutic pressures. While the concept of intra-tumoral heterogeneity has become a major paradigm in cancer research and care, systematic approaches to assess and document bio-variation in cancer are still in their infancy. Here we discuss existing approaches and challenges to documenting intra-tumoral heterogeneity and emerging computational approaches that leverage artificial intelligence to begin to overcome these limitations. We propose how these emerging techniques can be coupled with a diversity of molecular tools to address intra-tumoral heterogeneity more systematically in research and in practice, especially across larger specimens and longitudinal analyses. Systematic documentation and characterization of heterogeneity across entire tumor specimens and their longitudinal evolution has the potential to improve our understanding and treatment of cancer.

Machine learning for cryosection pathology predicts the 2021 WHO classification of glioma

BACKGROUND

Timely and accurate intraoperative cryosection evaluations remain the gold standard for guiding surgical treatments for gliomas. However, the tissue-freezing process often generates artifacts that make histologic interpretation difficult. In addition, the 2021 WHO Classification of Tumors of the Central Nervous System incorporates molecular profiles in the diagnostic categories, so standard visual evaluation of cryosections alone cannot completely inform diagnoses based on the new classification system.

METHODS

To address these challenges, we develop the context-aware Cryosection Histopathology Assessment and Review Machine (CHARM) using samples from 1,524 glioma patients from three different patient populations to systematically analyze cryosection slides.

FINDINGS

Our CHARM models successfully identified malignant cells (AUROC = 0.98 ± 0.01 in the independent validation cohort), distinguished isocitrate dehydrogenase (IDH)-mutant tumors from wild type (AUROC = 0.79-0.82), classified three major types of molecularly defined gliomas (AUROC = 0.88-0.93), and identified the most prevalent subtypes of IDH-mutant tumors (AUROC = 0.89-0.97). CHARM further predicts clinically important genetic alterations in low-grade glioma, including ATRX, TP53, and CIC mutations, CDKN2A/B homozygous deletion, and 1p/19q codeletion via cryosection images.

CONCLUSIONS

Our approaches accommodate the evolving diagnostic criteria informed by molecular studies, provide real-time clinical decision support, and will democratize accurate cryosection diagnoses.

FUNDING

Supported in part by the National Institute of General Medical Sciences grant R35GM142879, the Google Research Scholar Award, the Blavatnik Center for Computational Biomedicine Award, the Partners' Innovation Discovery Grant, and the Schlager Family Award for Early Stage Digital Health Innovations.

Artificial Intelligence-Assisted Classification of Gliomas Using Whole Slide Images

CONTEXT.—

Glioma is the most common primary brain tumor in adults. The diagnosis and grading of different pathological subtypes of glioma is essential in treatment planning and prognosis.

OBJECTIVE.—

To propose a deep learning-based approach for the automated classification of glioma histopathology images. Two classification methods, the ensemble method based on 2 binary classifiers and the multiclass method using a single multiclass classifier, were implemented to classify glioma images into astrocytoma, oligodendroglioma, and glioblastoma, according to the 5th edition of the World Health Organization classification of central nervous system tumors, published in 2021.

DESIGN.—

We tested 2 different deep neural network architectures (VGG19 and ResNet50) and extensively validated the proposed approach based on The Cancer Genome Atlas data set (n = 700). We also studied the effects of stain normalization and data augmentation on the glioma classification task.

RESULTS.—

With the binary classifiers, our model could distinguish astrocytoma and oligodendroglioma (combined) from glioblastoma with an accuracy of 0.917 (area under the curve [AUC] = 0.976) and astrocytoma from oligodendroglioma (accuracy = 0.821, AUC = 0.865). The multiclass method (accuracy = 0.861, AUC = 0.961) outperformed the ensemble method (accuracy = 0.847, AUC = 0.933) with the best performance displayed by the ResNet50 architecture.

CONCLUSIONS.—

With the high performance of our model (>80%), the proposed method can assist pathologists and physicians to support examination and differential diagnosis of glioma histopathology images, with the aim to expedite personalized medical care.

Recent advances of pathomics in colorectal cancer diagnosis and prognosis

Colorectal cancer (CRC) is one of the most common malignancies, with the third highest incidence and the second highest mortality in the world. To improve the therapeutic outcome, the risk stratification and prognosis predictions would help guide clinical treatment decisions. Achieving these goals have been facilitated by the fast development of artificial intelligence (AI) -based algorithms using radiological and pathological data, in combination with genomic information. Among them, features extracted from pathological images, termed pathomics, are able to reflect sub-visual characteristics linking to better stratification and prediction of therapeutic responses. In this paper, we review recent advances in pathological image-based algorithms in CRC, focusing on diagnosis of benign and malignant lesions, micro-satellite instability, as well as prediction of neoadjuvant chemoradiotherapy and the prognosis of CRC patients.

Application of artificial intelligence to digital-rapid on-site cytopathology evaluation during endoscopic ultrasound-guided fine needle aspiration: A proof-of-concept study

BACKGROUND

During endoscopic ultrasound-guided fine needle aspiration (EUS-FNA), cytopathology with rapid on-site evaluation (ROSE) can improve diagnostic yield and accuracy. However, ROSE is unavailable in most Asian and European institutions because of the shortage of cytopathologists. Therefore, developing computer-assisted diagnostic tools to replace manual ROSE is crucial. Herein, we reported the validation of an artificial intelligence (AI)-based model (ROSE-AI model) to substitute manual ROSE during EUS-FNA.

METHODS

A total of 467 digitized images from Diff-Quik (D&F)-stained EUS-FNA slides were divided into training (3642 tiles from 367 images) and internal validation (916 tiles from 100 images) datasets. The ROSE-AI model was trained and validated using training and internal validation datasets, respectively. The specificity was emphasized while developing the model. Then, we evaluated the AI model on a 693-image external dataset. We assessed the performance of the AI model to detect cancer cells (CCs) regarding the accuracy, sensitivity, specificity, positive predictive value (PPV), and negative predictive value (NPV).

RESULTS

The ROSE-AI model achieved an accuracy of 83.4% in the internal validation dataset and 88.7% in the external test dataset. The sensitivity and PPV were 79.1% and 71.7% in internal validation dataset and 78.0% and 60.7% in external test dataset, respectively.

CONCLUSION

We provided a proof of concept that AI can be used to replace manual ROSE during EUS-FNA. The ROSE-AI model can address the shortage of cytopathologists and make ROSE available in more institutes.

Histopathological auxiliary system for brain tumour (HAS-Bt) based on weakly supervised learning using a WHO CNS5-style pipeline

PURPOSE

Classification and grading of central nervous system (CNS) tumours play a critical role in the clinic. When WHO CNS5 simplifies the histopathology diagnosis and places greater emphasis on molecular pathology, artificial intelligence (AI) has been widely used to meet the increased need for an automatic histopathology scheme that could liberate pathologists from laborious work. This study was to explore the diagnosis scope and practicality of AI.

METHODS

A one-stop Histopathology Auxiliary System for Brain tumours (HAS-Bt) is introduced based on a pipeline-structured multiple instance learning (pMIL) framework developed with 1,385,163 patches from 1038 hematoxylin and eosin (H&E) slides. The system provides a streamlined service including slide scanning, whole-slide image (WSI) analysis and information management. A logical algorithm is used when molecular profiles are available.

RESULTS

The pMIL achieved an accuracy of 0.94 in a 9-type classification task on an independent dataset composed of 268 H&E slides. Three auxiliary functions are developed and a built-in decision tree with multiple molecular markers is used to automatically formed integrated diagnosis. The processing efficiency was 443.0 s per slide.

CONCLUSION

HAS-Bt shows outstanding performance and provides a novel aid for the integrated neuropathological diagnostic workflow of brain tumours using CNS 5 pipeline.

Combination of multi-modal MRI radiomics and liquid biopsy technique for preoperatively non-invasive diagnosis of glioma based on deep learning: protocol for a double-center, ambispective, diagnostical observational study

Background

2021 World Health Organization (WHO) Central Nervous System (CNS) tumor classification increasingly emphasizes the important role of molecular markers in glioma diagnoses. Preoperatively non-invasive "integrated diagnosis" will bring great benefits to the treatment and prognosis of these patients with special tumor locations that cannot receive craniotomy or needle biopsy. Magnetic resonance imaging (MRI) radiomics and liquid biopsy (LB) have great potential for non-invasive diagnosis of molecular markers and grading since they are both easy to perform. This study aims to build a novel multi-task deep learning (DL) radiomic model to achieve preoperative non-invasive "integrated diagnosis" of glioma based on the 2021 WHO-CNS classification and explore whether the DL model with LB parameters can improve the performance of glioma diagnosis.

Methods

This is a double-center, ambispective, diagnostical observational study. One public database named the 2019 Brain Tumor Segmentation challenge dataset (BraTS) and two original datasets, including the Second Affiliated Hospital of Nanchang University, and Renmin Hospital of Wuhan University, will be used to develop the multi-task DL radiomic model. As one of the LB techniques, circulating tumor cell (CTC) parameters will be additionally applied in the DL radiomic model for assisting the "integrated diagnosis" of glioma. The segmentation model will be evaluated with the Dice index, and the performance of the DL model for WHO grading and all molecular subtype will be evaluated with the indicators of accuracy, precision, and recall.

Discussion

Simply relying on radiomics features to find the correlation with the molecular subtypes of gliomas can no longer meet the need for "precisely integrated prediction." CTC features are a promising biomarker that may provide new directions in the exploration of "precision integrated prediction" based on the radiomics, and this is the first original study that combination of radiomics and LB technology for glioma diagnosis. We firmly believe that this innovative work will surely lay a good foundation for the "precisely integrated prediction" of glioma and point out further directions for future research.

Clinical trail registration

This study was registered on ClinicalTrails.gov on 09/10/2022 with Identifier NCT05536024.

Relationship between IDH1/2 and TERT promoter mutation and the prognosis of human glioma patients

Objective

To investigate the relationship between isocitrate dehydrogenase (IDH) 1/2 mutation, telomerase reverse transcriptase (TERT) gene promoter mutation and the prognosis of human glioma patients.

Methods

One hundred fifteen patients with human glioma, treated surgically in The First Affiliated Hospital of Hebei North University from January 2019 to January 2020, were included. All patients were followed up until January 31, 2022. The mutations of IDH1/2 and TERT promoter were analyzed, and risk factors affecting survival of the patients with glioma were assessed.

Results

IDH1 gene mutation occurred in 82 cases, IDH2 gene mutation occurred in five cases and TERT promoter mutation occurred in 54 cases. Univariate analysis showed that tumor WHO grade, resection range, preoperative Karnofsky performance status score, postoperative radiotherapy and chemotherapy, IDH1/2 gene and TERT promoter mutation influenced postoperative survival of patients with glioma (P<0.05). Kaplan-Meier survival curve showed that IDH1/2 gene and TERT promoter mutation were significantly different from those of wild-type patients (P<0.05).

Conclusion

IDH1/2 gene and TERT promoter mutations are more frequent in patients with human glioma. These related factors can be used as molecular markers to aid in the prognosis of patients with glioma.

Contribution of whole slide imaging-based deep learning in the assessment of intraoperative and postoperative sections in neuropathology

The pathological diagnosis of intracranial germinoma (IG), oligodendroglioma, and low-grade astrocytoma on intraoperative frozen section (IFS) and hematoxylin-eosin (HE)-staining section directly determines patients' treatment options, but it is a difficult task for pathologists. We aimed to investigate whether whole-slide imaging (WSI)-based deep learning can contribute new precision to the diagnosis of IG, oligodendroglioma, and low-grade astrocytoma. Two types of WSIs (500 IFSs and 832 HE-staining sections) were collected from 379 patients at multiple medical centers. Patients at Center 1 were split into the training, testing, and internal validation sets (3:1:1), while the other centers were the external validation sets. First, we subdivided WSIs into small tiles and selected tissue tiles using a tissue tile selection model. Then a tile-level classification model was established, and the majority voting method was used to determine the final diagnoses. Color jitter was applied to the tiles so that the deep learning (DL) models could adapt to the variations in the staining. Last, we investigated the effectiveness of model assistance. The internal validation accuracies of the IFS and HE models were 93.9% and 95.3%, respectively. The external validation accuracies of the IFS and HE models were 82.0% and 76.9%, respectively. Furthermore, the IFS and HE models can predict Ki-67 positive cell areas with R² of 0.81 and 0.86, respectively. With model assistance, the IFS and HE diagnosis accuracy of pathologists improved from 54.6%-69.7% and 53.5%-83.7% to 87.9%-93.9% and 86.0%-90.7%, respectively. Both the IFS model and the HE model can differentiate the three tumors, predict the expression of Ki-67, and improve the diagnostic accuracy of pathologists. The use of our model can assist clinicians in providing patients with optimal and timely treatment options.

uRP: An integrated research platform for one-stop analysis of medical images

Introduction

Medical image analysis is of tremendous importance in serving clinical diagnosis, treatment planning, as well as prognosis assessment. However, the image analysis process usually involves multiple modality-specific software and relies on rigorous manual operations, which is time-consuming and potentially low reproducible.

Methods

We present an integrated platform - uAI Research Portal (uRP), to achieve one-stop analyses of multimodal images such as CT, MRI, and PET for clinical research applications. The proposed uRP adopts a modularized architecture to be multifunctional, extensible, and customizable.

Results and Discussion

The uRP shows 3 advantages, as it 1) spans a wealth of algorithms for image processing including semi-automatic delineation, automatic segmentation, registration, classification, quantitative analysis, and image visualization, to realize a one-stop analytic pipeline, 2) integrates a variety of functional modules, which can be directly applied, combined, or customized for specific application domains, such as brain, pneumonia, and knee joint analyses, 3) enables full-stack analysis of one disease, including diagnosis, treatment planning, and prognosis assessment, as well as full-spectrum coverage for multiple disease applications. With the continuous development and inclusion of advanced algorithms, we expect this platform to largely simplify the clinical scientific research process and promote more and better discoveries.

Predicting the recurrence and overall survival of patients with glioma based on histopathological images using deep learning

Background

A deep learning (DL) model based on representative biopsy tissues can predict the recurrence and overall survival of patients with glioma, leading to optimized personalized medicine. This research aimed to develop a DL model based on hematoxylin-eosin (HE) stained pathological images and verify its diagnostic accuracy.

Methods

Our study retrospectively collected 162 patients with glioma and randomly divided them into a training set (n = 113) and a validation set (n = 49) to build a DL model. The HE-stained slide was segmented into a size of 180 × 180 pixels without overlapping. The patch-level features were extracted by the pre-trained ResNet50 to predict the recurrence and overall survival. Additionally, a light-strategy was introduced where low-size digital biopsy images with clinical information were inputted into the DL model to ensure minimum memory occupation.

Results

Our study extracted 512 histopathological features from the HE-stained slides of each glioma patient. We identified 36 and 18 features as significantly related to disease-free survival (DFS) and overall survival (OS), respectively, (P &lt; 0.05) using the univariate Cox proportional-hazards model. Pathomics signature showed a C-index of 0.630 and 0.652 for DFS and OS prediction, respectively. The time-dependent receiver operating characteristic (ROC) curves, along with nomograms, were used to assess the diagnostic accuracy at a fixed time point. In the validation set (n = 49), the area under the curve (AUC) in the 1- and 2-year DFS was 0.955 and 0.904, respectively, and the 2-, 3-, and 5-year OS were 0.969, 0.955, and 0.960, respectively. We stratified the patients into low- and high-risk groups using the median hazard score (0.083 for DFS and−0.177 for OS) and showed significant differences between these groups (P &lt; 0.001).

Conclusion

Our results demonstrated that the DL model based on the HE-stained slides showed the predictability of recurrence and survival in patients with glioma. The results can be used to assist oncologists in selecting the optimal treatment strategy in clinical practice.

Emerging role of artificial intelligence in diagnosis, classification and clinical management of glioma

Glioma represents a dominant primary intracranial malignancy in the central nervous system. Artificial intelligence that mainly includes machine learning, and deep learning computational approaches, presents a unique opportunity to enhance clinical management of glioma through improving tumor segmentation, diagnosis, differentiation, grading, treatment, prediction of clinical outcomes (prognosis, and recurrence), molecular features, clinical classification, characterization of the tumor microenvironment, and drug discovery. A growing body of recent studies apply artificial intelligence-based models to disparate data sources of glioma, covering imaging modalities, digital pathology, high-throughput multi-omics data (especially emerging single-cell RNA sequencing and spatial transcriptome), etc. While these early findings are promising, future studies are required to normalize artificial intelligence-based models to improve the generalizability and interpretability of the results. Despite prominent issues, targeted clinical application of artificial intelligence approaches in glioma will facilitate the development of precision medicine of this field. If these challenges can be overcome, artificial intelligence has the potential to profoundly change the way patients with or at risk of glioma are provided with more rational care.

Intraoperative cytological diagnosis of brain tumours: A preliminary study using a deep learning model

BACKGROUND

Intraoperative pathological diagnosis of central nervous system (CNS) tumours is essential to planning patient management in neuro-oncology. Frozen section slides and cytological preparations provide architectural and cellular information that is analysed by pathologists to reach an intraoperative diagnosis. Progress in the fields of artificial intelligence and machine learning means that AI systems have significant potential for the provision of highly accurate real-time diagnosis in cytopathology.

OBJECTIVE

To investigate the efficiency of machine-learning models in the intraoperative cytological diagnosis of CNS tumours.

MATERIALS AND METHODS

We trained a deep neural network to classify biopsy material for intraoperative tissue diagnosis of four major brain lesions. Overall, 205 medical images were obtained from squash smear slides of histologically correlated cases, with 18 high-grade and 11 low-grade gliomas, 17 metastatic carcinomas, and 9 non-neoplastic pathological brain tissue samples. The neural network model was trained and evaluated using 5-fold cross-validation.

RESULTS

The model achieved 95% and 97% diagnostic accuracy in the patch-level classification and patient-level classification tasks, respectively.

CONCLUSIONS

We conclude that deep learning-based classification of cytological preparations may be a promising complementary method for the rapid and accurate intraoperative diagnosis of CNS tumours.

Relationship between the deep features of the full-scan pathological map of mucinous gastric carcinoma and related genes based on deep learning

Background

Long-term differential expression of disease-associated genes is a crucial driver of pathological changes in mucinous gastric carcinoma. Therefore, there should be a correlation between depth features extracted from pathology-based full-scan images using deep learning and disease-associated gene expression. This study tried to provides preliminary evidence that long-term differentially expressed (disease-associated) genes lead to subtle changes in disease pathology by exploring their correlation, and offer a new ideas for precise analysis of pathomics and combined analysis of pathomics and genomics.

Methods

Full pathological scans, gene sequencing data, and clinical data of patients with mucinous gastric carcinoma were downloaded from TCGA data. The VGG-16 network architecture was used to construct a binary classification model to explore the potential of VGG-16 applications and extract the deep features of the pathology-based full-scan map. Differential gene expression analysis was performed and a protein-protein interaction network was constructed to screen disease-related core genes. Differential, Lasso regression, and extensive correlation analyses were used to screen for valuable deep features. Finally, a correlation analysis was used to determine whether there was a correlation between valuable deep features and disease-related core genes.

Result

The accuracy of the binary classification model was 0.775 ± 0.129. A total of 24 disease-related core genes were screened, including ASPM, AURKA, AURKB, BUB1, BUB1B, CCNA2, CCNB1, CCNB2, CDCA8, CDK1, CENPF, DLGAP5, KIF11, KIF20A, KIF2C, KIF4A, MELK, PBK, RRM2, TOP2A, TPX2, TTK, UBE2C, and ZWINT. In addition, differential, Lasso regression, and extensive correlation analyses were used to screen eight valuable deep features, including features 51, 106, 109, 118, 257, 282, 326, and 487. Finally, the results of the correlation analysis suggested that valuable deep features were either positively or negatively correlated with core gene expression.

Conclusion

The preliminary results of this study support our hypotheses. Deep learning may be an important bridge for the joint analysis of pathomics and genomics and provides preliminary evidence for long-term abnormal expression of genes leading to subtle changes in pathology.

Annotation-free glioma grading from pathological images using ensemble deep learning

Glioma grading is critical for treatment selection, and the fine classification between glioma grades II and III is still a pathological challenge. Traditional systems based on a single deep learning (DL) model can only show relatively low accuracy in distinguishing glioma grades II and III. Introducing ensemble DL models by combining DL and ensemble learning techniques, we achieved annotation-free glioma grading (grade II or III) from pathological images. We established multiple tile-level DL models using residual network ResNet-18 architecture and then used DL models as component classifiers to develop ensemble DL models to achieve patient-level glioma grading. Whole-slide images of 507 subjects with low-grade glioma (LGG) from the Cancer Genome Atlas (TCGA) were included. The 30 DL models exhibited an average area under the curve (AUC) of 0.7991 in patient-level glioma grading. Single DL models showed large variation, and the median between-model cosine similarity was 0.9524, significantly smaller than the threshold of 1.0. The ensemble model based on logistic regression (LR) methods with a 14-component DL classifier (LR-14) demonstrated a mean patient-level accuracy and AUC of 0.8011 and 0.8945, respectively. Our proposed LR-14 ensemble DL model achieved state-of-the-art performance in glioma grade II and III classifications based on unannotated pathological images.

Code-free machine learning for classification of central nervous system histopathology images

Machine learning (ML), an application of artificial intelligence, is currently transforming the analysis of biomedical data and specifically of biomedical images including histopathology. The promises of this technology contrast, however, with its currently limited application in routine clinical practice. This discrepancy is in part due to the extent of informatics expertise typically required for implementation of ML. Therefore, we assessed the suitability of 2 publicly accessible code-free ML platforms (Microsoft Custom Vision and Google AutoML), for classification of histopathological images of diagnostic central nervous system tissue samples. When trained with typically 100 to more than 1000 images, both systems were able to perform nontrivial classifications (glioma vs brain metastasis; astrocytoma vs astrocytosis, prediction of 1p/19q co-deletion in IDH-mutant tumors) based on hematoxylin and eosin-stained images with high accuracy (from ∼80% to nearly 100%). External validation of the predicted accuracy and negative control experiments were found to be crucial for verification of the accuracy predicted by the algorithms. Furthermore, we propose a possible diagnostic workflow for pathologists to implement classification of histopathological images based on code-free machine platforms.