Development and validation of an abnormality-derived deep-learning diagnostic system for major respiratory diseases

China Protocol for early screening, precise diagnosis, and individualized treatment of lung cancer

Early screening, diagnosis, and treatment of lung cancer are pivotal in clinical practice since the tumor stage remains the most dominant factor that affects patient survival. Previous initiatives have tried to develop new tools for decision-making of lung cancer. In this study, we proposed the China Protocol, a complete workflow of lung cancer tailored to the Chinese population, which is implemented by steps including early screening by evaluation of risk factors and three-dimensional thin-layer image reconstruction technique for low-dose computed tomography (Tre-LDCT), accurate diagnosis via artificial intelligence (AI) and novel biomarkers, and individualized treatment through non-invasive molecule visualization strategies. The application of this protocol has improved the early diagnosis and 5-year survival rates of lung cancer in China. The proportion of early-stage (stage I) lung cancer has increased from 46.3% to 65.6%, along with a 5-year survival rate of 90.4%. Moreover, especially for stage IA1 lung cancer, the diagnosis rate has improved from 16% to 27.9%; meanwhile, the 5-year survival rate of this group achieved 97.5%. Thus, here we defined stage IA1 lung cancer, which cohort benefits significantly from early diagnosis and treatment, as the "ultra-early stage lung cancer", aiming to provide an intuitive description for more precise management and survival improvement. In the future, we will promote our findings to multicenter remote areas through medical alliances and mobile health services with the desire to move forward the diagnosis and treatment of lung cancer.

The potential of large language models to advance precision oncology

With the rapid development of artificial intelligence (AI) within medicine, the emergence of large language models (LLMs) has gradually reached the forefront of clinical research. In oncology, by mining the underlying connection between a text or image input and the desired output, LLMs demonstrate great potential for managing tumours. In this review, we provide a brief description of the development of LLMs, followed by model construction strategies and general medical functions. We then elaborate on the role of LLMs in cancer screening and diagnosis, metastasis identification, tumour staging, treatment recommendation, and documentation processing tasks by decoding various types of clinical data. Moreover, the current barriers faced by LLMs, such as hallucinations, ethical problems, limited application, and so on, are outlined along with corresponding solutions, where the further purpose is to inspire improvement and innovation in this field with respect to harnessing LLMs for advancing precision oncology.

Data-driven risk stratification and precision management of pulmonary nodules detected on chest computed tomography

The widespread implementation of low-dose computed tomography (LDCT) in lung cancer screening has led to the increasing detection of pulmonary nodules. However, precisely evaluating the malignancy risk of pulmonary nodules remains a formidable challenge. Here we propose a triage-driven Chinese Lung Nodules Reporting and Data System (C-Lung-RADS) utilizing a medical checkup cohort of 45,064 cases. The system was operated in a stepwise fashion, initially distinguishing low-, mid-, high- and extremely high-risk nodules based on their size and density. Subsequently, it progressively integrated imaging information, demographic characteristics and follow-up data to pinpoint suspicious malignant nodules and refine the risk scale. The multidimensional system achieved a state-of-the-art performance with an area under the curve (AUC) of 0.918 (95% confidence interval (CI) 0.918-0.919) on the internal testing dataset, outperforming the single-dimensional approach (AUC of 0.881, 95% CI 0.880-0.882). Moreover, C-Lung-RADS exhibited a superior sensitivity compared with Lung-RADS v2022 (87.1% versus 63.3%) in an independent cohort, which was screened using mobile computed tomography scanners to broaden screening accessibility in resource-constrained settings. With its foundation in precise risk stratification and tailored management, this system has minimized unnecessary invasive procedures for low-risk cases and recommended prompt intervention for extremely high-risk nodules to avert diagnostic delays. This approach has the potential to enhance the decision-making paradigm and facilitate a more efficient diagnosis of lung cancer during routine checkups as well as screening scenarios.

Model based on the automated AI-driven CT quantification is effective for the diagnosis of refractory Mycoplasma pneumoniae pneumonia

The prediction of refractory Mycoplasma pneumoniae pneumonia (RMPP) remains a clinically significant challenge. This study aimed to develop an early predictive model utilizing artificial intelligence (AI)-derived quantitative assessment of lung lesion extent on initial computed tomography (CT) scans and clinical indicators for RMPP in pediatric inpatients. A retrospective cohort study was conducted on patients with M. pneumoniae pneumonia (MP) admitted to the Children's Hospital of Nanjing Medical University, China from January 2019 to December 2020. An early prediction model was developed by stratifying the patients with Mycoplasma pneumoniae pneumonia (MPP) into two cohorts according to the presence or absence of refractory pneumonia. A retrospective cohort of 126 children diagnosed with Mycoplasma pneumoniae pneumonia (MPP) was utilized as a training set, with 85 cases classified as RMPP. Subsequently, a prospective cohort comprising 54 MPP cases, including 37 instances of RMPP, was assembled as a validation set to assess the performance of the predictive model for RMPP from January to December 2021. We defined a constant Φ which can combine the volume and CT value of pulmonary lesions and be further used to calculate the logarithm of Φ to the base of 2 (Log2Φ). A clinical-imaging prediction model was then constructed utilizing Log2Φ and clinical characteristics. Performance was evaluated by the area under the receiver operating characteristic curve (AUC). The clinical model demonstrated AUC values of 0.810 and 0.782, while the imaging model showed AUC values of 0.764 and 0.769 in the training and test sets, respectively. The clinical-imaging model, incorporating Log2Φ, temperature(T), aspartate aminotransferase (AST), preadmission fever duration (PFD), and preadmission macrolides therapy duration (PMTD), achieved the highest AUC values of 0.897 and 0.895 in the training and test sets, respectively. A prognostic model developed through automated quantification of lung disease on CT scans, in conjunction with clinical data in MPP may be utilized for the early identification of RMPP.

A multimodal integration pipeline for accurate diagnosis, pathogen identification, and prognosis prediction of pulmonary infections

Pulmonary infections pose formidable challenges in clinical settings with high mortality rates across all age groups worldwide. Accurate diagnosis and early intervention are crucial to improve patient outcomes. Artificial intelligence (AI) has the capability to mine imaging features specific to different pathogens and fuse multimodal features to reach a synergistic diagnosis, enabling more precise investigation and individualized clinical management. In this study, we successfully developed a multimodal integration (MMI) pipeline to differentiate among bacterial, fungal, and viral pneumonia and pulmonary tuberculosis based on a real-world dataset of 24,107 patients. The area under the curve (AUC) of the MMI system comprising clinical text and computed tomography (CT) image scans yielded 0.910 (95% confidence interval [CI]: 0.904-0.916) and 0.887 (95% CI: 0.867-0.909) in the internal and external testing datasets respectively, which were comparable to those of experienced physicians. Furthermore, the MMI system was utilized to rapidly differentiate between viral subtypes with a mean AUC of 0.822 (95% CI: 0.805-0.837) and bacterial subtypes with a mean AUC of 0.803 (95% CI: 0.775-0.830). Here, the MMI system harbors the potential to guide tailored medication recommendations, thus mitigating the risk of antibiotic misuse. Additionally, the integration of multimodal factors in the AI-driven system also provided an evident advantage in predicting risks of developing critical illness, contributing to more informed clinical decision-making. To revolutionize medical care, embracing multimodal AI tools in pulmonary infections will pave the way to further facilitate early intervention and precise management in the foreseeable future.

A deep learning-based algorithm for pulmonary tuberculosis detection in chest radiography

In tuberculosis (TB), chest radiography (CXR) patterns are highly variable, mimicking pneumonia and many other diseases. This study aims to evaluate the efficacy of Google teachable machine, a deep neural network-based image classification tool, to develop algorithm for predicting TB probability of CXRs. The training dataset included 348 TB CXRs and 3806 normal CXRs for training TB detection. We also collected 1150 abnormal CXRs and 627 normal CXRs for training abnormality detection. For external validation, we collected 250 CXRs from our hospital. We also compared the accuracy of the algorithm to five pulmonologists and radiological reports. In external validation, the AI algorithm showed areas under the curve (AUC) of 0.951 and 0.975 in validation dataset 1 and 2. The accuracy of the pulmonologists on validation dataset 2 showed AUC range of 0.936-0.995. When abnormal CXRs other than TB were added, AUC decreased in both human readers (0.843-0.888) and AI algorithm (0.828). When combine human readers with AI algorithm, the AUC further increased to 0.862-0.885. The TB CXR AI algorithm developed by using Google teachable machine in this study is effective, with the accuracy close to experienced clinical physicians, and may be helpful for detecting tuberculosis by CXR.

Deep learning for precise diagnosis and subtype triage of drug-resistant tuberculosis on chest computed tomography

Deep learning, transforming input data into target prediction through intricate network structures, has inspired novel exploration in automated diagnosis based on medical images. The distinct morphological characteristics of chest abnormalities between drug-resistant tuberculosis (DR-TB) and drug-sensitive tuberculosis (DS-TB) on chest computed tomography (CT) are of potential value in differential diagnosis, which is challenging in the clinic. Hence, based on 1176 chest CT volumes from the equal number of patients with tuberculosis (TB), we presented a Deep learning-based system for TB drug resistance identification and subtype classification (DeepTB), which could automatically diagnose DR-TB and classify crucial subtypes, including rifampicin-resistant tuberculosis, multidrug-resistant tuberculosis, and extensively drug-resistant tuberculosis. Moreover, chest lesions were manually annotated to endow the model with robust power to assist radiologists in image interpretation and the Circos revealed the relationship between chest abnormalities and specific types of DR-TB. Finally, DeepTB achieved an area under the curve (AUC) up to 0.930 for thoracic abnormality detection and 0.943 for DR-TB diagnosis. Notably, the system demonstrated instructive value in DR-TB subtype classification with AUCs ranging from 0.880 to 0.928. Meanwhile, class activation maps were generated to express a human-understandable visual concept. Together, showing a prominent performance, DeepTB would be impactful in clinical decision-making for DR-TB.

The long-term health outcomes, pathophysiological mechanisms and multidisciplinary management of long COVID

There have been hundreds of millions of cases of coronavirus disease 2019 (COVID-19), which is caused by severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2). With the growing population of recovered patients, it is crucial to understand the long-term consequences of the disease and management strategies. Although COVID-19 was initially considered an acute respiratory illness, recent evidence suggests that manifestations including but not limited to those of the cardiovascular, respiratory, neuropsychiatric, gastrointestinal, reproductive, and musculoskeletal systems may persist long after the acute phase. These persistent manifestations, also referred to as long COVID, could impact all patients with COVID-19 across the full spectrum of illness severity. Herein, we comprehensively review the current literature on long COVID, highlighting its epidemiological understanding, the impact of vaccinations, organ-specific sequelae, pathophysiological mechanisms, and multidisciplinary management strategies. In addition, the impact of psychological and psychosomatic factors is also underscored. Despite these crucial findings on long COVID, the current diagnostic and therapeutic strategies based on previous experience and pilot studies remain inadequate, and well-designed clinical trials should be prioritized to validate existing hypotheses. Thus, we propose the primary challenges concerning biological knowledge gaps and efficient remedies as well as discuss the corresponding recommendations.

Overcoming the Challenges in the Development and Implementation of Artificial Intelligence in Radiology: A Comprehensive Review of Solutions Beyond Supervised Learning

Artificial intelligence (AI) in radiology is a rapidly developing field with several prospective clinical studies demonstrating its benefits in clinical practice. In 2022, the Korean Society of Radiology held a forum to discuss the challenges and drawbacks in AI development and implementation. Various barriers hinder the successful application and widespread adoption of AI in radiology, such as limited annotated data, data privacy and security, data heterogeneity, imbalanced data, model interpretability, overfitting, and integration with clinical workflows. In this review, some of the various possible solutions to these challenges are presented and discussed; these include training with longitudinal and multimodal datasets, dense training with multitask learning and multimodal learning, self-supervised contrastive learning, various image modifications and syntheses using generative models, explainable AI, causal learning, federated learning with large data models, and digital twins.

Recent advancements in artificial intelligence for breast cancer: Image augmentation, segmentation, diagnosis, and prognosis approaches

Breast cancer is a significant global health burden, with increasing morbidity and mortality worldwide. Early screening and accurate diagnosis are crucial for improving prognosis. Radiographic imaging modalities such as digital mammography (DM), digital breast tomosynthesis (DBT), magnetic resonance imaging (MRI), ultrasound (US), and nuclear medicine techniques, are commonly used for breast cancer assessment. And histopathology (HP) serves as the gold standard for confirming malignancy. Artificial intelligence (AI) technologies show great potential for quantitative representation of medical images to effectively assist in segmentation, diagnosis, and prognosis of breast cancer. In this review, we overview the recent advancements of AI technologies for breast cancer, including 1) improving image quality by data augmentation, 2) fast detection and segmentation of breast lesions and diagnosis of malignancy, 3) biological characterization of the cancer such as staging and subtyping by AI-based classification technologies, 4) prediction of clinical outcomes such as metastasis, treatment response, and survival by integrating multi-omics data. Then, we then summarize large-scale databases available to help train robust, generalizable, and reproducible deep learning models. Furthermore, we conclude the challenges faced by AI in real-world applications, including data curating, model interpretability, and practice regulations. Besides, we expect that clinical implementation of AI will provide important guidance for the patient-tailored management.

Novel tools for early diagnosis and precision treatment based on artificial intelligence

Lung cancer has the highest mortality rate among all cancers in the world. Hence, early diagnosis and personalized treatment plans are crucial to improving its 5-year survival rate. Chest computed tomography (CT) serves as an essential tool for lung cancer screening, and pathology images are the gold standard for lung cancer diagnosis. However, medical image evaluation relies on manual labor and suffers from missed diagnosis or misdiagnosis, and physician heterogeneity. The rapid development of artificial intelligence (AI) has brought a whole novel opportunity for medical task processing, demonstrating the potential for clinical application in lung cancer diagnosis and treatment. AI technologies, including machine learning and deep learning, have been deployed extensively for lung nodule detection, benign and malignant classification, and subtype identification based on CT images. Furthermore, AI plays a role in the non-invasive prediction of genetic mutations and molecular status to provide the optimal treatment regimen, and applies to the assessment of therapeutic efficacy and prognosis of lung cancer patients, enabling precision medicine to become a reality. Meanwhile, histology-based AI models assist pathologists in typing, molecular characterization, and prognosis prediction to enhance the efficiency of diagnosis and treatment. However, the leap to extensive clinical application still faces various challenges, such as data sharing, standardized label acquisition, clinical application regulation, and multimodal integration. Nevertheless, AI holds promising potential in the field of lung cancer to improve cancer care.

A transformer-based representation-learning model with unified processing of multimodal input for clinical diagnostics

During the diagnostic process, clinicians leverage multimodal information, such as the chief complaint, medical images and laboratory test results. Deep-learning models for aiding diagnosis have yet to meet this requirement of leveraging multimodal information. Here we report a transformer-based representation-learning model as a clinical diagnostic aid that processes multimodal input in a unified manner. Rather than learning modality-specific features, the model leverages embedding layers to convert images and unstructured and structured text into visual tokens and text tokens, and uses bidirectional blocks with intramodal and intermodal attention to learn holistic representations of radiographs, the unstructured chief complaint and clinical history, and structured clinical information such as laboratory test results and patient demographic information. The unified model outperformed an image-only model and non-unified multimodal diagnosis models in the identification of pulmonary disease (by 12% and 9%, respectively) and in the prediction of adverse clinical outcomes in patients with COVID-19 (by 29% and 7%, respectively). Unified multimodal transformer-based models may help streamline the triaging of patients and facilitate the clinical decision-making process.

Dynamic learning for imbalanced data in learning chest X-ray and CT images

Massive annotated datasets are necessary for networks of deep learning. When a topic is being researched for the first time, as in the situation of the viral epidemic, handling it with limited annotated datasets might be difficult. Additionally, the datasets are quite unbalanced in this situation, with limited findings coming from significant instances of the novel illness. We offer a technique that allows a class balancing algorithm to understand and detect lung disease signs from chest X-ray and CT images. Deep learning techniques are used to train and evaluate images, enabling the extraction of basic visual attributes. The training objects' characteristics, instances, categories, and relative data modeling are all represented probabilistically. It is possible to identify a minority category in the classification process by using an imbalance-based sample analyzer. In order to address the imbalance problem, learning samples from the minority class are examined. The Support Vector Machine (SVM) is used to categorize images in clustering. Physicians and medical professionals can use the CNN model to validate their initial assessments of malignant and benign categorization. The proposed technique for class imbalance (3-Phase Dynamic Learning (3PDL)) and parallel CNN model (Hybrid Feature Fusion (HFF)) for multiple modalities achieve a high F1 score of 96.83 and precision is 96.87, its outstanding accuracy and generalization suggest that it may be utilized to create a pathologist's help tool.

Predicting gene mutation status via artificial intelligence technologies based on multimodal integration (MMI) to advance precision oncology

Personalized treatment strategies for cancer frequently rely on the detection of genetic alterations which are determined by molecular biology assays. Historically, these processes typically required single-gene sequencing, next-generation sequencing, or visual inspection of histopathology slides by experienced pathologists in a clinical context. In the past decade, advances in artificial intelligence (AI) technologies have demonstrated remarkable potential in assisting physicians with accurate diagnosis of oncology image-recognition tasks. Meanwhile, AI techniques make it possible to integrate multimodal data such as radiology, histology, and genomics, providing critical guidance for the stratification of patients in the context of precision therapy. Given that the mutation detection is unaffordable and time-consuming for a considerable number of patients, predicting gene mutations based on routine clinical radiological scans or whole-slide images of tissue with AI-based methods has become a hot issue in actual clinical practice. In this review, we synthesized the general framework of multimodal integration (MMI) for molecular intelligent diagnostics beyond standard techniques. Then we summarized the emerging applications of AI in the prediction of mutational and molecular profiles of common cancers (lung, brain, breast, and other tumor types) pertaining to radiology and histology imaging. Furthermore, we concluded that there truly exist multiple challenges of AI techniques in the way of its real-world application in the medical field, including data curation, feature fusion, model interpretability, and practice regulations. Despite these challenges, we still prospect the clinical implementation of AI as a highly potential decision-support tool to aid oncologists in future cancer treatment management.

Radiogenomic System for Non-Invasive Identification of Multiple Actionable Mutations and PD-L1 Expression in Non-Small Cell Lung Cancer Based on CT Images

PURPOSE

Personalized treatments such as targeted therapy and immunotherapy have revolutionized the predominantly therapeutic paradigm for non-small cell lung cancer (NSCLC). However, these treatment decisions require the determination of targetable genomic and molecular alterations through invasive genetic or immunohistochemistry (IHC) tests. Numerous previous studies have demonstrated that artificial intelligence can accurately predict the single-gene status of tumors based on radiologic imaging, but few studies have achieved the simultaneous evaluation of multiple genes to reflect more realistic clinical scenarios.

METHODS

We proposed a multi-label multi-task deep learning (MMDL) system for non-invasively predicting actionable NSCLC mutations and PD-L1 expression utilizing routinely acquired computed tomography (CT) images. This radiogenomic system integrated transformer-based deep learning features and radiomic features of CT volumes from 1096 NSCLC patients based on next-generation sequencing (NGS) and IHC tests.

RESULTS

For each task cohort, we randomly split the corresponding dataset into training (80%), validation (10%), and testing (10%) subsets. The area under the receiver operating characteristic curves (AUCs) of the MMDL system achieved 0.862 (95% confidence interval (CI), 0.758-0.969) for discrimination of a panel of 8 mutated genes, including EGFR, ALK, ERBB2, BRAF, MET, ROS1, RET and KRAS, 0.856 (95% CI, 0.663-0.948) for identification of a 10-molecular status panel (previous 8 genes plus TP53 and PD-L1); and 0.868 (95% CI, 0.641-0.972) for classifying EGFR / PD-L1 subtype, respectively.

CONCLUSIONS

To the best of our knowledge, this study is the first deep learning system to simultaneously analyze 10 molecular expressions, which might be utilized as an assistive tool in conjunction with or in lieu of ancillary testing to support precision treatment options.