Deep learning with ultrasonography: automated classification of liver fibrosis using a deep convolutional neural network

Deep learning without borders: recent advances in ultrasound image classification for liver diseases diagnosis

INTRODUCTION

Liver diseases are among the top global health burdens. Recently, there has been an increasing significance of diagnostics without discomfort to the patient; among them, ultrasound is the most used. Deep learning, in particular convolutional neural networks, has revolutionized the classification of liver diseases by automatically performing some specific analyses of difficult images.

AREAS COVERED

This review summarizes the progress that has been made in deep learning techniques for the classification of liver diseases using ultrasound imaging. It evaluates various models from CNNs to their hybrid versions, such as CNN-Transformer, for detecting fatty liver, fibrosis, and liver cancer, among others. Several challenges in the generalization of data and models across a different clinical environment are also discussed.

EXPERT OPINION

Deep learning has great prospects for automatic diagnosis of liver diseases. Most of the models have performed with high accuracy in different clinical studies. Despite this promise, challenges relating to generalization have remained. Future hardware developments and access to quality clinical data continue to further improve the performance of these models and ensure their vital role in the diagnosis of liver diseases.

Artificial intelligence in imaging for liver disease diagnosis

Liver diseases, including hepatitis, non-alcoholic fatty liver disease (NAFLD), cirrhosis, and hepatocellular carcinoma (HCC), remain a major global health concern, with early and accurate diagnosis being essential for effective management. Imaging modalities such as ultrasound (US), computed tomography (CT), and magnetic resonance imaging (MRI) play a crucial role in non-invasive diagnosis, but their sensitivity and diagnostic accuracy can be limited. Recent advancements in artificial intelligence (AI) have improved imaging-based liver disease assessment by enhancing pattern recognition, automating fibrosis and steatosis quantification, and aiding in HCC detection. AI-driven imaging techniques have shown promise in fibrosis staging through US, CT, MRI, and elastography, reducing the reliance on invasive liver biopsy. For liver steatosis, AI-assisted imaging methods have improved sensitivity and grading consistency, while in HCC detection and characterization, AI models have enhanced lesion identification, classification, and risk stratification across imaging modalities. The growing integration of AI into liver imaging is reshaping diagnostic workflows and has the potential to improve accuracy, efficiency, and clinical decision-making. This review provides an overview of AI applications in liver imaging, focusing on their clinical utility and implications for the future of liver disease diagnosis.

Ultrasound detection of nonalcoholic steatohepatitis using convolutional neural networks with dual-branch global-local feature fusion architecture

Nonalcoholic steatohepatitis (NASH) is a contributing factor to liver cancer, with ultrasound B-mode imaging as the first-line diagnostic tool. This study applied deep learning to ultrasound B-scan images for NASH detection and introduced an ultrasound-specific data augmentation (USDA) technique with a dual-branch global-local feature fusion architecture (DG-LFFA) to improve model performance and adaptability across imaging conditions. A total of 137 participants were included. Ultrasound images underwent data augmentation (rotation and USDA) for training and testing convolutional neural networks-AlexNet, Inception V3, VGG16, VGG19, ResNet50, and DenseNet201. Gradient-weighted class activation mapping (Grad-CAM) analyzed model attention patterns, guiding the selection of the optimal backbone for DG-LFFA implementation. The models achieved testing accuracies of 0.81-0.83 with rotation-based data augmentation. Grad-CAM analysis showed that ResNet50 and DenseNet201 exhibited stronger liver attention. When USDA simulated datasets from different imaging conditions, DG-LFFA (based on ResNet50 and DenseNet201) improved accuracy (0.79 to 0.84 and 0.78 to 0.83), recall (0.72 to 0.81 and 0.70 to 0.78), and F1 score (0.80 to 0.84 for both models). In conclusion, deep architectures (ResNet50 and DenseNet201) enable focused analysis of liver regions for NASH detection. Under USDA-simulated imaging variations, the proposed DG-LFFA framework further improves diagnostic performance.

CT-based artificial intelligence system complementing deep learning model and radiologist for liver fibrosis staging

Noninvasive methods for liver fibrosis staging are urgently needed due to its significance in predicting significant morbidity and mortality. In this study, we developed an automated DL-based segmentation and classification model (Model-C). Test-time adaptation was used to address data distribution shifts. We then established a deep learning-radiologist complementarity decision system (DRCDS) via a decision model determining whether to adopt Model-C's diagnosis or defer to radiologists. Model-C (AUCs of 0.89-0.92) outperformed models based on liver (AUCs: 0.84-0.90) or spleen (AUCs: 0.69-0.70). With test-time adaptation, the Obuchowski index values of Model-C in three external sets improved from 0.81, 0.73, and 0.73 to 0.85, 0.85, and 0.81. DRCDS performed slightly better than Model-C or senior radiologists, with 73.7%-92.0% of cases adopting Model-C's diagnosis. In conclusion, DRCDS could diagnose liver fibrosis with high accuracy. Additionally, we provided solutions to model generalization and human-machine complementarity issues in multi-classification problems.

Study on Ultrasound-Assisted Diagnosis of CHB Complicated with NAFLD Hepatic Fibrosis Based on Deep Learning

This study aimed to develop and evaluate an automated classification model for diagnosing chronic hepatitis B (CHB) complicated with non-alcoholic fatty liver disease (NAFLD) using deep learning techniques applied to two-dimensional liver imaging. We retrospectively analyzed data from 2803 patients diagnosed with CHB and NAFLD via two-dimensional ultrasound and FibroScan at our hospital between June 2019 and December 2022. These patients contributed a total of 20,540 two-dimensional liver images. An additional 150 patients with CHB complicated with NAFLD, who had liver biopsy results, were selected (with a total of 922 liver 2D ultrasound images) for validation of the deep learning model. The diagnostic performance was assessed using sensitivity, specificity, and accuracy metrics for independent diagnoses made by clinicians or with AI assistance. Diagnostic performance was assessed under blinded conditions by physicians of varying expertise, primary, intermediate, and senior clinicians, either independently or assisted by the AI model. We then conducted a statistical analysis of the results, focusing on sensitivity, specificity, and accuracy metrics. The sensitivity, specificity, and accuracy for independent diagnoses made by primary, intermediate, and senior physicians were 24.3%, 29.1%, and 33.8% for sensitivity; 100% for specificity; and 25.3%, 30.0%, and 34.7% for accuracy, respectively. The AI model alone achieved 68.9% sensitivity, 100% specificity, and 69.3% accuracy. When diagnoses were AI-assisted, the sensitivity increased significantly to 73.7% for primary physicians, 73.7% for intermediate physicians, and 75.7% for senior physicians, while specificity remained at 100%, and accuracy was 74.0%, 74.0%, and 76.0%, respectively. The area under the receiver operating characteristic curve (AUC) for AI-independent diagnosis (0.845) was significantly higher than that for primary, intermediate, and senior physicians (0.622, 0.645, and 0.669, respectively;P < 0.001 ). Similarly, AI-assisted diagnosis AUC values (0.868, 0.868, and 0.878, respectively) surpassed those for independent diagnosis by physicians at each level of expertise (P < 0.001 ). In conclusion, the deep learning model based on two-dimensional liver imaging demonstrated feasibility for assisting in the identification of liver fibrosis in patients with CHB and NAFLD. Our results suggest that integrating AI models into the diagnostic process can significantly improve the accuracy of diagnosing liver conditions in this patient population.

Development of an artificial intelligence-based multimodal diagnostic system for early detection of biliary atresia

BACKGROUND

Early diagnosis of biliary atresia (BA) is crucial for improving patient outcomes, yet remains a significant global challenge. This challenge may be ameliorated through the application of artificial intelligence (AI). Despite the promise of AI in medical diagnostics, its application to multimodal BA data has not yet achieved substantial breakthroughs. This study aims to leverage diverse data sources and formats to develop an intelligent diagnostic system for BA.

METHODS

We constructed the largest known multimodal BA dataset, comprising ultrasound images, clinical data, and laboratory results. Using this dataset, we developed a novel deep learning model and simplified it using easily obtainable data, eliminating the need for blood samples. The models were externally validated in a prospective study. We compared the performance of our model with human experts of varying expertise levels and evaluated the AI system's potential to enhance its diagnostic accuracy.

RESULTS

The retrospective study included 1579 participants. The multimodal model achieved an AUC of 0.9870 on the internal test set, outperforming human experts. The simplified model yielded an AUC of 0.9799. In the prospective study's external test set of 171 cases, the multimodal model achieved an AUC of 0.9740, comparable to that of a radiologist with over 10 years of experience (AUC = 0.9766). For less experienced radiologists, the AI-assisted diagnostic AUC improved from 0.6667 to 0.9006.

CONCLUSIONS

This AI-based screening application effectively facilitates early diagnosis of BA and serves as a valuable reference for addressing common challenges in rare diseases. The model's high accuracy and its ability to enhance the diagnostic performance of human experts underscore its potential for significant clinical impact.

Multi-site, multi-vendor development and validation of a deep learning model for liver stiffness prediction using abdominal biparametric MRI

BACKGROUND

Chronic liver disease (CLD) is a substantial cause of morbidity and mortality worldwide. Liver stiffness, as measured by MR elastography (MRE), is well-accepted as a surrogate marker of liver fibrosis.

PURPOSE

To develop and validate deep learning (DL) models for predicting MRE-derived liver stiffness using routine clinical non-contrast abdominal T1-weighted (T1w) and T2-weighted (T2w) data from multiple institutions/system manufacturers in pediatric and adult patients.

MATERIALS AND METHODS

We identified pediatric and adult patients with known or suspected CLD from four institutions, who underwent clinical MRI with MRE from 2011 to 2022. We used T1w and T2w data to train DL models for liver stiffness classification. Patients were categorized into two groups for binary classification using liver stiffness thresholds (≥ 2.5 kPa, ≥ 3.0 kPa, ≥ 3.5 kPa, ≥ 4 kPa, or ≥ 5 kPa), reflecting various degrees of liver stiffening.

RESULTS

We identified 4695 MRI examinations from 4295 patients (mean ± SD age, 47.6 ± 18.7 years; 428 (10.0%) pediatric; 2159 males [50.2%]). With a primary liver stiffness threshold of 3.0 kPa, our model correctly classified patients into no/minimal (< 3.0 kPa) vs moderate/severe (≥ 3.0 kPa) liver stiffness with AUROCs of 0.83 (95% CI: 0.82, 0.84) in our internal multi-site cross-validation (CV) experiment, 0.82 (95% CI: 0.80, 0.84) in our temporal hold-out validation experiment, and 0.79 (95% CI: 0.75, 0.81) in our external leave-one-site-out CV experiment. The developed model is publicly available ( https://github.com/almahdir1/Multi-channel-DeepLiverNet2.0.git ).

CONCLUSION

Our DL models exhibited reasonable diagnostic performance for categorical classification of liver stiffness on a large diverse dataset using T1w and T2w MRI data.

KEY POINTS

Question Can DL models accurately predict liver stiffness using routine clinical biparametric MRI in pediatric and adult patients with CLD? Findings DeepLiverNet2.0 used biparametric MRI data to classify liver stiffness, achieving AUROCs of 0.83, 0.82, and 0.79 for multi-site CV, hold-out validation, and external CV. Clinical relevance Our DeepLiverNet2.0 AI model can categorically classify the severity of liver stiffening using anatomic biparametric MR images in children and young adults. Model refinements and incorporation of clinical features may decrease the need for MRE.

The role of noninvasive biomarkers for monitoring cell injury in advanced liver fibrosis

INTRODUCTION

Accurate and reliable diagnosis and monitoring of hepatic fibrosis is increasingly important given the variable natural history in chronic liver disease (CLD) and expanding antifibrotic therapeutic options targeting reversibility of early-stage cirrhosis. This highlights the need to develop more refined and effective noninvasive techniques for the dynamic assessment of fibrogenesis and fibrolysis.

AREAS COVERED

We conducted a literature review on PubMed, from 1 December 1970, to 1 November 2024, to evaluate and compare available blood-based and imaging-based noninvasive tools for hepatic fibrosis diagnosis and monitoring. Simple scores such as FIB-4 and NAFLD fibrosis score are suitable for excluding significant or advanced fibrosis, while tertiary centers should adopt complex scores and liver stiffness measurement as part of a secondary diagnostic and more comprehensive evaluation. Moreover, the advent of multiomics for high-resolution molecular profiling, and integration of artificial intelligence for noninvasive diagnostics holds promise for revolutionizing fibrosis monitoring and treatment through novel biomarker discovery and predictive omics-based algorithms.

EXPERT OPINION

The increased shift toward noninvasive diagnostics for liver fibrosis needs to align with personalized medicine, enabling more effective, tailored management strategies for patients with liver disease in the future.

TransEBUS: The interpretation of endobronchial ultrasound image using hybrid transformer for differentiating malignant and benign mediastinal lesions

The purpose of this study is to establish a deep learning automatic assistance diagnosis system for benign and malignant classification of mediastinal lesions in endobronchial ultrasound (EBUS) images. EBUS images are in the form of video and contain multiple imaging modes. Different imaging modes and different frames can reflect the different characteristics of lesions. Compared with previous studies, the proposed model can efficiently extract and integrate the spatiotemporal relationships between different modes and does not require manual selection of representative frames. In recent years, Vision Transformer has received much attention in the field of computer vision. Combined with convolutional neural networks, hybrid transformers can also perform well on small datasets. This study designed a novel deep learning architecture based on hybrid transformer called TransEBUS. By adding learnable parameters in the temporal dimension, TransEBUS was able to extract spatiotemporal features from insufficient data. In addition, we designed a two-stream module to integrate information from three different imaging modes of EBUS. Furthermore, we applied contrastive learning when training TransEBUS, enabling it to learn discriminative representation of benign and malignant mediastinal lesions. The results show that TransEBUS achieved a diagnostic accuracy of 82% and an area under the curve of 0.8812 in the test dataset, outperforming other methods. It also shows that several models can improve performance by incorporating two-stream module. Our proposed system has shown its potential to help physicians distinguishing benign and malignant mediastinal lesions, thereby ensuring the accuracy of EBUS examination.

Non-invasive imaging biomarkers in chronic liver disease

Chronic liver disease (CLD) is a global and worldwide clinical challenge, considering that different underlying liver entities can lead to hepatic dysfunction. In the past, blood tests and clinical evaluation were the main noninvasive tools used to detect, diagnose and follow-up patients with CLD; in case of clinical suspicion of CLD or unclear diagnosis, liver biopsy has been considered as the reference standard to rule out different chronic liver conditions. Nowadays, noninvasive tests have gained a central role in the clinical pathway. Particularly, liver stiffness measurement (LSM) and cross-sectional imaging techniques can provide transversal information to clinicians, helping them to correctly manage, treat and follow patients during time. Cross-sectional imaging techniques, namely computed tomography (CT) and magnetic resonance imaging (MRI), have plenty of potential. Both techniques allow to compute the liver surface nodularity (LSN), associated with CLDs and risk of decompensation. MRI can also help quantify fatty liver infiltration, mainly with the proton density fat fraction (PDFF) sequences, and detect and quantify fibrosis, especially thanks to elastography (MRE). Advanced techniques, such as intravoxel incoherent motion (IVIM), T1- and T2- mapping are promising tools for detecting fibrosis deposition. Furthermore, the injection of hepatobiliary contrast agents has gained an important role not only in liver lesion characterization but also in assessing liver function, especially in CLDs. Finally, the broad development of radiomics signatures, applied to CT and MR, can be considered the next future approach to CLDs. The aim of this review is to provide a comprehensive overview of the current advancements and applications of both invasive and noninvasive imaging techniques in the evaluation and management of CLD.

GLCV-NET: An automatic diagnosis system for advanced liver fibrosis using global-local cross view in B-mode ultrasound images

BACKGROUND AND OBJECTIVE

Advanced liver fibrosis is a critical stage in the evaluation of chronic liver disease (CLD), holding clinical significance in the development of treatment strategies and estimating the disease progression.

METHODS

This paper proposes an innovative Global-Local Cross-View Network (GLCV-Net) for the automatic diagnosis of advanced liver fibrosis from ultrasound (US) B-mode images. The proposed method consists of three main components: 1. A Segmentation-enhanced Global Hybrid Feature Extractor for segmenting the liver parenchyma and extracting global features; 2. A Heatmap-weighted Local Feature Extractor for selecting candidate regions and automatically identifying suspicious areas to construct local features; 3. A Scale-adaptive Fusion Module to balance the contributions of global and local scales in evaluating advanced liver fibrosis.

RESULTS

The predictive performance of the model was validated on an internal dataset of 1003 chronic liver disease (CLD) patients and an external dataset of 46 CLD patients, both subjected to liver fibrosis staging through pathological assessment. On the internal dataset, GLCV-Net achieved 86.9% accuracy, 85.0% recall, 85.4% precision, and 85.2% F1-score. Further validation on the external dataset confirmed its robustness, with scores of 86.1% in accuracy, 83.1% in recall, 80.8% in precision, and 81.9% in F1-score.

CONCLUSION

These results underscore the GLCV-Net's potential as a promising approach for non-invasively and accurately diagnosing advanced liver fibrosis in CLD patients, breaking through the limitations of traditional methods by integrating global and local information of liver fibrosis, significantly enhancing diagnostic accuracy.

Automated classification of multiple ophthalmic diseases using ultrasound images by deep learning

BACKGROUND

Ultrasound imaging is suitable for detecting and diagnosing ophthalmic abnormalities. However, a shortage of experienced sonographers and ophthalmologists remains a problem. This study aims to develop a multibranch transformer network (MBT-Net) for the automated classification of multiple ophthalmic diseases using B-mode ultrasound images.

METHODS

Ultrasound images with six clinically confirmed categories, including normal, retinal detachment, vitreous haemorrhage, intraocular tumour, posterior scleral staphyloma and other abnormalities, were used to develop and evaluate the MBT-Net. Images were derived from five different ultrasonic devices operated by different sonographers and divided into training set, validation set, internal testing set and temporal external testing set. Two senior ophthalmologists and two junior ophthalmologists were recruited to compare the model's performance.

RESULTS

A total of 10 184 ultrasound images were collected. The MBT-Net got an accuracy of 87.80% (95% CI 86.26% to 89.18%) in the internal testing set, which was significantly higher than junior ophthalmologists (95% CI 67.37% to 79.16%; both p<0.05) and lower than senior ophthalmologists (95% CI 89.45% to 92.61%; both p<0.05). The micro-average area under the curve of the six-category classification was 0.98. With reference to comprehensive clinical diagnosis, the measurements of agreement were almost perfect in the MBT-Net (kappa=0.85, p<0.05). There was no significant difference in the accuracy of the MBT-Net across five ultrasonic devices (p=0.27). The MBT-Net got an accuracy of 82.21% (95% CI 78.45% to 85.44%) in the temporal external testing set.

CONCLUSIONS

The MBT-Net showed high accuracy for screening and diagnosing multiple ophthalmic diseases using only ultrasound images across mutioperators and mutidevices.

Artificial Intelligence in Coronary Artery Calcium Scoring

Cardiovascular disease (CVD), particularly coronary heart disease (CHD), is the leading cause of death in the US, with a high economic impact. Coronary artery calcium (CAC) is a known marker for CHD and a useful tool for estimating the risk of atherosclerotic cardiovascular disease (ASCVD). Although CACS is recommended for informing the decision to initiate statin therapy, the current standard requires a dedicated CT protocol, which is time-intensive and contributes to radiation exposure. Non-dedicated CT protocols can be taken advantage of to visualize calcium and reduce overall cost and radiation exposure; however, they mainly provide visual estimates of coronary calcium and have disadvantages such as motion artifacts. Artificial intelligence is a growing field involving software that independently performs human-level tasks, and is well suited for improving CACS efficiency and repurposing non-dedicated CT for calcium scoring. We present a review of the current studies on automated CACS across various CT protocols and discuss consideration points in clinical application and some barriers to implementation.

Serum biomarkers for liver fibrosis assessment

Liver fibrosis is the result of chronic liver injury of different etiologies produced by an imbalance between the synthesis and degeneration of the extracellular matrix and dysregulation of physiological mechanisms. Liver has a high regenerative capacity in the early stage of chronic diseases so a prompt liver fibrosis detection is important. Consequently, an easy and economic tool that could identify patients with liver fibrosis at the initial stages is needed. To achieve this, many non-invasive serum direct, such as hyaluronic acid or metalloproteases, and indirect biomarkers have been proposed to evaluate liver fibrosis. Also, there have been developed formulas that combine these biomarkers, some of them also introduce clinical and/or demographic parameters, like FIB-4, non-alcoholic fatty liver disease fibrosis score (NFS), enhance liver fibrosis (ELF) or Hepamet fibrosis score (HFS). In this manuscript we critically reviewed different serum biomarkers and formulas for their utility in the diagnosis and progression of liver fibrosis.

Biomarcadores séricos para la evaluación de la fibrosis hepática

La fibrosis hepática se desarrolla como respuesta a la presencia de daño hepático crónico de diferentes etiologías, provocando un desequilibrio entre la síntesis y degeneración de la matriz extracelular y la desregulación de diversos mecanismos fisiológicos. En los estadios iniciales de las patologías crónicas, el hígado posee una elevada capacidad de regeneración, por lo que la detección temprana de la fibrosis hepática resulta esencial. En este contexto, es preciso contar con herramientas sencillas y económicas que permitan detectar la fibrosis hepática en sus fases iniciales. Para evaluar la fibrosis hepática, se han propuesto multitud de biomarcadores séricos no invasivos, tanto directos, como el ácido hialurónico o las metaloproteasas, como indirectos. Así mismo, se han desarrollado diversas fórmulas que combinan dichos biomarcadores junto con parámetros demográficos, como el índice FIB-4, el índice de fibrosis en la enfermedad de hígado graso no alcohólico (NFS, por sus siglas en inglés), la prueba ELF o el score de fibrosis Hepamet (HFS, por sus siglas en inglés). En el presente manuscrito, realizamos una revisión crítica del valor diagnóstico y pronóstico de los diferentes biomarcadores séricos y fórmulas actualmente existentes.

Scope and caveats: Artificial intelligence in gastroenterology

The use of Artificial intelligence (AI) has evolved from its mid-20^th century origins to playing a pivotal tool in modern medicine. It leverages digital data and computational hardware for diverse applications, including diagnosis, prognosis, and treatment responses in gastrointestinal and hepatic conditions. AI has had an impact in diagnostic techniques, particularly endoscopy, ultrasound, and histopathology. AI encompasses machine learning, natural language processing, and robotics, with machine learning being central. This involves sophisticated algorithms capable of managing complex datasets, far surpassing traditional statistical methods. These algorithms, both supervised and unsupervised, are integral for interpreting large datasets. In liver diseases, AI's non-invasive diagnostic applications, particularly in non-alcoholic fatty liver disease, and its role in characterizing hepatic lesions is promising. AI aids in distinguishing between normal and cirrhotic livers and improves the accuracy of lesion characterization and prognostication of hepatocellular carcinoma. AI enhances lesion identification during endoscopy, showing potential in the diagnosis and management of early-stage esophageal carcinoma. In peptic ulcer disease, AI technologies influence patient management strategies. AI is useful in colonoscopy, particularly in detecting smaller colonic polyps. However, its applicability in non-academic settings requires further validation. Addressing these issues is vital for harnessing the potential of AI. In conclusion, while AI offers transformative possibilities in gastroenterology, careful integration and balancing of technical possibilities with ethical and practical application, is essential for optimal use.

SEG-LUS: A novel ultrasound segmentation method for liver and its accessory structures based on muti-head self-attention

Although liver ultrasound (US) is quick and convenient, it presents challenges due to patient variations. Previous research has predominantly focused on computer-aided diagnosis (CAD), particularly for disease analysis. However, characterizing liver US images is complex due to structural diversity and a limited number of samples. Normal liver US images are crucial, especially for standard section diagnosis. This study explicitly addresses Liver US standard sections (LUSS) and involves detailed labeling of eight anatomical structures. We propose SEG-LUS, a US image segmentation model for the liver and its accessory structures. In SEG-LUS, we have adopted the shifted windows feature encoder combined with the cross-attention mechanism to adapt to capturing image information at different scales and resolutions and address context mismatch and sample imbalance in the segmentation task. By introducing the UUF module, we achieve the perfect fusion of shallow and deep information, making the information retained by the network in the feature extraction process more comprehensive. We have improved the Focal Loss to tackle the imbalance of pixel-level distribution. The results show that the SEG-LUS model exhibits significant performance improvement, with mPA, mDice, mIOU, and mASD reaching 85.05%, 82.60%, 74.92%, and 0.31, respectively. Compared with seven state-of-the-art semantic segmentation methods, the mPA improves by 5.32%. SEG-LUS is positioned to serve as a crucial reference for research in computer-aided modeling using liver US images, thereby advancing the field of US medicine research.

US-based Sequential Algorithm Integrating an AI Model for Advanced Liver Fibrosis Screening

Background Noninvasive tests can be used to screen patients with chronic liver disease for advanced liver fibrosis; however, the use of single tests may not be adequate. Purpose To construct sequential clinical algorithms that include a US deep learning (DL) model and compare their ability to predict advanced liver fibrosis with that of other noninvasive tests. Materials and Methods This retrospective study included adult patients with a history of chronic liver disease or unexplained abnormal liver function test results who underwent B-mode US of the liver between January 2014 and September 2022 at three health care facilities. A US-based DL network (FIB-Net) was trained on US images to predict whether the shear-wave elastography (SWE) value was 8.7 kPa or higher, indicative of advanced fibrosis. In the internal and external test sets, a two-step algorithm (Two-step#1) using the Fibrosis-4 Index (FIB-4) followed by FIB-Net and a three-step algorithm (Three-step#1) using FIB-4 followed by FIB-Net and SWE were used to simulate screening scenarios where liver stiffness measurements were not or were available, respectively. Measures of diagnostic accuracy were calculated using liver biopsy as the reference standard and compared between FIB-4, SWE, FIB-Net, and European Association for the Study of the Liver guidelines (ie, FIB-4 followed by SWE), along with sequential algorithms. Results The training, validation, and test data sets included 3067 (median age, 42 years [IQR, 33-53 years]; 2083 male), 1599 (median age, 41 years [IQR, 33-51 years]; 1124 male), and 1228 (median age, 44 years [IQR, 33-55 years]; 741 male) patients, respectively. FIB-Net obtained a noninferior specificity with a margin of 5% (P < .001) compared with SWE (80% vs 82%). The Two-step#1 algorithm showed higher specificity and positive predictive value (PPV) than FIB-4 (specificity, 79% vs 57%; PPV, 44% vs 32%) while reducing unnecessary referrals by 42%. The Three-step#1 algorithm had higher specificity and PPV compared with European Association for the Study of the Liver guidelines (specificity, 94% vs 88%; PPV, 73% vs 64%) while reducing unnecessary referrals by 35%. Conclusion A sequential algorithm combining FIB-4 and a US DL model showed higher diagnostic accuracy and improved referral management for all-cause advanced liver fibrosis compared with FIB-4 or the DL model alone. © RSNA, 2024 Supplemental material is available for this article. See also the editorial by Ghosh in this issue.

Artificial Intelligence in Liver Diseases: Recent Advances

Liver diseases cause a significant burden on public health worldwide. In spite of great advances during recent years, there are still many challenges in the diagnosis and treatment of liver diseases. During recent years, artificial intelligence (AI) has been widely used for the diagnosis, risk stratification, and prognostic prediction of various diseases based on clinical datasets and medical images. Accumulative studies have shown its performance for diagnosing patients with nonalcoholic fatty liver disease and liver fibrosis and assessing their severity, and for predicting treatment response and recurrence of hepatocellular carcinoma, outcomes of liver transplantation recipients, and risk of drug-induced liver injury. Herein, we aim to comprehensively summarize the current evidence regarding diagnostic, prognostic, and/or therapeutic role of AI in these common liver diseases.

Automated classification of liver fibrosis stages using ultrasound imaging

BACKGROUND

Ultrasound imaging is the most frequently performed for the patients with chronic hepatitis or liver cirrhosis. However, ultrasound imaging is highly operator dependent and interpretation of ultrasound images is subjective, thus well-trained radiologist is required for evaluation. Automated classification of liver fibrosis could alleviate the shortage of skilled radiologist especially in low-to-middle income countries. The purposed of this study is to evaluate deep convolutional neural networks (DCNNs) for classifying the degree of liver fibrosis according to the METAVIR score using US images.

METHODS

We used ultrasound (US) images from two tertiary university hospitals. A total of 7920 US images from 933 patients were used for training/validation of DCNNs. All patient were underwent liver biopsy or hepatectomy, and liver fibrosis was categorized based on pathology results using the METAVIR score. Five well-established DCNNs (VGGNet, ResNet, DenseNet, EfficientNet and ViT) was implemented to predict the METAVIR score. The performance of DCNNs for five-level (F0/F1/F2/F3/F4) classification was evaluated through area under the receiver operating characteristic curve (AUC) with 95% confidential interval, accuracy, sensitivity, specificity, positive and negative likelihood ratio.

RESULTS

Similar mean AUC values were achieved for five models; VGGNet (0.96), ResNet (0.96), DenseNet (0.95), EfficientNet (0.96), and ViT (0.95). The same mean accuracy (0.94) and specificity values (0.96) were yielded for all models. In terms of sensitivity, EffcientNet achieved highest mean value (0.85) while the other models produced slightly lower values range from 0.82 to 0.84.

CONCLUSION

In this study, we demonstrated that DCNNs can classify the staging of liver fibrosis according to METAVIR score with high performance using conventional B-mode images. Among them, EfficientNET that have fewer parameters and computation cost produced highest performance. From the results, we believe that DCNNs based classification of liver fibrosis may allow fast and accurate diagnosis of liver fibrosis without needs of additional equipment for add-on test and may be powerful tool for supporting radiologists in clinical practice.

The Integration of Artificial Intelligence into Clinical Practice

The purpose of this literature review is to provide a fundamental synopsis of current research pertaining to artificial intelligence (AI) within the domain of clinical practice. Artificial intelligence has revolutionized the field of medicine and healthcare by providing innovative solutions to complex problems. One of the most important benefits of AI in clinical practice is its ability to investigate extensive volumes of data with efficiency and precision. This has led to the development of various applications that have improved patient outcomes and reduced the workload of healthcare professionals. AI can support doctors in making more accurate diagnoses and developing personalized treatment plans. Successful examples of AI applications are outlined for a series of medical specialties like cardiology, surgery, gastroenterology, pneumology, nephrology, urology, dermatology, orthopedics, neurology, gynecology, ophthalmology, pediatrics, hematology, and critically ill patients, as well as diagnostic methods. Special reference is made to legal and ethical considerations like accuracy, informed consent, privacy issues, data security, regulatory framework, product liability, explainability, and transparency. Finally, this review closes by critically appraising AI use in clinical practice and its future perspectives. However, it is also important to approach its development and implementation cautiously to ensure ethical considerations are met.

Liver fibrosis classification from ultrasound using machine learning: a systematic literature review

PURPOSE

Liver biopsy was considered the gold standard for diagnosing liver fibrosis; however, with advancements in medical technology and increasing awareness of potential complications, the reliance on liver biopsy has diminished. Ultrasound is gaining popularity due to its wider availability and cost-effectiveness. This study examined the machine learning / deep learning (ML/DL) models for non-invasive liver fibrosis classification from ultrasound.

METHODS

Following the preferred reporting items for systematic reviews and meta-analyses (PRISMA) protocol, we searched five academic databases using the query. We defined population, intervention, comparison, outcomes, and study design (PICOS) framework for the inclusion. Furthermore, Joana Briggs Institute (JBI) checklist for analytical cross-sectional studies is used for quality assessment.

RESULTS

Among the 188 screened studies, 17 studies are selected. The methods are categorized as off-the-shelf (OTS), attention, generative, and ensemble classifiers. Most studies used OTS classifiers that combined pre-trained ML/DL methods with radiomics features to determine fibrosis staging. Although machine learning shows potential for fibrosis classification, there are limited external comparisons of interventions and prospective clinical trials, which limits their applicability.

CONCLUSION

With the recent success of ML/DL toward biomedical image analysis, automated solutions using ultrasound are developed for predicting liver diseases. However, their applicability is bounded by the limited and imbalanced retrospective studies having high heterogeneity. This challenge could be addressed by generating a standard protocol for study design by selecting appropriate population, interventions, outcomes, and comparison.

Automatic Diagnosis of Significant Liver Fibrosis From Ultrasound B-Mode Images Using a Handcrafted-Feature-Assisted Deep Convolutional Neural Network

The accurate diagnosis of significant liver fibrosis ( ≥ F2) in patients with chronic liver disease (CLD) is critical, as ≥ F2 is a crucial factor that should be considered in selecting an antiviral therapy for these patients. This article proposes a handcrafted-feature-assisted deep convolutional neural network (HFA-DCNN) that helps radiologists automatically and accurately diagnose significant liver fibrosis from ultrasound (US) brightness (B)-mode images. The HFA-DCNN model has three main branches: one for automatic region of interest (ROI) segmentation in the US images, another for attention deep feature learning from the segmented ROI, and the third for handcrafted feature extraction. The attention deep learning features and handcrafted features are fused in the back end of the model to enable more accurate diagnosis of significant liver fibrosis. The usefulness and effectiveness of the proposed model were validated on a dataset built upon 321 CLD patients with liver fibrosis stages confirmed by pathological evaluations. In a fivefold cross validation (FFCV), the proposed model achieves accuracy, sensitivity, specificity, and area under the receiver-operating-characteristic (ROC) curve (AUC) values of 0.863 (95% confidence interval (CI) 0.820-0.899), 0.879 (95% CI 0.823-0.920), 0.872 (95% CI 0.800-0.925), and 0.925 (95% CI 0.891-0.952), which are significantly better than those obtained by the comparative methods. Given its excellent performance, the proposed HFA-DCNN model can serve as a promising tool for the noninvasive and accurate diagnosis of significant liver fibrosis in CLD patients.

The Big Bang of Deep Learning in Ultrasound-Guided Surgery: A Review

Ultrasound (US) imaging is a paramount modality in many image-guided surgeries and percutaneous interventions, thanks to its high portability, temporal resolution, and cost-efficiency. However, due to its imaging principles, the US is often noisy and difficult to interpret. Appropriate image processing can greatly enhance the applicability of the imaging modality in clinical practice. Compared with the classic iterative optimization and machine learning (ML) approach, deep learning (DL) algorithms have shown great performance in terms of accuracy and efficiency for US processing. In this work, we conduct a comprehensive review on deep-learning algorithms in the applications of US-guided interventions, summarize the current trends, and suggest future directions on the topic.

A deep learning model with data integration of ultrasound contrast-enhanced micro-flow cines, B-mode images, and clinical parameters for diagnosing significant liver fibrosis in patients with chronic hepatitis B

OBJECTIVE

To develop and investigate a deep learning model with data integration of ultrasound contrast-enhanced micro-flow (CEMF) cines, B-mode images, and patients' clinical parameters to improve the diagnosis of significant liver fibrosis (≥ F2) in patients with chronic hepatitis B (CHB).

METHODS

Of 682 CHB patients who underwent ultrasound and histopathological examinations between October 2016 and May 2020, 218 subjects were included in this retrospective study. We devised a data integration-based deep learning (DIDL) model for assessing ≥ F2 in CHB patients. The model contained three convolutional neural network branches to automatically extract features from ultrasound CEMF cines, B-mode images, and clinical data. The extracted features were fused at the backend of the model for decision-making. The diagnostic performance was evaluated across fivefold cross-validation and compared against the other methods in terms of the area under the receiver operating characteristic curve (AUC), with histopathological results as the reference standard.

RESULTS

The mean AUC achieved by the DIDL model was 0.901 [95% CI, 0.857-0.939], which was significantly higher than those of the comparative methods, including the models trained by using only CEMF cines (0.850 [0.794-0.893]), B-mode images (0.813 [0.754-0.862]), or clinical data (0.757 [0.694-0.812]), as well as the conventional TIC method (0.752 [0.689-0.808]), APRI (0.792 [0.734-0.845]), FIB-4 (0.776 [0.714-0.829]), and visual assessments of two radiologists (0.812 [0.754-0.862], and 0.800 [0.739-0.849]), all ps < 0.01, DeLong test.

CONCLUSION

The DIDL model with data integration of ultrasound CEMF cines, B-mode images, and clinical parameters showed promising performance in diagnosing significant liver fibrosis for CHB patients.

KEY POINTS

• The combined use of ultrasound contrast-enhanced micro-flow cines, B-mode images, and clinical data in a deep learning model has potential to improve the diagnosis of significant liver fibrosis. • The deep learning model with the fusion of features extracted from multimodality data outperformed the conventional methods including mono-modality data-based models, the time-intensity curve-based recognizer, fibrosis biomarkers, and visual assessments by experienced radiologists. • The interpretation of the feature attention maps in the deep learning model may help radiologists get better understanding of liver fibrosis-related features and hence potentially enhancing their diagnostic capacities.

Radiological Diagnosis of Chronic Liver Disease and Hepatocellular Carcinoma: A Review

Medical image analysis plays a pivotal role in the evaluation of diseases, including screening, surveillance, diagnosis, and prognosis. Liver is one of the major organs responsible for key functions of metabolism, protein and hormone synthesis, detoxification, and waste excretion. Patients with advanced liver disease and Hepatocellular Carcinoma (HCC) are often asymptomatic in the early stages; however delays in diagnosis and treatment can lead to increased rates of decompensated liver diseases, late-stage HCC, morbidity and mortality. Ultrasound (US) is commonly used imaging modality for diagnosis of chronic liver diseases that includes fibrosis, cirrhosis and portal hypertension. In this paper, we first provide an overview of various diagnostic methods for stages of liver diseases and discuss the role of Computer-Aided Diagnosis (CAD) systems in diagnosing liver diseases. Second, we review the utility of machine learning and deep learning approaches as diagnostic tools. Finally, we present the limitations of existing studies and outline future directions to further improve diagnostic accuracy, as well as reduce cost and subjectivity, while also improving workflow for the clinicians.

Convolutional neural network classification of ultrasound images by liver fibrosis stages based on echo-envelope statistics

Introduction: Assessing the stage of liver fibrosis during the diagnosis and follow-up of patients with diffuse liver disease is crucial. The tissue structure in the fibrotic liver is reflected in the texture and contrast of an ultrasound image, with the pixel brightness indicating the intensity of the echo envelope. Therefore, the progression of liver fibrosis can be evaluated non-invasively by analyzing ultrasound images.

Methods: A convolutional-neural-network (CNN) classification of ultrasound images was applied to estimate liver fibrosis. In this study, the colorization of the ultrasound images using echo-envelope statistics that correspond to the features of the images is proposed to improve the accuracy of CNN classification. In the proposed method, the ultrasound image is modulated by the 3rd- and 4th-order moments of pixel brightness. The two modulated images and the original image were then synthesized into a color image of RGB representation.

Results and Discussion: The colorized ultrasound images were classified via transfer learning of VGG-16 to evaluate the effect of colorization. Of the 80 ultrasound images with liver fibrosis stages F1–F4, 38 images were accurately classified by the CNN using the original ultrasound images, whereas 47 images were classified by the proposed method.

A Hierarchical Siamese Network for Noninvasive Staging of Liver Fibrosis Based on US Image Pairs of the Liver and Spleen

Due to the heterogeneity of ultrasound (US) images and the indeterminate US texture of liver fibrosis (LF), automatic evaluation of LF based on US images is still challenging. Thus, this study aimed to propose a hierarchical Siamese network that combines the information from liver and spleen US images to improve the accuracy of LF grading. There were two stages in the proposed method. In stage one, a dual-channel Siamese network was trained to extract features from paired liver and spleen patches that were cropped from US images to avoid vascular interferences. Subsequently, the L1 distance was used to quantify the liver-spleen differences (LSDs). In stage two, the pretrained weights from stage one were transferred into the Siamese feature extractor of the LF staging model, and a classifier was trained using the fusion of the liver and LSD features for LF staging. This study was retrospectively conducted on US images of 286 patients with histologically proven liver fibrosis stages. Our method achieved a precision and sensitivity of 93.92% and 91.65%, respectively, for cirrhosis (S4) diagnosis, which is about 8% higher than that of the baseline model. The accuracy of the advanced fibrosis (≥S3) diagnosis and the multi-staging of fibrosis (≤S2 vs. S3 vs. S4) both improved about 5% to reach 90.40% and 83.93%, respectively. This study proposed a novel method that combined hepatic and splenic US images and improved the accuracy of LF staging, which indicates the great potential of liver-spleen texture comparison in noninvasive assessment of LF based on US images.

Artificial intelligence - based ultrasound elastography for disease evaluation - a narrative review

Ultrasound elastography (USE) provides complementary information of tissue stiffness and elasticity to conventional ultrasound imaging. It is noninvasive and free of radiation, and has become a valuable tool to improve diagnostic performance with conventional ultrasound imaging. However, the diagnostic accuracy will be reduced due to high operator-dependence and intra- and inter-observer variability in visual observations of radiologists. Artificial intelligence (AI) has great potential to perform automatic medical image analysis tasks to provide a more objective, accurate and intelligent diagnosis. More recently, the enhanced diagnostic performance of AI applied to USE have been demonstrated for various disease evaluations. This review provides an overview of the basic concepts of USE and AI techniques for clinical radiologists and then introduces the applications of AI in USE imaging that focus on the following anatomical sites: liver, breast, thyroid and other organs for lesion detection and segmentation, machine learning (ML) - assisted classification and prognosis prediction. In addition, the existing challenges and future trends of AI in USE are also discussed.

Radiobioinformatics: A novel bridge between basic research and clinical practice for clinical decision support in diffuse liver diseases

The liver is a multifaceted organ that is responsible for many critical functions encompassing amino acid, carbohydrate, and lipid metabolism, all of which make a healthy liver essential for the human body. Contemporary imaging methodologies have remarkable diagnostic accuracy in discerning focal liver lesions; however, a comprehensive understanding of diffuse liver diseases is a requisite for radiologists to accurately diagnose or predict the progression of such lesions within clinical contexts. Nonetheless, the conventional attributes of radiological features, including morphology, size, margin, density, signal intensity, and echoes, limit their clinical utility. Radiomics is a widely used approach that is characterized by the extraction of copious image features from radiographic depictions, which gives it considerable potential in addressing this limitation. It is worth noting that functional or molecular alterations occur significantly prior to the morphological shifts discernible by imaging modalities. Consequently, the explication of potential mechanisms by multiomics analyses (encompassing genomics, epigenomics, transcriptomics, proteomics, and metabolomics) is essential for investigating putative signal pathway regulations from a radiological viewpoint. In this review, we elaborate on the principal pathological categorizations of diffuse liver diseases, the evaluation of multiomics approaches pertaining to diffuse liver diseases, and the prospective value of predictive models. Accordingly, the overarching objective of this review is to scrutinize the interrelations between radiological features and bioinformatics as well as to consider the development of prediction models predicated on radiobioinformatics as integral components of clinical decision support systems for diffuse liver diseases.

Diagnosis of Liver Fibrosis Using Artificial Intelligence: A Systematic Review

Background and Objectives: The development of liver fibrosis as a consequence of continuous inflammation represents a turning point in the evolution of chronic liver diseases. The recent developments of artificial intelligence (AI) applications show a high potential for improving the accuracy of diagnosis, involving large sets of clinical data. For this reason, the aim of this systematic review is to provide a comprehensive overview of current AI applications and analyze the accuracy of these systems to perform an automated diagnosis of liver fibrosis. Materials and Methods: We searched PubMed, Cochrane Library, EMBASE, and WILEY databases using predefined keywords. Articles were screened for relevant publications about AI applications capable of diagnosing liver fibrosis. Exclusion criteria were animal studies, case reports, abstracts, letters to the editor, conference presentations, pediatric studies, studies written in languages other than English, and editorials. Results: Our search identified a total of 24 articles analyzing the automated imagistic diagnosis of liver fibrosis, out of which six studies analyze liver ultrasound images, seven studies analyze computer tomography images, five studies analyze magnetic resonance images, and six studies analyze liver biopsies. The studies included in our systematic review showed that AI-assisted non-invasive techniques performed as accurately as human experts in detecting and staging liver fibrosis. Nevertheless, the findings of these studies need to be confirmed through clinical trials to be implemented into clinical practice. Conclusions: The current systematic review provides a comprehensive analysis of the performance of AI systems in diagnosing liver fibrosis. Automatic diagnosis, staging, and risk stratification for liver fibrosis is currently possible considering the accuracy of the AI systems, which can overcome the limitations of non-invasive diagnosis methods.

Artificial Intelligence (AI)-Enhanced Ultrasound Techniques Used in Non-Alcoholic Fatty Liver Disease: Are They Ready for Prime Time?

Non-alcoholic fatty liver disease (NAFLD) is the most prevalent cause of chronic liver disease, affecting approximately 2 billion individuals worldwide with a spectrum that can range from simple steatosis to cirrhosis. Typically, the diagnosis of NAFLD is based on imaging studies, but the gold standard remains liver biopsies. Hence, the use of artificial intelligence (AI) in this field, which has recently undergone rapid development in various aspects of medicine, has the potential to accurately diagnose NAFLD and steatohepatitis (NASH). This paper provides an overview of the latest research that employs AI for the diagnosis and staging of NAFLD, as well as applications for future developments in this field.

Development and evaluation of an artificial intelligence system for children intussusception diagnosis using ultrasound images

Accurate identification of intussusception in children is critical for timely non-surgical management. We propose an end-to-end artificial intelligence algorithm, the Children Intussusception Diagnosis Network (CIDNet) system, that utilizes ultrasound images to rapidly diagnose intussusception. 9999 ultrasound images of 4154 pediatric patients were divided into training, validation, test, and independent reader study datasets. The independent reader study cohort was used to compare the diagnostic performance of the CIDNet system to six radiologists. Performance was evaluated using, among others, balance accuracy (BACC) and area under the receiver operating characteristic curve (AUC). The CIDNet system performed the best in diagnosing intussusception with a BACC of 0.8464 and AUC of 0.9716 in the test dataset compared to other deep learning algorithms. The CIDNet system compared favorably with expert radiologists by outstanding identification performance and robustness (BACC:0.9297; AUC:0.9769). CIDNet is a stable and precise technological tool for identifying intussusception in ultrasound scans of children.

Artificial intelligence-aided method to detect uterine fibroids in ultrasound images: a retrospective study

We explored a new artificial intelligence-assisted method to assist junior ultrasonographers in improving the diagnostic performance of uterine fibroids and further compared it with senior ultrasonographers to confirm the effectiveness and feasibility of the artificial intelligence method. In this retrospective study, we collected a total of 3870 ultrasound images from 667 patients with a mean age of 42.45 years ± 6.23 [SD] for those who received a pathologically confirmed diagnosis of uterine fibroids and 570 women with a mean age of 39.24 years ± 5.32 [SD] without uterine lesions from Shunde Hospital of Southern Medical University between 2015 and 2020. The DCNN model was trained and developed on the training dataset (2706 images) and internal validation dataset (676 images). To evaluate the performance of the model on the external validation dataset (488 images), we assessed the diagnostic performance of the DCNN with ultrasonographers possessing different levels of seniority. The DCNN model aided the junior ultrasonographers (Averaged) in diagnosing uterine fibroids with higher accuracy (94.72% vs. 86.63%, P < 0.001), sensitivity (92.82% vs. 83.21%, P = 0.001), specificity (97.05% vs. 90.80%, P = 0.009), positive predictive value (97.45% vs. 91.68%, P = 0.007), and negative predictive value (91.73% vs. 81.61%, P = 0.001) than they achieved alone. Their ability was comparable to that of senior ultrasonographers (Averaged) in terms of accuracy (94.72% vs. 95.24%, P = 0.66), sensitivity (92.82% vs. 93.66%, P = 0.73), specificity (97.05% vs. 97.16%, P = 0.79), positive predictive value (97.45% vs. 97.57%, P = 0.77), and negative predictive value (91.73% vs. 92.63%, P = 0.75). The DCNN-assisted strategy can considerably improve the uterine fibroid diagnosis performance of junior ultrasonographers to make them more comparable to senior ultrasonographers.

A novel modified long short term memory architecture for automatic liver disease prediction from patient records

The liver is the second largest organ in the human body after the skin and liver disease mainly impacts the liver's functionality by properly separating the nutrients and waste into the digestive system and also causes scarring (cirrhosis) as time passes. The scarring over time affects the healthy liver tissue and also affects its proper functioning and if left untreated for a prolonged period it can also result in severe complications such as liver failure or liver cancer. The patients can be prevented from the severe complications if the disease is detected at an earlier stage and the existing research for liver disease prediction mainly encouraged the usage of intelligent machine learning‐based techniques. However, these techniques have several complexities such as low accuracy, overfitting, higher training time, poor feature extraction capabilities and so on. To overcome these problems, we present modified long short term emory (MLSTM) architecture for chronic liver disease prediction. The proposed methodology has three stages: information enhancement, feature extraction, and classification. The modified generative adversarial network uses an autoencoder system for sample augmentation which helps to enrich the diversity present in both the normal and abnormal classes. The outlier information is eliminated via the criminal search algorithm which captures the differences and correlation associated with multiple samples. The fast independent component analysis algorithm and enhanced whale optimization algorithm are used for feature extraction. This step mainly identifies the crucial features for liver disease prediction and leaves out the irrelevant and duplicate features thus enhancing the convergence, computational time, and prediction accuracy. The MLSTM architecture is used to classify the samples present in the liver disease datasets into normal and abnormal (liver disease) classes. The proposed methodology offers improved performance in terms of accuracy, recall, means square error, and F‐measure. The results show that the proposed methodology will be efficient for doctors to diagnose liver disease in the earlier stage.

Evaluation of Hepatic Fibrosis Using Ultrasound Backscattered Radiofrequency Signals and One-Dimensional Convolutional Neural Networks

The early detection of hepatic fibrosis is of critical importance. Ultrasound backscattered radiofrequency signals from the liver contain abundant information about its microstructure. We proposed a method for characterizing human hepatic fibrosis using one-dimensional convolutional neural networks (CNNs) based on ultrasound backscattered signals. The proposed CNN model was composed of four one-dimensional convolutional layers, four one-dimensional max-pooling layers, and four fully connected layers. Ultrasound radiofrequency signals collected from 230 participants (F0: 23; F1: 46; F2: 51; F3: 49; F4: 61) with a 3-MHz transducer were analyzed. Liver regions of interest (ROIs) that contained most of the liver ultrasound backscattered signals were manually delineated using B-mode images reconstructed from the backscattered signals. ROI signals were normalized and augmented by using a sliding window technique. After data augmentation, the radiofrequency signal segments were divided into training sets, validation sets and test sets at a ratio of 80%:10%:10%. In the test sets, the proposed algorithm produced an area under the receive operating characteristic curve of 0.933 (accuracy: 91.30%; sensitivity: 92.00%; specificity: 90.48%), 0.997 (accuracy: 94.29%; sensitivity: 94.74%; specificity: 93.75%), 0.818 (accuracy: 75.00%; sensitivity: 69.23%; specificity: 81.82%), and 0.934 (accuracy: 91.67%; sensitivity: 88.89%; specificity: 94.44%) for diagnosis liver fibrosis stage ≥F1, ≥F2, ≥F3, and ≥F4, respectively. Experimental results indicated that the proposed deep learning algorithm based on ultrasound backscattered signals yields a satisfying performance when diagnosing hepatic fibrosis stages. The proposed method may be used as a new quantitative ultrasound approach to characterizing hepatic fibrosis.

Artificial intelligence based liver portal tract region identification and quantification with transplant biopsy whole-slide images

Liver fibrosis staging is clinically important for liver disease progression prediction. As the portal tract fibrotic quantity and size in a liver biopsy correlate with the fibrosis stage, an accurate analysis of portal tract regions is clinically critical. Manual annotations of portal tract regions, however, are time-consuming and subject to large inter- and intra-observer variability. To address such a challenge, we develop a Multiple Up-sampling and Spatial Attention guided UNet model (MUSA-UNet) to segment liver portal tract regions in whole-slide images of liver tissue slides. To enhance the segmentation performance, we propose to use depth-wise separable convolution, the spatial attention mechanism, the residual connection, and multiple up-sampling paths in the developed model. This study includes 53 histopathology whole slide images from patients who received liver transplantation. In total, 6,012 patches derived from 30 images are used for our deep learning model training and validation. The remaining 23 whole slide images are utilized for the model testing. The average liver portal tract segmentation performance of the developed MUSA-UNet is 0.94 (Precision), 0.85 (Recall), 0.89 (F1 Score), 0.89 (Accuracy), 0.80 (Jaccard Index), and 0.91 (Fowlkes-Mallows Index), respectively. The clinical Scheuer fibrosis stage presents a strong correlation with the resulting average portal tract fibrotic area (R = 0.681, p<0.001) and portal tract percentage (R = 0.335, p = 0.02) computed from the MUSA-UNet segmentation results. In conclusion, our developed deep learning model MUSA-UNet can accurately segment portal tract regions from whole-slide images of liver tissue biopsies, presenting its promising potential to assist liver disease diagnosis in a computational manner.

Ultrasound-based artificial intelligence in gastroenterology and hepatology

Artificial intelligence (AI), especially deep learning, is gaining extensive attention for its excellent performance in medical image analysis. It can automatically make a quantitative assessment of complex medical images and help doctors to make more accurate diagnoses. In recent years, AI based on ultrasound has been shown to be very helpful in diffuse liver diseases and focal liver lesions, such as analyzing the severity of nonalcoholic fatty liver and the stage of liver fibrosis, identifying benign and malignant liver lesions, predicting the microvascular invasion of hepatocellular carcinoma, curative transarterial chemoembolization effect, and prognoses after thermal ablation. Moreover, AI based on endoscopic ultrasonography has been applied in some gastrointestinal diseases, such as distinguishing gastric mesenchymal tumors, detection of pancreatic cancer and intraductal papillary mucinous neoplasms, and predicting the preoperative tumor deposits in rectal cancer. This review focused on the basic technical knowledge about AI and the clinical application of AI in ultrasound of liver and gastroenterology diseases. Lastly, we discuss the challenges and future perspectives of AI.

Automated prediction of early spontaneous miscarriage based on the analyzing ultrasonographic gestational sac imaging by the convolutional neural network: a case-control and cohort study

Background

It is challenging to predict the outcome of the pregnancy when fetal heart activity is detected in early pregnancy. However, an accurate prediction is of importance for obstetricians as it helps to provide appropriate consultancy and determine the frequency of ultrasound examinations. The purpose of this study was to investigate the role of the convolutional neural network (CNN) in the prediction of spontaneous miscarriage risk through the analysis of early ultrasound gestational sac images.

Methods

A total of 2196 ultrasound images from 1098 women with early singleton pregnancies of gestational age between 6 and 8 weeks were used for training a CNN for the prediction of the miscarriage in the retrospective study. The patients who had positive fetal cardiac activity on their first ultrasound but then experienced a miscarriage were enrolled. The control group was randomly selected in the same database from the fetuses confirmed to be normal during follow-up. Diagnostic performance of the algorithm was validated and tested in two separate test sets of 136 patients with 272 images, respectively. Performance in prediction of the miscarriage was compared between the CNN and the manual measurement of ultrasound characteristics in the prospective study.

Results

The accuracy of the predictive model was 80.32% and 78.1% in the retrospective and prospective study, respectively. The area under the receiver operating characteristic curve (AUC) for classification was 0.857 (95% confidence interval [CI], 0.793–0.922) in the retrospective study and 0.885 (95%CI, 0.846–0.925) in the prospective study, respectively. Correspondingly, the predictive power of the CNN was higher compared with manual ultrasound characteristics, for which the AUCs of the crown-rump length combined with fetal heart rate was 0.687 (95%CI, 0.587–0.775).

Conclusions

The CNN model showed high accuracy for predicting miscarriage through the analysis of early pregnancy ultrasound images and achieved better performance than that of manual measurement.

Supplementary Information

The online version contains supplementary material available at 10.1186/s12884-022-04936-0.

Performance of a generative adversarial network using ultrasound images to stage liver fibrosis and predict cirrhosis based on a deep-learning radiomics nomogram

AIM

To investigate the performance of a generative adversarial network (GAN) model for staging liver fibrosis and its radiomics-based nomogram for predicting cirrhosis.

MATERIALS AND METHODS

This two-centre retrospective study included 434 patients for whom input data of ultrasound images and histopathological data (obtained within 1 month of ultrasound examinations) were assigned to the training cohort (249 patients), the internal cohort (92 patients), and the external (93 patients) cohort. A data augmentation method based on a GAN model was used. The discriminative performance was evaluated for classifying fibrosis of S4 and ≥S3. Deep-learning radiomics features were extracted for the prediction of cirrhosis (S4). To perform feature reduction and selection, the least absolute shrinkage and selection operator (LASSO) algorithm was applied. Radiomics scores, along with clinical factors, were incorporated into a nomogram using multivariable logistic regression analysis. The performance of the models was estimated with respect to discrimination power, calibration, and clinical benefits.

RESULTS

The areas under the receiver operating characteristic curve (AUCs) values of the GAN were 0.832/0.762 (≥S3), and 0.867/0.835 (S4) for internal/external test sets, respectively. The radiomics nomogram that intergrated radiomics scores and clinical factors showed good calibration and discrimination ability of 0.922 (AUC) in the training dataset, 0.896 in the internal dataset, and 0.861 in the external dataset. Decision curve analysis (DCA) demonstrated that the nomogram outperformed radiologist and haematological indices in terms of the most clinical benefits.

CONCLUSIONS

The GAN model could be applied to discriminate fibrosis stages, and a favourable predictive accuracy for diagnosing cirrhosis was achieved using a deep-learning radiomics nomogram.

MultiHeadGAN: A deep learning method for low contrast retinal pigment epithelium cell segmentation with fluorescent flatmount microscopy images

BACKGROUND

Retinal pigment epithelium (RPE) aging is an important cause of vision loss. As RPE aging is accompanied by changes in cell morphological features, an accurate segmentation of RPE cells is a prerequisite to such morphology analyses. Due the overwhelmingly large cell number, manual annotations of RPE cell borders are time-consuming. Computer based methods do not work well on cells with weak or missing borders in the impaired RPE sheet regions.

METHOD

To address such a challenge, we develop a semi-supervised deep learning approach, namely MultiHeadGAN, to segment low contrast cells from impaired regions in RPE flatmount images. The developed deep learning model has a multi-head structure that allows model training with only a small scale of human annotated data. To strengthen model learning, we further train our model with RPE cells without ground truth cell borders by generative adversarial networks. Additionally, we develop a new shape loss to guide the network to produce closed cell borders in the segmentation results.

RESULTS

In this study, 155 annotated and 1,640 unlabeled image patches are included for model training. The testing dataset consists of 200 image patches presenting large impaired RPE regions. The average RPE segmentation performance of the developed model MultiHeadGAN is 85.4 (correct rate), 88.8 (weighted correct rate), 87.3 (precision), and 80.1 (recall), respectively. Compared with other state-of-the-art deep learning approaches, our method demonstrates its superior qualitative and quantitative performance.

CONCLUSIONS

Suggested by our extensive experimental results, our developed deep learning method can accurately segment cells in RPE flatmount microscopy images and is promising to support large scale cell morphological analyses for RPE aging investigations.