Code-free deep learning for multi-modality medical image classification

Code-Free Deep Learning Glaucoma Detection on Color Fundus Images

Objective

Code-free deep learning (CFDL) allows clinicians with no coding experience to build their own artificial intelligence models. This study assesses the performance of CFDL in glaucoma detection from fundus images in comparison to expert-designed models.

Design

Deep learning model development, testing, and validation.

Subjects

A total of 101 442 labeled fundus images from the Rotterdam EyePACS Artificial Intelligence for Robust Glaucoma Screening (AIROGS) dataset were included.

Methods

Ophthalmology trainees without coding experience designed a CFDL binary model using the Rotterdam EyePACS AIROGS dataset of fundus images (101 442 labeled images) to differentiate glaucoma from normal optic nerves. We compared our results with bespoke models from the literature. We then proceeded to externally validate our model using 2 datasets, the Retinal Fundus Glaucoma Challenge (REFUGE) and the Glaucoma grading from Multi-Modality imAges (GAMMA) at 0.1, 0.3, and 0.5 confidence thresholds.

Main Outcome Measures

Area under the precision-recall curve (AuPRC), sensitivity at 95% specificity (SE@95SP), accuracy, area under the receiver operating curve (AUC), and positive predictive value (PPV).

Results

The CFDL model showed high performance metrics that were comparable to the bespoke deep learning models. Our single-label classification model had an AuPRC of 0.988, an SE@95SP of 95%, and an accuracy of 91% (compared with 85% SE@95SP for the top bespoke models). Using the REFUGE dataset for external validation, our model had an SE@95SP, AUC, PPV, and accuracy of 83%, 0.960%, 73% to 94%, and 95% to 98%, respectively, at the 0.1, 0.3, and 0.5 confidence threshold cutoffs. Using the GAMMA dataset for external validation at the same confidence threshold cutoffs, our model had an SE@95SP, AUC, PPV, and accuracy of 98%, 0.994%, 94% to 96%, and 94% to 97%, respectively.

Conclusion

The capacity of CFDL models to perform glaucoma screening using fundus images presents a compelling proof of concept, empowering clinicians to explore innovative model designs for broad glaucoma screening in the near future.

Financial Disclosures

Proprietary or commercial disclosure may be found in the Footnotes and Disclosures at the end of this article.

Application of open domain adaptive models in image annotation and classification

In the field of computer vision, the task of image annotation and classification has attracted much attention due to its wide demand in applications such as medical image analysis, intelligent surveillance, and image retrieval. However, existing methods have significant limitations in dealing with unknown target domain data, which are manifested in the problems of reduced classification accuracy and insufficient generalization ability. To this end, the study proposes an adaptive image annotation classification model for open-set domains based on dynamic threshold control and subdomain alignment strategy to address the impact of the difference between the source and target domain distributions on the classification performance. The model combines the channel attention mechanism to dynamically extract important features, optimizes the cross-domain feature alignment effect using dynamic weight adjustment and subdomain alignment strategy, and balances the classification performance of known and unknown categories by dynamic threshold control. The experiments are conducted on ImageNet and COCO datasets, and the results show that the proposed model has a classification accuracy of up to 93.5% in the unknown target domain and 89.6% in the known target domain, which is better than the best results of existing methods. Meanwhile, the model check accuracy and recall rate reach up to 89.6% and 90.7%, respectively, and the classification time is only 1.2 seconds, which significantly improves the classification accuracy and efficiency. It is shown that the method can effectively improve the robustness and generalization ability of the image annotation and classification task in open-set scenarios, and provides a new idea for solving the domain adaptation problem in real scenarios.

Asymmetric Adaptive Heterogeneous Network for Multi-Modality Medical Image Segmentation

Existing studies of multi-modality medical image segmentation tend to aggregate all modalities without discrimination and employ multiple symmetric encoders or decoders for feature extraction and fusion. They often overlook the different contributions to visual representation and intelligent decisions among multi-modality images. Motivated by this discovery, this paper proposes an asymmetric adaptive heterogeneous network for multi-modality image feature extraction with modality discrimination and adaptive fusion. For feature extraction, it uses a heterogeneous two-stream asymmetric feature-bridging network to extract complementary features from auxiliary multi-modality and leading single-modality images, respectively. For feature adaptive fusion, the proposed Transformer-CNN Feature Alignment and Fusion (T-CFAF) module enhances the leading single-modality information, and the Cross-Modality Heterogeneous Graph Fusion (CMHGF) module further fuses multi-modality features at a high-level semantic layer adaptively. Comparative evaluation with ten segmentation models on six datasets demonstrates significant efficiency gains as well as highly competitive segmentation accuracy. (Our code is publicly available at https://github.com/joker-527/AAHN).

Advances in machine learning for keratoconus diagnosis

PURPOSE

To review studies reporting the role of Machine Learning (ML) techniques in the diagnosis of keratoconus (KC) over the past decade, shedding light on recent developments while also highlighting the existing gaps between academic research and practical implementation in clinical settings.

METHODS

The review process begins with a systematic search of primary digital libraries using relevant keywords. A rigorous set of inclusion and exclusion criteria is then applied, resulting in the identification of 62 articles for analysis. Key research questions are formulated to address advancements in ML for KC diagnosis, corneal imaging modalities, types of datasets utilised, and the spectrum of KC conditions investigated over the past decade. A significant gap between academic research and practical implementation in clinical settings is identified, forming the basis for actionable recommendations tailored for both ML developers and ophthalmologists. Additionally, a proposed roadmap model is presented to facilitate the integration of ML models into clinical practice, enhancing diagnostic accuracy and patient care.

RESULTS

The analysis revealed that the diagnosis of KC predominantly relies on supervised classifiers (97%), with Random Forest being the most used algorithm (27%), followed by Deep Learning including Convolution Neural Networks (16%), Feedforward and Feedback Neural Networks (12%), and Support Vector Machines (12%). Pentacam is identified as the leading corneal imaging modality (56%), and a substantial majority of studies (91%) utilize local datasets, primarily consisting of numerical corneal parameters (77%). The most studied KC conditions were non-KC (NKC) vs. clinical KC (CKC) (29%), NKC vs. Subclinical KC (SCKC) (24%), NKC vs. SCKC vs. CKC (20%), SCKC vs. CKC (7%). However, only 20% of studies focused on addressing KC severity stages, emphasizing the need for more research in this area. These findings highlight the current landscape of ML in KC diagnosis and uncover existing challenges, and suggest potential avenues for further research and development, with particular emphasis on the dominance of certain algorithms and imaging modalities.

CONCLUSION

Key obstacles include the lack of consensus on an objective diagnostic standard for early KC detection and severity staging, limited multidisciplinary collaboration, and restricted access to public datasets. Further research is crucial to overcome these challenges and apply findings in clinical practice.

Artificial intelligence model on images of functional dyspepsia

BACKGROUND

Recently, it has been suggested that the duodenum may be the pathological locus of functional dyspepsia (FD). Additionally, an image-based artificial intelligence (AI) model was shown to discriminate colonoscopy images of irritable bowel syndrome from healthy subjects with an area under the curve (AUC) 0.95.

AIM

To evaluate an AI model to distinguish duodenal images of FD patients from healthy subjects.

METHODS

Duodenal images were collected from hospital records and labeled as "functional dyspepsia" or non-FD in electronic medical records. Helicobacter pylori (HP) infection status was obtained from the Japan Endoscopy Database. Google Cloud AutoML Vision was used to classify four groups: FD/HP current infection (n = 32), FD/HP uninfected (n = 35), non-FD/HP current infection (n = 39), and non-FD/HP uninfected (n = 33). Patients with organic diseases (e.g., cancer, ulcer, postoperative abdomen, reflux) and narrow-band or dye-spread images were excluded. Sensitivity, specificity, and AUC were calculated.

RESULTS

In total, 484 images were randomly selected for FD/HP current infection, FD/HP uninfected, non-FD/current infection, and non-FD/HP uninfected. The overall AUC for the four groups was 0.47. The individual AUC values were as follows: FD/HP current infection (0.20), FD/HP uninfected (0.35), non-FD/current infection (0.46), and non-FD/HP uninfected (0.74). Next, using the same images, we constructed models to determine the presence or absence of FD in the HP-infected or uninfected patients. The model exhibited a sensitivity of 58.3%, specificity of 100%, positive predictive value of 100%, negative predictive value of 77.3%, and an AUC of 0.85 in HP uninfected patients.

CONCLUSION

We developed an image-based AI model to distinguish duodenal images of FD from healthy subjects, showing higher accuracy in HP-uninfected patients. These findings suggest AI-assisted endoscopic diagnosis of FD may be feasible.

Retinal vein occlusion risk prediction without fundus examination using a no-code machine learning tool for tabular data: a nationwide cross-sectional study from South Korea

BACKGROUND

Retinal vein occlusion (RVO) is a leading cause of vision loss globally. Routine health check-up data-including demographic information, medical history, and laboratory test results-are commonly utilized in clinical settings for disease risk assessment. This study aimed to develop a machine learning model to predict RVO risk in the general population using such tabular health data, without requiring coding expertise or retinal imaging.

METHODS

We utilized data from the Korea National Health and Nutrition Examination Surveys (KNHANES) collected between 2017 and 2020 to develop the RVO prediction model, with external validation performed using independent data from KNHANES 2021. Model construction was conducted using Orange Data Mining, an open-source, code-free, component-based tool with a user-friendly interface, and Google Vertex AI. An easy-to-use oversampling function was employed to address class imbalance, enhancing the usability of the workflow. Various machine learning algorithms were trained by incorporating all features from the health check-up data in the development set. The primary outcome was the area under the receiver operating characteristic curve (AUC) for identifying RVO.

RESULTS

All machine learning training was completed without the need for coding experience. An artificial neural network (ANN) with a ReLU activation function, developed using Orange Data Mining, demonstrated superior performance, achieving an AUC of 0.856 (95% confidence interval [CI], 0.835-0.875) in internal validation and 0.784 (95% CI, 0.763-0.803) in external validation. The ANN outperformed logistic regression and Google Vertex AI models, though differences were not statistically significant in internal validation. In external validation, the ANN showed a marginally significant improvement over logistic regression (P = 0.044), with no significant difference compared to Google Vertex AI. Key predictive variables included age, household income, and blood pressure-related factors.

CONCLUSION

This study demonstrates the feasibility of developing an accessible, cost-effective RVO risk prediction tool using health check-up data and no-code machine learning platforms. Such a tool has the potential to enhance early detection and preventive strategies in general healthcare settings, thereby improving patient outcomes.

Comparing No-Code Platforms and Deep Learning Models for Glaucoma Detection From Fundus Images

PURPOSE

This study compares the performance of two no-code machine learning platforms, Google's Teachable Machine (TM) (Google LLC, Mountain View, CA, USA) and Apple's Create ML (Apple Inc., Cupertino, CA, USA), alongside a traditional deep learning model, ResNet200d, in classifying optic nerve fundus images into glaucoma and non-glaucoma categories using the ACRIMA dataset.

METHODS

A comparative cross-sectional analysis was conducted using 705 labeled fundus images from the ACRIMA dataset (326 glaucomatous, 239 non-glaucomatous). Models were trained separately on each platform, and a validation set comprising 70 glaucomatous and 70 non-glaucomatous images was used to assess performance. Performance metrics, such as sensitivity, specificity, F1 score, and Cohen's kappa, were assessed with 95% confidence intervals. Statistical analysis was performed using DATAtab (DATAtab e.U. Graz, Austria (https://datatab.net)).

RESULTS

The ResNet200d model demonstrated the highest performance, with an accuracy of 99.29%, a sensitivity of 98.57%, a specificity of 100%, and an F1 score of 99.29%. Create ML achieved a sensitivity of 93.24%, a specificity of 98.48%, and an F1 score of 95.83%. TM exhibited a sensitivity of 95.71%, a specificity of 94.29%, and an F1 score of 95.04%. Both no-code platforms demonstrated strong performance, with Create ML excelling in specificity and TM showing higher sensitivity.

CONCLUSION

While the ResNet200d model outperformed both no-code platforms in diagnostic accuracy, the no-code platforms demonstrated robust capabilities, highlighting their potential to democratize artificial intelligence (AI) in healthcare. These results highlight the potential of no-code platforms for democratizing medical image analysis, especially in resource-limited contexts. Further studies with diverse datasets are recommended to validate these results.

A review of AutoML optimization techniques for medical image applications

Automatic analysis of medical images using machine learning techniques has gained significant importance over the years. A large number of approaches have been proposed for solving different medical image analysis tasks using machine learning and deep learning approaches. These approaches are quite effective thanks to their ability to analyze large volume of medical imaging data. Moreover, they can also identify patterns that may be difficult for human experts to detect. Manually designing and tuning the parameters of these algorithms is a challenging and time-consuming task. Furthermore, designing a generalized model that can handle different imaging modalities is difficult, as each modality has specific characteristics. To solve these problems and automate the whole pipeline of different medical image analysis tasks, numerous Automatic Machine Learning (AutoML) techniques have been proposed. These techniques include Hyper-parameter Optimization (HPO), Neural Architecture Search (NAS), and Automatic Data Augmentation (ADA). This study provides an overview of several AutoML-based approaches for different medical imaging tasks in terms of optimization search strategies. The usage of optimization techniques (evolutionary, gradient-based, Bayesian optimization, etc.) is of significant importance for these AutoML approaches. We comprehensively reviewed existing AutoML approaches, categorized them, and performed a detailed analysis of different proposed approaches. Furthermore, current challenges and possible future research directions are also discussed.

PlanText: Gradually Masked Guidance to Align Image Phenotypes with Trait Descriptions for Plant Disease Texts

Plant diseases are a critical driver of the global food crisis. The integration of advanced artificial intelligence technologies can substantially enhance plant disease diagnostics. However, current methods for early and complex detection remain challenging. Employing multimodal technologies, akin to medical artificial intelligence diagnostics that combine diverse data types, may offer a more effective solution. Presently, the reliance on single-modal data predominates in plant disease research, which limits the scope for early and detailed diagnosis. Consequently, developing text modality generation techniques is essential for overcoming the limitations in plant disease recognition. To this end, we propose a method for aligning plant phenotypes with trait descriptions, which diagnoses text by progressively masking disease images. First, for training and validation, we annotate 5,728 disease phenotype images with expert diagnostic text and provide annotated text and trait labels for 210,000 disease images. Then, we propose a PhenoTrait text description model, which consists of global and heterogeneous feature encoders as well as switching-attention decoders, for accurate context-aware output. Next, to generate a more phenotypically appropriate description, we adopt 3 stages of embedding image features into semantic structures, which generate characterizations that preserve trait features. Finally, our experimental results show that our model outperforms several frontier models in multiple trait descriptions, including the larger models GPT-4 and GPT-4o. Our code and dataset are available at https://plantext.samlab.cn/.

Automated Machine Learning versus Expert-Designed Models in Ocular Toxoplasmosis: Detection and Lesion Localization Using Fundus Images

PURPOSE

Automated machine learning (AutoML) allows clinicians without coding experience to build their own deep learning (DL) models. This study assesses the performance of AutoML in detecting and localizing ocular toxoplasmosis (OT) lesions in fundus images and compares it to expert-designed models.

METHODS

Ophthalmology trainees without coding experience designed AutoML models using 304 labelled fundus images. We designed a binary model to differentiate OT from normal and an object detection model to visually identify OT lesions.

RESULTS

The AutoML model had an area under the precision-recall curve (AuPRC) of 0.945, sensitivity of 100%, specificity of 83% and accuracy of 93.5% (vs. 94%, 86% and 91% for the bespoke models). The AutoML object detection model had an AuPRC of 0.600 with a precision of 93.3% and recall of 56%. Using a diversified external validation dataset, our model correctly labeled 15 normal fundus images (100%) and 15 OT fundus images (100%), with a mean confidence score of 0.965 and 0.963, respectively.

CONCLUSION

AutoML models created by ophthalmologists without coding experience were comparable or better than expert-designed bespoke models trained on the same dataset. By creatively using AutoML to identify OT lesions on fundus images, our approach brings the whole spectrum of DL model design into the hands of clinicians.

A Convolutional Neural Network Model for Distinguishing Hemangioblastomas From Other Cerebellar-and-Brainstem Tumors Using Contrast-Enhanced MRI

BACKGROUND

Hemangioblastoma (HB) is a highly vascularized tumor most commonly occurring in the posterior cranial fossa, requiring accurate preoperative diagnosis to avoid accidental intraoperative hemorrhage and even death.

PURPOSE

To accurately distinguish HBs from other cerebellar-and-brainstem tumors using a convolutional neural network model based on a contrast-enhanced brain MRI dataset.

STUDY TYPE

Retrospective.

POPULATION

Four hundred five patients (182 = HBs; 223 = other cerebellar-and brainstem tumors): 305 cases for model training, and 100 for evaluation.

FIELD STRENGTH/SEQUENCE

3 T/contrast-enhanced T1-weighted imaging (T1WI + C).

ASSESSMENT

A CNN-based 2D classification network was trained by using sliced data along the z-axis. To improve the performance of the network, we introduced demographic information, various data-augmentation methods and an auxiliary task to segment tumor region. Then, this method was compared with the evaluations performed by experienced and intermediate-level neuroradiologists, and the heatmap of deep feature, which indicates the contribution of each pixel to model prediction, was visualized by Grad-CAM for analyzing the misclassified cases.

STATISTICAL TESTS

The Pearson chi-square test and an independent t-test were used to test for distribution difference in age and sex. And the independent t-test was exploited to evaluate the performance between experts and our proposed method. P value <0.05 was considered significant.

RESULTS

The trained network showed a higher accuracy for identifying HBs (accuracy = 0.902 ± 0.031, F1 = 0.891 ± 0.035, AUC = 0.926 ± 0.040) than experienced (accuracy = 0.887 ± 0.013, F1 = 0.868 ± 0.011, AUC = 0.881 ± 0.008) and intermediate-level (accuracy = 0.827 ± 0.037, F1 = 0.768 ± 0.068, AUC = 0.810 ± 0.047) neuroradiologists. The recall values were 0.910 ± 0.050, 0.659 ± 0.084, and 0.828 ± 0.019 for the trained network, intermediate and experienced neuroradiologists, respectively. Additional ablation experiments verified the utility of the introduced demographic information, data augmentation, and the auxiliary-segmentation task.

DATA CONCLUSION

Our proposed method can successfully distinguish HBs from other cerebellar-and-brainstem tumors and showed diagnostic efficiency comparable to that of experienced neuroradiologists.

EVIDENCE LEVEL

3 TECHNICAL EFFICACY: Stage 2.

A comparative evaluation of deep learning approaches for ophthalmology

There is a growing number of publicly available ophthalmic imaging datasets and open-source code for Machine Learning algorithms. This allows ophthalmic researchers and practitioners to independently perform various deep-learning tasks. With the advancement in artificial intelligence (AI) and in the field of imaging, the choice of the most appropriate AI architecture for different tasks will vary greatly. The best-performing AI-dataset combination will depend on the specific problem that needs to be solved and the type of data available. The article discusses different machine learning models and deep learning architectures currently used for various ophthalmic imaging modalities and for different machine learning tasks. It also proposes the most appropriate models based on accuracy and other important factors such as training time, the ability to deploy the model on clinical devices/smartphones, heatmaps that enhance the self-explanatory nature of classification decisions, and the ability to train/adapt on small image datasets to determine if further data collection is worthwhile. The article extensively reviews the existing state-of-the-art AI methods focused on useful machine-learning applications for ophthalmology. It estimates their performance and viability through training and evaluating architectures with different public and private image datasets of different modalities, such as full-color retinal images, OCT images, and 3D OCT scans. The article is expected to benefit the readers by enriching their knowledge of artificial intelligence applied to ophthalmology.

Characterizing Sentinel Lymph Node Status in Breast Cancer Patients Using a Deep-Learning Model Compared With Radiologists' Analysis of Grayscale Ultrasound and Lymphosonography

ABSTRACT

The objective of the study was to use a deep learning model to differentiate between benign and malignant sentinel lymph nodes (SLNs) in patients with breast cancer compared to radiologists' assessments.Seventy-nine women with breast cancer were enrolled and underwent lymphosonography and contrast-enhanced ultrasound (CEUS) examination after subcutaneous injection of ultrasound contrast agent around their tumor to identify SLNs. Google AutoML was used to develop image classification model. Grayscale and CEUS images acquired during the ultrasound examination were uploaded with a data distribution of 80% for training/20% for testing. The performance metric used was area under precision/recall curve (AuPRC). In addition, 3 radiologists assessed SLNs as normal or abnormal based on a clinical established classification. Two-hundred seventeen SLNs were divided in 2 for model development; model 1 included all SLNs and model 2 had an equal number of benign and malignant SLNs. Validation results model 1 AuPRC 0.84 (grayscale)/0.91 (CEUS) and model 2 AuPRC 0.91 (grayscale)/0.87 (CEUS). The comparison between artificial intelligence (AI) and readers' showed statistical significant differences between all models and ultrasound modes; model 1 grayscale AI versus readers, P = 0.047, and model 1 CEUS AI versus readers, P < 0.001. Model 2 r grayscale AI versus readers, P = 0.032, and model 2 CEUS AI versus readers, P = 0.041.The interreader agreement overall result showed κ values of 0.20 for grayscale and 0.17 for CEUS.In conclusion, AutoML showed improved diagnostic performance in balance volume datasets. Radiologist performance was not influenced by the dataset's distribution.

Comparing code-free and bespoke deep learning approaches in ophthalmology

AIM

Code-free deep learning (CFDL) allows clinicians without coding expertise to build high-quality artificial intelligence (AI) models without writing code. In this review, we comprehensively review the advantages that CFDL offers over bespoke expert-designed deep learning (DL). As exemplars, we use the following tasks: (1) diabetic retinopathy screening, (2) retinal multi-disease classification, (3) surgical video classification, (4) oculomics and (5) resource management.

METHODS

We performed a search for studies reporting CFDL applications in ophthalmology in MEDLINE (through PubMed) from inception to June 25, 2023, using the keywords 'autoML' AND 'ophthalmology'. After identifying 5 CFDL studies looking at our target tasks, we performed a subsequent search to find corresponding bespoke DL studies focused on the same tasks. Only English-written articles with full text available were included. Reviews, editorials, protocols and case reports or case series were excluded. We identified ten relevant studies for this review.

RESULTS

Overall, studies were optimistic towards CFDL's advantages over bespoke DL in the five ophthalmological tasks. However, much of such discussions were identified to be mono-dimensional and had wide applicability gaps. High-quality assessment of better CFDL applicability over bespoke DL warrants a context-specific, weighted assessment of clinician intent, patient acceptance and cost-effectiveness. We conclude that CFDL and bespoke DL are unique in their own assets and are irreplaceable with each other. Their benefits are differentially valued on a case-to-case basis. Future studies are warranted to perform a multidimensional analysis of both techniques and to improve limitations of suboptimal dataset quality, poor applicability implications and non-regulated study designs.

CONCLUSION

For clinicians without DL expertise and easy access to AI experts, CFDL allows the prototyping of novel clinical AI systems. CFDL models concert with bespoke models, depending on the task at hand. A multidimensional, weighted evaluation of the factors involved in the implementation of those models for a designated task is warranted.

Exploring Publicly Accessible Optical Coherence Tomography Datasets: A Comprehensive Overview

Artificial intelligence has transformed medical diagnostic capabilities, particularly through medical image analysis. AI algorithms perform well in detecting abnormalities with a strong performance, enabling computer-aided diagnosis by analyzing the extensive amounts of patient data. The data serve as a foundation upon which algorithms learn and make predictions. Thus, the importance of data cannot be underestimated, and clinically corresponding datasets are required. Many researchers face a lack of medical data due to limited access, privacy concerns, or the absence of available annotations. One of the most widely used diagnostic tools in ophthalmology is Optical Coherence Tomography (OCT). Addressing the data availability issue is crucial for enhancing AI applications in the field of OCT diagnostics. This review aims to provide a comprehensive analysis of all publicly accessible retinal OCT datasets. Our main objective is to compile a list of OCT datasets and their properties, which can serve as an accessible reference, facilitating data curation for medical image analysis tasks. For this review, we searched through the Zenodo repository, Mendeley Data repository, MEDLINE database, and Google Dataset search engine. We systematically evaluated all the identified datasets and found 23 open-access datasets containing OCT images, which significantly vary in terms of size, scope, and ground-truth labels. Our findings indicate the need for improvement in data-sharing practices and standardized documentation. Enhancing the availability and quality of OCT datasets will support the development of AI algorithms and ultimately improve diagnostic capabilities in ophthalmology. By providing a comprehensive list of accessible OCT datasets, this review aims to facilitate better utilization and development of AI in medical image analysis.

American Society of Retina Specialists Artificial Intelligence Task Force Report

Since the Artificial Intelligence Committee of the American Society of Retina Specialists developed the initial task force report in 2020, the artificial intelligence (AI) field has seen further adoption of US Food and Drug Administration-approved AI platforms and significant development of AI for various retinal conditions. With expansion of this technology comes further areas of challenges, including the data sources used in AI, the democracy of AI, commercialization, bias, and the need for provider education on the technology of AI. The overall focus of this committee report is to explore these recent issues as they relate to the continued development of AI and its integration into ophthalmology and retinal practice.

Vision-Language Models for Feature Detection of Macular Diseases on Optical Coherence Tomography

Importance

Vision-language models (VLMs) are a novel artificial intelligence technology capable of processing image and text inputs. While demonstrating strong generalist capabilities, their performance in ophthalmology has not been extensively studied.

Objective

To assess the performance of the Gemini Pro VLM in expert-level tasks for macular diseases from optical coherence tomography (OCT) scans.

Design, Setting, and Participants

This was a cross-sectional diagnostic accuracy study evaluating a generalist VLM on ophthalmology-specific tasks using the open-source Optical Coherence Tomography Image Database. The dataset included OCT B-scans from 50 unique patients: healthy individuals and those with macular hole, diabetic macular edema, central serous chorioretinopathy, and age-related macular degeneration. Each OCT scan was labeled for 10 key pathological features, referral recommendations, and treatments. The images were captured using a Cirrus high definition OCT machine (Carl Zeiss Meditec) at Sankara Nethralaya Eye Hospital, Chennai, India, and the dataset was published in December 2018. Image acquisition dates were not specified.

Exposures

Gemini Pro, using a standard prompt to extract structured responses on December 15, 2023.

Main Outcomes and Measures

The primary outcome was model responses compared against expert labels, calculating F1 scores for each pathological feature. Secondary outcomes included accuracy in diagnosis, referral urgency, and treatment recommendation. The model's internal concordance was evaluated by measuring the alignment between referral and treatment recommendations, independent of diagnostic accuracy.

Results

The mean F1 score was 10.7% (95% CI, 2.4-19.2). Measurable F1 scores were obtained for macular hole (36.4%; 95% CI, 0-71.4), pigment epithelial detachment (26.1%; 95% CI, 0-46.2), subretinal hyperreflective material (24.0%; 95% CI, 0-45.2), and subretinal fluid (20.0%; 95% CI, 0-45.5). A correct diagnosis was achieved in 17 of 50 cases (34%; 95% CI, 22-48). Referral recommendations varied: 28 of 50 were correct (56%; 95% CI, 42-70), 10 of 50 were overcautious (20%; 95% CI, 10-32), and 12 of 50 were undercautious (24%; 95% CI, 12-36). Referral and treatment concordance were very high, with 48 of 50 (96%; 95 % CI, 90-100) and 48 of 49 (98%; 95% CI, 94-100) correct answers, respectively.

Conclusions and Relevance

In this study, a generalist VLM demonstrated limited vision capabilities for feature detection and management of macular disease. However, it showed low self-contradiction, suggesting strong language capabilities. As VLMs continue to improve, validating their performance on large benchmarking datasets will help ascertain their potential in ophthalmology.

Machine Learning Analysis of Human Skin by Optoacoustic Mesoscopy for Automated Extraction of Psoriasis and Aging Biomarkers

Ultra-wideband raster-scan optoacoustic mesoscopy (RSOM) is a novel modality that has demonstrated unprecedented ability to visualize epidermal and dermal structures in-vivo. However, an automatic and quantitative analysis of three-dimensional RSOM datasets remains unexplored. In this work we present our framework: Deep Learning RSOM Analysis Pipeline (DeepRAP), to analyze and quantify morphological skin features recorded by RSOM and extract imaging biomarkers for disease characterization. DeepRAP uses a multi-network segmentation strategy based on convolutional neural networks with transfer learning. This strategy enabled the automatic recognition of skin layers and subsequent segmentation of dermal microvasculature with an accuracy equivalent to human assessment. DeepRAP was validated against manual segmentation on 25 psoriasis patients under treatment and our biomarker extraction was shown to characterize disease severity and progression well with a strong correlation to physician evaluation and histology. In a unique validation experiment, we applied DeepRAP in a time series sequence of occlusion-induced hyperemia from 10 healthy volunteers. We observe how the biomarkers decrease and recover during the occlusion and release process, demonstrating accurate performance and reproducibility of DeepRAP. Furthermore, we analyzed a cohort of 75 volunteers and defined a relationship between aging and microvascular features in-vivo. More precisely, this study revealed that fine microvascular features in the dermal layer have the strongest correlation to age. The ability of our newly developed framework to enable the rapid study of human skin morphology and microvasculature in-vivo promises to replace biopsy studies, increasing the translational potential of RSOM.

A comparative study of an on premise AutoML solution for medical image classification

Automated machine learning (AutoML) allows for the simplified application of machine learning to real-world problems, by the implicit handling of necessary steps such as data pre-processing, feature engineering, model selection and hyperparameter optimization. This has encouraged its use in medical applications such as imaging. However, the impact of common parameter choices such as the number of trials allowed, and the resolution of the input images, has not been comprehensively explored in existing literature. We therefore benchmark AutoKeras (AK), an open-source AutoML framework, against several bespoke deep learning architectures, on five public medical datasets representing a wide range of imaging modalities. It was found that AK could outperform the bespoke models in general, although at the cost of increased training time. Moreover, our experiments suggest that a large number of trials and higher resolutions may not be necessary for optimal performance to be achieved.

Automated machine learning model for fundus image classification by health-care professionals with no coding experience

To assess the feasibility of code-free deep learning (CFDL) platforms in the prediction of binary outcomes from fundus images in ophthalmology, evaluating two distinct online-based platforms (Google Vertex and Amazon Rekognition), and two distinct datasets. Two publicly available datasets, Messidor-2 and BRSET, were utilized for model development. The Messidor-2 consists of fundus photographs from diabetic patients and the BRSET is a multi-label dataset. The CFDL platforms were used to create deep learning models, with no preprocessing of the images, by a single ophthalmologist without coding expertise. The performance metrics employed to evaluate the models were F1 score, area under curve (AUC), precision and recall. The performance metrics for referable diabetic retinopathy and macular edema were above 0.9 for both tasks and CDFL. The Google Vertex models demonstrated superior performance compared to the Amazon models, with the BRSET dataset achieving the highest accuracy (AUC of 0.994). Multi-classification tasks using only BRSET achieved similar overall performance between platforms, achieving AUC of 0.994 for laterality, 0.942 for age grouping, 0.779 for genetic sex identification, 0.857 for optic, and 0.837 for normality with Google Vertex. The study demonstrates the feasibility of using automated machine learning platforms for predicting binary outcomes from fundus images in ophthalmology. It highlights the high accuracy achieved by the models in some tasks and the potential of CFDL as an entry-friendly platform for ophthalmologists to familiarize themselves with machine learning concepts.

Comparing code-free deep learning models to expert-designed models for detecting retinal diseases from optical coherence tomography

BACKGROUND

Code-free deep learning (CFDL) is a novel tool in artificial intelligence (AI). This study directly compared the discriminative performance of CFDL models designed by ophthalmologists without coding experience against bespoke models designed by AI experts in detecting retinal pathologies from optical coherence tomography (OCT) videos and fovea-centered images.

METHODS

Using the same internal dataset of 1,173 OCT macular videos and fovea-centered images, model development was performed simultaneously but independently by an ophthalmology resident (CFDL models) and a postdoctoral researcher with expertise in AI (bespoke models). We designed a multi-class model to categorize video and fovea-centered images into five labels: normal retina, macular hole, epiretinal membrane, wet age-related macular degeneration and diabetic macular edema. We qualitatively compared point estimates of the performance metrics of the CFDL and bespoke models.

RESULTS

For videos, the CFDL model demonstrated excellent discriminative performance, even outperforming the bespoke models for some metrics: area under the precision-recall curve was 0.984 (vs. 0.901), precision and sensitivity were both 94.1% (vs. 94.2%) and accuracy was 94.1% (vs. 96.7%). The fovea-centered CFDL model overall performed better than video-based model and was as accurate as the best bespoke model.

CONCLUSION

This comparative study demonstrated that code-free models created by clinicians without coding expertise perform as accurately as expert-designed bespoke models at classifying various retinal pathologies from OCT videos and images. CFDL represents a step forward towards the democratization of AI in medicine, although its numerous limitations must be carefully addressed to ensure its effective application in healthcare.

Leveraging code-free deep learning for pill recognition in clinical settings: A multicenter, real-world study of performance across multiple platforms

BACKGROUND

Preventable patient harm, particularly medication errors, represent significant challenges in healthcare settings. Dispensing the wrong medication is often associated with mix-up of lookalike and soundalike drugs in high workload environments. Replacing manual dispensing with automated unit dose and medication dispensing systems to reduce medication errors is not always feasible in clinical facilities experiencing high patient turn-around or frequent dose changes. Artificial intelligence (AI) based pill recognition tools and smartphone applications could potentially aid healthcare workers in identifying pills in situations where more advanced dispensing systems are not implemented.

OBJECTIVE

Most of the published research on pill recognition focuses on theoretical aspects of model development using traditional coding and deep learning methods. The use of code-free deep learning (CFDL) as a practical alternative for accessible model development, and implementation of such models in tools intended to aid decision making in clinical settings, remains largely unexplored. In this study, we sought to address this gap in existing literature by investigating whether CFDL is a viable approach for developing pill recognition models using a custom dataset, followed by a thorough evaluation of the model across various deployment scenarios, and in multicenter clinical settings. Furthermore, we aimed to highlight challenges and propose solutions to achieve optimal performance and real-world applicability of pill recognition models, including when deployed on smartphone applications.

METHODS

A pill recognition model was developed utilizing Microsoft Azure Custom Vision platform and a large custom training dataset of 26,880 images captured from the top 30 most dispensed solid oral dosage forms (SODFs) at the three participating hospitals. A comprehensive internal and external testing strategy was devised, model's performance was investigated through the online API, and offline using exported TensorFlow Lite model running on a Windows PC and on Android, using a tailor-made testing smartphone application. Additionally, model's calibration, degree of reliance on color features and device dependency was thoroughly evaluated. Real-world performance was assessed using images captured by hospital pharmacists at three participating clinical centers.

RESULTS

The pill recognition model showed high performance in Microsoft Azure Custom Vision platform with 98.7 % precision, 95.1 % recall, and 98.2 % mean average precision (mAP), with thresholds set to 50 %. During internal testing utilizing the online API, the model reached 93.7 % precision, 88.96 % recall, 90.81 % F1-score and 87.35 % mAP. Testing the offline TensorFlow Lite model on Windows PC showed a slight performance reduction, with 91.16 % precision, 83.82 % recall, 86.18 % F1-score and 82.55 % mAP. Performance of the model running offline on the Android application was further reduced to 86.50 % precision, 75.00 % recall, 77.83 % F1-score and 69.24 % mAP. During external clinical testing through the online API an overall precision of 83.10 %, recall of 71.39 %, and F1-score of 75.76 % was achieved.

CONCLUSION

Our study demonstrates that using a CFDL approach is a feasible and cost-effective method for developing AI-based pill recognition systems. Despite the limitations encountered, our model performed well, particularly when accessed through the online API. The use of CFDL facilitates interdisciplinary collaboration, resulting in human-centered AI models with enhanced real-world applicability. We suggest that rather than striving to build a universally applicable pill recognition system, models should be tailored to the medications in a regional formulary or needs of a specific clinic, which can in turn lead to improved performance in real-world deployment in these locations. Parallel to focusing on model development, it is crucial to employ a human centered approach by training the end users on how to properly interact with the AI based system to maximize benefits. Future research is needed on refining pill recognition models for broader adaptability. This includes investigating image pre-processing and optimization techniques to enhance offline performance and operation on handheld devices. Moreover, future studies should explore methods to overcome limitations of CFDL development to enhance the robustness of models and reduce overfitting. Collaborative efforts between researchers in this domain and sharing of best practices are vital to improve pill recognition systems, ultimately enhancing patient safety and healthcare outcomes.

Clinical performance of automated machine learning: A systematic review

Introduction

Automated machine learning (autoML) removes technical and technological barriers to building artificial intelligence models. We aimed to summarise the clinical applications of autoML, assess the capabilities of utilised platforms, evaluate the quality of the evidence trialling autoML, and gauge the performance of autoML platforms relative to conventionally developed models, as well as each other.

Method

This review adhered to a prospectively registered protocol (PROSPERO identifier CRD42022344427). The Cochrane Library, Embase, MEDLINE and Scopus were searched from inception to 11 July 2022. Two researchers screened abstracts and full texts, extracted data and conducted quality assessment. Disagreement was resolved through discussion and if required, arbitration by a third researcher.

Results

There were 26 distinct autoML platforms featured in 82 studies. Brain and lung disease were the most common fields of study of 22 specialties. AutoML exhibited variable performance: area under the receiver operator characteristic curve (AUCROC) 0.35-1.00, F1-score 0.16-0.99, area under the precision-recall curve (AUPRC) 0.51-1.00. AutoML exhibited the highest AUCROC in 75.6% trials; the highest F1-score in 42.3% trials; and the highest AUPRC in 83.3% trials. In autoML platform comparisons, AutoPrognosis and Amazon Rekognition performed strongest with unstructured and structured data, respectively. Quality of reporting was poor, with a median DECIDE-AI score of 14 of 27.

Conclusion

A myriad of autoML platforms have been applied in a variety of clinical contexts. The performance of autoML compares well to bespoke computational and clinical benchmarks. Further work is required to improve the quality of validation studies. AutoML may facilitate a transition to data-centric development, and integration with large language models may enable AI to build itself to fulfil user-defined goals.

A Deep Ensemble Dynamic Learning Network for Corona Virus Disease 2019 Diagnosis

Corona virus disease 2019 is an extremely fatal pandemic around the world. Intelligently recognizing X-ray chest radiography images for automatically identifying corona virus disease 2019 from other types of pneumonia and normal cases provides clinicians with tremendous conveniences in diagnosis process. In this article, a deep ensemble dynamic learning network is proposed. After a chain of image preprocessing steps and the division of image dataset, convolution blocks and the final average pooling layer are pretrained as a feature extractor. For classifying the extracted feature samples, two-stage bagging dynamic learning network is trained based on neural dynamic learning and bagging algorithms, which diagnoses the presence and types of pneumonia successively. Experimental results manifest that using the proposed deep ensemble dynamic learning network obtains 98.7179% diagnosis accuracy, which indicates more excellent diagnosis effect than existing state-of-the-art models on the open image dataset. Such accurate diagnosis effects provide convincing evidences for further detections and treatments.

Quantitative analysis of deep learning-based denoising model efficacy on optical coherence tomography images with different noise levels

BACKGROUND

To quantitatively evaluate the effectiveness of the Noise2Noise (N2N) model, a deep learning (DL)-based noise reduction algorithm, on enhanced depth imaging-optical coherence tomography (EDI-OCT) images with different noise levels.

METHODS

The study included 30 subfoveal EDI-OCT images averaged with 100 frames from 30 healthy participants. Artificial Gaussian noise at 25.00, 50.00, and 75.00 standard deviations were added to the averaged (original) images, and the images were grouped as 25N, 50N, and 75N. Afterward, noise-added images were denoised with the N2N model and grouped as 25dN, 50dN, and 75dN, according to previous noise levels. The choroidal vascularity index (CVI) and deep choroidal contrast-to-noise ratio (CNR) were calculated for all images, and noise-added and denoised images were compared with the original images. The structural similarity of the noise-added and denoised images to the original images was assessed by the Multi-Scale Structural Similarity Index (MS-SSI).

RESULTS

The CVI and CNR parameters of the original images (68.08 ± 2.47 %, and 9.71 ± 2.80) did not differ from the only 25dN images (67.97 ± 2.34 % and 8.50 ± 2.43) (p:1.000, and p:0.062, respectively). Noise reduction improved the MS-SSI at each noise level (p < 0.001). However, the highest MS-SSI was achieved in 25dN images.

CONCLUSIONS

The DL-based N2N denoising model can be used effectively for images with low noise levels, but at increasing noise levels, this model may be insufficient to provide both the original structural features of the choroid and structural similarity to the original image.

Revolutionizing tumor detection and classification in multimodality imaging based on deep learning approaches: Methods, applications and limitations

BACKGROUND

The emergence of deep learning (DL) techniques has revolutionized tumor detection and classification in medical imaging, with multimodal medical imaging (MMI) gaining recognition for its precision in diagnosis, treatment, and progression tracking.

OBJECTIVE

This review comprehensively examines DL methods in transforming tumor detection and classification across MMI modalities, aiming to provide insights into advancements, limitations, and key challenges for further progress.

METHODS

Systematic literature analysis identifies DL studies for tumor detection and classification, outlining methodologies including convolutional neural networks (CNNs), recurrent neural networks (RNNs), and their variants. Integration of multimodality imaging enhances accuracy and robustness.

RESULTS

Recent advancements in DL-based MMI evaluation methods are surveyed, focusing on tumor detection and classification tasks. Various DL approaches, including CNNs, YOLO, Siamese Networks, Fusion-Based Models, Attention-Based Models, and Generative Adversarial Networks, are discussed with emphasis on PET-MRI, PET-CT, and SPECT-CT.

FUTURE DIRECTIONS

The review outlines emerging trends and future directions in DL-based tumor analysis, aiming to guide researchers and clinicians toward more effective diagnosis and prognosis. Continued innovation and collaboration are stressed in this rapidly evolving domain.

CONCLUSION

Conclusions drawn from literature analysis underscore the efficacy of DL approaches in tumor detection and classification, highlighting their potential to address challenges in MMI analysis and their implications for clinical practice.

Potential for artificial intelligence in medicine and its application to male infertility

Background

The third AI boom, which began in 2010, has been characterized by the rapid evolution and diversification of AI and marked by the development of key technologies such as machine learning and deep learning. AI is revolutionizing the medical field, enhancing diagnostic accuracy, surgical outcomes, and drug production.

Methods

This review includes explanations of digital transformation (DX), the history of AI, the difference between machine learning and deep learning, recent AI topics, medical AI, and AI research in male infertility.

Main Findings Results

In research on male infertility, I established an AI-based prediction model for Johnsen scores and an AI predictive model for sperm retrieval in non-obstructive azoospermia, both by no-code AI.

Conclusions

AI is making constant progress. It would be ideal for physicians to acquire a knowledge of AI and even create AI models. No-code AI tools have revolutionized model creation, allowing individuals to independently handle data preparation and model development. Previously a team effort, this shift empowers users to craft customized AI models solo, offering greater flexibility and control in the model creation process.

Uncertainty-aware network for fine-grained and imbalanced reflux esophagitis grading

Computer-aided diagnosis (CAD) assists endoscopists in analyzing endoscopic images, reducing misdiagnosis rates and enabling timely treatment. A few studies have focused on CAD for gastroesophageal reflux disease, but CAD studies on reflux esophagitis (RE) are still inadequate. This paper presents a CAD study on RE using a dataset collected from hospital, comprising over 3000 images. We propose an uncertainty-aware network with handcrafted features, utilizing representation and classifier decoupling with metric learning to address class imbalance and achieve fine-grained RE classification. To enhance interpretability, the network estimates uncertainty through test time augmentation. The experimental results demonstrate that the proposed network surpasses previous methods, achieving an accuracy of 90.2% and an F1 score of 90.1%.

Clinician-Driven AI: Code-Free Self-Training on Public Data for Diabetic Retinopathy Referral

Importance

Democratizing artificial intelligence (AI) enables model development by clinicians with a lack of coding expertise, powerful computing resources, and large, well-labeled data sets.

Objective

To determine whether resource-constrained clinicians can use self-training via automated machine learning (ML) and public data sets to design high-performing diabetic retinopathy classification models.

Design, Setting, and Participants

This diagnostic quality improvement study was conducted from January 1, 2021, to December 31, 2021. A self-training method without coding was used on 2 public data sets with retinal images from patients in France (Messidor-2 [n = 1748]) and the UK and US (EyePACS [n = 58 689]) and externally validated on 1 data set with retinal images from patients of a private Egyptian medical retina clinic (Egypt [n = 210]). An AI model was trained to classify referable diabetic retinopathy as an exemplar use case. Messidor-2 images were assigned adjudicated labels available on Kaggle; 4 images were deemed ungradable and excluded, leaving 1744 images. A total of 300 images randomly selected from the EyePACS data set were independently relabeled by 3 blinded retina specialists using the International Classification of Diabetic Retinopathy protocol for diabetic retinopathy grade and diabetic macular edema presence; 19 images were deemed ungradable, leaving 281 images. Data analysis was performed from February 1 to February 28, 2021.

Exposures

Using public data sets, a teacher model was trained with labeled images using supervised learning. Next, the resulting predictions, termed pseudolabels, were used on an unlabeled public data set. Finally, a student model was trained with the existing labeled images and the additional pseudolabeled images.

Main Outcomes and Measures

The analyzed metrics for the models included the area under the receiver operating characteristic curve (AUROC), accuracy, sensitivity, specificity, and F1 score. The Fisher exact test was performed, and 2-tailed P values were calculated for failure case analysis.

Results

For the internal validation data sets, AUROC values for performance ranged from 0.886 to 0.939 for the teacher model and from 0.916 to 0.951 for the student model. For external validation of automated ML model performance, AUROC values and accuracy were 0.964 and 93.3% for the teacher model, 0.950 and 96.7% for the student model, and 0.890 and 94.3% for the manually coded bespoke model, respectively.

Conclusions and Relevance

These findings suggest that self-training using automated ML is an effective method to increase both model performance and generalizability while decreasing the need for costly expert labeling. This approach advances the democratization of AI by enabling clinicians without coding expertise or access to large, well-labeled private data sets to develop their own AI models.

Detection of dental restorations using no-code artificial intelligence

OBJECTIVES

The purpose of this study was to utilize a no-code computer vision platform to develop, train, and evaluate a model specifically designed for segmenting dental restorations on panoramic radiographs.

METHODS

One hundred anonymized panoramic radiographs were selected for this study. Accurate labeling of dental restorations was performed by calibrated dental faculty and students, with subsequent final review by an oral radiologist. The radiographs were automatically split within the platform into training (70%), development (20%), and testing (10%) subgroups. The model was trained for 40 epochs using a medium model size. Data augmentation techniques available within the platform, namely horizontal and vertical flip, were utilized on the training set to improve the model's predictions. Post-training, the model was tested for independent predictions. The model's diagnostic validity was assessed through the calculation of sensitivity, specificity, accuracy, precision, F1-score by pixel and by tooth, and by ROC-AUC.

RESULTS

A total of 1,108 restorations were labeled on 960 teeth. At a confidence threshold of 0.95, the model achieved 86.64% sensitivity, 99.78% specificity, 99.63% accuracy, 82.4% precision and an F1-score of 0.844 by pixel. The model achieved 98.34% sensitivity, 98.13% specificity, 98.21% accuracy, 98.85% precision and an F1-score of 0.98 by tooth. ROC curve showed high performance with an AUC of 0.978.

CONCLUSIONS

The no-code computer vision platform used in this study accurately detected dental restorations on panoramic radiographs. However, further research and validation are required to evaluate the performance of no-code platforms on larger and more diverse datasets, as well as for other detection and segmentation tasks.

CLINICAL SIGNIFICANCE

The advent of no-code computer vision holds significant promise in dentistry and dental research by eliminating the requirement for coding skills, democratizing access to artificial intelligence tools, and potentially revolutionizing dental diagnostics.

Democratizing Artificial Intelligence Imaging Analysis With Automated Machine Learning: Tutorial

Deep learning-based clinical imaging analysis underlies diagnostic artificial intelligence (AI) models, which can match or even exceed the performance of clinical experts, having the potential to revolutionize clinical practice. A wide variety of automated machine learning (autoML) platforms lower the technical barrier to entry to deep learning, extending AI capabilities to clinicians with limited technical expertise, and even autonomous foundation models such as multimodal large language models. Here, we provide a technical overview of autoML with descriptions of how autoML may be applied in education, research, and clinical practice. Each stage of the process of conducting an autoML project is outlined, with an emphasis on ethical and technical best practices. Specifically, data acquisition, data partitioning, model training, model validation, analysis, and model deployment are considered. The strengths and limitations of available code-free, code-minimal, and code-intensive autoML platforms are considered. AutoML has great potential to democratize AI in medicine, improving AI literacy by enabling "hands-on" education. AutoML may serve as a useful adjunct in research by facilitating rapid testing and benchmarking before significant computational resources are committed. AutoML may also be applied in clinical contexts, provided regulatory requirements are met. The abstraction by autoML of arduous aspects of AI engineering promotes prioritization of data set curation, supporting the transition from conventional model-driven approaches to data-centric development. To fulfill its potential, clinicians must be educated on how to apply these technologies ethically, rigorously, and effectively; this tutorial represents a comprehensive summary of relevant considerations.

OCT-based deep-learning models for the identification of retinal key signs

A new system based on binary Deep Learning (DL) convolutional neural networks has been developed to recognize specific retinal abnormality signs on Optical Coherence Tomography (OCT) images useful for clinical practice. Images from the local hospital database were retrospectively selected from 2017 to 2022. Images were labeled by two retinal specialists and included central fovea cross-section OCTs. Nine models were developed using the Visual Geometry Group 16 architecture to distinguish healthy versus abnormal retinas and to identify eight different retinal abnormality signs. A total of 21,500 OCT images were screened, and 10,770 central fovea cross-section OCTs were included in the study. The system achieved high accuracy in identifying healthy retinas and specific pathological signs, ranging from 93 to 99%. Accurately detecting abnormal retinal signs from OCT images is crucial for patient care. This study aimed to identify specific signs related to retinal pathologies, aiding ophthalmologists in diagnosis. The high-accuracy system identified healthy retinas and pathological signs, making it a useful diagnostic aid. Labelled OCT images remain a challenge, but our approach reduces dataset creation time and shows DL models' potential to improve ocular pathology diagnosis and clinical decision-making.

MLP-Like Model With Convolution Complex Transformation for Auxiliary Diagnosis Through Medical Images

Medical images such as facial and tongue images have been widely used for intelligence-assisted diagnosis, which can be regarded as the multi-label classification task for disease location (DL) and disease nature (DN) of biomedical images. Compared with complicated convolutional neural networks and Transformers for this task, recent MLP-like architectures are not only simple and less computationally expensive, but also have stronger generalization capabilities. However, MLP-like models require better input features from the image. Thus, this study proposes a novel convolution complex transformation MLP-like (CCT-MLP) model for the multi-label DL and DN recognition task for facial and tongue images. Notably, the convolutional Tokenizer and multiple convolutional layers are first used to extract the better shallow features from input biomedical images to make up for the loss of spatial information obtained by the simple MLP structure. Subsequently, the Channel-MLP architecture with complex transformations is used to extract deep-level contextual features. In this way, multi-channel features are extracted and mixed to perform the multi-label classification of the input biomedical images. Experimental results on our constructed multi-label facial and tongue image datasets demonstrate that our method outperforms existing methods in terms of both accuracy (Acc) and mean average precision (mAP).

Performance of Automated Machine Learning for Diabetic Retinopathy Image Classification from Multi-field Handheld Retinal Images

PURPOSE

To create and validate code-free automated deep learning models (AutoML) for diabetic retinopathy (DR) classification from handheld retinal images.

DESIGN

Prospective development and validation of AutoML models for DR image classification.

PARTICIPANTS

A total of 17 829 deidentified retinal images from 3566 eyes with diabetes, acquired using handheld retinal cameras in a community-based DR screening program.

METHODS

AutoML models were generated based on previously acquired 5-field (macula-centered, disc-centered, superior, inferior, and temporal macula) handheld retinal images. Each individual image was labeled using the International DR and diabetic macular edema (DME) Classification Scale by 4 certified graders at a centralized reading center under oversight by a senior retina specialist. Images for model development were split 8-1-1 for training, optimization, and testing to detect referable DR ([refDR], defined as moderate nonproliferative DR or worse or any level of DME). Internal validation was performed using a published image set from the same patient population (N = 450 images from 225 eyes). External validation was performed using a publicly available retinal imaging data set from the Asia Pacific Tele-Ophthalmology Society (N = 3662 images).

MAIN OUTCOME MEASURES

Area under the precision-recall curve (AUPRC), sensitivity (SN), specificity (SP), positive predictive value (PPV), negative predictive value (NPV), accuracy, and F1 scores.

RESULTS

Referable DR was present in 17.3%, 39.1%, and 48.0% of the training set, internal validation, and external validation sets, respectively. The model's AUPRC was 0.995 with a precision and recall of 97% using a score threshold of 0.5. Internal validation showed that SN, SP, PPV, NPV, accuracy, and F1 scores were 0.96 (95% confidence interval [CI], 0.884-0.99), 0.98 (95% CI, 0.937-0.995), 0.96 (95% CI, 0.884-0.99), 0.98 (95% CI, 0.937-0.995), 0.97, and 0.96, respectively. External validation showed that SN, SP, PPV, NPV, accuracy, and F1 scores were 0.94 (95% CI, 0.929-0.951), 0.97 (95% CI, 0.957-0.974), 0.96 (95% CI, 0.952-0.971), 0.95 (95% CI, 0.935-0.956), 0.97, and 0.96, respectively.

CONCLUSIONS

This study demonstrates the accuracy and feasibility of code-free AutoML models for identifying refDR developed using handheld retinal imaging in a community-based screening program. Potentially, the use of AutoML may increase access to machine learning models that may be adapted for specific programs that are guided by the clinical need to rapidly address disparities in health care delivery.

FINANCIAL DISCLOSURE(S)

Proprietary or commercial disclosure may be found after the references.

Crowd-Sourced Deep Learning for Intracranial Hemorrhage Identification: Wisdom of Crowds or Laissez-Faire

BACKGROUND AND PURPOSE

Researchers and clinical radiology practices are increasingly faced with the task of selecting the most accurate artificial intelligence tools from an ever-expanding range. In this study, we sought to test the utility of ensemble learning for determining the best combination from 70 models trained to identify intracranial hemorrhage. Furthermore, we investigated whether ensemble deployment is preferred to use of the single best model. It was hypothesized that any individual model in the ensemble would be outperformed by the ensemble.

MATERIALS AND METHODS

In this retrospective study, de-identified clinical head CT scans from 134 patients were included. Every section was annotated with "no-intracranial hemorrhage" or "intracranial hemorrhage," and 70 convolutional neural networks were used for their identification. Four ensemble learning methods were researched, and their accuracies as well as receiver operating characteristic curves and the corresponding areas under the curve were compared with those of individual convolutional neural networks. The areas under the curve were compared for a statistical difference using a generalized U-statistic.

RESULTS

The individual convolutional neural networks had an average test accuracy of 67.8% (range, 59.4%-76.0%). Three ensemble learning methods outperformed this average test accuracy, but only one achieved an accuracy above the 95th percentile of the individual convolutional neural network accuracy distribution. Only 1 ensemble learning method achieved a similar area under the curve as the single best convolutional neural network (Δarea under the curve = 0.03; 95% CI, -0.01-0.06; P = .17).

CONCLUSIONS

None of the ensemble learning methods outperformed the accuracy of the single best convolutional neural network, at least in the context of intracranial hemorrhage detection.

Development and international validation of custom-engineered and code-free deep-learning models for detection of plus disease in retinopathy of prematurity: a retrospective study

BACKGROUND

Retinopathy of prematurity (ROP), a leading cause of childhood blindness, is diagnosed through interval screening by paediatric ophthalmologists. However, improved survival of premature neonates coupled with a scarcity of available experts has raised concerns about the sustainability of this approach. We aimed to develop bespoke and code-free deep learning-based classifiers for plus disease, a hallmark of ROP, in an ethnically diverse population in London, UK, and externally validate them in ethnically, geographically, and socioeconomically diverse populations in four countries and three continents. Code-free deep learning is not reliant on the availability of expertly trained data scientists, thus being of particular potential benefit for low resource health-care settings.

METHODS

This retrospective cohort study used retinal images from 1370 neonates admitted to a neonatal unit at Homerton University Hospital NHS Foundation Trust, London, UK, between 2008 and 2018. Images were acquired using a Retcam Version 2 device (Natus Medical, Pleasanton, CA, USA) on all babies who were either born at less than 32 weeks gestational age or had a birthweight of less than 1501 g. Each images was graded by two junior ophthalmologists with disagreements adjudicated by a senior paediatric ophthalmologist. Bespoke and code-free deep learning models (CFDL) were developed for the discrimination of healthy, pre-plus disease, and plus disease. Performance was assessed internally on 200 images with the majority vote of three senior paediatric ophthalmologists as the reference standard. External validation was on 338 retinal images from four separate datasets from the USA, Brazil, and Egypt with images derived from Retcam and the 3nethra neo device (Forus Health, Bengaluru, India).

FINDINGS

Of the 7414 retinal images in the original dataset, 6141 images were used in the final development dataset. For the discrimination of healthy versus pre-plus or plus disease, the bespoke model had an area under the curve (AUC) of 0·986 (95% CI 0·973-0·996) and the CFDL model had an AUC of 0·989 (0·979-0·997) on the internal test set. Both models generalised well to external validation test sets acquired using the Retcam for discriminating healthy from pre-plus or plus disease (bespoke range was 0·975-1·000 and CFDL range was 0·969-0·995). The CFDL model was inferior to the bespoke model on discriminating pre-plus disease from healthy or plus disease in the USA dataset (CFDL 0·808 [95% CI 0·671-0·909, bespoke 0·942 [0·892-0·982]], p=0·0070). Performance also reduced when tested on the 3nethra neo imaging device (CFDL 0·865 [0·742-0·965] and bespoke 0·891 [0·783-0·977]).

INTERPRETATION

Both bespoke and CFDL models conferred similar performance to senior paediatric ophthalmologists for discriminating healthy retinal images from ones with features of pre-plus or plus disease; however, CFDL models might generalise less well when considering minority classes. Care should be taken when testing on data acquired using alternative imaging devices from that used for the development dataset. Our study justifies further validation of plus disease classifiers in ROP screening and supports a potential role for code-free approaches to help prevent blindness in vulnerable neonates.

FUNDING

National Institute for Health Research Biomedical Research Centre based at Moorfields Eye Hospital NHS Foundation Trust and the University College London Institute of Ophthalmology.

TRANSLATIONS

For the Portuguese and Arabic translations of the abstract see Supplementary Materials section.

Self-Curable Synaptic Ferroelectric FET Arrays for Neuromorphic Convolutional Neural Network

With the recently increasing prevalence of deep learning, both academia and industry exhibit substantial interest in neuromorphic computing, which mimics the functional and structural features of the human brain. To realize neuromorphic computing, an energy-efficient and reliable artificial synapse must be developed. In this study, the synaptic ferroelectric field-effect-transistor (FeFET) array is fabricated as a component of a neuromorphic convolutional neural network. Beyond the single transistor level, the long-term potentiation and depression of synaptic weights are achieved at the array level, and a successful program-inhibiting operation is demonstrated in the synaptic array, achieving a learning accuracy of 79.84% on the Canadian Institute for Advanced Research (CIFAR)-10 dataset. Furthermore, an efficient self-curing method is proposed to improve the endurance of the FeFET array by tenfold, utilizing the punch-through current inherent to the device. Low-frequency noise spectroscopy is employed to quantitatively evaluate the curing efficiency of the proposed self-curing method. The results of this study provide a method to fabricate and operate reliable synaptic FeFET arrays, thereby paving the way for further development of ferroelectric-based neuromorphic computing.

Comparison of Gray-scale Inversion to Improve Detection of Pulmonary Nodules on Chest X-rays Between Radiologists and a Deep Convolutional Neural Network

Detection of pulmonary nodules on chest x-rays is an important task for radiologists. Previous studies have shown improved detection rates using gray-scale inversion. The purpose of our study was to compare the efficacy of gray-scale inversion in improving the detection of pulmonary nodules on chest x-rays for radiologists and machine learning models (ML). We created a mixed dataset consisting of 60, 2-view (posteroanterior view - PA and lateral view) chest x-rays with computed tomography confirmed nodule(s) and 62 normal chest x-rays. Twenty percent of the cases were separated for a testing dataset (24 total images). Data augmentation through mirroring and transfer learning was used for the remaining cases (784 total images) for supervised training of 4 ML models (grayscale PA, grayscale lateral, gray-scale inversion PA, and gray-scale inversion lateral) on Google's cloud-based AutoML platform. Three cardiothoracic radiologists analyzed the complete 2-view dataset (n=120) and, for comparison to the ML, the single-view testing subsets (12 images each). Gray-scale inversion (area under the curve (AUC) 0.80, 95% confidence interval (CI) 0.75-0.85) did not improve diagnostic performance for radiologists compared to grayscale (AUC 0.84, 95% CI 0.79-0.88). Gray-scale inversion also did not improve diagnostic performance for the ML. The ML did demonstrate higher sensitivity and negative predictive value for grayscale PA (72.7% and 75.0%), grayscale lateral (63.6% and 66.6%), and gray-scale inversion lateral views (72.7% and 76.9%), comparing favorably to the radiologists (63.9% and 72.3%, 27.8% and 58.3%, 19.5% and 50.5% respectively). In the limited testing dataset, the ML did demonstrate higher sensitivity and negative predictive value for grayscale PA (72.7% and 75.0%), grayscale lateral (63.6% and 66.6%), and gray-scale inversion lateral views (72.7% and 76.9%), comparing favorably to the radiologists (63.9% and 72.3%, 27.8% and 58.3%, 19.5% and 50.5%, respectively). Further investigation of other post-processing algorithms to improve diagnostic performance of ML is warranted.

Code-free machine learning for classification of central nervous system histopathology images

Machine learning (ML), an application of artificial intelligence, is currently transforming the analysis of biomedical data and specifically of biomedical images including histopathology. The promises of this technology contrast, however, with its currently limited application in routine clinical practice. This discrepancy is in part due to the extent of informatics expertise typically required for implementation of ML. Therefore, we assessed the suitability of 2 publicly accessible code-free ML platforms (Microsoft Custom Vision and Google AutoML), for classification of histopathological images of diagnostic central nervous system tissue samples. When trained with typically 100 to more than 1000 images, both systems were able to perform nontrivial classifications (glioma vs brain metastasis; astrocytoma vs astrocytosis, prediction of 1p/19q co-deletion in IDH-mutant tumors) based on hematoxylin and eosin-stained images with high accuracy (from ∼80% to nearly 100%). External validation of the predicted accuracy and negative control experiments were found to be crucial for verification of the accuracy predicted by the algorithms. Furthermore, we propose a possible diagnostic workflow for pathologists to implement classification of histopathological images based on code-free machine platforms.

Ultrasound-Based Machine Learning Approach for Detection of Nonalcoholic Fatty Liver Disease

OBJECTIVES

Current diagnosis of nonalcoholic fatty liver disease (NAFLD) relies on biopsy or MR-based fat quantification. This prospective study explored the use of ultrasound with artificial intelligence for the detection of NAFLD.

METHODS

One hundred and twenty subjects with clinical suspicion of NAFLD and 10 healthy volunteers consented to participate in this institutional review board-approved study. Subjects were categorized as NAFLD and non-NAFLD according to MR proton density fat fraction (PDFF) findings. Ultrasound images from 10 different locations in the right and left hepatic lobes were collected following a standard protocol. MRI-based liver fat quantification was used as the reference standard with >6.4% indicative of NAFLD. A supervised machine learning model was developed for assessment of NAFLD. To validate model performance, a balanced testing dataset of 24 subjects was used. Sensitivity, specificity, positive predictive value, negative predictive value, and overall accuracy with 95% confidence interval were calculated.

RESULTS

A total of 1119 images from 106 participants was used for model development. The internal evaluation achieved an average precision of 0.941, recall of 88.2%, and precision of 89.0%. In the testing set AutoML achieved a sensitivity of 72.2% (63.1%-80.1%), specificity of 94.6% (88.7%-98.0%), positive predictive value (PPV) of 93.1% (86.0%-96.7%), negative predictive value of 77.3% (71.6%-82.1%), and accuracy of 83.4% (77.9%-88.0%). The average agreement for an individual subject was 92%.

CONCLUSIONS

An ultrasound-based machine learning model for identification of NAFLD showed high specificity and PPV in this prospective trial. This approach may in the future be used as an inexpensive and noninvasive screening tool for identifying NAFLD in high-risk patients.

Artificial intelligence model for analyzing colonic endoscopy images to detect changes associated with irritable bowel syndrome

IBS is not considered to be an organic disease and usually shows no abnormality on lower gastrointestinal endoscopy, although biofilm formation, dysbiosis, and histological microinflammation have recently been reported in patients with IBS. In this study, we investigated whether an artificial intelligence (AI) colorectal image model can identify minute endoscopic changes, which cannot typically be detected by human investigators, that are associated with IBS. Study subjects were identified based on electronic medical records and categorized as IBS (Group I; n = 11), IBS with predominant constipation (IBS-C; Group C; n = 12), and IBS with predominant diarrhea (IBS-D; Group D; n = 12). The study subjects had no other diseases. Colonoscopy images from IBS patients and from asymptomatic healthy subjects (Group N; n = 88) were obtained. Google Cloud Platform AutoML Vision (single-label classification) was used to construct AI image models to calculate sensitivity, specificity, predictive value, and AUC. A total of 2479, 382, 538, and 484 images were randomly selected for Groups N, I, C and D, respectively. The AUC of the model discriminating between Group N and I was 0.95. Sensitivity, specificity, positive predictive value, and negative predictive value of Group I detection were 30.8%, 97.6%, 66.7%, and 90.2%, respectively. The overall AUC of the model discriminating between Groups N, C, and D was 0.83; sensitivity, specificity, and positive predictive value of Group N were 87.5%, 46.2%, and 79.9%, respectively. Using the image AI model, colonoscopy images of IBS could be discriminated from healthy subjects at AUC 0.95. Prospective studies are needed to further validate whether this externally validated model has similar diagnostic capabilities at other facilities and whether it can be used to determine treatment efficacy.

A Neural Network for Automated Image Quality Assessment of Optic Disc Photographs

This study describes the development of a convolutional neural network (CNN) for automated assessment of optic disc photograph quality. Using a code-free deep learning platform, a total of 2377 optic disc photographs were used to develop a deep CNN capable of determining optic disc photograph quality. Of these, 1002 were good-quality images, 609 were acceptable-quality, and 766 were poor-quality images. The dataset was split 80/10/10 into training, validation, and test sets and balanced for quality. A ternary classification model (good, acceptable, and poor quality) and a binary model (usable, unusable) were developed. In the ternary classification system, the model had an overall accuracy of 91% and an AUC of 0.98. The model had higher predictive accuracy for images of good (93%) and poor quality (96%) than for images of acceptable quality (91%). The binary model performed with an overall accuracy of 98% and an AUC of 0.99. When validated on 292 images not included in the original training/validation/test dataset, the model's accuracy was 85% on the three-class classification task and 97% on the binary classification task. The proposed system for automated image-quality assessment for optic disc photographs achieves high accuracy in both ternary and binary classification systems, and highlights the success achievable with a code-free platform. There is wide clinical and research potential for such a model, with potential applications ranging from integration into fundus camera software to provide immediate feedback to ophthalmic photographers, to prescreening large databases before their use in research.

Non-coding deep learning models for tomato biotic and abiotic stress classification using microscopic images

Plant disease classification is quite complex and, in most cases, requires trained plant pathologists and sophisticated labs to accurately determine the cause. Our group for the first time used microscopic images (×30) of tomato plant diseases, for which representative plant samples were diagnostically validated to classify disease symptoms using non-coding deep learning platforms (NCDL). The mean F1 scores (SD) of the NCDL platforms were 98.5 (1.6) for Amazon Rekognition Custom Label, 93.9 (2.5) for Clarifai, 91.6 (3.9) for Teachable Machine, 95.0 (1.9) for Google AutoML Vision, and 97.5 (2.7) for Microsoft Azure Custom Vision. The accuracy of the NCDL platform for Amazon Rekognition Custom Label was 99.8% (0.2), for Clarifai 98.7% (0.5), for Teachable Machine 98.3% (0.4), for Google AutoML Vision 98.9% (0.6), and for Apple CreateML 87.3 (4.3). Upon external validation, the model's accuracy of the tested NCDL platforms dropped no more than 7%. The potential future use for these models includes the development of mobile- and web-based applications for the classification of plant diseases and integration with a disease management advisory system. The NCDL models also have the potential to improve the early triage of symptomatic plant samples into classes that may save time in diagnostic lab sample processing.

Automatic detection of glaucoma via fundus imaging and artificial intelligence: A review

Glaucoma is a leading cause of irreversible vision impairment globally, and cases are continuously rising worldwide. Early detection is crucial, allowing timely intervention that can prevent further visual field loss. To detect glaucoma an examination of the optic nerve head via fundus imaging can be performed, at the center of which is the assessment of the optic cup and disc boundaries. Fundus imaging is noninvasive and low-cost; however, image examination relies on subjective, time-consuming, and costly expert assessments. A timely question to ask is: "Can artificial intelligence mimic glaucoma assessments made by experts?" Specifically, can artificial intelligence automatically find the boundaries of the optic cup and disc (providing a so-called segmented fundus image) and then use the segmented image to identify glaucoma with high accuracy? We conducted a comprehensive review on artificial intelligence-enabled glaucoma detection frameworks that produce and use segmented fundus images and summarized the advantages and disadvantages of such frameworks. We identified 36 relevant papers from 2011 to 2021 and 2 main approaches: 1) logical rule-based frameworks, based on a set of rules; and 2) machine learning/statistical modeling-based frameworks. We critically evaluated the state-of-art of the 2 approaches, identified gaps in the literature and pointed at areas for future research.

Deep learning for ultra-widefield imaging: a scoping review

PURPOSE

This article is a scoping review of published and peer-reviewed articles using deep-learning (DL) applied to ultra-widefield (UWF) imaging. This study provides an overview of the published uses of DL and UWF imaging for the detection of ophthalmic and systemic diseases, generative image synthesis, quality assessment of images, and segmentation and localization of ophthalmic image features.

METHODS

A literature search was performed up to August 31st, 2021 using PubMed, Embase, Cochrane Library, and Google Scholar. The inclusion criteria were as follows: (1) deep learning, (2) ultra-widefield imaging. The exclusion criteria were as follows: (1) articles published in any language other than English, (2) articles not peer-reviewed (usually preprints), (3) no full-text availability, (4) articles using machine learning algorithms other than deep learning. No study design was excluded from consideration.

RESULTS

A total of 36 studies were included. Twenty-three studies discussed ophthalmic disease detection and classification, 5 discussed segmentation and localization of ultra-widefield images (UWFIs), 3 discussed generative image synthesis, 3 discussed ophthalmic image quality assessment, and 2 discussed detecting systemic diseases via UWF imaging.

CONCLUSION

The application of DL to UWF imaging has demonstrated significant effectiveness in the diagnosis and detection of ophthalmic diseases including diabetic retinopathy, retinal detachment, and glaucoma. DL has also been applied in the generation of synthetic ophthalmic images. This scoping review highlights and discusses the current uses of DL with UWF imaging, and the future of DL applications in this field.

Feasibility of the Machine Learning Network to Diagnose Tympanic Membrane Lesions without Coding Experience

A machine learning platform operated without coding knowledge (Teachable machine^®) has been introduced. The aims of the present study were to assess the performance of the Teachable machine^® for diagnosing tympanic membrane lesions. A total of 3024 tympanic membrane images were used to train and validate the diagnostic performance of the network. Tympanic membrane images were labeled as normal, otitis media with effusion (OME), chronic otitis media (COM), and cholesteatoma. According to the complexity of the categorization, Level I refers to normal versus abnormal tympanic membrane; Level II was defined as normal, OME, or COM + cholesteatoma; and Level III distinguishes between all four pathologies. In addition, eighty representative test images were used to assess the performance. Teachable machine^® automatically creates a classification network and presents diagnostic performance when images are uploaded. The mean accuracy of the Teachable machine^® for classifying tympanic membranes as normal or abnormal (Level I) was 90.1%. For Level II, the mean accuracy was 89.0% and for Level III it was 86.2%. The overall accuracy of the classification of the 80 representative tympanic membrane images was 78.75%, and the hit rates for normal, OME, COM, and cholesteatoma were 95.0%, 70.0%, 90.0%, and 60.0%, respectively. Teachable machine^® could successfully generate the diagnostic network for classifying tympanic membrane.

Machine Learning and Deep Learning in Cardiothoracic Imaging: A Scoping Review

Machine-learning (ML) and deep-learning (DL) algorithms are part of a group of modeling algorithms that grasp the hidden patterns in data based on a training process, enabling them to extract complex information from the input data. In the past decade, these algorithms have been increasingly used for image processing, specifically in the medical domain. Cardiothoracic imaging is one of the early adopters of ML/DL research, and the COVID-19 pandemic resulted in more research focus on the feasibility and applications of ML/DL in cardiothoracic imaging. In this scoping review, we systematically searched available peer-reviewed medical literature on cardiothoracic imaging and quantitatively extracted key data elements in order to get a big picture of how ML/DL have been used in the rapidly evolving cardiothoracic imaging field. During this report, we provide insights on different applications of ML/DL and some nuances pertaining to this specific field of research. Finally, we provide general suggestions on how researchers can make their research more than just a proof-of-concept and move toward clinical adoption.

Automated detection of pain levels using deep feature extraction from shutter blinds-based dynamic-sized horizontal patches with facial images

Pain intensity classification using facial images is a challenging problem in computer vision research. This work proposed a patch and transfer learning-based model to classify various pain intensities using facial images. The input facial images were segmented into dynamic-sized horizontal patches or "shutter blinds". A lightweight deep network DarkNet19 pre-trained on ImageNet1K was used to generate deep features from the shutter blinds and the undivided resized segmented input facial image. The most discriminative features were selected from these deep features using iterative neighborhood component analysis, which were then fed to a standard shallow fine k-nearest neighbor classifier for classification using tenfold cross-validation. The proposed shutter blinds-based model was trained and tested on datasets derived from two public databases-University of Northern British Columbia-McMaster Shoulder Pain Expression Archive Database and Denver Intensity of Spontaneous Facial Action Database-which both comprised four pain intensity classes that had been labeled by human experts using validated facial action coding system methodology. Our shutter blinds-based classification model attained more than 95% overall accuracy rates on both datasets. The excellent performance suggests that the automated pain intensity classification model can be deployed to assist doctors in the non-verbal detection of pain using facial images in various situations (e.g., non-communicative patients or during surgery). This system can facilitate timely detection and management of pain.