Automated Diagnosis of Plus Disease in Retinopathy of Prematurity Using Deep Convolutional Neural Networks

Detection of Optic Disc Abnormalities in Color Fundus Photographs Using Deep Learning

BACKGROUND

To date, deep learning-based detection of optic disc abnormalities in color fundus photographs has mostly been limited to the field of glaucoma. However, many life-threatening systemic and neurological conditions can manifest as optic disc abnormalities. In this study, we aimed to extend the application of deep learning (DL) in optic disc analyses to detect a spectrum of nonglaucomatous optic neuropathies.

METHODS

Using transfer learning, we trained a ResNet-152 deep convolutional neural network (DCNN) to distinguish between normal and abnormal optic discs in color fundus photographs (CFPs). Our training data set included 944 deidentified CFPs (abnormal 364; normal 580). Our testing data set included 151 deidentified CFPs (abnormal 71; normal 80). Both the training and testing data sets contained a wide range of optic disc abnormalities, including but not limited to ischemic optic neuropathy, atrophy, compressive optic neuropathy, hereditary optic neuropathy, hypoplasia, papilledema, and toxic optic neuropathy. The standard measures of performance (sensitivity, specificity, and area under the curve of the receiver operating characteristic curve (AUC-ROC)) were used for evaluation.

RESULTS

During the 10-fold cross-validation test, our DCNN for distinguishing between normal and abnormal optic discs achieved the following mean performance: AUC-ROC 0.99 (95 CI: 0.98-0.99), sensitivity 94% (95 CI: 91%-97%), and specificity 96% (95 CI: 93%-99%). When evaluated against the external testing data set, our model achieved the following mean performance: AUC-ROC 0.87, sensitivity 90%, and specificity 69%.

CONCLUSION

In summary, we have developed a deep learning algorithm that is capable of detecting a spectrum of optic disc abnormalities in color fundus photographs, with a focus on neuro-ophthalmological etiologies. As the next step, we plan to validate our algorithm prospectively as a focused screening tool in the emergency department, which if successful could be beneficial because current practice pattern and training predict a shortage of neuro-ophthalmologists and ophthalmologists in general in the near future.

Artificial intelligence in myopia: current and future trends

PURPOSE OF REVIEW

Myopia is one of the leading causes of visual impairment, with a projected increase in prevalence globally. One potential approach to address myopia and its complications is early detection and treatment. However, current healthcare systems may not be able to cope with the growing burden. Digital technological solutions such as artificial intelligence (AI) have emerged as a potential adjunct for myopia management.

RECENT FINDINGS

There are currently four significant domains of AI in myopia, including machine learning (ML), deep learning (DL), genetics and natural language processing (NLP). ML has been demonstrated to be a useful adjunctive for myopia prediction and biometry for cataract surgery in highly myopic individuals. DL techniques, particularly convoluted neural networks, have been applied to various image-related diagnostic and predictive solutions. Applications of AI in genomics and NLP appear to be at a nascent stage.

SUMMARY

Current AI research is mainly focused on disease classification and prediction in myopia. Through greater collaborative research, we envision AI will play an increasingly critical role in big data analysis by aggregating a greater variety of parameters including genomics and environmental factors. This may enable the development of generalizable adjunctive DL systems that could help realize predictive and individualized precision medicine for myopic patients.

Generative adversarial networks in ophthalmology: what are these and how can they be used?

PURPOSE OF REVIEW

The development of deep learning (DL) systems requires a large amount of data, which may be limited by costs, protection of patient information and low prevalence of some conditions. Recent developments in artificial intelligence techniques have provided an innovative alternative to this challenge via the synthesis of biomedical images within a DL framework known as generative adversarial networks (GANs). This paper aims to introduce how GANs can be deployed for image synthesis in ophthalmology and to discuss the potential applications of GANs-produced images.

RECENT FINDINGS

Image synthesis is the most relevant function of GANs to the medical field, and it has been widely used for generating 'new' medical images of various modalities. In ophthalmology, GANs have mainly been utilized for augmenting classification and predictive tasks, by synthesizing fundus images and optical coherence tomography images with and without pathologies such as age-related macular degeneration and diabetic retinopathy. Despite their ability to generate high-resolution images, the development of GANs remains data intensive, and there is a lack of consensus on how best to evaluate the outputs produced by GANs.

SUMMARY

Although the problem of artificial biomedical data generation is of great interest, image synthesis by GANs represents an innovation with yet unclear relevance for ophthalmology.

The Impact of Artificial Intelligence and Deep Learning in Eye Diseases: A Review

Artificial intelligence (AI) is a subset of computer science dealing with the development and training of algorithms that try to replicate human intelligence. We report a clinical overview of the basic principles of AI that are fundamental to appreciating its application to ophthalmology practice. Here, we review the most common eye diseases, focusing on some of the potential challenges and limitations emerging with the development and application of this new technology into ophthalmology.

Teleophthalmology and Artificial Intelligence As Game Changers in Ophthalmic Care After the COVID-19 Pandemic

The current COVID-19 pandemic has boosted a sudden demand for telemedicine due to quarantine and travel restrictions. The exponential increase in the use of telemedicine is expected to affect ophthalmology drastically. The aim of this review is to discuss the utility, effectiveness and challenges of teleophthalmological new tools for eyecare delivery as well as its implementation and possible facilitation with artificial intelligence. We used the terms: “teleophthalmology,” “telemedicine and COVID-19,” “retinal diseases and telemedicine,” “virtual ophthalmology,” “cost effectiveness of teleophthalmology,” “pediatric teleophthalmology,” “Artificial intelligence and ophthalmology,” “Glaucoma and teleophthalmology” and “teleophthalmology limitations” in the database of PubMed and selected the articles being published in the course of 2015-2020. After the initial search, 321 articles returned as relevant. A meticulous screening followed and eventually 103 published manuscripts were included and used as our references. Emerging in the market, teleophthalmology is showing great potential for the future of ophthalmological care, benefiting both patients and ophthalmologists in times of pandemics. The spectrum of eye diseases that could benefit from teleophthalmology is wide, including mostly retinal diseases such as diabetic retinopathy, retinopathy of prematurity, age-related macular degeneration but also glaucoma and anterior segment conditions. Simultaneously, artificial intelligence provides ways of implementing teleophthalmology easier and with better outcomes, contributing as significant changing factors for ophthalmology practice after the COVID-19 pandemic.

Automatic detection of 39 fundus diseases and conditions in retinal photographs using deep neural networks

Retinal fundus diseases can lead to irreversible visual impairment without timely diagnoses and appropriate treatments. Single disease-based deep learning algorithms had been developed for the detection of diabetic retinopathy, age-related macular degeneration, and glaucoma. Here, we developed a deep learning platform (DLP) capable of detecting multiple common referable fundus diseases and conditions (39 classes) by using 249,620 fundus images marked with 275,543 labels from heterogenous sources. Our DLP achieved a frequency-weighted average F1 score of 0.923, sensitivity of 0.978, specificity of 0.996 and area under the receiver operating characteristic curve (AUC) of 0.9984 for multi-label classification in the primary test dataset and reached the average level of retina specialists. External multihospital test, public data test and tele-reading application also showed high efficiency for multiple retinal diseases and conditions detection. These results indicate that our DLP can be applied for retinal fundus disease triage, especially in remote areas around the world.

Accuracy of Deep Learning Algorithms for the Diagnosis of Retinopathy of Prematurity by Fundus Images: A Systematic Review and Meta-Analysis

Background

Retinopathy of prematurity (ROP) occurs in preterm infants and may contribute to blindness. Deep learning (DL) models have been used for ophthalmologic diagnoses. We performed a systematic review and meta-analysis of published evidence to summarize and evaluate the diagnostic accuracy of DL algorithms for ROP by fundus images.

Methods

We searched PubMed, EMBASE, Web of Science, and Institute of Electrical and Electronics Engineers Xplore Digital Library on June 13, 2021, for studies using a DL algorithm to distinguish individuals with ROP of different grades, which provided accuracy measurements. The pooled sensitivity and specificity values and the area under the curve (AUC) of summary receiver operating characteristics curves (SROC) summarized overall test performance. The performances in validation and test datasets were assessed together and separately. Subgroup analyses were conducted between the definition and grades of ROP. Threshold and nonthreshold effects were tested to assess biases and evaluate accuracy factors associated with DL models.

Results

Nine studies with fifteen classifiers were included in our meta-analysis. A total of 521,586 objects were applied to DL models. For combined validation and test datasets in each study, the pooled sensitivity and specificity were 0.953 (95% confidence intervals (CI): 0.946-0.959) and 0.975 (0.973-0.977), respectively, and the AUC was 0.984 (0.978-0.989). For the validation dataset and test dataset, the AUC was 0.977 (0.968-0.986) and 0.987 (0.982-0.992), respectively. In the subgroup analysis of ROP vs. normal and differentiation of two ROP grades, the AUC was 0.990 (0.944-0.994) and 0.982 (0.964-0.999), respectively.

Conclusions

Our study shows that DL models can play an essential role in detecting and grading ROP with high sensitivity, specificity, and repeatability. The application of a DL-based automated system may improve ROP screening and diagnosis in the future.

Detection of Diabetic Eye Disease from Retinal Images Using a Deep Learning Based CenterNet Model

Diabetic retinopathy (DR) is an eye disease that alters the blood vessels of a person suffering from diabetes. Diabetic macular edema (DME) occurs when DR affects the macula, which causes fluid accumulation in the macula. Efficient screening systems require experts to manually analyze images to recognize diseases. However, due to the challenging nature of the screening method and lack of trained human resources, devising effective screening-oriented treatment is an expensive task. Automated systems are trying to cope with these challenges; however, these methods do not generalize well to multiple diseases and real-world scenarios. To solve the aforementioned issues, we propose a new method comprising two main steps. The first involves dataset preparation and feature extraction and the other relates to improving a custom deep learning based CenterNet model trained for eye disease classification. Initially, we generate annotations for suspected samples to locate the precise region of interest, while the other part of the proposed solution trains the Center Net model over annotated images. Specifically, we use DenseNet-100 as a feature extraction method on which the one-stage detector, CenterNet, is employed to localize and classify the disease lesions. We evaluated our method over challenging datasets, namely, APTOS-2019 and IDRiD, and attained average accuracy of 97.93% and 98.10%, respectively. We also performed cross-dataset validation with benchmark EYEPACS and Diaretdb1 datasets. Both qualitative and quantitative results demonstrate that our proposed approach outperforms state-of-the-art methods due to more effective localization power of CenterNet, as it can easily recognize small lesions and deal with over-fitted training data. Our proposed framework is proficient in correctly locating and classifying disease lesions. In comparison to existing DR and DME classification approaches, our method can extract representative key points from low-intensity and noisy images and accurately classify them. Hence our approach can play an important role in automated detection and recognition of DR and DME lesions.

Detecting lumbar lesions in 99m Tc-MDP SPECT by deep learning: Comparison with physicians

PURPOSE

^99m Tc-MDP single-photon emission computed tomography (SPECT) is an established tool for diagnosing lumbar stress, a common cause of low back pain (LBP) in pediatric patients. However, detection of small stress lesions is complicated by the low quality of SPECT, leading to significant interreader variability. The study objectives were to develop an approach based on a deep convolutional neural network (CNN) for detecting lumbar lesions in ^99m Tc-MDP scans and to compare its performance to that of physicians in a localization receiver operating characteristic (LROC) study.

METHODS

Sixty-five lesion-absent (LA) ^99m Tc-MDP studies performed in pediatric patients for evaluating LBP were retrospectively identified. Projections for an artificial focal lesion were acquired separately by imaging a ^99m Tc capillary tube at multiple distances from the collimator. An approach was developed to automatically insert lesions into LA scans to obtain realistic lesion-present (LP) ^99m Tc-MDP images while ensuring knowledge of the ground truth. A deep CNN was trained using 2.5D views extracted in LP and LA ^99m Tc-MDP image sets. During testing, the CNN was applied in a sliding-window fashion to compute a 3D "heatmap" reporting the probability of a lesion being present at each lumbar location. The algorithm was evaluated using cross-validation on a ^99m Tc-MDP test dataset which was also studied by five physicians in a LROC study. LP images in the test set were obtained by incorporating lesions at sites selected by a physician based on clinical likelihood of injury in this population.

RESULTS

The deep learning (DL) system slightly outperformed human observers, achieving an area under the LROC curve (AUC_LROC ) of 0.830 (95% confidence interval [CI]: [0.758, 0.924]) compared with 0.785 (95% CI: [0.738, 0.830]) for physicians. The AUC_LROC for the DL system was higher than that of two readers (difference in AUC_LROC [ΔAUC_LROC ] = 0.049 and 0.053) who participated to the study and slightly lower than that of two other readers (ΔAUC_LROC  = -0.006 and -0.012). Another reader outperformed DL by a more substantial margin (ΔAUC_LROC  = -0.053).

CONCLUSION

The DL system provides comparable or superior performance than physicians in localizing small ^99m Tc-MDP positive lumbar lesions.

Assistive Framework for Automatic Detection of All the Zones in Retinopathy of Prematurity Using Deep Learning

Retinopathy of prematurity (ROP) is a potentially blinding disorder seen in low birth weight preterm infants. In India, the burden of ROP is high, with nearly 200,000 premature infants at risk. Early detection through screening and treatment can prevent this blindness. The automatic screening systems developed so far can detect "severe ROP" or "plus disease," but this information does not help schedule follow-up. Identifying vascularized retinal zones and detecting the ROP stage is essential for follow-up or discharge from screening. There is no automatic system to assist these crucial decisions to the best of the authors' knowledge. The low contrast of images, incompletely developed vessels, macular structure, and lack of public data sets are a few challenges in creating such a system. In this paper, a novel method using an ensemble of "U-Network" and "Circle Hough Transform" is developed to detect zones I, II, and III from retinal images in which macula is not developed. The model developed is generic and trained on mixed images of different sizes. It detects zones in images of variable sizes captured by two different imaging systems with an accuracy of 98%. All images of the test set (including the low-quality images) are considered. The time taken for training was only 14 min, and a single image was tested in 30 ms. The present study can help medical experts interpret retinal vascular status correctly and reduce subjective variation in diagnosis.

Evaluation of a Deep Learning-Derived Quantitative Retinopathy of Prematurity Severity Scale

PURPOSE

To evaluate the clinical usefulness of a quantitative deep learning-derived vascular severity score for retinopathy of prematurity (ROP) by assessing its correlation with clinical ROP diagnosis and by measuring clinician agreement in applying a novel scale.

DESIGN

Analysis of existing database of posterior pole fundus images and corresponding ophthalmoscopic examinations using 2 methods of assigning a quantitative scale to vascular severity.

PARTICIPANTS

Images were from clinical examinations of patients in the Imaging and Informatics in ROP Consortium. Four ophthalmologists and 1 study coordinator evaluated vascular severity on a scale from 1 to 9.

METHODS

A quantitative vascular severity score (1-9) was applied to each image using a deep learning algorithm. A database of 499 images was developed for assessment of interobserver agreement.

MAIN OUTCOME MEASURES

Distribution of deep learning-derived vascular severity scores with the clinical assessment of zone (I, II, or III), stage (0, 1, 2, or 3), and extent (<3 clock hours, 3-6 clock hours, and >6 clock hours) of stage 3 evaluated using multivariate linear regression and weighted κ values and Pearson correlation coefficients for interobserver agreement on a 1-to-9 vascular severity scale.

RESULTS

For deep learning analysis, a total of 6344 clinical examinations were analyzed. A higher deep learning-derived vascular severity score was associated with more posterior disease, higher disease stage, and higher extent of stage 3 disease (P < 0.001 for all). For a given ROP stage, the vascular severity score was higher in zone I than zones II or III (P < 0.001). Multivariate regression found zone, stage, and extent all were associated independently with the severity score (P < 0.001 for all). For interobserver agreement, the mean ± standard deviation weighted κ value was 0.67 ± 0.06, and the Pearson correlation coefficient ± standard deviation was 0.88 ± 0.04 on the use of a 1-to-9 vascular severity scale.

CONCLUSIONS

A vascular severity scale for ROP seems feasible for clinical adoption; corresponds with zone, stage, extent of stage 3, and plus disease; and facilitates the use of objective technology such as deep learning to improve the consistency of ROP diagnosis.

Semi-supervised generative adversarial networks for closed-angle detection on anterior segment optical coherence tomography images: an empirical study with a small training dataset

BACKGROUND

Semi-supervised learning algorithms can leverage an unlabeled dataset when labeling is limited or expensive to obtain. In the current study, we developed and evaluated a semi-supervised generative adversarial networks (GANs) model that detects closed-angle on anterior segment optical coherence tomography (AS-OCT) images using a small labeled dataset.

METHODS

In this cross-sectional study, a semi-supervised GANs model was developed for automatic closed-angle detection training on a small labeled and large unsupervised training dataset collected from the Joint Shantou International Eye Center of Shantou University and the Chinese University of Hong Kong (JSIEC). The closed-angle was defined as iris-trabecular contact beyond the scleral spur in AS-OCT images. We further developed two supervised deep learning (DL) models training on the same supervised dataset and the whole dataset separately. The semi-supervised GANs model and supervised DL models' performance were compared on two independent testing datasets from JSIEC (515 images) and the Department of Ophthalmology (84 images), National University Health System, respectively. The diagnostic performance was assessed by evaluation matrices, including the accuracy, sensitivity, specificity, and area under the receiver operating characteristic curve (AUC).

RESULTS

For closed-angle detection using clinician grading of AS-OCT imaging as the reference standard, the semi-supervised GANs model showed comparable performance, with AUCs of 0.97 (95% CI, 0.96-0.99) and 0.98 (95% CI, 0.94-1.00), compared with the supervised DL model (using the whole dataset) [AUCs of 0.97 (95% CI, 0.96-0.99), and 0.97 (95% CI, 0.94-1.00)]. When training on the same small supervised dataset, the semi-supervised GANs achieved performance at least as well as, if not better than, the supervised DL model [AUCs of 0.90 (95% CI: 0.84-0.96), and 0.92 (95% CI, 0.86-0.97)].

CONCLUSIONS

The semi-supervised GANs method achieves diagnostic performance at least as good as a supervised DL model when trained on small labeled datasets. Further development of semi-supervised learning methods could be useful within clinical and research settings.

TRIAL REGISTRATION NUMBER

ChiCTR2000037892.

Deep Learning and Transfer Learning for Optic Disc Laterality Detection: Implications for Machine Learning in Neuro-Ophthalmology

BACKGROUND

Deep learning (DL) has demonstrated human expert levels of performance for medical image classification in a wide array of medical fields, including ophthalmology. In this article, we present the results of our DL system designed to determine optic disc laterality, right eye vs left eye, in the presence of both normal and abnormal optic discs.

METHODS

Using transfer learning, we modified the ResNet-152 deep convolutional neural network (DCNN), pretrained on ImageNet, to determine the optic disc laterality. After a 5-fold cross-validation, we generated receiver operating characteristic curves and corresponding area under the curve (AUC) values to evaluate performance. The data set consisted of 576 color fundus photographs (51% right and 49% left). Both 30° photographs centered on the optic disc (63%) and photographs with varying degree of optic disc centration and/or wider field of view (37%) were included. Both normal (27%) and abnormal (73%) optic discs were included. Various neuro-ophthalmological diseases were represented, such as, but not limited to, atrophy, anterior ischemic optic neuropathy, hypoplasia, and papilledema.

RESULTS

Using 5-fold cross-validation (70% training; 10% validation; 20% testing), our DCNN for classifying right vs left optic disc achieved an average AUC of 0.999 (±0.002) with optimal threshold values, yielding an average accuracy of 98.78% (±1.52%), sensitivity of 98.60% (±1.72%), and specificity of 98.97% (±1.38%). When tested against a separate data set for external validation, our 5-fold cross-validation model achieved the following average performance: AUC 0.996 (±0.005), accuracy 97.2% (±2.0%), sensitivity 96.4% (±4.3%), and specificity 98.0% (±2.2%).

CONCLUSIONS

Small data sets can be used to develop high-performing DL systems for semantic labeling of neuro-ophthalmology images, specifically in distinguishing between right and left optic discs, even in the presence of neuro-ophthalmological pathologies. Although this may seem like an elementary task, this study demonstrates the power of transfer learning and provides an example of a DCNN that can help curate large medical image databases for machine-learning purposes and facilitate ophthalmologist workflow by automatically labeling images according to laterality.

Impact of Artificial Intelligence on Medical Education in Ophthalmology

Clinical care in ophthalmology is rapidly evolving as artificial intelligence (AI) algorithms are being developed. The medical community and national and federal regulatory bodies are recognizing the importance of adapting to AI. However, there is a gap in physicians’ understanding of AI and its implications regarding its potential use in clinical care, and there are limited resources and established programs focused on AI and medical education in ophthalmology. Physicians are essential in the application of AI in a clinical context. An AI curriculum in ophthalmology can help provide physicians with a fund of knowledge and skills to integrate AI into their practice. In this paper, we provide general recommendations for an AI curriculum for medical students, residents, and fellows in ophthalmology.

Strabismus and Artificial Intelligence App: Optimizing Diagnostic and Accuracy

Purpose

Clinical evaluation of eye versions plays an important role in the diagnosis of special strabismus. Despite the importance of versions, they are not standardized in clinical practice because they are subjective. Assuming that objectivity confers accuracy, this research aims to create an artificial intelligence app that can classify the eye versions into nine positions of gaze.

Methods

We analyzed photos of 110 strabismus patients from an outpatient clinic of a tertiary hospital at nine gazes. For each photo, the gaze was identified, and the corresponding version was rated by the same examiner during patient evaluation.

Results

The images were standardized by using the OpenCV library in Python language, so that the patient's eyes were located and sent to a multilabel model through the Keras framework regardless of the photo orientation. Then, the model was trained for each combination of the following groupings: eyes (left, right), gaze (1 to 9), and version (-4 to 4). Resnet50 was used as the neural network architecture, and the Data Augmentation technique was applied. For quick inference via web browser, the SteamLit app framework was employed. For use in Mobiles, the finished model was exported for use in through the Tensorflow Lite converter.

Conclusions

The results showed that the mobile app might be applied to complement evaluation of ocular motility based on objective classification of ocular versions. However, further exploratory research and validations are required.

Translational Relevance

Apart from the traditional clinical practice method, professionals will be able to envisage an easy-to-apply support app, to increase diagnostic accuracy.

Intraoperative assessment of canine soft tissue sarcoma by deep learning enhanced optical coherence tomography

Soft tissue sarcoma (STS) is a locally aggressive and infiltrative tumour in dogs. Surgical resection is the treatment of choice for local tumour control. Currently, post-operative pathology is performed for surgical margin assessment. Spectral-domain optical coherence tomography (OCT) has recently been evaluated for its value for surgical margin assessment in some tumour types in dogs. The purpose of this study was to develop an automatic diagnosis system that can assist clinicians in real-time for OCT image interpretation of tissues at surgical margins. We utilized a ResNet-50 network to classify healthy and cancerous tissues. A patch-based approach was adopted to achieve accurate classification with limited training data (80 cancer images, 80 normal images) and the validation set (20 cancer images, 20 normal images). The proposed method achieved an average accuracy of 97.1% with an excellent sensitivity of 94.3% on the validation set; the quadratic weighted κ was 0.94 for the STS diagnosis. In an independent test data set of 20 OCT images (10 cancer images, 10 normal images), the proposed method correctly differentiated all the STS images. Furthermore, we proposed a diagnostic curve, which could be evaluated in real-time to assist clinicians in detecting the specific location of a lesion. In short, the proposed method is accurate, operates in real-time and is non-invasive, which could be helpful for future surgical guidance.

Diagnosis of central serous chorioretinopathy by deep learning analysis of en face images of choroidal vasculature: A pilot study

Purpose

To diagnose central serous chorioretinopathy (CSC) by deep learning (DL) analyses of en face images of the choroidal vasculature obtained by optical coherence tomography (OCT) and to analyze the regions of interest for the DL from heatmaps.

Methods

One-hundred eyes were studied; 53 eyes with CSC and 47 normal eyes. Volume scans of 12×12 mm square were obtained at the same time as the OCT angiographic (OCTA) scans (Plex Elite 9000 Swept-Source OCT^®, Zeiss). High-quality en face images of the choroidal vasculature of the segmentation slab of one-half of the subfoveal choroidal thickness were created for the analyses. The 100 en face images were divided into 80 for training and 20 for validation. Thus, we divided it into five groups of 20 eyes each, trained the remaining 80 eyes in each group, and then calculated the correct answer rate for each group by validation with 20 eyes. The Neural Network Console (NNC) developed by Sony and the Keras-Tensorflow backend developed by Google were used as the software for the classification with 16 layers of convolutional neural networks. The active region of the heatmap based on the feature quantity extracted by DL was also evaluated as the percentages with gradient-weighted class activation mapping implemented in Keras.

Results

The mean accuracy rate of the validation was 95% for NNC and 88% for Keras. This difference was not significant (P >0.1). The mean active region in the heatmap image was 12.5% in CSC eyes which was significantly lower than the 79.8% in normal eyes (P<0.01).

Conclusions

CSC can be automatically diagnosed by DL with high accuracy from en face images of the choroidal vasculature with different programs, convolutional layer structures, and small data sets. Heatmap analyses showed that the DL focused on the area occupied by the choroidal vessels and their uniformity. We conclude that DL can help in the diagnosis of CSC.

Big data and predictive analytics in healthcare in Bangladesh: regulatory challenges

Big data analytics and artificial intelligence are revolutionizing the global healthcare industry. As the world accumulates unfathomable volumes of data and health technology grows more and more critical to the advancement of medicine, policymakers and regulators are faced with tough challenges around data security and data privacy. This paper reviews existing regulatory frameworks for artificial intelligence-based medical devices and health data privacy in Bangladesh. The study is legal research employing a comparative approach where data is collected from primary and secondary legal materials and filtered based on policies relating to medical data privacy and medical device regulation of Bangladesh. Such policies are then compared with benchmark policies of the European Union and the USA to test the adequacy of the present regulatory framework of Bangladesh and identify the gaps in the current regulation. The study highlights the gaps in policy and regulation in Bangladesh that are hampering the widespread adoption of big data analytics and artificial intelligence in the industry. Despite the vast benefits that big data would bring to Bangladesh's healthcare industry, it lacks the proper data governance and legal framework necessary to gain consumer trust and move forward. Policymakers and regulators must work collaboratively with clinicians, patients and industry to adopt a new regulatory framework that harnesses the potential of big data but ensures adequate privacy and security of personal data. The article opens valuable insight to regulators, academicians, researchers and legal practitioners regarding the present regulatory loopholes in Bangladesh involving exploiting the promise of big data in the medical field. The study concludes with the recommendation for future research into the area of privacy as it relates to artificial intelligence-based medical devices should consult the patients' perspective by employing quantitative analysis research methodology.

Accuracy of a Deep Learning System for Classification of Papilledema Severity on Ocular Fundus Photographs

OBJECTIVE

To evaluate the performance of a deep learning system (DLS) in classifying the severity of papilledema associated with increased intracranial pressure on standard retinal fundus photographs.

METHODS

A DLS was trained to automatically classify papilledema severity in 965 patients (2,103 mydriatic fundus photographs), representing a multiethnic cohort of patients with confirmed elevated intracranial pressure. Training was performed on 1,052 photographs with mild/moderate papilledema (MP) and 1,051 photographs with severe papilledema (SP) classified by a panel of experts. The performance of the DLS and that of 3 independent neuro-ophthalmologists were tested in 111 patients (214 photographs, 92 with MP and 122 with SP) by calculating the area under the receiver operating characteristics curve (AUC), accuracy, sensitivity, and specificity. Kappa agreement scores between the DLS and each of the 3 graders and among the 3 graders were calculated.

RESULTS

The DLS successfully discriminated between photographs of MP and SP, with an AUC of 0.93 (95% confidence interval [CI] 0.89-0.96) and an accuracy, sensitivity, and specificity of 87.9%, 91.8%, and 86.2%, respectively. This performance was comparable with that of the 3 neuro-ophthalmologists (84.1%, 91.8%, and 73.9%, p = 0.19, p = 1, p = 0.09, respectively). Misclassification by the DLS was mainly observed for moderate papilledema (Frisén grade 3). Agreement scores between the DLS and the neuro-ophthalmologists' evaluation was 0.62 (95% CI 0.57-0.68), whereas the intergrader agreement among the 3 neuro-ophthalmologists was 0.54 (95% CI 0.47-0.62).

CONCLUSIONS

Our DLS accurately classified the severity of papilledema on an independent set of mydriatic fundus photographs, achieving a comparable performance with that of independent neuro-ophthalmologists.

CLASSIFICATION OF EVIDENCE

This study provides Class II evidence that a DLS using mydriatic retinal fundus photographs accurately classified the severity of papilledema associated in patients with a diagnosis of increased intracranial pressure.

Heart, Eye, and Artificial Intelligence: A Review

Heart disease continues to be the leading cause of death in the USA. Deep learning-based artificial intelligence (AI) methods have become increasingly common in studying the various factors involved in cardiovascular disease. The usage of retinal scanning techniques to diagnose retinal diseases, such as diabetic retinopathy, age-related macular degeneration, glaucoma and others, using fundus photographs and optical coherence tomography angiography (OCTA) has been extensively documented. Researchers are now looking to combine the power of AI with the non-invasive ease of retinal scanning to examine the workings of the heart and predict changes in the macrovasculature based on microvascular features and function. In this review, we summarize the current state of the field in using retinal imaging to diagnose cardiovascular issues and other diseases.

Insights into the growing popularity of artificial intelligence in ophthalmology

Artificial intelligence (AI) in healthcare is the use of computer-algorithms in analyzing complex medical data to detect associations and provide diagnostic support outputs. AI and deep learning (DL) find obvious applications in fields like ophthalmology wherein huge amount of image-based data need to be analyzed; however, the outcomes related to image recognition are reasonably well-defined. AI and DL have found important roles in ophthalmology in early screening and detection of conditions such as diabetic retinopathy (DR), age-related macular degeneration (ARMD), retinopathy of prematurity (ROP), glaucoma, and other ocular disorders, being successful inroads as far as early screening and diagnosis are concerned and appear promising with advantages of high-screening accuracy, consistency, and scalability. AI algorithms need equally skilled manpower, trained optometrists/ophthalmologists (annotators) to provide accurate ground truth for training the images. The basis of diagnoses made by AI algorithms is mechanical, and some amount of human intervention is necessary for further interpretations. This review was conducted after tracing the history of AI in ophthalmology across multiple research databases and aims to summarise the journey of AI in ophthalmology so far, making a close observation of most of the crucial studies conducted. This article further aims to highlight the potential impact of AI in ophthalmology, the pitfalls, and how to optimally use it to the maximum benefits of the ophthalmologists, the healthcare systems and the patients, alike.

Automated Explainable Multidimensional Deep Learning Platform of Retinal Images for Retinopathy of Prematurity Screening

Importance

A retinopathy of prematurity (ROP) diagnosis currently relies on indirect ophthalmoscopy assessed by experienced ophthalmologists. A deep learning algorithm based on retinal images may facilitate early detection and timely treatment of ROP to improve visual outcomes.

Objective

To develop a retinal image-based, multidimensional, automated, deep learning platform for ROP screening and validate its performance accuracy.

Design, Setting, and Participants

A total of 14 108 eyes of 8652 preterm infants who received ROP screening from 4 centers from November 4, 2010, to November 14, 2019, were included, and a total of 52 249 retinal images were randomly split into training, validation, and test sets. Four main dimensional independent classifiers were developed, including image quality, any stage of ROP, intraocular hemorrhage, and preplus/plus disease. Referral-warranted ROP was automatically generated by integrating the results of 4 classifiers at the image, eye, and patient levels. DeepSHAP, a method based on DeepLIFT and Shapley values (solution concepts in cooperative game theory), was adopted as the heat map technology to explain the predictions. The performance of the platform was further validated as compared with that of the experienced ROP experts. Data were analyzed from February 12, 2020, to June 24, 2020.

Exposure

A deep learning algorithm.

Main Outcomes and Measures

The performance of each classifier included true negative, false positive, false negative, true positive, F1 score, sensitivity, specificity, receiver operating characteristic, area under curve (AUC), and Cohen unweighted κ.

Results

A total of 14 108 eyes of 8652 preterm infants (mean [SD] gestational age, 32.9 [3.1] weeks; 4818 boys [60.4%] of 7973 with known sex) received ROP screening. The performance of all classifiers achieved an F1 score of 0.718 to 0.981, a sensitivity of 0.918 to 0.982, a specificity of 0.949 to 0.992, and an AUC of 0.983 to 0.998, whereas that of the referral system achieved an F1 score of 0.898 to 0.956, a sensitivity of 0.981 to 0.986, a specificity of 0.939 to 0.974, and an AUC of 0.9901 to 0.9956. Fine-grained and class-discriminative heat maps were generated by DeepSHAP in real time. The platform achieved a Cohen unweighted κ of 0.86 to 0.98 compared with a Cohen κ of 0.93 to 0.98 by the ROP experts.

Conclusions and Relevance

In this diagnostic study, an automated ROP screening platform was able to identify and classify multidimensional pathologic lesions in the retinal images. This platform may be able to assist routine ROP screening in general and children hospitals.

Digital technology, tele-medicine and artificial intelligence in ophthalmology: A global perspective

The simultaneous maturation of multiple digital and telecommunications technologies in 2020 has created an unprecedented opportunity for ophthalmology to adapt to new models of care using tele-health supported by digital innovations. These digital innovations include artificial intelligence (AI), 5th generation (5G) telecommunication networks and the Internet of Things (IoT), creating an inter-dependent ecosystem offering opportunities to develop new models of eye care addressing the challenges of COVID-19 and beyond. Ophthalmology has thrived in some of these areas partly due to its many image-based investigations. Tele-health and AI provide synchronous solutions to challenges facing ophthalmologists and healthcare providers worldwide. This article reviews how countries across the world have utilised these digital innovations to tackle diabetic retinopathy, retinopathy of prematurity, age-related macular degeneration, glaucoma, refractive error correction, cataract and other anterior segment disorders. The review summarises the digital strategies that countries are developing and discusses technologies that may increasingly enter the clinical workflow and processes of ophthalmologists. Furthermore as countries around the world have initiated a series of escalating containment and mitigation measures during the COVID-19 pandemic, the delivery of eye care services globally has been significantly impacted. As ophthalmic services adapt and form a "new normal", the rapid adoption of some of telehealth and digital innovation during the pandemic is also discussed. Finally, challenges for validation and clinical implementation are considered, as well as recommendations on future directions.

Impact and Challenges of Integrating Artificial Intelligence and Telemedicine into Clinical Ophthalmology

ABSTRACT

Aging populations and worsening burden of chronic, treatable disease is increasingly creating a global shortfall in ophthalmic care provision. Remote and automated systems carry the promise to expand the scale and potential of health care interventions, and reduce strain on health care services through safe, personalized, efficient, and cost-effective services. However, significant challenges remain. Forward planning in service design is paramount to safeguard patient safety, trust in digital services, data privacy, medico-legal implications, and digital exclusion. We explore the impact and challenges facing patients and clinicians in integrating AI and telemedicine into ophthalmic care-and how these may influence its direction.

Considerations for Artificial Intelligence Real-World Implementation in Ophthalmology: Providers' and Patients' Perspectives

ABSTRACT

Artificial Intelligence (AI), in particular deep learning, has made waves in the health care industry, with several prominent examples shown in ophthalmology. Despite the burgeoning reports on the development of new AI algorithms for detection and management of various eye diseases, few have reached the stage of regulatory approval for real-world implementation. To better enable real-world translation of AI systems, it is important to understand the demands, needs, and concerns of both health care professionals and patients, as providers and recipients of clinical care are impacted by these solutions. This review outlines the advantages and concerns of incorporating AI in ophthalmology care delivery, from both the providers' and patients' perspectives, and the key enablers for seamless transition to real-world implementation.

Artificial intelligence for the detection of age-related macular degeneration in color fundus photographs: A systematic review and meta-analysis

BACKGROUND

Age-related macular degeneration (AMD) is one of the leading causes of vision loss in the elderly population. The application of artificial intelligence (AI) provides convenience for the diagnosis of AMD. This systematic review and meta-analysis aimed to quantify the performance of AI in detecting AMD in fundus photographs.

METHODS

We searched PubMed, Embase, Web of Science and the Cochrane Library before December 31st, 2020 for studies reporting the application of AI in detecting AMD in color fundus photographs. Then, we pooled the data for analysis. PROSPERO registration number: CRD42020197532.

FINDINGS

19 studies were finally selected for systematic review and 13 of them were included in the quantitative synthesis. All studies adopted human graders as reference standard. The pooled area under the receiver operating characteristic curve (AUROC) was 0.983 (95% confidence interval (CI):0.979-0.987). The pooled sensitivity, specificity, and diagnostic odds ratio (DOR) were 0.88 (95% CI:0.88-0.88), 0.90 (95% CI:0.90-0.91), and 275.27 (95% CI:158.43-478.27), respectively. Threshold analysis was performed and a potential threshold effect was detected among the studies (Spearman correlation coefficient: -0.600, P = 0.030), which was the main cause for the heterogeneity. For studies applying convolutional neural networks in the Age-Related Eye Disease Study database, the pooled AUROC, sensitivity, specificity, and DOR were 0.983 (95% CI:0.978-0.988), 0.88 (95% CI:0.88-0.88), 0.91 (95% CI:0.91-0.91), and 273.14 (95% CI:130.79-570.43), respectively.

INTERPRETATION

Our data indicated that AI was able to detect AMD in color fundus photographs. The application of AI-based automatic tools is beneficial for the diagnosis of AMD.

FUNDING

Capital Health Research and Development of Special (2020-1-2052).

Computer-aided detection and abnormality score for the outer retinal layer in optical coherence tomography

BACKGROUND

To develop computer-aided detection (CADe) of ORL abnormalities in the retinal pigmented epithelium, interdigitation zone and ellipsoid zone via optical coherence tomography (OCT).

METHODS

In this retrospective study, healthy participants with normal ORL, and patients with abnormality of ORL including choroidal neovascularisation (CNV) or retinitis pigmentosa (RP) were included. First, an automatic segmentation deep learning (DL) algorithm, CADe, was developed for the three outer retinal layers using 120 handcraft masks of ORL. This automatic segmentation algorithm generated 4000 segmentations, which included 2000 images with normal ORL and 2000 (1000 CNV and 1000 RP) images with focal or wide defects in ORL. Second, based on the automatically generated segmentation images, a binary classifier (normal vs abnormal) was developed. Results were evaluated by area under the receiver operating characteristic curve (AUC).

RESULTS

The DL algorithm achieved an AUC of 0.984 (95% CI 0.976 to 0.993) for individual image evaluation in the internal test set of 797 images. In addition, performance analysis of a publicly available external test set (n=968) had an AUC of 0.957 (95% CI 0.944 to 0.970) and a second clinical external test set (n=1124) had an AUC of 0.978 (95% CI 0.970 to 0.986). Moreover, the CADe highlighted well normal parts of ORL and omitted highlights in abnormal ORLs of CNV and RP.

CONCLUSION

The CADe can use OCT images to segment ORL and differentiate between normal ORL and abnormal ORL. The CADe classifier also performs visualisation and may aid future physician diagnosis and clinical applications.

Diagnostic accuracy of deep learning in medical imaging: a systematic review and meta-analysis

Deep learning (DL) has the potential to transform medical diagnostics. However, the diagnostic accuracy of DL is uncertain. Our aim was to evaluate the diagnostic accuracy of DL algorithms to identify pathology in medical imaging. Searches were conducted in Medline and EMBASE up to January 2020. We identified 11,921 studies, of which 503 were included in the systematic review. Eighty-two studies in ophthalmology, 82 in breast disease and 115 in respiratory disease were included for meta-analysis. Two hundred twenty-four studies in other specialities were included for qualitative review. Peer-reviewed studies that reported on the diagnostic accuracy of DL algorithms to identify pathology using medical imaging were included. Primary outcomes were measures of diagnostic accuracy, study design and reporting standards in the literature. Estimates were pooled using random-effects meta-analysis. In ophthalmology, AUC's ranged between 0.933 and 1 for diagnosing diabetic retinopathy, age-related macular degeneration and glaucoma on retinal fundus photographs and optical coherence tomography. In respiratory imaging, AUC's ranged between 0.864 and 0.937 for diagnosing lung nodules or lung cancer on chest X-ray or CT scan. For breast imaging, AUC's ranged between 0.868 and 0.909 for diagnosing breast cancer on mammogram, ultrasound, MRI and digital breast tomosynthesis. Heterogeneity was high between studies and extensive variation in methodology, terminology and outcome measures was noted. This can lead to an overestimation of the diagnostic accuracy of DL algorithms on medical imaging. There is an immediate need for the development of artificial intelligence-specific EQUATOR guidelines, particularly STARD, in order to provide guidance around key issues in this field.

DeepNeuro: an open-source deep learning toolbox for neuroimaging

Translating deep learning research from theory into clinical practice has unique challenges, specifically in the field of neuroimaging. In this paper, we present DeepNeuro, a Python-based deep learning framework that puts deep neural networks for neuroimaging into practical usage with a minimum of friction during implementation. We show how this framework can be used to design deep learning pipelines that can load and preprocess data, design and train various neural network architectures, and evaluate and visualize the results of trained networks on evaluation data. We present a way of reproducibly packaging data pre- and postprocessing functions common in the neuroimaging community, which facilitates consistent performance of networks across variable users, institutions, and scanners. We show how deep learning pipelines created with DeepNeuro can be concisely packaged into shareable Docker and Singularity containers with user-friendly command-line interfaces.

Recent Applications of Artificial Intelligence in detection of Gastrointestinal, Hepatic and Pancreatic Diseases

There is substantial progress in artificial intelligence (AI) algorithms and their medical sciences applications in the last two decades. AI-assisted programs have already been established for remotely health monitoring using sensors and smartphones. A variety of AI-based prediction models available for the gastrointestinal inflammatory, non-malignant diseases, and bowel bleeding using wireless capsule endoscopy, electronic medical records for hepatitis-associated fibrosis, pancreatic carcinoma using endoscopic ultrasounds. AI-based models may be of immense help for healthcare professionals in the identification, analysis, and decision support using endoscopic images to establish prognosis and risk assessment of patient's treatment using multiple factors. Although enough randomized clinical trials are warranted to establish the efficacy of AI-algorithms assisted and non-AI based treatments before approval of such techniques from medical regulatory authorities. In this article, available AI approaches and AI-based prediction models for detecting gastrointestinal, hepatic, and pancreatic diseases are reviewed. The limitation of AI techniques in such disease prognosis, risk assessment, and decision support are discussed.

Assessment of Generative Adversarial Networks for Synthetic Anterior Segment Optical Coherence Tomography Images in Closed-Angle Detection

PURPOSE

To develop generative adversarial networks (GANs) that synthesize realistic anterior segment optical coherence tomography (AS-OCT) images and evaluate deep learning (DL) models that are trained on real and synthetic datasets for detecting angle closure.

METHODS

The GAN architecture was adopted and trained on the dataset with AS-OCT images collected from the Joint Shantou International Eye Center of Shantou University and the Chinese University of Hong Kong, synthesizing open- and closed-angle AS-OCT images. A visual Turing test with two glaucoma specialists was performed to assess the image quality of real and synthetic images. DL models, trained on either real or synthetic datasets, were developed. Using the clinicians' grading of the AS-OCT images as the reference standard, we compared the diagnostic performance of open-angle vs. closed-angle detection of DL models and the AS-OCT parameter, defined as a trabecular-iris space area 750 µm anterior to the scleral spur (TISA750), in a small independent validation dataset.

RESULTS

The GAN training included 28,643 AS-OCT anterior chamber angle (ACA) images. The real and synthetic datasets for DL model training have an equal distribution of open- and closed-angle images (all with 10,000 images each). The independent validation dataset included 238 open-angle and 243 closed-angle AS-OCT ACA images. The image quality of real versus synthetic AS-OCT images was similar, as assessed by the two glaucoma specialists, except for the scleral spur visibility. For the independent validation dataset, both DL models achieved higher areas under the curve compared with TISA750. Two DL models had areas under the curve of 0.97 (95% confidence interval, 0.96-0.99) and 0.94 (95% confidence interval, 0.92-0.96).

CONCLUSIONS

The GAN synthetic AS-OCT images appeared to be of good quality, according to the glaucoma specialists. The DL models, trained on all-synthetic AS-OCT images, can achieve high diagnostic performance.

TRANSLATIONAL RELEVANCE

The GANs can generate realistic AS-OCT images, which can also be used to train DL models.

Development and evaluation of a deep learning model for the detection of multiple fundus diseases based on colour fundus photography

AIM

To explore and evaluate an appropriate deep learning system (DLS) for the detection of 12 major fundus diseases using colour fundus photography.

METHODS

Diagnostic performance of a DLS was tested on the detection of normal fundus and 12 major fundus diseases including referable diabetic retinopathy, pathologic myopic retinal degeneration, retinal vein occlusion, retinitis pigmentosa, retinal detachment, wet and dry age-related macular degeneration, epiretinal membrane, macula hole, possible glaucomatous optic neuropathy, papilledema and optic nerve atrophy. The DLS was developed with 56 738 images and tested with 8176 images from one internal test set and two external test sets. The comparison with human doctors was also conducted.

RESULTS

The area under the receiver operating characteristic curves of the DLS on the internal test set and the two external test sets were 0.950 (95% CI 0.942 to 0.957) to 0.996 (95% CI 0.994 to 0.998), 0.931 (95% CI 0.923 to 0.939) to 1.000 (95% CI 0.999 to 1.000) and 0.934 (95% CI 0.929 to 0.938) to 1.000 (95% CI 0.999 to 1.000), with sensitivities of 80.4% (95% CI 79.1% to 81.6%) to 97.3% (95% CI 96.7% to 97.8%), 64.6% (95% CI 63.0% to 66.1%) to 100% (95% CI 100% to 100%) and 68.0% (95% CI 67.1% to 68.9%) to 100% (95% CI 100% to 100%), respectively, and specificities of 89.7% (95% CI 88.8% to 90.7%) to 98.1% (95%CI 97.7% to 98.6%), 78.7% (95% CI 77.4% to 80.0%) to 99.6% (95% CI 99.4% to 99.8%) and 88.1% (95% CI 87.4% to 88.7%) to 98.7% (95% CI 98.5% to 99.0%), respectively. When compared with human doctors, the DLS obtained a higher diagnostic sensitivity but lower specificity.

CONCLUSION

The proposed DLS is effective in diagnosing normal fundus and 12 major fundus diseases, and thus has much potential for fundus diseases screening in the real world.

Automated assessment of the substantia nigra on susceptibility map-weighted imaging using deep convolutional neural networks for diagnosis of Idiopathic Parkinson's disease

OBJECTIVES

Despite its use in determining nigrostriatal degeneration, the lack of a consistent interpretation of nigrosome 1 susceptibility map-weighted imaging (SMwI) limits its generalized applicability. To implement and evaluate a diagnostic algorithm based on convolutional neural networks for interpreting nigrosome 1 SMwI for determining nigrostriatal degeneration in idiopathic Parkinson's disease (IPD).

METHODS

In this retrospective study, we enrolled 267 IPD patients and 160 control subjects (125 patients with drug-induced parkinsonism and 35 healthy subjects) at our institute, and 24 IPD patients and 27 control subjects at three other institutes on approval of the local institutional review boards. Dopamine transporter imaging served as the reference standard for the presence or absence of abnormalities of nigrosome 1 on SMwI. Diagnostic performance was compared between visual assessment by an experienced neuroradiologist and the developed deep learning-based diagnostic algorithm in both internal and external datasets using a bootstrapping method with 10000 re-samples by the "pROC" package of R (version 1.16.2).

RESULTS

The area under the receiver operating characteristics curve (AUC) (95% confidence interval [CI]) per participant by the bootstrap method was not significantly different between visual assessment and the deep learning-based algorithm (internal validation, .9622 [0.8912-1.0000] versus 0.9534 [0.8779-0.9956], P = .1511; external validation, 0.9367 [0.8843-0.9802] versus 0.9208 [0.8634-0.9693], P = .6267), indicative of a comparable performance to visual assessment.

CONCLUSIONS

Our deep learning-based algorithm for assessing abnormalities of nigrosome 1 on SMwI was found to have a comparable performance to that of an experienced neuroradiologist.

Artificial Intelligence: the unstoppable revolution in ophthalmology

Artificial Intelligence (AI) is an unstoppable force that is starting to permeate all aspects of our society as part of the revolution being brought into our lives (and into medicine) by the digital era, and accelerated by the current COVID-19 pandemic. As the population ages and developing countries move forward, AI-based systems may be a key asset in streamlining the screening, staging, and treatment planning of sight-threatening eye conditions, offloading the most tedious tasks from the experts, allowing for a greater population coverage, and bringing the best possible care to every patient. This paper presents a review of the state of the art of AI in the field of ophthalmology, focusing on the strengths and weaknesses of current systems, and defining the vision that will enable us to advance scientifically in this digital era. It starts with a thorough yet accessible introduction to the algorithms underlying all modern AI applications. Then, a critical review of the main AI applications in ophthalmology is presented, including Diabetic Retinopathy, Age-Related Macular Degeneration, Retinopathy of Prematurity, Glaucoma, and other AI-related topics such as image enhancement. The review finishes with a brief discussion on the opportunities and challenges that the future of this field might hold.

Key factors in a rigorous longitudinal image-based assessment of retinopathy of prematurity

To describe a database of longitudinally graded telemedicine retinal images to be used as a comparator for future studies assessing grader recall bias and ability to detect typical progression (e.g. International Classification of Retinopathy of Prematurity (ICROP) stages) as well as incremental changes in retinopathy of prematurity (ROP). Cohort comprised of retinal images from 84 eyes of 42 patients who were sequentially screened for ROP over 6 consecutive weeks in a telemedicine program and then followed to vascular maturation or treatment, and then disease stabilization. De-identified retinal images across the 6 weekly exams (2520 total images) were graded by an ROP expert based on whether ROP had improved, worsened, or stayed the same compared to the prior week’s images, corresponding to an overall clinical “gestalt” score. Subsequently, we examined which parameters might have influenced the examiner’s ability to detect longitudinal change; images were graded by the same ROP expert by image view (central, inferior, nasal, superior, temporal) and by retinal components (vascular tortuosity, vascular dilation, stage, hemorrhage, vessel growth), again determining if each particular retinal component or ROP in each image view had improved, worsened, or stayed the same compared to the prior week’s images. Agreement between gestalt scores and view, component, and component by view scores was assessed using percent agreement, absolute agreement, and Cohen’s weighted kappa statistic to determine if any of the hypothesized image features correlated with the ability to predict ROP disease trajectory in patients. The central view showed substantial agreement with gestalt scores (κ = 0.63), with moderate agreement in the remaining views. Of retinal components, vascular tortuosity showed the most overall agreement with gestalt (κ = 0.42–0.61), with only slight to fair agreement for all other components. This is a well-defined ROP database graded by one expert in a real-world setting in a masked fashion that correlated with the actual (remote in time) exams and known outcomes. This provides a foundation for subsequent study of telemedicine’s ability to longitudinally assess ROP disease trajectory, as well as for potential artificial intelligence approaches to retinal image grading, in order to expand patient access to timely, accurate ROP screening.

Applications of Artificial Intelligence for Retinopathy of Prematurity Screening

OBJECTIVES

Childhood blindness from retinopathy of prematurity (ROP) is increasing as a result of improvements in neonatal care worldwide. We evaluate the effectiveness of artificial intelligence (AI)-based screening in an Indian ROP telemedicine program and whether differences in ROP severity between neonatal care units (NCUs) identified by using AI are related to differences in oxygen-titrating capability.

METHODS

External validation study of an existing AI-based quantitative severity scale for ROP on a data set of images from the Retinopathy of Prematurity Eradication Save Our Sight ROP telemedicine program in India. All images were assigned an ROP severity score (1-9) by using the Imaging and Informatics in Retinopathy of Prematurity Deep Learning system. We calculated the area under the receiver operating characteristic curve and sensitivity and specificity for treatment-requiring retinopathy of prematurity. Using multivariable linear regression, we evaluated the mean and median ROP severity in each NCU as a function of mean birth weight, gestational age, and the presence of oxygen blenders and pulse oxygenation monitors.

RESULTS

The area under the receiver operating characteristic curve for detection of treatment-requiring retinopathy of prematurity was 0.98, with 100% sensitivity and 78% specificity. We found higher median (interquartile range) ROP severity in NCUs without oxygen blenders and pulse oxygenation monitors, most apparent in bigger infants (>1500 g and 31 weeks' gestation: 2.7 [2.5-3.0] vs 3.1 [2.4-3.8]; P = .007, with adjustment for birth weight and gestational age).

CONCLUSIONS

Integration of AI into ROP screening programs may lead to improved access to care for secondary prevention of ROP and may facilitate assessment of disease epidemiology and NCU resources.

Current Application of Digital Diagnosing Systems for Retinopathy of Prematurity

BACKGROUND AND OBJECTIVE

Retinopathy of prematurity (ROP), a proliferative vascular eye disease, is one of the leading causes of blindness in childhood and prevails in premature infants with low-birth-weight. The recent progress in digital image analysis offers novel strategies for ROP diagnosis. This paper provides a comprehensive review on the development of digital diagnosing systems for ROP to software researchers. It may also be adopted as a guide to ophthalmologists for selecting the most suitable diagnostic software in the clinical setting, particularly for the remote ophthalmic support.

METHODS

We review the latest literatures concerning the application of digital diagnosing systems for ROP. The diagnosing systems are analyzed and categorized. Articles published between 1998 and 2020 were screened with the two searching engines Pubmed and Google Scholar.

RESULTS

Telemedicine is a method of remote image interpretation that can provide medical service to remote regions, and yet requires training to local operators. On the basis of image collection in telemedicine, computer-based image analytical systems for ROP were later developed. So far, the aforementioned systems have been mainly developed by virtue of classic machine learning, deep learning (DL) and multiple machine learning. During the past two decades, various computer-aided systems for ROP based on classic machine learning (e.g. RISA, ROPtool, CAIER) became available and have achieved satisfactory performance. Further, automated systems for ROP diagnosis based on DL are developed for clinical applications and exhibit high accuracy. Moreover, multiple instance learning is another method to establish an automated system for ROP detection besides DL, which, however, warrants further investigation in future.

CONCLUSION

At present, the incorporation of computer-based image analysis with telemedicine potentially enables the detection, supervision and in-time treatment of ROP for the preterm babies.

Machine Learning Applications in Pediatric Ophthalmology

Purpose: To describe emerging applications of machine learning (ML) in pediatric ophthalmology with an emphasis on the diagnosis and treatment of disorders affecting visual development. Methods: Literature review of studies applying ML algorithms to problems in pediatric ophthalmology. Results: At present, the ML literature emphasizes applications in retinopathy of prematurity. However, there are increasing efforts to apply ML techniques in the diagnosis of amblyogenic conditions such as pediatric cataracts, strabismus, and high refractive error. Conclusions: A greater understanding of the principles governing ML will enable pediatric eye care providers to apply the methodology to unexplored challenges within the subspecialty.

Application of Deep Learning for Diagnosing, Classifying, and Treating Age-Related Macular Degeneration

Age-related macular degeneration (AMD) affects nearly 200 million people and is the third leading cause of irreversible vision loss worldwide. Deep learning, a branch of artificial intelligence that can learn image recognition based on pre-existing datasets, creates an opportunity for more accurate and efficient diagnosis, classification, and treatment of AMD on both individual and population levels. Current algorithms based on fundus photography and optical coherence tomography imaging have already achieved diagnostic accuracy levels comparable to human graders. This accuracy can be further increased when deep learning algorithms are simultaneously applied to multiple diagnostic imaging modalities. Combined with advances in telemedicine and imaging technology, deep learning can enable large populations of patients to be screened than would otherwise be possible and allow ophthalmologists to focus on seeing those patients who are in need of treatment, thus reducing the number of patients with significant visual impairment from AMD.

Deep learning for differentiation of benign and malignant solid liver lesions on ultrasonography

PURPOSE

The ability to reliably distinguish benign from malignant solid liver lesions on ultrasonography can increase access, decrease costs, and help to better triage patients for biopsy. In this study, we used deep learning to differentiate benign from malignant focal solid liver lesions based on their ultrasound appearance.

METHODS

Among the 596 patients who met the inclusion criteria, there were 911 images of individual liver lesions, of which 535 were malignant and 376 were benign. Our training set contained 660 lesions augmented dynamically during training for a total of 330,000 images; our test set contained 79 images. A neural network with ResNet50 architecture was fine-tuned using pre-trained weights on ImageNet. Non-cystic liver lesions with definite diagnosis by histopathology or MRI were included. Accuracy of the final model was compared with expert interpretation. Two separate datasets were used in training and evaluation, one with all lesions and one with lesions deemed to be of uncertain diagnosis based on the Code Abdomen rating system.

RESULTS

Our model trained on the complete set of all lesions achieved a test accuracy of 0.84 (95% CI 0.74-0.90) compared to expert 1 with a test accuracy of 0.80 (95% CI 0.70-0.87) and expert 2 with a test accuracy of 0.73 (95% CI 0.63-0.82). Our model trained on the uncertain set of lesions achieved a test accuracy of 0.79 (95% CI 0.69-0.87) compared to expert 1 with a test accuracy of 0.70 (95% CI 0.59-0.78) and expert 2 with a test accuracy of 0.66 (95% CI 0.55-0.75). On the uncertain dataset, compared to all experts averaged, the model had higher test accuracy (0.79 vs. 0.68, p = 0.025).

CONCLUSION

Deep learning algorithms proposed in the current study improve differentiation of benign from malignant ultrasound-captured solid liver lesions and perform comparably to expert radiologists. Deep learning tools can potentially be used to improve the accuracy and efficiency of clinical workflows.

Retinal imaging in infants

Digital retinal imaging is at the core of a revolution that is continually improving the screening, diagnosis, documentation, monitoring, and treatment of infant retinal diseases. Historically, imaging the retina of infants had been limited and difficult to obtain. Recent advances in photographic instrumentation have significantly improved the ability to obtain high quality multimodal images of the infant retina. These include color fundus photography with different camera angles, ultrasonography, fundus fluorescein angiography, optical coherence tomography, and optical coherence tomography angiography. We provide a summary of the current literature on retinal imaging in infants and highlight areas where further research is required.

Diagnosability of Synthetic Retinal Fundus Images for Plus Disease Detection in Retinopathy of Prematurity

Advances in generative adversarial networks have allowed for engineering of highly-realistic images. Many studies have applied these techniques to medical images. However, evaluation of generated medical images often relies upon image quality and reconstruction metrics, and subjective evaluation by laypersons. This is acceptable for generation of images depicting everyday objects, but not for medical images, where there may be subtle features experts rely upon for diagnosis. We implemented the pix2pix generative adversarial network for retinal fundus image generation, and evaluated the ability of experts to identify generated images as such and to form accurate diagnoses of plus disease in retinopathy of prematurity. We found that, while experts could discern between real and generated images, the diagnoses between image sets were similar. By directly evaluating and confirming physicians' abilities to diagnose generated retinal fundus images, this work supports conclusions that generated images may be viable for dataset augmentation and physician training.

Applications of deep learning in fundus images: A review

The use of fundus images for the early screening of eye diseases is of great clinical importance. Due to its powerful performance, deep learning is becoming more and more popular in related applications, such as lesion segmentation, biomarkers segmentation, disease diagnosis and image synthesis. Therefore, it is very necessary to summarize the recent developments in deep learning for fundus images with a review paper. In this review, we introduce 143 application papers with a carefully designed hierarchy. Moreover, 33 publicly available datasets are presented. Summaries and analyses are provided for each task. Finally, limitations common to all tasks are revealed and possible solutions are given. We will also release and regularly update the state-of-the-art results and newly-released datasets at https://github.com/nkicsl/Fundus_Review to adapt to the rapid development of this field.

Translational Research in Retinopathy of Prematurity: From Bedside to Bench and Back Again

Retinopathy of prematurity (ROP), a vascular proliferative disease affecting preterm infants, is a leading cause of childhood blindness. Various studies have investigated the pathogenesis of ROP. Clinical experience indicates that oxygen levels are strongly correlated with ROP development, which led to the development of oxygen-induced retinopathy (OIR) as an animal model of ROP. OIR has been used extensively to investigate the molecular mechanisms underlying ROP and to evaluate the efficacy of new drug candidates. Large clinical trials have demonstrated the efficacy of anti-vascular endothelial growth factor (VEGF) agents to treat ROP, and anti-VEGF therapy is presently becoming the first-line treatment worldwide. Anti-VEGF therapy has advantages over conventional treatments, including being minimally invasive with a low risk of refractive error. However, long-term safety concerns and the risk of late recurrence limit this treatment. There is an unmet medical need for novel ROP therapies, which need to be addressed by safe and minimally invasive therapies. The recent progress in biotechnology has contributed greatly to translational research. In this review, we outline how basic ROP research has evolved with clinical experience and the subsequent emergence of new drugs. We discuss previous and ongoing trials and present the candidate molecules expected to become novel targets.