Health Economic and Safety Considerations for Artificial Intelligence Applications in Diabetic Retinopathy Screening

Economic evaluation for medical artificial intelligence: accuracy vs. cost-effectiveness in a diabetic retinopathy screening case

Artificial intelligence (AI) models have shown great accuracy in health screening. However, for real-world implementation, high accuracy may not guarantee cost-effectiveness. Improving AI's sensitivity finds more high-risk patients but may raise medical costs while increasing specificity reduces unnecessary referrals but may weaken detection capability. To evaluate the trade-off between AI model performance and the long-running cost-effectiveness, we conducted a cost-effectiveness analysis in a nationwide diabetic retinopathy (DR) screening program in China, comprising 251,535 participants with diabetes over 30 years. We tested a validated AI model in 1100 different diagnostic performances (presented as sensitivity/specificity pairs) and modeled annual screening scenarios. The status quo was defined as the scenario with the most accurate AI performance. The incremental cost-effectiveness ratio (ICER) was calculated for other scenarios against the status quo as cost-effectiveness metrics. Compared to the status quo (sensitivity/specificity: 93.3%/87.7%), six scenarios were cost-saving and seven were cost-effective. To achieve cost-saving or cost-effective, the AI model should reach a minimum sensitivity of 88.2% and specificity of 80.4%. The most cost-effective AI model exhibited higher sensitivity (96.3%) and lower specificity (80.4%) than the status quo. In settings with higher DR prevalence and willingness-to-pay levels, the AI needed higher sensitivity for optimal cost-effectiveness. Urban regions and younger patient groups also required higher sensitivity in AI-based screening. In real-world DR screening, the most accurate AI model may not be the most cost-effective. Cost-effectiveness should be independently evaluated, which is most likely to be affected by the AI's sensitivity.

Managing risk and resilience in autonomous and intelligent systems: Exploring safety in the development, deployment, and use of artificial intelligence in healthcare

Autonomous and intelligent systems (AIS) are being developed and deployed across a wide range of sectors and encompass a variety of technologies designed to engage in different forms of independent reasoning and self-directed behavior. These technologies may bring considerable benefits to society but also pose a range of risk management challenges, particularly when deployed in safety-critical sectors where complex interactions between human, social, and technical processes underpin safety and resilience. Healthcare is one safety-critical sector at the forefront of efforts to develop and deploy intelligent technologies, such as through artificial intelligence (AI) systems intended to automate key aspects of healthcare tasks such as reading medical images to identify signs of pathology. This article develops a qualitative analysis of the sociotechnical sources of risk and resilience associated with the development, deployment, and use of AI in healthcare, drawing on 40 in-depth interviews with participants involved in the development, management, and regulation of AI. Qualitative template analysis is used to examine sociotechnical sources of risk and resilience, drawing on and elaborating Macrae's (2022, Risk Analysis, 42(9), 1999-2025) SOTEC framework that integrates structural, organizational, technological, epistemic, and cultural sources of risk in AIS. This analysis explores an array of sociotechnical sources of risk associated with the development, deployment, and use of AI in healthcare and identifies an array of sociotechnical patterns of resilience that may counter those risks. In doing so, the SOTEC framework is elaborated and translated to define key sources of both risk and resilience in AIS.

Artificial intelligence in retinal imaging: current status and future prospects

INTRODUCTION

The steadily growing and aging world population, in conjunction with continuously increasing prevalences of vision-threatening retinal diseases, is placing an increasing burden on the global healthcare system. The main challenges within retinology involve identifying the comparatively few patients requiring therapy within the large mass, the assurance of comprehensive screening for retinal disease and individualized therapy planning. In order to sustain high-quality ophthalmic care in the future, the incorporation of artificial intelligence (AI) technologies into our clinical practice represents a potential solution.

AREAS COVERED

This review sheds light onto already realized and promising future applications of AI techniques in retinal imaging. The main attention is directed at the application in diabetic retinopathy and age-related macular degeneration. The principles of use in disease screening, grading, therapeutic planning and prediction of future developments are explained based on the currently available literature.

EXPERT OPINION

The recent accomplishments of AI in retinal imaging indicate that its implementation into our daily practice is likely to fundamentally change the ophthalmic healthcare system and bring us one step closer to the goal of individualized treatment. However, it must be emphasized that the aim is to optimally support clinicians by gradually incorporating AI approaches, rather than replacing ophthalmologists.

Biological effects of stored platelet-rich plasma eye-drops in corneal wound healing

BACKGROUND/AIMS

This study aimed to assess the efficacy and sterility of stored platelet-rich plasma (PRP) eye-drops for corneal epithelial wound healing compared with those of autologous serum (AS) eye-drops.

METHODS

At our single institution, PRP and AS eye-drops were prepared using peripheral blood obtained from six healthy volunteers and stored at 4°C. Platelet and leucocyte counts and transforming growth factor (TGF)-β1, epidermal growth factor (EGF), and fibronectin levels were assessed during storage for up to 4 weeks. Sterility was assessed by culturing 4-week poststorage samples. PRP, AS, and phosphate-buffered saline (PBS) eye-drop efficacies were compared using corneal epithelial wound healing assays in vitro and in vivo and monitoring wound areas under a microscope every 3 hours.

RESULTS

Higher platelet and lower leucocyte counts were seen in PRP than in whole blood on the day of preparation. After storage, TGF-β1, EGF, and fibronectin levels were significantly higher in PRP than in AS eye-drops. In vitro and in vivo, PRP eye-drops used on the day of preparation significantly promoted corneal epithelial wound healing compared with PBS. Moreover, PRP eye-drops stored for 4 weeks significantly promoted corneal wound healing compared with PBS and AS eye-drops.

CONCLUSION

PRP eye-drops stored at 4°C for 4 weeks promoted corneal epithelial wound healing with higher levels of growth factors than those observed in AS eye-drops, while maintaining sterility, suggesting that this preparation satisfies the unmet medical needs in the treatment of refractory keratoconjunctival epithelial disorders.

A multi-centre prospective evaluation of THEIA™ to detect diabetic retinopathy (DR) and diabetic macular oedema (DMO) in the New Zealand screening program

PURPOSE

To validate the potential application of THEIA™ as clinical decision making assistant in a national screening program.

METHODS

A total of 900 patients were recruited from either an urban large eye hospital, or a semi-rural optometrist led screening provider, as they were attending their appointment as part of New Zealand Diabetic Eye Screening Programme. The de-identified images were independently graded by three senior specialists, and final results were aggregated using New Zealand grading scheme, which was then converted to referable/non-referable and Healthy/mild/more than mild/sight threatening categories.

RESULTS

THEIA™ managed to grade all images obtained during the study. Comparing the adjudicated images from the specialist grading team, "ground truth", with the grading by the AI platform in detecting "sight threatening" disease, at the patient level THEIA™ achieved 100% imageability, 100% [98.49-100.00%] sensitivity and [97.02-99.16%] specificity, and negative predictive value of 100%. In other words, THEIA™ did not miss any patients with "more than mild" or "sight threatening" disease. The level of agreement between the clinicians and the aggregated results was (k value: 0.9881, 0.9557, and 0.9175), and the level of agreement between THEIA™ and the aggregated labels was (k value: 0.9515).

CONCLUSION

This multi-centre prospective trial showed that THEIA™ did not miss referable disease when screening for diabetic retinopathy and maculopathy. It also had a very high level of granularity in reporting the disease level. As THEIA™ has been tested on a variety of cameras, operating in a range of clinics (rural/urban, ophthalmologist-led\optometrist-led), we believe that it will be a suitable addition to a public diabetic screening program.

Health Economic Implications of Artificial Intelligence Implementation for Ophthalmology in Australia: A Systematic Review

PURPOSE

The health care industry is an inherently resource-intense sector. Emerging technologies such as artificial intelligence (AI) are at the forefront of advancements in health care. The health economic implications of this technology have not been clearly established and represent a substantial barrier to adoption both in Australia and globally. This review aims to determine the health economic impact of implementing AI to ophthalmology in Australia.

METHODS

A systematic search of the databases PubMed/MEDLINE, EMBASE, and CENTRAL was conducted to March 2022, before data collection and risk of bias analysis in accordance with preferred reporting items for systematic ceviews and meta-analyses 2020 guidelines (PROSPERO number CRD42022325511). Included were full-text primary research articles analyzing a population of patients who have or are being evaluated for an ophthalmological diagnosis, using a health economic assessment system to assess the cost-effectiveness of AI.

RESULTS

Seven articles were identified for inclusion. Economic viability was defined as direct cost to the patient that is equal to or less than costs incurred with human clinician assessment. Despite the lack of Australia-specific data, foreign analyses overwhelmingly showed that AI is just as economically viable, if not more so, than traditional human screening programs while maintaining comparable clinical effectiveness. This evidence was largely in the setting of diabetic retinopathy screening.

CONCLUSIONS

Primary Australian research is needed to accurately analyze the health economic implications of implementing AI on a large scale. Further research is also required to analyze the economic feasibility of adoption of AI technology in other areas of ophthalmology, such as glaucoma and cataract screening.

Value assessment of artificial intelligence in medical imaging: a scoping review

BACKGROUND

Artificial intelligence (AI) is seen as one of the major disrupting forces in the future healthcare system. However, the assessment of the value of these new technologies is still unclear, and no agreed international health technology assessment-based guideline exists. This study provides an overview of the available literature in the value assessment of AI in the field of medical imaging.

METHODS

We performed a systematic scoping review of published studies between January 2016 and September 2020 using 10 databases (Medline, Scopus, ProQuest, Google Scholar, and six related databases of grey literature). Information about the context (country, clinical area, and type of study) and mentioned domains with specific outcomes and items were extracted. An existing domain classification, from a European assessment framework, was used as a point of departure, and extracted data were grouped into domains and content analysis of data was performed covering predetermined themes.

RESULTS

Seventy-nine studies were included out of 5890 identified articles. An additional seven studies were identified by searching reference lists, and the analysis was performed on 86 included studies. Eleven domains were identified: (1) health problem and current use of technology, (2) technology aspects, (3) safety assessment, (4) clinical effectiveness, (5) economics, (6) ethical analysis, (7) organisational aspects, (8) patients and social aspects, (9) legal aspects, (10) development of AI algorithm, performance metrics and validation, and (11) other aspects. The frequency of mentioning a domain varied from 20 to 78% within the included papers. Only 15/86 studies were actual assessments of AI technologies. The majority of data were statements from reviews or papers voicing future needs or challenges of AI research, i.e. not actual outcomes of evaluations.

CONCLUSIONS

This review regarding value assessment of AI in medical imaging yielded 86 studies including 11 identified domains. The domain classification based on European assessment framework proved useful and current analysis added one new domain. Included studies had a broad range of essential domains about addressing AI technologies highlighting the importance of domains related to legal and ethical aspects.

Reporting guideline for the early-stage clinical evaluation of decision support systems driven by artificial intelligence: DECIDE-AI

A growing number of artificial intelligence (AI)-based clinical decision support systems are showing promising performance in preclinical, in silico evaluation, but few have yet demonstrated real benefit to patient care. Early-stage clinical evaluation is important to assess an AI system's actual clinical performance at small scale, ensure its safety, evaluate the human factors surrounding its use and pave the way to further large-scale trials. However, the reporting of these early studies remains inadequate. The present statement provides a multi-stakeholder, consensus-based reporting guideline for the Developmental and Exploratory Clinical Investigations of DEcision support systems driven by Artificial Intelligence (DECIDE-AI). We conducted a two-round, modified Delphi process to collect and analyze expert opinion on the reporting of early clinical evaluation of AI systems. Experts were recruited from 20 pre-defined stakeholder categories. The final composition and wording of the guideline was determined at a virtual consensus meeting. The checklist and the Explanation & Elaboration (E&E) sections were refined based on feedback from a qualitative evaluation process. In total, 123 experts participated in the first round of Delphi, 138 in the second round, 16 in the consensus meeting and 16 in the qualitative evaluation. The DECIDE-AI reporting guideline comprises 17 AI-specific reporting items (made of 28 subitems) and ten generic reporting items, with an E&E paragraph provided for each. Through consultation and consensus with a range of stakeholders, we developed a guideline comprising key items that should be reported in early-stage clinical studies of AI-based decision support systems in healthcare. By providing an actionable checklist of minimal reporting items, the DECIDE-AI guideline will facilitate the appraisal of these studies and replicability of their findings.

Cost-Utility Analysis of Screening for Diabetic Retinopathy in China

Background. Diabetic retinopathy (DR) has been primarily indicated to cause vision impairment and blindness, while no studies have focused on the cost-utility of telemedicine-based and community screening programs for DR in China, especially in rural and urban areas, respectively.Methods. We developed a Markov model to calculate the cost-utility of screening programs for DR in DM patients in rural and urban settings from the societal perspective. The incremental cost-utility ratio (ICUR) was calculated for the assessment.Results. In the rural setting, the community screening program obtained 1 QALY with a cost of $4179 (95% CI 3859 to 5343), and the telemedicine screening program had an ICUR of $2323 (95% CI 1023 to 3903) compared with no screening, both of which satisfied the criterion of a significantly cost-effective health intervention. Likewise, community screening programs in urban areas generated an ICUR of $3812 (95% CI 2906 to 4167) per QALY gained, with telemedicine screening at an ICUR of $2437 (95% CI 1242 to 3520) compared with no screening, and both were also cost-effective. By further comparison, compared to community screening programs, telemedicine screening yielded an ICUR of 1212 (95% CI 896 to 1590) per incremental QALY gained in rural setting and 1141 (95% CI 859 to 1403) in urban setting, which both meet the criterion for a significantly cost-effective health intervention.Conclusions. Both telemedicine and community screening for DR in rural and urban settings were cost-effective in China, and telemedicine screening programs were more cost-effective.

Efficacy of artificial intelligence-based screening for diabetic retinopathy in type 2 diabetes mellitus patients

AIM

To explore the efficacy of artificial intelligence (AI)-based screening for diabetic retinopathy (DR) in type 2 diabetes mellitus (T2DM) patients.

METHODS

Data were obtained from 549 T2DM patients who visited the Fundus Disease Center at Henan Provincial People's Hospital from 2018/10-2020/09. DR identification and grading were conducted by two retina specialists, EyeWisdom®DSS and EyeWisdom®MCS, with ophthalmologist grading as reference standard, efficacy of EyeWisdom was evaluated according to sensitivity, specificity, positive predictive value, and negative predictive value.

RESULTS

Ophthalmologists detected 324 DR cases. Among them, there were 43 of mild non-proliferative DR (NPDR), 79 of moderate NPDR, 61 of severe NPDR, and 141 of proliferative DR (PDR). EyeWisdom®DSS detected 337 DR and EyeWisdom®MCS detected 264 DR. Sensitivity and specificity of EyeWisdom®DSS were 91.0%(95 %CI: 87.3%-93.8%) and 81.3% (95 %CI: 75.5%-86.1%), while EyeWisdom®MCS correctly identified 76.2%(95 %CI: 71.1%-80.7%) of patients with DR and 92.4%(95 %CI: 87.9%-95.4%) of patients without DR. EyeWisdom®DSS showed 76.5%(95 %CI: 69.6%-82.3%) sensitivity and 78.4%(95 %CI: 73.7%-82.5%) specificity for detecting NPDR and 64.5%(95 %CI: 56.0%-72.3%) sensitivity and 93.1%(95 %CI: 90.1%-95.3%) specificity for diagnosing PDR.

CONCLUSION

EyeWisdom®DSS is effective in screening for DR, and the accuracy of EyeWisdom®MCS was higher for identifying patients without DR. It is valuable to carry out AI-based DR screening in poorer regions.

Clinical validation of an artificial intelligence-based diabetic retinopathy screening tool for a national health system

OBJECTIVE

To evaluate the accuracy and validity of an automated diabetic retinopathy (DR) screening tool (DART, TeleDx, Santiago, Chile) that uses artificial intelligence to analyze ocular fundus photographs for potential implementation in the national Chilean DR screening programme.

METHOD

This was an observational study of 1123 diabetic eye exams using a validation protocol designed by the commission of the Chilean Ministry of Health personnel and retina specialists.

RESULTS

Receiver operating characteristic (ROC) analysis indicated a sensitivity of 94.6% (95% CI: 90.9-96.9%), specificity of 74.3% (95% CI: 73.3-75%), and negative predictive value of 98.1% (95% CI: 96.8-98.9%) for the automated tool at the optimal operating point for DR screening. The area under the ROC curve was 0.915.

CONCLUSIONS

The results of this study suggest that DART is a valid tool that could be implemented in a heterogeneous health network such as the Chilean system.

Trustworthy AI: Closing the gap between development and integration of AI systems in ophthalmic practice

An increasing number of artificial intelligence (AI) systems are being proposed in ophthalmology, motivated by the variety and amount of clinical and imaging data, as well as their potential benefits at the different stages of patient care. Despite achieving close or even superior performance to that of experts, there is a critical gap between development and integration of AI systems in ophthalmic practice. This work focuses on the importance of trustworthy AI to close that gap. We identify the main aspects or challenges that need to be considered along the AI design pipeline so as to generate systems that meet the requirements to be deemed trustworthy, including those concerning accuracy, resiliency, reliability, safety, and accountability. We elaborate on mechanisms and considerations to address those aspects or challenges, and define the roles and responsibilities of the different stakeholders involved in AI for ophthalmic care, i.e., AI developers, reading centers, healthcare providers, healthcare institutions, ophthalmological societies and working groups or committees, patients, regulatory bodies, and payers. Generating trustworthy AI is not a responsibility of a sole stakeholder. There is an impending necessity for a collaborative approach where the different stakeholders are represented along the AI design pipeline, from the definition of the intended use to post-market surveillance after regulatory approval. This work contributes to establish such multi-stakeholder interaction and the main action points to be taken so that the potential benefits of AI reach real-world ophthalmic settings.

Updates in deep learning research in ophthalmology

Ophthalmology has been one of the early adopters of artificial intelligence (AI) within the medical field. Deep learning (DL), in particular, has garnered significant attention due to the availability of large amounts of data and digitized ocular images. Currently, AI in Ophthalmology is mainly focused on improving disease classification and supporting decision-making when treating ophthalmic diseases such as diabetic retinopathy, age-related macular degeneration (AMD), glaucoma and retinopathy of prematurity (ROP). However, most of the DL systems (DLSs) developed thus far remain in the research stage and only a handful are able to achieve clinical translation. This phenomenon is due to a combination of factors including concerns over security and privacy, poor generalizability, trust and explainability issues, unfavorable end-user perceptions and uncertain economic value. Overcoming this challenge would require a combination approach. Firstly, emerging techniques such as federated learning (FL), generative adversarial networks (GANs), autonomous AI and blockchain will be playing an increasingly critical role to enhance privacy, collaboration and DLS performance. Next, compliance to reporting and regulatory guidelines, such as CONSORT-AI and STARD-AI, will be required to in order to improve transparency, minimize abuse and ensure reproducibility. Thirdly, frameworks will be required to obtain patient consent, perform ethical assessment and evaluate end-user perception. Lastly, proper health economic assessment (HEA) must be performed to provide financial visibility during the early phases of DLS development. This is necessary to manage resources prudently and guide the development of DLS.

Economic Evaluations of Artificial Intelligence in Ophthalmology

ABSTRACT

Artificial intelligence (AI) is expected to cause significant medical quality enhancements and cost-saving improvements in ophthalmology. Although there has been a rapid growth of studies on AI in the recent years, real-world adoption of AI is still rare. One reason may be because the data derived from economic evaluations of AI in health care, which policy makers used for adopting new technology, have been fragmented and scarce. Most data on economics of AI in ophthalmology are from diabetic retinopathy (DR) screening. Few studies classified costs of AI software, which has been considered as a medical device, into direct medical costs. These costs of AI are composed of initial and maintenance costs. The initial costs may include investment in research and development, and costs for validation of different datasets. Meanwhile, the maintenance costs include costs for algorithms upgrade and hardware maintenance in the long run. The cost of AI should be balanced between manufacturing price and reimbursements since it may pose significant challenges and barriers to providers. Evidence from cost-effectiveness analyses showed that AI, either standalone or used with humans, was more cost-effective than manual DR screening. Notably, economic evaluation of AI for DR screening can be used as a model for AI to other ophthalmic diseases.

Feasibility Study of a Multimodal, Cloud-Based, Diabetic Retinal Screening Program in a Workplace Environment

Purpose

To evaluate the feasibility of capturing and interpreting retinal images in a workplace environment using a multimodal, cloud-based, diabetic retinal screening program combined with electronic self-reported questionnaires. The burden of diabetic retinopathy (DR) and other retinal conditions, healthcare utilization, and visual function were also assessed.

Methods

A cross-sectional feasibility study was conducted at the Genentech, Inc., Campus Health Center. Eyes of participants were imaged using ultra-widefield (UWF) color fundus photography (CFP) and spectral-domain optical coherence tomography (SD-OCT). A cloud-based platform was used for the automated, seamless transfer of images to a remote reading center for evaluation for DR and other retinal pathologies. Electronic surveys collected participants’ self-reported medical histories, healthcare utilization, and visual function data.

Results

Among 100 participants (mean age, 43.9 years; 44% male), 33% of them self-reported diabetes. Eye examinations within the past 12 months were reported by 71% of all participants (n = 71/100) and by 85% (n = 28/33) of those with self-reported diabetes. Among participants with complete screening images from both UWF-CFP and SD-OCT, 20% (n = 6/30) of those with self-reported diabetes and 8.5% (n = 5/59) of participants with no history of diabetes were unaware they had mild/moderate nonproliferative DR. Among all participants, 20% (20/100) had a retinal finding, on either UWF-CFP or SD-OCT, or both, which prompted a referral for further evaluation.

Conclusions

A retinal screening program deployed via a secure, scalable, and interoperable cloud-based platform was feasible and conveniently integrated into the workplace.

Translational Relevance

Cloud-based platforms could be used to promote a secure, scalable, and interoperable system for retinal screening in nontraditional environments.

Impact and Challenges of Integrating Artificial Intelligence and Telemedicine into Clinical Ophthalmology

ABSTRACT

Aging populations and worsening burden of chronic, treatable disease is increasingly creating a global shortfall in ophthalmic care provision. Remote and automated systems carry the promise to expand the scale and potential of health care interventions, and reduce strain on health care services through safe, personalized, efficient, and cost-effective services. However, significant challenges remain. Forward planning in service design is paramount to safeguard patient safety, trust in digital services, data privacy, medico-legal implications, and digital exclusion. We explore the impact and challenges facing patients and clinicians in integrating AI and telemedicine into ophthalmic care-and how these may influence its direction.

Multicenter, Head-to-Head, Real-World Validation Study of Seven Automated Artificial Intelligence Diabetic Retinopathy Screening Systems

OBJECTIVE

With rising global prevalence of diabetic retinopathy (DR), automated DR screening is needed for primary care settings. Two automated artificial intelligence (AI)-based DR screening algorithms have U.S. Food and Drug Administration (FDA) approval. Several others are under consideration while in clinical use in other countries, but their real-world performance has not been evaluated systematically. We compared the performance of seven automated AI-based DR screening algorithms (including one FDA-approved algorithm) against human graders when analyzing real-world retinal imaging data.

RESEARCH DESIGN AND METHODS

This was a multicenter, noninterventional device validation study evaluating a total of 311,604 retinal images from 23,724 veterans who presented for teleretinal DR screening at the Veterans Affairs (VA) Puget Sound Health Care System (HCS) or Atlanta VA HCS from 2006 to 2018. Five companies provided seven algorithms, including one with FDA approval, that independently analyzed all scans, regardless of image quality. The sensitivity/specificity of each algorithm when classifying images as referable DR or not were compared with original VA teleretinal grades and a regraded arbitrated data set. Value per encounter was estimated.

RESULTS

Although high negative predictive values (82.72-93.69%) were observed, sensitivities varied widely (50.98-85.90%). Most algorithms performed no better than humans against the arbitrated data set, but two achieved higher sensitivities, and one yielded comparable sensitivity (80.47%, P = 0.441) and specificity (81.28%, P = 0.195). Notably, one had lower sensitivity (74.42%) for proliferative DR (P = 9.77 × 10^-4) than the VA teleretinal graders. Value per encounter varied at $15.14-$18.06 for ophthalmologists and $7.74-$9.24 for optometrists.

CONCLUSIONS

The DR screening algorithms showed significant performance differences. These results argue for rigorous testing of all such algorithms on real-world data before clinical implementation.

Digital health during COVID-19: lessons from operationalising new models of care in ophthalmology

The COVID-19 pandemic has resulted in massive disruptions within health care, both directly as a result of the infectious disease outbreak, and indirectly because of public health measures to mitigate against transmission. This disruption has caused rapid dynamic fluctuations in demand, capacity, and even contextual aspects of health care. Therefore, the traditional face-to-face patient-physician care model has had to be re-examined in many countries, with digital technology and new models of care being rapidly deployed to meet the various challenges of the pandemic. This Viewpoint highlights new models in ophthalmology that have adapted to incorporate digital health solutions such as telehealth, artificial intelligence decision support for triaging and clinical care, and home monitoring. These models can be operationalised for different clinical applications based on the technology, clinical need, demand from patients, and manpower availability, ranging from out-of-hospital models including the hub-and-spoke pre-hospital model, to front-line models such as the inflow funnel model and monitoring models such as the so-called lighthouse model for provider-led monitoring. Lessons learnt from operationalising these models for ophthalmology in the context of COVID-19 are discussed, along with their relevance for other specialty domains.

Clinician checklist for assessing suitability of machine learning applications in healthcare

Machine learning algorithms are being used to screen and diagnose disease, prognosticate and predict therapeutic responses. Hundreds of new algorithms are being developed, but whether they improve clinical decision making and patient outcomes remains uncertain. If clinicians are to use algorithms, they need to be reassured that key issues relating to their validity, utility, feasibility, safety and ethical use have been addressed. We propose a checklist of 10 questions that clinicians can ask of those advocating for the use of a particular algorithm, but which do not expect clinicians, as non-experts, to demonstrate mastery over what can be highly complex statistical and computational concepts. The questions are: (1) What is the purpose and context of the algorithm? (2) How good were the data used to train the algorithm? (3) Were there sufficient data to train the algorithm? (4) How well does the algorithm perform? (5) Is the algorithm transferable to new clinical settings? (6) Are the outputs of the algorithm clinically intelligible? (7) How will this algorithm fit into and complement current workflows? (8) Has use of the algorithm been shown to improve patient care and outcomes? (9) Could the algorithm cause patient harm? and (10) Does use of the algorithm raise ethical, legal or social concerns? We provide examples where an algorithm may raise concerns and apply the checklist to a recent review of diagnostic imaging applications. This checklist aims to assist clinicians in assessing algorithm readiness for routine care and identify situations where further refinement and evaluation is required prior to large-scale use.

Digital Ophthalmology in the UK - Diabetic Retinopathy Screening and Virtual Glaucoma Clinics in the National Health Service

The customary doctor and patient interactions are currently undergoing significant changes through technological advances in imaging and data processing and the need for reducing person-to person contacts during the COVID-19 crisis. There is a trend away from face-to-face examinations to virtual assessments and decision making. Ophthalmology is particularly amenable to such changes, as a high proportion of clinical decisions are based on routine tests and imaging results, which can be assessed remotely. The uptake of digital ophthalmology varies significantly between countries. Due to financial constraints within the National Health Service, specialized ophthalmology units in the UK have been early adopters of digital technology. For more than a decade, patients have been managed remotely in the diabetic retinopathy screening service and virtual glaucoma clinics. We describe the day-to-day running of such services and the doctor and patient experiences with digital ophthalmology in daily practice.

The Evolution of Diabetic Retinopathy Screening Programmes: A Chronology of Retinal Photography from 35 mm Slides to Artificial Intelligence

As a third of people with diabetes mellitus (DM) will suffer the microvascular complications of diabetic retinopathy (DR) and therapeutic options can effectively prevent visual impairment, systematic screening has substantially reduced disease burden in developed countries. In an effort to tackle the rising incidence of DM, screening programmes have modernized in synchrony with technical and infrastructural advancements. Patient evaluation has shifted from face-to-face ophthalmologist-based review delivered through community grassroots to asynchronous store-and-forward modern telemedicine platforms commissioned on a nationwide scale. First pioneered with primitive 35-mm slide film retinal photography, the last decade has seen an emergence of high resolution and widefield imaging devices, which may reveal extents of DR indiscernible to the clinician but with implications of potential earlier identification. Similar progress has been seen in image analysis approaches - automated image analysis of retinal photographs of DR has evolved from qualitative feature detection to rules-based algorithms to autonomous artificial intelligence-powered classification. Such models have, relatively rapidly, been validated and are now receiving approval from health regulation authorities with deployment into the clinical sphere. In this review, we chart the evolution of global DR screening programmes since their inception highlighting major milestones in healthcare infrastructure, telemedicine approaches and imaging devices that have shaped the robust and effective frameworks recognised today. We also provide an outlook for the future of DR screening in the context of recent technological advancements with respect to their limitations in current times.

Evaluating Safety and Efficacy of Follow-up for Patients With Abdominal Pain Using Video Consultation (SAVED Study): Randomized Controlled Trial

BACKGROUND

The benefits of telemedicine include cost savings and decentralized care. Video consultation is one form that enables early detection of deteriorating patients and promotion of self-efficacy in patients who are well but anxious. Abdominal pain is a common symptom presented by patients in emergency departments. These patients could benefit from video consultation, as it enables remote follow-up of patients who do not require admission and facilitates early discharge of patients from overcrowded hospitals.

OBJECTIVE

The study aimed to evaluate the safety and efficacy of the use of digital telereview in patients presenting with undifferentiated acute abdominal pain.

METHODS

The SAVED study was a prospective randomized controlled trial in which follow-up using existing telephone-based telereview (control) was compared with digital telereview (intervention). Patients with undifferentiated acute abdominal pain discharged from the emergency department observation ward were studied based on intention-to-treat. The control arm received routine, provider-scheduled telereview with missed reviews actively coordinated and rescheduled by emergency department staff. The intervention arm received access to a platform for digital telereview (asynchronous and synchronous format) that enabled patient-led appointment rescheduling. Patients were followed-up for 2 weeks for outcomes of service utilization, efficacy (compliance with their disposition plan), and safety (re-presentation for the same condition).

RESULTS

A total of 70 patients participated, with patients randomly assigned to each arm (1:1 ratio). Patients were a mean age of 40.0 (SD 13.8; range 22-71) years, predominantly female (47/70, 67%), and predominantly of Chinese ethnicity (39/70, 56%). The telereview service was used by 32 patients in the control arm (32/35, 91%) and 18 patients in the intervention arm (18/35, 51%). Most patients in control (33/35, 94%; 95% CI 79.5%-99.0%) and intervention (34/35, 97%; 95% CI 83.4%-99.9%) arms were compliant with their final disposition. There was a low rate of re-presentation at 72 hours and 2 weeks for both control (72 hours: 2/35, 6%; 95% CI 1.0%-20.5%; 2 weeks: 2/35, 6%, 95% CI 1.0%-20.5%) and intervention (72 hours: 2/35, 6%; 95% CI 1.0%-20.5%; 2 weeks: 3/35, 9%, 95% CI 2.2%-24.2%) arms. There were no significant differences in safety (P>.99) and efficacy (P>.99) between the two groups.

CONCLUSIONS

The application of digital telereview for the follow-up of patients with abdominal pain may be safe and effective. Future studies are needed to evaluate its cost-effectiveness and usefulness for broader clinical application.

TRIAL REGISTRATION

ISRCTN Registry ISRCTN28468556; http://www.isrctn.com/ISRCTN28468556.