Deploying digital health tools within large, complex health systems: key considerations for adoption and implementation

Implementing mobile health in your surgical practice: Results of a multidisciplinary convening

The perioperative journey remains complex and difficult to navigate for patients and caregivers. Poor communication and lack of care coordination lead to diminished patient satisfaction, outcomes, and system performance. Mobile health platforms have the potential to overcome some of these issues by improving care delivery through timely individualized assessments, improved patient education, and care coordination. Yet mobile health implementation in surgical practice remains limited. Based on a convening of experts using human-centered design techniques, an implementation guide for the integration of mobile health in perioperative care was created to assist with (1) identification of the use of mobile health within a specific surgical practice, (2) identification of the pathway to mobile health implementation, and (3) measurement of successful implementation including patient and surgical system impact. This article reviews those recommendations and provides references to additional literature, including the full implementation guide, to aid those seeking to implement mobile health in a surgical practice or system.

Selection and Implementation of Virtual Scribe Solutions to Reduce Documentation Burden: A Mixed Methods Pilot

Electronic health record (EHR) documentation is a leading reason for clinician burnout. While technology-enabled solutions like virtual and digital scribes aim to improve this, there is limited evidence of their effectiveness and minimal guidance for healthcare systems around solution selection and implementation. A transdisciplinary approach, informed by clinician interviews and other considerations, was used to evaluate and select a virtual scribe solution to pilot in a rapid iterative sprint over 12 weeks. Surveys, interviews, and EHR metadata were analyzed over a staggered 30 day implementation with live and asynchronous virtual scribe solutions. Among 16 pilot clinicians, documentation burden metrics decreased for some but not all. Some clinicians had highly positive comments, and others had concerns regarding scribe training and quality. Our findings demonstrate that virtual scribes may reduce documentation burden for some clinicians and describe a method for a collaborative and iterative technology selection process for digital tools in practice.

An operational guide to translational clinical machine learning in academic medical centers

Few published data science tools are ever translated from academia to real-world clinical settings for which they were intended. One dimension of this problem is the software engineering task of turning published academic projects into tools that are usable at the bedside. Given the complexity of the data ecosystem in large health systems, this task often represents a significant barrier to the real-world deployment of data science tools for prospective piloting and evaluation. Many information technology companies have created Machine Learning Operations (MLOps) teams to help with such tasks at scale, but the low penetration of home-grown data science tools in regular clinical practice precludes the formation of such teams in healthcare organizations. Based on experiences deploying data science tools at two large academic medical centers (Beth Israel Deaconess Medical Center, Boston, MA; Mayo Clinic, Rochester, MN), we propose a strategy to facilitate this transition from academic product to operational tool, defining the responsibilities of the principal investigator, data scientist, machine learning engineer, health system IT administrator, and clinician end-user throughout the process. We first enumerate the technical resources and stakeholders needed to prepare for model deployment. We then propose an approach to planning how the final product will work from data extraction and analysis to visualization of model outputs. Finally, we describe how the team should execute on this plan. We hope to guide health systems aiming to deploy minimum viable data science tools and realize their value in clinical practice.

International perspectives on measuring national digital public health system maturity through a multidisciplinary Delphi study

Unlocking the full potential of digital public health (DiPH) systems requires a comprehensive tool to assess their maturity. While the World Health Organization and the International Telecommunication Union released a toolkit in 2012 covering various aspects of digitalizing national healthcare systems, a holistic maturity assessment tool has been lacking ever since. To bridge this gap, we conducted a pioneering Delphi study, to which 54 experts from diverse continents and academic fields actively contributed to at least one of three rounds. 54 experts participated in developing and rating multidisciplinary quality indicators to measure the maturity of national digital public health systems. Participants established consensus on these indicators with a threshold of 70% agreement on indicator importance. Eventually, 96 indicators were identified and agreed upon by experts. Notably, 48% of these indicators were found to align with existing validated tools, highlighting their relevance and reliability. However, further investigation is required to assess the suitability and applicability of all the suggestions put forward by our participants. Nevertheless, this Delphi study is an essential initial stride toward a comprehensive measurement tool for DiPH system maturity. By working towards a standardized assessment of DiPH system maturity, we aim to empower decision-makers to make informed choices, optimize resource allocation, and drive innovation in healthcare delivery. The results of this study mark a significant milestone in advancing DiPH on a global scale.

Patient Engagement with Conversational Agents in Health Applications 2016-2022: A Systematic Review and Meta-Analysis

Clinicians and patients seeking electronic health applications face challenges in selecting effective solutions due to a high market failure rate. Conversational agent applications ("chatbots") show promise in increasing healthcare user engagement by creating bonds between the applications and users. It is unclear if chatbots improve patient adherence or if past trends to include chatbots in electronic health applications were due to technology hype dynamics and competitive pressure to innovate. We conducted a systematic literature review using Preferred Reporting Items for Systematic reviews and Meta-Analyses methodology on health chatbot randomized control trials. The goal of this review was to identify if user engagement indicators are published in eHealth chatbot studies. A meta-analysis examined patient clinical trial retention of chatbot apps. The results showed no chatbot arm patient retention effect. The small number of studies suggests a need for ongoing eHealth chatbot research, especially given the claims regarding their effectiveness made outside the scientific literatures.

Pilot Study of a Mobile Phone Chatbot for Medication Adherence and Toxicity Management Among Patients With GI Cancers on Capecitabine

PURPOSE

Capecitabine is an oral chemotherapy used to treat many gastrointestinal cancers. Its complex dosing and narrow therapeutic index make medication adherence and toxicity management crucial for quality care.

METHODS

We conducted a pilot study of PENNY-GI, a mobile phone text messaging-based chatbot that leverages algorithmic surveys and natural language processing to promote medication adherence and toxicity management among patients with gastrointestinal cancers on capecitabine. Eligibility initially included all capecitabine-containing regimens but was subsequently restricted to capecitabine monotherapy because of challenges in integrating PENNY-GI with radiation and intravenous chemotherapy schedules. We used design thinking principles and real-time data on safety, accuracy, and usefulness to make iterative refinements to PENNY-GI with the goal of minimizing the proportion of text messaging exchanges with incorrect medication or symptom management recommendations. All patients were invited to participate in structured exit interviews to provide feedback on PENNY-GI.

RESULTS

We enrolled 40 patients (median age 64.5 years, 52.5% male, 62.5% White, 55.0% with colorectal cancer, 50.0% on capecitabine monotherapy). We identified 284 of 3,895 (7.3%) medication-related and 13 of 527 (2.5%) symptom-related text messaging exchanges with incorrect recommendations. In exit interviews with 24 patients, participants reported finding the medication reminders reliable and user-friendly, but the symptom management tool was too simplistic to be helpful.

CONCLUSION

Although PENNY-GI provided accurate recommendations in >90% of text messaging exchanges, we identified multiple limitations with respect to the intervention's generalizability, usefulness, and scalability. Lessons from this pilot study should inform future efforts to develop and implement digital health interventions in oncology.

Implementation of a Continuous Patient Monitoring System in the Hospital Setting: A Qualitative Study

BACKGROUND

Technology can improve care delivery, patient outcomes, and staff satisfaction, but integration into the clinical workflow remains challenging. To contribute to this knowledge area, this study examined the implementation continuum of a contact-free, continuous monitoring system (CFCM) in an inpatient setting. CFCM monitors vital signs and uses the information to alert clinicians of important changes, enabling early detection of patient deterioration.

METHODS

Data were collected throughout the entire implementation continuum at a community teaching hospital. Throughout the study, 3 group and 24 individual interviews and five process observations were conducted. Postimplementation alarm response data were collected. Analysis was conducted using triangulation of information sources and two-coder consensus.

RESULTS

Preimplementation perceived barriers were alarm fatigue, questions about accuracy and trust, impact on patient experience, and challenges to the status quo. Stakeholders identified the value of CFCM as preventing deterioration and benefitting patients who are not good candidates for telemetry. Educational materials addressed each barrier and emphasized the shared CFCM values. Mean alarm response times were below the desired target of two minutes. Postimplementation interview analysis themes revealed lessened concerns of alarm fatigue and improved trust in CFCM than anticipated. Postimplementation challenges included insufficient training for secondary users and impact on patient experience.

CONCLUSION

In addition to understanding the preimplementation anticipated barriers to implementation and establishing shared value before implementation, future recommendations include studying strategies for optimal tailoring of education to each user group, identifying and reinforcing positive process changes after implementation, and including patient experience as the overarching element in frameworks for digital tool implementation.

The Fondazione Toscana Gabriele Monasterio app: a digital health system to improve wellbeing of inpatients with heart or lung disease

BACKGROUND

An app providing material for education and entertaining is a possible way to support patients and healthcare providers in achieving person-centered care.

METHODS

An app tailored on the Fondazione Toscana Gabriele Monasterio (FTGM), a research hospital treating cardiac and lung disorders, was created. A pilot evaluation project was conducted on consecutive patients hospitalized for heart or lung disorders. Patients were asked to complete an assessment questionnaire.

RESULTS

The FTGM app provides information on diagnostic and therapeutic investigations, hospital and healthcare personnel, and includes content for entertainment and learning. It was tested on 215 consecutive patients (75% men, 66% aged >60 years, and 40% with a primary or middle school degree). Sixty-nine percentage of patients used the FTGM app, including 67% of patients aged >80 years and 65% of those with an elementary education (65%). Patients gave positive feedback on the app layout. Many (76%) looked for information on doctors and nurses in the 'People' section. Sixty-five percent of responders had used at least one of the sections called 'Music' and 'Museum visits'. The app helped many patients perceive the hospital as a more liveable place (68%), and to feel less anxious (76%), and more engaged in the diagnostic and therapeutic workup (65%). Overall, the majority of responders (87%) rated the app as 'excellent' or 'good', and almost all (95%) would have recommended other patients to use the app.

CONCLUSIONS

The FTGM app is a possible tool to improve patient wellbeing during hospitalization.

Remote digital health technologies for improving the care of people with respiratory disorders

Respiratory diseases are a leading cause of morbidity and mortality globally. However, existing systems of care, built around scheduled appointments, are not well designed to support the needs of people with chronic and acute respiratory conditions that can change rapidly and unexpectedly. Home-based and personal digital health technologies (DHTs) allow implementation of new models of care catering to the unique needs of individuals. The high number of respiratory triggers and unique responses to them require a personalised solution for each patient. The real-world, repetitive monitoring capabilities of DHTs enable identification of the normal operating characteristics for each individual and, therefore, recognition of the earliest deviations from that state. However, despite this potential, the number of clinical efficacy studies of DHTs is quite small. Evaluation of clinical effectiveness of DHTs in improving health quality in real-world settings is urgently needed.

Clinician Needs for Electronic Health Record Pediatric and Adolescent Weight Management Tools: A Mixed-Methods Study

BACKGROUND

 Clinicians play an important role in addressing pediatric and adolescent obesity, but their effectiveness is restricted by time constraints, competing clinical demands, and the lack of effective electronic health record (EHR) tools. EHR tools are rarely developed with provider input.

OBJECTIVES

 We conducted a mixed method study of clinicians who provide weight management care to children and adolescents to determine current barriers for effective care and explore the role of EHR weight management tools to overcome these barriers.

METHODS

 In this mixed-methods study, we conducted three 1-hour long virtual focus groups at one medium-sized academic health center in Missouri and analyzed the focus group scripts using thematic analysis. We sequentially conducted a descriptive statistical analysis of a survey emailed to pediatric and family medicine primary care clinicians (n = 52) at two private and two academic health centers in Missouri.

RESULTS

 Surveyed clinicians reported that they effectively provided health behavior lifestyle counseling at well-child visits (mean of 60 on a scale of 1-100) and child obesity visits (63); however, most felt the current health care system (27) and EHR tools (41) do not adequately support pediatric weight management. Major themes from the clinician focus groups were that EHR weight management tools should display data in a way that (1) improves clinical efficiency, (2) supports patient-centered communication, (3) improves patient continuity between visits, and (4) reduces documentation burdens. An additional theme was (5) clinicians trust patient data entered in real time over patient recalled data.

CONCLUSION

 Study participants report that the health care system status quo and currently available EHR tools do not sufficiently support clinicians working to manage pediatric or adolescent obesity and provide health behavior counseling. Clinician input in the development and testing of EHR weight management tools provides opportunities to address barriers, inform content, and improve efficiencies of EHR use.

A Scoping Review on Malaria Prevention and Control Intervention in Fragile and Conflict-Affected States (FCAS): A Need for Renewed Focus to Enhance International Cooperation

Malaria is a major public health problem in developing countries. The burden of malaria in fragile and conflict-affected states (FCAS) is increasing year by year. Moreover, the population living in FCAS is often the most vulnerable and at high risk of malaria due to factors, such as deteriorating healthcare system, mass relocations, and reduced resilience to shocks. Therefore, this scoping review aims to map the interventions that are conducted at the FCAS on malaria prevention among the general population. In addition, this review can help policy-makers and international health bodies, providing a comprehensive overview that can lead to more targeted, effective, and context-specific interventions. Databases, such as PubMed, EBSCO-CINAHL, Web of Science, ProQuest, and Cochrane Central Register of Controlled Trials, were searched using specified search terms. A total of 3601 studies were retrieved from the search. After screening, 62 studies were included in the synthesis that met the eligibility criteria. Narrative analysis of the findings was done. The results revealed that in fragile countries, interventions for children below 5 years of age included IPTi, TDA, and ACT. In conflicted countries, interventions for children below 5 years of age included TDA, LLINs, SMC, drug trials, and vaccination. Similar interventions were reported for other age groups and populations. Despite ongoing conflicts, malaria interventions have been maintained in these countries, but a persistent high burden of malaria remains. To achieve the goals of malaria elimination, the results of the review highlight the need for continued research and evaluation of malaria control interventions to assess their effectiveness and impact. Strengthening health systems, building partnerships, utilizing digital health technologies, and conducting context-specific research are recommended to improve healthcare access and reduce the burden of malaria in FCAS.

Ethical and regulatory challenges of AI technologies in healthcare: A narrative review

Over the past decade, there has been a notable surge in AI-driven research, specifically geared toward enhancing crucial clinical processes and outcomes. The potential of AI-powered decision support systems to streamline clinical workflows, assist in diagnostics, and enable personalized treatment is increasingly evident. Nevertheless, the introduction of these cutting-edge solutions poses substantial challenges in clinical and care environments, necessitating a thorough exploration of ethical, legal, and regulatory considerations. A robust governance framework is imperative to foster the acceptance and successful implementation of AI in healthcare. This article delves deep into the critical ethical and regulatory concerns entangled with the deployment of AI systems in clinical practice. It not only provides a comprehensive overview of the role of AI technologies but also offers an insightful perspective on the ethical and regulatory challenges, making a pioneering contribution to the field. This research aims to address the current challenges in digital healthcare by presenting valuable recommendations for all stakeholders eager to advance the development and implementation of innovative AI systems.

Development, piloting and evaluation of an app-supported psychosocial prevention intervention to strengthen participation in working life: a study protocol of a mixed-methods approach

INTRODUCTION

Rates of incapacity to work due to mental disorders have increased in many European countries. The consequences of persistent stress can impact individuals' physical and psychological well-being and gradually develop into chronic stress. Mental disorders or symptoms of burn-out syndrome can have severe consequences. Mental disorders leading to work incapacity significantly burden the health system. Prevention interventions can protect against burn-out, depression, anxiety and other mental health disorders. Digital health is a promising approach to increase the utilisation of effective prevention interventions. This mixed-methods study evaluates a newly developed app-supported psychosocial prevention intervention called 'RV Fit Mental Health' to strengthen participation in working life.

METHODS AND ANALYSIS

The study uses a three-stage parallel mixed-methods design. This study accompanies the development (stage 1), piloting (stage 2) and evaluation (stage 3) of the new intervention. Within the stages, there is a quantitative as well as a qualitative research strand. Employed persons with an incipient mental disorder will be included. Additionally, experts within the project or connected areas will be included. Quantitative data will be analysed using multifactorial variance analyses in a pre-post design. Qualitative data will be analysed using qualitative content analysis. The study is a comprehensive research approach to investigate the development, piloting and evaluation of an app-supported psychosocial app-based prevention intervention. The rigour of the study will be achieved through data triangulation.

ETHICS AND DISSEMINATION

All participants will receive detailed study information and give written informed consent before data collection. Ethical approval was obtained from the Technical University of Munich Ethics Committee. All data collection will follow all legislative rules regarding data protection, also following the Declaration of Helsinki. The study results will be disseminated in peer-reviewed journals and presented at international conferences.

TRIAL REGISTRATION NUMBERS

DRKS00030818 and DRKS00033080.

Navigating the doctor-patient-AI relationship - a mixed-methods study of physician attitudes toward artificial intelligence in primary care

BACKGROUND

Artificial intelligence (AI) is a rapidly advancing field that is beginning to enter the practice of medicine. Primary care is a cornerstone of medicine and deals with challenges such as physician shortage and burnout which impact patient care. AI and its application via digital health is increasingly presented as a possible solution. However, there is a scarcity of research focusing on primary care physician (PCP) attitudes toward AI. This study examines PCP views on AI in primary care. We explore its potential impact on topics pertinent to primary care such as the doctor-patient relationship and clinical workflow. By doing so, we aim to inform primary care stakeholders to encourage successful, equitable uptake of future AI tools. Our study is the first to our knowledge to explore PCP attitudes using specific primary care AI use cases rather than discussing AI in medicine in general terms.

METHODS

From June to August 2023, we conducted a survey among 47 primary care physicians affiliated with a large academic health system in Southern California. The survey quantified attitudes toward AI in general as well as concerning two specific AI use cases. Additionally, we conducted interviews with 15 survey respondents.

RESULTS

Our findings suggest that PCPs have largely positive views of AI. However, attitudes often hinged on the context of adoption. While some concerns reported by PCPs regarding AI in primary care focused on technology (accuracy, safety, bias), many focused on people-and-process factors (workflow, equity, reimbursement, doctor-patient relationship).

CONCLUSION

Our study offers nuanced insights into PCP attitudes towards AI in primary care and highlights the need for primary care stakeholder alignment on key issues raised by PCPs. AI initiatives that fail to address both the technological and people-and-process concerns raised by PCPs may struggle to make an impact.

National Multi-Institutional Validation of a Surgical Transfusion Risk Prediction Model

BACKGROUND

Accurate estimation of surgical transfusion risk is important for many aspects of surgical planning, yet few methods for estimating are available for estimating such risk. There is a need for reliable validated methods for transfusion risk stratification to support effective perioperative planning and resource stewardship.

STUDY DESIGN

This study was conducted using the American College of Surgeons NSQIP datafile from 2019. S-PATH performance was evaluated at each contributing hospital, with and without hospital-specific model tuning. Linear regression was used to assess the relationship between hospital characteristics and area under the receiver operating characteristic (AUROC) curve.

RESULTS

A total of 1,000,927 surgical cases from 414 hospitals were evaluated. Aggregate AUROC was 0.910 (95% CI 0.904 to 0.916) without model tuning and 0.925 (95% CI 0.919 to 0.931) with model tuning. AUROC varied across individual hospitals (median 0.900, interquartile range 0.849 to 0.944), but no statistically significant relationships were found between hospital-level characteristics studied and model AUROC.

CONCLUSIONS

S-PATH demonstrated excellent discriminative performance, although there was variation across hospitals that was not well-explained by hospital-level characteristics. These results highlight the S-PATH's viability as a generalizable surgical transfusion risk prediction tool.

Optimizing Digital Tools for the Field of Substance Use and Substance Use Disorders: Backcasting Exercise

BACKGROUND

Substance use trends are complex; they often rapidly evolve and necessitate an intersectional approach in research, service, and policy making. Current and emerging digital tools related to substance use are promising but also create a range of challenges and opportunities.

OBJECTIVE

This paper reports on a backcasting exercise aimed at the development of a roadmap that identifies values, challenges, facilitators, and milestones to achieve optimal use of digital tools in the substance use field by 2030.

METHODS

A backcasting exercise method was adopted, wherein the core elements are identifying key values, challenges, facilitators, milestones, cornerstones and a current, desired, and future scenario. A structured approach was used by means of (1) an Open Science Framework page as a web-based collaborative working space and (2) key stakeholders' collaborative engagement during the 2022 Lisbon Addiction Conference.

RESULTS

The identified key values were digital rights, evidence-based tools, user-friendliness, accessibility and availability, and person-centeredness. The key challenges identified were ethical funding, regulations, commercialization, best practice models, digital literacy, and access or reach. The key facilitators identified were scientific research, interoperable infrastructure and a culture of innovation, expertise, ethical funding, user-friendly designs, and digital rights and regulations. A range of milestones were identified. The overarching identified cornerstones consisted of creating ethical frameworks, increasing access to digital tools, and continuous trend analysis.

CONCLUSIONS

The use of digital tools in the field of substance use is linked to a range of risks and opportunities that need to be managed. The current trajectories of the use of such tools are heavily influenced by large multinational for-profit companies with relatively little involvement of key stakeholders such as people who use drugs, service providers, and researchers. The current funding models are problematic and lack the necessary flexibility associated with best practice business approaches such as lean and agile principles to design and execute customer discovery methods. Accessibility and availability, digital rights, user-friendly design, and person-focused approaches should be at the forefront in the further development of digital tools. Global legislative and technical infrastructures by means of a global action plan and strategy are necessary and should include ethical frameworks, accessibility of digital tools for substance use, and continuous trend analysis as cornerstones.

Clinical decision support versus a paper-based protocol for massive transfusion: Impact on decision outcomes in a simulation study

BACKGROUND

Management of major hemorrhage frequently requires massive transfusion (MT) support, which should be delivered effectively and efficiently. We have previously developed a clinical decision support system (CDS) for MT using a multicenter multidisciplinary user-centered design study. Here we examine its impact when administering a MT.

STUDY DESIGN AND METHODS

We conducted a randomized simulation trial to compare a CDS for MT with a paper-based MT protocol for the management of simulated hemorrhage. A total of 44 specialist physicians, trainees (residents), and nurses were recruited across critical care to participate in two 20-min simulated bleeding scenarios. The primary outcome was the decision velocity (correct decisions per hour) and overall task completion. Secondary outcomes included cognitive workload and System Usability Scale (SUS).

RESULTS

There was a statistically significant increase in decision velocity for CDS-based management (mean 8.5 decisions per hour) compared to paper based (mean 6.9 decisions per hour; p .003, 95% CI 0.6-2.6). There was no significant difference in the overall task completion using CDS-based management (mean 13.3) compared to paper-based (mean 13.2; p .92, 95% CI -1.2-1.3). Cognitive workload was statistically significantly lower using the CDS compared to the paper protocol (mean 57.1 vs. mean 64.5, p .005, 95% CI 2.4-12.5). CDS usability was assessed as a SUS score of 82.5 (IQR 75-87.5).

DISCUSSION

Compared to paper-based management, CDS-based MT supports more time-efficient decision-making by users with limited CDS training and achieves similar overall task completion while reducing cognitive load. Clinical implementation will determine whether the benefits demonstrated translate to improved patient outcomes.

Implementation and evaluation of an electronic health record-integrated app for postpartum monitoring of hypertensive disorders of pregnancy using patient-contributed data collection

Remote monitoring of women experiencing hypertensive disorders of pregnancy (HDP) can provide timely life-saving data, particularly if these data are integrated into existing patient and clinical workflows. This pilot intervention of a smartphone application (app) for postpartum monitoring of hypertensive disorders integrates patient-contributed data into electronic health records (EHRs) to support monitoring and clinical decision-making. Results from the evaluation of the pilot highlight the resources needed when implementing the app, challenges for integrating an app into the EHR, and the usability and utility of the HDP monitoring app for patient and clinician users. The implementation team's key observations included the importance of a local clinical champion, more robust patient involvement and support for the remote patient monitoring program, an impetus for EHR developers to adopt data integration standards, and a need to expand the capabilities of the standards to support interventions using patient-contributed data.

Digital health technologies and major depressive disorder

There is an urgent need to improve the clinical management of major depressive disorder (MDD), which has become increasingly prevalent over the past two decades. Several gaps and challenges in the awareness, detection, treatment, and monitoring of MDD remain to be addressed. Digital health technologies have demonstrated utility in relation to various health conditions, including MDD. Factors related to the COVID-19 pandemic have accelerated the development of telemedicine, mobile medical apps, and virtual reality apps and have continued to introduce new possibilities across mental health care. Growing access to and acceptance of digital health technologies present opportunities to expand the scope of care and to close gaps in the management of MDD. Digital health technology is rapidly evolving the options for nonclinical support and clinical care for patients with MDD. Iterative efforts to validate and optimize such digital health technologies, including digital therapeutics and digital biomarkers, continue to improve access to and quality of personalized detection, treatment, and monitoring of MDD. The aim of this review is to highlight the existing gaps and challenges in depression management and discuss the current and future landscape of digital health technology as it applies to the challenges faced by patients with MDD and their healthcare providers.

Professionals Evaluating Clients' Suitability for Digital Health and Social Care: Scoping Review of Assessment Instruments

BACKGROUND

Increased digital health and social care services are generally considered to improve people's access to services. However, not everyone can equally access and use these resources. Health and social care professionals should assess clients' suitability for digital solutions, but to succeed, they need information about what to evaluate and how.

OBJECTIVE

This scoping review aimed to identify evaluation tools that professionals can use when assessing clients' suitability for digital health and social care. We summarized the dimensions and the practical usefulness of the instruments.

METHODS

The MEDLINE (Ovid), CINAHL, Web of Science, and ASSIA databases were searched in February 2023 following the Joanna Briggs Institute's Manual for Evidence Synthesis. Studies were included if they focused on health and social care clients and professionals, examined clients' suitability for using digital health or social care, and applied related assessment methods in the direct client work of professionals. Studies focusing primarily on instruments intended for research use without clear applicability to professionals' practical contexts were excluded. Details of the eligible studies were extracted, and qualitative content analysis according to the research objectives was performed.

RESULTS

A total of 19 articles introducing 12 different assessment instruments intended for the health care context were included in the review. No instruments were found for evaluating the suitability for digital social care. The instruments contained 60 dimensions of the client's suitability for digital health, which reflected four perspectives: (1) skill-based suitability, (2) suitability based on general ability to maintain health, (3) suitability based on attitude and experience, and (4) suitability based on practical matters. The described practical usefulness of the instruments included professionals' possibility to (1) identify clients most in need of education and support, (2) direct and recommend the right clients for the right digital services, (3) ensure that clients can use digital health, (4) improve effectiveness and maximize the provision of digital health, (5) develop and redesign services, and (6) empower clients.

CONCLUSIONS

Based on the diverse assessment instruments available and the dimensions they measure, there seems to be no comprehensive evaluation tool for assessing clients' prerequisites to use digital solutions. It is important to further develop comprehensive screening tools applicable to professionals' busy work (both in health and social care) with defined threshold values for suitability.

Parent and clinician perceptions and recommendations on a pediatric cancer pain management app: A qualitative co-design study

Pain is one of the most prevalent and burdensome pediatric cancer symptoms for young children and their families. A significant proportion of pain episodes are experienced in environments where management options are limited, including at home. Digital innovations such as apps may have positive impacts on pain outcomes for young children in these environments. Our overall aim is to co-design such an app and the objective of this study was to explore the perceptions of children's parents about app utility, needed system features, and challenges. We recruited parents of young children with cancer and multidisciplinary pediatric oncology clinicians from two pediatric cancer care centers to participate in audio-recorded, semi-structured, co-design interviews. We conducted interviews structured around technology acceptance and family caregiving theories until data saturation was reached. Audio-recordings were then transcribed, coded, and analyzed using thematic analysis. Forty-two participants took part in the process. Participants endorsed the concept of an app as a useful, safe, and convenient way to engage caregivers in managing their young child's pain. Overall, the app was valued as a means to provide real-time, multimodal informational and procedural pain support to parents, while also reducing the emotional burden of pain care. Recommendations for intervention design included accessibility-focused features, comprehensive symptom tracking, and embedded scientific- and clinically-sound symptom assessments and management advice. Predicted challenges to app use included the workload burden it may place on parents and clinicians. The insights gathered will inform the design principles of our future childhood cancer pain digital research.

EviGUIDE - a tool for evidence-based decision making in image-guided adaptive brachytherapy for cervical cancer

PURPOSE

To develop a novel decision-support system for radiation oncology that incorporates clinical, treatment and outcome data, as well as outcome models from a large clinical trial on magnetic resonance image-guided adaptive brachytherapy (MR-IGABT) for locally advanced cervical cancer (LACC).

METHODS

A system, called EviGUIDE, was developed that combines dosimetric information from the treatment planning system, patient and treatment characteristics, and established tumor control probability (TCP), and normal tissue complication probability (NTCP) models, to predict clinical outcome of radiotherapy treatment of LACC. Six Cox Proportional Hazards models based on data from 1341 patients of the EMBRACE-I study have been integrated. One TCP model for local tumor control, and five NTCP models for OAR morbidities.

RESULTS

EviGUIDE incorporates TCP-NTCP graphs to help users visualize the clinical impact of different treatment plans and provides feedback on achievable doses based on a large reference population. It enables holistic assessment of the interplay between multiple clinical endpoints and tumour and treatment variables. Retrospective analysis of 45 patients treated with MR-IGABT showed that there exists a sub-cohort of patients (20%) with increased risk factors, that could greatly benefit from the quantitative and visual feedback.

CONCLUSION

A novel digital concept was developed that can enhance clinical decision- making and facilitate personalized treatment. It serves as a proof of concept for a new generation of decision support systems in radiation oncology, which incorporate outcome models and high-quality reference data, and aids the dissemination of evidence-based knowledge about optimal treatment and serve as a blueprint for other sites in radiation oncology.

The Constrained-Disorder Principle Assists in Overcoming Significant Challenges in Digital Health: Moving from "Nice to Have" to Mandatory Systems

The success of artificial intelligence depends on whether it can penetrate the boundaries of evidence-based medicine, the lack of policies, and the resistance of medical professionals to its use. The failure of digital health to meet expectations requires rethinking some of the challenges faced. We discuss some of the most significant challenges faced by patients, physicians, payers, pharmaceutical companies, and health systems in the digital world. The goal of healthcare systems is to improve outcomes. Assisting in diagnosing, collecting data, and simplifying processes is a "nice to have" tool, but it is not essential. Many of these systems have yet to be shown to improve outcomes. Current outcome-based expectations and economic constraints make "nice to have," "assists," and "ease processes" insufficient. Complex biological systems are defined by their inherent disorder, bounded by dynamic boundaries, as described by the constrained disorder principle (CDP). It provides a platform for correcting systems' malfunctions by regulating their degree of variability. A CDP-based second-generation artificial intelligence system provides solutions to some challenges digital health faces. Therapeutic interventions are held to improve outcomes with these systems. In addition to improving clinically meaningful endpoints, CDP-based second-generation algorithms ensure patient and physician engagement and reduce the health system's costs.

Sociotechnical Challenges of Digital Health in Nursing Practice During the COVID-19 Pandemic: National Study

BACKGROUND

The COVID-19 pandemic has accelerated the use of digital health innovations, which has greatly impacted nursing practice. However, little is known about the use of digital health services by nurses and how this has changed during the pandemic.

OBJECTIVE

This study explored the sociotechnical challenges that nurses encountered in using digital health services implemented during the pandemic and, accordingly, what digital health capabilities they expect from the emerging workforce.

METHODS

Five groups of nurses, including chief nursing information officers, nurses, clinical educators, nurse representatives at digital health vendor companies, and nurse representatives in government bodies across Australia were interviewed. They were asked about their experience of digital health during the pandemic, their sociotechnical challenges, and their expectations of the digital health capabilities of emerging nurses to overcome these challenges. Interviews were deductively analyzed based on 8 sociotechnical themes, including technical challenges, nurse-technology interaction, clinical content management, training and human resources, communication and workflow, internal policies and guidelines, external factors, and effectiveness assessment of digital health for postpandemic use.

RESULTS

Sixteen participants were interviewed. Human factors and clinical workflow challenges were highly mentioned. Nurses' lack of knowledge and involvement in digital health implementation and evaluation led to inefficient use of these technologies during the pandemic. They expected the emerging workforce to be digitally literate and actively engaged in digital health interventions beyond documentation, such as data analytics and decision-making.

CONCLUSIONS

Nurses should be involved in digital health interventions to efficiently use these technologies and provide safe and quality care. Collaborative efforts among policy makers, vendors, and clinical and academic industries can leverage digital health capabilities in the nursing workforce.

Systematic review of externally validated machine learning models for predicting acute kidney injury in general hospital patients

Acute kidney injury (AKI) is one of the most common and consequential complications among hospitalized patients. Timely AKI risk prediction may allow simple interventions that can minimize or avoid the harm associated with its development. Given the multifactorial and complex etiology of AKI, machine learning (ML) models may be best placed to process the available health data to generate accurate and timely predictions. Accordingly, we searched the literature for externally validated ML models developed from general hospital populations using the current definition of AKI. Of 889 studies screened, only three were retrieved that fit these criteria. While most models performed well and had a sound methodological approach, the main concerns relate to their development and validation in populations with limited diversity, comparable digital ecosystems, use of a vast number of predictor variables and over-reliance on an easily accessible biomarker of kidney injury. These are potentially critical limitations to their applicability in diverse socioeconomic and cultural settings, prompting a need for simpler, more transportable prediction models which can offer a competitive advantage over the current tools used to predict and diagnose AKI.

Access to Technology and Digital Literacy as Determinants of Health and Health Care

Access to and the skills to use technology provide the digital equity necessary for civic and cultural participation, employment, lifelong learning, and access to essential services. However, existing digital disparities and the resultant 'digital divide' risk exacerbating health and health-care inequalities. The COVID-19 pandemic amplified these disparities and accelerated the adoption of technology-driven health care such as telehealth, electronic health records, and digital health technologies. Unfortunately, pre-existing disparities influence the adoption and utilization of these technologies, often leaving disadvantaged groups further behind. Efforts toward digital inclusion, access to technology, and digital literacy are necessary to ensure universal access to and meaningful engagement with digital resources. Nurses play a vital role in promoting digital equity, serving as educators, advocates, and digital navigators, guiding patients through the complexities of the digital health landscape.

The Emerging Role of Artificial Intelligence in Valvular Heart Disease

Valvular heart disease (VHD) is a morbid condition in which timely identification and evidence-based treatments can lead to improved outcomes. Artificial intelligence broadly refers to the ability for computers to perform tasks and problem solve like the human mind. Studies applying AI to VHD have used a variety of structured (eg, sociodemographic, clinical) and unstructured (eg, electrocardiogram, phonocardiogram, and echocardiograms) and machine learning modeling approaches. Additional researches in diverse populations, including prospective clinical trials, are needed to evaluate the effectiveness and value of AI-enabled medical technologies in clinical care for patients with VHD.

Evolving the Cybersecurity of Clinical Photography in Plastic Surgery

Point-of-care photography and photo sharing optimize patient outcomes and facilitate remote consultation imperative for resident surgeons. This literature review and external pilot survey study highlight the risks associated with current practices concerning patient privacy and biometric security. In a survey of 30 plastic surgeon residents and attendings, we found that the majority took photos of patients with their iPhones and shared them with colleagues via Apple iMessage. These findings corroborate previous reports and highlight a lack of physician user acceptance of secure photo-sharing platforms. Finally, we frame a successful example from the literature in the context of a postulated framework for institutional change. Prioritizing the privacy and safety of patients requires a strategic approach that preserves the ease and frequency of use of current practices.

A Review of the Role of Artificial Intelligence in Healthcare

Artificial intelligence (AI) applications have transformed healthcare. This study is based on a general literature review uncovering the role of AI in healthcare and focuses on the following key aspects: (i) medical imaging and diagnostics, (ii) virtual patient care, (iii) medical research and drug discovery, (iv) patient engagement and compliance, (v) rehabilitation, and (vi) other administrative applications. The impact of AI is observed in detecting clinical conditions in medical imaging and diagnostic services, controlling the outbreak of coronavirus disease 2019 (COVID-19) with early diagnosis, providing virtual patient care using AI-powered tools, managing electronic health records, augmenting patient engagement and compliance with the treatment plan, reducing the administrative workload of healthcare professionals (HCPs), discovering new drugs and vaccines, spotting medical prescription errors, extensive data storage and analysis, and technology-assisted rehabilitation. Nevertheless, this science pitch meets several technical, ethical, and social challenges, including privacy, safety, the right to decide and try, costs, information and consent, access, and efficacy, while integrating AI into healthcare. The governance of AI applications is crucial for patient safety and accountability and for raising HCPs' belief in enhancing acceptance and boosting significant health consequences. Effective governance is a prerequisite to precisely address regulatory, ethical, and trust issues while advancing the acceptance and implementation of AI. Since COVID-19 hit the global health system, the concept of AI has created a revolution in healthcare, and such an uprising could be another step forward to meet future healthcare needs.

Digital health in chronic obstructive pulmonary disease

Chronic obstructive pulmonary disease (COPD) can be prevented and treated through effective care, reducing exacerbations and hospitalizations. Early identification of individuals at high risk of COPD exacerbation is an opportunity for preventive measures. However, many patients struggle to follow their treatment plans because of a lack of knowledge about the disease, limited access to resources, and insufficient clinical support. The growth of digital health-which encompasses advancements in health information technology, artificial intelligence, telehealth, the Internet of Things, mobile health, wearable technology, and digital therapeutics-offers opportunities for improving the early diagnosis and management of COPD. This study reviewed the field of digital health in terms of COPD. The findings showed that despite significant advances in digital health, there are still obstacles impeding its effectiveness. Finally, we highlighted some of the major challenges and possibilities for developing and integrating digital health in COPD management.

The digital transformation of surgery

Rapid advances in digital technology and artificial intelligence in recent years have already begun to transform many industries, and are beginning to make headway into healthcare. There is tremendous potential for new digital technologies to improve the care of surgical patients. In this piece, we highlight work being done to advance surgical care using machine learning, computer vision, wearable devices, remote patient monitoring, and virtual and augmented reality. We describe ways these technologies can be used to improve the practice of surgery, and discuss opportunities and challenges to their widespread adoption and use in operating rooms and at the bedside.

iCHECK-DH: Guidelines and Checklist for the Reporting on Digital Health Implementations

BACKGROUND

Implementation of digital health technologies has grown rapidly, but many remain limited to pilot studies due to challenges, such as a lack of evidence or barriers to implementation. Overcoming these challenges requires learning from previous implementations and systematically documenting implementation processes to better understand the real-world impact of a technology and identify effective strategies for future implementation.

OBJECTIVE

A group of global experts, facilitated by the Geneva Digital Health Hub, developed the Guidelines and Checklist for the Reporting on Digital Health Implementations (iCHECK-DH, pronounced "I checked") to improve the completeness of reporting on digital health implementations.

METHODS

A guideline development group was convened to define key considerations and criteria for reporting on digital health implementations. To ensure the practicality and effectiveness of the checklist, it was pilot-tested by applying it to several real-world digital health implementations, and adjustments were made based on the feedback received. The guiding principle for the development of iCHECK-DH was to identify the minimum set of information needed to comprehensively define a digital health implementation, to support the identification of key factors for success and failure, and to enable others to replicate it in different settings.

RESULTS

The result was a 20-item checklist with detailed explanations and examples in this paper. The authors anticipate that widespread adoption will standardize the quality of reporting and, indirectly, improve implementation standards and best practices.

CONCLUSIONS

Guidelines for reporting on digital health implementations are important to ensure the accuracy, completeness, and consistency of reported information. This allows for meaningful comparison and evaluation of results, transparency, and accountability and informs stakeholder decision-making. i-CHECK-DH facilitates standardization of the way information is collected and reported, improving systematic documentation and knowledge transfer that can lead to the development of more effective digital health interventions and better health outcomes.

A mixed-methods study exploring women's perceptions and recommendations for a pregnancy app with monitoring tools

Digital health tools such as apps are being increasingly used by women to access pregnancy-related information. Conducted during the COVID-19 pandemic, this study investigated: (i) pregnant women's current usage of digital health tools to self-monitor and (ii) their interest in theoretical pregnancy app features (a direct patient-to-healthcare-professional communication tool and a body measurement tool). Using a mixed methods approach, 108 pregnant women were surveyed and 15 currently or recently pregnant women were interviewed online. We found that pregnant women used digital health tools to mainly access pregnancy related information and less so to self-monitor. Most participants were interested and enthusiastic about a patient-to-healthcare-professional communication tool. About half of the survey participants (49%) felt comfortable using a body measurement tool to monitor their body parts and 80% of interview participants were interested in using the body measurement to track leg/ankle swelling. Participants also shared additional pregnancy app features that they thought would be beneficial such as a "Digital Wallet" and a desire for a holistic pregnancy app that allowed for more continuous and personalised care. This study highlights the gaps and needs of pregnant women and should inform all stakeholders designing pregnancy digital healthcare. This study offers a unique insight into the needs of pregnant women during a very particular and unique period in human history.

APLUS: A Python library for usefulness simulations of machine learning models in healthcare

Despite the creation of thousands of machine learning (ML) models, the promise of improving patient care with ML remains largely unrealized. Adoption into clinical practice is lagging, in large part due to disconnects between how ML practitioners evaluate models and what is required for their successful integration into care delivery. Models are just one component of care delivery workflows whose constraints determine clinicians' abilities to act on models' outputs. However, methods to evaluate the usefulness of models in the context of their corresponding workflows are currently limited. To bridge this gap we developed APLUS, a reusable framework for quantitatively assessing via simulation the utility gained from integrating a model into a clinical workflow. We describe the APLUS simulation engine and workflow specification language, and apply it to evaluate a novel ML-based screening pathway for detecting peripheral artery disease at Stanford Health Care.

Implementation costs of hospital-based computerised decision support systems: a systematic review

BACKGROUND

The importance of accurately costing implementation strategies is increasingly recognised within the field of implementation science. However, there is a lack of methodological guidance for costing implementation, particularly within digital health settings. This study reports on a systematic review of costing analyses conducted alongside implementation of hospital-based computerised decision support systems.

METHODS

PubMed, Embase, Scopus and CINAHL databases were searched between January 2010 and August 2021. Two reviewers independently screened and selected original research studies that were conducted in a hospital setting, examined the implementation of a computerised decision support systems and reported implementation costs. The Expert Recommendations for Implementing Change Framework was used to identify and categorise implementation strategies into clusters. A previously published costing framework was applied to describe the methods used to measure and value implementation costs. The reporting quality of included studies was assessed using the Consolidated Health Economic Evaluation Reporting Standards checklist.

RESULTS

Titles and abstracts of 1836 articles were screened, with nine articles eligible for inclusion in the review. Implementation costs were most frequently reported under the 'evaluative and iterative strategies' cluster, followed by 'provide interactive assistance'. Labour was the largest implementation-related cost in the included papers, irrespective of implementation strategy. Other reported costs included consumables, durable assets and physical space, which was mostly associated with stakeholder training. The methods used to cost implementation were often unclear. There was variation across studies in the overall quality of reporting.

CONCLUSIONS

A relatively small number of papers have described computerised decision support systems implementation costs, and the methods used to measure and value these costs were not well reported. Priorities for future research should include establishing consistent terminology and appropriate methods for estimating and reporting on implementation costs.

TRIAL REGISTRATION

The review protocol is registered with PROSPERO (ID: CRD42021272948).

Creating an Implementation Enhancement Plan for a Digital Patient Fall Prevention Platform Using the CFIR-ERIC Approach: A Qualitative Study

(1) Background: Inpatient falls are a major cause of hospital-acquired complications (HAC) and inpatient harm. Interventions to prevent falls exist, but it is unclear which are most effective and what implementation strategies best support their use. This study uses existing implementation theory to develop an implementation enhancement plan to improve the uptake of a digital fall prevention workflow. (2) Methods: A qualitative approach using focus groups/interview included 12 participants across four inpatient wards, from a newly built, 300-bed rural referral hospital. Interviews were coded to the Consolidated Framework for Implementation Research (CFIR) and then converted to barrier and enabler statements using consensus agreement. Barriers and enablers were mapped to the Expert Recommendations for Implementing Change (ERIC) tool to develop an implementation enhancement plan. (3) Results: The most prevalent CFIR enablers included: relative advantage (n = 12), access to knowledge and information (n = 11), leadership engagement (n = 9), patient needs and resources (n = 8), cosmopolitanism (n = 5), knowledge and beliefs about the intervention (n = 5), self-efficacy (n = 5) and formally appointed internal implementation leaders (n = 5). Commonly mentioned CFIR barriers included: access to knowledge and information (n = 11), available resources (n = 8), compatibility (n = 8), patient needs and resources (n = 8), design quality and packaging (n = 10), adaptability (n = 7) and executing (n = 7). After mapping the CFIR enablers and barriers to the ERIC tool, six clusters of interventions were revealed: train and educate stakeholders, utilize financial strategies, adapt and tailor to context, engage consumers, use evaluative and iterative strategies and develop stakeholder interrelations. (4) Conclusions: The enablers and barriers identified are similar to those described in the literature. Given there is close agreement between the ERIC consensus framework recommendations and the evidence, this approach will likely assist in enhancing the implementation of Rauland's Concentric Care fall prevention platform and other similar workflow technologies that have the potential to disrupt team and organisational routines. The results of this study will provide a blueprint to enhance implementation that will be tested for effectiveness at a later stage.

Triage to electrocardiogram sign-off time in patients with acute coronary syndrome at a metropolitan Sydney hospital

OBJECTIVE

To compare the time from triage to ECG sign-off in patients with acute coronary syndrome, before and after the introduction of an electronic medical record-integrated ECG workflow system (Epiphany). Additionally, to assess for any correlation between patient characteristics and ECG sign-off times.

METHODS

A retrospective, single-centre cohort study was performed at Prince of Wales Hospital, Sydney. Patients were included if they were over 18 years, presented to Prince of Wales Hospital ED during 2021, had an ED diagnosis code of 'ACS', 'UA', 'NSTEMI' or 'STEMI' and were subsequently admitted under the cardiology team. ECG sign-off times and demographic data were compared between patients presenting prior to 29 June (pre-Epiphany group) and those presenting after (post-Epiphany group). Those without ECGs signed-off were excluded.

RESULTS

There were 200 patients (100 each group) included in the statistical analysis. There was a significant decrease in the median triage to ECG sign-off time, from 35 min (IQR 18-69) pre-Epiphany, to 21 min (IQR 13-37) post-Epiphany. There were only 10 (5%) patients in the pre-Epiphany group and 16 (8%) in the post-Epiphany group, who had ECG sign-off times less than the 10-min. There was no correlation between gender, triage category, age or time of shift with triage to ECG sign-off time.

CONCLUSIONS

The introduction of the Epiphany system has significantly reduced the triage to ECG sign-off time in the ED. Despite this, there remains a large proportion of patients with acute coronary syndrome who do not have an ECG signed-off within the guideline-recommended 10 min.

Patient Perspectives and Preferences for Consent in the Digital Health Context: State-of-the-art Literature Review

BACKGROUND

The increasing integration of digital health tools into care may result in a greater flow of personal health information (PHI) between patients and providers. Although privacy legislation governs how entities may collect, use, or share PHI, such legislation has not kept pace with digital health innovations, resulting in a lack of guidance on implementing meaningful consent. Understanding patient perspectives when implementing meaningful consent is critical to ensure that it meets their needs. Consent for research in the context of digital health is limited.

OBJECTIVE

This state-of-the-art review aimed to understand the current state of research as it relates to patient perspectives on digital health consent. Its objectives were to explore what is known about the patient perspective and experience with digital health consent and provide recommendations on designing and implementing digital health consent based on the findings.

METHODS

A structured literature search was developed and deployed in 4 electronic databases-MEDLINE, IEEE Xplore, Scopus, and Web of Science-for articles published after January 2010. The initial literature search was conducted in March 2021 and updated in March 2022. Articles were eligible for inclusion if they discussed electronic consent or consent, focused on the patient perspective or preference, and were related to digital health or digital PHI. Data were extracted using an extraction template and analyzed using qualitative content analysis.

RESULTS

In total, 75 articles were included for analysis. Most studies were published within the last 5 years (58/75, 77%) and conducted in a clinical care context (33/75, 44%) and in the United States (48/75, 64%). Most studies aimed to understand participants' willingness to share PHI (25/75, 33%) and participants' perceived usability and comprehension of an electronic consent notice (25/75, 33%). More than half (40/75, 53%) of the studies did not describe the type of consent model used. The broad open consent model was the most explored (11/75, 15%). Of the 75 studies, 68 (91%) found that participants were willing to provide consent; however, their consent behaviors and preferences were context-dependent. Common patient consent requirements included clear and digestible information detailing who can access PHI, for what purpose their PHI will be used, and how privacy will be ensured.

CONCLUSIONS

There is growing interest in understanding the patient perspective on digital health consent in the context of providing clinical care. There is evidence suggesting that many patients are willing to consent for various purposes, especially when there is greater transparency on how the PHI is used and oversight mechanisms are in place. Providing this transparency is critical for fostering trust in digital health tools and the innovative uses of data to optimize health and system outcomes.

Stakeholder Perspectives of Clinical Artificial Intelligence Implementation: Systematic Review of Qualitative Evidence

BACKGROUND

The rhetoric surrounding clinical artificial intelligence (AI) often exaggerates its effect on real-world care. Limited understanding of the factors that influence its implementation can perpetuate this.

OBJECTIVE

In this qualitative systematic review, we aimed to identify key stakeholders, consolidate their perspectives on clinical AI implementation, and characterize the evidence gaps that future qualitative research should target.

METHODS

Ovid-MEDLINE, EBSCO-CINAHL, ACM Digital Library, Science Citation Index-Web of Science, and Scopus were searched for primary qualitative studies on individuals' perspectives on any application of clinical AI worldwide (January 2014-April 2021). The definition of clinical AI includes both rule-based and machine learning-enabled or non-rule-based decision support tools. The language of the reports was not an exclusion criterion. Two independent reviewers performed title, abstract, and full-text screening with a third arbiter of disagreement. Two reviewers assigned the Joanna Briggs Institute 10-point checklist for qualitative research scores for each study. A single reviewer extracted free-text data relevant to clinical AI implementation, noting the stakeholders contributing to each excerpt. The best-fit framework synthesis used the Nonadoption, Abandonment, Scale-up, Spread, and Sustainability (NASSS) framework. To validate the data and improve accessibility, coauthors representing each emergent stakeholder group codeveloped summaries of the factors most relevant to their respective groups.

RESULTS

The initial search yielded 4437 deduplicated articles, with 111 (2.5%) eligible for inclusion (median Joanna Briggs Institute 10-point checklist for qualitative research score, 8/10). Five distinct stakeholder groups emerged from the data: health care professionals (HCPs), patients, carers and other members of the public, developers, health care managers and leaders, and regulators or policy makers, contributing 1204 (70%), 196 (11.4%), 133 (7.7%), 129 (7.5%), and 59 (3.4%) of 1721 eligible excerpts, respectively. All stakeholder groups independently identified a breadth of implementation factors, with each producing data that were mapped between 17 and 24 of the 27 adapted Nonadoption, Abandonment, Scale-up, Spread, and Sustainability subdomains. Most of the factors that stakeholders found influential in the implementation of rule-based clinical AI also applied to non-rule-based clinical AI, with the exception of intellectual property, regulation, and sociocultural attitudes.

CONCLUSIONS

Clinical AI implementation is influenced by many interdependent factors, which are in turn influenced by at least 5 distinct stakeholder groups. This implies that effective research and practice of clinical AI implementation should consider multiple stakeholder perspectives. The current underrepresentation of perspectives from stakeholders other than HCPs in the literature may limit the anticipation and management of the factors that influence successful clinical AI implementation. Future research should not only widen the representation of tools and contexts in qualitative research but also specifically investigate the perspectives of all stakeholder HCPs and emerging aspects of non-rule-based clinical AI implementation.

TRIAL REGISTRATION

PROSPERO (International Prospective Register of Systematic Reviews) CRD42021256005; https://www.crd.york.ac.uk/prospero/display_record.php?RecordID=256005.

INTERNATIONAL REGISTERED REPORT IDENTIFIER (IRRID)

RR2-10.2196/33145.

Promotors and barriers to the implementation and adoption of assistive technology and telecare for people with dementia and their caregivers: a systematic review of the literature

BACKGROUND

One of the most pressing issues in our society is the provision of proper care and treatment for the growing global health challenge of ageing. Assistive Technology and Telecare (ATT) is a key component in facilitation of safer, longer, and independent living for people with dementia (PwD) and has the potential to extend valuable care and support for caregivers globally. The objective of this study was to identify promotors and barriers to implementation and adoption of ATT for PwD and their informal (family and friends) and formal (healthcare professionals) caregivers.

METHODS

Five databases Medline (Ovid), CINAHL, Web of Science, APA PsycINFO and EMBASE were searched. PRISMA guidelines have been used to guide all processes and results. Retrieved studies were qualitative, mixed-method and quantitative, screened using Rayyan and overall quality assessed using Critical Appraisal Skills Programme (CASP) and Mixed Methods Assessment Tool (MMAT). Certainty of evidence was assessed using Grading of Recommendations Assessment, Development and Evaluation (GRADE) criteria and assigned within categories of high, moderate, or low. NVivo was used for synthesis and analysis of article content. A narrative synthesis combines the study findings.

RESULTS

Thirty studies (7 quantitative, 19 qualitative and 4 mixed methods) met the inclusion criteria. Identified primary promotors for the implementation and adoption of ATT were: personalized training and co-designed solutions, safety for the PwD, involvement of all relevant stakeholders, ease of use and support, and cultural relevance. Main barriers for the implementation and adoption of ATT included: unintended adverse consequences, timing and disease progress, technology anxiety, system failures, digital divide, and lack of access to or knowledge of available ATT.

CONCLUSION

The most crucial elements for the adoption of ATT in the future will be a focus on co-design, improved involvement of relevant stakeholders, and the adaptability (tailoring related to context) of ATT solutions over time (disease process).

The Evaluation of a Clinical Decision Support Tool Using Natural Language Processing to Screen Hospitalized Adults for Unhealthy Substance Use: Protocol for a Quasi-Experimental Design

BACKGROUND

Automated and data-driven methods for screening using natural language processing (NLP) and machine learning may replace resource-intensive manual approaches in the usual care of patients hospitalized with conditions related to unhealthy substance use. The rigorous evaluation of tools that use artificial intelligence (AI) is necessary to demonstrate effectiveness before system-wide implementation. An NLP tool to use routinely collected data in the electronic health record was previously validated for diagnostic accuracy in a retrospective study for screening unhealthy substance use. Our next step is a noninferiority design incorporated into a research protocol for clinical implementation with prospective evaluation of clinical effectiveness in a large health system.

OBJECTIVE

This study aims to provide a study protocol to evaluate health outcomes and the costs and benefits of an AI-driven automated screener compared to manual human screening for unhealthy substance use.

METHODS

A pre-post design is proposed to evaluate 12 months of manual screening followed by 12 months of automated screening across surgical and medical wards at a single medical center. The preintervention period consists of usual care with manual screening by nurses and social workers and referrals to a multidisciplinary Substance Use Intervention Team (SUIT). Facilitated by a NLP pipeline in the postintervention period, clinical notes from the first 24 hours of hospitalization will be processed and scored by a machine learning model, and the SUIT will be similarly alerted to patients who flagged positive for substance misuse. Flowsheets within the electronic health record have been updated to capture rates of interventions for the primary outcome (brief intervention/motivational interviewing, medication-assisted treatment, naloxone dispensing, and referral to outpatient care). Effectiveness in terms of patient outcomes will be determined by noninferior rates of interventions (primary outcome), as well as rates of readmission within 6 months, average time to consult, and discharge rates against medical advice (secondary outcomes) in the postintervention period by a SUIT compared to the preintervention period. A separate analysis will be performed to assess the costs and benefits to the health system by using automated screening. Changes from the pre- to postintervention period will be assessed in covariate-adjusted generalized linear mixed-effects models.

RESULTS

The study will begin in September 2022. Monthly data monitoring and Data Safety Monitoring Board reporting are scheduled every 6 months throughout the study period. We anticipate reporting final results by June 2025.

CONCLUSIONS

The use of augmented intelligence for clinical decision support is growing with an increasing number of AI tools. We provide a research protocol for prospective evaluation of an automated NLP system for screening unhealthy substance use using a noninferiority design to demonstrate comprehensive screening that may be as effective as manual screening but less costly via automated solutions.

TRIAL REGISTRATION

ClinicalTrials.gov NCT03833804; https://clinicaltrials.gov/ct2/show/NCT03833804.

INTERNATIONAL REGISTERED REPORT IDENTIFIER (IRRID)

DERR1-10.2196/42971.

Governing partnerships with technology companies as part of the COVID-19 response in Canada: A qualitative case study

Cross-sector partnerships are vital for maintaining resilient health systems; however, few studies have sought to empirically assess the barriers and enablers of effective and responsible partnerships during public health emergencies. Through a qualitative, multiple case study, we analyzed 210 documents and conducted 26 interviews with stakeholders in three real-world partnerships between Canadian health organizations and private technology startups during the COVID-19 pandemic. The three partnerships involved: 1) deploying a virtual care platform to care for COVID-19 patients at one hospital, 2) deploying a secure messaging platform for physicians at another hospital, and 3) using data science to support a public health organization. Our results demonstrate that a public health emergency created time and resource pressures throughout a partnership. Given these constraints, early and sustained alignment on the core problem was critical for success. Moreover, governance processes designed for normal operations, such as procurement, were triaged and streamlined. Social learning, or the process of learning from observing others, offset some time and resource pressures. Social learning took many forms ranging from informal conversations between individuals at peer organisations (e.g., hospital chief information officers) to standing meetings at the local university's city-wide COVID-19 response table. We also found that startups' flexibility and understanding of the local context enabled them to play a highly valuable role in emergency response. However, pandemic fueled "hypergrowth" created risks for startups, such as introducing opportunities for deviation away from their core value proposition. Finally, we found each partnership navigated intense workloads, burnout, and personnel turnover through the pandemic. Strong partnerships required healthy, motivated teams. Visibility into and engagement in partnership governance, belief in partnership impact, and strong emotional intelligence in managers promoted team well-being. Taken together, these findings can help to bridge the theory-to-practice gap and guide effective cross-sector partnerships during public health emergencies.

Methodological guidance for the evaluation and updating of clinical prediction models: a systematic review

BACKGROUND

Clinical prediction models are often not evaluated properly in specific settings or updated, for instance, with information from new markers. These key steps are needed such that models are fit for purpose and remain relevant in the long-term. We aimed to present an overview of methodological guidance for the evaluation (i.e., validation and impact assessment) and updating of clinical prediction models.

METHODS

We systematically searched nine databases from January 2000 to January 2022 for articles in English with methodological recommendations for the post-derivation stages of interest. Qualitative analysis was used to summarize the 70 selected guidance papers.

RESULTS

Key aspects for validation are the assessment of statistical performance using measures for discrimination (e.g., C-statistic) and calibration (e.g., calibration-in-the-large and calibration slope). For assessing impact or usefulness in clinical decision-making, recent papers advise using decision-analytic measures (e.g., the Net Benefit) over simplistic classification measures that ignore clinical consequences (e.g., accuracy, overall Net Reclassification Index). Commonly recommended methods for model updating are recalibration (i.e., adjustment of intercept or baseline hazard and/or slope), revision (i.e., re-estimation of individual predictor effects), and extension (i.e., addition of new markers). Additional methodological guidance is needed for newer types of updating (e.g., meta-model and dynamic updating) and machine learning-based models.

CONCLUSION

Substantial guidance was found for model evaluation and more conventional updating of regression-based models. An important development in model evaluation is the introduction of a decision-analytic framework for assessing clinical usefulness. Consensus is emerging on methods for model updating.

eHealth or e-Chaos: The use of Digital Health Interventions for Health Systems Strengthening in sub-Saharan Africa over the last 10 years: A scoping review

Background

Digital health solutions are a potent and complementary intervention in health system strengthening to accelerate universal access to health services. Implementing scalable, sustainable, and integrated digital solutions in a coordinated manner is necessary to experience the benefits of digital interventions in health systems. We sought to establish the breadth and scope of available digital health interventions (DHIs) and their functions in sub-Saharan Africa.

Methods

We conducted a scoping review according to the Joanne Briggs Institute's reviewers manual and followed the Preferred Reporting Items for Systematic Reviews and Meta-Analyses - Extension for Scoping Reviews (PRISMA-ScR) checklist and explanation. We retrieved data from the WHO Digital Health Atlas (DHA), the WHO e-Health country profiles report of 2015, and electronic databases. The protocol has been deposited in an open-source platform - the Open Science Framework at https://osf.io/5kzq7.

Results

The researchers retrieved 983 digital tools used to strengthen health systems in sub-Saharan Africa over the past 10 years. We included 738 DHIs in the analysis while 245 were excluded for not meeting the inclusion criteria. We observed a disproportionate distribution of DHIs towards service delivery (81.7%, n = 603), health care providers (91.8%, n = 678), and access and use of information (84.1%, n = 621). Fifty-three percent (53.4%, n = 394) of the solutions are established and 47.5% (n = 582) were aligned to 20% (n = 5) of the system categories.

Conclusions

Sub-Saharan Africa is endowed with digital health solutions in both numbers and distinct functions. It is lacking in coordination, integration, scalability, sustainability, and equitable distribution of investments in digital health. Digital health policymakers in sub-Saharan Africa need to urgently institute coordination mechanisms to terminate unending duplication and disjointed vertical implementations and manage solutions for scale. Central to this would be to build digital health leadership in countries within SSA, adopt standards and interoperability frameworks; advocate for more investments into lagging components, and promote multi-purpose solutions to halt the seeming "e-chaos" and progress to sustainable e-health solutions.

A Systematic Review of Artificial Intelligence Applications in Plastic Surgery: Looking to the Future

Artificial intelligence (AI) is presently employed in several medical specialties, particularly those that rely on large quantities of standardized data. The integration of AI in surgical subspecialties is under preclinical investigation but is yet to be widely implemented. Plastic surgeons collect standardized data in various settings and could benefit from AI. This systematic review investigates the current clinical applications of AI in plastic and reconstructive surgery.

METHODS

A comprehensive literature search of the Medline, EMBASE, Cochrane, and PubMed databases was conducted for AI studies with multiple search terms. Articles that progressed beyond the title and abstract screening were then subcategorized based on the plastic surgery subspecialty and AI application.

RESULTS

The systematic search yielded a total of 1820 articles. Forty-four studies met inclusion criteria warranting further analysis. Subcategorization of articles by plastic surgery subspecialties revealed that most studies fell into aesthetic and breast surgery (27%), craniofacial surgery (23%), or microsurgery (14%). Analysis of the research study phase of included articles indicated that the current research is primarily in phase 0 (discovery and invention; 43.2%), phase 1 (technical performance and safety; 27.3%), or phase 2 (efficacy, quality improvement, and algorithm performance in a medical setting; 27.3%). Only one study demonstrated translation to clinical practice.

CONCLUSIONS

The potential of AI to optimize clinical efficiency is being investigated in every subfield of plastic surgery, but much of the research to date remains in the preclinical status. Future implementation of AI into everyday clinical practice will require collaborative efforts.

The Role of Academic Health Systems in Leading the "Third Wave" of Digital Health Innovation

Investors, entrepreneurs, health care pundits, and venture capital firms all agree that the health care sector is awaiting a digital revolution. Steven Case, in 2016, predicted a "third wave" of innovation that would leverage big data, artificial intelligence, and machine learning to transform medicine and finally achieve reduced costs, improved efficiency, and better patient outcomes. Academic medical centers (AMCs) have the infrastructure and resources needed by digital health intrapreneurs and entrepreneurs to innovate, iterate, and optimize technology solutions for the major pain points of modern medicine. With large unique patient data sets, strong research programs, and subject matter experts, AMCs have the ability to assess, optimize, and integrate new digital health tools with feedback at the point of care and research-based clinical validation. As AMCs begin to explore digital health solutions, they must decide between forming internal teams to develop these innovations or collaborating with external companies. Although each has its drawbacks and benefits, AMCs can both benefit from and drive forward the digital health innovations that will result from this journey. This viewpoint will provide an explanation as to why AMCs are ideal incubators for digital health solutions and describe what these organizations will need to be successful in leading this "third wave" of innovation.

Advancing digital health Equity: Directions for behavioral and social science research

The field of digital health is evolving rapidly and encompasses a wide range of complex and changing technologies used to support individual and population health. The COVID-19 pandemic has augmented digital health expansion and significantly changed how digital health technologies are used. To ensure that these technologies do not create or exacerbate existing health disparities, a multi-pronged and comprehensive research approach is needed. In this commentary, we outline five recommendations for behavioral and social science researchers that are critical to promoting digital health equity. These recommendations include: (i) centering equity in research teams and theoretical approaches, (ii) focusing on issues of digital health literacy and engagement, (iii) using methods that elevate perspectives and needs of underserved populations, (iv) ensuring ethical approaches for collecting and using digital health data, and (v) developing strategies for integrating digital health tools within and across systems and settings. Taken together, these recommendations can help advance the science of digital health equity and justice.