Beyond Adoption: A New Framework for Theorizing and Evaluating Nonadoption, Abandonment, and Challenges to the Scale-Up, Spread, and Sustainability of Health and Care Technologies

"Let's put it this way: you can't really live without it" - digital technologies in routine palliative care delivery: an explorative qualitative study with patients and their family caregivers in Germany

BACKGROUND

Despite ongoing efforts to integrate palliative care into the German healthcare system, challenges persist, particularly in areas where infrastructure does not fully support digital technologies (DT). The increasing importance of digital technology (DT) in palliative care delivery presents both opportunities and challenges.

OBJECTIVE

This study aimed to explore the perspectives and preferences of palliative care patients and their family caregivers regarding the use of DT in care delivery.

METHODS

An exploratory qualitative study was conducted using semi-structured interviews with palliative care patients and their family caregivers across various settings. Participants were selected through gatekeeper-supported purposive sampling. Interviews were analysed using structured qualitative content analysis.

RESULTS

Nineteen interviews were conducted.Three themes emerged: (1) Application of DTs in palliative care; (2) Potential of DTs; (3) Barriers to the use of DTs. Key findings highlighted the preference for real-time communication using DTs that participants are familiar with. Participants reported limited perceived value for digital transformation in the presence of in-person care. The study identified requirements for DT development and use in palliative care, including the need for direct and immediate functionality, efficiency in healthcare professional (HCP) work, and continuous access to services.

CONCLUSION

The findings highlight a demonstrate the importance of familiarity with DTs and real-time access for patients and their families. While DT can enhance palliative care efficiency and accessibility, its integration must complement, not replace, in-person interaction in palliative care. As DTs continue to grow in scope and use in palliative care, maintaining continued user engagement is essential to optimise their adoption and ensure they benefit patients and their caregivers.

Understanding the integration of artificial intelligence in healthcare organisations and systems through the NASSS framework: a qualitative study in a leading Canadian academic centre

BACKGROUND

Artificial intelligence (AI) technologies are expected to "revolutionise" healthcare. However, despite their promises, their integration within healthcare organisations and systems remains limited. The objective of this study is to explore and understand the systemic challenges and implications of their integration in a leading Canadian academic hospital.

METHODS

Semi-structured interviews were conducted with 29 stakeholders concerned by the integration of a large set of AI technologies within the organisation (e.g., managers, clinicians, researchers, patients, technology providers). Data were collected and analysed using the Non-Adoption, Abandonment, Scale-up, Spread, Sustainability (NASSS) framework.

RESULTS

Among enabling factors and conditions, our findings highlight: a supportive organisational culture and leadership leading to a coherent organisational innovation narrative; mutual trust and transparent communication between senior management and frontline teams; the presence of champions, translators, and boundary spanners for AI able to build bridges and trust; and the capacity to attract technical and clinical talents and expertise. Constraints and barriers include: contrasting definitions of the value of AI technologies and ways to measure such value; lack of real-life and context-based evidence; varying patients' digital and health literacy capacities; misalignments between organisational dynamics, clinical and administrative processes, infrastructures, and AI technologies; lack of funding mechanisms covering the implementation, adaptation, and expertise required; challenges arising from practice change, new expertise development, and professional identities; lack of official professional, reimbursement, and insurance guidelines; lack of pre- and post-market approval legal and governance frameworks; diversity of the business and financing models for AI technologies; and misalignments between investors' priorities and the needs and expectations of healthcare organisations and systems.

CONCLUSION

Thanks to the multidimensional NASSS framework, this study provides original insights and a detailed learning base for analysing AI technologies in healthcare from a thorough socio-technical perspective. Our findings highlight the importance of considering the complexity characterising healthcare organisations and systems in current efforts to introduce AI technologies within clinical routines. This study adds to the existing literature and can inform decision-making towards a judicious, responsible, and sustainable integration of these technologies in healthcare organisations and systems.

Dropout rate in clinical trials of smartphone apps for diabetes management: A meta-analysis

Applicability of smartphone-based digital health in diabetes management still face challenges due tolow user retention or engagement. Thus, this systematic and meta-analysis aimed to estimate the dropout rate from the clinical trials. Search of literature was performedon 4 September 2023 through various databases (PubMed, Scilit, Scopus, Embase, and Web of Science). Those reporting clinical trials of smartphone apps for diabetic controls (either type 1 or type 2 diabetes mellitus) were screened and selected in accordance with PRISMA guideline. Of 5,429 identified records, as many as 36 studies were found eligible with a total of 3,327 patients in the intervention group. The overall dropout rate was 29.6 % (95 %CI: 25 %-34.3 %) with high heterogeneity (p-Het < 0.001;I2 = 84.84 %). Sample size, intervention duration, patients' age and gender, and cultural adaptation on the app appeared to be non-significant moderators (p > 0.05). In sub-group levels, notably high dropout rates were observed in studies performing cultural adaptation (34.6 %) and conducted in high-income countries (31.9 %). Given the high dropout rate, the engagement level toward diabetic management apps in real-world setting is expected to be low. High heterogeneity in this study, however, requires careful interpretation of the foregoing results. PROSPERO: CRD42023460365 (14 September 2023).

Applying the Non-adoption, Abandonment, Scale-up, Spread and Sustainability (NASSS) framework to evaluate automated evidence synthesis in health behaviour change

Automated tools to speed up the process of evidence synthesis are increasingly apparent within health behaviour research. This brief review explores the potential of the Non-adoption, Abandonment, Scale-up, Spread and Sustainability framework for supporting automated evidence synthesis in health behaviour change by applying it to the ongoing Human Behaviour-Change Project, which aims to revolutionize evidence synthesis within behaviour change intervention research. To increase the relevance of NASSS for health behaviour change, we recommend i) terminology changes ('condition' to 'behaviour' and 'patient' to 'end user') and ii) that it is used prospectively address complexities iteratively. We draw conclusions about i) the need to specify the organizations that will use the technology, ii) identifying what to do if interdependencies fail and iii) even though we have focused on automated evidence synthesis, NASSS would arguably be beneficial for technology developments in health behaviour change more generally, particularly for invention development.

Service innovations for people with multiple long-term conditions: reflections of a rapid evaluation team

Background

People living with multiple long-term conditions represent a significant concern for National Health Service policy and practice, and their care is a major theme in the 2019 National Health Service Long Term Plan. The Birmingham RAND and Cambridge Rapid Evaluation Centre team has undertaken a thematic synthesis of the 10 evaluations it has conducted from 2018 to 2023, exploring the needs, priorities and implications for people with multiple long-term conditions.

Objectives

The aims for this overarching study were to: (1) build a body of learning about service innovations in primary and community settings for people of all ages with multiple long-term conditions, focused on questions that matter most to people with multimorbidity; and (2) develop methodological insights about how rapid evaluation can be used to inform the scoping, testing and implementation of service innovations for people with multiple long-term conditions.

Design

The focus on multiple long-term conditions came from a Birmingham RAND and Cambridge Rapid Evaluation Centre prioritisation process undertaken in 2018 using James Lind Alliance methods. Cross-analysis of the findings from the 10 individual rapid evaluations was supplemented by (1) building aspects of multimorbidity into the design of later evaluations; (2) interviewing national and regional stakeholders (n=19) working in or alongside integrated care systems; (3) undertaking a rapid review of evidence on remote monitoring for people with multiple long-term conditions (19 papers included); and (4) testing overall insights with organisations representing patients and carers through a patient, public and professional engagement workshop with 10 participants plus members of the research team.

Results

While living with multiple long-term conditions is common and is the norm for people over the age of 50 using health and care services, it is not often a focus of health service provision or innovation, nor of research and evaluation activity. We discuss six themes emerging from the totality of the study: (1) our health system is mainly organised around single conditions and not multiple long-term conditions; (2) research calls and studies usually focus on single conditions and associated services; (3) building opportunities for engaged, informed individuals and carers and improved self-management; (4) the importance of measures that matter for patients and carers; (5) barriers to developing and implementing service innovations for people with multiple long-term conditions; and (6) what is needed to make patients with multiple long-term conditions a priority in healthcare planning and delivery.

Limitations

Care of people with multiple long-term conditions was not the principal focus of several of the rapid evaluations. While this was a finding in itself, it limited our learning about designing and implementing, as well as methodological approaches to evaluating, service innovations for people with multiple long-term conditions.

Conclusions

Through a thematic analysis of the portfolio of evaluations, we have deduced a set of suggested implications for how the needs of people with multiple long-term conditions can be better embedded in policy, research and practice.

Future work

Areas of uncertainty related to the care of people with multiple long-term conditions should be further explored, including developing and testing measures of patient experience of (un)co-ordinated care across settings, and interrogating the experience of health and care staff when working with people with multiple long-term conditions, to understand what works.

Funding

This award was funded by the National Institute for Health and Care Research (NIHR) Health and Social Care Delivery Research programme (NIHR award ref: NIHR134284) and is published in full in Health and Social Care Delivery Research; Vol. 12, No. 15. See the NIHR Funding and Awards website for further award information.

DIGIPREDICT: physiological, behavioural and environmental predictors of asthma attacks-a prospective observational study using digital markers and artificial intelligence-study protocol

INTRODUCTION

Asthma attacks are a leading cause of morbidity and mortality but are preventable in most if detected and treated promptly. However, the changes that occur physiologically and behaviourally in the days and weeks preceding an attack are not always recognised, highlighting a potential role for technology. The aim of this study 'DIGIPREDICT' is to identify early digital markers of asthma attacks using sensors embedded in smart devices including watches and inhalers, and leverage health and environmental datasets and artificial intelligence, to develop a risk prediction model to provide an early, personalised warning of asthma attacks.

METHODS AND ANALYSIS

A prospective sample of 300 people, 12 years or older, with a history of a moderate or severe asthma attack in the last 12 months will be recruited in New Zealand. Each participant will be given a smart watch (to assess physiological measures such as heart and respiratory rate), peak flow meter, smart inhaler (to assess adherence and inhalation) and a cough monitoring application to use regularly over 6 months with fortnightly questionnaires on asthma control and well-being. Data on sociodemographics, asthma control, lung function, dietary intake, medical history and technology acceptance will be collected at baseline and at 6 months. Asthma attacks will be measured by self-report and confirmed with clinical records. The collected data, along with environmental data on weather and air quality, will be analysed using machine learning to develop a risk prediction model for asthma attacks.

ETHICS AND DISSEMINATION

Ethical approval has been obtained from the New Zealand Health and Disability Ethics Committee (2023 FULL 13541). Enrolment began in August 2023. Results will be presented at local, national and international meetings, including dissemination via community groups, and submission for publication to peer-reviewed journals.

TRIAL REGISTRATION NUMBER

Australian New Zealand Clinical Trials Registry ACTRN12623000764639; Australian New Zealand Clinical Trials Registry.

Evaluation of usability and user feedback to guide telepharmacy application development in Indonesia: a mixed-methods study

BACKGROUND

In Indonesia, the adoption of telepharmacy was propelled by the COVID-19 pandemic, prompting the need for a user-friendly application to support both the general population and pharmacists in accessing healthcare services. Therefore, this study aimed to evaluate usability and user feedback of a pioneering telepharmacy application known as Tanya Obat (translating to "Ask about Medications") in Indonesia, from the perspectives of the general population and pharmacists.

METHODS

A mixed-methods sequential study was conducted with the early-stage Tanya Obat application in Bandung City. Participants, including the general population and pharmacists, were instructed to use the application for a week. Questionnaires for the general population and pharmacists were distributed from March to May and February to June 2023, respectively. The System Usability Scale questionnaire was adopted to describe usability of the developed application. Further exploration of the quantitative results required collecting open-ended feedback to assess the impressions of the participants, difficulties encountered, and desired features for enhanced user-friendliness. The collected statements were summarized and clustered using thematic analysis. Subsequently, the association between the characteristics of participants and perceived usability was determined with the Chi-square test.

RESULT

A total of 176 participants, comprising 100 individuals from the general population and 76 pharmacists, engaged in this study. In terms of usability, the questionnaire showed that Tanya Obat application was on the borderline of acceptability, with mean scores of 63.4 and 64.1 from the general population and pharmacists, respectively. Additionally, open-ended feedback targeted at achieving a more compelling user experience was categorized into two themes, including concerns regarding the functionality of certain features and recommendations for improved visual aesthetics and bug fixes. No significant associations were observed between the characteristics of participants and perceived usability (p-value > 0.05).

CONCLUSION

The results showed that the perceived usability of Tanya Obat developed for telepharmacy was below average. Therefore, feature optimizations should be performed to facilitate usability of this application in Indonesia.

"Notification! You May Have Cancer." Could Smartphones and Wearables Help Detect Cancer Early?

This viewpoint paper considers the authors' perspectives on the potential role of smartphones, wearables, and other technologies in the diagnosis of cancer. We believe that these technologies could be valuable additions in the pursuit of early cancer diagnosis, as they offer solutions to the timely detection of signals or symptoms and monitoring of subtle changes in behavior that may otherwise be missed. In addition to signal detection, technologies could assist symptom interpretation and guide and facilitate access to health care. This paper aims to provide an overview of the scientific rationale as to why these technologies could be valuable for early cancer detection, as well as outline the next steps for research and development to drive investigation into the potential for smartphones and wearables in this context and optimize implementation. We draw attention to potential barriers to successful implementation, including the difficulty of the development of signals and sensors with sufficient utility and accuracy through robust research with the target group. There are regulatory challenges; the potential for innovations to exacerbate inequalities; and questions surrounding acceptability, uptake, and correct use by the intended target group and health care practitioners. Finally, there is potential for unintended consequences on individuals and health care services including unnecessary anxiety, increased symptom burden, overinvestigation, and inappropriate use of health care resources.

Therapies go digital. What drives physicians' acceptance?

National healthcare systems face multiple challenges, including the increasing demand for care and decreasing availability of healthcare professionals. Digital health technologies represent opportunities that offer improved efficiency, accessibility, and patient care. In this scenario, Digital Therapeutics are technological advancements to treat or alleviate a disease and deliver a medical intervention with evidence-based therapeutic impacts and regulatory approval. Digital Therapeutics are a paradigm shift for physicians, who exercise caution in terms of trust and wide usage. Digital Therapeutics represents an opportunity and a challenge in healthcare system integration. The research investigates the factors explaining physicians' acceptance of Digital Therapeutics. A research model that combines organizational mechanisms derived from Institutional Theory and rational factors derived from the Technology Acceptance model was developed. The model was tested through 107 responses from a survey distributed to the members of the leading Italian scientific society in Diabetology. Literature-based hypotheses were empirically tested through Structural Equation Modelling. The analysis confirmed the influence of Perceived Ease of Use on Perceived Usefulness and Perceived Usefulness on the Intention To Use Digital Therapeutics. Rules and norms impact Perceived Usefulness when considering the influence of the scientific society. Culture and mindset towards innovation within the hospital positively affect Perceived Ease of Use. The readiness of hospital facilities enhances the extent to which physicians perceive the ease of employing Digital Therapeutics in their daily practice. Instead, esteemed colleagues' opinions and guidelines from the scientific society reveal to physicians the value of Digital Therapeutics in patients' care pathways. Institutions should prioritize cultural, normative, and regulative aspects to accelerate physicians' endorsement of Digital Therapeutics. Findings advance the theoretical knowledge around clinicians' adoption of innovative digital health technologies, unveiling the interaction between rational and institutional factors. The results highlight practical implications for healthcare institutions and Digital Therapeutics manufacturers willing to promote their adoption.

Veteran Experiences With an mHealth App to Support Measurement-Based Mental Health Care: Results From a Mixed Methods Evaluation

BACKGROUND

Mental health conditions are highly prevalent among US veterans. The Veterans Health Administration (VHA) is committed to enhancing mental health care through the integration of measurement-based care (MBC) practices, guided by its Collect-Share-Act model. Incorporating the use of remote mobile apps may further support the implementation of MBC for mental health care.

OBJECTIVE

This study aims to evaluate veteran experiences with Mental Health Checkup (MHC), a VHA mobile app to support remote MBC for mental health.

METHODS

Our mixed methods sequential explanatory evaluation encompassed mailed surveys with veterans who used MHC and follow-up semistructured interviews with a subset of survey respondents. We analyzed survey data using descriptive statistics. We then compared responses between veterans who indicated having used MHC for ≥3 versus <3 months using χ2 tests. We analyzed interview data using thematic analysis.

RESULTS

We received 533 surveys (533/2631, for a 20% response rate) and completed 20 interviews. Findings from these data supported one another and highlighted 4 key themes. (1) The MHC app had positive impacts on care processes for veterans: a majority of MHC users overall, and a greater proportion who had used MHC for ≥3 months (versus <3 months), agreed or strongly agreed that using MHC helped them be more engaged in their health and health care (169/262, 65%), make decisions about their treatment (157/262, 60%), and set goals related to their health and health care (156/262, 60%). Similarly, interviewees described that visualizing progress through graphs of their assessment data over time motivated them to continue therapy and increased self-awareness. (2) A majority of respondents overall, and a greater proportion who had used MHC for ≥3 months (versus <3 months), agreed/strongly agreed that using MHC enhanced their communication (112/164, 68% versus 51/98, 52%; P=.009) and rapport (95/164, 58% versus 42/98, 43%; P=.02) with their VHA providers. Likewise, interviewees described how MHC helped focus therapy time and facilitated trust. (3) However, veterans also endorsed some challenges using MHC. Among respondents overall, these included difficulty understanding graphs of their assessment data (102/245, 42%), not receiving enough training on the app (73/259, 28%), and not being able to change responses to assessment questions (72/256, 28%). (4) Interviewees offered suggestions for improving the app (eg, facilitating ease of log-in, offering additional reminder features) and for increasing adoption (eg, marketing the app and its potential advantages for veterans receiving mental health care).

CONCLUSIONS

Although experiences with the MHC app varied, veterans were positive overall about its use. Veterans described associations between the use of MHC and engagement in their own care, self-management, and interactions with their VHA mental health providers. Findings support the potential of MHC as a technology capable of supporting the VHA's Collect-Share-Act model of MBC.

Digital Mental Health for Schizophrenia and Other Severe Mental Illnesses: An International Consensus on Current Challenges and Potential Solutions

BACKGROUND

Digital approaches may be helpful in augmenting care to address unmet mental health needs, particularly for schizophrenia and severe mental illness (SMI).

OBJECTIVE

An international multidisciplinary group was convened to reach a consensus on the challenges and potential solutions regarding collecting data, delivering treatment, and the ethical challenges in digital mental health approaches for schizophrenia and SMI.

METHODS

The consensus development panel method was used, with an in-person meeting of 2 groups: the expert group and the panel. Membership was multidisciplinary including those with lived experience, with equal participation at all stages and coproduction of the consensus outputs and summary. Relevant literature was shared in advance of the meeting, and a systematic search of the recent literature on digital mental health interventions for schizophrenia and psychosis was completed to ensure that the panel was informed before the meeting with the expert group.

RESULTS

Four broad areas of challenge and proposed solutions were identified: (1) user involvement for real coproduction; (2) new approaches to methodology in digital mental health, including agreed standards, data sharing, measuring harms, prevention strategies, and mechanistic research; (3) regulation and funding issues; and (4) implementation in real-world settings (including multidisciplinary collaboration, training, augmenting existing service provision, and social and population-focused approaches). Examples are provided with more detail on human-centered research design, lived experience perspectives, and biomedical ethics in digital mental health approaches for SMI.

CONCLUSIONS

The group agreed by consensus on a number of recommendations: (1) a new and improved approach to digital mental health research (with agreed reporting standards, data sharing, and shared protocols), (2) equal emphasis on social and population research as well as biological and psychological approaches, (3) meaningful collaborations across varied disciplines that have previously not worked closely together, (4) increased focus on the business model and product with planning and new funding structures across the whole development pathway, (5) increased focus and reporting on ethical issues and potential harms, and (6) organizational changes to allow for true communication and coproduction with those with lived experience of SMI. This study approach, combining an international expert meeting with patient and public involvement and engagement throughout the process, consensus methodology, discussion, and publication, is a helpful way to identify directions for future research and clinical implementation in rapidly evolving areas and can be combined with measurements of real-world clinical impact over time. Similar initiatives will be helpful in other areas of digital mental health and similarly fast-evolving fields to focus research and organizational change and effect improved real-world clinical implementation.

Implementation of digital health technologies for older adults: a scoping review

The critical importance of technological innovation in home care for older adults is indisputable. Less well understood is the question of how to measure its performance and impact on the delivery of healthcare to older adults who are living with chronic illness and disability. Knowing how well digital technologies, such as smartphones, tablets, wearable devices, and Ambient Assisted Living Technologies (AAL) systems "work" should certainly include assessing their impact on older adults' health and ability to function in daily living but that will not guarantee that it will necessarily be adopted by the user or implemented by a healthcare facility or the healthcare system. Technology implementation is a process of planned and guided activities to launch, introduce and support technologies in a certain context to innovate or improve healthcare, which delivers the evidence for adoption and upscaling a technology in healthcare practices. Factors in addition to user acceptance and clinical effectiveness require investigation. Failure to appreciate these factors can result in increased likelihood of technology rejection or protracted procurement decision at the "adoption decision" stage or delayed or incomplete implementation or discontinuance (following initial adoption) during implementation. The aim of our research to analyze research studies on the effectiveness of digital health technologies for older adults to answer the question, "How well do these studies address factors that affect the implementation of technology?" We found common problems with the conceptualization, design, and methodology in studies of digital technology that have contributed to the slow pace of implementation in home care and long-term care. We recommend a framework for improving the quality of research in this critical area. Systematic Review Registration: https://archive.org/details/osf-registrations-f56rb-v1, identifier osf-registrations-f56rb-v1.

Telehealth for Indigenous Children Worldwide: A Scoping Review

PURPOSE

Indigenous children worldwide face healthcare disparities due, in part, to resource scarcity in remote settings which may be mitigated with technology. This study aims to determine the use of telehealth for this population, with respect to feasibility, acceptability, and the degree of patient/family involvement in reported interventions. We focused on the use of telehealth to support perioperative care.

METHODS

To identify relevant studies, five databases were searched to find articles that focused on the role of telehealth in caring for Indigenous populations worldwide, with an emphasis on the pediatric population. Studies that lacked insight into those themes, as well as protocols and review articles, were excluded. Analysis was done according to the non-adoption, abandonment, scale-up, spread, and sustainability (NASSS) framework, the Montreal Model (patient involvement), and the theoretical framework of acceptability (TFA).

RESULTS

Of the 1690 articles screened, 34 met the eligibility criteria. The most frequent uses of telehealth for Indigenous children were in ENT and psychiatry. Most of those had a low degree of complexity across the NASSS framework domains, suggesting greater feasibility. In 13 articles, the patient involvement was limited to information (lowest level of involvement in the Montreal Model). Only 11 articles directly assessed patient/family-perceived acceptability. Finally, two articles addressed telehealth in the surgical context.

CONCLUSIONS

The relative simplicity of the proposed telehealth applications may support their sustained impact and use in other settings such as for perioperative care. Early and longitudinal involvement of communities is essential for responsible telehealth development that addresses local needs.

LEVEL OF EVIDENCE

Level V.

Blended care in psychosis - A systematic review

Blended-care, a psychosocial intervention combining traditional, face-to-face therapy with digital mental health tools, has shown potential for improving therapeutic processes, fostering patient engagement, and augmenting clinical outcomes. This systematic review aimed to evaluate the development and effectiveness of blended-care interventions tailored for adults diagnosed with schizophrenia-spectrum disorders or other conditions with psychotic features. Our search strategy spanned three electronic databases (PsycINFO, Web of Science, and PubMed) in accordance with the reporting guidelines outlined by the Preferred Reporting Items for Systematic Reviews and Meta-Analyses. We identified 11 papers, each examining the use of one of eight distinct blended-care interventions. Significantly, the majority of these papers (10/11, 91 %) examined these interventions exclusively within controlled research environments, demonstrating both acceptability and favourable impacts on symptomatology and recovery (e.g., 0.22 ≤ Cohen's ds ≤ 1.00). Only one intervention was examined in research settings and real-world conditions, and the shift resulted in low real-world uptake (e.g., only 50 % of practitioners were able to engage at least one of their clients with the intervention) and an inability to reproduce positive changes in clinical outcomes. Additional research is needed to determine the viability of successfully developing and implementing blended-care interventions for psychosis in real-world conditions. An exploration of the developmental processes that could facilitate the transition from research settings to routine clinical practice is vital.

Electronic Collection of Patient-Reported Outcomes to Improve Kidney Care: Benefits, Drawbacks, and Next Steps

Kidney services worldwide are increasingly using digital health technologies to deliver care. This includes kidney electronic patient-reported outcome (ePRO) systems: ambulatory digital technologies that enable the capture of PRO data electronically from people with kidney disease remotely and in real time to be shared with their kidney care team. Current kidney ePRO systems commonly aim to support the monitoring and management of symptoms in patients with kidney disease. The majority have thus far only been implemented in research settings and are not yet routinely used in clinical practice, leaving their readiness for real-world implementation largely unknown. Compared with paper-based PRO collection, ePRO systems have certain advantages, which we categorize as efficiency benefits (e.g., lower administrative burden), direct patient care benefits (e.g., automated PRO-based patient education), and health system and research benefits (e.g., collecting ePRO data once for multiple purposes). At the same time, kidney ePRO systems come with drawbacks, such as their potential to exacerbate existing inequities in care and outcomes and to negatively affect staff burden and patients' experience of kidney care. Areas that hold promise for expediting the development and uptake of kidney ePRO systems at the local, organizational, and national level include harnessing national kidney registries as enabling infrastructures; using novel data-driven technologies (e.g., computerized adaptive test systems, configurable dashboards); applying implementation science and action research approaches to enhance translation of ePRO research findings into clinical practice; and engaging stakeholders, including patients and carers, health care professionals, policymakers, payers, ePRO experts, technology providers, and organizations that monitor and improve the quality of kidney services.

Understanding the Barriers and Facilitators of Digital Health Technology (DHT) Implementation in Neurological Rehabilitation: An Integrative Systematic Review

Background

Digital Health technologies (DHT) have potential to deliver intensive, novel and engaging rehabilitation for people with neurological conditions, yet health services lack a strong track record in embedding DHT into practice. The aim of this review was to synthesise factors that have been shown to influence implementation of DHT into neurological rehabilitation.

Method

An integrative review was undertaken. An extensive search of MEDLINE, CINAHL, AMED, EMBASE was undertaken. The title and abstract of all retrieved sources were screened against pre-defined criteria. Retained sources underwent full text review. The quality of all included sources was assessed. A meta-ethnographic synthesis explored commonalities and contradictions of the included studies.

Results

Fourteen studies (1 quantitative, 8 qualitative and 5 mixed methods) were included. Eleven implementation theories/models/frameworks were used across the 14 studies. Five themes were identified: (i) individual factors; (ii) user experience of the technology; (iii) the content of the intervention; (iv) access to the technology and (v) supporting use.

Conclusions

Key factors which appear to influence the implementation of DHT into clinical settings are highlighted. Implementation theories, models and frameworks are under-utilised in DHT rehabilitation research. This needs to be addressed if DHT are to realise their potential in neurological rehabilitation.

Registration

The protocol was registered and is available from PROSPERO (CRD42021268984).

Exploring the Role of Complexity in Health Care Technology Bottom-Up Innovations: Multiple-Case Study Using the Nonadoption, Abandonment, Scale-Up, Spread, and Sustainability Complexity Assessment Tool

BACKGROUND

New digital technology presents new challenges to health care on multiple levels. There are calls for further research that considers the complex factors related to digital innovations in complex health care settings to bridge the gap when moving from linear, logistic research to embracing and testing the concept of complexity. The nonadoption, abandonment, scale-up, spread, and sustainability (NASSS) framework was developed to help study complexity in digital innovations.

OBJECTIVE

This study aims to investigate the role of complexity in the development and deployment of innovations by retrospectively assessing challenges to 4 digital health care innovations initiated from the bottom up.

METHODS

A multicase retrospective, deductive, and explorative analysis using the NASSS complexity assessment tool LONG was conducted. In total, 4 bottom-up innovations developed in Region Västra Götaland in Sweden were explored and compared to identify unique and shared complexity-related challenges.

RESULTS

The analysis resulted in joint insights and individual learning. Overall, the complexity was mostly found outside the actual innovation; more specifically, it related to the organization's readiness to integrate new innovations, how to manage and maintain innovations, and how to finance them. The NASSS framework sheds light on various perspectives that can either facilitate or hinder the adoption, scale-up, and spread of technological innovations. In the domain of condition or diagnosis, a well-informed understanding of the complexity related to the condition or illness (diabetes, cancer, bipolar disorders, and schizophrenia disorders) is of great importance for the innovation. The value proposition needs to be clearly described early to enable an understanding of costs and outcomes. The questions in the NASSS complexity assessment tool LONG were sometimes difficult to comprehend, not only from a language perspective but also due to a lack of understanding of the surrounding organization's system and its setting.

CONCLUSIONS

Even when bottom-up innovations arise within the same support organization, the complexity can vary based on the developmental phase and the unique characteristics of each project. Identifying, defining, and understanding complexity may not solve the issues but substantially improves the prospects for successful deployment. Successful innovation within complex organizations necessitates an adaptive leadership and structures to surmount cultural resistance and organizational impediments. A rigid, linear, and stepwise approach risks disregarding interconnected variables and dependencies, leading to suboptimal outcomes. Success lies in embracing the complexity with its uncertainty, nurturing creativity, and adopting a nonlinear methodology that accommodates the iterative nature of innovation processes within complex organizations.

Patient and Staff Experience of Remote Patient Monitoring-What to Measure and How: Systematic Review

BACKGROUND

Patient and staff experience is a vital factor to consider in the evaluation of remote patient monitoring (RPM) interventions. However, no comprehensive overview of available RPM patient and staff experience-measuring methods and tools exists.

OBJECTIVE

This review aimed at obtaining a comprehensive set of experience constructs and corresponding measuring instruments used in contemporary RPM research and at proposing an initial set of guidelines for improving methodological standardization in this domain.

METHODS

Full-text papers reporting on instances of patient or staff experience measuring in RPM interventions, written in English, and published after January 1, 2011, were considered for eligibility. By "RPM interventions," we referred to interventions including sensor-based patient monitoring used for clinical decision-making; papers reporting on other kinds of interventions were therefore excluded. Papers describing primary care interventions, involving participants under 18 years of age, or focusing on attitudes or technologies rather than specific interventions were also excluded. We searched 2 electronic databases, Medline (PubMed) and EMBASE, on February 12, 2021.We explored and structured the obtained corpus of data through correspondence analysis, a multivariate statistical technique.

RESULTS

In total, 158 papers were included, covering RPM interventions in a variety of domains. From these studies, we reported 546 experience-measuring instances in RPM, covering the use of 160 unique experience-measuring instruments to measure 120 unique experience constructs. We found that the research landscape has seen a sizeable growth in the past decade, that it is affected by a relative lack of focus on the experience of staff, and that the overall corpus of collected experience measures can be organized in 4 main categories (service system related, care related, usage and adherence related, and health outcome related). In the light of the collected findings, we provided a set of 6 actionable recommendations to RPM patient and staff experience evaluators, in terms of both what to measure and how to measure it. Overall, we suggested that RPM researchers and practitioners include experience measuring as part of integrated, interdisciplinary data strategies for continuous RPM evaluation.

CONCLUSIONS

At present, there is a lack of consensus and standardization in the methods used to measure patient and staff experience in RPM, leading to a critical knowledge gap in our understanding of the impact of RPM interventions. This review offers targeted support for RPM experience evaluators by providing a structured, comprehensive overview of contemporary patient and staff experience measures and a set of practical guidelines for improving research quality and standardization in this domain.

Integrating Explainable Machine Learning in Clinical Decision Support Systems: Study Involving a Modified Design Thinking Approach

BACKGROUND

Though there has been considerable effort to implement machine learning (ML) methods for health care, clinical implementation has lagged. Incorporating explainable machine learning (XML) methods through the development of a decision support tool using a design thinking approach is expected to lead to greater uptake of such tools.

OBJECTIVE

This work aimed to explore how constant engagement of clinician end users can address the lack of adoption of ML tools in clinical contexts due to their lack of transparency and address challenges related to presenting explainability in a decision support interface.

METHODS

We used a design thinking approach augmented with additional theoretical frameworks to provide more robust approaches to different phases of design. In particular, in the problem definition phase, we incorporated the nonadoption, abandonment, scale-up, spread, and sustainability of technology in health care (NASSS) framework to assess these aspects in a health care network. This process helped focus on the development of a prognostic tool that predicted the likelihood of admission to an intensive care ward based on disease severity in chest x-ray images. In the ideate, prototype, and test phases, we incorporated a metric framework to assess physician trust in artificial intelligence (AI) tools. This allowed us to compare physicians' assessments of the domain representation, action ability, and consistency of the tool.

RESULTS

Physicians found the design of the prototype elegant, and domain appropriate representation of data was displayed in the tool. They appreciated the simplified explainability overlay, which only displayed the most predictive patches that cumulatively explained 90% of the final admission risk score. Finally, in terms of consistency, physicians unanimously appreciated the capacity to compare multiple x-ray images in the same view. They also appreciated the ability to toggle the explainability overlay so that both options made it easier for them to assess how consistently the tool was identifying elements of the x-ray image they felt would contribute to overall disease severity.

CONCLUSIONS

The adopted approach is situated in an evolving space concerned with incorporating XML or AI technologies into health care software. We addressed the alignment of AI as it relates to clinician trust, describing an approach to wire framing and prototyping, which incorporates the use of a theoretical framework for trust in the design process itself. Moreover, we proposed that alignment of AI is dependent upon integration of end users throughout the larger design process. Our work shows the importance and value of engaging end users prior to tool development. We believe that the described approach is a unique and valuable contribution that outlines a direction for ML experts, user experience designers, and clinician end users on how to collaborate in the creation of trustworthy and usable XML-based clinical decision support tools.

Implementation of Video-Based Care in Interdisciplinary Primary Care Settings at the Veterans Health Administration: Qualitative Study

BACKGROUND

With the rapid shift to telehealth, there remains a knowledge gap in how video-based care is implemented in interdisciplinary primary care (PC) settings.

OBJECTIVE

The objective of this study was to gain an in-depth understanding of how video telehealth services were implemented in PC from the perspectives of patients and interdisciplinary PC team members at the Veterans Health Administration (VHA) 2 years after the onset of the COVID-19 pandemic.

METHODS

We applied a positive and negative deviance approach and selected the 6% highest (n=8) and the 6% lowest (n=8) video-using PC sites in 2022 from a total of 130 VHA medical centers nationally. A total of 12 VHA sites were included in the study, where 43 PC interdisciplinary team members (August-October 2022) and 25 patients (February-May 2023) were interviewed. The 5 domains from the diffusion of innovation theory and the nonadoption, abandonment, scale-up, spread, and sustainability (NASSS) framework guided the development of the 2 study interview guides (provider and patient). We identified themes that emerged across all interviews that were associated with the implementation of video-based care in interdisciplinary PC settings, using directed-content rapid analysis of the interview transcripts. The analysis was guided by 5 a priori NASSS domains: (1) patient condition or characteristic, (2) technology, (3) adopter system, (4) health care organization, and (5) adaptation over time.

RESULTS

The study findings include the following common themes and factors, organized by the 5 NASSS domains: (1) patient condition or characteristic-visit type or purpose (eg, follow-up visits that do not require physical examination), health condition (eg, homebound or semihomebound patients), and sociodemographic characteristic (eg, patients who have a long commute time); (2) technology-key features (eg, access to video-enabled devices), knowledge (eg, how to use videoconferencing software), and technical support for patients and providers; (3) adopter system-changes in staff roles and clinical practice (eg, coordination of video-based care), provider and patient preference or comfort to use video-based care, and caregiver's role (eg, participation of caregivers during video visits); (4) health care organization-leadership support and access to resources, scheduling for video visits (eg, schedule or block off digital half or full days), and training and telehealth champions (eg, hands-on or on-site training for staff, patients, or caregivers); (5) adaptation over time-capacity to improve all aspects of video-based care and provide continued access to resources (eg, effective communication about updates).

CONCLUSIONS

This study identified key factors associated with the implementation of video-based services in interdisciplinary PC settings at the VHA from the perspectives of PC team members and patients. The identified multifaceted factors may inform recommendations on how to sustain and improve the provision of video-based care in VHA PC settings as well as non-VHA patient-centered medical homes.

Use of a digital tool to support the diagnostic process in memory clinics-a usability study

BACKGROUND

Both memory clinic professionals and patients see value in digital tools, yet these hardly find their way to clinical practice. We explored the usability of a digital tool to support the diagnostic work-up in daily memory clinic practice. We evaluated four modules that integrate multi-modal patient data (1.cognitive test; cCOG, and 2. MRI quantification; cMRI) into useful diagnostic information for clinicians (3. cDSI) and understandable and personalized information for patients (4. patient report).

METHODS

We conducted a mixed-methods study in five Dutch memory clinics. Fourteen clinicians (11 geriatric specialists/residents, two neurologists, one nurse practitioner) were invited to integrate the tool into routine care with 43 new memory clinic patients. We evaluated usability and user experiences through quantitative data from questionnaires (patients, care partners, clinicians), enriched with thematically analyzed qualitative data from interviews (clinicians).

RESULTS

We observed wide variation in tool use among clinicians. Our core findings were that clinicians: 1) were mainly positive about the patient report, since it contributes to patient-centered and personalized communication. This was endorsed by patients and care partners, who indicated that the patient report was useful and understandable and helped them to better understand their diagnosis, 2) considered the tool acceptable in addition to their own clinical competence, 3) indicated that the usefulness of the tool depended on the patient population and purpose of the diagnostic process, 4) addressed facilitators (ease of use, practice makes perfect) and barriers (high workload, lack of experience, data unavailability).

CONCLUSION

This multicenter usability study revealed a willingness to adopt a digital tool to support the diagnostic process in memory clinics. Clinicians, patients, and care partners appreciated the personalized diagnostic report. More attention to education and training of clinicians is needed to utilize the full functionality of the tool and foster implementation in actual daily practice. These findings provide an important step towards a lasting adoption of digital tools in memory clinic practice.

Qualitative study exploring the design of a patient-reported symptom-based risk stratification system for suspected head and neck cancer referrals: protocol for work packages 1 and 2 within the EVEREST-HN programme

INTRODUCTION

Between 2009/2010 and 2019/2020, England witnessed an increase in suspected head and neck cancer (sHNC) referrals from 140 to 404 patients per 100 000 population. 1 in 10 patients are not seen within the 2-week target, contributing to patient anxiety. We will develop a pathway for sHNC referrals, based on the Head and Neck Cancer Risk Calculator. The evolution of a patient-reported symptom-based risk stratification system to redesign the sHNC referral pathway (EVEREST-HN) Programme comprises six work packages (WPs). This protocol describes WP1 and WP2. WP1 will obtain an understanding of language to optimise the SYmptom iNput Clinical (SYNC) system patient-reported symptom questionnaire for sHNC referrals and outline requirements for the SYNC system. WP2 will codesign key elements of the SYNC system, including the SYNC Questionnaire, and accompanying behaviour change materials.

METHODS AND ANALYSIS

WP1 will be conducted at three acute National Health Service (NHS) trusts with variation in service delivery models and ensuring a broad mixture of social, economic and cultural backgrounds of participants. Up to 150 patients with sHNC (n=50 per site) and 15 clinicians (n=5 per site) will be recruited. WP1 will use qualitative methods including interviews, observation and recordings of consultations. Rapid qualitative analysis and inductive thematic analysis will be used to analyse the data. WP2 will recruit lay patient representatives to participate in online focus groups (n=8 per focus group), think-aloud technique and experience-based codesign and will be analysed using qualitative and quantitative approaches.

ETHICS AND DISSEMINATION

The committee for clinical research at The Royal Marsden, a research ethics committee and the Health Research Authority approved this protocol. All participants will give informed consent. Ethical issues of working with patients on an urgent cancer diagnostic pathway have been considered. Findings will be disseminated via journal publications, conference presentations and public engagement activities.

Implementation of new technologies designed to improve cervical cancer screening and completion of care in low-resource settings: a case study from the Proyecto Precancer

BACKGROUND

This case study details the experience of the Proyecto Precancer in applying the Integrative Systems Praxis for Implementation Research (INSPIRE) methodology to guide the co-development, planning, implementation, adoption, and sustainment of new technologies and screening practices in a cervical cancer screening and management (CCSM) program in the Peruvian Amazon. We briefly describe the theoretical grounding of the INSPIRE framework, the phases of the INSPIRE process, the activities within each phase, and the RE-AIM outcomes used to evaluate program outcomes.

METHODS

Proyecto Precancer iteratively engaged over 90 stakeholders in the Micro Red Iquitos Sur (MRIS) health network in the Amazonian region of Loreto, Perú, through the INSPIRE phases. INSPIRE is an integrative research methodology grounded in systems thinking, participatory action research, and implementation science frameworks such as the Consolidated Framework for Implementation Research. An interrupted time-series design with a mixed-methods RE-AIM (Reach, Effectiveness, Adoption, Implementation, and Maintenance) evaluation framework was used to examine the adoption of human papillomavirus (HPV) testing (including self-sampling), with direct treatment after visual inspection with portable thermal ablation, at the primary level.

RESULTS

This approach, blending participatory action research, implementation science, and systems-thinking, led to rapid adoption and successful implementation of the new cervical cancer screening and management program within 6 months, using an HPV-based screen-and-treat strategy across 17 health facilities in one of the largest public health networks of the Peruvian Amazon. Monitoring and evaluation data revealed that, within 6 months, the MRIS had surpassed their monthly screening goals, tripling their original screening rate, with approximately 70% of HPV-positive women reaching a completion of care endpoint, compared with around 30% prior to the new CCSM strategy.

CONCLUSIONS

Proyecto Precancer facilitated the adoption and sustainment of HPV testing with subsequent treatment of HPV-positive women (after visual inspection) using portable thermal ablation at the primary level. This was accompanied by the de-implementation of existing visual inspection-based screening strategies and colposcopy for routine precancer triage at the hospital level. This case study highlights how implementation science approaches were used to guide the sustained adoption of a new screen-and-treat strategy in the Peruvian Amazon, while facilitating de-implementation of older screening practices.

Implementation of an Electronic Clinical Decision Support System for the Early Recognition and Management of Dysglycemia in an Inpatient Mental Health Setting Using CogStack: Protocol for a Pilot Hybrid Type 3 Effectiveness-Implementation Randomized Controlled Cluster Trial

BACKGROUND

Severe mental illnesses (SMIs), including schizophrenia, bipolar affective disorder, and major depressive disorder, are associated with an increased risk of physical health comorbidities and premature mortality from conditions including cardiovascular disease and diabetes. Digital technologies such as electronic clinical decision support systems (eCDSSs) could play a crucial role in improving the clinician-led management of conditions such as dysglycemia (deranged blood sugar levels) and associated conditions such as diabetes in people with a diagnosis of SMI in mental health settings.

OBJECTIVE

We have developed a real-time eCDSS using CogStack, an information retrieval and extraction platform, to automatically alert clinicians with National Health Service Trust-approved, guideline-based recommendations for dysglycemia monitoring and management in secondary mental health care. This novel system aims to improve the management of dysglycemia and associated conditions, such as diabetes, in SMI. This protocol describes a pilot study to explore the acceptability, feasibility, and evaluation of its implementation in a mental health inpatient setting.

METHODS

This will be a pilot hybrid type 3 effectiveness-implementation randomized controlled cluster trial in inpatient mental health wards. A ward will be the unit of recruitment, where it will be randomly allocated to receive either access to the eCDSS plus usual care or usual care alone over a 4-month period. We will measure implementation outcomes, including the feasibility and acceptability of the eCDSS to clinicians, as primary outcomes, alongside secondary outcomes relating to the process of care measures such as dysglycemia screening rates. An evaluation of other implementation outcomes relating to the eCDSS will be conducted, identifying facilitators and barriers based on established implementation science frameworks.

RESULTS

Enrollment of wards began in April 2022, after which clinical staff were recruited to take part in surveys and interviews. The intervention period of the trial began in February 2023, and subsequent data collection was completed in August 2023. Data are currently being analyzed, and results are expected to be available in June 2024.

CONCLUSIONS

An eCDSS can have the potential to improve clinician-led management of dysglycemia in inpatient mental health settings. If found to be feasible and acceptable, then, in combination with the results of the implementation evaluation, the system can be refined and improved to support future successful implementation. A larger and more definitive effectiveness trial should then be conducted to assess its impact on clinical outcomes and to inform scalability and application to other conditions in wider mental health care settings.

TRIAL REGISTRATION

ClinicalTrials.gov NCT04792268; https://clinicaltrials.gov/study/NCT04792268.

INTERNATIONAL REGISTERED REPORT IDENTIFIER (IRRID)

DERR1-10.2196/49548.

Question answering systems for health professionals at the point of care-a systematic review

OBJECTIVES

Question answering (QA) systems have the potential to improve the quality of clinical care by providing health professionals with the latest and most relevant evidence. However, QA systems have not been widely adopted. This systematic review aims to characterize current medical QA systems, assess their suitability for healthcare, and identify areas of improvement.

MATERIALS AND METHODS

We searched PubMed, IEEE Xplore, ACM Digital Library, ACL Anthology, and forward and backward citations on February 7, 2023. We included peer-reviewed journal and conference papers describing the design and evaluation of biomedical QA systems. Two reviewers screened titles, abstracts, and full-text articles. We conducted a narrative synthesis and risk of bias assessment for each study. We assessed the utility of biomedical QA systems.

RESULTS

We included 79 studies and identified themes, including question realism, answer reliability, answer utility, clinical specialism, systems, usability, and evaluation methods. Clinicians' questions used to train and evaluate QA systems were restricted to certain sources, types and complexity levels. No system communicated confidence levels in the answers or sources. Many studies suffered from high risks of bias and applicability concerns. Only 8 studies completely satisfied any criterion for clinical utility, and only 7 reported user evaluations. Most systems were built with limited input from clinicians.

DISCUSSION

While machine learning methods have led to increased accuracy, most studies imperfectly reflected real-world healthcare information needs. Key research priorities include developing more realistic healthcare QA datasets and considering the reliability of answer sources, rather than merely focusing on accuracy.

Remote and technology-mediated working during the COVID-19 pandemic: A qualitative exploration of the experiences of nurses working in general practice (the GenCo Study)

AIM

To explore how nurses working in general practice experienced remote and technology-mediated working during the COVID-19 pandemic.

DESIGN

Exploratory qualitative study with nursing team members working in general practices in England and national nurse leaders.

METHODS

Data were collected between April and August 2022. Forty participants took part in either semi-structured interviews or focus groups. Data were analysed using Framework Analysis informed by the PERCS (Planning and Evaluating Remote Consultation Services) Framework. University of York ethics approval [HSRGC/2021/458/I] and Health Research Authority approval were obtained [IRAS:30353. Protocol number: R23982. Ref 21/HRA/5132. CPMS: 51834]. The study was funded by The General Nursing Council for England and Wales Trust.

RESULTS

Participants continued to deliver a significant proportion of patient care in-person. However, remote and technology-mediated care could meet patients' needs and broaden access in some circumstances. When remote and technology-mediated working were used this was often part of a blended model which was expected to continue. This could support some workforce issues, but also increase workload. Participants did not always have access to remote technology and were not involved in decision-making about what was used and how this was implemented. They rarely used video consultations, which were not seen to add value in comparison to telephone consultations. Some participants expressed concern that care had become more transactional than therapeutic and there were potential safety risks.

CONCLUSION

The study explored how nurses working in general practice during the COVID-19 pandemic engaged with remote and technology-mediated working. It identifies specific issues of access to technology, workload, hybrid working, disruption to therapeutic relationships, safety risks and lack of involvement in decision-making. Changes were implemented quickly with little strategic input from nurses. There is now an opportunity to reflect and build on what has been learned in relation to remote and technology-mediated working to ensure the future development of safe and effective nursing care in general practice.

IMPACT

The paper contributes to understanding of remote and technology-mediated working by nurses working in general practice during the COVID-19 pandemic and indicates to employers and policy makers how this can be supported moving forward.

REPORTING METHOD

Standards for Reporting Qualitative Research (O'Brien et al., 2014).

PATIENT OR PUBLIC CONTRIBUTION

This was a workforce study so there was no patient or public contribution.

IMPLICATIONS FOR THE PROFESSION AND PATIENT CARE

The paper highlights specific issues which have implications for the development of remote, technology-mediated and blended working for nurses in general practice, care quality and patient safety. These require full attention to ensure the future development of safe and effective nursing care in general practice moving forward.

Exploring challenges and opportunities in detecting emerging drug trends: A socio-technical analysis of the Canadian context

OBJECTIVES

This study aimed to apply a systems thinking approach to explore factors influencing the detection of emerging drug trends in Canada's provinces and territories to better understand how the local context can influence the design and performance of a pan-Canadian (i.e., national) substance use early warning system (EWS). This study also presents a set of actionable recommendations arising from the results. METHODOLOGY AND METHODS: Semi-structured interviews were conducted with 13 purposively recruited Medical Officers of Health and epidemiologists from across Canada working in the field of substance use. Thematic and social network analysis guided by the socio-technical systems framework were subsequently employed.

RESULTS

Barriers and facilitators for detecting emerging drug trends in provinces and territories are a product of the collective linkages and interactions between social (objectives, people, culture), technical (tools, practices, infrastructure), and external environmental (financial, regulatory frameworks, stakeholders) factors. Shortcomings in several of these areas shaped the system's behaviour and together contributed to fragmented operations that lacked strategic focus, poorly designed cross-sector partnerships, and unactionable information outputs. Participants' experiences shaped perceptions of a national substance use EWS, with some voicing potential opportunities and others expressing doubts about its effectiveness.

CONCLUSION

This study highlights interconnected social, technical, and external environmental considerations for the design and implementation of a national substance use EWS in Canada. It also demonstrates the value of using the socio-technical systems framework to understand a complex public health surveillance issue and how it can be used to inform a path forward.

Beyond forced telehealth adoption: A framework to sustain telehealth among allied health services

INTRODUCTION

As COVID-19 restrictions reduce globally, services will determine what components of care will continue via telehealth. We aimed to determine the clinician, service, and system level factors that influence sustained use of telehealth and develop a framework to enhance sustained use where appropriate.

METHODS

This study was conducted across 16 allied health departments over four health service facilities (Brisbane, Australia). It used a multi-method observational study design, involving telehealth service activity data from hospital administrative databases and qualitative interviews with allied health staff (n  =  80). Data were integrated and analysed using Greenhalgh's Non-adoption, Abandonment, Scale-up, Spread, and Sustainability framework.

RESULTS

Increased telehealth use during the peak COVID period reverted to in-person activity as restrictions eased. Telehealth is unlikely to be sustained without a clear strategy including determination of roles and responsibilities across the organisation. Clinician resistance due to forced adoption remains a key issue. The main motivator for clinicians to use telehealth was improved consumer-centred care. Benefits beyond this are needed to sustain telehealth and improvements are required to make the telehealth experience seamless for providers and recipients. Data were synthesised into a comprehensive framework that can be used as a blueprint for system-wide improvements and service enhancement or redesign.

DISCUSSION

Sustainability of telehealth activity beyond the peak COVID period is unlikely without implementation strategies to address consumer, clinician, service, and system factors. The framework can inform how these strategies can be enacted. Whilst developed for allied health disciplines, it is likely applicable to other disciplines.

Brave New World of Artificial Intelligence: Its Use in Antimicrobial Stewardship-A Systematic Review

Antimicrobial resistance (AMR) is a growing public health problem in the One Health dimension. Artificial intelligence (AI) is emerging in healthcare, since it is helpful to deal with large amounts of data and as a prediction tool. This systematic review explores the use of AI in antimicrobial stewardship programs (ASPs) and summarizes the predictive performance of machine learning (ML) algorithms, compared with clinical decisions, in inpatients and outpatients who need antimicrobial prescriptions. This review includes eighteen observational studies from PubMed, Scopus, and Web of Science. The exclusion criteria comprised studies conducted only in vitro, not addressing infectious diseases, or not referencing the use of AI models as predictors. Data such as study type, year of publication, number of patients, study objective, ML algorithms used, features, and predictors were extracted from the included publications. All studies concluded that ML algorithms were useful to assist antimicrobial stewardship teams in multiple tasks such as identifying inappropriate prescribing practices, choosing the appropriate antibiotic therapy, or predicting AMR. The most extracted performance metric was AUC, which ranged from 0.64 to 0.992. Despite the risks and ethical concerns that AI raises, it can play a positive and promising role in ASP.

Target Product Profile for a Machine Learning-Automated Retinal Imaging Analysis Software for Use in English Diabetic Eye Screening: Protocol for a Mixed Methods Study

BACKGROUND

Diabetic eye screening (DES) represents a significant opportunity for the application of machine learning (ML) technologies, which may improve clinical and service outcomes. However, successful integration of ML into DES requires careful product development, evaluation, and implementation. Target product profiles (TPPs) summarize the requirements necessary for successful implementation so these can guide product development and evaluation.

OBJECTIVE

This study aims to produce a TPP for an ML-automated retinal imaging analysis software (ML-ARIAS) system for use in DES in England.

METHODS

This work will consist of 3 phases. Phase 1 will establish the characteristics to be addressed in the TPP. A list of candidate characteristics will be generated from the following sources: an overview of systematic reviews of diagnostic test TPPs; a systematic review of digital health TPPs; and the National Institute for Health and Care Excellence's Evidence Standards Framework for Digital Health Technologies. The list of characteristics will be refined and validated by a study advisory group (SAG) made up of representatives from key stakeholders in DES. This includes people with diabetes; health care professionals; health care managers and leaders; and regulators and policy makers. In phase 2, specifications for these characteristics will be drafted following a series of semistructured interviews with participants from these stakeholder groups. Data collected from these interviews will be analyzed using the shortlist of characteristics as a framework, after which specifications will be drafted to create a draft TPP. Following approval by the SAG, in phase 3, the draft will enter an internet-based Delphi consensus study with participants sought from the groups previously identified, as well as ML-ARIAS developers, to ensure feasibility. Participants will be invited to score characteristic and specification pairs on a scale from "definitely exclude" to "definitely include," and suggest edits. The document will be iterated between rounds based on participants' feedback. Feedback on the draft document will be sought from a group of ML-ARIAS developers before its final contents are agreed upon in an in-person consensus meeting. At this meeting, representatives from the stakeholder groups previously identified (minus ML-ARIAS developers, to avoid bias) will be presented with the Delphi results and feedback of the user group and asked to agree on the final contents by vote.

RESULTS

Phase 1 was completed in November 2023. Phase 2 is underway and expected to finish in March 2024. Phase 3 is expected to be complete in July 2024.

CONCLUSIONS

The multistakeholder development of a TPP for an ML-ARIAS for use in DES in England will help developers produce tools that serve the needs of patients, health care providers, and their staff. The TPP development process will also provide methods and a template to produce similar documents in other disease areas.

INTERNATIONAL REGISTERED REPORT IDENTIFIER (IRRID)

DERR1-10.2196/50568.

"FREED instils a bit of hope in the eating disorder community… that things can change.": an investigation of clinician views on implementation facilitators and challenges from the rapid scaling of the First Episode Rapid Early Intervention for Eating Disorders programme

Introduction

First Episode Rapid Early Intervention for Eating Disorders (FREED) is the leading eating disorder (ED) early intervention model for young people. Research has shown that it reduces the duration of untreated illness, improves clinical outcomes, and has cost savings. However, less is known about the experience of implementing FREED. This study aimed to investigate the views and experiences of adopting, implementing, and sustaining FREED from the perspective of clinical staff.

Methods

Seven focus groups were conducted involving 26 clinicians. Thematic analysis was used, with the Non-Adoption, Abandonment and Challenges to Scale-up, Spread and Sustainability (The NASSS framework) framework being applied to organise subthemes and determine facilitators and barriers. The NASSS framework was also used to rate the complexity of themes as either simple (straightforward, predictable, few components), complicated (multiple interrelating components), or complex (dynamic, unpredictable, not easily divisible into constituent components).

Results

There were 16 subthemes identified under seven broader themes representing each domain of the NASSS framework. Key barriers and areas of complexity included factors related to EDs as an illness (e.g., high acuity and prevalence), and organisational complexity (e.g., staffing shortages, lack of managerial/team support). Key facilitators included positive clinician/adopter attitudes, a supportive national network, and the ability for FREED to be flexible/adaptable over time.

Conclusion

The FREED model appears to be desirable to clinical staff. Wider team and managerial support was perceived to be particularly important to its successful implementation, as were the national network and supervision. Key areas of complexity include staffing issues and high ED acuity/prevalence. These barriers to implementation need to be managed and investment continued to expand and improve early intervention for EDs further.

How to co-design a prototype of a clinical practice tool: a framework with practical guidance and a case study

Clinical tools for use in practice-such as medicine reconciliation charts, diagnosis support tools and track-and-trigger charts-are endemic in healthcare, but relatively little attention is given to how to optimise their design. User-centred design approaches and co-design principles offer potential for improving usability and acceptability of clinical tools, but limited practical guidance is currently available. We propose a framework (FRamework for co-dESign of Clinical practice tOols or 'FRESCO') offering practical guidance based on user-centred methods and co-design principles, organised in five steps: (1) establish a multidisciplinary advisory group; (2) develop initial drafts of the prototype; (3) conduct think-aloud usability evaluations; (4) test in clinical simulations; (5) generate a final prototype informed by workshops. We applied the framework in a case study to support co-design of a prototype track-and-trigger chart for detecting and responding to possible fetal deterioration during labour. This started with establishing an advisory group of 22 members with varied expertise. Two initial draft prototypes were developed-one based on a version produced by national bodies, and the other with similar content but designed using human factors principles. Think-aloud usability evaluations of these prototypes were conducted with 15 professionals, and the findings used to inform co-design of an improved draft prototype. This was tested with 52 maternity professionals from five maternity units through clinical simulations. Analysis of these simulations and six workshops were used to co-design the final prototype to the point of readiness for large-scale testing. By codifying existing methods and principles into a single framework, FRESCO supported mobilisation of the expertise and ingenuity of diverse stakeholders to co-design a prototype track-and-trigger chart in an area of pressing service need. Subject to further evaluation, the framework has potential for application beyond the area of clinical practice in which it was applied.

Early Impressions and Adoption of the AtriAmp for Managing Arrhythmias Following Congenital Heart Surgery

Objective

AtriAmp is a new medical device that displays a continuous real-time atrial electrogram on telemetry using temporary atrial pacing leads. Our objective was to evaluate early adoption of this device into patient care, understand how it affected clinical workflow, and identify unforeseen benefits or limitations.

Design

Qualitative study using inductive analysis of semi-structured interviews to identify dominant themes.

Setting

Single center, tertiary, academic 21-bed mixed pediatric intensive care unit (PICU).

Subjects

PICU multidisciplinary team members (Pediatric intensivists, PICU Nurse Practitioners, PICU nurses and Pediatric Cardiologists) who were early adopters of the AtriAmp (n=14).

Results

Three prominent themes emerged from qualitative analysis of the early adopters' experiences. (1) Accelerated time from arrhythmia event to diagnosis, treatment, and determination of treatment effectiveness; (2) Increased confidence and security in the accuracy of providers' arrhythmia diagnosis; and (3) Improvement in the ability to educate providers about post-operative arrythmias where reliance on time consuming consultation is a default. Providers also noted some learning curves with the device; none of which compromised medical care or clinical workflow.

Conclusions

Insights from early adopters of AtriAmp signal the need for simplicity and fidelity in new technologies within the PICU. Further research in the qualitative and observational sphere is needed to understand how technologies, such as AtriAmp, find expanded use in the PICU environment. Our research suggests that such technologies can be pivotal to the support and growth of multi-disciplinary teams, even among those who do not participate in early implementation.

Adoption of C-reactive protein rapid tests for the management of acute childhood infections in hospitals in the Netherlands and England: a comparative health systems analysis

BACKGROUND

The adoption of C-reactive protein point-of-care tests (CRP POCTs) in hospitals varies across Europe. We aimed to understand the factors that contribute to different levels of adoption of CRP POCTs for the management of acute childhood infections in two countries.

METHODS

Comparative qualitative analysis of the implementation of CRP POCTs in the Netherlands and England. The study was informed by the non-adoption, abandonment, spread, scale-up, and sustainability (NASSS) framework. Data were collected through document analysis and qualitative interviews with stakeholders. Documents were identified by a scoping literature review, search of websites, and through the stakeholders. Stakeholders were sampled purposively initially, and then by snowballing. Data were analysed thematically.

RESULTS

Forty-one documents resulted from the search and 46 interviews were conducted. Most hospital healthcare workers in the Netherlands were familiar with CRP POCTs as the tests were widely used and trusted in primary care. Moreover, although diagnostics were funded through similar Diagnosis Related Group reimbursement mechanisms in both countries, the actual funding for each hospital was more constrained in England. Compared to primary care, laboratory-based CRP tests were usually available in hospitals and their use was encouraged in both countries because they were cheaper. However, CRP POCTs were perceived as useful in some hospitals of the two countries in which the laboratory could not provide CRP measures 24/7 or within a short timeframe, and/or in emergency departments where expediting patient care was important.

CONCLUSIONS

CRP POCTs are more available in hospitals in the Netherlands because of the greater familiarity of Dutch healthcare workers with the tests which are widely used in primary care in their country and because there are more funding constraints in England. However, most hospitals in the Netherlands and England have not adopted CRP POCTs because the alternative CRP measurements from the hospital laboratory are available in a few hours and at a lower cost.

Generative AI in healthcare: an implementation science informed translational path on application, integration and governance

BACKGROUND

Artificial intelligence (AI), particularly generative AI, has emerged as a transformative tool in healthcare, with the potential to revolutionize clinical decision-making and improve health outcomes. Generative AI, capable of generating new data such as text and images, holds promise in enhancing patient care, revolutionizing disease diagnosis and expanding treatment options. However, the utility and impact of generative AI in healthcare remain poorly understood, with concerns around ethical and medico-legal implications, integration into healthcare service delivery and workforce utilisation. Also, there is not a clear pathway to implement and integrate generative AI in healthcare delivery.

METHODS

This article aims to provide a comprehensive overview of the use of generative AI in healthcare, focusing on the utility of the technology in healthcare and its translational application highlighting the need for careful planning, execution and management of expectations in adopting generative AI in clinical medicine. Key considerations include factors such as data privacy, security and the irreplaceable role of clinicians' expertise. Frameworks like the technology acceptance model (TAM) and the Non-Adoption, Abandonment, Scale-up, Spread and Sustainability (NASSS) model are considered to promote responsible integration. These frameworks allow anticipating and proactively addressing barriers to adoption, facilitating stakeholder participation and responsibly transitioning care systems to harness generative AI's potential.

RESULTS

Generative AI has the potential to transform healthcare through automated systems, enhanced clinical decision-making and democratization of expertise with diagnostic support tools providing timely, personalized suggestions. Generative AI applications across billing, diagnosis, treatment and research can also make healthcare delivery more efficient, equitable and effective. However, integration of generative AI necessitates meticulous change management and risk mitigation strategies. Technological capabilities alone cannot shift complex care ecosystems overnight; rather, structured adoption programs grounded in implementation science are imperative.

CONCLUSIONS

It is strongly argued in this article that generative AI can usher in tremendous healthcare progress, if introduced responsibly. Strategic adoption based on implementation science, incremental deployment and balanced messaging around opportunities versus limitations helps promote safe, ethical generative AI integration. Extensive real-world piloting and iteration aligned to clinical priorities should drive development. With conscientious governance centred on human wellbeing over technological novelty, generative AI can enhance accessibility, affordability and quality of care. As these models continue advancing rapidly, ongoing reassessment and transparent communication around their strengths and weaknesses remain vital to restoring trust, realizing positive potential and, most importantly, improving patient outcomes.

Co-design and evaluation of a youth-informed organisational tool to enhance trauma-informed practices in the UK public sector: a study protocol

INTRODUCTION

A trauma-informed approach (TIA) means working with awareness that people's histories of trauma may shape the way they engage with services, organisations or institutions. Young people with adverse childhood experiences may be at risk of retraumatisation by organisational practices in schools and universities and by employers and health agencies when they seek support. There are limited evidence-based resources to help people working in the public sector to work with adolescents in trauma-informed ways and the needs of adolescents have not been central in resource development. This study contributes to public sector capacity to work in trauma-informed ways with adolescents by codesigning and evaluating the implementation of a youth-informed organisational resource.

METHODS AND ANALYSIS

This is an Accelerated Experience-based Co-design (AEBCD) Study followed by pre-post evaluation. Public sector organisations or services, and adolescents connected with them, will collaboratively reflect on lived experience data assembled through creative arts practice, alongside data from epidemiological national data sets. These will present knowledge about the impact of adverse childhood experiences on adolescents' mental health (stage 1). Collaboratively, priorities (touch points) for organisational responses will be identified (stage 2), and a low-burden resource will be codesigned (stage 3) and offered for implementation (stage 4) and evaluation (stage 5) in diverse settings. The study will provide insights into what adolescents and public sector organisations in the UK want from a TIA resource, the experience of services/organisations in implementing this and recommendations for resource development and implementation.

ETHICS AND DISSEMINATION

The UK National Health Service Health Research Authority approved this study (23/WM/0105). Learning will be shared across study participants in a workshop at the end of the study. Knowledge products will include a website detailing the created resource and a youth-created film documenting the study process, the elements of the codesigned resource and experiences of implementation. Dissemination will target academic, healthcare, education, social care, third sector and local government settings via knowledge exchange events, social media, accessible briefings, conference presentations and publications.

eHealth implementation in Europe: a scoping review on legal, ethical, financial, and technological aspects

Background

The evolution of eHealth development has shifted from standalone tools to comprehensive digital health environments, fostering data exchange among diverse stakeholders and systems. Nevertheless, existing research and implementation frameworks have primarily emphasized technological and organizational aspects of eHealth implementation, overlooking the intricate legal, ethical, and financial considerations. It is essential to discover what legal, ethical, financial, and technological challenges should be considered to ensure successful and sustainable implementation of eHealth.

Objective

This review aims to provide insights into barriers and facilitators of legal, ethical, financial, and technological aspects for successful implementation of complex eHealth technologies, which impacts multiple levels and multiple stakeholders.

Methods

A scoping review was conducted by querying PubMed, Scopus, Web of Science, and ACM Digital Library (2018-2023) for studies describing the implementation process of eHealth technologies that facilitate data exchange. Studies solely reporting clinical outcomes or conducted outside Europe were excluded. Two independent reviewers selected the studies. A conceptual framework was constructed through axial and inductive coding, extracting data from literature on legal, ethical, financial, and technological aspects of eHealth implementation. This framework guided systematic extraction and interpretation.

Results

The search resulted in 7.308 studies that were screened for eligibility, of which 35 (0.48%) were included. Legal barriers revolve around data confidentiality and security, necessitating clear regulatory guidelines. Ethical barriers span consent, responsibility, liability, and validation complexities, necessitating robust frameworks. Financial barriers stem from inadequate funding, requiring (commercial) partnerships and business models. Technological issues include interoperability, integration, and malfunctioning, necessitating strategies for enhancing data reliability, improving accessibility, and aligning eHealth technology with existing systems for smoother integration.

Conclusions

This research highlights the multifaceted nature of eHealth implementation, encompassing legal, ethical, financial, and technological considerations. Collaborative stakeholder engagement is paramount for effective decision-making and aligns with the transition from standalone eHealth tools to integrated digital health environments. Identifying suitable stakeholders and recognizing their stakes and values enriches implementation strategies with expertise and guidance across all aspects. Future research should explore the timing of these considerations and practical solutions for regulatory compliance, funding, navigation of responsibility and liability, and business models for reimbursement strategies.

Developing an integrated clinical decision support system for the early identification and management of kidney disease-building cross-sectoral partnerships

BACKGROUND

The burden of chronic conditions is growing in Australia with people in remote areas experiencing high rates of disease, especially kidney disease. Health care in remote areas of the Northern Territory (NT) is complicated by a mobile population, high staff turnover, poor communication between health services and complex comorbid health conditions requiring multidisciplinary care.

AIM

This paper aims to describe the collaborative process between research, government and non-government health services to develop an integrated clinical decision support system to improve patient care.

METHODS

Building on established partnerships in the government and Aboriginal Community-Controlled Health Service (ACCHS) sectors, we developed a novel digital clinical decision support system for people at risk of developing kidney disease (due to hypertension, diabetes, cardiovascular disease) or with kidney disease. A cross-organisational and multidisciplinary Steering Committee has overseen the design, development and implementation stages. Further, the system's design and functionality were strongly informed by experts (Clinical Reference Group and Technical Working Group), health service providers, and end-user feedback through a formative evaluation.

RESULTS

We established data sharing agreements with 11 ACCHS to link patient level data with 56 government primary health services and six hospitals. Electronic Health Record (EHR) data, based on agreed criteria, is automatically and securely transferred from 15 existing EHR platforms. Through clinician-determined algorithms, the system assists clinicians to diagnose, monitor and provide guideline-based care for individuals, as well as service-level risk stratification and alerts for clinically significant events.

CONCLUSION

Disconnected health services and separate EHRs result in information gaps and a health and safety risk, particularly for patients who access multiple health services. However, barriers to clinical data sharing between health services still exist. In this first phase, we report how robust partnerships and effective governance processes can overcome these barriers to support clinical decision making and contribute to holistic care.

Smartphone Apps for Food Purchase Choices: Scoping Review of Designs, Opportunities, and Challenges

BACKGROUND

Smartphone apps can aid consumers in making healthier and more sustainable food purchases. However, there is still a limited understanding of the different app design approaches and their impact on food purchase choices. An overview of existing food purchase choice apps and an understanding of common challenges can help speed up effective future developments.

OBJECTIVE

We examined the academic literature on food purchase choice apps and provided an overview of the design characteristics, opportunities, and challenges for effective implementation. Thus, we contribute to an understanding of how technologies can effectively improve food purchase choice behavior and provide recommendations for future design efforts.

METHODS

Following the PRISMA-ScR (Preferred Reporting Items for Systematic Reviews and Meta-Analyses extension for Scoping Reviews) guidelines, we considered peer-reviewed literature on food purchase choice apps within IEEE Xplore, PubMed, Scopus, and ScienceDirect. We inductively coded and summarized design characteristics. Opportunities and challenges were addressed from both quantitative and qualitative perspectives. From the quantitative perspective, we coded and summarized outcomes of comparative evaluation trials. From the qualitative perspective, we performed a qualitative content analysis of commonly discussed opportunities and challenges.

RESULTS

We retrieved 55 articles, identified 46 unique apps, and grouped them into 5 distinct app types. Each app type supports a specific purchase choice stage and shares a common functional design. Most apps support the product selection stage (selection apps; 27/46, 59%), commonly by scanning the barcode and displaying a nutritional rating. In total, 73% (8/11) of the evaluation trials reported significant findings and indicated the potential of food purchase choice apps to support behavior change. However, relatively few evaluations covered the selection app type, and these studies showed mixed results. We found a common opportunity in apps contributing to learning (knowledge gain), whereas infrequent engagement presents a common challenge. The latter was associated with perceived burden of use, trust, and performance as well as with learning. In addition, there were technical challenges in establishing comprehensive product information databases or achieving performance accuracy with advanced identification methods such as image recognition.

CONCLUSIONS

Our findings suggest that designs of food purchase choice apps do not encourage repeated use or long-term adoption, compromising the effectiveness of behavior change through nudging. However, we found that smartphone apps can enhance learning, which plays an important role in behavior change. Compared with nudging as a mechanism for behavior change, this mechanism is less dependent on continued use. We argue that designs that optimize for learning within each interaction have a better chance of achieving behavior change. This review concludes with design recommendations, suggesting that food purchase choice app designers anticipate the possibility of early abandonment as part of their design process and design apps that optimize the learning experience.

Navigating Quality Assessment Hurdles in Clinical Laboratory Services: A Comprehensive Review in Resource-Limited Settings

External quality assessment is the process of evaluating the performance of a laboratory and the competence of professionals. Participation in EQA and standardizing the quality of laboratory services are a mandatory requirements for accreditation. This review is aimed at identifying and discussing challenges that hinder the effective implementation of an EQA program in countries with resource limited setting. To obtain abundant information, articles were identified by searching the literature publishedin English using the National Library of Medicine, PubMed, Science Direct and AMC digital library databases. The articles identified in the references were manually searched and included. After the article was identified, it was imported to Endnote version 8.1 and exported to Microsoft Word for citation. Based on this review, the major identified challenges that hinder the implementation of an EQA program include the high cost of control materials, malfunction and lack of maintenance for equipment failure and breakdown, a knowledge gap among laboratory professionals, noncommutability of control samples, and difficulty in assigning target values. In addition, failing to participate in EQA and failing to take corrective action are the major challenges identified. As a result, applying to an EQA program in resource-limited counties was highly challenging. To attain high performance in the laboratory and to provide quality laboratory service for patient care, the EQA supplier and the user laboratory must pay attention to these issues and take appropriate corrective actions for ongoing quality improvement and accreditation.

Systems archetypes to investigate the unintended consequences of telehealth in rural Australia: A systems thinking approach to telehealth evaluation and policymaking

As an alternative model of delivery to standard care, telehealth offers a promising solution to health access issues faced by rural and remote communities in Australia and worldwide. However, research typically focuses on its expected benefits and pitfalls, with little to no consideration of its unintended consequences and factors influencing its better utilisation. Drawing on systems thinking and informed by complexity science, we propose using systems archetypes-systems thinking tools - as a magnifying lens to investigate potential telehealth unintended consequences or outcomes. We conceptualise telehealth implementation in rural and remote Australia as a sociotechnical system whereby the interactions between its various agents shape telehealth implementation and, in turn, are shaped by it. When introducing new policies or interventions to any system, these interactions often lead to outcomes other than those initially planned or intended. Although systems archetypes cannot necessarily predict these outcomes, they are valuable for helping anticipate unintended, unforeseen outcomes and facilitating discussions about them to mitigate their negative impact and maximise their benefits. Outcomes are not necessarily adverse; they can also be positive. So, investigating such outcomes will minimise their negative impact and maximise their benefit. Our method was to review existing research and a selection of complexity and systems informed frameworks. Then, we assessed systems archetypes. And how they can be utilised to investigate unintended consequences. A worked example of what an unintended consequence in the implementation of telehealth in rural and remote Australia is presented.

Telemedicine Implementation for Safety Net Populations: A Systematic Review

Background: Telemedicine systems were rapidly implemented in response to COVID-19. However, little is known about their effectiveness, acceptability, and sustainability for safety net populations. This study systematically reviewed primary care telemedicine implementation and effectiveness in safety net settings. Methods: We searched PubMed for peer-reviewed articles on telemedicine implementation from 2013 to 2021. The search was done between June and December 2021. Included articles focused on health care organizations that primarily serve low-income and/or rural populations in the United States. We screened 244 articles from an initial search of 343 articles and extracted and analyzed data from N = 45 articles. Results: Nine (20%) of 45 articles were randomized controlled trials. N = 22 reported findings for at least one marginalized group (i.e., racial/ethnic minority, 65 years+, limited English proficiency). Only n = 19 (42%) included African American/Black patients in demographics descriptions, n = 14 (31%) LatinX/Hispanic patients, n = 4 (9%) Asian patients, n = 4 (9%) patients aged 65+ years, and n = 4 (9%) patients with limited English proficiency. Results show telemedicine can provide high-quality primary care that is more accessible and affordable. Fifteen studies assessed barriers and facilitators to telemedicine implementation. Common barriers were billing/administrative workflow disruption (n = 9, 20%), broadband access/quality (n = 5, 11%), and patient preference for in-person care (n = 4, 9%). Facilitators included efficiency gains (n = 6, 13%), patient acceptance (n = 3, 7%), and enhanced access (n = 3, 7%). Conclusions: Telemedicine is an acceptable care modality to deliver primary care in safety net settings. Future studies should compare telemedicine and in-person care quality and test strategies to improve telemedicine implementation in safety net settings.

Digitally augmented, parent-led CBT versus treatment as usual for child anxiety problems in child mental health services in England and Northern Ireland: a pragmatic, non-inferiority, clinical effectiveness and cost-effectiveness randomised controlled trial

BACKGROUND

Anxiety problems are common in children, yet few affected children access evidence-based treatment. Digitally augmented psychological therapies bring potential to increase availability of effective help for children with mental health problems. This study aimed to establish whether therapist-supported, digitally augmented, parent-led cognitive behavioural therapy (CBT) could increase the efficiency of treatment without compromising clinical effectiveness and acceptability.

METHODS

We conducted a pragmatic, unblinded, two-arm, multisite, randomised controlled non-inferiority trial to evaluate the clinical effectiveness and cost-effectiveness of therapist-supported, parent-led CBT using the Online Support and Intervention (OSI) for child anxiety platform compared with treatment as usual for child (aged 5-12 years) anxiety problems in 34 Child and Adolescent Mental Health Services in England and Northern Ireland. We examined acceptability of OSI plus therapist support via qualitative interviews. Participants were randomly assigned (1:1) to OSI plus therapist support or treatment as usual, minimised by child age, gender, service type, and baseline child anxiety interference. Outcomes were assessed at week 14 and week 26 after randomisation. The primary clinical outcome was parent-reported interference caused by child anxiety at week 26 assessment, using the Child Anxiety Impact Scale-parent report (CAIS-P). The primary measure of health economic effect was quality-adjusted life-years (QALYs). Outcome analyses were conducted blind in the intention-to-treat (ITT) population with a standardised non-inferiority margin of 0·33 for clinical analyses. The trial was registered with ISRCTN, 12890382.

FINDINGS

Between Dec 5, 2020, and Aug 3, 2022, 706 families (706 children and their parents or carers) were referred to the study information. 444 families were enrolled. Parents reported 255 (58%) child participants' gender to be female, 184 (41%) male, three (<1%) other, and one (<1%) preferred not to report their child's gender. 400 (90%) children were White and the mean age was 9·20 years (SD 1·79). 85% of families for whom clinicians provided information in the treatment as usual group received CBT. OSI plus therapist support was non-inferior for parent-reported anxiety interference on the CAIS-P (SMD 0·01, 95% CI -0·15 to 0·17; p<0·0001) and all secondary outcomes. The mean difference in QALYs across trial arms approximated to zero, and OSI plus therapist support was associated with lower costs than treatment as usual. OSI plus therapist support was likely to be cost effective under certain scenarios, but uncertainty was high. OSI plus therapist support acceptability was good. No serious adverse events were reported.

INTERPRETATION

Digitally augmented intervention brought promising savings without compromising outcomes and as such presents a valuable tool for increasing access to psychological therapies and meeting the demand for treatment of child anxiety problems.

FUNDING

Department for Health and Social Care and United Kingdom Research and Innovation Research Grant, National Institute for Health and Care (NIHR) Research Policy Research Programme, Oxford and Thames Valley NIHR Applied Research Collaboration, Oxford Health NIHR Biomedical Research Centre.

Safer and more efficient vital signs monitoring protocols to identify the deteriorating patients in the general hospital ward: an observational study

Background

The frequency at which patients should have their vital signs (e.g. blood pressure, pulse, oxygen saturation) measured on hospital wards is currently unknown. Current National Health Service monitoring protocols are based on expert opinion but supported by little empirical evidence. The challenge is finding the balance between insufficient monitoring (risking missing early signs of deterioration and delays in treatment) and over-observation of stable patients (wasting resources needed in other aspects of care).

Objective

Provide an evidence-based approach to creating monitoring protocols based on a patient's risk of deterioration and link these to nursing workload and economic impact.

Design

Our study consisted of two parts: (1) an observational study of nursing staff to ascertain the time to perform vital sign observations; and (2) a retrospective study of historic data on patient admissions exploring the relationships between National Early Warning Score and risk of outcome over time. These were underpinned by opinions and experiences from stakeholders.

Setting and participants

Observational study: observed nursing staff on 16 randomly selected adult general wards at four acute National Health Service hospitals. Retrospective study: extracted, linked and analysed routinely collected data from two large National Health Service acute trusts; data from over 400,000 patient admissions and 9,000,000 vital sign observations.

Results

Observational study found a variety of practices, with two hospitals having registered nurses take the majority of vital sign observations and two favouring healthcare assistants or student nurses. However, whoever took the observations spent roughly the same length of time. The average was 5:01 minutes per observation over a 'round', including time to locate and prepare the equipment and travel to the patient area. Retrospective study created survival models predicting the risk of outcomes over time since the patient was last observed. For low-risk patients, there was little difference in risk between 4 hours and 24 hours post observation.

Conclusions

We explored several different scenarios with our stakeholders (clinicians and patients), based on how 'risk' could be managed in different ways. Vital sign observations are often done more frequently than necessary from a bald assessment of the patient's risk, and we show that a maximum threshold of risk could theoretically be achieved with less resource. Existing resources could therefore be redeployed within a changed protocol to achieve better outcomes for some patients without compromising the safety of the rest. Our work supports the approach of the current monitoring protocol, whereby patients' National Early Warning Score 2 guides observation frequency. Existing practice is to observe higher-risk patients more frequently and our findings have shown that this is objectively justified. It is worth noting that important nurse-patient interactions take place during vital sign monitoring and should not be eliminated under new monitoring processes. Our study contributes to the existing evidence on how vital sign observations should be scheduled. However, ultimately, it is for the relevant professionals to decide how our work should be used.

Study registration

This study is registered as ISRCTN10863045.

Funding

This award was funded by the National Institute for Health and Care Research (NIHR) Health and Social Care Delivery Research programme (NIHR award ref: 17/05/03) and is published in full in Health and Social Care Delivery Research; Vol. 12, No. 6. See the NIHR Funding and Awards website for further award information.

Using a Novel Gameplay Intervention to Target Intrusive Memories After Work-Related Trauma: Iterative Qualitative Analysis of Intensive Care Unit Staff Experiences

BACKGROUND

Many intensive care unit (ICU) staff experience intrusive memories following work-related traumatic events, which can lead to long-term mental health outcomes and impact work functioning. There is a need for interventions that target intrusive memories in this population; however, factors such as mental health stigma and difficulty in fitting interventions into busy schedules can pose barriers. The Brief Gameplay Intervention For National Health Service Intensive Care Unit Staff Affected By COVID-19 Trauma (GAINS) study tested a brief, digital imagery-competing task intervention (including computer gameplay) with the aim of reducing the recurrence of intrusive memories, which holds promise for overcoming some of these barriers.

OBJECTIVE

This substudy aims to explore barriers and facilitators to the uptake and practical use of the intervention by ICU staff, along with its acceptability, and iteratively explore the impact of intervention optimizations to further refine the intervention.

METHODS

The GAINS study is a randomized controlled trial comparing access to a brief digital imagery-competing task intervention for 4 weeks with usual care followed by delayed access to the intervention. The participants were ICU staff who worked during the COVID-19 pandemic and experienced intrusive memories. All participants were sent a questionnaire at 4 weeks to gather data about intervention acceptability. Nested within the randomized controlled trial, a subset of 16 participants was interviewed, and data were analyzed using thematic analysis drawing from a framework approach.

RESULTS

Both quantitative and qualitative data indicated high acceptability of the intervention. Intervention use data show that, on average, staff were able to target approximately 73% (3.64/4.88) of their intrusive memories and engaged with the Tetris component for the full 20 minutes per session. Overall, on the acceptability questionnaire, staff found the intervention easy to use, helpful, and highly acceptable. The interviews generated four themes: approach to the intervention, positives of the intervention, negatives of the intervention, and improvements and optimizations. Findings highlighted barriers that ICU staff experienced: stigma, feeling weak for seeking help, not wanting colleagues to know they were struggling, and skepticism. However, they provided suggestions on how barriers could be overcome and discussed the advantages of the intervention when compared with other treatments. Although participants described many positive aspects of the intervention, such as being easy to use, enjoyable, and leading to a reduction in the frequency or intensity of intrusive memories, they also raised practical issues for implementation.

CONCLUSIONS

The intervention has the potential to overcome stigma and reduce the frequency of intrusive memories after traumatic events among ICU staff. Further refinement is needed to improve the adoption and reach of this intervention. A limitation is that we could not interview the National Health Service staff who were unable or unwilling to take part in the trial.

Understanding inherent influencing factors to digital health adoption in general practices through a mixed-methods analysis

Extensive research has shown the potential value of digital health solutions and highlighted the importance of clinicians' adoption. As general practitioners (GPs) are patients' first point of contact, understanding influencing factors to their digital health adoption is especially important to derive personalized practical recommendations. Using a mixed-methods approach, this study broadly identifies adoption barriers and potential improvement strategies in general practices, including the impact of GPs' inherent characteristics - especially their personality - on digital health adoption. Results of our online survey with 216 GPs reveal moderate overall barriers on a 5-point Likert-type scale, with required workflow adjustments (M = 4.13, SD = 0.93), inadequate reimbursement (M = 4.02, SD = 1.02), and high training effort (M = 3.87, SD = 1.01) as substantial barriers. Improvement strategies are considered important overall, with respondents especially wishing for improved interoperability (M = 4.38, SD = 0.81), continued technical support (M = 4.33, SD = 0.91), and improved usability (M = 4.20, SD = 0.88). In our regression model, practice-related characteristics, the expected future digital health usage, GPs' digital affinity, several personality traits, and digital maturity are significant predictors of the perceived strength of barriers. For the perceived importance of improvement strategies, only demographics and usage-related variables are significant predictors. This study provides strong evidence for the impact of GPs' inherent characteristics on barriers and improvement strategies. Our findings highlight the need for comprehensive approaches integrating personal and emotional elements to make digitization in practices more engaging, tangible, and applicable.

Towards co-design of rehabilitation technologies: a collaborative approach to prioritize usability issues

Introduction

Early stakeholder engagement is critical to the successful development and translation of rehabilitation technologies, a pivotal step of which is usability testing with intended end-users. To this end, several methods employ end-user feedback to identify usability and implementation issues. However, the process of prioritizing identified issues seldom leverages the knowledge and expertise of the range of stakeholders who will ultimately affect the demand and supply of a device. This paper describes a novel method to prioritize end-user feedback using transdisciplinary stakeholder consultation and address it in subsequent product development. The proposed approach was demonstrated using a case study relating to the development of a novel technology for neural recovery after spinal cord injury.

Method

Feedback from five individuals with chronic spinal cord injury was collected during two-hour usability evaluation sessions with a fully functional high-fidelity system prototype. A think-aloud and semi-structured interview protocol was used with each participant to identify usability and acceptability issues relating to the system in a 3-phase approach. Phase 1 involved extracting usability issues from think-aloud and semi-structured interview data. Phase 2 involved rating the usability issues based on their significance, technical feasibility, and implementation priority by relevant internal and external stakeholders. Finally, Phase 3 involved aggregating the usability issues according to design and implementation elements to facilitate solution generation, and these solutions were then raised as action tasks for future design iterations.

Results

Sixty usability issues representing nine facets of usability were rated. Eighty percent of issues were rated to be of moderate to high significance, 83% were rated as being feasible to address, and 75% were rated as addressable using existing project resources. Fifty percent of the issues were rated to be a high priority for implementation. Evaluation of the grouped issues identified 21 tasks which were mapped to the product roadmap for integration into future design iterations.

Discussion

This paper presents a method for meaningful transdisciplinary stakeholder engagement in rehabilitation technology development that can extended to other projects. Alongside a worked example, we offer practical considerations for others seeking to co-develop rehabilitation technologies.