The NASSS framework for ex post theorisation of technology-supported change in healthcare: worked example of the TORPEDO programme

Question answering systems for health professionals at the point of care-a systematic review

OBJECTIVES

Question answering (QA) systems have the potential to improve the quality of clinical care by providing health professionals with the latest and most relevant evidence. However, QA systems have not been widely adopted. This systematic review aims to characterize current medical QA systems, assess their suitability for healthcare, and identify areas of improvement.

MATERIALS AND METHODS

We searched PubMed, IEEE Xplore, ACM Digital Library, ACL Anthology, and forward and backward citations on February 7, 2023. We included peer-reviewed journal and conference papers describing the design and evaluation of biomedical QA systems. Two reviewers screened titles, abstracts, and full-text articles. We conducted a narrative synthesis and risk of bias assessment for each study. We assessed the utility of biomedical QA systems.

RESULTS

We included 79 studies and identified themes, including question realism, answer reliability, answer utility, clinical specialism, systems, usability, and evaluation methods. Clinicians' questions used to train and evaluate QA systems were restricted to certain sources, types and complexity levels. No system communicated confidence levels in the answers or sources. Many studies suffered from high risks of bias and applicability concerns. Only 8 studies completely satisfied any criterion for clinical utility, and only 7 reported user evaluations. Most systems were built with limited input from clinicians.

DISCUSSION

While machine learning methods have led to increased accuracy, most studies imperfectly reflected real-world healthcare information needs. Key research priorities include developing more realistic healthcare QA datasets and considering the reliability of answer sources, rather than merely focusing on accuracy.

How can we improve Comprehensive Geriatric Assessment for older people living with frailty in primary care and community settings? A qualitative study

OBJECTIVE

With advancing age comes the increasing prevalence of frailty and increased risk of adverse outcomes (eg, hospitalisation). Evidence for comprehensive geriatric assessment (CGA), a multidimensional holistic model of care, is mixed in community settings. Uncertainties remain, such as the key components of CGA, who delivers it, and the use of technology. This study aimed to understand the perspectives, beliefs and experiences, of both older people and health professionals, to improve the current CGA and explore factors that may impact on CGA delivery in community settings.

DESIGN

A qualitative interview study was conducted with older people and healthcare professionals (HCPs) identified using a maximum variation strategy. Data were analysed using an abductive analysis approach. The non-adoption, abandonment, scale-up, spread and sustainability framework and the theoretical framework of acceptability guided the categorisation of the codes and identified categories were mapped to the two frameworks.

SETTING

England, UK.

RESULTS

27 people were interviewed, constituting 14 older people and 13 HCPs. We identified limitations in the current CGA: a lack of information sharing between different HCPs who deliver CGA; poor communication between older people and their HCPs and a lack of follow-up as part of CGA. When we discussed the potential for CGA to use technology, HCPs and older people varied in their readiness to engage with it.

CONCLUSIONS

Viable solutions to address gaps in the current delivery of CGA include the provision of training and support to use digital technology and a designated comprehensive care coordinator. The next stage of this research will use these findings, existing evidence and stakeholder engagement, to develop and refine a model of community-based CGA that can be assessed for feasibility and acceptability.

Applying the Non-adoption, Abandonment, Scale-up, Spread and Sustainability (NASSS) framework to evaluate automated evidence synthesis in health behaviour change

Automated tools to speed up the process of evidence synthesis are increasingly apparent within health behaviour research. This brief review explores the potential of the Non-adoption, Abandonment, Scale-up, Spread and Sustainability framework for supporting automated evidence synthesis in health behaviour change by applying it to the ongoing Human Behaviour-Change Project, which aims to revolutionize evidence synthesis within behaviour change intervention research. To increase the relevance of NASSS for health behaviour change, we recommend i) terminology changes ('condition' to 'behaviour' and 'patient' to 'end user') and ii) that it is used prospectively address complexities iteratively. We draw conclusions about i) the need to specify the organizations that will use the technology, ii) identifying what to do if interdependencies fail and iii) even though we have focused on automated evidence synthesis, NASSS would arguably be beneficial for technology developments in health behaviour change more generally, particularly for invention development.

Recommendation to implementation of remote patient monitoring in rheumatology: lessons learned and barriers to take

Remote patient monitoring (RPM) leverages advanced technology to monitor and manage patients' health remotely and continuously. In 2022 European Alliance of Associations for Rheumatology (EULAR) points-to-consider for remote care were published to foster adoption of RPM, providing guidelines on where to position RPM in our practices. Sample papers and studies describe the value of RPM. But for many rheumatologists, the unanswered question remains the 'how to?' implement RPM.Using the successful, though not frictionless example of the Southmead rheumatology department, we address three types of barriers for the implementation of RPM: service, clinician and patients, with subsequent learning points that could be helpful for new teams planning to implement RPM. These address, but are not limited to, data governance, selecting high quality cost-effective solutions and ensuring compliance with data protection regulations. In addition, we describe five lacunas that could further improve RPM when addressed: establishing quality standards, creating a comprehensive database of available RPM tools, integrating data with electronic patient records, addressing reimbursement uncertainties and improving digital literacy among patients and healthcare professionals.

'Smart' BLE wearables for digital contact tracing in care homes during the COVID-19 pandemic-a process evaluation of the CONTACT feasibility study

BACKGROUND

Rapid and mass transmission of the SARS-CoV-2 virus amongst vulnerable people led to devastating effects from COVID-19 in care homes. The CONTACT intervention introduced Bluetooth Low Energy 'smart' wearable devices (BLE wearables) as a basis for automated contact tracing in, and feedback on infection risks and patterns to, care homes to try and improve infection prevention and control (IPC). We planned a cluster randomised controlled trial (RCT) of CONTACT. To be feasible, homes had to adopt CONTACT's technology and new ways of working. This paper reports on the process evaluation conducted alongside CONTACT's feasibility study and explains why it lacked the feasibility and acceptability for a definitive RCT.

METHODS

This mixed method process evaluation used Normalisation Process Theory (NPT) qualitative (interviews, field notes, study case report forms and documents, and observation) and quantitative (survey instruments, counts of activity) data to plan, implement, and analyse the mechanisms, effects, and contextual factors that shaped the feasibility and acceptability of the CONTACT intervention.

RESULTS

Thirteen themes within four core NPT constructs explained CONTACT's lack of feasibility. Coherence: the home's varied in the scale and extent of commitment and understanding of the technology and study procedures. Leadership credibility was important but compromised by competing priorities. Management and direct care staff saw CONTACT differently. Work to promote (cognitive participation) and enact (collective action) CONTACT was burdensome and failed to be prioritised over competing COVID-19-related demands on time and scarce human and cognitive resources. Ultimately, staff appraisal of the value of CONTACT-generated information and study procedures (reflexivity) was that any utility for IPC was insufficient to outweigh the perceived burden and complexity involved.

CONCLUSIONS

Despite implementation failure, dismissing BLE wearables' potential for contact tracing is premature. In non-pandemic conditions, with more time, better co-design and integration of theory-driven implementation strategies tailored to care homes' unique contexts, researchers could enhance normalisation in readiness for future pandemic challenges.

TRIAL REGISTRATION

ISRCTN registration: 11,204,126 registered 17/02/2021.

How digital health translational research is prioritised: a qualitative stakeholder-driven approach to decision support evaluation

OBJECTIVES

Digital health is now routinely being applied in clinical care, and with a variety of clinician-facing systems available, healthcare organisations are increasingly required to make decisions about technology implementation and evaluation. However, few studies have examined how digital health research is prioritised, particularly research focused on clinician-facing decision support systems. This study aimed to identify criteria for prioritising digital health research, examine how these differ from criteria for prioritising traditional health research and determine priority decision support use cases for a collaborative implementation research programme.

METHODS

Drawing on an interpretive listening model for priority setting and a stakeholder-driven approach, our prioritisation process involved stakeholder identification, eliciting decision support use case priorities from stakeholders, generating initial use case priorities and finalising preferred use cases based on consultations. In this qualitative study, online focus group session(s) were held with stakeholders, audiorecorded, transcribed and analysed thematically.

RESULTS

Fifteen participants attended the online priority setting sessions. Criteria for prioritising digital health research fell into three themes, namely: public health benefit, health system-level factors and research process and feasibility. We identified criteria unique to digital health research as the availability of suitable governance frameworks, candidate technology's alignment with other technologies in use,and the possibility of data-driven insights from health technology data. The final selected use cases were remote monitoring of patients with pulmonary conditions, sepsis detection and automated breast screening.

CONCLUSION

The criteria for determining digital health research priority areas are more nuanced than that of traditional health condition focused research and can neither be viewed solely through a clinical lens nor technological lens. As digital health research relies heavily on health technology implementation, digital health prioritisation criteria comprised enablers of successful technology implementation. Our prioritisation process could be applied to other settings and collaborative projects where research institutions partner with healthcare delivery organisations.

Identifying barriers and facilitators to successful implementation of computerized clinical decision support systems in hospitals: a NASSS framework-informed scoping review

BACKGROUND

Successful implementation and utilization of Computerized Clinical Decision Support Systems (CDSS) in hospitals is complex and challenging. Implementation science, and in particular the Nonadoption, Abandonment, Scale-up, Spread and Sustainability (NASSS) framework, may offer a systematic approach for identifying and addressing these challenges. This review aimed to identify, categorize, and describe barriers and facilitators to CDSS implementation in hospital settings and map them to the NASSS framework. Exploring the applicability of the NASSS framework to CDSS implementation was a secondary aim.

METHODS

Electronic database searches were conducted (21 July 2020; updated 5 April 2022) in Ovid MEDLINE, Embase, Scopus, PyscInfo, and CINAHL. Original research studies reporting on measured or perceived barriers and/or facilitators to implementation and adoption of CDSS in hospital settings, or attitudes of healthcare professionals towards CDSS were included. Articles with a primary focus on CDSS development were excluded. No language or date restrictions were applied. We used qualitative content analysis to identify determinants and organize them into higher-order themes, which were then reflexively mapped to the NASSS framework.

RESULTS

Forty-four publications were included. These comprised a range of study designs, geographic locations, participants, technology types, CDSS functions, and clinical contexts of implementation. A total of 227 individual barriers and 130 individual facilitators were identified across the included studies. The most commonly reported influences on implementation were fit of CDSS with workflows (19 studies), the usefulness of the CDSS output in practice (17 studies), CDSS technical dependencies and design (16 studies), trust of users in the CDSS input data and evidence base (15 studies), and the contextual fit of the CDSS with the user's role or clinical setting (14 studies). Most determinants could be appropriately categorized into domains of the NASSS framework with barriers and facilitators in the "Technology," "Organization," and "Adopters" domains most frequently reported. No determinants were assigned to the "Embedding and Adaptation Over Time" domain.

CONCLUSIONS

This review identified the most common determinants which could be targeted for modification to either remove barriers or facilitate the adoption and use of CDSS within hospitals. Greater adoption of implementation theory should be encouraged to support CDSS implementation.

Using the Non-Adoption, Abandonment, Scale-Up, Spread, and Sustainability (NASSS) Framework to Identify Barriers and Facilitators for the Implementation of Digital Twins in Cardiovascular Medicine

A digital twin is a computer-based "virtual" representation of a complex system, updated using data from the "real" twin. Digital twins are established in product manufacturing, aviation, and infrastructure and are attracting significant attention in medicine. In medicine, digital twins hold great promise to improve prevention of cardiovascular diseases and enable personalised health care through a range of Internet of Things (IoT) devices which collect patient data in real-time. However, the promise of such new technology is often met with many technical, scientific, social, and ethical challenges that need to be overcome-if these challenges are not met, the technology is therefore less likely on balance to be adopted by stakeholders. The purpose of this work is to identify the facilitators and barriers to the implementation of digital twins in cardiovascular medicine. Using, the Non-adoption, Abandonment, Scale-up, Spread, and Sustainability (NASSS) framework, we conducted a document analysis of policy reports, industry websites, online magazines, and academic publications on digital twins in cardiovascular medicine, identifying potential facilitators and barriers to adoption. Our results show key facilitating factors for implementation: preventing cardiovascular disease, in silico simulation and experimentation, and personalised care. Key barriers to implementation included: establishing real-time data exchange, perceived specialist skills required, high demand for patient data, and ethical risks related to privacy and surveillance. Furthermore, the lack of empirical research on the attributes of digital twins by different research groups, the characteristics and behaviour of adopters, and the nature and extent of social, regulatory, economic, and political contexts in the planning and development process of these technologies is perceived as a major hindering factor to future implementation.

Digital behaviour change interventions to increase vegetable intake in adults: a systematic review

BACKGROUND

Digital interventions may help address low vegetable intake in adults, however there is limited understanding of the features that make them effective. We systematically reviewed digital interventions to increase vegetable intake to 1) describe the effectiveness of the interventions; 2) examine links between effectiveness and use of co-design, personalisation, behavioural theories, and/or a policy framework; and 3) identify other features that contribute to effectiveness.

METHODS

A systematic search strategy was used to identify eligible studies from MEDLINE, Embase, PsycINFO, Scopus, CINAHL, Cochrane Library, INFORMIT, IEEE Xplore and Clinical Trial Registries, published between January 2000 and August 2022. Digital interventions to increase vegetable intake were included, with effective interventions identified based on statistically significant improvement in vegetable intake. To identify policy-action gaps, studies were mapped across the three domains of the NOURISHING framework (i.e., behaviour change communication, food environment, and food system). Risk of bias was assessed using Cochrane tools for randomized, cluster randomized and non-randomized trials.

RESULTS

Of the 1,347 records identified, 30 studies were included. Risk of bias was high or serious in most studies (n = 25/30; 83%). Approximately one quarter of the included interventions (n = 8) were effective at improving vegetable intake. While the features of effective and ineffective interventions were similar, embedding of behaviour change theories (89% vs 61%) and inclusion of stakeholders in the design of the intervention (50% vs 38%) were more common among effective interventions. Only one (ineffective) intervention used true co-design. Although fewer effective interventions included personalisation (67% vs 81%), the degree of personalisation varied considerably between studies. All interventions mapped across the NOURISHING framework behaviour change communication domain, with one ineffective intervention also mapping across the food environment domain.

CONCLUSION

Few digital interventions identified in this review were effective for increasing vegetable intake. Embedding behaviour change theories and involving stakeholders in intervention design may increase the likelihood of success. The under-utilisation of comprehensive co-design methods presents an opportunity to ensure that personalisation approaches better meet the needs of target populations. Moreover, future digital interventions should address both behaviour change and food environment influences on vegetable intake.

Scale-up of a novel vital signs alert device to improve maternity care in Sierra Leone: a mixed methods evaluation of adoption

BACKGROUND

The CRADLE (Community blood pressure monitoring in Rural Africa: Detection of underLying pre-Eclampsia) Vital Signs Alert device-designed specifically to improve maternity care in low resource settings-had varying impact when trialled in different countries. To better understand the contextual factors that may contribute to this variation, this study retrospectively evaluated the adoption of CRADLE, during scale-up in Sierra Leone.

METHODS

This was a mixed methods study. A quantitative indicator of adoption (the proportion of facilities trained per district) was calculated from existing training records, then focus groups were held with 'CRADLE Champions' in each district (n = 32), to explore adoption qualitatively. Template Analysis was used to deductively interpret qualitative data, guided by the NASSS (non-adoption, abandonment, scale-up, spread, sustainability) Framework.

FINDINGS

Substantial but non-significant variation was found in the proportion of facilities trained in each district (range 59-90%) [X² (7, N = 8) = 10.419, p = 0.166]. Qualitative data identified complexity in two NASSS domains that may have contributed to this variation: 'the technology' (for example, charging issues, difficulty interpreting device output and concerns about ongoing procurement) and 'the organisation' (for example, logistical barriers to implementing training, infighting and high staff turnover). Key strategies mentioned to mitigate against these issues included: transparent communication at all levels; encouraging localised adaptations during implementation (including the involvement of community leaders); and selecting Champions with strong soft skills (particularly conflict resolution and problem solving).

CONCLUSIONS

Complexity related to the technology and the organisational context were found to influence the adoption of CRADLE in Sierra Leone, with substantial inter-district variation. These findings emphasise the importance of gaining an in-depth understanding of the specific system and context in which a new healthcare technology is being implemented. This study has implications for the ongoing scale-up of CRADLE, and for those implementing or evaluating other health technologies in similar contexts.

Adoption and scale-up of the cardiovascular Polypill: a realist institutional analysis

Efforts to promote the adoption and scale-up of health system innovations must contend with the existing institutional context. But there are no commonly used frameworks to ensure that the insights of actors involved in such institutional efforts connect to one another. To test and modify a potential framework-the 'four-by-four' framework-we interviewed researcher-entrepreneurs involved in the unfolding story of the cardiovascular Polypill. The framework has four types/levels of institutions that affect adoption and scale-up: (1) informal institutions (L1, e.g. social norms), (2) formal institutions (L2, e.g. government policies and regulation), (3) organizational structures (L3, e.g. organizational boards and mission) and (4) everyday exchange (L4, e.g. service delivery), vis-à-vis four potential entrepreneurial strategies in response: (1) abide by existing institutions, (2) evade them, (3) alter them and/or (4) exit entrepreneurial action. Using this framework, we conducted a realist-informed analysis to understand how context (i.e. institutions) and mechanism (i.e. entrepreneurial strategies) influence each other to shape outcomes (i.e. adoption and scale-up). We found that researcher-entrepreneurs began with efforts to abide with existing institutions but encountered institutional obstacles at each level. Efforts to abide were followed by seeking to evade and/or alter unfavourable institutions, with greater success evading and/or altering lower (L3 and L4) than upper (L1 and L2) institutions. Exit considerations followed the failure of the evade or alter strategy. Shifts between strategies were propelled by 'learning'. The 'four-by-four' framework can be used as a scaffold to generate narratives of adoption or scale-up efforts, a sensitizing tool to prospectively map out contingencies and a matrix to synthesize narratives and experiences across multiple innovations or settings. Used in these ways, the 'four-by-four' framework can help to optimize the transferability and cumulation of insights on how to promote the adoption and scale-up of health system innovations.

Barriers and Facilitators to the Use of Clinical Decision Support Systems in Primary Care: A Mixed-Methods Systematic Review

PURPOSE

To identify and quantify the barriers and facilitators to the use of clinical decision support systems (CDSSs) by primary care professionals (PCPs).

METHODS

A mixed-methods systematic review was conducted using a sequential synthesis design. PubMed/MEDLINE, PsycInfo, Embase, CINAHL, and the Cochrane library were searched in July 2021. Studies that evaluated CDSSs providing recommendations to PCPs and intended for use during a consultation were included. We excluded CDSSs used only by patients, described as concepts or prototypes, used with simulated cases, and decision supports not considered as CDSSs. A framework synthesis was performed according to the HOT-fit framework (Human, Organizational, Technology, Net Benefits), then a quantitative synthesis evaluated the impact of the HOT-fit categories on CDSS use.

RESULTS

A total of 48 studies evaluating 45 CDSSs were included, and 186 main barriers or facilitators were identified. Qualitatively, barriers and facilitators were classified as human (eg, perceived usefulness), organizational (eg, disruption of usual workflow), and technological (eg, CDSS user-friendliness), with explanatory elements. The greatest barrier to using CDSSs was an increased workload. Quantitatively, the human and organizational factors had negative impacts on CDSS use, whereas the technological factor had a neutral impact and the net benefits dimension a positive impact.

CONCLUSIONS

Our findings emphasize the need for CDSS developers to better address human and organizational issues, in addition to technological challenges. We inferred core CDSS features covering these 3 factors, expected to improve their usability in primary care.

Cocreation of a conversational agent to help patients adhere to their varenicline treatment: A study protocol

Objective

Varenicline is the most efficacious approved smoking cessation medication, making it one of the most cost-effective clinical interventions for reducing tobacco-related morbidity and mortality. Adhering to varenicline is strongly associated with smoking cessation. Healthbots have the potential to help people adhere to their medications by scaling up evidence-based behavioral interventions. In this protocol, we outline how we will follow the UK's Medical Research Council's guidance to codesign a theory-informed, evidence-based, and patient-centered healthbot to help people adhere to varenicline.

Methods

The study will utilize the Discover, Design and Build, and Test framework and will include three phases: (a) a rapid review and interviews with 20 patients and 20 healthcare providers to understand barriers and facilitators to varenicline adherence (Discover phase); (b) Wizard of Oz test to design the healthbot and get a sense of the questions that chatbot has to be able to answer (Design phase); and (c) building, training, and beta-testing the healthbot (Building and Testing phases) where the Nonadoption, Abandonment, Scale-up, Spread, and Sustainability framework will be used to develop the healthbot using the simplest sensible solution, and 20 participants will beta test the healthbot. We will use the Capability, Opportunity, Motivation-Behavior (COM-B) model of behavior change and its associated framework, the Theoretical Domains Framework, to organize the findings.

Conclusions

The present approach will enable us to systematically identify the most appropriate features for the healthbot based on a well-established behavioral theory, the latest scientific evidence, and end users' and healthcare providers' knowledge.

Process evaluation of the scale-up of integrated diabetes and hypertension care in Belgium, Cambodia and Slovenia (the SCUBY Project): a study protocol

INTRODUCTION

Integrated care interventions for type 2 diabetes (T2D) and hypertension (HT) are effective, yet challenges exist with regard to their implementation and scale-up. The 'SCale-Up diaBetes and hYpertension care' (SCUBY) Project aims to facilitate the scale-up of integrated care for T2D and HT through the co-creation and implementation of contextualised scale-up roadmaps in Belgium, Cambodia and Slovenia. We hereby describe the plan for the process and scale-up evaluation of the SCUBY Project. The specific goals of the process and scale-up evaluation are to (1) analyse how, and to what extent, the roadmap has been implemented, (2) assess how the differing contexts can influence the implementation process of the scale-up strategies and (3) assess the progress of the scale-up.

METHODS AND ANALYSIS

A comprehensive framework was developed to include process and scale-up evaluation embedded in implementation science theory. Key implementation outcomes include acceptability, feasibility, relevance, adaptation, adoption and cost of roadmap activities. A diverse range of predominantly qualitative tools-including a policy dialogue reporting form, a stakeholder follow-up interview and survey, project diaries and policy mapping-were developed to assess how stakeholders perceive the scale-up implementation process and adaptations to the roadmap. The role of context is considered relevant, and barriers and facilitators to scale-up will be continuously assessed.

ETHICS AND DISSEMINATION

Ethical approval has been obtained from the Institutional Review Board (ref. 1323/19) at the Institute of Tropical Medicine (Antwerp, Belgium). The SCUBY Project presents a comprehensive framework to guide the process and scale-up evaluation of complex interventions in different health systems. We describe how implementation outcomes, mechanisms of impact and scale-up outcomes can be a basis to monitor adaptations through a co-creation process and to guide other scale-up interventions making use of knowledge translation and co-creation activities.

Barriers and enablers to implementing and using clinical decision support systems for chronic diseases: a qualitative systematic review and meta-aggregation

Background

Clinical decision support (CDS) is increasingly used to facilitate chronic disease care. Despite increased availability of electronic health records and the ongoing development of new CDS technologies, uptake of CDS into routine clinical settings is inconsistent. This qualitative systematic review seeks to synthesise healthcare provider experiences of CDS—exploring the barriers and enablers to implementing, using, evaluating, and sustaining chronic disease CDS systems.

Methods

A search was conducted in Medline, CINAHL, APA PsychInfo, EconLit, and Web of Science from 2011 to 2021. Primary research studies incorporating qualitative findings were included if they targeted healthcare providers and studied a relevant chronic disease CDS intervention. Relevant CDS interventions were electronic health record-based and addressed one or more of the following chronic diseases: cardiovascular disease, diabetes, chronic kidney disease, hypertension, and hypercholesterolaemia. Qualitative findings were synthesised using a meta-aggregative approach.

Results

Thirty-three primary research articles were included in this qualitative systematic review. Meta-aggregation of qualitative data revealed 177 findings and 29 categories, which were aggregated into 8 synthesised findings. The synthesised findings related to clinical context, user, external context, and technical factors affecting CDS uptake. Key barriers to uptake included CDS systems that were simplistic, had limited clinical applicability in multimorbidity, and integrated poorly into existing workflows. Enablers to successful CDS interventions included perceived usefulness in providing relevant clinical knowledge and structured chronic disease care; user confidence gained through training and post training follow-up; external contexts comprised of strong clinical champions, allocated personnel, and technical support; and CDS technical features that are both highly functional, and attractive.

Conclusion

This systematic review explored healthcare provider experiences, focussing on barriers and enablers to CDS use for chronic diseases. The results provide an evidence-base for designing, implementing, and sustaining future CDS systems. Based on the findings from this review, we highlight actionable steps for practice and future research.

Trial registration

PROSPERO CRD42020203716

Supplementary Information

The online version contains supplementary material available at 10.1186/s43058-022-00326-x.

Identifying existing approaches used to evaluate the sustainability of evidence-based interventions in healthcare: an integrative review

BACKGROUND

There is limited evidence to evaluate the sustainability of evidence-based interventions (EBIs) for healthcare improvement. Through an integrative review, we aimed to identify approaches to evaluate the sustainability of evidence-based interventions (EBIs) and sustainability outcomes.

METHODS

Following Whittemore and Knafl's methodological process: (1) problem identification; (2) literature search; (3) data evaluation; (4) data analysis; and (5) presentation, a comprehensive search strategy was applied across five databases. Included studies were not restricted by research design; and had to evaluate the sustainability of an EBI in a healthcare context. We assessed the methodological quality of studies using the Mixed Methods Appraisal Tool.

RESULTS

Of 18,783 articles retrieved, 64 fit the inclusion criteria. Qualitative designs were most commonly used for evaluation (48%), with individual interviews as the predominant data collection method. Timing of data collection varied widely with post-intervention data collection most frequent (89%). Of the 64 studies, 44% used a framework, 26% used a model, 11% used a tool, 5% used an instrument, and 14% used theory as their primary approach to evaluate sustainability. Most studies (77%) did not measure sustainability outcomes, rather these studies focused on sustainability determinants.

DISCUSSION

It is unclear which approach/approaches are most effective for evaluating sustainability and what measures and outcomes are most commonly used. There is a disconnect between evaluating the factors that may shape sustainability and the outcomes approaches employed to measure sustainability. Our review offers methodological recommendations for sustainability evaluation research and highlights the importance in understanding mechanisms of sustainability to advance the field.

Information and communications technology, health, and gender equality: Empirical evidence from a panel of Pacific developing economies

Information and communications technology (ICT) has been widely embraced in many developing economies in recent times. Extant research reveals that ICT increases economic growth. Beyond economic growth, improved access to information, markets and economic opportunities via information and communications technology have the potential to influence other dimensions of public welfare. This study quantitatively examines the effects of ICT on selected health and gender dimensions of Pacific Island developing countries’ populations. The results show a statistically significant and positive impact of ICT on health and gender outcomes. Our results are robust with an alternative modeling approach, different control variables, and different measures of health and gender outcomes. We further establish that the health outcome of technology has a valid pass-through of income. The study suggests policy implications for the Pacific and other developing countries striving to enhance the health and gender outcomes of SGDs.

Assessing the Potential of Wearable Health Monitors for Health System Strengthening in Low- and Middle-Income Countries: A Prospective Study of Technology Adoption in Cambodia

Wearable health monitors are a rapidly evolving technology that may offer new opportunities for strengthening health system responses to cardiovascular and other non-communicable diseases (NCDs) in low- and middle-income countries (LMICs). In light of this, we explored opportunities for, and potential challenges to, technology adoption in Cambodia, considering the complexity of contextual factors that may influence product uptake and sustainable health system integration. Data collection for this study involved in-depth interviews with national and international stakeholders and a literature review. The analytical approach was guided by concepts and categories derived from the non-adoption, abandonment, scale-up, spread, and sustainability (NASSS) framework—an evidence-based framework that was developed for studying health technology adoption and the challenges to scale-up, spread and sustainability of such technologies in health service organizations. Three potential applications of health wearables for the prevention and control of NCDs in Cambodia were identified: health promotion, follow-up and monitoring of patients and surveys of NCD risk factors. However, several challenges to technology adoption emerged across the research domains, associated with the intended adopters, the organization of the national health system, the wider infrastructure, the regulatory environment and the technology itself. Our findings indicate that, currently, wearables could be best used to conduct surveys of NCD risk factors in Cambodia and in other LMICs with similar health system profiles. In the future, a more integrated use of wearables to strengthen monitoring and management of patients could be envisaged, although this would require careful consideration of feasibility and organizational issues.

The evaluation of the role of technology in the pathways to comorbidity care implementation project to improve management of comorbid substance use and mental disorders

In substance use treatment settings, comorbid mental health problems can occur in up to 70% of people. An integrated approach for managing comorbidity, implementing evidence-based intervention in drug and alcohol settings, remains problematic. Technology can help in adopting evidence-based practice to implement effective treatment healthcare pathways. This study sought to examine aspects of tailored portal utilization (barriers and facilitators) by participants taking part in a program aimed at improving the implementation of evidence-based practice for comorbidity management Pathways to Comorbidity Care (PCC).

Scaling Digital Health Innovation: Developing a New 'Service Readiness Level' Framework of Evidence

Digital innovation has scaled exponentially in many sectors including tourism, banking, and retail. It is well cited that the health sector is slower to embrace digital health innovations (DHI) beyond the pilot stage and consequently, many successful DHI pilot projects have failed to scale up. Such failure arises in part from a knowledge gap around what type and level of evidence are needed to convince implementers and decision makers to fund, endorse, or adopt new innovations into care delivery systems and sustainable practice. Much is known about the range of DHI evaluation methods used; however, less is published on the evidence that decision makers need to move innovations to scale. This paper draws on interviews (N = 18) with decision makers/project leads engaged in DHI in Scotland to identify what evidence matters when making DHI adoption/scale decisions. The results are used to present a heuristic service readiness level (SRL) framework that captures the changing nature of the evidence base required over a project lifecycle for progression to scale. We utilise this framework to discuss ‘what evidence’ is required and ‘how data accumulate’ over time to assist project teams to build a ‘DHI case for scale’.

Planning and Evaluating Remote Consultation Services: A New Conceptual Framework Incorporating Complexity and Practical Ethics

Establishing and running remote consultation services is challenging politically (interest groups may gain or lose), organizationally (remote consulting requires implementation work and new roles and workflows), economically (costs and benefits are unevenly distributed across the system), technically (excellent care needs dependable links and high-quality audio and images), relationally (interpersonal interactions are altered), and clinically (patients are unique, some examinations require contact, and clinicians have deeply-held habits, dispositions and norms). Many of these challenges have an under-examined ethical dimension. In this paper, we present a novel framework, Planning and Evaluating Remote Consultation Services (PERCS), built from a literature review and ongoing research. PERCS has 7 domains-the reason for consulting, the patient, the clinical relationship, the home and family, technologies, staff, the healthcare organization, and the wider system-and considers how these domains interact and evolve over time as a complex system. It focuses attention on the organization's digital maturity and digital inclusion efforts. We have found that both during and beyond the pandemic, policymakers envisaged an efficient, safe and accessible remote consultation service delivered through state-of-the art digital technologies and implemented via rational allocation criteria and quality standards. In contrast, our empirical data reveal that strategic decisions about establishing remote consultation services, allocation decisions for appointment type (phone, video, e-, face-to-face), and clinical decisions when consulting remotely are fraught with contradictions and tensions-for example, between demand management and patient choice-leading to both large- and small-scale ethical dilemmas for managers, support staff, and clinicians. These dilemmas cannot be resolved by standard operating procedures or algorithms. Rather, they must be managed by attending to here-and-now practicalities and emergent narratives, drawing on guiding principles applied with contextual judgement. We complement the PERCS framework with a set of principles for informing its application in practice, including education of professionals and patients.

A tale of 'politics and stars aligning': analysing the sustainability of scaled up digital tools for front-line health workers in India

INTRODUCTION

India has become a lighthouse for large-scale digital innovation in the health sector, particularly for front-line health workers (FLHWs). However, among scaled digital health solutions, ensuring sustainability remains elusive. This study explores the factors underpinning scale-up of digital health solutions for FLHWs in India, and the potential implications of these factors for sustainability.

METHODS

We assessed five FLHW digital tools scaled at the national and/or state level in India. We conducted in-depth interviews with implementers, technology and technical partners (n=11); senior government stakeholders (n=5); funders (n=1) and evaluators/academics (n=3). Emergent themes were grouped according to a broader framework that considered the (1) digital solution; (2) actors; (3) processes and (4) context.

RESULTS

The scale-up of digital solutions was facilitated by their perceived value, bounded adaptability, support from government champions, cultivation of networks, sustained leadership and formative research to support fit with the context and population. However, once scaled, embedding digital health solutions into the fabric of the health system was hampered by challenges related to transitioning management and ownership to government partners; overcoming government procurement hurdles; and establishing committed funding streams in government budgets. Strong data governance, continued engagement with FLHWs and building a robust evidence base, while identified in the literature as critical for sustainability, did not feature strongly among respondents. Sustainability may be less elusive once there is more consensus around the roles played between national and state government actors, implementing and technical partners and donors.

CONCLUSION

The use of digital tools by FLHWs offers much promise for improving service delivery and health outcomes in India. However, the pathway to sustainability is bespoke to each programme and should be planned from the outset by investing in people, relationships and service delivery adjustments to navigate the challenges involved given the dynamic nature of digital tools in complex health systems.

Adopt, adapt, or abandon technology-supported person-centred care initiatives: healthcare providers' beliefs matter

BACKGROUND

Technology support and person-centred care are the new mantra for healthcare programmes in Western societies. While few argue with the overarching philosophy of person-centred care or the potential of information technologies, there is less agreement on how to make them a reality in everyday clinical practice. In this paper, we investigate how individual healthcare providers at four innovation arenas in Scandinavia experienced the implementation of technology-supported person-centred care for people with long-term care needs by using the new analytical framework nonadoption, abandonment, and challenges to the scale-up, spread, and sustainability (NASSS) of health and care technologies. We also discuss the usability and sensitivity of the NASSS framework for those seeking to plan, implement, and evaluate technology-supported healthcare programmes. This study is part of an interdisciplinary research and development project called Patients and Professionals in Partnership (2016-2020). It originates at one of ten work packages in this project.

METHOD

The main data consist of ethnographic field observations at the four innovation arenas and 29 interviews with involved healthcare providers. To ensure continuous updates and status on work in the four innovation arenas, we have also participated in a total of six annual network meetings arranged by the project.

RESULTS

While the NASSS framework is very useful for identifying and communicating challenges with the adoption and spread of technology-supported person-centred care initiatives, we found it less sensitive towards capturing the dedication, enthusiasm, and passion for care transformation that we found among the healthcare providers in our study. When it comes to technology-supported person-centred care, the point of no return has passed for the involved healthcare providers. To them, it is already a definite part of the future of healthcare services. How to overcome barriers and obstacles is pragmatically approached.

CONCLUSION

Increased knowledge about healthcare providers and their visions as potential assets for care transformation might be critical for those seeking to plan, implement, and evaluate technology-supported healthcare programmes.

A Novel Educational Prescription Web-Based Application to Support Education for Caregivers of People Living With Dementia: Development and Usability Study With Clinicians

Background

It is estimated that 564,000 Canadians are currently living with dementia and there are approximately 486,000 to 1.1 million informal family/friend caregivers. Family/friend caregivers often receive little to no education or training about dementia but are expected to provide ongoing support for a complex condition. Web-based family/friend caregiver interventions may be helpful, but little is known about how best to implement them.

Objective

The objectives of this study were to 1) design and develop a novel education prescription application to help scale and spread web-based dementia education to family/friend caregivers, 2) conduct user testing, and 3) conduct a larger-scale field trial.

Methods

A novel education prescription web-based application was designed and developed. Initial user testing used task completion and the “think aloud” technique with a small sample of representative clinicians who work with people living with dementia and family/friend caregivers. Following iterative incorporation of feedback, a larger field trial was conducted with a convenience sample of clinicians. Account invitations were sent to 55 clinicians and, following a 2-month trial period, surveys were administered to participants including the System Usability Scale and the Net Promoter Score.

Results

During the initial user testing phase, participants (N=7) from representative disciplines easily completed associated tasks, and had very positive feedback with respect to the usability of the application. The System Usability Scale score during this phase was 91.4. Suggestions from feedback were incorporated into the application. During the larger field trial phase, participants (total N=55; activated account n=17; did not activate account n=38) were given access to the iGeriCare education prescription application. During this period, 2 participants created educational prescriptions; a total of 3 educational prescriptions were sent. Survey completers who did not activate their account (n=5) identified that their lack of use was due to time constraints, competing priorities, or forgetting to use the application. Survey completers who activated their account (n=5) identified their lower use was due to lack of time, lack of eligible family/friend caregivers during trial period, and competing priorities due to the COVID-19 pandemic. The System Usability Scale score during this phase was 78.75, and the Net Promoter Score was 50.

Conclusions

Study findings indicate a generally positive response for the usability of a web-based application for clinicians to prescribe dementia education to family/friend caregivers. The dissonance between the promising data and widespread enthusiasm for the design and purpose of the education prescription application found in the initial user testing phase and subsequent lack of significant adoption in the field trial represents both an important lesson for other novel health technologies and a potential area for further investigation. Further research is required to better understand factors associated with implementation of this type of intervention and impact on dissemination of education to family/friend caregivers.

Research protocol for the validation of a new portable technology for real-time continuous monitoring of Early Warning Score (EWS) in hospital practice and for an early-stage multistakeholder assessment

INTRODUCTION

The real-time continuous monitoring of vital parameters in patients affected by multiple chronic conditions and/or COVID-19 can lead to several benefits to the Italian National Healthcare System (IT-NHS). The UBiquitous Integrated CARE (UBICARE) technology is a novel health digital platform at the validation stage in hospital setting. UBICARE might support the urgent need for digitalisation and early intervention, as well as minimise the face-to-face delivery of care in both hospital and community-based care settings. This research protocol aims to design an early-stage assessment of the multidimensional impact induced by UBICARE within the IT-NHS alongside technology validation in a hospital ward.

METHODS AND ANALYSIS

The targeted patients will be medium/high-risk hypertensive individuals as an illustrative first example of how UBICARE might bring benefits to susceptible patients. A mixed-method study will be applied to incorporate to the validation study a multistakeholder perspective, including perceived patient experiences and preferences, and facilitate technology adoption. First, semistructured interviews will be undertaken with a variety of stakeholders including clinicians, health managers and policy-makers to capture views on the likely technology utility, economic sustainability, impact of adoption in hospital practice and alternative adoption scenarios. Second, a monocentric, non-randomised and non-comparative clinical study, supplemented by the administration of standardised usability questionnaires to patients and health professionals, will validate the use of UBICARE in hospital practice. Finally, the results of the previous stages will be discussed in a multidisciplinary-facilitated workshop with IT-NHS relevant stakeholders to reconcile stakeholders' perspectives. Limitations include a non-random recruitment strategy in the clinical study, small sample size of the key stakeholders and potential stakeholder recruitment bias introduced by the research technique.

ETHICS AND DISSEMINATION

The Ethics Committee for Clinical Experimentation of Tuscany Region approved the protocol. Participation in this study is voluntary. Study results will be disseminated through peer-reviewed publications and academic conferences.