Theoretical and methodological considerations in evaluating large-scale health information technology change programmes

Researching big IT in the UK National Health Service: A systematic review of theory-based studies

OBJECTIVE

To identify and discuss theory-based studies of large-scale health information technology programs in the UK National Health Service.

MATERIALS AND METHODS

Using the PRISMA systematic review framework, we searched Scopus, PubMed and CINAHL databases from inception to March 2022 for theory-based studies of large-scale health IT implementations. We undertook detailed full-text analyses of papers meeting our inclusion criteria.

RESULTS

Forty-six studies were included after assessment for eligibility, of which twenty-five applied theories from the information systems arena (socio-technical approaches, normalization process theory, user acceptance theories, diffusion of innovation), twelve from sociology (structuration theory, actor-network theory, institutional theory), while nine adopted other theories. Most investigated England's National Program for IT (2002-2011), exploring various technologies among which electronic records predominated. Research themes were categorized into user factors, program factors, process outcomes, clinical impact, technology, and organizational factors. Most research was qualitative, often using a case study strategy with a longitudinal or cross-sectional approach. Data were typically collected through interviews, observation, and document analysis; sampling was generally purposive; and most studies used thematic or related analyses. Theories were generally applied in a superficial or fragmentary manner; and articles frequently lacked detail on how theoretical constructs and relationships aided organization, analysis, and interpretation of data.

CONCLUSION

Theory-based studies of large NHS IT programs are relatively uncommon. As large healthcare programs evolve over a long timeframe in complex and dynamic environments, wider adoption of theory-based methods could strengthen the explanatory and predictive utility of research findings across multiple evaluation studies. Our review has confirmed earlier suggestions for theory selection, and we suggest there is scope for more explicit use of such theoretical constructs to strengthen the conceptual foundations of health informatics research. Additionally, the challenges of large national health informatics programs afford wide-ranging opportunities to test, refine, and adapt sociological and information systems theories.

Innovative solutions for disease management

The increasing prevalence of chronic diseases is a driver for emerging big data technologies for healthcare including digital platforms for data collection, systems for active patient engagement and education, therapy specific predictive models, optimized patient pathway models. Powerful bioelectronic medicine tools for data collection, analysis and visualization allow for joint processing of large volumes of heterogeneous data, which in turn can produce new insights about patient outcomes and alternative interpretations of clinical patterns that can lead to implementation of optimized clinical decisions and clinical patient pathway by healthcare professionals.With this perspective, we identify innovative solutions for disease management and evaluate their impact on patients, payers and society, by analyzing their impact in terms of clinical outcomes (effectiveness, safety, and quality of life) and economic outcomes (cost-effectiveness, savings, and productivity).As a result, we propose a new approach based on the main pillars of innovation in the disease management area, i.e. progressive patient care models, patient-centric approaches, bioelectronics for precise medicine, and lean management that, combined with an increase in appropriate private-public-citizen-partnership, leads towards Patient-Centric Healthcare.

The Large-Scale Implementation of a Health Information System in Brazilian University Hospitals: Process and Outcomes

Governments around the globe are paving the way for healthcare services that can have a profound impact on the overall well-being and development of their nations. However, government programs to implement health information technologies on a large-scale are challenging, especially in developing countries. In this article, the process and outcomes of the large-scale implementation of a hospital information system for the management of Brazilian university hospitals are analyzed. Based on a qualitative approach, this research involved 21 hospitals and comprised a documentary search, interviews with 24 hospital managers and two system user focus groups, and a questionnaire of 736 respondents. Generally, we observed that aspects relating to the wider context of system implementation (macro level), the managerial structure, cultural nuances, and political dynamics within each hospital (meso level), as well as the technology, work activities, and individuals themselves (micro level) acted as facilitators and/or obstacles to the implementation process. The dynamics and complex interactions established between these aspects had repercussions on the process, including the extended time necessary to implement the national program and the somewhat mixed outcomes obtained by hospitals in the national network. Mostly positive, these outcomes were linked to the eight emerging dimensions of practices and work processes; planning, control, and decision making; transparency and accountability; optimization in the use of resources; productivity of professionals; patient information security; safety and quality of care; and improvement in teaching and research. We argued here that to maximize the potential of information technology in healthcare on a large-scale, an integrative and cooperative vision is required, along with a high capacity for change management, considering the different regional, local, and institutional contexts.

Using Theories, Models, and Frameworks to Inform Implementation Cycles of Computerized Clinical Decision Support Systems in Tertiary Health Care Settings: Scoping Review

BACKGROUND

Computerized clinical decision support systems (CDSSs) are essential components of modern health system service delivery, particularly within acute care settings such as hospitals. Theories, models, and frameworks may assist in facilitating the implementation processes associated with CDSS innovation and its use within these care settings. These processes include context assessments to identify key determinants, implementation plans for adoption, promoting ongoing uptake, adherence, and long-term evaluation. However, there has been no prior review synthesizing the literature regarding the theories, models, and frameworks that have informed the implementation and adoption of CDSSs within hospitals.

OBJECTIVE

This scoping review aims to identify the theory, model, and framework approaches that have been used to facilitate the implementation and adoption of CDSSs in tertiary health care settings, including hospitals. The rationales reported for selecting these approaches, including the limitations and strengths, are described.

METHODS

A total of 5 electronic databases were searched (CINAHL via EBSCOhost, PubMed, Scopus, PsycINFO, and Embase) to identify studies that implemented or adopted a CDSS in a tertiary health care setting using an implementation theory, model, or framework. No date or language limits were applied. A narrative synthesis was conducted using full-text publications and abstracts. Implementation phases were classified according to the "Active Implementation Framework stages": exploration (feasibility and organizational readiness), installation (organizational preparation), initial implementation (initiating implementation, ie, training), full implementation (sustainment), and nontranslational effectiveness studies.

RESULTS

A total of 81 records (42 full text and 39 abstracts) were included. Full-text studies and abstracts are reported separately. For full-text studies, models (18/42, 43%), followed by determinants frameworks (14/42,33%), were most frequently used to guide adoption and evaluation strategies. Most studies (36/42, 86%) did not list the limitations associated with applying a specific theory, model, or framework.

CONCLUSIONS

Models and related quality improvement methods were most frequently used to inform CDSS adoption. Models were not typically combined with each other or with theory to inform full-cycle implementation strategies. The findings highlight a gap in the application of implementation methods including theories, models, and frameworks to facilitate full-cycle implementation strategies for hospital CDSSs.

Analysis of the nature and contributory factors of medication safety incidents following hospital discharge using National Reporting and Learning System (NRLS) data from England and Wales: a multi-method study

Introduction

Improving medication safety during transition of care is an international healthcare priority. While existing research reveals that medication-related incidents and associated harms may be common following hospital discharge, there is limited information about their nature and contributory factors at a national level which is crucial to inform improvement strategy.

Aim

To characterise the nature and contributory factors of medication-related incidents during transition of care from secondary to primary care.

Method

A retrospective analysis of medication incidents reported to the National Reporting and Learning System (NRLS) in England and Wales between 2015 and 2019. Descriptive analysis identified the frequency and nature of incidents and content analysis of free text data, coded using the Patient Safety Research Group (PISA) classification, examined the contributory factors and outcome of incidents.

Results

A total of 1121 medication-related incident reports underwent analysis. Most incidents involved patients over 65 years old (55%, n = 626/1121). More than one in 10 (12.6%, n = 142/1121) incidents were associated with patient harm. The drug monitoring (17%) and administration stages (15%) were associated with a higher proportion of harmful incidents than any other drug use stages. Common medication classes associated with incidents were the cardiovascular (n = 734) and central nervous (n = 273) systems. Among 408 incidents reporting 467 contributory factors, the most common contributory factors were organisation factors (82%, n = 383/467) (mostly related to continuity of care which is the delivery of a seamless service through integration, co-ordination, and the sharing of information between different providers), followed by staff factors (16%, n = 75/467).

Conclusion

Medication incidents after hospital discharge are associated with patient harm. Several targets were identified for future research that could support the development of remedial interventions, including commonly observed medication classes, older adults, increase patient engagement, and improve shared care agreement for medication monitoring post hospital discharge.

Plain language summary

Study using reports about unsafe or substandard care mainly written by healthcare professionals to better understand the type and causes of medication safety problems following hospital discharge Why was the study done? The safe use of medicines after hospital discharge has been highlighted by the World Health Organization as an important target for improvement in patient care. Yet, the type of medication problems which occur, and their causes are poorly understood across England and Wales, which may hamper our efforts to create ways to improve care as they may not be based on what we know causes the problem in the first place.What did the researchers do? The research team studied medication safety incident reports collected across England and Wales over a 5-year period to better understand what kind of medication safety problems occur after hospital discharge and why they happen, so we can find ways to prevent them from happening in future.What did the researchers find? The total number of incident reports studied was 1121, and the majority (n = 626) involved older people. More than one in ten of these incidents caused harm to patients. The most common medications involved in the medication safety incidents were for cardiovascular diseases such as high blood pressure, conditions such as mental illness, pain and neurological conditions (e.g., epilepsy) and other illnesses such as diabetes. The most common causes of these incidents were because of the organisation rules, such as information sharing, followed by staff issues, such as not following protocols, individual mistakes and not having the right skills for the task.What do the findings mean? This study has identified some important targets that can be a focus of future efforts to improve the safe use of medicines after hospital discharge. These include concentrating attention on medication for the cardiovascular and central nervous systems (e.g., via incorporating them in prescribing safety indicators and pharmaceutical prioritisation tools), staff skill mix (e.g., embedding clinical pharmacist roles at key parts of the care pathway where greatest risk is suspected), and implementation of electronic interventions to improve timely communication of medication and other information between healthcare providers.

Evaluation of the FAST-M maternal sepsis intervention in Pakistan: A qualitative exploratory study

INTRODUCTION

The World Health Organization and partners developed and evaluated a maternity-specific sepsis care bundle called 'FAST-M' for low-resource settings. However, this bundle has not yet been studied in Asia. Our study sought to evaluate the perceptions of healthcare providers about the implementation of the FAST-M intervention in Pakistan.

MATERIALS AND METHODS

The study was conducted at a public sector hospital in Hyderabad. We conducted three focus group discussions with healthcare providers including doctors, nurses, and healthcare administrators (n = 22) who implemented the FAST-M intervention. The Consolidated Framework for Implementation Research was used as a guiding framework for data collection and analysis. The data were analyzed using a thematic analysis approach and deductive methods.

RESULTS

Five overarching themes emerged: (I) FAST-M intervention and its significance including HCPs believing in the advantages of using the intervention to improve clinical practices; (II) Influence of outer and inner settings including non-availability of resources in the facility for sepsis care; (III) HCPs perceptions about sustainability, which were positive (IV) Integration into the clinical setting including HCPs views on the existing gaps, for example, shortage of HCPs and communication gaps, and their recommendations to improve these; and (V) Outcomes of the intervention including improved clinical processes and outcomes using the FAST-M intervention. Significant improvement in patient monitoring and FAST-M bundle completion within an hour of diagnosis of sepsis was reported by the HCPs.

CONCLUSIONS

The healthcare providers' views were positive about the intervention, its outcomes, and long-term sustainability. The qualitative data provided findings on the acceptability of the overall implementation processes to support subsequent scaling up of the intervention.

The Tension between National and Local Concerns in Preparing for Large-Scale Generic Systems in Healthcare

Large-scale generic systems are typically adapted to local practice through configuration. This is especially important in healthcare, which involves a plurality of institutions and users. However, the decision to acquire a generic system in public healthcare is typically founded on regional and national health policy goals, which often are translated into various forms of standardization. As a result, national and regional health policy interests may stand in contrast to interests on the local level. Therefore, we analyze how national and local concerns are weighed against each other in the preparations for implementing large-scale generic systems in healthcare. We explore what role configuration plays and what the prospects are for long-term development. We contribute with insight into how the organizational consequences of generic systems are formed already in the preparation phase and point to how configuration easily results in standardization, thereby basically privileging national and regional health goals at the expense of local needs. Empirically, we focus on the preparations for implementing the Epic electronic health record in Central Norway.

Driving digital health transformation in hospitals: a formative qualitative evaluation of the English Global Digital Exemplar programme

BACKGROUND

There is currently a strong drive internationally towards creating digitally advanced healthcare systems through coordinated efforts at a national level. The English Global Digital Exemplar (GDE) programme is a large-scale national health information technology change programme aiming to promote digitally-enabled transformation in secondary healthcare provider organisations by supporting relatively digitally mature provider organisations to become international centres of excellence.

AIM

To qualitatively evaluate the impact of the GDE programme in promoting digital transformation in provider organisations that took part in the programme.

METHODS

We conducted a series of in-depth case studies in 12 purposively selected provider organisations and a further 24 wider case studies of the remaining organisations participating in the GDE programme. Data collected included 628 interviews, non-participant observations of 190 meetings and workshops and analysis of 9 documents. We used thematic analysis aided by NVivo software and drew on sociotechnical theory to analyse the data.

RESULTS

We found the GDE programme accelerated digital transformation within participating provider organisations. This acceleration was triggered by: (1) dedicated funding and the associated requirement for matched internal funding, which in turn helped to prioritise digital transformation locally; (2) governance requirements put in place by the programme that helped strengthen existing local governance and project management structures and supported the emergence of a cadre of clinical health informatics leaders locally; and (3) reputational benefits associated with being recognised as a centre of digital excellence, which facilitated organisational buy-in for digital transformation and increased negotiating power with vendors.

CONCLUSION

The GDE programme has been successful in accelerating digital transformation in participating provider organisations. Large-scale digital transformation programmes in healthcare can stimulate local progress through protected funding, putting in place governance structures and leveraging reputational benefits for participating provider organisations, around a coherent vision of transformation.

Promoting inter-organisational knowledge sharing: A qualitative evaluation of England's Global Digital Exemplar and Fast Follower Programme

BACKGROUND

The Global Digital Exemplar (GDE) Programme was designed to promote the digitisation of hospital services in England. Selected provider organisations that were reasonably digitally-mature were funded with the expectation that they would achieve internationally recognised levels of excellence and act as exemplars ('GDE sites') and share their learning with somewhat less digitally-mature Fast Follower (FF) sites.

AIMS

This paper explores how partnerships between GDE and FF sites have promoted knowledge sharing and learning between organisations.

METHODS

We conducted an independent qualitative longitudinal evaluation of the GDE Programme, collecting data across 36 provider organisations (including acute, mental health and speciality), 12 of which we studied as in-depth ethnographic case studies. We used a combination of semi-structured interviews with programme leads, vendors and national policy leads, non-participant observations of meetings and workshops, and analysed national and local documents. This allowed us to explore both how inter-organisational learning and knowledge sharing was planned, and how it played out in practice. Thematic qualitative analysis, combining findings from diverse data sources, was facilitated by NVivo 11 and drew on sociotechnical systems theory.

RESULTS

Formally established GDE and FF partnerships were perceived to enhance learning and accelerate adoption of technologies in most pairings. They were seen to be most successful where they had encouraged, and were supported by, informal knowledge networking, driven by the mutual benefits of information sharing. Informal networking was enhanced where the benefits were maximised (for example where paired sites had implemented the same technological system) and networking costs minimised (for example by geographical proximity, prior links and institutional alignment). Although the intervention anticipated uni-directional learning between exemplar sites and 'followers', in most cases we observed a two-way flow of information, with GDEs also learning from FFs, through informal networking which also extended to other health service providers outside the Programme. The efforts of the GDE Programme to establish a learning ecosystem has enhanced the profile of shared learning within the NHS.

CONCLUSIONS

Inter-organisational partnerships have produced significant gains for both follower (FF) and exemplar (GDE) sites. Formal linkages were most effective where they had facilitated, and were supported by, informal networking. Informal networking was driven by the mutual benefits of information sharing and was optimised where sites were well aligned in terms of technology, geography and culture. Misalignments that created barriers to networking between organisations in a few cases were attributed to inappropriate choice of partners. Policy makers seeking to promote learning through centrally directed mechanisms need to create a framework that enables networking and informal knowledge transfer, allowing local organisations to develop bottom-up collaboration and exchanges, where they are productive, in an organic manner.

Development, Status Quo, and Challenges to China's Health Informatization During COVID-19: Evaluation and Recommendations

By applying advanced health information technology to the health care field, health informatization helps optimize health resource allocation, improve health care services, and realize universal health coverage. COVID-19 has tested the status quo of China's health informatization, revealing challenges to the health care system. This viewpoint evaluates the development, status quo, and practice of China's health informatization, especially during COVID-19, and makes recommendations to address the health informatization challenges. We collected, assessed, and evaluated data on the development of China's health informatization from five perspectives-health information infrastructure, information technology (IT) applications, financial and intellectual investment, health resource allocation, and standard system-and discussed the status quo of the internet plus health care service pattern during COVID-19. The main data sources included China's policy documents and national plans on health informatization, commercial and public welfare sources and websites, public reports, institutional reports, and academic papers. In particular, we extracted data from the 2019 National Health Informatization Survey released by the National Health Commission in China. We found that China developed its health information infrastructure and IT applications, made significant financial and intellectual informatization investments, and improved health resource allocations. Tested during COVID-19, China's current health informatization system, especially the internet plus health care system, has played a crucial role in monitoring and controlling the pandemic and allocating medical resources. However, an uneven distribution of health resources and insufficient financial and intellectual investment continue to challenge China's health informatization. China's rapid development of health informatization played a crucial role during COVID-19, providing a reference point for global pandemic prevention and control. To further promote health informatization, China's health informatization needs to strengthen top-level design, increase investment and training, upgrade the health infrastructure and IT applications, and improve internet plus health care services.

Implementation of distributed automated medication dispensing units in a new hospital: Nursing and pharmacy experience

AIMS AND OBJECTIVES

To explore the structures, processes and outcomes involved in an Automated Medication Dispensing system implementation and its impact on patient safety.

BACKGROUND

Increasing digitalisation of medication prescribing, dispensing, administration and stock management has occurred over the past two decades. While automated medication dispensing units aim to provide safe, high-quality, patient-centred care, the implementation may result in unintended consequences leading to suboptimal outcomes.

DESIGN

This study uses a qualitative approach guided by Donabedian's structure, process and outcome framework.

METHODS

Twenty-six registered nurses and pharmacy assistant staff, from clinical areas equipped with automated medication dispensing cabinets, participated in semi-structured interviews. In-depth, thematic analysis explored the structures and processes. Together with interview data, content analysis of text data generated by internal risk management and critical incident reporting systems was undertaken to evaluate outcomes. Findings were considered in light of the Interactive Sociotechnical Analysis approach to health information technology. The COREQ checklist was used in preparation of this article.

RESULTS

Pharmacy assistants reported better satisfaction with the system at implementation than nurses. Training provided for nurses and their involvement in system implementation was reported as insufficient; however, nurses' use of and satisfaction with the system improved over time. A recursive relationship between the changes imposed by the system and nurses' creative problem solving (workarounds) used to manage these changes, impacted work productivity for nurses and safety for patients.

CONCLUSIONS

The individualised nature of "workarounds" employed offered both risks and opportunities which require further identification, investigation and management.

RELEVANCE TO CLINICAL PRACTICE

Nurses are the majority of the health workforce. Digitalisation of traditionally paper-based activities in health care, impacting nursing work, requires similar strategies to any practice change.