Towards an evaluation framework for medical device-integrated electronic medical record

INTRODUCTION

Medical device (MD)-integrated (I) electronic medical record (EMR) (MDI-EMR) poses cyber threats that undermine patient safety, and thus, they require effective control mechanisms. We reviewed the related literature, including existing EMR and MD risk assessment approaches, to identify MDI-EMR comprehensive evaluation dimensions and measures.

AREAS COVERED

We searched multiple databases, including PubMed, Web of Knowledge, Scopus, ACM, Embase, IEEE and Ingenta. We explored various evaluation aspects of MD and EMR to gain a better understanding of their complex integration. We reviewed numerous risk management and assessment frameworks related to MD and EMR security aspects and mitigation controls and then identified their common evaluation aspects. Our review indicated that previous evaluation frameworks assessed MD and EMR independently. To address this gap, we proposed an evaluation framework based on the sociotechnical dimensions of health information systems and risk assessment approaches for MDs to evaluate MDI-EMR integratively.

EXPERT OPINION

The emergence of MDI-EMR cyber threats requires appropriate evaluation tools to ensure the safe development and application of MDI-EMR. Consequently, our proposed framework will continue to evolve through subsequent validations and refinements. This process aims to establish its applicability in informing stakeholders of the safety level and assessing its effectiveness in mitigating risks for future improvements.

What kind of a problem is loneliness? Representations of connectedness and participation from a study of telepresence technologies in the UK

Loneliness is represented in UK policy as a public health problem with consequences in terms of individual suffering, population burden and service use. However, loneliness is historically and culturally produced; manifestations of loneliness and social isolation also require social and cultural analysis. We explored meanings of loneliness and social isolation in the UK 2020-2022 and considered what the solutions of telepresence technologies reveal about the problems they are used to address. Through qualitative methods we traced the introduction and use of two telepresence technologies and representations of these, and other technologies, in policy and UK media. Our dataset comprises interviews, fieldnotes, policy documents, grey literature and newspaper articles. We found loneliness was represented as a problem of individual human connection and of collective participation in social life, with technology understood as having the potential to enhance and inhibit connections and participation. Technologically-mediated connections were frequently perceived as inferior to in-person contact, particularly in light of the enforced social isolation of the COVID-19 pandemic. We argue that addressing loneliness requires attending to other, related, health and social problems and introducing technological solutions requires integration into the complex social and organisational dynamics that shape technology adoption. We conclude that loneliness is primarily understood as a painful lack of co-presence, no longer regarded as simply a subjective experience, but as a social and policy problem demanding resolution.

"Are we in this together?": embedding social identity detection in drones improves emergency coordination

Autonomous systems, such as drones, are critical for emergency mitigation, management, and recovery. They provide situational awareness and deliver communication services which effectively guide emergency responders' decision making. This combination of technology and people comprises a socio-technical system. Yet, focusing on the use of drone technology as a solely operational tool, underplays its potential to enhance coordination between the different agents involved in mass emergencies, both human and non-human. This paper proposes a new methodological approach that capitalizes on social identity principles to enable this coordination in an evacuation operation. In the proposed approach, an adaptive drone uses sensor data to infer the group membership of the survivors it encounters during the operation. A corpus of 200 interactions of survivors' talk during real-life emergencies was computationally classified as being indicative of a shared identity or personal/no identity. This classification model, then, informed a game-theoretic model of human-robot interactions. Bayesian Nash Equilibrium analysis determined the predicted behavior for the human agent and the strategy that the drone needs to adopt to help with survivor evacuation. Using linguistic and synthetic data, we show that the identity-adaptive architecture outperformed two non-adaptive architectures in the number of successful evacuations. The identity-adaptive drone can infer which victims are likely to be helped by survivors and where help from emergency teams is needed. This facilitates effective coordination and adaptive performance. This study shows decision-making can be an emergent capacity that arises from the interactions of both human and non-human agents in a socio-technical system.

Sustainable Development for Mobile Health Apps Using the Human-Centered Design Process

Well-documented scientific evidence indicates that mobile health (mHealth) apps can improve the quality of life, relieve symptoms, and restore health for patients. In addition to improving patients' health outcomes, mHealth apps reduce health care use and the cost burdens associated with disease management. Currently, patients and health care providers have a wide variety of choices among commercially available mHealth apps. However, due to the high resource costs and low user adoption of mHealth apps, the cost-benefit relationship remains controversial. When compared to traditional expert-driven approaches, applying human-centered design (HCD) may result in more useable, acceptable, and effective mHealth apps. In this paper, we summarize current HCD practices in mHealth development studies and make recommendations to improve the sustainability of mHealth. These recommendations include consideration of factors regarding culture norms, iterative evaluations on HCD practice, use of novelty in mHealth app, and consideration of privacy and reliability across the entire HCD process. Additionally, we suggest a sociotechnical lens toward HCD practices to promote the sustainability of mHealth apps. Future research should consider standardizing the HCD practice to help mHealth researchers and developers avoid barriers associated with inadequate HCD practices.

Socio-Organizational Dimensions: The Key to Advancing the Shared Care Record Agenda in Health and Social Care

Integrating health and social care delivery with the help of digital technologies is a grand challenge. We argue that previous attempts have largely failed to achieve their objectives because implementers and decision makers disregard the complex socio-organizational dimensions of change associated with initiatives. These include structural and organizational complexity inhibiting the development of shared care pathways; professional jurisdictions, interests, and expertise; and existing data and governance structures. We provide an overview of those dimensions that can inform strategic decisions going forward, thereby contributing to the chances of success of shared care initiatives.

DECENT: A sociotechnical approach for developing mobile health apps in underserved settings

Objective

Despite the fact that user engagement is critical to the efficacy of mobile health (mHealth) interventions in the Global South, many of these interventions lack user engagement features. This is because sociotechnical aspects of such initiatives are frequently ignored during the design, development, and implementation stages. This research highlighted the importance of considering sociotechnical factors when developing mHealth apps. The intended users for the mHealth technologies in this study are care professionals.

Materials and Methods

Five semi-structured interviews and a pilot interview were conducted to identify user engagement facilitators and barriers. The interview data were analysed using NVivo. The Capability, Opportunity, Motivation - Behaviour (COM-B) model is then used to map the facilitators and barriers to mHealth app engagement, allowing researchers to better understand how users engage/disengage with mHealth apps.

Results and Discussion

Capability facilitators included features that assist users in learning more about the app (e.g. a user manual and statistical data) as well as features that assist users in developing a routine. The lack of app skills and cognitive overload limit capability. While social connectedness and offline functionality were identified as facilitators of user engagement, non-user-friendly design and cultural dimensions were identified as barriers. Early user engagement and rewards were identified as motivational facilitators that influence user engagement. Furthermore, perceived non-utility and a lack of encouragement were identified as motivational barriers to engagement.

Conclusion

Several factors were discovered across all COM-B model components that could be used to develop more engaging mHealth apps. Adopting a techno-centric approach that ignores sociotechnical factors can reduce user engagement. The design process engagement enhancement system (DECENT) framework was proposed based on the findings.

Designing and implementing smart glass technology for emergency medical services: a sociotechnical perspective

Objective

This study aims to investigate key considerations and critical factors that influence the implementation and adoption of smart glasses in fast-paced medical settings such as emergency medical services (EMS).

Materials and Methods

We employed a sociotechnical theoretical framework and conducted a set of participatory design workshops with 15 EMS providers to elicit their opinions and concerns about using smart glasses in real practice.

Results

Smart glasses were recognized as a useful tool to improve EMS workflow given their hands-free nature and capability of processing and capturing various patient data. Out of the 8 dimensions of the sociotechnical model, we found that hardware and software, human-computer interface, workflow, and external rules and regulations were cited as the major factors that could influence the adoption of this novel technology. EMS participants highlighted several key requirements for the successful implementation of smart glasses in the EMS context, such as durable devices, easy-to-use and minimal interface design, seamless integration with existing systems and workflow, and secure data management.

Discussion

Applications of the sociotechnical model allowed us to identify a range of factors, including not only technical aspects, but also social, organizational, and human factors, that impact the implementation and uptake of smart glasses in EMS. Our work informs design implications for smart glass applications to fulfill EMS providers' needs.

Conclusion

The successful implementation of smart glasses in EMS and other dynamic healthcare settings needs careful consideration of sociotechnical issues and close collaboration between different stakeholders.

Best Papers in Human Factors and Sociotechnical Development

OBJECTIVES

To select the best papers that made original and high impact contributions in human factors and organizational issues in biomedical informatics in 2021.

METHODS

A rigorous extraction process based on queries from Web of Science® and PubMed/Medline was conducted to identify the scientific contributions published in 2021 that address human factors and organizational issues in biomedical informatics. The screening of papers on titles and abstracts independently by the two section editors led to a total of 3,206 papers. These papers were discussed for a selection of 12 finalist papers, which were then reviewed by the two section editors, two chief editors, and by three external reviewers from internationally renowned research teams.

RESULTS

The query process resulted in 12 papers that reveal interesting and rigorous methods and important studies in human factors that move the field forward, particularly in clinical informatics and emerging technologies such as brain-computer interfaces and mobile health. This year three papers were clearly outstanding and help advance in the field. They provide examples of examining novel and important topics such as the nature of human-machine interaction behavior and norms, use of social-media based design for an electronic health record, and emerging topics such as brain-computer interfaces. thematic development of electronic health records and usability techniques, and condition-focused patient facing tools. Those concerning the Corona Virus Disease 2019 (COVID-19) were included as part of that section.

CONCLUSION

The selected papers make important contributions to human factors and organizational issues, expanding and deepening our knowledge of how to apply theory and applications of new technologies in health.

Sociotechnical Intervention for Improved Delivery of Preventive Cardiovascular Care to Rural Communities: Participatory Design Approach

Background

Clinical practice guidelines recommend antiplatelet and statin therapies as well as blood pressure control and tobacco cessation for secondary prevention in patients with established atherosclerotic cardiovascular diseases (ASCVDs). However, these strategies for risk modification are underused, especially in rural communities. Moreover, resources to support the delivery of preventive care to rural patients are fewer than those for their urban counterparts. Transformative interventions for the delivery of tailored preventive cardiovascular care to rural patients are needed.

Objective

A multidisciplinary team developed a rural-specific, team-based model of care intervention assisted by clinical decision support (CDS) technology using participatory design in a sociotechnical conceptual framework. The model of care intervention included redesigned workflows and a novel CDS technology for the coordination and delivery of guideline recommendations by primary care teams in a rural clinic.

Methods

The design of the model of care intervention comprised 3 phases: problem identification, experimentation, and testing. Input from team members (n=35) required 150 hours, including observations of clinical encounters, provider workshops, and interviews with patients and health care professionals. The intervention was prototyped, iteratively refined, and tested with user feedback. In a 3-month pilot trial, 369 patients with ASCVDs were randomized into the control or intervention arm.

Results

New workflows and a novel CDS tool were created to identify patients with ASCVDs who had gaps in preventive care and assign the right care team member for delivery of tailored recommendations. During the pilot, the intervention prototype was iteratively refined and tested. The pilot demonstrated feasibility for successful implementation of the sociotechnical intervention as the proportion of patients who had encounters with advanced practice providers (nurse practitioners and physician assistants), pharmacists, or tobacco cessation coaches for the delivery of guideline recommendations in the intervention arm was greater than that in the control arm.

Conclusions

Participatory design and a sociotechnical conceptual framework enabled the development of a rural-specific, team-based model of care intervention assisted by CDS technology for the transformation of preventive health care delivery for ASCVDs.

The biopsychosociotechnical model: a systems-based framework for human-centered health improvement

The biopsychosocial model is among the most influential frameworks for human-centered health improvement but has faced significant criticism- both conceptual and pragmatic. This paper extends and fundamentally re-structures the biopsychosocial model by combining it with sociotechnical systems theory. The resulting biopsychosociotechnical model addresses key critiques of the biopsychosocial model, providing a more "practical theory" for human-centered health improvement. It depicts the determinants of health as complex adaptive system of systems; includes the the artificial world (technology); and provides a roadmap for systems improvement by: differentiating between "health status" and "health and needs assessment", [promoting problem framing]; explaining health as an emergent property of the biopsychosociotechnical context [imposing a systems orientation]; focusing on "interventions" vs. "treatments" to modify the biopsychosociotechnical determinants of health, [expanding the solution space]; calling for a participatory design process [supporting systems awareness and goal-orientation]; and including intervention management to support the full lifecycle of health improvement.

Infrastructuring an organizational node for a federated research and data network: A case study from a sociotechnical perspective

Background:

Local nodes on federated research and data networks (FR&DNs) provide enabling infrastructure for collaborative clinical and translational research. Studies in other fields note that infrastructuring, that is, work to identify and negotiate relationships among people, technologies, and organizations, is invisible, unplanned, and undervalued. This may explain the limited literature on nodes in FR&DNs in health care.

Methods:

A retrospective case study of one PCORnet® node explored 3 questions: (1) how were components of infrastructure assembled; (2) what specific work was required; and (3) what theoretically grounded, pragmatic questions should be considered when infrastructuring a node for sustainability. Artifacts, work efforts, and interviews generated during node development and implementation were reviewed. A sociotechnical lens was applied to the analysis. Validity was established with internal and external partners.

Results:

Resources, services, and expertise needed to establish the node existed within the organization, but were scattered across work units. Aligning, mediating, and institutionalizing for sustainability among network and organizational teams, governance, and priorities consumed more work efforts than deploying technical aspects of the node. A theoretically based set of questions relevant to infrastructuring a node was developed and organized within a framework of infrastructuring emphasizing enacting technology, organizing work, and institutionalizing; validity was established with internal and external partners.

Conclusions:

FR&DNs are expanding; we provide a sociotechnical perspective on infrastructuring a node. Future research should evaluate the applicability of the framework and questions to other node and network configurations, and more broadly the infrastructuring required to enable and support federated clinical and translational science.

Teamwork in clinical genomics: A dynamic sociotechnical healthcare setting

RATIONALE, AIMS AND OBJECTIVES

Teamworking across sociotechnical boundaries in healthcare is growing as technological advances in medicine abound. With this progress, teams need to find new ways of working together in non-traditional settings. The novel field of clinical genomics provides the opportunity to rethink the existing approach to teamworking and how it needs to evolve. Our aim was to identify the key factors influencing teamworking in the emerging field of clinical genomics and how can they be applied in practice.

METHOD

We drew on three qualitative datasets from interviews undertaken in Australia, 2018/2019, that explored determinants of implementation of clinical genomics with laboratory scientists (n = 7), service and programme leads (n = 21), project officers (n = 2), clinical genetics staff (n = 26) and other medical specialists (n = 21). Data were analysed using a theory-informed matrix approach to identify themes related to teamworking.

RESULTS

We identify that teams in clinical genomics work in an elongated adaptive context where there is rapid evolution of the knowledge base, shifting expectations of staff roles, and fast changes of technology. Delivering care in this setting brings additional challenges to teamworking as members strive to stay abreast of current knowledge and technology. We identify four themes: (a) the role of the team in keeping knowledge up-to-date; (b) professional identity; (c) team adaptability, and (d) practical/organisational considerations.

CONCLUSION

Challenges to teamworking that arise in the elongated adaptive context do not always fit traditional ways of working, and innovative strategies will need to be adopted to ensure the diagnostic advances of clinical genomics are realised. Provision of time and permission for team members to share knowledge and evolve, promoting capacity building, nurturing trustful relationships and establishing boundaries are amongst the practice recommendations for organisational and team leaders, even though these activities may disrupt existing ways of working or hierarchical structures.

Clinician Preimplementation Perspectives of a Decision-Support Tool for the Prediction of Cardiac Arrhythmia Based on Machine Learning: Near-Live Feasibility and Qualitative Study

Background

Artificial intelligence (AI), such as machine learning (ML), shows great promise for improving clinical decision-making in cardiac diseases by outperforming statistical-based models. However, few AI-based tools have been implemented in cardiology clinics because of the sociotechnical challenges during transitioning from algorithm development to real-world implementation.

Objective

This study explored how an ML-based tool for predicting ventricular tachycardia and ventricular fibrillation (VT/VF) could support clinical decision-making in the remote monitoring of patients with an implantable cardioverter defibrillator (ICD).

Methods

Seven experienced electrophysiologists participated in a near-live feasibility and qualitative study, which included walkthroughs of 5 blinded retrospective patient cases, use of the prediction tool, and questionnaires and interview questions. All sessions were video recorded, and sessions evaluating the prediction tool were transcribed verbatim. Data were analyzed through an inductive qualitative approach based on grounded theory.

Results

The prediction tool was found to have potential for supporting decision-making in ICD remote monitoring by providing reassurance, increasing confidence, acting as a second opinion, reducing information search time, and enabling delegation of decisions to nurses and technicians. However, the prediction tool did not lead to changes in clinical action and was found less useful in cases where the quality of data was poor or when VT/VF predictions were found to be irrelevant for evaluating the patient.

Conclusions

When transitioning from AI development to testing its feasibility for clinical implementation, we need to consider the following: expectations must be aligned with the intended use of AI; trust in the prediction tool is likely to emerge from real-world use; and AI accuracy is relational and dependent on available information and local workflows. Addressing the sociotechnical gap between the development and implementation of clinical decision-support tools based on ML in cardiac care is essential for succeeding with adoption. It is suggested to include clinical end-users, clinical contexts, and workflows throughout the overall iterative approach to design, development, and implementation.

Stem Cell Treatments in India: An Ethnography of Regular Practice

In this article I explore how experimental stem cell treatments have become a therapeutic choice in India. Drawing on ethnographic fieldwork, I illustrate six sociotechnical pathways by which clinically unproven treatment is rendered regular health care practice for consumers and providers across the country. Through each pathway, based on the themes of bioconsumption, integration, consumer contacts, treatment experience and rituals, and political culture, I demonstrate how the experimental status of stem cell treatment is undermined and thereby experienced by key stakeholders as just another medical option. In analyzing the pathways, I argue that unproven stem cell treatment practices are promoted and sustained by multiple social, political and technical forces. The discussion on pathways is therefore situated within a conceptual framework of "normalization," derived from studies that investigate the multidimensional aspects to controversial and/or new medical technologies becoming routine.

Developing and Applying a Formative Evaluation Framework for Health Information Technology Implementations: Qualitative Investigation

Background

There is currently a lack of comprehensive, intuitive, and usable formative evaluation frameworks for health information technology (HIT) implementations. We therefore sought to develop and apply such a framework. This study describes the Technology, People, Organizations, and Macroenvironmental factors (TPOM) framework we developed.

Objective

The aim was to develop and apply a formative evaluation framework for HIT implementations, highlighting interrelationships between identified dimensions and offering guidance for implementers.

Methods

We drew on an initial prototype framework developed as part of a literature review exploring factors for the effective implementation of HIT. In addition, we used qualitative data from three national formative evaluations of different HIT interventions (electronic health record, electronic prescribing, and clinical decision support functionality). The combined data set comprised 19 case studies of primarily hospital settings, and included 703 semistructured interviews, 663 hours of observations, and 864 documents gathered from a range of care settings across National Health Service (NHS) England and NHS Scotland. Data analysis took place over a period of 10 years and was guided by a framework informed by the existing evidence base.

Results

TPOM dimensions are intimately related and each include a number of subthemes that evaluators need to consider. Although technological functionalities are crucial in getting an initiative off the ground, system design needs to be cognizant of the accompanying social and organizational transformations required to ensure that technologies deliver the desired value for a variety of stakeholders. Wider structural changes, characterized by shifting policy landscapes and markets, influence technologies and the ways they are used by organizations and staff.

Conclusions

The TPOM framework supports formative evaluations of HIT implementation and digitally enabled transformation efforts. There is now a need for prospective application of the TPOM framework to determine its value.

Data integration strategies for predictive analytics in precision medicine

With the rapid growth of health-related data including genomic, proteomic, imaging and clinical, the arduous task of data integration can be overwhelmed by the complexity of the environment including data size and diversity. This report examines the role of data integration strategies for big data predictive analytics in precision medicine research. Infrastructure-as-code methodologies will be discussed as a means of integrating and managing data. This includes a discussion on how and when these strategies can be used to lower barriers and address issues of consistency and interoperability within medical research environments. The goal is to support translational research and enable healthcare organizations to integrate and utilize infrastructure to accelerate the adoption of precision medicine.

Health Care Robotics: Qualitative Exploration of Key Challenges and Future Directions

Background

The emergence of robotics is transforming industries around the world. Robot technologies are evolving exponentially, particularly as they converge with other functionalities such as artificial intelligence to learn from their environment, from each other, and from humans.

Objective

The goal of the research was to understand the emerging role of robotics in health care and identify existing and likely future challenges to maximize the benefits associated with robotics and related convergent technologies.

Methods

We conducted qualitative semistructured one-to-one interviews exploring the role of robotic applications in health care contexts. Using purposive sampling, we identified a diverse range of stakeholders involved in conceiving, procuring, developing, and using robotics in a range of national and international health care settings. Interviews were digitally recorded, transcribed verbatim, and analyzed thematically, supported by NVivo 10 (QSR International) software. Theoretically, this work was informed by the sociotechnical perspective, where social and technical systems are understood as being interdependent.

Results

We conducted 21 interviews and these accounts suggested that there are significant opportunities for improving the safety, quality, and efficiency of health care through robotics, but our analysis identified 4 major barriers that need to be effectively negotiated to realize these: (1) no clear pull from professionals and patients, (2) appearance of robots and associated expectations and concerns, (3) disruption of the way work is organized and distributed, and (4) new ethical and legal challenges requiring flexible liability and ethical frameworks.

Conclusions

Sociotechnical challenges associated with the effective integration of robotic applications in health care settings are likely to be significant, particularly for patient-facing functions. These need to be identified and addressed for effective innovation and adoption.

Sociotechnical factors influencing unsafe use of hospital information systems: A qualitative study in Malaysian government hospitals

The objective of this study is to identify factors influencing unsafe use of hospital information systems in Malaysian government hospitals. Semi-structured interviews with 31 medical doctors in three Malaysian government hospitals implementing total hospital information systems were conducted between March and May 2015. A thematic qualitative analysis was performed on the resultant data to deduce the relevant themes. A total of five themes emerged as the factors influencing unsafe use of a hospital information system: (1) knowledge, (2) system quality, (3) task stressor, (4) organization resources, and (5) teamwork. These qualitative findings highlight that factors influencing unsafe use of a hospital information system originate from multidimensional sociotechnical aspects. Unsafe use of a hospital information system could possibly lead to the incidence of errors and thus raises safety risks to the patients. Hence, multiple interventions (e.g. technology systems and teamwork) are required in shaping high-quality hospital information system use.

Designing Novel Health ICTs to Support Work, Not Generate It: Five Principles

In this paper, we offer five principles to inform how health ICT designers and healthcare organizations address and mitigate issues relating to clinician documentation burden. We draw on our experience and empirical work designing an ICT intervention, ActionADE, to illustrate how our team developed and will use these principles to ease documentation burden for clinician-users.

Patient Perceptions About Data Sharing & Privacy: Insights from ActionADE

Information communication technologies (ICTs) may improve health delivery by enhancing informational continuity of care and enabling secondary use of health data including public health surveillance and research. ICTs also introduce concerns related to privacy. In this paper, we examine and address this tension in the context of the development and implementation of a novel platform that will enable the documentation and communication of patient-specific ADE information, titled ActionADE. We explored privacy concerns qualitatively from the perspective of patients. Our findings will inform a series of recommendations for system design that seek to balance the need to both share and protect personal health information.

Naturalistic Usability Testing of Inpatient Medication Reconciliation Software

Medication history errors are common at admission, but can be mitigated through the implementation of medication reconciliation (MR). We designed multi-media software to assist clinicians with collection of an admission history. This manuscript describes a naturalistic usability study conducted on the hospital wards. Our goals were to 1) estimate the impact of our workflow upon departmental productivity and 2) determine the ability of our software to detect discrepancies. We furnished clinical pharmacists with our application on a tablet PC and asked them to collect a bedside history. We used 1) time-motion analysis to estimate cycle-time and 2) chart reviews to estimate error detection rates. Our intervention detected an average of 7.7 discrepancies per admission (11.7 per pharmacy-shift). A panel rated 67% of these discrepancies as 'high' or 'very high' risk. The cycle-time per admission was slightly longer than usual care processes (20.5 min vs. 17.9 min), but included a bedside interview. In general, pharmacists agreed that the technology improved the completeness and accuracy of a medication history. However, workflow leveling strategies are important to implementing a durable process. In conclusion, a pharmacist-mediated, patient-centered technology holds promise for improving the quality of MR and overall clinical performance.

An analysis of electronic health record-related patient safety incidents

The aim of this study was to analyse electronic health record-related patient safety incidents in the patient safety incident reporting database in fully digital hospitals in Finland. We compare Finnish data to similar international data and discuss their content with regard to the literature. We analysed the types of electronic health record-related patient safety incidents that occurred at 23 hospitals during a 2-year period. A procedure of taxonomy mapping served to allow comparisons. This study represents a rare examination of patient safety risks in a fully digital environment. The proportion of electronic health record-related incidents was markedly higher in our study than in previous studies with similar data. Human-computer interaction problems were the most frequently reported. The results show the possibility of error arising from the complex interaction between clinicians and computers.

An analysis of electronic health record-related patient safety concerns

OBJECTIVE

A recent Institute of Medicine report called for attention to safety issues related to electronic health records (EHRs). We analyzed EHR-related safety concerns reported within a large, integrated healthcare system.

METHODS

The Informatics Patient Safety Office of the Veterans Health Administration (VA) maintains a non-punitive, voluntary reporting system to collect and investigate EHR-related safety concerns (ie, adverse events, potential events, and near misses). We analyzed completed investigations using an eight-dimension sociotechnical conceptual model that accounted for both technical and non-technical dimensions of safety. Using the framework analysis approach to qualitative data, we identified emergent and recurring safety concerns common to multiple reports.

RESULTS

We extracted 100 consecutive, unique, closed investigations between August 2009 and May 2013 from 344 reported incidents. Seventy-four involved unsafe technology and 25 involved unsafe use of technology. A majority (70%) involved two or more model dimensions. Most often, non-technical dimensions such as workflow, policies, and personnel interacted in a complex fashion with technical dimensions such as software/hardware, content, and user interface to produce safety concerns. Most (94%) safety concerns related to either unmet data-display needs in the EHR (ie, displayed information available to the end user failed to reduce uncertainty or led to increased potential for patient harm), software upgrades or modifications, data transmission between components of the EHR, or 'hidden dependencies' within the EHR.

DISCUSSION

EHR-related safety concerns involving both unsafe technology and unsafe use of technology persist long after 'go-live' and despite the sophisticated EHR infrastructure represented in our data source. Currently, few healthcare institutions have reporting and analysis capabilities similar to the VA.

CONCLUSIONS

Because EHR-related safety concerns have complex sociotechnical origins, institutions with long-standing as well as recent EHR implementations should build a robust infrastructure to monitor and learn from them.

"Reducing unnecessary testing in a CPOE system through implementation of a targeted CDS intervention"

BACKGROUND

We describe and evaluate the development and use of a Clinical Decision Support (CDS) intervention; an alert, in response to an identified medical error of overuse of a diagnostic laboratory test in a Computerized Physician Order Entry (CPOE) system. CPOE with embedded CDS has been shown to improve quality of care and reduce medical errors. CPOE can also improve resource utilization through more appropriate use of laboratory tests and diagnostic studies. Observational studies are necessary in order to understand how these technologies can be successfully employed by healthcare providers.

METHODS

The error was identified by the Test Utilization Committee (TUC) in September, 2008 when they noticed critical care patients were being tested daily, and sometimes twice daily, for B-Type Natriuretic Peptide (BNP). Repeat and/or serial BNP testing is inappropriate for guiding the management of heart failure and may be clinically misleading. The CDS intervention consists of an expert rule that searches the system for a BNP lab value on the patient. If there is a value and the value is within the current hospital stay, an advisory is displayed to the ordering clinician. In order to isolate the impact of this intervention on unnecessary BNP testing we applied multiple regression analysis to the sample of 41,306 patient admissions with at least one BNP test at LVHN between January, 2008 and September, 2011.

RESULTS

Our regression results suggest the CDS intervention reduced BNP orders by 21% relative to the mean. The financial impact of the rule was also significant. Multiplying by the direct supply cost of $28.04 per test, the intervention saved approximately $92,000 per year.

CONCLUSIONS

The use of alerts has great positive potential to improve care, but should be used judiciously and in the appropriate environment. While these savings may not be generalizable to other interventions, the experience at LVHN suggests that appropriately designed and carefully implemented CDS interventions can have a substantial impact on the efficiency of care provision.

Building a house on shifting sand: methodological considerations when evaluating the implementation and adoption of national electronic health record systems

BACKGROUND

A commitment to Electronic Health Record (EHR) systems now constitutes a core part of many governments' healthcare reform strategies. The resulting politically-initiated large-scale or national EHR endeavors are challenging because of their ambitious agendas of change, the scale of resources needed to make them work, the (relatively) short timescales set, and the large number of stakeholders involved, all of whom pursue somewhat different interests. These initiatives need to be evaluated to establish if they improve care and represent value for money.

METHODS

Critical reflections on these complexities in the light of experience of undertaking the first national, longitudinal, and sociotechnical evaluation of the implementation and adoption of England's National Health Service's Care Records Service (NHS CRS).

RESULTS/DISCUSSION

We advance two key arguments. First, national programs for EHR implementations are likely to take place in the shifting sands of evolving sociopolitical and sociotechnical and contexts, which are likely to shape them in significant ways. This poses challenges to conventional evaluation approaches which draw on a model of baseline operations → intervention → changed operations (outcome). Second, evaluation of such programs must account for this changing context by adapting to it. This requires careful and creative choice of ontological, epistemological and methodological assumptions.

SUMMARY

New and significant challenges are faced in evaluating national EHR implementation endeavors. Based on experiences from this national evaluation of the implementation and adoption of the NHS CRS in England, we argue for an approach to these evaluations which moves away from seeing EHR systems as Information and Communication Technologies (ICT) projects requiring an essentially outcome-centred assessment towards a more interpretive approach that reflects the situated and evolving nature of EHR seen within multiple specific settings and reflecting a constantly changing milieu of policies, strategies and software, with constant interactions across such boundaries.

Towards an integrative cognitive-socio-technical approach in health informatics: analyzing technology-induced error involving health information systems to improve patient safety

The purpose of this paper is to argue for an integration of cognitive and socio-technical approaches to assessing the impact of health information systems. Historically, health informatics research has examined the cognitive and socio-technical aspects of health information systems separately. In this paper we argue that evaluations of health information systems should consider aspects related to cognition as well as socio-technical aspects including impact on workflow (i.e. an integrated view). Using examples from the study of technology-induced error in healthcare, we argue for the use of simulations to evaluate the cognitive-socio-technical impacts of health information technology [36]. Implications of clinical simulations and analysis of cognitive-social-technical impacts are discussed within the context of the system development life cycle to improve health information system design, implementation and evaluation.

Conceptual challenges for advancing the socio-technical underpinnings of health informatics

This discussion paper considers the adoption of socio-technical perspectives and their theoretical and practical influence within the discipline of health informatics. The paper highlights the paucity of discussion of the philosophy, theory and concepts of socio-technical perspectives within health informatics. Instead of a solid theoretical base from which to describe, study and understand human-information technology interactions we continue to have fragmented, unelaborated understandings. This has resulted in a continuing focus on technical system performance and increasingly managerial outputs to the detriment of social and technical systems analysis. It has also limited critical analyses and the adaptation of socio-technical approaches beyond the immediate environment to the broader social systems of contemporary society, an expansion which is increasingly mandated in today’s complex health environment.