User-Centered Design and Implementation of an Interoperable FHIR Application for Pediatric Pneumonia Prognostication in a Randomized Trial

OBJECTIVE

To support a pragmatic, electronic health record (EHR)-based randomized controlled trial, we applied user-centered design (UCD) principles, evidence-based risk communication strategies, and interoperable software architecture to design, test, and deploy a prognostic tool for children in emergency departments (EDs) with pneumonia.

METHODS

Risk for severe in-hospital outcomes was estimated using a validated ordinal logistic regression model to classify pneumonia severity. To render the results usable for ED clinicians, we created an integrated SMART on FHIR web application built for interoperable use in two pediatric EDs using different EHR vendors: Epic and Cerner. We followed a UCD framework, including problem analysis and user research, conceptual design and early prototyping, user interface development, formative evaluation, and post-deployment summative evaluation.

RESULTS

Problem analysis and user research from 39 clinicians and nurses revealed user preferences for risk aversion, accessibility, and timing of risk communication. Early prototyping and iterative design incorporated evidence-based design principles, including numeracy, risk framing, and best-practice visualization techniques. After rigorous unit and end-to-end testing, the application was successfully deployed in both EDs, which facilitatd enrollment, randomization, model visualization, data capture, and reporting for trial purposes.

CONCLUSIONS

The successful implementation of a custom application for pneumonia prognosis and clinical trial support in two health systems on different EHRs demonstrates the importance of UCD, adherence to modern clinical data standards, and rigorous testing. Key lessons included the need for understanding users' real-world needs, regular knowledge management, application maintenance, and the recognition that FHIR applications require careful configuration for interoperability.

Antibiotic clinical decision support for pneumonia in the ED: A randomized trial

BACKGROUND

Electronic health record-based clinical decision support (CDS) is a promising antibiotic stewardship strategy. Few studies have evaluated the effectiveness of antibiotic CDS in the pediatric emergency department (ED).

OBJECTIVE

To compare the effectiveness of antibiotic CDS vs. usual care for promoting guideline-concordant antibiotic prescribing for pneumonia in the pediatric ED.

DESIGN

Pragmatic randomized clinical trial.

SETTING AND PARTICIPANTS

Encounters for children (6 months-18 years) with pneumonia presenting to two tertiary care children s hospital EDs in the United States.

INTERVENTION

CDS or usual care was randomly assigned during 4-week periods within each site. The CDS intervention provided antibiotic recommendations tailored to each encounter and in accordance with national guidelines.

MAIN OUTCOME AND MEASURES

The primary outcome was exclusive guideline-concordant antibiotic prescribing within the first 24 h of care. Safety outcomes included time to first antibiotic order, encounter length of stay, delayed intensive care, and 3- and 7-day revisits.

RESULTS

1027 encounters were included, encompassing 478 randomized to usual care and 549 to CDS. Exclusive guideline-concordant prescribing did not differ at 24 h (CDS, 51.7% vs. usual care, 53.3%; odds ratio [OR] 0.94 [95% confidence interval [CI]: 0.73, 1.20]). In pre-specified stratified analyses, CDS was associated with guideline-concordant prescribing among encounters discharged from the ED (74.9% vs. 66.0%; OR 1.53 [95% CI: 1.01, 2.33]), but not among hospitalized encounters. Mean time to first antibiotic was shorter in the CDS group (3.0 vs 3.4 h; p = .024). There were no differences in safety outcomes.

CONCLUSIONS

Effectiveness of ED-based antibiotic CDS was greatest among those discharged from the ED. Longitudinal interventions designed to target both ED and inpatient clinicians and to address common implementation challenges may enhance the effectiveness of CDS as a stewardship tool.

Decision-Making During High-Risk Events: A Systematic Literature Review

Effective decision-making in crisis events is challenging due to time pressure, uncertainty, and dynamic decisional environments. We conducted a systematic literature review in PubMed and PsycINFO, identifying 32 empiric research papers that examine how trained professionals make naturalistic decisions under pressure. We used structured qualitative analysis methods to extract key themes. The studies explored different aspects of decision-making across multiple domains. The majority (19) focused on healthcare; military, fire and rescue, oil installation, and aviation domains were also represented. We found appreciable variability in research focus, methodology, and decision-making descriptions. We identified five main themes: (1) decision-making strategy, (2) time pressure, (3) stress, (4) uncertainty, and (5) errors. Recognition-primed decision-making (RPD) strategies were reported in all studies that analyzed this aspect. Analytical strategies were also prominent, appearing more frequently in contexts with less time pressure and explicit training to generate multiple explanations. Practitioner experience, time pressure, stress, and uncertainty were major influencing factors. Professionals must adapt to the time available, types of uncertainty, and individual skills when making decisions in high-risk situations. Improved understanding of these decisional factors can inform evidence-based enhancements to training, technology, and process design.

A pilot study to determine the incidence, type, and severity of non-routine events in neonates undergoing gastrostomy tube placement

BACKGROUND

Non-routine events (NRE) are defined as any suboptimal occurrences in a process being measured in the opinion of the reporter and comes from the field of human factors engineering. These typically occur well up-stream of an adverse event and NRE measurement has not been applied to the complex context of neonatal surgery. We sought to apply this novel safety event measurement methodology to neonates in the NICU undergoing gastrostomy tube placement.

METHODS

A prospective pilot study was conducted between November 2016 and August 2020 in the Level IV NICU and the pediatric operating rooms of an urban academic children's hospital to determine the incidence, severity, impact, and contributory factors of clinician-reported non-routine events (NREs, i.e., deviations from optimal care) and 30-day NSQIP occurrences in neonates receiving a G-tube.

RESULTS

Clinicians reported at least one NRE in 32 of 36 (89%) G-tube cases, averaging 3.0 (Standard deviation: 2.5) NRE reports per case. NSQIP-P review identified 7 cases (19%) with NSQIP-P occurrences and each of these cases had multiple reported NREs. One case in which NREs were not reported was without NSQIP-P occurrences. The odds ratio of having a NSQIP-P occurrence with the presence of an NRE was 0.695 (95% CI 0.06-17.04).

CONCLUSION

Despite being considered a "simple" operation, >80% of neonatal G-tube placement operations had at least one reported NRE by an operative team member. In this pilot study, NRE occurrence was not significantly associated with the subsequent reporting of an NSQIP-P occurrence. Understanding contributory factors of NREs that occur in neonatal surgery may promote surgical safety efforts and should be evaluated in larger and more diverse populations.

LEVEL OF EVIDENCE

IV.

Perioperative Care Structures and Non-Routine Events: Network Analysis

Non-routine events (NREs) are any aspect of care perceived by clinicians as a deviation from optimal care. The reporting of NREs to peers (or care teams) may help healthcare organizations improve patient safety in high-risk work environments (e.g., surgery). While various factors, including care structure and organizational factors may influence a clinician’s NRE reporting behavior, their role has not been systematically studied. We conducted a retrospective study relying on NREs and electronic health records to determine if perioperative interaction structures among clinicians are associated with the frequency of NRE reporting in a large academic medical center. The data covers November 1, 2016, to January 31, 2019 and includes 295 perioperative clinicians, 225 neonatal surgical cases, and 543 NREs. Using network analysis, we measured a clinician’s status in interaction structures according to the sociometric factors of degree, betweenness, and eigenvector centrality. We applied a proportional odds model to measure the relationship between each sociometric factor and NRE reporting frequency. Our findings indicate that the centrality of clinicians is directly associated with the quantity of NREs per surgical case.

Preventing clinical deterioration in cancer outpatients: Human centered design of a predictive model and response system

e13567

Background: Patients (pts) with cancer in the outpatient setting are at a high-risk for adverse events, such as unplanned hospitalizations and ER visits. A recent study found that up to 30% of hospital admissions were preventable. Identifying pts at risk of avoidable clinical deterioration remains a challenge, as clinicians may not be aware of pts’ experiences at home. The growing use of health IT presents an opportunity to identify and respond to clinical deterioration in patients before an adverse event occurs. In this study, we describe a human-centered design (HCD) process used to develop a clinical deterioration risk prediction system to improve the detection of and response to deterioration in cancer outpatients. Methods: Predictive model: We enrolled eligible cancer pts and collected data from each one including: FitBit, geolocation, EHR, and weekly patient-reported outcome measures (PROMs). Pts and their family caregivers could also report non-routine events (NREs), defined as any deviation from expected optimal care. We also captured unplanned treatment events (UTEs), a clinically meaningful change in the pt’s treatment course or care pathway. We developed a predictive model that generates a pt’s 7-day risk of clinical deterioration. Response system: We are developing a risk communication system (RCS) to communicate predicted risk scores to clinical teams. Using a HCD process, we first conducted 36 observations across 100 patient encounters to understand the environment of use. Next, we conducted 18 clinician interviews to define user needs. We have conducted 7 multi-disciplinary design sessions to iteratively develop prototypes of the RCS. We are currently conducting formative usability testing to assess the prototype and gather clinician feedback. Results: Predictive model: We have enrolled 36 cancer outpatients (24 head & neck, 9 gastrointestinal, and 3 lung). Pts completed a total of 219 weekly PROM surveys, reported 107 NREs and experienced 18 UTEs (e.g., infection). So far, models using EHR and PROM data are the most sensitive and precise (AUC: 0.983; 0.999). More patient data are required to develop higher quality stable models. Response system: We identified key design elements to include in the RCS, such as the caregiver’s phone number and the pt’s weight over time. Preliminary findings demonstrate high usability of the prototype RCS. Oncologists identified opportunities for the system to better support team communication and coordination, and to improve the identification and response to clinical deterioration in cancer outpatients. Conclusions: We have developed and tested a clinical deterioration risk prediction system for cancer outpatients. Future studies will implement the response system and evaluate its impact on clinical care.

Defining the Epidemiology of Safety Risks in Neonatal Intensive Care Unit Patients Requiring Surgery

OBJECTIVE

The aim of the study was to determine the incidence, type, severity, preventability, and contributing factors of nonroutine events (NREs)-events perceived by care providers or skilled observers as a deviations from optimal care based on the clinical situation-in the perioperative (i.e., preoperative, operative, and postoperative) care of surgical neonates in the neonatal intensive care unit and operating room.

METHODS

A prospective observational study of noncardiac surgical neonates, who received preoperative and postoperative neonatal intensive care unit care, was conducted at an urban academic children's hospital between November 1, 2016, and March 31, 2018. One hundred twenty-nine surgical cases in 109 neonates were observed. The incidence and description of NREs were collected via structured researcher-administered survey tool of involved clinicians. Primary measurements included clinicians' ratings of NRE severity and contributory factors and trained research assistants' ratings of preventability.

RESULTS

One or more NREs were reported in 101 (78%) of 129 observed cases for 247 total NREs. Clinicians reported 2 (2) (median, interquartile range) NREs per NRE case with a maximum severity of 3 (1) (possible range = 1-5). Trained research assistants rated 47% of NREs as preventable and 11% as severe and preventable. The relative risks for National Surgical Quality Improvement Program - pediatric major morbidity and 30-day mortality were 1.17 (95% confidence interval = 0.92-1.48) and 1.04 (95% confidence interval = 1.00-1.08) in NRE cases versus non-NRE cases.

CONCLUSIONS

The incidence of NREs in neonatal perioperative care at an academic children's hospital was high and of variable severity with a myriad of contributory factors.

Measuring Collaboration Through Concurrent Electronic Health Record Usage: Network Analysis Study

BACKGROUND

Collaboration is vital within health care institutions, and it allows for the effective use of collective health care worker (HCW) expertise. Human-computer interactions involving electronic health records (EHRs) have become pervasive and act as an avenue for quantifying these collaborations using statistical and network analysis methods.

OBJECTIVE

We aimed to measure HCW collaboration and its characteristics by analyzing concurrent EHR usage.

METHODS

By extracting concurrent EHR usage events from audit log data, we defined concurrent sessions. For each HCW, we established a metric called concurrent intensity, which was the proportion of EHR activities in concurrent sessions over all EHR activities. Statistical models were used to test the differences in the concurrent intensity between HCWs. For each patient visit, starting from admission to discharge, we measured concurrent EHR usage across all HCWs, which we called temporal patterns. Again, we applied statistical models to test the differences in temporal patterns of the admission, discharge, and intermediate days of hospital stay between weekdays and weekends. Network analysis was leveraged to measure collaborative relationships among HCWs. We surveyed experts to determine if they could distinguish collaborative relationships between high and low likelihood categories derived from concurrent EHR usage. Clustering was used to aggregate concurrent activities to describe concurrent sessions. We gathered 4 months of EHR audit log data from a large academic medical center's neonatal intensive care unit (NICU) to validate the effectiveness of our framework.

RESULTS

There was a significant difference (P<.001) in the concurrent intensity (proportion of concurrent activities: ranging from mean 0.07, 95% CI 0.06-0.08, to mean 0.36, 95% CI 0.18-0.54; proportion of time spent on concurrent activities: ranging from mean 0.32, 95% CI 0.20-0.44, to mean 0.76, 95% CI 0.51-1.00) between the top 13 HCW specialties who had the largest amount of time spent in EHRs. Temporal patterns between weekday and weekend periods were significantly different on admission (number of concurrent intervals per hour: 11.60 vs 0.54; P<.001) and discharge days (4.72 vs 1.54; P<.001), but not during intermediate days of hospital stay. Neonatal nurses, fellows, frontline providers, neonatologists, consultants, respiratory therapists, and ancillary and support staff had collaborative relationships. NICU professionals could distinguish high likelihood collaborative relationships from low ones at significant rates (3.54, 95% CI 3.31-4.37 vs 2.64, 95% CI 2.46-3.29; P<.001). We identified 50 clusters of concurrent activities. Over 87% of concurrent sessions could be described by a single cluster, with the remaining 13% of sessions comprising multiple clusters.

CONCLUSIONS

Leveraging concurrent EHR usage workflow through audit logs to analyze HCW collaboration may improve our understanding of collaborative patient care. HCW collaboration using EHRs could potentially influence the quality of patient care, discharge timeliness, and clinician workload, stress, or burnout.

Using Fitbit data to predict clinical deterioration and unplanned treatment events in cancer outpatients

e13560

Background: A common cause of preventable harm is the failure to detect and appropriately respond to clinical deterioration. Timely intervention is needed, particularly in cancer patients, to mitigate the effects of adverse events, disease progression, and medical error. This problem requires effective clinical surveillance, early recognition, timely notification of the appropriate clinician, and effective intervention. Methods: Applying a user-centered systems engineering design approach, we designed and implemented a surveillance-and-response system to improve the detection and response to clinical deterioration in cancer outpatients. The surveillance system predicts 7-day risk of UTEs, defined as clinically meaningful changes in the patient’s treatment course or cancer care pathway (e.g., any unplanned/unexpected: clinic or ER visit, hospital admission, or major treatment change and/or delays, and/or death). Data inputs consist of: 1) patient activity and health data collected by a Fitbit monitor; 2) geolocation data to measure activity outside the home (i.e., locations preselected at study onset); 3) clinical data from the hospital’s electronic health record; and 4) patient-reported outcomes measures (i.e., PROMs; the NCCN Distress Thermometer, the Comprehensive OpeN-Ended Survey or CONES, Global Health Score, items from the Consumer Assessment of Healthcare Providers and Systems (CAHPS)). Herein, we measured the effectiveness of Fitbit data alone to UTEs in a pilot sample of patients. Dimension reduction of Fitbit variables was first carried out by using Pearson correlation analysis to eliminate redundant variables. As UTEs are rare events, they were oversampled using the Synthetic Minority Oversampling Technique (SMOTE) to balance the dataset. A random forest classification model was trained to predict 7-day UTE risk. Model accuracy was determined by calculating the mean of Stratified 5-Fold Cross-Validation with 10 repeats. Results: Fitbit data was collected over a 6-8-week period from 14 head and neck cancer patients receiving surgical resection, outpatient chemotherapy, and/or radiotherapy. We identified six UTEs in 5 patients. A random forest classification model was developed from 10 variables derived from 7 Fitbit measures. The following variables were averaged or summed daily: average heart rate (HR), resting HR, below 50% or zone 1 of maximum HR, zone 2 and zone 3 HR combined (i.e., 70-100% of max HR), total daily calories, steps, and sleep in minutes. We achieved a model accuracy of 94% (ROC AUC: 0.984, Precision-Recall AUC: 0.985). Conclusions: Activity and health data collected by a commercial activity monitor demonstrated effectiveness in predicting patient UTEs when an oversampling procedure was used to adjust for class imbalance (i.e., low UTE rate). Future studies are recommended to verify and validate this result in a larger patient sample.

Reliability of simulation-based assessment for practicing physicians: performance is context-specific

INTRODUCTION

Even physicians who routinely work in complex, dynamic practices may be unprepared to optimally manage challenging critical events. High-fidelity simulation can realistically mimic critical clinically relevant events, however the reliability and validity of simulation-based assessment scores for practicing physicians has not been established.

METHODS

Standardised complex simulation scenarios were developed and administered to board-certified, practicing anesthesiologists who volunteered to participate in an assessment study during formative maintenance of certification activities. A subset of the study population agreed to participate as the primary responder in a second scenario for this study. The physicians were assessed independently by trained raters on both teamwork/behavioural and technical performance measures. Analysis using Generalisability and Decision studies were completed for the two scenarios with two raters.

RESULTS

The behavioural score was not more reliable than the technical score. With two raters > 20 scenarios would be required to achieve a reliability estimate of 0.7. Increasing the number of raters for a given scenario would have little effect on reliability.

CONCLUSIONS

The performance of practicing physicians on simulated critical events may be highly context-specific. Realistic simulation-based assessment for practicing physicians is resource-intensive and may be best-suited for individualized formative feedback. More importantly, aggregate data from a population of participants may have an even higher impact if used to identify skill or knowledge gaps to be addressed by training programs and inform continuing education improvements across the profession.

Shared Decisions: A Qualitative Study on Clinician and Patient Perspectives on Statin Therapy and Statin-Associated Side Effects

Background Despite guideline recommendations and clinical trial data suggesting benefit, statin therapy use in patients with atherosclerotic cardiovascular disease remains suboptimal. The aim of this study was to understand clinician and patient views on statin therapy, statin-associated side effects (SASEs), SASE management, and communication around statin risks and benefits. Methods and Results We conducted qualitative interviews of patients with atherosclerotic cardiovascular disease who had SASEs (n=17) and clinicians who regularly prescribe statins (n=20). We used directed content analysis, facilitated by Atlas.ti software, to develop and revise codebooks for clinician and patient interviews. The most relevant codes were "pile sorted" into 5 main topic domains: (1) SASEs vary in severity, duration, and time of onset; (2) communication practices by clinicians around statins and SASEs are variable and impacted by clinician time limitations and patient preconceived notions of SASEs; (3) although a "trial and error" approach to managing SASEs may be effective in allowing clinicians to keep patients with atherosclerotic cardiovascular disease on a statin, it can be frustrating for patients; (4) outside sources, such as the media, internet, social networks, and social circles, influence patients' perceptions and often impact the risk benefit discussion; and (5) a decision aid would be beneficial in facilitating clinician decision-making around SASEs and discussion of SASEs with the patients. Conclusions Statin use among patients with atherosclerotic cardiovascular disease remains suboptimal because of various patient- and clinician-related factors. The development of a decision aid to facilitate discussion of SASEs, clinician decision-making, and SASE management may improve statin use in this high-risk population.

Artificial neural network discovery of a switchable metasurface reflector

Optical materials engineered to dynamically and selectively manipulate electromagnetic waves are essential to the future of modern optical systems. In this paper, we simulate various metasurface configurations consisting of periodic 1D bars or 2D pillars made of the ternary phase change material Ge₂Sb₂Te₅ (GST). Dynamic switching behavior in reflectance is exploited due to a drastic refractive index change between the crystalline and amorphous states of GST. Selectivity in the reflection and transmission spectra is manipulated by tailoring the geometrical parameters of the metasurface. Due to the immense number of possible metasurface configurations, we train deep neural networks capable of exploring all possible designs within the working parameter space. The data requirements, predictive accuracy, and robustness of these neural networks are benchmarked against a ground truth by varying quality and quantity of training data. After ensuring trustworthy neural network advisory, we identify and validate optimal GST metasurface configurations best suited as dynamic switchable mirrors depending on selected light and manufacturing constraints.

Towards a Maturity Model for Clinical Decision Support Operations

Clinical decision support (CDS) systems delivered through the electronic health record are an important element of quality and safety initiatives within a health care system. However, managing a large CDS knowledge base can be an overwhelming task for informatics teams. Additionally, it can be difficult for these informatics teams to communicate their goals with external operational stakeholders and define concrete steps for improvement. We aimed to develop a maturity model that describes a roadmap toward organizational functions and processes that help health care systems use CDS more effectively to drive better outcomes. We developed a maturity model for CDS operations through discussions with health care leaders at 80 organizations, iterative model development by four clinical informaticists, and subsequent review with 19 health care organizations. We ceased iterations when feedback from three organizations did not result in any changes to the model. The proposed CDS maturity model includes three main "pillars": "Content Creation," "Analytics and Reporting," and "Governance and Management." Each pillar contains five levels-advancing along each pillar provides CDS teams a deeper understanding of the processes CDS systems are intended to improve. A "roof" represents the CDS functions that become attainable after advancing along each of the pillars. Organizations are not required to advance in order and can develop in one pillar separately from another. However, we hypothesize that optimal deployment of preceding levels and advancing in tandem along the pillars increase the value of organizational investment in higher levels of CDS maturity. In addition to describing the maturity model and its development, we also provide three case studies of health care organizations using the model for self-assessment and determine next steps in CDS development.

CYP2D6 Genotype-guided Metoprolol Therapy in Cardiac Surgery Patients: Rationale and Design of the Pharmacogenetic-guided Metoprolol Management for Postoperative Atrial Fibrillation in Cardiac Surgery (PREEMPTIVE) Pilot Study

OBJECTIVES

The Preemptive Pharmacogenetic-guided Metoprolol Management for Atrial Fibrillation in Cardiac Surgery (PREEMPTIVE) pilot trial aims to use existing institutional resources to develop a process for integrating CYP2D6 pharmacogenetic test results into the patient electronic health record, to develop an evidence-based clinical decision support tool to facilitate CYP2D6 genotype-guided metoprolol administration in the cardiac surgery setting, and to determine the impact of implementing this CYP2D6 genotype-guided integrated approach on the incidence of postoperative atrial fibrillation (AF), provider, and cost outcomes.

DESIGN

One-arm Bayesian adaptive design clinical trial.

SETTING

Single center, university hospital.

PARTICIPANTS

The authors will screen (including CYP2D6 genotype) up to 600 (264 ± 144 expected under the adaptive design) cardiac surgery patients, and enroll up to 200 (88 ± 48 expected) poor, intermediate, and ultrarapid CYP2D6 metabolizers over a period of 2 years at a tertiary academic center.

INTERVENTIONS

All consented and enrolled patients will receive the intervention of CYP2D6 genotype-guided metoprolol management based on CYP2D6 phenotype classified as a poor, intermediate, extensive (normal), or ultrarapid metabolizer.

MEASUREMENTS AND MAIN RESULTS

The primary outcome will be the incidence of postoperative AF. Secondary outcomes relating to rates of CYP2D6 genotype-guided prescription changes, costs, lengths of stay, and implementation metrics also will be investigated.

CONCLUSIONS

The PREEMPTIVE pilot study is the first perioperative pilot trial to provide essential information for the design of a future, large-scale trial comparing CYP2D6 genotype-guided metoprolol management with a nontailored strategy in terms of managing AF. In addition, secondary outcomes regarding implementation, clinical benefit, safety, and cost-effectiveness in patients undergoing cardiac surgery will be examined.

The use of structured data elements to identify ASCVD patients with statin-associated side effects: Insights from the Department of Veterans Affairs

BACKGROUND

Accurate identification of patients with statin-associated side effects (SASEs) is critical for health care systems to institute strategies to improve guideline-concordant statin use.

OBJECTIVE

The objective of this study was to determine whether adverse drug reaction (ADR) entry by clinicians in the electronic medical record can accurately identify SASEs.

METHODS

We identified 1,248,214 atherosclerotic cardiovascular disease (ASCVD) patients seeking care in the Department of Veterans Affairs. Using an ADR data repository, we identified SASEs in 15 major symptom categories. Sensitivity, specificity, positive predictive value (PPV), and negative predictive value (NPV) were assessed using a chart review of 256 ASCVD patients with identified SASEs, who were not on high-intensity statin therapy.

RESULTS

We identified 171,189 patients (13.71%) with documented SASEs over a 15-year period (9.9%, 2.7%, and 1.1% to 1, 2, or >2 statins, respectively). Statin use, high-intensity statin use, low-density lipoprotein cholesterol, and non-high-density lipoprotein cholesterol levels were 72%, 28.1%, 99 mg/dL, and 129 mg/dL among those with vs 81%, 31.1%, 84 mg/dL, and 111 mg/dL among those without SASEs. Progressively lower statin and high-intensity statin use, and higher low-density lipoprotein cholesterol and non-high-density lipoprotein cholesterol levels were noted among those with SASEs to 1, 2, or >2 statins. Two-thirds of SASEs were related to muscle symptoms. Sensitivity, specificity, PPV, NPV compared with manual chart review were 63.4%, 100%, 100%, and 85.3%, respectively.

CONCLUSION

A strategy of using ADR entry in the electronic medical record is feasible to identify SASEs with modest sensitivity and NPV but high specificity and PPV. Health care systems can use this strategy to identify ASCVD patients with SASEs and operationalize efforts to improve guideline-concordant lipid-lowering therapy use in such patients. The sensitivity of this approach can be further enhanced by the use of unstructured text data.

Descriptive Usability Study of CirrODS: Clinical Decision and Workflow Support Tool for Management of Patients With Cirrhosis

Background

There are gaps in delivering evidence-based care for patients with chronic liver disease and cirrhosis.

Objective

Our objective was to use interactive user-centered design methods to develop the Cirrhosis Order Set and Clinical Decision Support (CirrODS) tool in order to improve clinical decision-making and workflow.

Methods

Two work groups were convened with clinicians, user experience designers, human factors and health services researchers, and information technologists to create user interface designs. CirrODS prototypes underwent several rounds of formative design. Physicians (n=20) at three hospitals were provided with clinical scenarios of patients with cirrhosis, and the admission orders made with and without the CirrODS tool were compared. The physicians rated their experience using CirrODS and provided comments, which we coded into categories and themes. We assessed the safety, usability, and quality of CirrODS using qualitative and quantitative methods.

Results

We created an interactive CirrODS prototype that displays an alert when existing electronic data indicate a patient is at risk for cirrhosis. The tool consists of two primary frames, presenting relevant patient data and allowing recommended evidence-based tests and treatments to be ordered and categorized. Physicians viewed the tool positively and suggested that it would be most useful at the time of admission. When using the tool, the clinicians placed fewer orders than they placed when not using the tool, but more of the orders placed were considered to be high priority when the tool was used than when it was not used. The physicians’ ratings of CirrODS indicated above average usability.

Conclusions

We developed a novel Web-based combined clinical decision-making and workflow support tool to alert and assist clinicians caring for patients with cirrhosis. Further studies are underway to assess the impact on quality of care for patients with cirrhosis in actual practice.

Assessing Medication Safety Technology in the Intensive Care Unit

Understanding the strengths and weaknesses of a technology in the context of the distributed system in which it is working is critical to assessing and improving the performance of that system. Taking a systems approach requires knowledge about how all agents in a system work together to achieve the goals of that system. With these aims, the alerting mechanism of infusion pumps containing Dose-Error Reduction Software (DERS) was studied to determine its effectiveness in the Intensive Care Units (ICU's) of three hospitals. In 1,146 of the 9,557 pump alerts (12.0%), the alert caused the clinician to change the input. Of these, 1,030 were changed to within the hospital's recommended dosing limits. The alert was overridden for 8,400 (88.0%) of the alerts. The data show that this technology successfully informed clinicians over 1000 times that unintended doses had been inputted and stopped those doses from reaching the patient, thereby averting potential Medication Events. The data also suggest that, because nearly 90% of the alerts were overridden, a well-intended and valuable alert may be perceived by the clinicians as a false alarm and may be overlooked. Another key finding from this analysis was that clinicians may have used potentially unsafe workarounds to administer intravenous drug boluses (i.e., more rapid infusion of a defined dose or volume) and to keep the patient's line active between infusions. In a separate parallel study, clinician self-report of potentially harmful medication events was studied. During 559 hours of direct observation, clinicians detected 27 (IV and non-IV) medication events. All of the reported events were outside of the scope of what DERS technology was designed to detect. In addition, during the same time period the technology detected five potentially harmful IV medication events that the clinicians did not report. The results of these two studies indicate two possible classes of solutions that could reduce the impact and likelihood of medication administration errors. One class of solutions involves the procedures and policies of the hospital, ensuring that process and technology implementations are optimally tuned, taking human performance and the current practice of the clinicians into account. The other class of solutions involves using new strategies and technologies to ensure that each system agent has access to other agents' perspectives, and the broader system's perspective. Studies such as these can provide insight into the use of safety technology during critical care processes and provide direction for future research, including more effective design of alerting mechanisms of ICU devices.

Abstract 226: Patient and Clinician Reported Non-Routine Events During Periprocedural Cardiac Catheterization

Awareness in the realm of patient safety and quality improvement has become increasingly important in healthcare. Understanding the factors that influence these tenets can help improve the overall care for patients and potentially improve patient outcomes. One tool to assist in identifying those factors is the study of non-routine events (NREs). A non-routine event is defined as any event that deviates from optimal or expected care for a specific patient in a specific clinical situation. The goals of this study are to ascertain what aspects of their clinical encounters do patients and families view as “non-routine” and reflective of lower care safety or quality, delineate factors that influence the reporting of NREs and affect the nature of the reported NREs, and determine whether NREs obtained from patients/families significantly add to evidence about clinical system failure modes beyond that obtained from clinicians caring for the same patients. We concurrently captured and compared NRE’s reported by patients/families and their clinicians in patients having a cardiac catheterization. To date, the use of NREs to advance the quality of healthcare has primarily been identified in the surgical literature, making this project unique in regards to the use of NREs in the catheterization lab. At defined times in each care episode, trained researchers used a structured survey to identify and elucidate NREs from the patient, family members, and care providers. After we obtained informed consent, we obtained participant and system factors including individual (age, education, literacy, health status, satisfaction), contextual (self-reported stress, frustration, and performance level), and system (staffing and unit workload) factors. NREs will be characterized by incidence, source, type, and severity by two independent raters. Qualitative and multivariate statistical analyses will be performed. Of the 130 cases studied, we collected 189 patient reported NREs and 107 clinician-reported NRE’s. Ninety cases (69%) contained patient reported NREs while sixty cases (46%) contained clinician reported NREs, and forty-five (35%) cases contained both clinician and patient reported NREs. This site had 129 total patients and 130 total cases (one patient was interviewed on two separate encounters). Given ongoing data analysis, only preliminary findings can be concluded at this time: 1) patient and family members reported a substantial number of NREs with many related to the patients’ health, 2) virtually none of the patient reported NREs were reported or known by clinicians yet a substantial number were deemed relevant by patients to the quality and safety of their care, and 3) the reported NREs provided useful information about care delivery systems and how to address their shortcomings. Further studies are needed to elucidate the effect of studying patient NREs on outcomes such as morbidity and mortality.

The feasibility of text reminders to improve medication adherence in adolescents with asthma

OBJECTIVE

Personal health applications have the potential to help patients with chronic disease by improving medication adherence, self-efficacy, and quality of life. The goal of this study was to assess the impact of MyMediHealth (MMH) - a website and a short messaging service (SMS)-based reminder system - on medication adherence and perceived self-efficacy in adolescents with asthma.

METHODS

We conducted a block-randomized controlled study in academic pediatric outpatient settings. There were 98 adolescents enrolled. Subjects who were randomized to use MMH were asked to create a medication schedule and receive SMS reminders at designated medication administration times for 3 weeks. Control subjects received action lists as a part of their usual care. Primary outcome measures included MMH usage patterns and self-reports of system usability, medication adherence, asthma control, self-efficacy, and quality of life.

RESULTS

Eighty-nine subjects completed the study, of whom 46 were randomized to the intervention arm. Compared to controls, we found improvements in self-reported medication adherence (P = .011), quality of life (P = .037), and self-efficacy (P = .016). Subjects reported high satisfaction with MMH; however, the level of system usage varied widely, with lower use among African American patients.

CONCLUSIONS

MMH was associated with improved medication adherence, perceived quality of life, and self-efficacy.Trial Registration This project was registered under http://clinicaltrials.gov/ identifier NCT01730235.

Impact of electronic health record technology on the work and workflow of physicians in the intensive care unit

OBJECTIVE

To assess the impact of EHR technology on the work and workflow of ICU physicians and compare time spent by ICU resident and attending physicians on various tasks before and after EHR implementation.

DESIGN

EHR technology with electronic order management (CPOE, medication administration and pharmacy system) and physician documentation was implemented in October 2007.

MEASUREMENT

We collected a total of 289 h of observation pre- and post-EHR implementation. We directly observed the work of residents in three ICUs (adult medical/surgical ICU, pediatric ICU and neonatal ICU) and attending physicians in one ICU (adult medical/surgical ICU).

RESULTS

EHR implementation had an impact on the time distribution of tasks as well as the temporal patterns of tasks. After EHR implementation, both residents and attending physicians spent more of their time on clinical review and documentation (40% and 55% increases, respectively). EHR implementation also affected the frequency of switching between tasks, which increased for residents (from 117 to 154 tasks per hour) but decreased for attendings (from 138 to 106 tasks per hour), and the temporal flow of tasks, in particular around what tasks occurred before and after clinical review and documentation. No changes in the time spent in conversational tasks or the physical care of the patient were observed.

CONCLUSIONS

The use of EHR technology has a major impact on ICU physician work (e.g., increased time spent on clinical review and documentation) and workflow (e.g., clinical review and documentation becoming the focal point of many other tasks). Further studies should evaluate the impact of changes in physician work on the quality of care provided.

Technology and the Ecology of Chronic Illness in Everyday Life

A major challenge in the design of useful technological tools is effectively conceptualizing the context in which users engage the technology. Contextually specific research on activities of patients and their caregivers - and how those activities are supported by social and material arrangements--can result in insights for design of consumer health informatics technologies and infrastructural advancements that can better support patients outside of institutional settings. This chapter describes an ecosystem focused on activity--how activity is shaped by cultural institutions, and the negotiations that arise between actors and institutions.

Barriers to Effective Preoperative Handover Communication in the Neonatal Intensive Care Unit

We sought to better understand the preoperative neonatal intensive care unit to operating room (NICU-to-OR) handover process and to elicit barriers to effective handovers. We first conducted observations of NICU-to-OR handovers to ascertain current handover practices, including the participants involved, handover content, as well as barriers and facilitators of effective handovers. We then developed a survey tool to assess the generalizability of our findings to other NICUs across the country. The resulting pilot data highlight key areas for future research and potential interventions to improve the quality of NICU-to-OR care transitions.

The work of adult and pediatric intensive care unit nurses

BACKGROUND

Researchers have used various methods to describe and quantify the work of nurses. Many of these studies were focused on nursing in general care settings; therefore, less is known about the unique work nurses perform in intensive care units (ICUs).

OBJECTIVES

The aim of this study was to observe adult and pediatric ICU nurses in order to quantify and compare the duration and frequency of nursing tasks across four ICUs as well as within two discrete workflows: nurse handoffs at shift change and patient interdisciplinary rounds.

METHODS

A behavioral task analysis of adult and pediatric nurses was used to allow unobtrusive, real-time observation. A total of 147 hours of observation were conducted in an adult medical-surgical, a cardiac, a pediatric, and a neonatal ICU at one rural, tertiary care community teaching hospital.

RESULTS

Over 75% of ICU nurses' time was spent on patient care activities. Approximately 50% of this time was spent on direct patient care, over 20% on care coordination, 28% on nonpatient care, and approximately 2% on indirect patient care activities. Variations were observed between units; for example, nurses in the two adult units spent more time using monitors and devices. A high rate and variety of tasks were also observed: Nurses performed about 125 activities per hour, averaging a switch between tasks every 29 seconds.

DISCUSSION

This study provides useful information about how nurses spend their time in various ICUs. The methodology can be used in future research to examine changes in work related to, for example, implementation of health information technology.

Differences in day and night shift clinical performance in anesthesiology

OBJECTIVE

This study examined whether anesthesia residents (physicians in training) performed clinical duties in the operating room differently during the day versus at night.

BACKGROUND

Fatigue from sleep deprivation and working through the night is common for physicians, particularly during residency training.

METHODS

Using a repeated-measures design, we studied 13 pairs of day-night matched anesthesia cases. Dependent measures included task times, workload ratings, response to an alarm light latency task, and mood.

RESULTS

Residents spent significantly less time on manual tasks and more time on monitoring tasks during the maintenance phase at night than during the day. Residents reported more negative mood at night than during the day, both pre- and postoperation. However, time of day had no effect on the mood change between pre- and postoperation. Workload ratings and the response time to an alarm light latency task were not significantly different between night and day cases.

CONCLUSIONS

Because night shift residents had been awake and working for more than 16 hr, the observed differences in task performance and mood may be attributed to fatigue. The changes in task distribution during night shift work may represent compensatory strategies to maintain patient care quality while keeping perceived workload at a manageable level.

APPLICATIONS

Fatigue effects during night shifts should be considered when designing work-rest schedules for clinicians. This matched-case control scheme can also be applied to study other phenomena associated with patient safety in the actual clinical environment.

Shifting Toward Balance: Measuring the Distribution of Workload Among Emergency Physician Teams

STUDY OBJECTIVE

The objective of this investigation is to determine time-dependent workload patterns for emergency department (ED) physician teams across work shifts. A secondary aim was to demonstrate how ED demand patterns and the timing of shift changes influence the balance of workload among a physician team.

METHODS

Operational measurements of an adult ED were collected from a clinical information system to characterize physician workload patterns during all current work shifts. Plots of patient load versus time were developed for each physician shift, in which patient load was defined as the number of patients a physician simultaneously managed at a point in time. Patient-load curves for each shift were superimposed during 24 hours to display how patient load was distributed among a team of physicians.

RESULTS

Resident shift changes during daily peak occupancy periods caused patient load imbalances so that residents on a particular shift consistently managed a disproportionate number of patients (mean 9.4 patients; 95% confidence interval [CI] 6.7 to 12.1 patients) compared with other residents on duty (mean 3.4 patients; 95% CI 2.1 to 4.7 patients).

CONCLUSION

Physician patient load patterns and ED demand patterns should be taken into consideration when physician shift times are scheduled so that patient load may be balanced among a team. Real-time monitoring of physician patient load may reduce stress and prevent physicians from exceeding their safe capacity for workload.

Video capture of clinical care to enhance patient safety

Experience from other domains suggests that videotaping and analyzing actual clinical care can provide valuable insights for enhancing patient safety through improvements in the process of care. Methods are described for the videotaping and analysis of clinical care using a high quality portable multi-angle digital video system that enables simultaneous capture of vital signs and time code synchronization of all data streams. An observer can conduct clinician performance assessment (such as workload measurements or behavioral task analysis) either in real time (during videotaping) or while viewing previously recorded videotapes. Supplemental data are synchronized with the video record and stored electronically in a hierarchical database. The video records are transferred to DVD, resulting in a small, cheap, and accessible archive. A number of technical and logistical issues are discussed, including consent of patients and clinicians, maintaining subject privacy and confidentiality, and data security. Using anesthesiology as a test environment, over 270 clinical cases (872 hours) have been successfully videotaped and processed using the system.

Changes in intensive care unit nurse task activity after installation of a third-generation intensive care unit information system

OBJECTIVE To determine the percentage of time that intensive care unit (ICU) nurses spend on documentation and other nursing activities before and after installation of a third-generation ICU information system.

DESIGN

Prospective data collection using real-time time-motion analysis, before and after installation of the ICU information system.

SETTING

A ten-bed surgical ICU at a Veterans Affairs medical center.

SUBJECTS

ICU nurses.

INTERVENTIONS

Installation of a third-generation ICU information system.

MEASUREMENTS AND MAIN RESULTS

Ten ICU nurses were studied before and after installation of the ICU information system. Each ICU nurse's activities and tasks, during 4-hr observation periods, were categorized in real-time by a nurse observer and recorded in a laptop computer. Each recorded task was automatically time-stamped and logged into a data file. The percentage of time spent on documentation decreased from 35.1 +/- 8.3% to 24.2 +/- 7.6% (p =.025) after the ICU information system was installed. The percentage of time providing direct patient care increased from 31.3 +/- 9.2% to 40.1 +/- 11.7% (p =.085). The percentage of time doing patient assessment, a direct patient care task, increased from 4.0 +/- 4.7% to 9.4 +/- 4.4% (p =.001).

CONCLUSIONS

Installation of a third-generation ICU information system decreased the percentage of time ICU nurses spent on documentation by >30%. Almost half of the time saved on documentation was spent on patient assessment, a direct patient care task.

Retrospective data collection and analytical techniques for patient safety studies

To enhance patient safety, data about actual clinical events must be collected and scrutinized. This paper has two purposes. First, it provides an overview of some of the methods available to collect and analyze retrospective data about medical errors, near misses, and other relevant patient safety events. Second, it introduces a methodological approach that focuses on non-routine events (NRE), defined as all events that deviate from optimal clinical care. In intermittent in-person surveys of anesthesia providers, 75 of 277 (27%) recently completed anesthetic cases contained a non-routine event (98 total NRE). Forty-six of the cases (17%) had patient impact while only 20 (7%) led to patient injury. In contrast, in the same hospitals over a two-year period, we collected event data on 135 cases identified with traditional quality improvement processes (event incidence of 0.7-2.7%). In these quality improvement cases, 120 (89%) had patient impact and 74 (55%) led to patient injury. Preliminary analyses not only illustrate some of the analytical methods applicable to safety data but also provide insight into the potential value of the non-routine event approach for the early detection of risks to patient safety before serious patient harm occurs.

Assessment of the intrarater and interrater reliability of an established clinical task analysis methodology

BACKGROUND

Task analysis may be useful for assessing how anesthesiologists alter their behavior in response to different clinical situations. In this study, the authors examined the intraobserver and interobserver reliability of an established task analysis methodology.

METHODS

During 20 routine anesthetic procedures, a trained observer sat in the operating room and categorized in real-time the anesthetist's activities into 38 task categories. Two weeks later, the same observer performed task analysis from videotapes obtained intraoperatively. A different observer performed task analysis from the videotapes on two separate occasions. Data were analyzed for percent of time spent on each task category, average task duration, and number of task occurrences. Rater reliability and agreement were assessed using intraclass correlation coefficients.

RESULTS

Intrarater reliability was generally good for categorization of percent time on task and task occurrence (mean intraclass correlation coefficients of 0.84-0.97). There was a comparably high concordance between real-time and video analyses. Interrater reliability was generally good for percent time and task occurrence measurements. However, the interrater reliability of the task duration metric was unsatisfactory, primarily because of the technique used to capture multitasking.

CONCLUSIONS

A task analysis technique used in anesthesia research for several decades showed good intrarater reliability. Off-line analysis of videotapes is a viable alternative to real-time data collection. Acceptable interrater reliability requires the use of strict task definitions, sophisticated software, and rigorous observer training. New techniques must be developed to more accurately capture multitasking. Substantial effort is required to conduct task analyses that will have sufficient reliability for purposes of research or clinical evaluation.

Human factors research in anesthesia patient safety

Patient safety has become a major public concern. Human factors research in other high-risk fields has demonstrated how rigorous study of factors that affect job performance can lead to improved outcome and reduced errors after evidence-based redesign of tasks or systems. These techniques have increasingly been applied to the anesthesia work environment. This paper describes data obtained recently using task analysis and workload assessment during actual patient care and the use of cognitive task analysis to study clinical decision making. A novel concept of "non-routine events" is introduced and pilot data are presented. The results support the assertion that human factors research can make important contributions to patient safety. Information technologies play a key role in these efforts.