Informing the design of clinical decision support services for evaluation of children with minor blunt head trauma in the emergency department: A sociotechnical analysis

Overcoming Vulnerabilities in Our Emergency Care System Through Pediatric Readiness

Although children account for 20% of all emergency department (ED) visits, the majority of children seek emergency care in hospitals that see fewer than 10 children per day. The National Pediatric Readiness Project has defined key system-level standards for all EDs to safely care for ill and injured children. High pediatric readiness is associated with improvement in mortality for critically ill and injured children. However, to improve readiness and sustain system-level changes, hospitals must invest in pediatric champions and empower them to engage in continuous quality improvement. Finally, incorporating pediatric readiness into policy is crucial for its long-term sustainability.

Revolutionizing decision support: a systematic literature review of contextual implementation models for electronic health records systems

PURPOSE

This study aims to assess previously developed Electronic Health Records System (EHRS) implementation models and identify successful models for decision support.

DESIGN/METHODOLOGY/APPROACH

A systematic review followed the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) guidelines. The data sources used were Scopus, PubMed and Google Scholar. The review identified peer-reviewed papers published in the English Language from January 2010 to April 2023, targeting well-defined implementation of EHRS with decision-support capabilities in healthcare. To comprehensively address the research question, we ensured that all potential sources of evidence were considered, and quantitative and qualitative studies reporting primary data and systematic review studies that directly addressed the research question were included in the review. By including these studies in our analysis, we aimed to provide a more thorough and reliable evaluation of the available evidence.

FINDINGS

The findings suggest that the success of EHRS implementation is determined by organizational and human factors rather than technical factors alone. Successful implementation is dependent on a suitable implementation framework and management of EHRS. The review identified the capabilities of Clinical Decision Support (CDS) tools as essential in the effectiveness of EHRS in supporting decision-making.

ORIGINALITY/VALUE

This study contributes to the existing literature on EHRS implementation models and identifies successful models for decision support. The findings can inform future implementations and guide decision-making in healthcare facilities.

Management of suicidal risk in the emergency department: A clinical pathway using the computerized adaptive screen for suicidal youth

Objective

Given the critical need for efficient and tailored suicide screening for youth presenting in the emergency department (ED), this study establishes validated screening score thresholds for the Computerized Adaptive Screen for Suicidal Youth (CASSY) and presents an example of a suicide risk classification pathway.

Methods

Participants were primarily from the Study One derivation cohort of the Emergency Department Screen for Teens at Risk for Suicide (ED-STARS) enrolled in collaboration with Pediatric Emergency Care Applied Research Networks (PECARN). CASSY scores corresponded to the predicted probabilities of a suicide attempt in the next 3 months and risk thresholds were classified as minimal (<1%), low (1%-5%), moderate (5%-10%), and high (>10%). CASSY scores were compared to risk thresholds derived from clinical consensus and ED complaints and dispositions. CASSY risk thresholds were also examined as predictors of future suicide attempts in the Study Two validation cohort of ED-STARS.

Results

A total of 1452 teens were enrolled with a median age of 15.2 years, 59.5% were female, 55.6% were White, 22% were Black, 22.3% were Latinx, and 42.8% received public assistance. The clinical consensus suicide risk groups were strongly associated with the CASSY-predicted risk thresholds. Suicide attempts in the Study Two cohort occurred at a frequency consistent with the CASSY-predicted thresholds.

Conclusions

The CASSY can be a valuable tool in providing patient-specific risk probabilities for a suicide attempt at 3 months and tailor the threshold cutoffs based on the availability of local mental health resources. We give an example of a clinical risk pathway, which should include segmentation of the ED population by medical versus psychiatric chief complaint.

Exploring the role of professional identity in the implementation of clinical decision support systems-a narrative review

BACKGROUND

Clinical decision support systems (CDSSs) have the potential to improve quality of care, patient safety, and efficiency because of their ability to perform medical tasks in a more data-driven, evidence-based, and semi-autonomous way. However, CDSSs may also affect the professional identity of health professionals. Some professionals might experience these systems as a threat to their professional identity, as CDSSs could partially substitute clinical competencies, autonomy, or control over the care process. Other professionals may experience an empowerment of the role in the medical system. The purpose of this study is to uncover the role of professional identity in CDSS implementation and to identify core human, technological, and organizational factors that may determine the effect of CDSSs on professional identity.

METHODS

We conducted a systematic literature review and included peer-reviewed empirical studies from two electronic databases (PubMed, Web of Science) that reported on key factors to CDSS implementation and were published between 2010 and 2023. Our explorative, inductive thematic analysis assessed the antecedents of professional identity-related mechanisms from the perspective of different health care professionals (i.e., physicians, residents, nurse practitioners, pharmacists).

RESULTS

One hundred thirty-one qualitative, quantitative, or mixed-method studies from over 60 journals were included in this review. The thematic analysis found three dimensions of professional identity-related mechanisms that influence CDSS implementation success: perceived threat or enhancement of professional control and autonomy, perceived threat or enhancement of professional skills and expertise, and perceived loss or gain of control over patient relationships. At the technological level, the most common issues were the system's ability to fit into existing clinical workflows and organizational structures, and its ability to meet user needs. At the organizational level, time pressure and tension, as well as internal communication and involvement of end users were most frequently reported. At the human level, individual attitudes and emotional responses, as well as familiarity with the system, most often influenced the CDSS implementation. Our results show that professional identity-related mechanisms are driven by these factors and influence CDSS implementation success. The perception of the change of professional identity is influenced by the user's professional status and expertise and is improved over the course of implementation.

CONCLUSION

This review highlights the need for health care managers to evaluate perceived professional identity threats to health care professionals across all implementation phases when introducing a CDSS and to consider their varying manifestations among different health care professionals. Moreover, it highlights the importance of innovation and change management approaches, such as involving health professionals in the design and implementation process to mitigate threat perceptions. We provide future areas of research for the evaluation of the professional identity construct within health care.

Physicians' Perceptions of Clinical Decision Support to Treat Patients With Heart Failure in the ED

Importance

Clinical decision support (CDS) could help emergency department (ED) physicians treat patients with heart failure (HF) by estimating risk, collating relevant history, and assisting with medication prescribing if physicians' perspectives inform its design and implementation.

Objective

To evaluate CDS usability and workflow integration in the hands of ED physician end users who use it in clinical practice.

Design, Setting, and Participants

This mixed-methods qualitative study administered semistructured interviews to ED physicians from 2 community EDs of Kaiser Permanente Northern California in 2023. The interview guide, based on the Usability Heuristics for User Interface Design and the Sociotechnical Environment models, yielded themes used to construct an electronic survey instrument sent to all ED physicians.

Main Outcomes and Measures

Main outcomes were physicians' perceptions of using CDS to complement clinical decision-making, usability, and integration into ED clinical workflow.

Results

Seven key informant physicians (5 [71.4%] female, median [IQR] 15.0 [9.5-15.0] years in practice) were interviewed and survey responses from 51 physicians (23 [45.1%] female, median [IQR] 14.0 [9.5-17.0] years in practice) were received from EDs piloting the CDS intervention. Response rate was 67.1% (51 of 76). Physicians suggested changes to CDS accessibility, functionality, and workflow integration. Most agreed that CDS would improve patient care and fewer than half of physicians expressed hesitation about their capacity to consistently comply with its recommendations, citing workload concerns. Physicians preferred a passive prompt that encouraged, but did not mandate, interaction with the CDS.

Conclusions and Relevance

In this qualitative study of physicians who were using a novel CDS intervention to assist with ED management of patients with acute HF, several opportunities were identified to improve usability as well as several key barriers and facilitators to CDS implementation.

Using Theories, Models, and Frameworks to Inform Implementation Cycles of Computerized Clinical Decision Support Systems in Tertiary Health Care Settings: Scoping Review

BACKGROUND

Computerized clinical decision support systems (CDSSs) are essential components of modern health system service delivery, particularly within acute care settings such as hospitals. Theories, models, and frameworks may assist in facilitating the implementation processes associated with CDSS innovation and its use within these care settings. These processes include context assessments to identify key determinants, implementation plans for adoption, promoting ongoing uptake, adherence, and long-term evaluation. However, there has been no prior review synthesizing the literature regarding the theories, models, and frameworks that have informed the implementation and adoption of CDSSs within hospitals.

OBJECTIVE

This scoping review aims to identify the theory, model, and framework approaches that have been used to facilitate the implementation and adoption of CDSSs in tertiary health care settings, including hospitals. The rationales reported for selecting these approaches, including the limitations and strengths, are described.

METHODS

A total of 5 electronic databases were searched (CINAHL via EBSCOhost, PubMed, Scopus, PsycINFO, and Embase) to identify studies that implemented or adopted a CDSS in a tertiary health care setting using an implementation theory, model, or framework. No date or language limits were applied. A narrative synthesis was conducted using full-text publications and abstracts. Implementation phases were classified according to the "Active Implementation Framework stages": exploration (feasibility and organizational readiness), installation (organizational preparation), initial implementation (initiating implementation, ie, training), full implementation (sustainment), and nontranslational effectiveness studies.

RESULTS

A total of 81 records (42 full text and 39 abstracts) were included. Full-text studies and abstracts are reported separately. For full-text studies, models (18/42, 43%), followed by determinants frameworks (14/42,33%), were most frequently used to guide adoption and evaluation strategies. Most studies (36/42, 86%) did not list the limitations associated with applying a specific theory, model, or framework.

CONCLUSIONS

Models and related quality improvement methods were most frequently used to inform CDSS adoption. Models were not typically combined with each other or with theory to inform full-cycle implementation strategies. The findings highlight a gap in the application of implementation methods including theories, models, and frameworks to facilitate full-cycle implementation strategies for hospital CDSSs.

Identifying barriers and facilitators to successful implementation of computerized clinical decision support systems in hospitals: a NASSS framework-informed scoping review

BACKGROUND

Successful implementation and utilization of Computerized Clinical Decision Support Systems (CDSS) in hospitals is complex and challenging. Implementation science, and in particular the Nonadoption, Abandonment, Scale-up, Spread and Sustainability (NASSS) framework, may offer a systematic approach for identifying and addressing these challenges. This review aimed to identify, categorize, and describe barriers and facilitators to CDSS implementation in hospital settings and map them to the NASSS framework. Exploring the applicability of the NASSS framework to CDSS implementation was a secondary aim.

METHODS

Electronic database searches were conducted (21 July 2020; updated 5 April 2022) in Ovid MEDLINE, Embase, Scopus, PyscInfo, and CINAHL. Original research studies reporting on measured or perceived barriers and/or facilitators to implementation and adoption of CDSS in hospital settings, or attitudes of healthcare professionals towards CDSS were included. Articles with a primary focus on CDSS development were excluded. No language or date restrictions were applied. We used qualitative content analysis to identify determinants and organize them into higher-order themes, which were then reflexively mapped to the NASSS framework.

RESULTS

Forty-four publications were included. These comprised a range of study designs, geographic locations, participants, technology types, CDSS functions, and clinical contexts of implementation. A total of 227 individual barriers and 130 individual facilitators were identified across the included studies. The most commonly reported influences on implementation were fit of CDSS with workflows (19 studies), the usefulness of the CDSS output in practice (17 studies), CDSS technical dependencies and design (16 studies), trust of users in the CDSS input data and evidence base (15 studies), and the contextual fit of the CDSS with the user's role or clinical setting (14 studies). Most determinants could be appropriately categorized into domains of the NASSS framework with barriers and facilitators in the "Technology," "Organization," and "Adopters" domains most frequently reported. No determinants were assigned to the "Embedding and Adaptation Over Time" domain.

CONCLUSIONS

This review identified the most common determinants which could be targeted for modification to either remove barriers or facilitate the adoption and use of CDSS within hospitals. Greater adoption of implementation theory should be encouraged to support CDSS implementation.

Clinical decision support to Optimize Care of patients with Atrial Fibrillation or flutter in the Emergency department: protocol of a stepped-wedge cluster randomized pragmatic trial (O'CAFÉ trial)

BACKGROUND

Management of adults with atrial fibrillation (AF) or atrial flutter in the emergency department (ED) includes rate reduction, cardioversion, and stroke prevention. Different approaches to these components of care may lead to variation in frequency of hospitalization and stroke prevention actions, with significant implications for patient experience, cost of care, and risk of complications. Standardization using evidence-based recommendations could reduce variation in management, preventable hospitalizations, and stroke risk.

METHODS

We describe the rationale for our ED-based AF treatment recommendations. We also describe the development of an electronic clinical decision support system (CDSS) to deliver these recommendations to emergency physicians at the point of care. We implemented the CDSS at three pilot sites to assess feasibility and solicit user feedback. We will evaluate the impact of the CDSS on hospitalization and stroke prevention actions using a stepped-wedge cluster randomized pragmatic clinical trial across 13 community EDs in Northern California.

DISCUSSION

We hypothesize that the CDSS intervention will reduce hospitalization of adults with isolated AF or atrial flutter presenting to the ED and increase anticoagulation prescription in eligible patients at the time of ED discharge and within 30 days. If our hypotheses are confirmed, the treatment protocol and CDSS could be recommended to other EDs to improve management of adults with AF or atrial flutter.

TRIAL REGISTRATION

ClinicalTrials.gov NCT05009225 .  Registered on 17 August 2021.

Barriers and Opportunities Regarding Implementation of a Machine Learning-Based Acute Heart Failure Risk Stratification Tool in the Emergency Department

Hospital admissions for patients with acute heart failure (AHF) remain high. There is an opportunity to improve alignment between patient risk and admission decision. We recently developed a machine learning (ML)-based model that stratifies emergency department (ED) patients with AHF based on predicted risk of a 30-day severe adverse event. Prior to deploying the algorithm and paired clinical decision support, we sought to understand barriers and opportunities regarding successful implementation. We conducted semi-structured interviews with eight front-line ED providers and surveyed 67 ED providers. Audio-recorded interviews were transcribed and analyzed using thematic analysis, and we had a 65% response rate to the survey. Providers wanted decision support to be streamlined into workflows with minimal disruptions. Most providers wanted assistance primarily with ED disposition decisions, and secondarily with medical management and post-discharge follow-up care. Receiving feedback on patient outcomes after risk tool use was seen as an opportunity to increase acceptance, and few providers (<10%) had significant hesitations with using an ML-based tool after education on its use. Engagement with key front-line users on optimal design of the algorithm and decision support may contribute to broader uptake, acceptance, and adoption of recommendations for clinical decisions.

Sustainability of a Clinical Decision Support Intervention for Outpatient Care for Emergency Department Patients With Acute Pulmonary Embolism

IMPORTANCE

Physicians commonly hospitalize patients presenting to the emergency department (ED) with acute pulmonary embolism (PE), despite eligibility for safe outpatient management. Risk stratification using electronic health record-embedded clinical decision support systems can aid physician site-of-care decision-making and increase safe outpatient management. The long-term sustainability of early improvements after the cessation of trial-based, champion-led promotion is uncertain.

OBJECTIVE

To evaluate the sustainability of recommended site-of-care decision-making support 4 years after initial physician champion-led interventions to increase outpatient management for patients with acute PE.

DESIGN, SETTING, AND PARTICIPANTS

This retrospective cohort study was conducted in 21 US community hospitals in an integrated health system. Participants included adult patients presenting to the ED with acute PE. Study sites had participated in an original decision-support intervention trial 4 years prior to the current study period: 10 sites were intervention sites, 11 sites were controls. In that trial, decision support with champion promotion resulted in significantly higher outpatient management at intervention sites compared with controls. After trial completion, all study sites were given continued access to a modified decision-support tool without further champion-led outreach. Data were analyzed from January 2019 to February 2020.

EXPOSURES

ED treatment with a modified clinical decision support tool.

MAIN OUTCOMES AND MEASURES

The main outcome was frequency of outpatient management, defined as discharge home directly from the ED, stratified by the PE Severity Index. The safety measure of outpatient care was 7-day PE-related hospitalization.

RESULTS

This study included 1039 patients, including 533 (51.3%) women, with a median (IQR) age of 65 (52-74) years. Nearly half (474 patients [45.6%]) were rated lower risk on the PE Severity Index. Overall, 278 patients (26.8%) were treated as outpatients, with only four 7-day PE-related hospitalizations (1.4%; 95% CI, 0.4%-3.6%). The practice gap in outpatient management created by the earlier trial persisted in the outpatient management for patients with lower risk: 109 of 236 patients (46.2%) at former intervention sites vs 81 of 238 patients (34.0%) at former control sites (difference, 12.2; [95% CI, 3.4-20.9] percentage points; P = .007), with wide interfacility variation (range, 7.1%-47.1%).

CONCLUSIONS AND RELEVANCE

In this cohort study, a champion-led, decision-support intervention to increase outpatient management for patients presenting to the ED with acute pulmonary embolism was associated with sustained higher rates of outpatient management 4 years later. The application of our findings to improving sustainability of practice change for other clinical conditions warrants further study.

Concussion Referral and Practice Patterns by Pediatric Emergency Medicine Providers

OBJECTIVES

Concussion is a commonly encountered diagnosis for pediatric emergency medicine (PEM) providers, yet little is known regarding referral patterns to specialists. Our goal was to assess PEM providers' referral patterns and current usage of standardized evaluation tools.

METHODS

This study was conducted as cross-sectional survey of PEM providers recruited from the American Academy of Pediatrics Section on Emergency Medicine Listserv. Surveys were distributed at 3 time points between December 1, 2020, and February 28, 2021, and included multiple choice, Likert scale, and free text questions. Descriptive statistics and bivariate analyses were used to describe the sample and compare responses between those with variable experience and confidence in concussion management.

RESULTS

In total, 162 of 491 Listserv members (33.0%) completed the survey. The factors most often reported to assist in referral decisions were history of severe (92.6%) or multiple (90.7%) prior concussions, prolonged symptom duration (89.5%), and severity of current symptoms (84.6%). Most providers reported having large experience (63.0%) and confidence (54.9%) in managing concussion. Standardized symptom scales (8.0%), vestibular (11.7%) and balance assessments (13.0%), and prognostic tools (6.8%) were infrequently used. Most (64.2%) providers felt specialty referral was important. More than 80% reported high likelihood to use an accurate risk stratification tool to facilitate referral.

CONCLUSIONS

Although most PEM providers reported significant experience and confidence in managing pediatric concussion, standardized assessment tools were infrequently used. Most were likely to use a risk stratification tool to assist in specialty referral. Future studies should assess the ability of targeted referral strategies to improve recovery for concussed youth.

User-Centered Design of A Novel Risk Prediction Behavior Change Tool Augmented With an Artificial Intelligence Engine (MyDiabetesIQ): A Sociotechnical Systems Approach

Background

Diabetes and its complications account for 10% of annual health care spending in the United Kingdom. Digital health care interventions (DHIs) can provide scalable care, fostering diabetes self-management and reducing the risk of complications. Tailorability (providing personalized interventions) and usability are key to DHI engagement/effectiveness. User-centered design of DHIs (aligning features to end users’ needs) can generate more usable interventions, avoiding unintended consequences and improving user engagement.

Objective

MyDiabetesIQ (MDIQ) is an artificial intelligence engine intended to predict users’ diabetes complications risk. It will underpin a user interface in which users will alter lifestyle parameters to see the impact on their future risks. MDIQ will link to an existing DHI, My Diabetes My Way (MDMW). We describe the user-centered design of the user interface of MDIQ as informed by human factors engineering.

Methods

Current users of MDMW were invited to take part in focus groups to gather their insights about users being shown their likelihood of developing diabetes-related complications and any risks they perceived from using MDIQ. Findings from focus groups informed the development of a prototype MDIQ interface, which was then user-tested through the “think aloud” method, in which users speak aloud about their thoughts/impressions while performing prescribed tasks. Focus group and think aloud transcripts were analyzed thematically, using a combination of inductive and deductive analysis. For think aloud data, a sociotechnical model was used as a framework for thematic analysis.

Results

Focus group participants (n=8) felt that some users could become anxious when shown their future complications risks. They highlighted the importance of easy navigation, jargon avoidance, and the use of positive/encouraging language. User testing of the prototype site through think aloud sessions (n=7) highlighted several usability issues. Issues included confusing visual cues and confusion over whether user-updated information fed back to health care teams. Some issues could be compounded for users with limited digital skills. Results from the focus groups and think aloud workshops were used in the development of a live MDIQ platform.

Conclusions

Acting on the input of end users at each iterative stage of a digital tool’s development can help to prioritize users throughout the design process, ensuring the alignment of DHI features with user needs. The use of the sociotechnical framework encouraged the consideration of interactions between different sociotechnical dimensions in finding solutions to issues, for example, avoiding the exclusion of users with limited digital skills. Based on user feedback, the tool could scaffold good goal setting, allowing users to balance their palatable future complications risk against acceptable lifestyle changes. Optimal control of diabetes relies heavily on self-management. Tools such as MDMW/ MDIQ can offer personalized support for self-management alongside access to users’ electronic health records, potentially helping to delay or reduce long-term complications, thereby providing significant reductions in health care costs.

Barriers to the Use of Clinical Decision Support for the Evaluation of Pulmonary Embolism: Qualitative Interview Study

BACKGROUND

Clinicians often disregard potentially beneficial clinical decision support (CDS).

OBJECTIVE

In this study, we sought to explore the psychological and behavioral barriers to the use of a CDS tool.

METHODS

We conducted a qualitative study involving emergency medicine physicians and physician assistants. A semistructured interview guide was created based on the Capability, Opportunity, and Motivation-Behavior model. Interviews focused on the barriers to the use of a CDS tool built based on Wells' criteria for pulmonary embolism to assist clinicians in establishing pretest probability of pulmonary embolism before imaging.

RESULTS

Interviews were conducted with 12 clinicians. Six barriers were identified, including (1) Bayesian reasoning, (2) fear of missing a pulmonary embolism, (3) time pressure or cognitive load, (4) gestalt includes Wells' criteria, (5) missed risk factors, and (6) social pressure.

CONCLUSIONS

Clinicians highlighted several important psychological and behavioral barriers to CDS use. Addressing these barriers will be paramount in developing CDS that can meet its potential to transform clinical care.

Electronic clinical decision support for children with minor head trauma and intracranial injuries: a sociotechnical analysis

BACKGROUND

Current management of children with minor head trauma (MHT) and intracranial injuries is not evidence-based and may place some children at risk of harm. Evidence-based electronic clinical decision support (CDS) for management of these children may improve patient safety and decrease resource use. To guide these efforts, we evaluated the sociotechnical environment impacting the implementation of electronic CDS, including workflow and communication, institutional culture, and hardware and software infrastructure, among other factors.

METHODS

Between March and May, 2020 semi-structured qualitative focus group interviews were conducted to identify sociotechnical influences on CDS implementation. Physicians from neurosurgery, emergency medicine, critical care, and pediatric general surgery were included, along with information technology specialists. Participants were recruited from nine health centers in the United States. Focus group transcripts were coded and analyzed using thematic analysis. The final themes were then cross-referenced with previously defined sociotechnical dimensions.

RESULTS

We included 28 physicians and four information technology specialists in seven focus groups (median five participants per group). Five physicians were trainees and 10 had administrative leadership positions. Through inductive thematic analysis, we identified five primary themes: (1) clinical impact; (2) stakeholders and users; (3) tool content; (4) clinical practice integration; and (5) post-implementation evaluation measures. Participants generally supported using CDS to determine an appropriate level-of-care for these children. However, some had mixed feelings regarding how the tool could best be used by different specialties (e.g. use by neurosurgeons versus non-neurosurgeons). Feedback from the interviews helped refine the tool content and also highlighted potential technical and workflow barriers to address prior to implementation.

CONCLUSIONS

We identified key factors impacting the implementation of electronic CDS for children with MHT and intracranial injuries. These results have informed our implementation strategy and may also serve as a template for future efforts to implement health information technology in a multidisciplinary, emergency setting.

National Imaging Trends in Pediatric Traumatic Brain Injury and Hydrocephalus

OBJECTIVE

Reduction in use of computed tomography (CT) in favor of rapid-sequence magnetic resonance imaging (MRI) to decrease pediatric radiation exposure has varied across institutions in the United States. The aims of this study were to understand national trends in CT and rapid-sequence MRI usage and identify variables affecting imaging practices and obstacles to CT reduction.

METHODS

This was a retrospective review of deidentified discharge data for children with hydrocephalus and traumatic brain injury (TBI) in the Healthcare Cost and Utilization Project Kids' Inpatient Database in 2000, 2003, 2006, 2009, 2012, and 2016. Utilization of MRI without contrast and CT was extracted using International Classification of Diseases, Ninth Revision, and International Classification of Diseases, Tenth Revision, codes. Hospital region and age cohorts were extracted and used to categorize data. χ2 tests and logistic regression were used for analysis.

RESULTS

Hospitalizations utilizing CT decreased (P < 0.05) and hospitalizations utilizing MRI increased (P < 0.05) overall in both diagnosis groups throughout the years analyzed. However, there was significant regional variation in imaging. The Northeast had higher CT rates (P < 0.05) and the South had lower CT rates in patients with hydrocephalus and TBI (P < 0.05). No regional variation was found for rates of MRI use in patients with TBI.

CONCLUSIONS

Nationwide, the average number of discharges after hospitalizations utilizing CT in patients with hydrocephalus and TBI has decreased, while discharges after hospitalizations utilizing MRI as an alternative imaging modality have increased. Despite successful overall CT reduction, significant regional variation exists within this trend showing inconsistent reduction of CT use.

Children With Minor Blunt Head Trauma Presenting to the Emergency Department

In our state-of-the-art review, we summarize the best-available evidence for the optimal emergency department management of children with minor blunt head trauma. Minor blunt head trauma in children is a common reason for emergency department evaluation, although clinically important traumatic brain injuries (TBIs) as a result are uncommon. Cranial computed tomography (CT) scanning is the reference standard for the diagnosis of TBIs, although they should be used judiciously because of the risk of lethal malignancy from ionizing radiation exposure, with the greatest risk to the youngest children. Available TBI prediction rules can assist with CT decision-making by identifying patients at either low risk for TBI, for whom CT scans may safely be obviated, or at high risk, for whom CT scans may be indicated. For clinical prediction rules to change practice, however, they require active implementation. Observation before CT decision-making in selected patients may further reduce CT rates without missing children with clinically important TBIs. Future work is also needed to incorporate patient and family preferences into these decision-making algorithms when the course of action is not clear.

Assessing workflow of emergency physicians in the use of clinical decision support

The emergency department (ED) is a complex environment where diagnoses must often be made quickly, based on incomplete information. Pulmonary embolism (PE) is an especially challenging diagnosis that is frequently delayed or missed due to its non-specific symptoms, and can be life-threatening when not treated. Clinical decision supports (CDS) have the potential to improve these difficult decisions; however, previous efforts to implement CDS in the ED have faced challenges due to poor usability and lack of workflow integration. The objective of this study is to identify potential barriers to workflow integration from the technology’s implementation and inform the CDS design; this is achieved by analyzing ED physicians’ workflow during a usability evaluation of two different CDS, a web-based risk scoring CDS and a CDS designed using an human-centered design (HCD) process and human factors (HF) design principles. The number of cases matching the guideline-based workflow and the percent of correct diagnostic decisions increased from the use of the HF-based CDS, but varied depending on the patient scenario. We identified three workflow variations, which had both positive and negative implications for the CDS design and implementation. The workflow analysis can be used to inform the CDS design and improve the technology’s usability and integration in physician workflow prior to implementation.

Developing a framework for evidence-based grading and assessment of predictive tools for clinical decision support

BACKGROUND

Clinical predictive tools quantify contributions of relevant patient characteristics to derive likelihood of diseases or predict clinical outcomes. When selecting predictive tools for implementation at clinical practice or for recommendation in clinical guidelines, clinicians are challenged with an overwhelming and ever-growing number of tools, most of which have never been implemented or assessed for comparative effectiveness. To overcome this challenge, we have developed a conceptual framework to Grade and Assess Predictive tools (GRASP) that can provide clinicians with a standardised, evidence-based system to support their search for and selection of efficient tools.

METHODS

A focused review of the literature was conducted to extract criteria along which tools should be evaluated. An initial framework was designed and applied to assess and grade five tools: LACE Index, Centor Score, Well's Criteria, Modified Early Warning Score, and Ottawa knee rule. After peer review, by six expert clinicians and healthcare researchers, the framework and the grading of the tools were updated.

RESULTS

GRASP framework grades predictive tools based on published evidence across three dimensions: 1) Phase of evaluation; 2) Level of evidence; and 3) Direction of evidence. The final grade of a tool is based on the highest phase of evaluation, supported by the highest level of positive evidence, or mixed evidence that supports a positive conclusion. Ottawa knee rule had the highest grade since it has demonstrated positive post-implementation impact on healthcare. LACE Index had the lowest grade, having demonstrated only pre-implementation positive predictive performance.

CONCLUSION

GRASP framework builds on widely accepted concepts to provide standardised assessment and evidence-based grading of predictive tools. Unlike other methods, GRASP is based on the critical appraisal of published evidence reporting the tools' predictive performance before implementation, potential effect and usability during implementation, and their post-implementation impact. Implementing the GRASP framework as an online platform can enable clinicians and guideline developers to access standardised and structured reported evidence of existing predictive tools. However, keeping GRASP reports up-to-date would require updating tools' assessments and grades when new evidence becomes available, which can only be done efficiently by employing semi-automated methods for searching and processing the incoming information.

Methodological standards for the development and evaluation of clinical prediction rules: a review of the literature

Clinical prediction rules (CPRs) that predict the absolute risk of a clinical condition or future outcome for individual patients are abundant in the medical literature; however, systematic reviews have demonstrated shortcomings in the methodological quality and reporting of prediction studies. To maximise the potential and clinical usefulness of CPRs, they must be rigorously developed and validated, and their impact on clinical practice and patient outcomes must be evaluated. This review aims to present a comprehensive overview of the stages involved in the development, validation and evaluation of CPRs, and to describe in detail the methodological standards required at each stage, illustrated with examples where appropriate. Important features of the study design, statistical analysis, modelling strategy, data collection, performance assessment, CPR presentation and reporting are discussed, in addition to other, often overlooked aspects such as the acceptability, cost-effectiveness and longer-term implementation of CPRs, and their comparison with clinical judgement. Although the development and evaluation of a robust, clinically useful CPR is anything but straightforward, adherence to the plethora of methodological standards, recommendations and frameworks at each stage will assist in the development of a rigorous CPR that has the potential to contribute usefully to clinical practice and decision-making and have a positive impact on patient care.

Decision-centred design in healthcare: The process of identifying a decision support tool for airway management

Current decision support interventions for airway management in anaesthesia lack the application of Human Factors Engineering; leading to interventions that can be disruptive, inefficient and error-inducing. This study followed a decision-centred design process to identify decision support that can assist anaesthesia teams with challenging airway management situations. Field observations, Critical Decision Method interviews and focus groups were conducted to identify the most difficult decisions and their requirements. Data triangulation narrowed the focus to key decisions related to preparation and planning, and the transitioning between airway techniques during difficulties. Five decision-support interventions were identified and positively rated by anaesthesia team members in relation to their perceived effectiveness. An organized airway equipment trolley was chosen as the most beneficial decision support intervention. This study reiterated the key importance of both Human Factors Engineering and data triangulation when designing for healthcare.

Increasing Safe Outpatient Management of Emergency Department Patients With Pulmonary Embolism: A Controlled Pragmatic Trial

BACKGROUND

Many low-risk patients with acute pulmonary embolism (PE) in the emergency department (ED) are eligible for outpatient care but are hospitalized nonetheless. One impediment to home discharge is the difficulty of identifying which patients can safely forgo hospitalization.

OBJECTIVE

To evaluate the effect of an integrated electronic clinical decision support system (CDSS) to facilitate risk stratification and decision making at the site of care for patients with acute PE.

DESIGN

Controlled pragmatic trial. (ClinicalTrials.gov: NCT03601676).

SETTING

All 21 community EDs of an integrated health care delivery system (Kaiser Permanente Northern California).

PATIENTS

Adult ED patients with acute PE.

INTERVENTION

Ten intervention sites selected by convenience received a multidimensional technology and education intervention at month 9 of a 16-month study period (January 2014 to April 2015); the remaining 11 sites served as concurrent controls.

MEASUREMENTS

The primary outcome was discharge to home from either the ED or a short-term (<24-hour) outpatient observation unit based in the ED. Adverse outcomes included return visits for PE-related symptoms within 5 days and recurrent venous thromboembolism, major hemorrhage, and all-cause mortality within 30 days. A difference-in-differences approach was used to compare pre-post changes at intervention versus control sites, with adjustment for demographic and clinical characteristics.

RESULTS

Among 881 eligible patients diagnosed with PE at intervention sites and 822 at control sites, adjusted home discharge increased at intervention sites (17.4% pre- to 28.0% postintervention) without a concurrent increase at control sites (15.1% pre- and 14.5% postintervention). The difference-in-differences comparison was 11.3 percentage points (95% CI, 3.0 to 19.5 percentage points; P = 0.007). No increases were seen in 5-day return visits related to PE or in 30-day major adverse outcomes associated with CDSS implementation.

LIMITATION

Lack of random allocation.

CONCLUSION

Implementation and structured promotion of a CDSS to aid physicians in site-of-care decision making for ED patients with acute PE safely increased outpatient management.

PRIMARY FUNDING SOURCE

Garfield Memorial National Research Fund and The Permanente Medical Group Delivery Science and Physician Researcher Programs.

Quality Improvement and Safety in Pediatric Emergency Medicine

Pediatric emergency medicine quality work continues to focus on the National Academies of Sciences, Engineering, and Medicine's 6 domains of quality, with a need for specific emphasis on equity and patient centeredness. Adopting the principles of high-reliability organizations, pediatric emergency departments should become increasing transparent with benchmarking and collaboration across institutions in order to develop an infrastructure for quality and safety to improve the care of pediatric patients in the emergency department.

Assessment of the Feasibility of automated, real-time clinical decision support in the emergency department using electronic health record data

BACKGROUND

The use of big data and machine learning within clinical decision support systems (CDSSs) has the potential to transform medicine through better prognosis, diagnosis and automation of tasks. Real-time application of machine learning algorithms, however, is dependent on data being present and entered prior to, or at the point of, CDSS deployment. Our aim was to determine the feasibility of automating CDSSs within electronic health records (EHRs) by investigating the timing, data categorization, and completeness of documentation of their individual components of two common Clinical Decision Rules (CDRs) in the Emergency Department.

METHODS

The CURB-65 severity score and HEART score were randomly selected from a list of the top emergency medicine CDRs. Emergency department (ED) visits with ICD-9 codes applicable to our CDRs were eligible. The charts were reviewed to determine the categorization components of the CDRs as structured and/or unstructured, median times of documentation, portion of charts with all data components documented as structured data, portion of charts with all structured CDR components documented before ED departure. A kappa score was calculated for interrater reliability.

RESULTS

The components of the CDRs were mainly documented as structured data for the CURB-65 severity score and HEART score. In the CURB-65 group, 26.8% of charts had all components documented as structured data, and 67.8% in the HEART score. Documentation of some CDR components often occurred late for both CDRs. Only 21 and 11% of patients had all CDR components documented as structured data prior to ED departure for the CURB-65 and HEART score groups, respectively. The interrater reliability for the CURB-65 score review was 0.75 and 0.65 for the HEART score.

CONCLUSION

Our study found that EHRs may be unable to automatically calculate popular CDRs-such as the CURB-65 severity score and HEART score-due to missing components and late data entry.

Applying the RE-AIM Framework for the Evaluation of a Clinical Decision Support Tool for Pediatric Head Trauma: A Mixed-Methods Study

BACKGROUND

The overuse of cranial computed tomography (CT) to diagnose potential traumatic brain injuries (TBIs) exposes children with minor blunt head trauma to unnecessary ionizing radiation. The Pediatric Emergency Care Applied Research Network and the Clinical Research on Emergency Services and Treatments Network implemented TBI prediction rules via electronic health record (EHR) clinical decision support (CDS) to decrease use of CTs in children with minor blunt head trauma.

OBJECTIVE

This article aims to facilitate implementation and dissemination of a CDS alert into emergency departments around the country.

METHODS

We evaluated the EHR CT CDS tool through a mixed-methods analysis of 38 audio-recorded interviews with health care stakeholders and quantitative data sources, using the Reach, Efficacy, Adoption, Implementation, and Maintenance framework.

RESULTS

Reach -: The demographics of participants enrolled in the clinical trial were consistent with national estimates of TBI prevalence. Efficacy-There was a variable and modest reduction in CT rates for the 8,067 children with minor head trauma whose clinicians received CDS. Adoption -: The EHR CT CDS tool was well matched with the organizational mission, values, and priorities of the implementation sites. Implementation- The most important predisposing factors for successful implementation were the presence of an approachable clinical champion at each site and belief that the tool was a relevant, reusable knowledge asset. Enabling factors included an effective integration within the clinical workflow, organizational investment in user training, and ease of use. Maintenance -: Reinforcing factors for the EHR CT CDS tool included a close fit with the institutional culture, belief that it was useful for providers and families, and a good educational and informational tool. As such, the EHR CT CDS tool was maintained in clinical practice long after study completion.

CONCLUSION

Data from this mixed-methods study complement findings from the efficacy trial and provide critical components for consideration prior to integration and subsequent dissemination of the EHR CT CDS tool.

TRIAL REGISTRATION

NCT01453621, Registered September 27, 2011.

Incorporating Guideline Adherence and Practice Implementation Issues into the Design of Decision Support for Beta-Blocker Titration for Heart Failure

BACKGROUND

The recognition of and response to undertreatment of heart failure (HF) patients can be complicated. A clinical reminder can facilitate use of guideline-concordant β-blocker titration for HF patients with depressed ejection fraction. However, the design must consider the cognitive demands on the providers and the context of the work.

OBJECTIVE

This study's purpose is to develop requirements for a clinical decision support tool (a clinical reminder) by analyzing the cognitive demands of the task along with the factors in the Cabana framework of physician adherence to guidelines, the health information technology (HIT) sociotechnical framework, and the Promoting Action on Research Implementation in Health Services (PARIHS) framework of health services implementation. It utilizes a tool that extracts information from medical records (including ejection fraction in free text reports) to identify qualifying patients at risk of undertreatment.

METHODS

We conducted interviews with 17 primary care providers, 5 PharmDs, and 5 Registered Nurses across three Veterans Health Administration outpatient clinics. The interviews were based on cognitive task analysis (CTA) methods and enhanced through the inclusion of the Cabana, HIT sociotechnical, and PARIHS frameworks. The analysis of the interview data led to the development of requirements and a prototype design for a clinical reminder. We conducted a small pilot usability assessment of the clinical reminder using realistic clinical scenarios.

RESULTS

We identified organizational challenges (such as time pressures and underuse of pharmacists), knowledge issues regarding the guideline, and information needs regarding patient history and treatment status. We based the design of the clinical reminder on how to best address these challenges. The usability assessment indicated the tool could help the decision and titration processes.

CONCLUSION

Through the use of CTA methods enhanced with adherence, sociotechnical, and implementation frameworks, we designed a decision support tool that considers important challenges in the decision and execution of β-blocker titration for qualifying HF patients at risk of undertreatment.

Development of a digital clinical pathway for emergency medicine: Lessons from usability testing and implementation failure

Clinical pathways are used to support the management of patients in emergency departments. An existing document-based clinical pathway was used as the foundation on which to design and build a digital clinical pathway for acute chest pain, with the aim of improving clinical calculations, clinician decision-making, documentation, and data collection. Established principles of decision support system design were used to build an application within the existing electronic health record, before testing with a multidisciplinary team of doctors using a think-aloud protocol. Technical authoring was successful, however, usability testing revealed that the user experience and the flexibility of workflow within the application were critical barriers to implementation. Emergency medicine and acute care decision support systems face particular challenges to existing models of linear workflow that should be deliberately addressed in digital pathway design. We make key recommendations regarding digital pathway design in emergency medicine.

Point-of-care Cognitive Support Technology in Emergency Departments: A Scoping Review of Technology Acceptance by Clinicians

OBJECTIVE

Cognitive support technologies that support clinical decisions and practices in the emergency department (ED) have the potential to optimize patient care. However, limited uptake by clinicians can prevent successful implementation. A better understanding of acceptance of these technologies from the clinician perspective is needed. We conducted a scoping review to synthesize diverse, emerging evidence on clinicians' acceptance of point-of-care (POC) cognitive support technology in the ED.

METHOD

We systematically searched 10 electronic databases and gray literature published from January 2006 to December 2016. Studies of any design assessing an ED-based POC cognitive support technology were considered eligible for inclusion. Studies were required to report outcome data for technology acceptance. Two reviewers independently screened studies for relevance and quality. Study quality was assessed using the Mixed-Methods Appraisal Tool. A descriptive analysis of the features of POC cognitive support technology for each study is presented, illustrating trends in technology development and evaluation. A thematic analysis of clinician, technical, patient, and organizational factors associated with technology acceptance is also presented.

RESULTS

Of the 1,563 references screened for eligibility, 24 met the inclusion criteria and were included in the review. Most studies were published from 2011 onward (88%), scored high for methodologic quality (79%), and examined POC technologies that were novel and newly introduced into the study setting (63%). Physician use of POC technology was the most commonly studied (67%). Technology acceptance was frequently conceptualized and measured by factors related to clinician attitudes and beliefs. Experience with the technology, intention to use, and actual use were also more common outcome measures of technology acceptance. Across studies, perceived usefulness was the most noteworthy factor impacting technology acceptance, and clinicians generally had positive perceptions of the use of POC cognitive support technology in the ED. However, the actual use of POC cognitive support technology reported by clinicians was low-use, by proportion of patient cases, ranged from 30% to 59%. Of the 24 studies, only two studies investigated acceptance of POC cognitive support technology currently implemented in the ED, offering "real-world" clinical practice data. All other studies focused on acceptance of novel technologies. Technical aspects such as an unfriendly user interface, presentation of redundant or ambiguous information, and required user effort had a negative impact on acceptance. Patient expectations were also found to have a negative impact, while patient safety implications had a positive impact. Institutional support was also reported to impact technology acceptance.

CONCLUSIONS

Findings from this scoping review suggest that while ED clinicians acknowledge the utility and value of using POC cognitive support technology, actual use of such technology can be low. Further, few studies have evaluated the acceptance and use of POC technologies in routine care. Prospective studies that evaluate how ED clinicians appraise and consider POC technology use in clinical practice are now needed with diverse clinician samples. While this review identified multiple factors contributing to technology acceptance, determining how clinician, technical, patient, and organizational factors mediate or moderate acceptance should also be a priority.

Working at the intersection of context, culture, and technology: Provider perspectives on antimicrobial stewardship in the emergency department using electronic health record clinical decision support

BACKGROUND

Antibiotic stewardship programs (ASPs) have not been fully developed for the emergency department (ED), in part the result of the barriers characteristic of this setting. Electronic health record-based clinical decision support (EHR CDS) represents a promising strategy to implement ASPs in the ED. We aimed to determine the cultural beliefs and structural barriers and facilitators to implementation of antimicrobial stewardship in the pediatric ED using EHR CDS.

METHODS

Interviews and focus groups were conducted with hospital and ED leadership, attending ED physicians, nurse practitioners, physician assistants, and residents at a single health system in Colorado. We reviewed and coded the data using constant comparative analysis and framework analysis until a final set of themes emerged.

RESULTS

Two dominant perceptions shaped providers' perspectives on ASPs in the ED and EHR CDS: (1) maintaining workflow efficiency and (2) constrained decision-making autonomy. Clinicians identified structural barriers to ASPs, such as pace of the ED, and various beliefs that shaped patterns of practice, including accommodating the prescribing decisions of other providers and managing parental expectations. Recommendations to enhance uptake focused on designing a simple yet flexible user interface, providing clinicians with performance data, and on-boarding clinicians to enhance buy-in.

CONCLUSIONS

Developing a successful ED-based ASP using EHR CDS should attend to technologic needs, the institutional context, and the cultural beliefs of practice associated with providers' antibiotic prescribing.

Use of Traumatic Brain Injury Prediction Rules With Clinical Decision Support

OBJECTIVES

We determined whether implementing the Pediatric Emergency Care Applied Research Network (PECARN) traumatic brain injury (TBI) prediction rules and providing risks of clinically important TBIs (ciTBIs) with computerized clinical decision support (CDS) reduces computed tomography (CT) use for children with minor head trauma.

METHODS

Nonrandomized trial with concurrent controls at 5 pediatric emergency departments (PEDs) and 8 general EDs (GEDs) between November 2011 and June 2014. Patients were <18 years old with minor blunt head trauma. Intervention sites received CDS with CT recommendations and risks of ciTBI, both for patients at very low risk of ciTBI (no Pediatric Emergency Care Applied Research Network rule factors) and those not at very low risk. The primary outcome was the rate of CT, analyzed by site, controlling for time trend.

RESULTS

We analyzed 16 635 intervention and 2394 control patients. Adjusted for time trends, CT rates decreased significantly (P < .05) but modestly (2.3%-3.7%) at 2 of 4 intervention PEDs for children at very low risk. The other 2 PEDs had small (0.8%-1.5%) nonsignificant decreases. CT rates did not decrease consistently at the intervention GEDs, with low baseline CT rates (2.1%-4.0%) in those at very low risk. The control PED had little change in CT use in similar children (from 1.6% to 2.9%); the control GED showed a decrease in the CT rate (from 7.1% to 2.6%). For all children with minor head trauma, intervention sites had small decreases in CT rates (1.7%-6.2%).

CONCLUSIONS

The implementation of TBI prediction rules and provision of risks of ciTBIs by using CDS was associated with modest, safe, but variable decreases in CT use. However, some secular trends were also noted.

Optimizing Clinical Decision Support in the Electronic Health Record. Clinical Characteristics Associated with the Use of a Decision Tool for Disposition of ED Patients with Pulmonary Embolism

OBJECTIVE

Adoption of clinical decision support (CDS) tools by clinicians is often limited by workflow barriers. We sought to assess characteristics associated with clinician use of an electronic health record-embedded clinical decision support system (CDSS).

METHODS

In a prospective study on emergency department (ED) activation of a CDSS tool across 14 hospitals between 9/1/14 to 4/30/15, the CDSS was deployed at 10 active sites with an on-site champion, education sessions, iterative feedback, and up to 3 gift cards/clinician as an incentive. The tool was also deployed at 4 passive sites that received only an introductory educational session. Activation of the CDSS - which calculated the Pulmonary Embolism Severity Index (PESI) score and provided guidance - and associated clinical data were collected prospectively. We used multivariable logistic regression with random effects at provider/facility levels to assess the association between activation of the CDSS tool and characteristics at: 1) patient level (PESI score), 2) provider level (demographics and clinical load at time of activation opportunity), and 3) facility level (active vs. passive site, facility ED volume, and ED acuity at time of activation opportunity).

RESULTS

Out of 662 eligible patient encounters, the CDSS was activated in 55%: active sites: 68% (346/512); passive sites 13% (20/150). In bivariate analysis, active sites had an increase in activation rates based on the number of prior gift cards the physician had received (96% if 3 prior cards versus 60% if 0, p<0.0001). At passive sites, physicians < age 40 had higher rates of activation (p=0.03). In multivariable analysis, active site status, low ED volume at the time of diagnosis and PESI scores I or II (compared to III or higher) were associated with higher likelihood of CDSS activation.

CONCLUSIONS

Performing on-site tool promotion significantly increased odds of CDSS activation. Optimizing CDSS adoption requires active education.

Clinical Decision Support for a Multicenter Trial of Pediatric Head Trauma: Development, Implementation, and Lessons Learned

INTRODUCTION

For children who present to emergency departments (EDs) due to blunt head trauma, ED clinicians must decide who requires computed tomography (CT) scanning to evaluate for traumatic brain injury (TBI). The Pediatric Emergency Care Applied Research Network (PECARN) derived and validated two age-based prediction rules to identify children at very low risk of clinically-important traumatic brain injuries (ciTBIs) who do not typically require CT scans. In this case report, we describe the strategy used to implement the PECARN TBI prediction rules via electronic health record (EHR) clinical decision support (CDS) as the intervention in a multicenter clinical trial.

METHODS

Thirteen EDs participated in this trial. The 10 sites receiving the CDS intervention used the Epic(®) EHR. All sites implementing EHR-based CDS built the rules by using the vendor's CDS engine. Based on a sociotechnical analysis, we designed the CDS so that recommendations could be displayed immediately after any provider entered prediction rule data. One central site developed and tested the intervention package to be exported to other sites. The intervention package included a clinical trial alert, an electronic data collection form, the CDS rules and the format for recommendations.

RESULTS

The original PECARN head trauma prediction rules were derived from physician documentation while this pragmatic trial led each site to customize their workflows and allow multiple different providers to complete the head trauma assessments. These differences in workflows led to varying completion rates across sites as well as differences in the types of providers completing the electronic data form. Site variation in internal change management processes made it challenging to maintain the same rigor across all sites. This led to downstream effects when data reports were developed.

CONCLUSIONS

The process of a centralized build and export of a CDS system in one commercial EHR system successfully supported a multicenter clinical trial.

Comparison of Prediction Rules and Clinician Suspicion for Identifying Children With Clinically Important Brain Injuries After Blunt Head Trauma

OBJECTIVE

Children with minor head trauma frequently present to emergency departments (EDs). Identifying those with traumatic brain injuries (TBIs) can be difficult, and it is unknown whether clinical prediction rules outperform clinician suspicion. Our primary objective was to compare the test characteristics of the Pediatric Emergency Care Applied Research Network (PECARN) TBI prediction rules to clinician suspicion for identifying children with clinically important TBIs (ciTBIs) after minor blunt head trauma. Our secondary objective was to determine the reasons for obtaining computed tomography (CT) scans when clinical suspicion of ciTBI was low.

METHODS

This was a planned secondary analysis of a previously conducted observational cohort study conducted in PECARN to derive and validate clinical prediction rules for ciTBI among children with minor blunt head trauma in 25 PECARN EDs. Clinicians recorded their suspicion of ciTBI before CT as <1, 1-5, 6-10, 11-50, or >50%. We defined ciTBI as 1) death from TBI, 2) neurosurgery, 3) intubation for more than 24 hours for TBI, or 4) hospital admission of 2 nights or more associated with TBI on CT. To avoid overfitting of the prediction rules, we performed comparisons of the prediction rules and clinician suspicion on the validation group only. On the validation group, we compared the test accuracies of clinician suspicion > 1% versus having at least one predictor in the PECARN TBI age-specific prediction rules for identifying children with ciTBIs (one rule for children <2 years [preverbal], the other rule for children >2 years [verbal]).

RESULTS

In the parent study, we enrolled 8,627 children to validate the prediction rules, after enrolling 33,785 children to derive the prediction rules. In the validation group, clinician suspicion of ciTBI was recorded in 8,496/8,627 (98.5%) patients, and 87 (1.0%) had ciTBIs. CT scans were obtained in 2,857 (33.6%) patients in the validation group for whom clinician suspicion of ciTBI was recorded, including 2,099/7,688 (27.3%) of those with clinician suspicion of ciTBI of <1% and 758/808 (93.8%) of those with clinician suspicion >1%. The PECARN prediction rules were significantly more sensitive than clinician suspicion >1% of ciTBI for preverbal (100% [95% confidence interval {CI} = 86.3% to 100%] vs. 60.0% [95% CI = 38.7% to 78.9%]) and verbal children (96.8% [95% CI = 88.8% to 99.6%] vs. 64.5% [95% CI = 51.3% to 76.3%]). Prediction rule specificity, however, was lower than clinician suspicion >1% for preverbal children (53.6% [95% CI = 51.5% to 55.7%] vs. 92.4% [95% CI = 91.2% to 93.5%]) and verbal children (58.2% [95% CI = 56.9% to 59.4%] vs. 90.6% [95% CI = 89.8% to 91.3%]). Of the 7,688 patients in the validation group with clinician suspicion recorded as <1%, CTs were nevertheless obtained in 2,099 (27.3%). Three of 16 (18.8%) patients undergoing neurosurgery had clinician suspicion of ciTBI <1%.

CONCLUSIONS

The PECARN TBI prediction rules had substantially greater sensitivity, but lower specificity, than clinician suspicion of ciTBI for children with minor blunt head trauma. Because CT ordering did not follow clinician suspicion of <1%, these prediction rules can augment clinician judgment and help obviate CT ordering for children at very low risk of ciTBI.

Assessment of Pediatric Neurotrauma Imaging Appropriateness at a Level I Pediatric Trauma Center

PURPOSE

The aim of this study was to assess the prevalence of appropriate neuroimaging on the basis of the ACR Appropriateness Criteria among pediatric patients presenting after head trauma to a level I emergency department.

METHODS

A retrospective emergency department record review was performed for patients <18 years of age undergoing head CT or MRI for the indication "head trauma" between January 2013 and December 2014. Clinical history and symptoms were compared with the ACR Appropriateness Criteria; the indication was deemed appropriate for ratings of ≥7. Patients were analyzed by age, gender, presentation, imaging obtained, follow-up, treatment, and outcomes.

RESULTS

Among 207 patients, 120 (58%) were imaged with CT and 107 (52%) with MRI; 20 patients underwent both CT and MRI. One hundred eighty-seven patients (90.3%) were appropriately imaged, with 90.0% of CT studies (108 of 120) deemed appropriate and 91.6% of MRI studies (98 of 107) deemed appropriate. Younger patients were more likely to be inappropriately imaged with CT or MRI than older patients (P = .02 and P < .01, respectively). Patients undergoing CT were older (mean age 9.9 ± 5.8 years) and more likely to be male (85.2%) than those undergoing MRI (5.6 ± 5.6 years and 55.1%, respectively) (P < .01 and P < .001, respectively). The diagnostic yield of positive imaging findings for intracranial trauma was significantly lower in the MRI group (P < .01), and patients undergoing MRI were significantly more likely to return to baseline with conservative management (P < .01).

CONCLUSIONS

Most pediatric patients undergoing neuroimaging for head trauma did so appropriately per ACR guidelines and had symptom resolution with conservative management. The minority not imaged appropriately represent a target for quality improvement efforts.

Optimizing Patient-centered Communication and Multidisciplinary Care Coordination in Emergency Diagnostic Imaging: A Research Agenda

Patient-centered emergency diagnostic imaging relies on efficient communication and multispecialty care coordination to ensure optimal imaging utilization. The construct of the emergency diagnostic imaging care coordination cycle with three main phases (pretest, test, and posttest) provides a useful framework to evaluate care coordination in patient-centered emergency diagnostic imaging. This article summarizes findings reached during the patient-centered outcomes session of the 2015 Academic Emergency Medicine consensus conference "Diagnostic Imaging in the Emergency Department: A Research Agenda to Optimize Utilization." The primary objective was to develop a research agenda focused on 1) defining component parts of the emergency diagnostic imaging care coordination process, 2) identifying gaps in communication that affect emergency diagnostic imaging, and 3) defining optimal methods of communication and multidisciplinary care coordination that ensure patient-centered emergency diagnostic imaging. Prioritized research questions provided the framework to define a research agenda for multidisciplinary care coordination in emergency diagnostic imaging.

Generation of Evidence and Translation Into Practice: Lessons Learned and Future Directions

The generation, validation, and then translation of definitive evidence to bedside evidence-based practice is inconsistent and presents many challenges to emergency department (ED) researchers and clinicians. This is particularly true for diagnostic imaging in the ED, where benefits and drawbacks may be difficult to assess in the chaotic ED setting. This article describes, in large part, the experience of the Pediatric Emergency Care Applied Research Network (PECARN) in deriving and validating the traumatic brain injury prediction rules and how PECARN is translating these prediction rules into clinical practice. Furthermore, we discuss the potential for patient/parent shared decision-making with a focus on patient-centered outcomes in ED research and the role this shared decision-making may play in enhancing knowledge translation and implementation of evidence-based care in the ED.

Knowledge Translation and Barriers to Imaging Optimization in the Emergency Department: A Research Agenda

Researchers have attempted to optimize imaging utilization by describing which clinical variables are more predictive of acute disease and, conversely, what combination of variables can obviate the need for imaging. These results are then used to develop evidence-based clinical pathways, clinical decision instruments, and clinical practice guidelines. Despite the validation of these results in subsequent studies, with some demonstrating improved outcomes, their actual use is often limited. This article outlines a research agenda to promote the dissemination and implementation (also known as knowledge translation) of evidence-based interventions for emergency department (ED) imaging, i.e., clinical pathways, clinical decision instruments, and clinical practice guidelines. We convened a multidisciplinary group of stakeholders and held online and telephone discussions over a 6-month period culminating in an in-person meeting at the 2015 Academic Emergency Medicine consensus conference. We identified the following four overarching research questions: 1) what determinants (barriers and facilitators) influence emergency physicians' use of evidence-based interventions when ordering imaging in the ED; 2) what implementation strategies at the institutional level can improve the use of evidence-based interventions for ED imaging; 3) what interventions at the health care policy level can facilitate the adoption of evidence-based interventions for ED imaging; and 4) how can health information technology, including electronic health records, clinical decision support, and health information exchanges, be used to increase awareness, use, and adherence to evidence-based interventions for ED imaging? Advancing research that addresses these questions will provide valuable information as to how we can use evidence-based interventions to optimize imaging utilization and ultimately improve patient care.

Comparison of Clinician Suspicion Versus a Clinical Prediction Rule in Identifying Children at Risk for Intra-abdominal Injuries After Blunt Torso Trauma

OBJECTIVES

Emergency department (ED) identification and radiographic evaluation of children with intra-abdominal injuries who need acute intervention can be challenging. To date, it is unclear if a clinical prediction rule is superior to unstructured clinician judgment in identifying these children. The objective of this study was to compare the test characteristics of clinician suspicion with a derived clinical prediction rule to identify children at risk of intra-abdominal injuries undergoing acute intervention following blunt torso trauma.

METHODS

This was a planned subanalysis of a prospective, multicenter observational study of children (<18 years old) with blunt torso trauma conducted in 20 EDs in the Pediatric Emergency Care Applied Research Network (PECARN). Clinicians documented their suspicion for the presence of intra-abdominal injuries needing acute intervention as <1, 1 to 5, 6 to 10, 11 to 50, or >50% prior to knowledge of abdominal computed tomography (CT) scanning (if performed). Intra-abdominal injuries undergoing acute intervention were defined by a therapeutic laparotomy, angiographic embolization, blood transfusion for abdominal hemorrhage, or intravenous fluid administration for 2 or more days in those with pancreatic or gastrointestinal injuries. Patients were considered to be positive for clinician suspicion if suspicion was documented as ≥1%. Suspicion ≥ 1% was compared to the presence of any variable in the prediction rule for identifying children with intra-abdominal injuries undergoing acute intervention.

RESULTS

Clinicians recorded their suspicion in 11,919 (99%) of 12,044 patients enrolled in the parent study. Intra-abdominal injuries undergoing acute intervention were diagnosed in 203 (2%) patients. Abdominal CT scans were obtained in the ED in 2,302 of the 2,667 (86%, 95% confidence interval [CI] = 85% to 88%) enrolled patients with clinician suspicion ≥1% and in 3,016 of the 9,252 (33%, 95% CI = 32% to 34%) patients with clinician suspicion < 1%. Sensitivity of the prediction rule for intra-abdominal injuries undergoing acute intervention (197 of 203; 97.0%, 95% CI = 93.7% to 98.9%) was higher than that of clinician suspicion ≥1% (168 of 203; 82.8%, 95% CI = 76.9% to 87.7%; difference = 14.2%, 95% CI = 8.6% to 20.0%). Specificity of the prediction rule (4,979 of the 11,716; 42.5%, 95% CI = 41.6% to 43.4%), however, was lower than that of clinician suspicion (9,217 of the 11,716, 78.7%, 95% CI = 77.9% to 79.4%; difference = -36.2%, 95% CI = -37.3% to -35.0%). Thirty-five (0.4%, 95% CI = 0.3% to 0.5%) patients with clinician suspicion < 1% had intra-abdominal injuries that underwent acute intervention.

CONCLUSIONS

The derived clinical prediction rule had a significantly higher sensitivity, but lower specificity, than clinician suspicion for identifying children with intra-abdominal injuries undergoing acute intervention. The higher specificity of clinician suspicion, however, did not translate into clinical practice, as clinicians frequently obtained abdominal CT scans in patients they considered very low risk. If validated, this prediction rule can assist in clinical decision-making around abdominal CT use in children with blunt torso trauma.

Designing Real-time Decision Support for Trauma Resuscitations

BACKGROUND

Use of electronic clinical decision support (eCDS) has been recommended to improve implementation of clinical decision rules. Many eCDS tools, however, are designed and implemented without taking into account the context in which clinical work is performed. Implementation of the pediatric traumatic brain injury (TBI) clinical decision rule at one Level I pediatric emergency department includes an electronic questionnaire triggered when ordering a head computed tomography using computerized physician order entry (CPOE). Providers use this CPOE tool in less than 20% of trauma resuscitation cases. A human factors engineering approach could identify the implementation barriers that are limiting the use of this tool.

OBJECTIVES

The objective was to design a pediatric TBI eCDS tool for trauma resuscitation using a human factors approach. The hypothesis was that clinical experts will rate a usability-enhanced eCDS tool better than the existing CPOE tool for user interface design and suitability for clinical use.

METHODS

This mixed-methods study followed usability evaluation principles. Pediatric emergency physicians were surveyed to identify barriers to using the existing eCDS tool. Using standard trauma resuscitation protocols, a hierarchical task analysis of pediatric TBI evaluation was developed. Five clinical experts, all board-certified pediatric emergency medicine faculty members, then iteratively modified the hierarchical task analysis until reaching consensus. The software team developed a prototype eCDS display using the hierarchical task analysis. Three human factors engineers provided feedback on the prototype through a heuristic evaluation, and the software team refined the eCDS tool using a rapid prototyping process. The eCDS tool then underwent iterative usability evaluations by the five clinical experts using video review of 50 trauma resuscitation cases. A final eCDS tool was created based on their feedback, with content analysis of the evaluations performed to ensure all concerns were identified and addressed.

RESULTS

Among 26 EPs (76% response rate), the main barriers to using the existing tool were that the information displayed is redundant and does not fit clinical workflow. After the prototype eCDS tool was developed based on the trauma resuscitation hierarchical task analysis, the human factors engineers rated it to be better than the CPOE tool for nine of 10 standard user interface design heuristics on a three-point scale. The eCDS tool was also rated better for clinical use on the same scale, in 84% of 50 expert-video pairs, and was rated equivalent in the remainder. Clinical experts also rated barriers to use of the eCDS tool as being low.

CONCLUSIONS

An eCDS tool for diagnostic imaging designed using human factors engineering methods has improved perceived usability among pediatric emergency physicians.

Quality Improvement Effort to Reduce Cranial CTs for Children With Minor Blunt Head Trauma

OBJECTIVE

Blunt head trauma is a common injury in children, although it rarely requires surgical intervention. Cranial computed tomography (CT) is the reference standard for the diagnosis of traumatic brain injury but has been associated with increased lifetime malignancy risk. We implemented a multifaceted quality improvement initiative to decrease the use of cranial CT for children with minor head injuries.

METHODS

We designed and implemented a quality improvement effort that included an evidence-based guideline as well as individual feedback for children aged 0 to 21 years who present to the emergency department (ED) for evaluation of minor blunt head trauma. Our primary outcome was cranial CT rate, and our balancing measure was any return to the ED within 72 hours that required hospitalization. We used statistical process control methodology to measure cranial CT rates over time.

RESULTS

We included 6851 ED visits of which 4242 (62%) occurred in the post-guideline implementation period. From a baseline CT rate of 21%, we observed an absolute reduction of 6% in cranial CT rate (95% confidence interval 3% to 9%) after initial guideline implementation and an additional absolute reduction of 6% (95% confidence interval 4% to 8%) after initiation of individual provider feedback. No children discharged from the ED required admission within 72 hours of initial evaluation.

CONCLUSIONS

An ED quality improvement effort that included an evidence-based guideline as well as individual provider feedback was associated with a reduction in cranial CT rates without an increase in missed significant head injuries.

Back to the Bedside: Developing a Bedside Aid for Concussion and Brain Injury Decisions in the Emergency Department

Context:

Current information-rich electronic health record (EHR) interfaces require large, high-resolution screens running on desktop computers. This interface compromises the provider’s already limited time at the bedside by physically separating the patient from the doctor. The case study presented here describes a patient-centered clinical decision support (CDS) design process that aims to bring the physician back to the bedside by integrating a patient decision aid with CDS for shared use by the patient and provider on a touchscreen tablet computer for deciding whether or not to obtain a CT scan for minor head injury in the emergency department, a clinical scenario that could benefit from CDS but has failed previous implementation attempts.

Case Description:

This case study follows the user-centered design (UCD) approach to build a bedside aid that is useful and usable, and that promotes shared decision-making between patients and their providers using a tablet computer at the bedside. The patient-centered decision support design process focuses on the prototype build using agile software development, but also describes the following: (1) the requirement gathering phase including triangulated qualitative research (focus groups and cognitive task analysis) to understand current challenges, (2) features for patient education, the physician, and shared decision-making, (3) system architecture and technical requirements, and (4) future plans for formative usability testing and field testing.

Lessons Learned:

We share specific lessons learned and general recommendations from critical insights gained in the patient-centered decision support design process about early stakeholder engagement, EHR integration, external expert feedback, challenges to two users on a single device, project management, and accessibility.

Conclusions:

Successful implementation of this tool will require seamless integration into the provider’s workflow. This protocol can create an effective interface for shared decision-making and safe resource reduction at the bedside in the austere and dynamic clinical environment of the ED and is generalizable for these purposes in other clinical environments as well.

The Pediatric Emergency Care Applied Research Network: a history of multicenter collaboration in the United States

In this article, we review the history and progress of a large multicenter research network pertaining to emergency medical services for children. We describe the history, organization, infrastructure, and research agenda of the Pediatric Emergency Care Applied Research Network and highlight some of the important accomplishments since its inception. We also describe the network's strategy to grow its research portfolio, train new investigators, and study how to translate new evidence into practice. This strategy ensures not only the sustainability of the network in the future but the growth of research in emergency medical services for children in general.

The Pediatric Emergency Care Applied Research Network: a history of multicenter collaboration in the United States

In this article, we review the history and progress of a large multicenter research network pertaining to emergency medical services for children. We describe the history, organization, infrastructure, and research agenda of the Pediatric Emergency Care Applied Research Network (PECARN), and highlight some of the important accomplishments since its inception. We also describe the network’s strategy to grow its research portfolio, train new investigators, and study how to translate new evidence into practice. This strategy ensures not only the sustainability of the network in the future, but the growth of research in emergency medical services for children in general.

Cognitive informatics in biomedicine and healthcare

Cognitive Informatics (CI) is a burgeoning interdisciplinary domain comprising of the cognitive and information sciences that focuses on human information processing, mechanisms and processes within the context of computing and computer applications. Based on a review of articles published in the Journal of Biomedical Informatics (JBI) between January 2001 and March 2014, we identified 57 articles that focused on topics related to cognitive informatics. We found that while the acceptance of CI into the mainstream informatics research literature is relatively recent, its impact has been significant - from characterizing the limits of clinician problem-solving and reasoning behavior, to describing coordination and communication patterns of distributed clinical teams, to developing sustainable and cognitively-plausible interventions for supporting clinician activities. Additionally, we found that most research contributions fell under the topics of decision-making, usability and distributed team activities with a focus on studying behavioral and cognitive aspects of clinical personnel, as they performed their activities or interacted with health information systems. We summarize our findings within the context of the current areas of CI research, future research directions and current and future challenges for CI researchers.