Clinical Decision Support Principles for Quality Improvement and Research

Pediatric hospitalists frequently interact with clinical decision support (CDS) tools in patient care and use these tools for quality improvement or research. In this method/ology paper, we provide an introduction and practical approach to developing and evaluating CDS tools within the electronic health record. First, we define CDS and describe the types of CDS interventions that exist. We then outline a stepwise approach to CDS development, which begins with defining the problem and understanding the system. We present a framework for metric development and then describe tools that can be used for CDS design (eg, 5 Rights of CDS, "10 commandments," usability heuristics, human-centered design) and testing (eg, validation, simulation, usability testing). We review approaches to evaluating CDS tools, which range from randomized studies to traditional quality improvement methods. Lastly, we discuss practical considerations for implementing CDS, including the assessment of a project team's skills and an organization's information technology resources.

Between-hospital variation in clinical decision support availability for common inpatient pediatric conditions: Results of a national Pediatric Research in Inpatient Settings (PRĪS) Network survey

Implementing pediatric-focused clinical decision support (CDS) into hospital electronic health records can lead to improvements in patient care and accelerate quality improvement and research initiatives. However, its design, development, and implementation can be a time-consuming and costly endeavor that may not be feasible for all hospital settings. In this cross-sectional study, we surveyed Pediatric Research in Inpatient Settings (PRĪS) Network hospitals about the availability of CDS tools to gain an understanding of the functionality available across 8 common inpatient pediatric diagnoses. Among the conditions, asthma had the most extensive CDS availability, while mood disorders had the least. Overall, freestanding children's hospitals had the greatest breadth in CDS coverage across conditions and depth in CDS types within conditions. Future initiatives should examine the relationship between CDS availability and clinical outcomes as well as its relationship with hospitals' performance executing multicenter informatics projects, quality improvement collaboratives, and implementation science strategies.

Clinical decision support with a comprehensive in-EHR patient tracking system improves genetic testing follow up

OBJECTIVE

We sought to develop and evaluate an electronic health record (EHR) genetic testing tracking system to address the barriers and limitations of existing spreadsheet-based workarounds.

MATERIALS AND METHODS

We evaluated the spreadsheet-based system using mixed effects logistic regression to identify factors associated with delayed follow up. These factors informed the design of an EHR-integrated genetic testing tracking system. After deployment, we assessed the system in 2 ways. We analyzed EHR access logs and note data to assess patient outcomes and performed semistructured interviews with users to identify impact of the system on work.

RESULTS

We found that patient-reported race was a significant predictor of documented genetic testing follow up, indicating a possible inequity in care. We implemented a CDS system including a patient data capture form and management dashboard to facilitate important care tasks. The system significantly sped review of results and significantly increased documentation of follow-up recommendations. Interviews with key system users identified a range of sociotechnical factors (ie, tools, tasks, collaboration) that contribute to safer and more efficient care.

DISCUSSION

Our new tracking system ended decades of workarounds for identifying and communicating test results and improved clinical workflows. Interview participants related that the system decreased cognitive and time burden which allowed them to focus on direct patient interaction.

CONCLUSION

By assembling a multidisciplinary team, we designed a novel patient tracking system that improves genetic testing follow up. Similar approaches may be effective in other clinical settings.

Opportunities to improve diagnosis in emergency transfers to the pediatric intensive care unit

BACKGROUND

Late recognition of in-hospital deterioration is a source of preventable harm. Emergency transfers (ET), when hospitalized patients require intensive care unit (ICU) interventions within 1 h of ICU transfer, are a proximal measure of late recognition associated with increased mortality and length of stay (LOS).

OBJECTIVE

To apply diagnostic process improvement frameworks to identify missed opportunities for improvement in diagnosis (MOID) in ETs and evaluate their association with outcomes.

DESIGN, SETTINGS, AND PARTICIPANTS

A single-center retrospective cohort study of ETs, January 2015 to June 2019. ET criteria include intubation, vasopressor initiation, or ≥ $\ge \phantom{\rule{}{0ex}}$ 60 mL/kg fluid resuscitation 1 h before to 1 h after ICU transfer. The primary exposure was the presence of MOID, determined using SaferDx. Cases were screened by an ICU and non-ICU physician. Final determinations were made by an interdisciplinary group. Diagnostic process improvement opportunities were identified.

MAIN OUTCOME AND MEASURES

Primary outcomes were in-hospital mortality and posttransfer LOS, analyzed by multivariable regression adjusting for age, service, deterioration category, and pretransfer LOS.

RESULTS

MOID was identified in 37 of 129 ETs (29%, 95% confidence interval [CI] 21%-37%). Cases with MOID differed in originating service, but not demographically. Recognizing the urgency of an identified condition was the most common diagnostic process opportunity. ET cases with MOID had higher odds of mortality (odds ratio 5.5; 95% CI 1.5-20.6; p = .01) and longer posttransfer LOS (rate ratio 1.7; 95% CI 1.1-2.6; p = .02).

CONCLUSION

MOID are common in ETs and are associated with increased mortality risk and posttransfer LOS. Diagnostic improvement strategies should be leveraged to support earlier recognition of clinical deterioration.

Clinician Perspectives on Specifications for Metrics to Inform Pediatric Alarm Management

Background: Ongoing management of monitor alarms is important for reducing alarm fatigue among clinicians (e.g., nurses, physicians). Strategies to enhance clinician engagement in active alarm management in pediatric acute care have not been well explored. Access to alarm summary metrics may enhance clinician engagement. Objective: To lay the foundation for intervention development, we sought to identify functional specifications for formulating, packaging, and delivering alarm metrics to clinicians. Methods: Our team of clinician scientists and human factors engineers conducted focus groups with clinicians from medical-surgical inpatient units in a children's hospital. We inductively coded transcripts, developed codes into themes, and grouped themes into "current state" and "future state." Results: We conducted five focus groups with 13 clinicians (eight registered nurses and five doctors of medicine). In the current state, information exchanged among team members about alarm burden is initiated by nurses on an ad hoc basis. For a future state, clinicians identified ways in which alarm metrics could help them manage alarms and described specific information, such as alarm trends, benchmarks, and contextual data, that would support decision-making. Conclusion: We developed four recommendations for future strategies to enhance clinicians' active management of patient alarms: (1) formulate alarm metrics for clinicians by categorizing alarm rates by type and summarizing alarm trends over time, (2) package alarm metrics with contextual patient data to facilitate clinicians' sensemaking, (3) deliver alarm metrics in a forum that facilitates interprofessional discussion, and (4) provide clinician education to establish a shared mental model about alarm fatigue and evidence-based alarm-reduction strategies.

Classification of Health Information Technology Safety Events in a Pediatric Tertiary Care Hospital

OBJECTIVE

State agencies have developed reporting systems of safety events that include events related to health information technology (HIT). These data come from hospital reporting systems where staff submit safety reports and nurses, in the role of safety managers, review, and code events. Safety managers may have varying degrees of experience with identifying events related to HIT. Our objective was to review events potentially involving HIT and compare those with what was reported to the state.

METHODS

We performed a structured review of 1 year of safety events from an academic pediatric healthcare system. We reviewed the free-text description of each event and applied a classification scheme derived from the AHRQ Health IT Hazard Manager and compared the results with events reported to the state as involving HIT.

RESULTS

Of 33,218 safety events for a 1-year period, 1247 included key words related to HIT and/or were indicated by safety managers as involving HIT. Of the 1247 events, the structured review identified 769 as involving HIT. In comparison, safety managers only identified 194 of the 769 events (25%) as involving HIT. Most events, 353 (46%), not identified by safety managers were documentation issues. Of the 1247 events, the structured review identified 478 as not involving HIT while safety managers identified and reported 81 of these 478 events (17%) as involving HIT.

CONCLUSIONS

The current process of reporting safety events lacks standardization in identifying health technology contributions to safety events, which may minimize the effectiveness of safety initiatives.

Returning integrated genomic risk and clinical recommendations: The eMERGE study

PURPOSE

Assessing the risk of common, complex diseases requires consideration of clinical risk factors as well as monogenic and polygenic risks, which in turn may be reflected in family history. Returning risks to individuals and providers may influence preventive care or use of prophylactic therapies for those individuals at high genetic risk.

METHODS

To enable integrated genetic risk assessment, the eMERGE (electronic MEdical Records and GEnomics) network is enrolling 25,000 diverse individuals in a prospective cohort study across 10 sites. The network developed methods to return cross-ancestry polygenic risk scores, monogenic risks, family history, and clinical risk assessments via a genome-informed risk assessment (GIRA) report and will assess uptake of care recommendations after return of results.

RESULTS

GIRAs include summary care recommendations for 11 conditions, education pages, and clinical laboratory reports. The return of high-risk GIRA to individuals and providers includes guidelines for care and lifestyle recommendations. Assembling the GIRA required infrastructure and workflows for ingesting and presenting content from multiple sources. Recruitment began in February 2022.

CONCLUSION

Return of a novel report for communicating monogenic, polygenic, and family history-based risk factors will inform the benefits of integrated genetic risk assessment for routine health care.

Measuring Training Disruptions Using an Informatics Based Tool

OBJECTIVE

Training disruptions, such as planned curricular adjustments or unplanned global pandemics, impact residency training in ways that are difficult to quantify. Informatics-based medical education tools can help measure these impacts. We tested the ability of a software platform driven by electronic health record data to quantify anticipated changes in trainee clinical experiences during the COVID-19 pandemic.

METHODS

We previously developed and validated the Trainee Individualized Learning System (TRAILS) to identify pediatric resident clinical experiences (i.e. shifts, resident provider-patient interactions (rPPIs), and diagnoses). We used TRAILS to perform a year-over-year analysis comparing pediatrics residents at a large academic children's hospital during March 15-June 15 in 2018 (Control #1), 2019 (Control #2), and 2020 (Exposure).

RESULTS

Residents in the exposure cohort had fewer shifts than those in both control cohorts (P < .05). rPPIs decreased an average of 43% across all PGY levels, with interns experiencing a 78% decrease in Continuity Clinic. Patient continuity decreased from 23% to 11%. rPPIs with common clinic and emergency department diagnoses decreased substantially during the exposure period.

CONCLUSIONS

Informatics tools like TRAILS may help program directors understand the impact of training disruptions on resident clinical experiences and target interventions to learners' needs and development.

Sustainable deimplementation of continuous pulse oximetry monitoring in children hospitalized with bronchiolitis: study protocol for the Eliminating Monitor Overuse (EMO) type III effectiveness-deimplementation cluster-randomized trial

Background

Methods of sustaining the deimplementation of overused medical practices (i.e., practices not supported by evidence) are understudied. In pediatric hospital medicine, continuous pulse oximetry monitoring of children with the common viral respiratory illness bronchiolitis is recommended only under specific circumstances. Three national guidelines discourage its use for children who are not receiving supplemental oxygen, but guideline-discordant practice (i.e., overuse) remains prevalent. A 6-hospital pilot of educational outreach with audit and feedback resulted in immediate reductions in overuse; however, the best strategies to optimize sustainment of deimplementation success are unknown.

Methods

The Eliminating Monitor Overuse (EMO) trial will compare two deimplementation strategies in a hybrid type III effectiveness-deimplementation trial. This longitudinal cluster-randomized design will be conducted in Pediatric Research in Inpatient Settings (PRIS) Network hospitals and will include baseline measurement, active deimplementation, and sustainment phases. After a baseline measurement period, 16–19 hospitals will be randomized to a deimplementation strategy that targets unlearning (educational outreach with audit and feedback), and the other 16–19 will be randomized to a strategy that targets unlearning and substitution (adding an EHR-integrated clinical pathway decision support tool). The primary outcome is the sustainment of deimplementation in bronchiolitis patients who are not receiving any supplemental oxygen, analyzed as a longitudinal difference-in-differences comparison of overuse rates across study arms. Secondary outcomes include equity of deimplementation and the fidelity to, and cost of, each deimplementation strategy. To understand how the deimplementation strategies work, we will test hypothesized mechanisms of routinization (clinicians developing new routines supporting practice change) and institutionalization (embedding of practice change into existing organizational systems).

Discussion

The EMO trial will advance the science of deimplementation by providing new insights into the processes, mechanisms, costs, and likelihood of sustained practice change using rigorously designed deimplementation strategies. The trial will also advance care for a high-incidence, costly pediatric lung disease.

Trial registration

ClinicalTrials.gov,NCT05132322. Registered on November 10, 2021.

Supplementary Information

The online version contains supplementary material available at 10.1186/s13012-022-01246-z.

Diagnostic Reasoning of Resident Physicians in the Age of Clinical Pathways

BACKGROUND

Development of skills in diagnostic reasoning is paramount to the transition from novice to expert clinicians. Efforts to standardize approaches to diagnosis and treatment using clinical pathways are increasingly common. The effects of implementing pathways into systems of care during diagnostic education and practice among pediatric residents are not well described.

OBJECTIVE

To characterize pediatric residents' perceptions of the tradeoffs between clinical pathway use and diagnostic reasoning.

METHODS

We conducted a qualitative study from May to December 2019. Senior pediatric residents from a high-volume general pediatric inpatient service at an academic hospital participated in semi-structured interviews. We utilized a basic interpretive qualitative approach informed by a dual process diagnostic reasoning framework.

RESULTS

Nine residents recruited via email were interviewed. Residents reported using pathways when admitting patients and during teaching rounds. All residents described using pathways primarily as management tools for patients with a predetermined diagnosis, rather than as aids in formulating a diagnosis. As such, pathways primed residents to circumvent crucial steps of deliberate diagnostic reasoning. However, residents relied on bedside assessment to identify when patients are "not quite fitting the mold" of the current pathway diagnosis, facilitating recalibration of the diagnostic process.

CONCLUSIONS

This study identifies important educational implications at the intersection of residents' cognitive diagnostic processes and use of clinical pathways. We highlight potential challenges clinical pathways pose for skill development in diagnostic reasoning by pediatric residents. We suggest opportunities for educators to leverage clinical pathways as a framework for development of these skills.

Characteristics of Emergency Room and Hospital Encounters Resulting From Consumer Home Monitors

BACKGROUND AND OBJECTIVES

Consumer home monitors (CHM), which measure vital signs, are popular products marketed to detect airway obstruction and arrhythmia. Yet, they lack evidence of infant death prevention, demonstrate suboptimal accuracy, and may result in false alarms that prompt unnecessary acute care visits. To better understand the hospital utilization and costs of CHM, we characterized emergency department (ED) and hospital encounters associated with CHM use at a children's hospital.

METHODS

We used structured query language to search the free text of all ED and admission notes between January 2013 and December 2019 to identify clinical documentation discussing CHM use. Two physicians independently reviewed the presence of CHM use and categorized encounter characteristics.

RESULTS

Evidence of CHM use contributed to the presentation of 36 encounters in a sample of over 300 000 encounters, with nearly half occurring in 2019. The leading discharge diagnoses were viral infection (13, 36%), gastroesophageal reflux (8, 22%) and false positive alarm (6, 17%). Median encounter duration was 20 hours (interquartile range: 3 hours to 2 days; max 10.5 days) and median cost of encounters was $2188 (interquartile range: $255 to $7632; max $84 928).

CONCLUSIONS

Although the annual rate of CHM-related encounters was low and did not indicate a major public health burden, for individual families who present to the ED or hospital for concerns related to CHMs, there may be important adverse financial and emotional consequences.

Using Data-Driven Machine Learning to Predict Unplanned ICU Transfers with Critical Deterioration from Electronic Health Records

OBJECTIVE

We aimed to develop a data-driven machine learning model for predicting critical deterioration events from routinely collected EHR data in hospitalized children.

MATERIALS

This retrospective cohort study included all pediatric inpatients hospitalized on a medical or surgical ward between 2014-2018 at a quaternary children's hospital.

METHODS

We developed a large data-driven approach and evaluated three machine learning models to predict pediatric critical deterioration events. We evaluated the models using a nested, stratified 10-fold cross-validation. The evaluation metrics included C-statistic, sensitivity, and positive predictive value. We also compared the machine learning models with patients identified as high-risk Watchers by bedside clinicians.

RESULTS

The study included 57,233 inpatient admissions from 34,976 unique patients. 3,943 variables were identified from the EHR data. The XGBoost model performed best (C-statistic=0.951, CI: 0.946 ∼ 0.956).

CONCLUSIONS

Our data-driven machine learning models accurately predicted patient deterioration. Future sociotechnical analysis will inform deployment within the clinical setting.

Clinical Decision Support Stewardship: Best Practices and Techniques to Monitor and Improve Interruptive Alerts

Interruptive clinical decision support systems, both within and outside of electronic health records, are a resource that should be used sparingly and monitored closely. Excessive use of interruptive alerting can quickly lead to alert fatigue and decreased effectiveness and ignoring of alerts. In this review, we discuss the evidence for effective alert stewardship as well as practices and methods we have found useful to assess interruptive alert burden, reduce excessive firings, optimize alert effectiveness, and establish quality governance at our institutions. We also discuss the importance of a holistic view of the alerting ecosystem beyond the electronic health record.

Leveraging EHR Data to Evaluate the Association of Late Recognition of Deterioration With Outcomes

OBJECTIVES

Emergency transfers (ETs), deterioration events with late recognition requiring ICU interventions within 1 hour of transfer, are associated with adverse outcomes. We leveraged electronic health record (EHR) data to assess the association between ETs and outcomes. We also evaluated the association between intervention timing (urgency) and outcomes.

METHODS

We conducted a propensity-score-matched study of hospitalized children requiring ICU transfer between 2015 and 2019 at a single institution. The primary exposure was ET, automatically classified using Epic Clarity Data stored in our enterprise data warehouse endotracheal tube in lines/drains/airway flowsheet, vasopressor in medication administration record, and/or ≥60 ml/kg intravenous fluids in intake/output flowsheets recorded within 1 hour of transfer. Urgent intervention was defined as interventions within 12 hours of transfer.

RESULTS

Of 2037 index transfers, 129 (6.3%) met ET criteria. In the propensity-score-matched cohort (127 ET, 374 matched controls), ET was associated with higher in-hospital mortality (13% vs 6.1%; odds ratio, 2.47; 95% confidence interval [95% CI], 1.24-4.9, P = .01), longer ICU length of stay (subdistribution hazard ratio of ICU discharge 0.74; 95% CI, 0.61-0.91, P < .01), and longer posttransfer length of stay (SHR of hospital discharge 0.71; 95% CI, 0.56-0.90, P < .01). Increased intervention urgency was associated with increased mortality risk: 4.1% no intervention, 6.4% urgent intervention, and 10% emergent intervention.

CONCLUSIONS

An EHR measure of deterioration with late recognition is associated with increased mortality and length of stay. Mortality risk increased with intervention urgency. Leveraging EHR automation facilitates generalizability, multicenter collaboratives, and metric consistency.

The Alarm Burden of Excess Continuous Pulse Oximetry Monitoring Among Patients With Bronchiolitis

Guidelines discourage continuous pulse oximetry monitoring of hospitalized infants with bronchiolitis who are not receiving supplemental oxygen. Excess monitoring is theorized to contribute to increased alarm burden, but this burden has not been quantified. We evaluated admissions of 201 children (aged 0-24 months) with bronchiolitis. We categorized time ≥60 minutes following discontinuation of supplemental oxygen as "continuously monitored (guideline-discordant)," "intermittently measured (guideline-concordant)," or "unable to classify." Across 4402 classifiable hours, 77% (11,101) of alarms occurred during periods of guideline-discordant monitoring. Patients experienced a median of 35 alarms (interquartile range [IQR], 10-81) during guideline-discordant, continuously monitored time, representing a rate of 6.7 alarms per hour (IQR, 2.1-12.3). In comparison, the median hourly alarm rate during periods of guideline-concordant intermittent measurement was 0.5 alarms per hour (IQR, 0.1-0.8). Reducing guideline-discordant monitoring in bronchiolitis patients would reduce nurse alarm burden.

Alert burden in pediatric hospitals: a cross-sectional analysis of six academic pediatric health systems using novel metrics

BACKGROUND

Excessive electronic health record (EHR) alerts reduce the salience of actionable alerts. Little is known about the frequency of interruptive alerts across health systems and how the choice of metric affects which users appear to have the highest alert burden.

OBJECTIVE

(1) Analyze alert burden by alert type, care setting, provider type, and individual provider across 6 pediatric health systems. (2) Compare alert burden using different metrics.

MATERIALS AND METHODS

We analyzed interruptive alert firings logged in EHR databases at 6 pediatric health systems from 2016-2019 using 4 metrics: (1) alerts per patient encounter, (2) alerts per inpatient-day, (3) alerts per 100 orders, and (4) alerts per unique clinician days (calendar days with at least 1 EHR log in the system). We assessed intra- and interinstitutional variation and how alert burden rankings differed based on the chosen metric.

RESULTS

Alert burden varied widely across institutions, ranging from 0.06 to 0.76 firings per encounter, 0.22 to 1.06 firings per inpatient-day, 0.98 to 17.42 per 100 orders, and 0.08 to 3.34 firings per clinician day logged in the EHR. Custom alerts accounted for the greatest burden at all 6 sites. The rank order of institutions by alert burden was similar regardless of which alert burden metric was chosen. Within institutions, the alert burden metric choice substantially affected which provider types and care settings appeared to experience the highest alert burden.

CONCLUSION

Estimates of the clinical areas with highest alert burden varied substantially by institution and based on the metric used.

EHR-Integrated Monitor Data to Measure Pulse Oximetry Use in Bronchiolitis

BACKGROUND AND OBJECTIVES

Continuous pulse oximetry (oxygen saturation [Spo₂]) monitoring in hospitalized children with bronchiolitis not requiring supplemental oxygen is discouraged by national guidelines, but determining monitoring status accurately requires in-person observation. Our objective was to determine if electronic health record (EHR) data can accurately estimate the extent of actual Spo₂ monitoring use in bronchiolitis.

METHODS

This repeated cross-sectional study included infants aged 8 weeks through 23 months hospitalized with bronchiolitis. In the validation phase at 3 children's hospitals, we calculated the test characteristics of the Spo₂ monitor data streamed into the EHR each minute when monitoring was active compared with in-person observation of Spo₂ monitoring use. In the application phase at 1 children's hospital, we identified periods when supplemental oxygen was administered using EHR flowsheet documentation and calculated the duration of Spo₂ monitoring that occurred in the absence of supplemental oxygen.

RESULTS

Among 668 infants at 3 hospitals (validation phase), EHR-integrated Spo₂ data from the same minute as in-person observation had a sensitivity of 90%, specificity of 98%, positive predictive value of 88%, and negative predictive value of 98% for actual Spo₂ monitoring use. Using EHR-integrated data in a sample of 317 infants at 1 hospital (application phase), infants were monitored in the absence of oxygen supplementation for a median 4.1 hours (interquartile range 1.4-9.4 hours). Those who received supplemental oxygen experienced a median 5.6 hours (interquartile range 3.0-10.6 hours) of monitoring after oxygen was stopped.

CONCLUSIONS

EHR-integrated monitor data are a valid measure of actual Spo₂ monitoring use that may help hospitals more efficiently identify opportunities to deimplement guideline-inconsistent use.

Physiologic Monitor Alarm Burden and Nurses' Subjective Workload in a Children's Hospital

BACKGROUND AND OBJECTIVES

Physiologic monitor alarms occur at high rates in children's hospitals; ≤1% are actionable. The burden of alarms has implications for patient safety and is challenging to measure directly. Nurse workload, measured by using a version of the National Aeronautics and Space Administration Task Load Index (NASA-TLX) validated among nurses, is a useful indicator of work burden that has been associated with patient outcomes. A recent study revealed that 5-point increases in the NASA-TLX score were associated with a 22% increased risk in missed nursing care. Our objective was to measure the relationship between alarm count and nurse workload by using the NASA-TLX.

METHODS

We conducted a repeated cross-sectional study of pediatric nurses in a tertiary care children's hospital to measure the association between NASA-TLX workload evaluations (using the nurse-validated scale) and alarm count in the 2 hours preceding NASA-TLX administration. Using a multivariable mixed-effects regression accounting for nurse-level clustering, we modeled the adjusted association of alarm count with workload.

RESULTS

The NASA-TLX score was assessed in 26 nurses during 394 nursing shifts over a 2-month period. In adjusted regression models, experiencing >40 alarms in the preceding 2 hours was associated with a 5.5 point increase (95% confidence interval 5.2 to 5.7; P < .001) in subjective workload.

CONCLUSION

Alarm count in the preceding 2 hours is associated with a significant increase in subjective nurse workload that exceeds the threshold associated with increased risk of missed nursing care and potential patient harm.

A maximum likelihood approach to electronic health record phenotyping using positive and unlabeled patients

OBJECTIVE

Phenotyping patients using electronic health record (EHR) data conventionally requires labeled cases and controls. Assigning labels requires manual medical chart review and therefore is labor intensive. For some phenotypes, identifying gold-standard controls is prohibitive. We developed an accurate EHR phenotyping approach that does not require labeled controls.

MATERIALS AND METHODS

Our framework relies on a random subset of cases, which can be specified using an anchor variable that has excellent positive predictive value and sensitivity independent of predictors. We proposed a maximum likelihood approach that efficiently leverages data from the specified cases and unlabeled patients to develop logistic regression phenotyping models, and compare model performance with existing algorithms.

RESULTS

Our method outperformed the existing algorithms on predictive accuracy in Monte Carlo simulation studies, application to identify hypertension patients with hypokalemia requiring oral supplementation using a simulated anchor, and application to identify primary aldosteronism patients using real-world cases and anchor variables. Our method additionally generated consistent estimates of 2 important parameters, phenotype prevalence and the proportion of true cases that are labeled.

DISCUSSION

Upon identification of an anchor variable that is scalable and transferable to different practices, our approach should facilitate development of scalable, transferable, and practice-specific phenotyping models.

CONCLUSIONS

Our proposed approach enables accurate semiautomated EHR phenotyping with minimal manual labeling and therefore should greatly facilitate EHR clinical decision support and research.

Progress of Self Help Group-Bank Linkage Programme in India

In this paper an attempt has been made to analyse the progress of SHG-Bank linkage programme in India during the period between 2007-2008 and 2019-2020. The progress of SHG-Bank linkage programme has been analyzed in terms growth of savings of SHGs with banks and growth of bank loans disbursed to SHGs. For this purpose the data required for the study were collected from the official publication of National Bank for Agricultural and Rural Development (NABARD), different published reports, journals and existing available literature. This study employed simple statistical tools such as percentage analysis and averages to analyze the data. The result of the study shows that there is significant raise in the amount of savings of SHGs with banking sector and amount of loans disbursed to SHGs, During this study period. However the agency-wise analyses of savings of SHGs and loans disbursed to SHGs show that the Commercial banks lead in getting savings of SHGs and loans disbursed to them followed by Regional Rural Banks and Co-operative banks. Not with standing the remarkable progress, geographically there has been skewed development of SHG-Bank linkage programme in India. There is wide regional disparity in the spread of SHGs, savings of SHGs with banks and loans disbursed to SHGs under this programme. The outreach of this programme is spectacular in Southern region while North, West and Eastern regions are lagging behind. In view of the large outreach, predominant position and the possible benefits to the poor, it is very important to see the benefits of this programme to reach across all sections of the society and regions. So far the SHG movement is India is mostly South-Centric and it is yet to take off the real sense in other regions of India.

Analysis: Protocol for a New Method to Measure Physiologic Monitor Alarm Responsiveness

Evaluating the clinical impacts of healthcare alarm management systems plays a critical role in assessing newly implemented monitoring technology, exposing latent threats to patient safety, and identifying opportunities for system improvement. We describe a novel, accurate, rapidly implementable, and readily reproducible in situ simulation approach to measure alarm response times and rates without the challenges and expense of video analysis. An interprofessional team consisting of biomedical engineers, human factors engineers, information technology specialists, nurses, physicians, facilitators from the hospital's simulation center, clinical informaticians, and hospital administrative leadership worked with three units at a pediatric hospital to design and conduct the simulations. Existing hospital technology was used to transmit a simulated, unambiguously critical alarm that appeared to originate from an actual patient to the nurse's mobile device, and discreet observers measured responses. Simulation observational data can be used to design and evaluate quality improvement efforts to address alarm responsiveness and to benchmark performance of different alarm communication systems.

Rapid Manufacturable Ventilator for Respiratory Emergencies of COVID-19 Disease

Influenza like pandemics are a severe threat to any established health care system as many thousands of patients would need emergency ventilator support during the acute respiratory failure stage, and this quickly overloads the existing facilities. The present article addresses the design and development of a human breathing assist machine (ventilator) prototype for use by qualified medical professionals in the emergency room, as well as in other locations, where a regular ventilator machine cannot be made available. The ventilator has been designed using readily available locally sourced materials, which can be assembled in a short time. This ensures the minimum required features to ventilate a patient in emergency conditions. The popular crank-rocker mechanism has been used to meet some of the vital design requirements of the emergency ventilator. The size of the links has been chosen to maintain a fixed inspiratory-to-expiratory (I:E) time ratio of 1:2. The kinematic linkage design has been kept modular by introducing a feature to adjust the location of the rocker tip to control the tidal volume from 100 ml to 600 ml of oxygenated air per breath. A virtual CAD model, based on the above-mentioned linkage design, has been designed to assess the variation of the position and velocity with time. Finally, a working prototype has been made, and it was observed that the I:E time ratio of 1:2 was achieved satisfactorily.

Performance of a Clinical Decision Support Tool to Identify PICU Patients at High Risk for Clinical Deterioration

OBJECTIVES

To evaluate the translation of a paper high-risk checklist for PICU patients at risk of clinical deterioration to an automated clinical decision support tool.

DESIGN

Retrospective, observational cohort study of an automated clinical decision support tool, the PICU Warning Tool, adapted from a paper checklist to predict clinical deterioration events in PICU patients within 24 hours.

SETTING

Two quaternary care medical-surgical PICUs-The Children's Hospital of Philadelphia and Cincinnati Children's Hospital Medical Center.

PATIENTS

The study included all patients admitted from July 1, 2014, to June 30, 2015, the year prior to the initiation of any focused situational awareness work at either institution.

INTERVENTIONS

We replicated the predictions of the real-time PICU Warning Tool by retrospectively querying the institutional data warehouse to identify all patients that would have flagged as high-risk by the PICU Warning Tool for their index deterioration.

MEASUREMENTS AND MAIN RESULTS

The primary exposure of interest was determination of high-risk status during PICU admission via the PICU Warning Tool. The primary outcome of interest was clinical deterioration event within 24 hours of a positive screen. The date and time of the deterioration event was used as the index time point. We evaluated the sensitivity, specificity, positive predictive value, and negative predictive value of the performance of the PICU Warning Tool. There were 6,233 patients evaluated with 233 clinical deterioration events experienced by 154 individual patients. The positive predictive value of the PICU Warning Tool was 7.1% with a number needed to screen of 14 patients for each index clinical deterioration event. The most predictive of the individual criteria were elevated lactic acidosis, high mean airway pressure, and profound acidosis.

CONCLUSIONS

Performance of a clinical decision support translation of a paper-based tool showed inferior test characteristics. Improved feasibility of identification of high-risk patients using automated tools must be balanced with performance.

Reducing Albuterol Use in Children With Bronchiolitis

OBJECTIVES

In 2014, the American Academy of Pediatrics published bronchiolitis guidelines recommending against the use of bronchodilators. For the winter of 2015 to 2016, we aimed to reduce the proportion of emergency department patients with bronchiolitis receiving albuterol from 43% (previous winter rate) to <35% and from 18% (previous winter rate) to <10% in the inpatient setting.

METHODS

A team identified key drivers of albuterol use and potential interventions. We implemented changes to our pathway and the associated order set recommending against routine albuterol use and designed education to accompany the pathway changes. We monitored albuterol use through weekly automated data extraction and reported results back to clinicians. We measured admission rate, length of stay, and revisit rate as balancing measures for the intervention.

RESULTS

The study period included 3834 emergency department visits and 1119 inpatient hospitalizations. In the emergency department, albuterol use in children with bronchiolitis declined from 43% to 20% and was <3 SD control limits established in the previous year, meeting statistical thresholds for special cause variation. Inpatient albuterol use decreased from 18% to 11% of patients, also achieving special cause variation and approaching our goal. The changes in both departments were sustained through the entire bronchiolitis season, and admission rate, length of stay, and revisit rates remained unchanged.

CONCLUSIONS

Using a multidisciplinary group that redesigned a clinical pathway and order sets for bronchiolitis, we substantially reduced albuterol use at a large children's hospital without impacting other outcome measures.

Economic Impact Of India-China Trade War: Future Directions

The growing trade war among India and China is creating imbalance in the among developing countries. Both countries are affecting in their business prospects. India is basically importing raw material for export of good and services. Growing tension leads to unnecessary growth impetus which affects industry growth, loss of employment opportunities and other trade related problems. India has considerable potential for reducing its trade deficit with China, as we can see from Made-in-China products sold on the Indian market. Most of them are low- and mid-range products. India can make these things itself. The value tune to the cores of rupees is loss for the both counties; it will create far reaching impact in Indian business environment. These papers highlight the possible causes and consequences of trade war between to Asian giants and suggest how to promote regional growth prospects for speedy development of economics.

Growing Dynamisms' Of Business And Technology/ Global Perspective

Technology is the essence of change in business and society. In this paper we can learn about technology, its characteristics, its historical phases and how innovation is fuelled today. Technological advances in the modern world have created new business opportunities. Leverage advanced tools to rise above the competition. Technology has revolutionized the way companies conduct business by enabling small businesses to level the playing field with larger organizations. Businesses nowadays, whether it is big or small, also rely on the help of technology. Most companies depends their daily operations on the use of technological innovation such as computers, internet connection, printers, applications, and file storages. That is why; every business can develop positively from small- scale to large-scale. Small businesses use an array of tech – everything from servers to mobile devices – to develop competitive advantages in the economic marketplace. Technology in business made it possible to have a wider reach in the global market. The basic example is the Internet, which is now a common marketing tool to attract more consumers in availing products and services offered by various businesses. The aim of the paper is to highlight the potential benefits of technology and advancement.

Formative Usability Testing Reduces Severe Blood Product Ordering Errors

BACKGROUND

Medical errors in blood product orders and administration are common, especially for pediatric patients. A failure modes and effects analysis in our health care system indicated high risk from the electronic blood ordering process.

OBJECTIVES

There are two objectives of this study as follows:(1) To describe differences in the design of the original blood product orders and order sets in the system (original design), new orders and order sets designed by expert committee (DEC), and a third-version developed through user-centered design (UCD).(2) To compare the number and type of ordering errors, task completion rates, time on task, and user preferences between the original design and that developed via UCD.

METHODS

A multidisciplinary expert committee proposed adjustments to existing blood product order sets resulting in the DEC order set. When that order set was tested with front-line users, persistent failure modes were detected, so orders and order sets were redesigned again via formative usability testing. Front-line users in their native clinical workspaces were observed ordering blood in realistic simulated scenarios using a think-aloud protocol. Iterative adjustments were made between participants. In summative testing, participants were randomized to use the original design or UCD for five simulated scenarios. We evaluated differences in ordering errors, time on task, and users' design preference with two-sample t-tests.

RESULTS

Formative usability testing with 27 providers from seven specialties led to 18 changes made to the DEC to produce the UCD. In summative testing, error-free task completion for the original design was 36%, which increased to 66% in UCD (30%, 95% confidence interval [CI]: 3.9-57%; p = 0.03). Time on task did not vary significantly.

CONCLUSION

UCD led to substantially different blood product orders and order sets than DEC. Users made fewer errors when ordering blood products for pediatric patients in simulated scenarios when using the UCD orders and order sets compared with the original design.

Towards a Maturity Model for Clinical Decision Support Operations

Clinical decision support (CDS) systems delivered through the electronic health record are an important element of quality and safety initiatives within a health care system. However, managing a large CDS knowledge base can be an overwhelming task for informatics teams. Additionally, it can be difficult for these informatics teams to communicate their goals with external operational stakeholders and define concrete steps for improvement. We aimed to develop a maturity model that describes a roadmap toward organizational functions and processes that help health care systems use CDS more effectively to drive better outcomes. We developed a maturity model for CDS operations through discussions with health care leaders at 80 organizations, iterative model development by four clinical informaticists, and subsequent review with 19 health care organizations. We ceased iterations when feedback from three organizations did not result in any changes to the model. The proposed CDS maturity model includes three main "pillars": "Content Creation," "Analytics and Reporting," and "Governance and Management." Each pillar contains five levels-advancing along each pillar provides CDS teams a deeper understanding of the processes CDS systems are intended to improve. A "roof" represents the CDS functions that become attainable after advancing along each of the pillars. Organizations are not required to advance in order and can develop in one pillar separately from another. However, we hypothesize that optimal deployment of preceding levels and advancing in tandem along the pillars increase the value of organizational investment in higher levels of CDS maturity. In addition to describing the maturity model and its development, we also provide three case studies of health care organizations using the model for self-assessment and determine next steps in CDS development.

Hidden health IT hazards: a qualitative analysis of clinically meaningful documentation discrepancies at transfer out of the pediatric intensive care unit

Objective

The risk of medical errors increases upon transfer out of the intensive care unit (ICU). Discrepancies in the documented care plan between notes at the time of transfer may contribute to communication errors. We sought to determine the frequency of clinically meaningful discrepancies in the documented care plan for patients transferred from the pediatric ICU to the medical wards and identified risk factors.

Materials and Methods

Two physician reviewers independently compared the transfer note and handoff document of 50 randomly selected transfers. Clinically meaningful discrepancies in the care plan between these two documents were identified using a coding procedure adapted from healthcare failure mode and effects analysis. We assessed the influence of risk factors via multivariable regression.

Results

We identified 34 clinically meaningful discrepancies in 50 patient transfers. Fourteen transfers (28%) had ≥1 discrepancy, and ≥2 were present in 7 transfers (14%). The most common discrepancy categories were differences in situational awareness notifications and documented current therapy. Transfers with handoff document length in the top quartile had 10.6 (95% CI: 1.2-90.2) times more predicted discrepancies than transfers with handoff length in the bottom quartile. Patients receiving more medications in the 24 hours prior to transfer had higher discrepancy counts, with each additional medication increasing the predicted number of discrepancies by 17% (95% CI: 6%-29%).

Conclusion

Clinically meaningful discrepancies in the documented care plan pose legitimate safety concerns and are common at the time of transfer out of the ICU among complex patients.

Parent eReferral to Tobacco Quitline: A Pragmatic Randomized Trial in Pediatric Primary Care

INTRODUCTION

Quitlines are effective in helping smokers quit, but pediatrician quitline referral rates are low, and few parents who smoke use the service. This study compared enrollment of parents who smoke in the quitline using electronic referral with that using manual referral.

STUDY DESIGN

The study was designed as a pragmatic RCT.

SETTING/PARTICIPANTS

Participants were recruited from one large, urban pediatric primary care site in Philadelphia, Pennsylvania with a high percentage of low-income families. Participants included adult parents who smoked and were present at their child's healthcare visit.

INTERVENTION

Pediatricians screened for tobacco use; smokers were given brief advice to quit and, if interested in quitting, were referred to the quitline. The eReferral ("warm handoff") involved electronically sending parent information to the quitline (parent received a call within 24-48 hours). Control group procedures were identical to eReferral, except the quitline number was provided to the parent. Data were collected between March 2017 and February 2018 and analyzed in 2018.

MAIN OUTCOME MEASURES

The primary outcome was the proportion of parents enrolled in quitline treatment. Secondary outcomes included parent factors (e.g., demographics, nicotine dependence, and quitting motivation) associated with successful enrollment. Number of quitline contacts was also explored.

RESULTS

During the study period, in the eReferral group, 10.3% (24 of 233) of parents who smoked and were interested in quitting enrolled in the quitline, whereas only 2.0% (5 of 251) of them in the control group enrolled in the quitline-a difference of 8.3% (95% CI=4.0, 12.6). Parents aged ≥50 years enrolled in the quitline more frequently. Although more parents in the eReferral group connected to the quitline, among parents who had at least one quitline contact, there was no significant difference in the mean number of quitline contacts between eReferral and control groups (mean, 2.04 vs 2.40 calls; difference, 0.36 [95% CI=0.35, 1.06]).

CONCLUSIONS

Smoking parent eReferral from pediatric primary care may increase quitline enrollment and could be adopted by practices interested in increasing rates of parent treatment.

TRIAL REGISTRATION

This study is registered at www.clinicaltrials.gov NCT02997735.

Identifying Electronic Health Record Usability And Safety Challenges In Pediatric Settings

Pediatric populations are uniquely vulnerable to the usability and safety challenges of electronic health records (EHRs), particularly those related to medication, yet little is known about the specific issues contributing to hazards. To understand specific usability issues and medication errors in the care of children, we analyzed 9,000 patient safety reports, made in the period 2012-17, from three different health care institutions that were likely related to EHR use. Of the 9,000 reports, 3,243 (36 percent) had a usability issue that contributed to the medication event, and 609 (18.8 percent) of the 3,243 might have resulted in patient harm. The general pattern of usability challenges and medication errors were the same across the three sites. The most common usability challenges were associated with system feedback and the visual display. The most common medication error was improper dosing.

Development and Evaluation of High-Value Pediatrics: A High-Value Care Pediatric Resident Curriculum

Low-value health care is pervasive in the United States, and clinicians need to be trained to be stewards of health care resources. Despite a mandate by the Accreditation Council for Graduate Medical Education to educate trainee physicians on cost awareness, only 10% of pediatric residency programs have a high-value care (HVC) curriculum. To meet this need, we set out to develop and evaluate the impact of High-Value Pediatrics, an open-access HVC curriculum. High-Value Pediatrics is a 3-part curriculum that includes 4 standardized didactics, monthly interactive morning reports, and an embedded HVC improvement project. Curriculum evaluation through an anonymous, voluntary survey revealed an improvement in the self-reported knowledge of health care costs, charges, reimbursement, and value (P < .05). Qualitative results revealed self-reported behavior changes, and HVC improvement projects resulted in higher-value patient care. The implementation of High-Value Pediatrics is feasible and reveals improved knowledge and attitudes about HVC. HVC improvement projects augmented curricular knowledge gains and revealed behavior changes. It is imperative that formal high-value education be taught to every pediatric trainee to lead the culture change that is necessary to turn the tide against low-value health care. In addition, simultaneous work on faculty education and attention to the hidden curriculum of low-value care is needed for sustained and long-term improvements.

User Testing an Information Foraging Tool for Ambulatory Surgical Site Infection Surveillance

BACKGROUND

Surveillance for surgical site infections (SSIs) after ambulatory surgery in children requires a detailed manual chart review to assess criteria defined by the National Health and Safety Network (NHSN). Electronic health records (EHRs) impose an inefficient search process where infection preventionists must manually review every postsurgical encounter (< 30 days). Using text mining and business intelligence software, we developed an information foraging application, the SSI Workbench, to visually present which postsurgical encounters included SSI-related terms and synonyms, antibiotic, and culture orders.

OBJECTIVE

This article compares the Workbench and EHR on four dimensions: (1) effectiveness, (2) efficiency, (3) workload, and (4) usability.

METHODS

Comparative usability test of Workbench and EHR. Objective test metrics are time per case, encounters reviewed per case, time per encounter, and retrieval of information meeting NHSN definitions. Subjective measures are cognitive load using the National Aeronautics and Space Administration (NASA) Task Load Index (NASA TLX), and a questionnaire on system usability and utility.

RESULTS

Eight infection preventionists participated in the test. There was no difference in effectiveness as subjects retrieved information from all cases, using both systems, to meet the NHSN criteria. There was no difference in efficiency in time per case between the Workbench and EHR (8.58 vs. 7.39 minutes, p = 0.36). However, with the Workbench subjects opened fewer encounters per case (3.0 vs. 7.5, p = 0.002), spent more time per encounter (2.23 vs. 0.92 minutes, p = 0.002), rated the Workbench lower in cognitive load (NASA TLX, 24 vs. 33, p = 0.02), and significantly higher in measures of usability.

CONCLUSION

Compared with the EHR, the Workbench was more usable, short, and reduced cognitive load. In overall efficiency, the Workbench did not save time, but demonstrated a shift from between-encounter foraging to within-encounter foraging and was rated as significantly more efficient. Our results suggest that infection surveillance can be better supported by systems applying information foraging theory.

A Heuristic Evaluation to Assess Use of After Visit Summaries for Supporting Continuity of Care

BACKGROUND

Outpatient providers often do not receive discharge summaries from acute care providers prior to follow-up visits. These outpatient providers may use the after-visit summaries (AVS) that are given to patients to obtain clinical information. It is unclear how effectively AVS support care coordination between clinicians.

OBJECTIVES

Goals for this effort include: (1) developing usability heuristics that may be applied both for assessment and to guide generation of medical documents in general, (2) conducting a heuristic evaluation to assess the use of AVS for communication between clinicians, and (3) providing recommendations for generating AVS that effectively support both patient/caregiver use and care coordination.

METHODS

We created a 17-item heuristic evaluation instrument for assessing usability of medical documents. Eight experts used the instrument to assess each of four simulated AVS. The simulations were created using examples from two hospitals and two pediatric patient cases developed by the National Institute of Standards and Technology.

RESULTS

Experts identified 224 unique usability problems ranging in severity from mild to catastrophic. Content issues (e.g., missing medical history, marital status of a 2-year-old) were rated as most severe, but widespread formatting and structural problems (e.g., inconsistent indentation, fonts, and headings; confusing ordering of information) were so distracting that they significantly reduced readers' ability to efficiently use the documents. Overall, issues in the AVS from Hospital 2 were more severe than those in the AVS from Hospital 1.

CONCLUSION

The new instrument allowed for quick, inexpensive evaluations of AVS. Usability issues such as unnecessary information, poor organization, missing information, and inconsistent formatting make it hard for patients, caregivers, and clinicians to use the AVS. The heuristics in the new instrument may be used as guidance to adapt electronic health record systems so that they generate more useful and usable medical documents.

Design and Implementation of a Pediatric ICU Acuity Scoring Tool as Clinical Decision Support

BACKGROUND AND OBJECTIVE

Pediatric in-hospital cardiac arrest most commonly occurs in the pediatric intensive care unit (PICU) and is frequently preceded by early warning signs of clinical deterioration. In this study, we describe the implementation and evaluation of criteria to identify high-risk patients from a paper-based checklist into a clinical decision support (CDS) tool in the electronic health record (EHR).

MATERIALS AND METHODS

The validated paper-based tool was first adapted by PICU clinicians and clinical informaticians and then integrated into clinical workflow following best practices for CDS design. A vendor-based rule engine was utilized. Littenberg's assessment framework helped guide the overall evaluation. Preliminary testing took place in EHR development environments with more rigorous evaluation, testing, and feedback completed in the live production environment. To verify data quality of the CDS rule engine, a retrospective Structured Query Language (SQL) data query was also created. As a process metric, preparedness was measured in pre- and postimplementation surveys.

RESULTS

The system was deployed, evaluating approximately 340 unique patients monthly across 4 clinical teams. The verification against retrospective SQL of 15-minute intervals over a 30-day period revealed no missing triggered intervals and demonstrated 99.3% concordance of positive triggers. Preparedness showed improvements across multiple domains to our a priori goal of 90%.

CONCLUSION

We describe the successful adaptation and implementation of a real-time CDS tool to identify PICU patients at risk of deterioration. Prospective multicenter evaluation of the tool's effectiveness on clinical outcomes is necessary before broader implementation can be recommended.