Development and evaluation of trigger tools to identify pediatric blood management errors

BACKGROUND

Pediatric patient blood management (PBM) programs require continuous surveillance of errors and near misses. However, most PBM programs rely on passive surveillance methods. Our objective was to develop and evaluate a set of automated trigger tools for active surveillance of pediatric PBM errors.

MATERIALS AND METHODS

We used the Rand-UCLA method with an expert panel of pediatric transfusion medicine specialists to identify and prioritize candidate trigger tools for all transfused blood products. We then iteratively developed automated queries of electronic health record (EHR) data for the highest priority triggers. Two physicians manually reviewed a subset of cases meeting trigger tool criteria and estimated each trigger tool's positive predictive value (PPV). We then estimated the rate of PBM errors, whether they reached the patient, and adverse events for each trigger tool across four years in a single pediatric health system.

RESULTS

We identified 28 potential triggers for pediatric PBM errors and developed 5 automated trigger tools (positive patient identification, missing irradiation, unwashed products despite prior anaphylaxis, transfusion lasting >4 hours, over-transfusion by volume). The PPV for ordering errors ranged from 38-100%. The most frequently detected near miss event reaching patients was first transfusions without positive patient identification (estimate 303, 95% CI: 288-318 per year). The only adverse events detected were from over-transfusions by volume, including 4 adverse events detected on manual review that had not been reported in passive surveillance systems.

DISCUSSION

It is feasible to automatically detect pediatric PBM errors using existing data captured in the EHR that enable active surveillance systems. Over-transfusions may be one of the most frequent causes of harm in the pediatric environment.

Associations between Child Opportunity Index and Pediatric Cardiac Surgical Outcomes

OBJECTIVE

To assess the relationship between the Child Opportunity Index (COI), a comprehensive measurement of social determinants of health, and specific COI domains on patient-specific outcomes following congenital cardiac surgery in the metropolitan region of Atlanta, Georgia.

STUDY DESIGN

In this retrospective chart review, we included patients who underwent an index operation for congenital heart disease between 2010 and 2020 in a single pediatric health care system. Patients' addresses were geocoded and mapped to census tracts. Descriptive statistics, univariable analysis, and multivariable regression models were employed to assess associations between variables and outcomes.

RESULTS

Of the 7460 index surgeries, 3798 (51%) met eligibility criteria. Presence of an adverse outcome, defined as either mortality or 1 of several other major postoperative morbidities, was significantly associated with COI in the univariable model (P = .008), but not the multivariable regression model (P = .39). Postoperative hospital length of stay was significantly associated with COI (P < .001) in univariable and multivariable regression models. There was no significant association between COI and readmission within 30 days of hospital discharge in univariable (P < .094) and multivariable (P = .49) models.

CONCLUSION

COI is associated with postoperative hospital length of stay but not all outcomes in patients after congenital heart surgery. By understanding the role of COI in outcomes related to cardiac surgery, targeted interventions can be developed to improve health equity.

Current Inpatient Screening Practices for Sexual History and STIs: An Opportunity to Seize

The American Academy of Pediatrics recommends utilizing hospitalizations as an opportunity to provide sexual health screenings for adolescents. This study aimed to describe the current practice of sexual history documentation (SHxD) and sexually transmitted infection (STI) testing among adolescents admitted to a pediatric hospital medicine service. Retrospective cross-sectional study of adolescents (14-19 years old) admitted to the PHM service from 2017-2019 was performed at an academic children's health system. Patient (demographics, history of complex chronic condition, and insurance), hospitalization (length of stay, diagnosis, STI tests ordered/results), and physician (level of training and gender) characteristics were extracted for each encounter. A natural language processing algorithm identified the presence of SHxD. Univariate analysis and multivariable analysis were performed to detect factors associated with SHxD and STI screening. The prevalence of STIs was calculated for those who were tested. Out of 2242 encounters, SHxD and STI testing rates were 40.9% and 17.2%, respectively. Patient gender, race, lack of complex chronic condition, and resident involvement were predictive of SHxD and STI testing. SHxD increased the odds of STI testing significantly (OR 5.06, CI 3.90-6.58). Among those who were tested, the prevalence of STIs was highest for chlamydia (37/329, 11.2%). Overall, sexual health screening rates remain low in the hospital setting and future improvement initiatives are needed.

Accurate Dosing Weight: When the 10% Really Matters

INTRODUCTION

Children are at increased risk of medication-associated adverse events, often due to weight-based dosing errors. We aimed to reduce the proportion of medications that were administered where the dosing weight was ≥ 10% different from the recorded weight.

METHODS

We adopted in-situ usability testing to iteratively improve design of clinical decision support that would enable accurate dosing weight documentation by prompting clinicians to update weight if recorded weight was > 10% different and it had been at least 7 days since the last dosing weight update.

RESULTS

The proportion of medication administrations with difference >10% between their recorded weight and dosing weight decreased from 13.1% (56,256/ 429,006) in the baseline period to 9.5% (35,560 / 372,443) in the intervention period (P < 0.001).

DISCUSSION AND CONCLUSION

User-centered design of an interruptive alert improved the accuracy of dosing weights during medication administrations without substantial alert burden. In-situ usability testing is an effective approach to rapidly obtain feedback from frontline users and iterate on the design to effect desired behavior changes.

Pharmacist Metrics in the Pediatric Intensive Care Unit: an Exploration of the Medication Regimen Complexity-Intensive Care Unit (MRC-ICU) Score

INTRODUCTION

The medication regimen complexity-intensive care unit (MRC-ICU) score has been developed and validated as an objective predictive metric for patient outcomes and pharmacist workload in the adult critically ill population. The purpose of this study was to explore the MRC-ICU and other workload metrics in the pediatric ICU (PICU).

METHODS

This study was a retrospective cohort of pediatric ICU patients admitted to a single institution -between February 2, 2022 - August 2, 2022. Two scores were calculated, including the MRC-ICU and the pediatric Daily Monitoring System (pDMS). Data were extracted from the electronic health record. The primary outcome was the correlation of the MRC-ICU to mortality, as measured by Pearson -correlation -coefficient. Additionally, the correlation of MRC-ICU to number of orders was evaluated. Secondary -analyses explored the correlation of the MRC-ICU with pDMS and with hospital and ICU length of stay.

RESULTS

A total of 2,232 patients were included comprising 2,405 encounters. The average age was 6.9 years (standard deviation [SD] 6.3 years). The average MRC-ICU score was 3.0 (SD 3.8). For the primary outcome, MRC-ICU was significantly positively correlated to mortality (0.22 95% confidence interval [CI 0.18 - 0.26]), p<0.05. Additionally, MRC-ICU was significantly positively correlated to ICU length of stay (0.38 [CI 0.34 - 0.41]), p<0.05. The correlation between the MRC-ICU and pDMS was (0.72 [CI 0.70 - 0.73]), p<0.05.

CONCLUSION

In this pilot study, MRC-ICU demonstrated an association with existing prioritization metrics and with mortality and length of ICU stay in PICU population. Further, larger scale studies are required.

Effectiveness of a Vendor Predictive Model for the Risk of Pediatric Asthma Exacerbation: A Difference-in-Differences Analysis

BACKGROUND

Asthma is a common cause of morbidity and mortality in children. Predictive models may help providers tailor asthma therapies to an individual's exacerbation risk. The effectiveness of asthma risk scores on provider behavior and pediatric asthma outcomes remains unknown.

OBJECTIVE

Determine the impact of an electronic health record (EHR) vendor-released model on outcomes for children with asthma.

METHODS

The Epic Systems Risk of Pediatric Asthma Exacerbation model was implemented on February 24, 2021, for volunteer pediatric allergy and pulmonology providers as a noninterruptive risk score visible in the patient schedule view. Asthma hospitalizations, emergency department (ED) visits, or oral steroid courses within 90 days of the index visit were compared from February 24, 2019, to February 23, 2022, using a difference-in-differences design with a control group of visits to providers in the same departments. Volunteer providers were interviewed to identify barriers and facilitators to model use.

RESULTS

In the intervention group, asthma hospitalizations within 90 days decreased from 1.4% (54/3,842) to 0.7% (14/2,165) after implementation with no significant change in the control group (0.9% [171/19,865] preimplementation to 1.0% [105/10,743] post). ED visits in the intervention group decreased from 5.8% (222/3,842) to 5.5% (118/2,164) but increased from 5.5% (1,099/19,865) to 6.8% (727/10,743) in the control group. The adjusted difference-in-differences estimators for hospitalization, ED visit, and oral steroid outcomes were -0.9% (95% confidence interval [CI]: -1.6 to -0.3), -2.4% (-3.9 to -0.8), and -1.9% (-4.3 to 0.5). In qualitative analysis, providers understood the purpose of the model and felt it was useful to flag high exacerbation risk. Trust in the model was calibrated against providers' own clinical judgement.

CONCLUSION

This EHR vendor model implementation was associated with a significant decrease in asthma hospitalization and ED visits within 90 days of pediatric allergy and pulmonology clinic visits, but not oral steroid courses.

User-Centered Design and Evaluation of Clinical Decision Support to Improve Early Peanut Introduction: Formative Study

BACKGROUND

Peanut allergy has recently become more prevalent. Peanut introduction recommendations have evolved from suggesting peanut avoidance until the age of 3 years to more recent guidelines encouraging early peanut introduction after the Learning Early about Peanut Allergy (LEAP) study in 2015. Guideline adherence is poor, leading to missed care opportunities.

OBJECTIVE

In this study, we aimed to develop a user-centered clinical decision support (CDS) tool to improve implementation of the most recent early peanut introduction guidelines in the primary care clinic setting.

METHODS

We edited the note template of the well-child check (WCC) visits at ages 4 and 6 months with CDS prompts and point-of-care education. Formative and summative usability testing were completed with pediatric residents in a simulated electronic health record (EHR). We estimated task completion rates and perceived usefulness of the CDS in summative testing, comparing a test EHR with and without the CDS.

RESULTS

Formative usability testing with the residents provided qualitative data that led to improvements in the build for both the 4-month and 6-month WCC note templates. During summative usability testing, the CDS tool significantly improved discussion of early peanut introduction at the 4-month WCC visit compared to scenarios without the CDS tool (9/15, 60% with CDS and 0/15, 0% without CDS). All providers except one at the 4-month WCC scenario gave at least an adequate score for the ease of use of the CDS tool for the history of present illness and assessment and plan sections. During the summative usability testing with the 6-month WCC new build note template, providers more commonly provided comprehensive care once obtaining a patient history concerning for an immunoglobulin E-mediated peanut reaction by placing a referral to allergy/immunology (P=.48), prescribing an epinephrine auto-injector (P=.07), instructing on how to avoid peanut products (P<.001), and providing an emergency treatment plan (P=.003) with CDS guidance. All providers gave at least an adequate score for ease of use of the CDS tool in the after-visit summary.

CONCLUSIONS

User-centered CDS improved application of early peanut introduction recommendations and comprehensive care for patients who have symptoms concerning for peanut allergy in a simulation.

Quasi-experimental, Nonrandomized Initiative to Minimize Sleep Disruptions among Hospitalized Children

Hospitalized children experience frequent sleep disruptions. We aimed to reduce caregiver-reported sleep disruptions of children hospitalized on the pediatric hospital medicine service by 10% over 12 months.

Methods

In family surveys, caregivers cited overnight vital signs (VS) as a primary contributor to sleep disruption. We created a new VS frequency order of "every 4 hours (unless asleep between 2300 and 0500)" as well as a patient list column in the electronic health record indicating patients with this active VS order. The outcome measure was caregiver-reported sleep disruptions. The process measure was adherence to the new VS frequency. The balancing measure was rapid responses called on patients with the new VS frequency.

Results

Physician teams ordered the new VS frequency for 11% (1,633/14,772) of patient nights on the pediatric hospital medicine service. Recorded VS between 2300 and 0500 was 89% (1,447/1,633) of patient nights with the new frequency ordered compared to 91% (11,895/13,139) of patient nights without the new frequency ordered (P = 0.01). By contrast, recorded blood pressure between 2300 and 0500 was only 36% (588/1,633) of patient nights with the new frequency but 87% (11,478/13,139) of patient nights without the new frequency (P < 0.001). Overall, caregivers reported sleep disruptions on 24% (99/419) of reported nights preintervention, which decreased to 8% (195/2,313) postintervention (P < 0.001). Importantly, there were no adverse safety issues related to this initiative.

Conclusion

This study safely implemented a new VS frequency with reduced overnight blood pressure readings and caregiver-reported sleep disruptions.

Reducing Therapeutic Duplication in Inpatient Medication Orders

BACKGROUND

Therapeutic duplication, the presence of multiple agents prescribed for the same indication without clarification for when each should be used, can contribute to serious medical errors. Joint Commission standards require that orders contain clarifying information about when each order should be given. In our system, as needed (PRN) acetaminophen and ibuprofen orders are major contributors to therapeutic duplication.

OBJECTIVE

The objective of this study is to design and evaluate effectiveness of clinical decision support (CDS) to reduce therapeutic duplication with acetaminophen and ibuprofen orders.

METHODS

This study was done in a pediatric health system with three freestanding hospitals. We iteratively designed and implemented two CDS strategies aimed at reducing the therapeutic duplication with these agents: (1) interruptive alert prompting clinicians for clarifying PRN comments at order entry and (2) addition of discrete "first-line" and "second-line" PRN reasons to orders. Therapeutic duplications were measured by manual review of orders for 30-day periods before and after each intervention and 6 months later.

RESULTS

Therapeutic duplications decreased from 1,485 in the 30 days prior to the first alert implementation to 818 in the 30 days after but rose back to 1,208 in the 30 days prior to the second intervention. After discrete reasons were added to the order, therapeutic duplication decreased to 336 in the immediate 30 days and 6 months later remained at 277. Alerts firing rates decreased from 76.0 per 1,000 PRN acetaminophen or ibuprofen orders to 42.9 after the second intervention.

CONCLUSION

Interruptive alerts may reduce therapeutic duplication but are associated with high rates of user frustration and alert fatigue. Leveraging discrete PRN reasons for "first line" and "second line" produced a greater reduction in therapeutic duplication as well as fewer interruptive alerts and less manual entry for providers.

The Effect of Implementation of Guideline Order Bundles Into a General Admission Order Set on Clinical Practice Guideline Adoption: Quasi-Experimental Study

BACKGROUND

Clinical practice guidelines (CPGs) and associated order sets can help standardize patient care and lead to higher-value patient care. However, difficult access and poor usability of these order sets can result in lower use rates and reduce the CPGs' impact on clinical outcomes. At our institution, we identified multiple CPGs for general pediatrics admissions where the appropriate order set was used in <50% of eligible encounters, leading to decreased adoption of CPG recommendations.

OBJECTIVE

We aimed to determine how integrating disease-specific order groups into a common general admission order set influences adoption of CPG-specific order bundles for patients meeting CPG inclusion criteria admitted to the general pediatrics service.

METHODS

We integrated order bundles for asthma, heavy menstrual bleeding, musculoskeletal infection, migraine, and pneumonia into a common general pediatrics order set. We compared pre- and postimplementation order bundle use rates for eligible encounters at both an intervention and nonintervention site for integrated CPGs. We also assessed order bundle adoption for nonintegrated CPGs, including bronchiolitis, acute gastroenteritis, and croup. In a post hoc analysis of encounters without order bundle use, we compared the pre- and postintervention frequency of diagnostic uncertainty at the time of admission.

RESULTS

CPG order bundle use rates for incorporated CPGs increased by +9.8% (from 629/856, 73.5% to 405/486, 83.3%) at the intervention site and by +5.1% (896/1351, 66.3% to 509/713, 71.4%) at the nonintervention site. Order bundle adoption for nonintegrated CPGs decreased from 84% (536/638) to 68.5% (148/216), driven primarily by decreases in bronchiolitis order bundle adoption in the setting of the COVID-19 pandemic. Diagnostic uncertainty was more common in admissions without CPG order bundle use after implementation (28/227, 12.3% vs 19/81, 23.4%).

CONCLUSIONS

The integration of CPG-specific order bundles into a general admission order set improved overall CPG adoption. However, integrating only some CPGs may reduce adoption of order bundles for excluded CPGs. Diagnostic uncertainty at the time of admission is likely an underrecognized barrier to guideline adherence that is not addressed by an integrated admission order set.

Integrating structured and unstructured data for timely prediction of bloodstream infection among children

BACKGROUND

Hospitalized children with central venous lines (CVLs) are at higher risk of hospital-acquired infections. Information in electronic health records (EHRs) can be employed in training deep learning models to predict the onset of these infections. We incorporated clinical notes in addition to structured EHR data to predict serious bloodstream infections, defined as positive blood culture followed by at least 4 days of new antimicrobial agent administration, among hospitalized children with CVLs.

METHODS

Structured EHR information and clinical notes were extracted for a retrospective cohort including all hospitalized patients with CVLs at a single tertiary care pediatric health system from 2013 to 2018. Deep learning models were trained to determine the added benefit of incorporating the information embedded in clinical notes in predicting serious bloodstream infection.

RESULTS

A total of 24,351 patient encounters met inclusion criteria. The best-performing model restricted to structured EHR data had a specificity of 0.951 and positive predictive value (PPV) of 0.056 when the sensitivity was set to 0.85. The addition of contextualized word embeddings improved the specificity to 0.981 and PPV to 0.113.

CONCLUSIONS

Integrating clinical notes with structured EHR data improved the prediction of serious bloodstream infections among pediatric patients with CVLs.

IMPACT

Developed an advanced infection prediction model in pediatrics that integrates the structured and unstructured EHRs. Extracted information from clinical notes to do timely prediction in a clinical setting. Developed a deep learning model framework that can be employed in predicting rare events in a complex and dynamic environment.

Free-Text Computerized Provider Order Entry Orders Used as Workaround for Communicating Medication Information

OBJECTIVES

Medication information is frequently communicated via free-text computerized provider order entry (CPOE) orders in electronic health records. When such information is transmitted separately from a structured CPOE medication order, there is a significant risk of medication error. Although prior studies have described the frequency of using free-text CPOE orders for communicating medication information, there is a gap in understanding the nature of the medication information contained in the free-text CPOE orders. The aims of this study are to (1) identify the most common medication names communicated in free-text CPOE orders and their risk levels and (2) identify what actions physicians expect that nurses will complete when they place free-text CPOE orders, and (3) describe differences in these patterns across hospitals.

METHODS

This study was a retrospective analysis of a sample of 26,524 free-text CPOE orders from 6 hospitals in the mid-Atlantic U.S. region.

RESULTS

Free-text CPOE orders contained in the sample mentioned 193 medication names. Free-text CPOE orders were used frequently to communicate information about naloxone, heparin, flumazenil, and dextrose. Twenty-two percent of the free-text CPOE orders related to discontinuing medication(s), whereas 7% of the free-text CPOE orders relate to giving medication(s). There was high variation across hospitals both in the percentage of free-text CPOE orders mentioning medication information and in the proportion of those that referred to high-risk medications.

CONCLUSIONS

The prevalence of medication information in free-text CPOE orders may suggest specific communication challenges in respect to urgency, uncertainty, planning, and other aspects of communication and clinical needs. Understanding and addressing communication challenges around commonly mentioned medication names and actions, especially those that are high risk, can help reduce the risk of medication errors.

A method to advance adolescent sexual health research: Automated algorithm finds sexual history documentation

Background:

We aimed to develop and validate a rule-based Natural Language Processing (NLP) algorithm to detect sexual history documentation and its five key components [partners, practices, past history of sexually transmitted infections (STIs), protection from STIs, and prevention of pregnancy] among adolescent encounters in the pediatric emergency and inpatient settings.

Methods

We iteratively designed a NLP algorithm using pediatric emergency department (ED) provider notes from adolescent ED visits with specific abdominal or genitourinary (GU) chief complaints. The algorithm is composed of regular expressions identifying commonly used phrases in sexual history documentation. We validated this algorithm with inpatient admission notes for adolescents. We calculated the sensitivity, specificity, negative predictive value, positive predictive value, and F1 score of the tool in each environment using manual chart review as the gold standard.

Results

In the ED test cohort with abdominal or GU complaints, 97/179 (54%) provider notes had a sexual history documented, and the NLP algorithm correctly classified each note. In the inpatient validation cohort, 97/321 (30%) admission notes included a sexual history, and the NLP algorithm had 100% sensitivity and 98.2% specificity. The algorithm demonstrated &gt;97% sensitivity and specificity in both settings for detection of elements of a high quality sexual history including protection used and contraception. Type of sexual practice and STI testing offered were also detected with &gt;97% sensitivity and specificity in the ED test cohort with slightly lower performance in the inpatient validation cohort.

Conclusion

This NLP algorithm automatically detects the presence of sexual history documentation and its key components in ED and inpatient settings.

Barriers to Seasonal Influenza Vaccine Uptake in a Pediatric Inpatient Healthcare Setting After Implementation of Clinical Decision Support

Every heatlhcare encounter is an opportunity to provide both acute care and health maintenance to children. A Clinical Decision Support (CDS) intervention was instituted in a tertiary pediatric health system to improve influenza vaccination rates during the 2019-2020 season among eligible children receiving care in an acute care inpatient healthcare setting. This study explores reasons for low vaccine uptake following implementation of a CDS aimed at improving vaccine administration as well as identifying possible solutions to improve flu vaccine coverage.

Clinical Decision Support Stewardship: Best Practices and Techniques to Monitor and Improve Interruptive Alerts

Interruptive clinical decision support systems, both within and outside of electronic health records, are a resource that should be used sparingly and monitored closely. Excessive use of interruptive alerting can quickly lead to alert fatigue and decreased effectiveness and ignoring of alerts. In this review, we discuss the evidence for effective alert stewardship as well as practices and methods we have found useful to assess interruptive alert burden, reduce excessive firings, optimize alert effectiveness, and establish quality governance at our institutions. We also discuss the importance of a holistic view of the alerting ecosystem beyond the electronic health record.

Automated Identification of Immunocompromised Status in Critically Ill Children

BACKGROUND

Easy identification of immunocompromised hosts (ICH) would allow for stratification of culture results based on host type.

METHODS

We utilized antimicrobial stewardship (ASP) team notes written during handshake stewardship rounds in the pediatric intensive care unit as the gold standard for host status; clinical notes from the primary team, medication orders during the encounter, problem list and billing diagnoses documented prior to the ASP documentation were extracted to develop models that predict host status. We calculated performance for three models based on diagnoses/medications, with and without natural language processing from clinical notes. The susceptibility of pathogens causing bacteremia to commonly used empiric antibiotic regimens was then stratified by host status.

RESULTS

We identified 844 antimicrobial episodes from 666 unique patients; 160 (18.9%) were identified as an ICH. We randomly selected 675 initiations (80%) for model training and 169 initiations (20%) for testing. A rule-based model using diagnoses and medications alone yielded sensitivity of 0.87 (08.6-0.88), specificity of 0.93 (0.92-0.93), and positive predictive value (PPV) of 0.74 (0.73-0.75). Adding clinical notes into XGBoost model led to improved specificity of 0.98 (0.98 - 0.98) and PPV of 0.9 (0.88 - 0.91), but with decreased sensitivity 0.77 (0.76 - 0.79). There were 77 bacteremia episodes during the study period identified and a host specific visualization was created.

CONCLUSIONS

An EHR phenotype based on notes, diagnoses and medications identifies ICH in the PICU with high specificity.

User-centered design of central venous access device documentation

Objective

Safe care of central venous access devices (CVAD) requires clinicians be able to identify key CVAD properties from insertion until safe removal. Our objective was to design and evaluate interfaces to improve CVAD documentation quality and information retrieval.

Materials and Methods

We applied user-centered design (UCD) to CVAD property documentation interfaces. We measured expert agreement and front-line clinician accuracy in retrieving key properties in CVADs documented pre- and postimplementation.

Results

The new approach (1) optimized searches for line types, (2) enabled discrete entry of key properties which propagated to the display name, and (3) facilitated error correction by experts. Expert agreement on key CVAD properties improved from 42% to 83% (P < 0.01). Frontline nurses’ perception of key CVAD properties improved from 31% to 86% (P < 0.01). Ease of use scores improved from 15/100 to 80/100 (P < 0.01).

Conclusions

UCD significantly improved data quality and nurse perception of CVAD properties to guide subsequent care.

Central venous access devices (CVADs) are inserted into the deep central veins to enable safe administration of fluids, medications, and other therapies to the bloodstream. Safe care of CVAD requires clinicians be able to identify key CVAD properties. We report the design and implementation of a new interface for CVAD documentation in the electronic health record using a user-centered design (UCD) approach. The new design optimized search for line types, enabled discrete entry of key properties that propagated to display names, and facilitated error correction by experts. Our evaluation found that the new approach was efficient, easy to use, and it significantly improved the quality of CVAD documentation and provider awareness of CVAD properties.

Missed Opportunities for Sexual History Documentation and Sexually Transmitted Infection Testing in the Pediatric Emergency Department

PURPOSE

Sexually transmitted infections (STIs) are disproportionally prevalent in adolescents, and adolescents often present to the pediatric emergency department (PED) for STI care. Prior studies have found low rates of sexual history documentation and STI testing in the PED. However, these studies have had limited sample sizes because of the burden of manual chart review. We aimed to estimate the rate of sexual history documentation and identify factors associated with STI testing in a large cohort of adolescents using natural language processing (NLP).

METHODS

We applied a validated NLP algorithm to all adolescent visits over a three-year period to the PED at a single large children's health care organization with a chief complaint potentially related to an STI. We utilized NLP to determine the prevalence of sexual history documentation in these patients. We applied logistic regression models to determine associations between sexual history documentation, patient demographic factors, and STI testing.

RESULTS

Of the 1,987 patient encounters included, only 56% had a sexual history documented, and only 40% of all patients were tested for STIs. Patients were more likely to have a sexual history documented and to be tested for STIs if they were of non-Hispanic black race/ethnicity, were >15 years of age, and had nonprivate insurance. Patients with a sexual history documented were seven times more likely to have STI testing ordered. Of patients tested (n = 728), 25% were positive for an STI.

CONCLUSIONS

Despite presenting to the PED with symptoms potentially related to an STI, many adolescents are not receiving recommended sexual health care. Rates of sexual history documentation and STI testing varied by demographic factors including race, age, and insurance status. Utilizing NLP technology allowed us to examine a larger sample size than previously documented in the adolescent sexual history and PED literature. This study highlights critical opportunities to improve sexual health provision and equity of care provided in the PED.

Alert burden in pediatric hospitals: a cross-sectional analysis of six academic pediatric health systems using novel metrics

BACKGROUND

Excessive electronic health record (EHR) alerts reduce the salience of actionable alerts. Little is known about the frequency of interruptive alerts across health systems and how the choice of metric affects which users appear to have the highest alert burden.

OBJECTIVE

(1) Analyze alert burden by alert type, care setting, provider type, and individual provider across 6 pediatric health systems. (2) Compare alert burden using different metrics.

MATERIALS AND METHODS

We analyzed interruptive alert firings logged in EHR databases at 6 pediatric health systems from 2016-2019 using 4 metrics: (1) alerts per patient encounter, (2) alerts per inpatient-day, (3) alerts per 100 orders, and (4) alerts per unique clinician days (calendar days with at least 1 EHR log in the system). We assessed intra- and interinstitutional variation and how alert burden rankings differed based on the chosen metric.

RESULTS

Alert burden varied widely across institutions, ranging from 0.06 to 0.76 firings per encounter, 0.22 to 1.06 firings per inpatient-day, 0.98 to 17.42 per 100 orders, and 0.08 to 3.34 firings per clinician day logged in the EHR. Custom alerts accounted for the greatest burden at all 6 sites. The rank order of institutions by alert burden was similar regardless of which alert burden metric was chosen. Within institutions, the alert burden metric choice substantially affected which provider types and care settings appeared to experience the highest alert burden.

CONCLUSION

Estimates of the clinical areas with highest alert burden varied substantially by institution and based on the metric used.

The influence of integrating clinical practice guideline order bundles into a general admission order set on guideline adoption

Objectives of this study were to (1) describe barriers to using clinical practice guideline (CPG) admission order sets in a pediatric hospital and (2) determine if integrating CPG order bundles into a general admission order set increases adoption of CPG-recommended orders compared to standalone CPG order sets. We identified CPG-eligible encounters and surveyed admitting physicians to understand reasons for not using the associated CPG order set. We then integrated CPG order bundles into a general admission order set and evaluated effectiveness through summative usability testing in a simulated environment. The most common reasons for the nonuse of CPG order sets were lack of awareness or forgetting about the CPG order set. In usability testing, CPG order bundle use increased from 27.8% to 66.6% while antibiotic ordering errors decreased from 62.9% to 18.5% with the new design. Integrating CPG-related order bundles into a general admission order set improves CPG order set use in simulation by addressing the most common barriers to CPG adoption.

Evaluation of a Clinical Decision Support Strategy to Increase Seasonal Influenza Vaccination Among Hospitalized Children Before Inpatient Discharge

IMPORTANCE

Hospitalized children are at increased risk of influenza-related complications, yet influenza vaccine coverage remains low among this group. Evidence-based strategies about vaccination of vulnerable children during all health care visits are especially important during the COVID-19 pandemic.

OBJECTIVE

To design and evaluate a clinical decision support (CDS) strategy to increase the proportion of eligible hospitalized children who receive a seasonal influenza vaccine prior to inpatient discharge.

DESIGN, SETTING, AND PARTICIPANTS

This quality improvement study was conducted among children eligible for the seasonal influenza vaccine who were hospitalized in a tertiary pediatric health system providing care to more than half a million patients annually in 3 hospitals. The study used a sequential crossover design from control to intervention and compared hospitalizations in the intervention group (2019-2020 season with the use of an intervention order set) with concurrent controls (2019-2020 season without use of an intervention order set) and historical controls (2018-2019 season with use of an order set that underwent intervention during the 2019-2020 season).

INTERVENTIONS

A CDS intervention was developed through a user-centered design process, including (1) placing a default influenza vaccine order into admission order sets for eligible patients, (2) a script to offer the vaccine using a presumptive strategy, and (3) just-in-time education for clinicians addressing vaccine eligibility in the influenza order group with links to further reference material. The intervention was rolled out in a stepwise fashion during the 2019-2020 influenza season.

MAIN OUTCOMES AND MEASURES

Proportion of eligible hospitalizations in which 1 or more influenza vaccines were administered prior to discharge.

RESULTS

Among 17 740 hospitalizations (9295 boys [52%]), the mean (SD) age was 8.0 (6.0) years, and the patients were predominantly Black (n = 8943 [50%]) or White (n = 7559 [43%]) and mostly had public insurance (n = 11 274 [64%]). There were 10 997 hospitalizations eligible for the influenza vaccine in the 2019-2020 season. Of these, 5449 (50%) were in the intervention group, and 5548 (50%) were concurrent controls. There were 6743 eligible hospitalizations in 2018-2019 that served as historical controls. Vaccine administration rates were 31% (n = 1676) in the intervention group, 19% (n = 1051) in concurrent controls, and 14% (n = 912) in historical controls (P < .001). In adjusted analyses, the odds of receiving the influenza vaccine were 3.25 (95% CI, 2.94-3.59) times higher in the intervention group and 1.28 (95% CI, 1.15-1.42) times higher in concurrent controls than in historical controls.

CONCLUSIONS AND RELEVANCE

This quality improvement study suggests that user-centered CDS may be associated with significantly improved influenza vaccination rates among hospitalized children. Stepwise implementation of CDS interventions was a practical method that was used to increase quality improvement rigor through comparison with historical and concurrent controls.

Clinician Perceptions on the Use of Free-Text Communication Orders

OBJECTIVE

The aim of this study was to investigate (1) why ordering clinicians use free-text orders to communicate medication information; (2) what risks physicians and nurses perceive when free-text orders are used for communicating medication information; and (3) how electronic health records (EHRs) could be improved to encourage the safe communication of medication information.

METHODS

We performed semi-structured, scenario-based interviews with eight physicians and eight nurses. Interview responses were analyzed and grouped into common themes.

RESULTS

Participants described eight reasons why clinicians use free-text medication orders, five risks relating to the use of free-text medication orders, and five recommendations for improving EHR medication-related communication. Poor usability, including reduced efficiency and limited functionality associated with structured order entry, was the primary reason clinicians used free-text orders to communicate medication information. Common risks to using free-text orders for medication communication included the increased likelihood of missing orders and the increased workload on nurses responsible for executing orders.

DISCUSSION

Clinicians' use of free-text orders is primarily due to limitations in the current structured order entry design. To encourage the safe communication of medication information between clinicians, the EHR's structured order entry must be redesigned to support clinicians' cognitive and workflow needs that are currently being addressed via the use of free-text orders.

CONCLUSION

Clinicians' use of free-text orders as a workaround to insufficient structured order entry can create unintended patient safety risks. Thoughtful solutions designed to address these workarounds can improve the medication ordering process and the subsequent medication administration process.

Predicting presumed serious infection among hospitalized children on central venous lines with machine learning

BACKGROUND

Presumed serious infection (PSI) is defined as a blood culture drawn and new antibiotic course of at least 4 days among pediatric patients with Central Venous Lines (CVLs). Early PSI prediction and use of medical interventions can prevent adverse outcomes and improve the quality of care.

METHODS

Clinical features including demographics, laboratory results, vital signs, characteristics of the CVLs and medications used were extracted retrospectively from electronic medical records. Data were aggregated across all hospitals within a single pediatric health system and used to train machine learning models (XGBoost and ElasticNet) to predict the occurrence of PSI 8 h prior to clinical suspicion. Prediction for PSI was benchmarked against PRISM-III.

RESULTS

Our model achieved an area under the receiver operating characteristic curve of 0.84 (95% CI = [0.82, 0.85]), sensitivity of 0.73 [0.69, 0.74], and positive predictive value (PPV) of 0.36 [0.34, 0.36]. The PRISM-III conversely achieved a lower sensitivity of 0.19 [0.16, 0.22] and PPV of 0.30 [0.26, 0.34] at a cut-off of ≥ 10. The features with the most impact on the PSI prediction were maximum diastolic blood pressure prior to PSI prediction (mean SHAP = 3.4), height (mean SHAP = 3.2), and maximum temperature prior to PSI prediction (mean SHAP = 2.6).

CONCLUSION

A machine learning model using common features in the electronic medical records can predict the onset of serious infections in children with central venous lines at least 8 h prior to when a clinical team drew a blood culture.

The Radiologist's Gaze: Mapping Three-Dimensional Visual Search in Computed Tomography of the Abdomen and Pelvis

A radiologist's search pattern can directly influence patient management. A missed finding is a missed opportunity for intervention. Multiple studies have attempted to describe and quantify search patterns but have mainly focused on chest radiographs and chest CTs. Here, we describe and quantify the visual search patterns of 17 radiologists as they scroll through 6 CTs of the abdomen and pelvis. Search pattern tracings varied among individuals and remained relatively consistent per individual between cases. Attendings and trainees had similar eye metric statistics with respect to time to first fixation (TTFF), number of fixations in the region of interest (ROI), fixation duration in ROI, mean saccadic amplitude, or total number of fixations. Attendings had fewer numbers of fixations per second versus trainees (p < 0.001), suggesting efficiency due to expertise. In those cases that were accurately interpreted, TTFF was shorter (p = 0.04), the number of fixations per second and number of fixations in ROI were higher (p = 0.04, p = 0.02, respectively), and fixation duration in ROI was increased (p = 0.02). We subsequently categorized radiologists as "scanners" or "drillers" by both qualitative and quantitative methods and found no differences in accuracy with most radiologists being categorized as "drillers." This study describes visual search patterns of radiologists in interpretation of CTs of the abdomen and pelvis to better approach future endeavors in determining the effects of manipulations such as fatigue, interruptions, and computer-aided detection.

Communication through the electronic health record: frequency and implications of free text orders

Communication for non-medication order (CNMO) is a type of free text communication order providers use for asynchronous communication about patient care. The objective of this study was to understand the extent to which non-medication orders are being used for medication-related communication. We analyzed a sample of 26 524 CNMOs placed in 6 hospitals. A total of 42% of non-medication orders contained medication information. There was large variation in the usage of CNMOs across hospitals, provider settings, and provider types. The use of CNMOs for communicating medication-related information may result in delayed or missed medications, receiving medications that should have been discontinued, or important clinical decision being made based on inaccurate information. Future studies should quantify the implications of these data entry patterns on actual medication error rates and resultant safety issues.

Evaluating visual analytics for health informatics applications: a systematic review from the American Medical Informatics Association Visual Analytics Working Group Task Force on Evaluation

OBJECTIVE

This article reports results from a systematic literature review related to the evaluation of data visualizations and visual analytics technologies within the health informatics domain. The review aims to (1) characterize the variety of evaluation methods used within the health informatics community and (2) identify best practices.

METHODS

A systematic literature review was conducted following PRISMA guidelines. PubMed searches were conducted in February 2017 using search terms representing key concepts of interest: health care settings, visualization, and evaluation. References were also screened for eligibility. Data were extracted from included studies and analyzed using a PICOS framework: Participants, Interventions, Comparators, Outcomes, and Study Design.

RESULTS

After screening, 76 publications met the review criteria. Publications varied across all PICOS dimensions. The most common audience was healthcare providers (n = 43), and the most common data gathering methods were direct observation (n = 30) and surveys (n = 27). About half of the publications focused on static, concentrated views of data with visuals (n = 36). Evaluations were heterogeneous regarding setting and measurements used.

DISCUSSION

When evaluating data visualizations and visual analytics technologies, a variety of approaches have been used. Usability measures were used most often in early (prototype) implementations, whereas clinical outcomes were most common in evaluations of operationally-deployed systems. These findings suggest opportunities for both (1) expanding evaluation practices, and (2) innovation with respect to evaluation methods for data visualizations and visual analytics technologies across health settings.

CONCLUSION

Evaluation approaches are varied. New studies should adopt commonly reported metrics, context-appropriate study designs, and phased evaluation strategies.

Electronic Progress Note Reading Patterns: An Eye Tracking Analysis

This study used eye-tracking to understand how the order of note sections influences the way physicians read electronic progress notes. Participants (n = 7) wore an eye-tracking device while reviewing progress notes for four patient cases and then provided a verbal summary. We reviewed and analyzed verbal summaries and eye tracking recordings. Wide variation in reading behaviors existed. There was no relationship between time spent reading a section and section origin of verbal summaries.

Designing a wrist-worn sensor to improve medication adherence: accommodating diverse user behaviors and technology preferences

Objectives

High medication adherence is important for HIV suppression (antiretroviral therapy) and pre-exposure prophylaxis efficacy. We are developing sensor-based technologies to detect pill-taking gestures, trigger reminders, and generate adherence reports.

Materials and Methods

We collected interview, observation, and questionnaire data from individuals with and at-risk for HIV (N = 17). We assessed their medication-taking practices and physical actions, and feedback on our initial design.

Results

While participants displayed diverse medication taking practices and physical actions, most (67%) wanted to use the system to receive real-time and summative feedback, and most (69%) wanted to share data with their physicians. Participants preferred reminders via the wrist-worn device or mobile app, and summative feedback via mobile app or email.

Discussion

Adoption of these systems is promising if designs accommodate diverse behaviors and preferences.

Conclusion

Our findings may help improve the accuracy and adoption of the system by accounting for user behaviors, physical actions, and preferences.

On the use of hydroxyurea/erythropoietin combination therapy for sickle cell disease

Seven sickle cell disease (SCD) patients [sickle cell anaemia = 4 (males 2, females 2, age range 18-40 years), and sickle cell beta (0)-thalassaemia = 3 (all females, age range 20-47 years)], suffering from a severe form of the disease were enrolled in a treatment protocol using hydroxyurea (HU) for up to 12 months followed by a combination therapy with HU and human recombinant erythropoietin (rHuEpo; using 400 U/kg/week i.v.) for 3-4 weeks. Following the withdrawal of rHuEpo the patients were maintained on HU alone. The patients were characterised on the basis of the 'severity index' prior to the initiation of the therapy. Haematological and relevant biochemical parameters, Hb A2 fetal haemoglobin (HbF), HbF cells, reticulocytes and platelet counts were estimated at least at three occasions to determine the mean and range of the parameters. During the treatment period the patients were followed every 2-4 weeks where the haematological and biochemical parameters were assessed. The results were separately analysed and mean +/- SD were obtained for each parameter at the end of each protocol. The statistical significance of the difference in the results obtained on treatment and the baseline results was examined using the paired t test. No toxic side effects of HU and rHuEpo (as judged from reduction in platelet and white blood cell count) were documented during and after the whole period of treatment. The patients showed a significant clinical improvement. Total haemoglobin, haematocrit, red cell count, HbF and HbF cells increased, while white blood cells, reticulocyte counts and bilirubin level decreased. Platelet count decreased but remained within the normal range. The results revealed that 5 of the patients on HU treatment showed a significant increase in the HbF level and HbF cells, while 2 patients (1 sickle cell anaemia and 1 Hb S/beta(0)-thalassaemia patient) did not and were considered as 'non-responders'. The rHuEpo and HU combination therapy elevated the HbF level, with a varying degree, in all patients except 2, who had already reached a high HbF level and showed a decrease in HbF during the rHuEpo protocol. Variable individual response to both HU and rHuEpo therapy was a common feature. We recommend the use of HU for the treatment of SCD and a combination therapy using HU and rHuEpo for the non-responders.

The pharmacological manipulation of fetal haemoglobin: trials using hydroxyurea and recombinant human erythropoietin

Hydroxyurea (HU) and recombinant human erythropoietin (rHuEpo) have been used in several studies to elevate Hb F level in sickle cell disease (SCD) patients and hence to ameliorate the clinical presentations of the disease. The treatment protocol and doses have varied in the different studies. We studied the effects of HU+rHuEpo combination therapy in sickle cell anaemia (SCA patients) to investigate the Hb F manipulation and hence treatment of SCA. Six patients with severe SCA were selected for treatment with HU (20-25 mg/kg body weight) and rHuEpo (400-800 U/kg body weight) combination therapy for 4 weeks followed by HU (20-25 mg/kg body weight) maintenance therapy for 6 months to 1 year. Iron and folic acid were administered during HU+rHuEpo treatment. Signs, symptoms and complications were recorded to obtain the severity index. Only patients with a severity index > or = 6 were included in the study. Haematological and biochemical parameter values, Hb A2, Hb F, Hb F distribution, Hb F cells, bilirubin level and reticulocyte count were assessed at least on 2-3 occasions prior to initiation of the therapy protocol to establish baseline values. During the treatment period, the clinical presentations were monitored and the estimation of the laboratory parameters was carried out every 4-8 weeks. The results of these parameters during HU and rHuEpo combination therapy and HU maintenance therapy were compared with baseline values using paired t test. The elevation in the level of Hb F, Hb F cells, total haemoglobin, red cell count and MCV were significant (p < 0.005), while reticulocyte count and total bilirubin were significantly decreased (p < 0.05). Each patient showed an individual pattern of Hb F elevation. The increase in Hb F level was correlated with the haematological and biochemical parameters using the General Linear Model Programme of Statistical Analysis System. In general, the clinical presentation improved as Hb F level increased in each patient. In addition, the positive correlation with the haematological parameters and negative correlation with reticulocytes and total bilirubin confirmed the beneficial effect of elevated Hb F level on reducing red cell haemolysis. No correlation could be demonstrated between the pretreatment Hb F level and the increase in Hb F during the treatment period. Daily doses of HU with a single intravenous rHuEpo and iron supplementation elevate Hb F and Hb F cells in SCA patients. The Hb F level can be maintained high on HU therapy alone.(ABSTRACT TRUNCATED AT 400 WORDS)

Amebic pericarditis

Amebic pericarditis in a 40-year-old man who presented with insidious onset of symptoms is reported. Pericarditis is an infrequent complication of liver abscess and accounts for 4 percent of all extraintestinal amebiasis. Communication between the left lobe of liver and pericardium has been demonstrated in this patient.