1
|
Cao L, Masino AJ, Harris MC, Ungar LH, Shaeffer G, Fidel A, McLaurin E, Srinivasan L, Karavite DJ, Grundmeier RW. Aligning prediction models with clinical information needs: infant sepsis case study. JAMIA Open 2025; 8:ooaf015. [PMID: 40059975 PMCID: PMC11887542 DOI: 10.1093/jamiaopen/ooaf015] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/20/2024] [Revised: 02/06/2025] [Accepted: 02/11/2025] [Indexed: 03/30/2025] Open
Abstract
Objective Sepsis recognition among infants in the Neonatal Intensive Care Unit (NICU) is challenging and delays in recognition can result in devastating consequences. Although predictive models may improve sepsis outcomes, clinical adoption has been limited. Our focus was to align model behavior with clinician information needs by developing a machine learning (ML) pipeline with two components: (1) a model to predict baseline sepsis risk and (2) a model to detect evolving (dynamic) sepsis risk due to physiologic changes. We then compared the performance of this two-component pipeline to a single model that combines all features reflecting both baseline risk and evolving risk. Materials and Methods We developed prediction models (two-stage pipeline and a single model) using logistic regression and XGBoost trained on electronic healthcare record data of an NICU cohort (1706 observations from 1094 patients, with a 1:1 ratio of cases to controls). We used nested 10-fold cross-validation to evaluate model performance on predictions made 1 h (T -1) before actual clinical recognition. Results The single model (XGBoost) achieved the best performance with a sensitivity of 0.77 (0.74, 0.80), specificity of 0.83 (0.80, 0.85), and positive predictive value (PPV) of 0.82 (0.79, 0.84), at 1 h prior to clinical sepsis recognition (T -1). The pipeline model (XGBoost) achieved a sensitivity of 0.72 (0.69, 0.75), specificity of 0.84 (0.82, 0.87), and PPV of 0.82 (0.80, 0.85) at T -1. Discussion Our findings highlight the challenges of aligning machine learning with NICU clinical decision-making processes. The two-stage pipeline, designed to mirror clinicians' reasoning, underperformed compared to the single model. Future work should explore integrating continuous physiological data to enhance real-time risk assessment. Conclusion Although a pipeline model that separately estimates baseline and dynamic sepsis risk aligns with clinical information needs, at similar levels of specificity the observed sensitivity of the pipeline is inferior to that of a single model. Additional research is needed to better align model outputs with clinician information needs.
Collapse
Affiliation(s)
- Lusha Cao
- Department of Biomedical and Health Informatics, Children’s Hospital of Philadelphia, Philadelphia, PA 19146, United States
| | - Aaron J Masino
- School of Computing, Clemson University, Clemson, SC 29634, United States
- Center for Human Genetics, Clemson University, Clemson, SC 29634, United States
- School of Health Research, Clemson University, Clemson, SC 29634, United States
| | - Mary Catherine Harris
- Division of Neonatology, Children’s Hospital of Philadelphia, Philadelphia, PA 19104, United States
- Perelman School of Medicine at the University of Pennsylvania, Philadelphia, PA 19104, United States
| | - Lyle H Ungar
- Department of Computer and Information Science, University of Pennsylvania, Philadelphia, PA 19104, United States
| | - Gerald Shaeffer
- Department of Biomedical and Health Informatics, Children’s Hospital of Philadelphia, Philadelphia, PA 19146, United States
| | - Alexander Fidel
- Department of Biomedical and Health Informatics, Children’s Hospital of Philadelphia, Philadelphia, PA 19146, United States
| | - Elease McLaurin
- School of Medicine, Emory University, Atlanta, GA 30322, United States
| | - Lakshmi Srinivasan
- Division of Neonatology, Children’s Hospital of Philadelphia, Philadelphia, PA 19104, United States
- Perelman School of Medicine at the University of Pennsylvania, Philadelphia, PA 19104, United States
| | - Dean J Karavite
- Department of Biomedical and Health Informatics, Children’s Hospital of Philadelphia, Philadelphia, PA 19146, United States
| | - Robert W Grundmeier
- Department of Biomedical and Health Informatics, Children’s Hospital of Philadelphia, Philadelphia, PA 19146, United States
- Perelman School of Medicine at the University of Pennsylvania, Philadelphia, PA 19104, United States
| |
Collapse
|
2
|
Sullivan BA, Grundmeier RW. Machine Learning Models as Early Warning Systems for Neonatal Infection. Clin Perinatol 2025; 52:167-183. [PMID: 39892951 DOI: 10.1016/j.clp.2024.10.011] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 02/04/2025]
Abstract
Neonatal infections pose a significant threat to the health of newborns. Associated morbidity and mortality risks underscore the urgency of prompt diagnosis and treatment with appropriate empiric antibiotics. Delay in treatment can be fatal; thus, early detection improves outcomes. However, diagnosing early is a challenge as signs and symptoms of neonatal infection are non-specific and overlap with non-infectious conditions. Machine learning (ML) offers promise in early detection, utilizing various data sources and methodologies. However, ML models require rigorous validation and consideration of various challenges, including false alarms and user acceptance requiring careful integration and ongoing evaluation for successful implementation.
Collapse
Affiliation(s)
- Brynne A Sullivan
- Division of Neonatology, Department of Pediatrics, University of Virginia School of Medicine, 1215 Lee Street, P.O. Box 800386, Charlottesville, VA 22947, USA.
| | - Robert W Grundmeier
- Department of Pediatrics, Perelman School of Medicine at the University of Pennsylvania; Division of Clinical Informatics, Department of Biomedical and Health Informatics, The Children's Hospital of Philadelphia, 3400 Civic Center Boulevard Ste 10, Philadelphia, PA 19104, USA
| |
Collapse
|
3
|
Yang M, Peng Z, van Pul C, Andriessen P, Dong K, Silvertand D, Li J, Liu C, Long X. Continuous prediction and clinical alarm management of late-onset sepsis in preterm infants using vital signs from a patient monitor. COMPUTER METHODS AND PROGRAMS IN BIOMEDICINE 2024; 255:108335. [PMID: 39047574 DOI: 10.1016/j.cmpb.2024.108335] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 02/11/2024] [Revised: 06/14/2024] [Accepted: 07/17/2024] [Indexed: 07/27/2024]
Abstract
BACKGROUND AND OBJECTIVE Continuous prediction of late-onset sepsis (LOS) could be helpful for improving clinical outcomes in neonatal intensive care units (NICU). This study aimed to develop an artificial intelligence (AI) model for assisting the bedside clinicians in successfully identifying infants at risk for LOS using non-invasive vital signs monitoring. METHODS In a retrospective study from the NICU of the Máxima Medical Center in Veldhoven, the Netherlands, a total of 492 preterm infants less than 32 weeks gestation were included between July 2016 and December 2018. Data on heart rate (HR), respiratory rate (RR), and oxygen saturation (SpO2) at 1 Hz were extracted from the patient monitor. We developed multiple AI models using 102 extracted features or raw time series to provide hourly LOS risk prediction. Shapley values were used to explain the model. For the best performing model, the effect of different vital signs and also the input type of signals on model performance was tested. To further assess the performance of applying the best performing model in a real-world clinical setting, we performed a simulation using four different alarm policies on continuous real-time predictions starting from three days after birth. RESULTS A total of 51 LOS patients and 68 controls were finally included according to the patient inclusion and exclusion criteria. When tested by seven-fold cross-validations, the mean (standard deviation) area under the receiver operating characteristic curve (AUC) six hours before CRASH was 0.875 (0.072) for the best performing model, compared to the other six models with AUC ranging from 0.782 (0.089) to 0.846 (0.083). The best performing model performed only slightly worse than the model learning from raw physiological waveforms (0.886 [0.068]), successfully detecting 96.1 % of LOS patients before CRASH. When setting the expected alarm window to 24 h and using a multi-threshold alarm policy, the sensitivity metric was 71.6 %, while the positive predictive value was 9.9 %, resulting in an average of 1.15 alarms per day per patient. CONCLUSIONS The proposed AI model, which learns from routinely collected vital signs, has the potential to assist clinicians in the early detection of LOS. Combined with interpretability and clinical alarm management, this model could be better translated into medical practice for future clinical implementation.
Collapse
Affiliation(s)
- Meicheng Yang
- State Key Laboratory of Digital Medical Engineering, School of Instrument Science and Engineering, Southeast University, Nanjing, China; Department of Electrical Engineering, Eindhoven University of Technology, Eindhoven, the Netherlands
| | - Zheng Peng
- Department of Electrical Engineering, Eindhoven University of Technology, Eindhoven, the Netherlands; Department of Clinical Physics, Máxima Medical Centre, Veldhoven, the Netherlands
| | - Carola van Pul
- Department of Electrical Engineering, Eindhoven University of Technology, Eindhoven, the Netherlands; Department of Clinical Physics, Máxima Medical Centre, Veldhoven, the Netherlands; Department of Applied Physics, Eindhoven University of Technology, Eindhoven, the Netherlands
| | - Peter Andriessen
- Department of Applied Physics, Eindhoven University of Technology, Eindhoven, the Netherlands; Department of Pediatrics, Máxima Medical Centre, Veldhoven, the Netherlands
| | - Kejun Dong
- Nell Hodgson Woodruff School of Nursing, Emory University, Atlanta, United States of America
| | - Demi Silvertand
- Department of Pediatrics, Máxima Medical Centre, Veldhoven, the Netherlands
| | - Jianqing Li
- State Key Laboratory of Digital Medical Engineering, School of Instrument Science and Engineering, Southeast University, Nanjing, China; School of Biomedical Engineering and Informatics, Nanjing Medical University, Nanjing, China
| | - Chengyu Liu
- State Key Laboratory of Digital Medical Engineering, School of Instrument Science and Engineering, Southeast University, Nanjing, China
| | - Xi Long
- Department of Electrical Engineering, Eindhoven University of Technology, Eindhoven, the Netherlands.
| |
Collapse
|
4
|
Kausch SL, Slevin CC, Duncan A, Fairchild KD, Lake DE, Keim-Malpass J, Vesoulis ZA, Sullivan BA. Clinical correlates of a high cardiorespiratory risk score for very low birth weight infants. Pediatr Res 2024:10.1038/s41390-024-03580-y. [PMID: 39300276 DOI: 10.1038/s41390-024-03580-y] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 12/19/2023] [Revised: 08/15/2024] [Accepted: 08/26/2024] [Indexed: 09/22/2024]
Abstract
BACKGROUND A pulse oximetry warning system (POWS) to analyze heart rate and oxygen saturation data and predict risk of sepsis was developed for very low birth weight (VLBW) infants. METHODS We determined the clinical correlates and positive predictive value (PPV) of a high POWS score in VLBW infants. In a two-NICU retrospective study, we identified times when POWS increased above 6 (POWS spike). We selected an equal number of control times, matched for gestational and chronologic age. We reviewed records for infection and non-infection events around POWS spikes and control times. We calculated the frequencies and PPV of a POWS spike for infection or another significant event. RESULTS We reviewed 111 POWS spike times and 111 control times. Days near POWS spikes were more likely to have clinical events than control days (77% vs 50%). A POWS spike had 52% PPV for suspected or confirmed infection and 77% for any clinically significant event. Respiratory deterioration occurred near more POWS spike times than control times (34% vs 18%). CONCLUSIONS In a retrospective cohort, infection and respiratory deterioration were common clinical correlations of a POWS spike. POWS had a high PPV for significant clinical events with or without infection. IMPACT There are significant gaps in understanding the best approach to implementing continuous sepsis prediction models so that clinicians can best respond to early signals of deterioration. Infection and respiratory deterioration were common clinical events identified at the time of a high predictive model score. Understanding the clinical correlates of a high-risk early warning score will inform future implementation efforts.
Collapse
Affiliation(s)
- Sherry L Kausch
- Department of Pediatrics, Division of Neonatology, University of Virginia School of Medicine, Charlottesville, VA, USA.
| | - Claire C Slevin
- Department of Pediatrics, Johns Hopkins University, Baltimore, MD, USA
| | - Amanda Duncan
- Department of Pediatrics, Division of Newborn Medicine, Washington University in St. Louis, St. Louis, MO, USA
| | - Karen D Fairchild
- Department of Pediatrics, Division of Neonatology, University of Virginia School of Medicine, Charlottesville, VA, USA
| | - Douglas E Lake
- Department of Medicine, Division of Cardiology, University of Virginia School of Medicine, Charlottesville, VA, USA
| | - Jessica Keim-Malpass
- Department of Pediatrics, Division of Hematology, University of Virginia School of Medicine, Charlottesville, VA, USA
| | - Zachary A Vesoulis
- Department of Pediatrics, Division of Newborn Medicine, Washington University in St. Louis, St. Louis, MO, USA
| | - Brynne A Sullivan
- Department of Pediatrics, Division of Neonatology, University of Virginia School of Medicine, Charlottesville, VA, USA
| |
Collapse
|
5
|
Coggins SA, Carr LH, Harris MC, Srinivasan L. Sepsis Huddles in the Neonatal Intensive Care Unit: A Retrospective Cohort Study of Late-onset Infection Recognition and Severity Assessment. J Pediatr 2024; 272:114117. [PMID: 38815749 DOI: 10.1016/j.jpeds.2024.114117] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 12/14/2023] [Revised: 04/15/2024] [Accepted: 05/22/2024] [Indexed: 06/01/2024]
Abstract
OBJECTIVE To analyze relationships between provider-documented signs prompting sepsis evaluations, assessments of illness severity, and late-onset infection (LOI). STUDY DESIGN Retrospective cohort study of all infants receiving a sepsis huddle in conjunction with a LOI evaluation. Participants were ≥3 days old and admitted to a level IV neonatal intensive care unit (NICU) from September 2018 through May 2021. Data were extracted from standardized sepsis huddle notes in the electronic health record, including clinical signs prompting LOI evaluations, illness severity assessments (from least to most severe: green, yellow, and red), and management plans. To analyze relationships of sepsis huddle characteristics with the detection of culture-confirmed LOI (bacteremia, urinary tract infection, or meningitis), we utilized diagnostic test statistics, area under the receiver-operator characteristic analyses, and multivariable logistic regression. RESULTS We identified 1209 eligible sepsis huddles among 604 infants. There were 111 culture-confirmed LOI episodes (9% of all huddles). Twelve clinical signs of infection poorly distinguished infants with and without LOI, with sensitivity for each ranging from 2% to 36% and area under the receiver-operator characteristic ranging 0.49-0.53. Multivariable logistic regression identified increasing odds of infection with higher perceived illness severity at the time of sepsis huddle, adjusted for gestational age and receipt of intensive care supports. CONCLUSIONS Clinical signs prompting sepsis huddles were nonspecific and not predictive of concurrent LOI. Higher perceived illness severity was associated with presence of infection, despite some misclassification based on objective criteria. In level IV NICUs, antimicrobial stewardship through development of criteria for antibiotic noninitiation may be challenging, as presenting signs of LOI are similar among infants with and without confirmed infection.
Collapse
Affiliation(s)
- Sarah A Coggins
- Division of Neonatology, Children's Hospital of Philadelphia, Philadelphia, PA; Department of Pediatrics, University of Pennsylvania, Philadelphia, PA; Clinical Futures, Children's Hospital of Philadelphia, Philadelphia, PA.
| | - Leah H Carr
- Division of Neonatology, Children's Hospital of Philadelphia, Philadelphia, PA; Department of Pediatrics, University of Pennsylvania, Philadelphia, PA; Department of Biomedical and Health Informatics, Children's Hospital of Philadelphia, Philadelphia, PA
| | - Mary Catherine Harris
- Division of Neonatology, Children's Hospital of Philadelphia, Philadelphia, PA; Department of Pediatrics, University of Pennsylvania, Philadelphia, PA
| | - Lakshmi Srinivasan
- Division of Neonatology, Children's Hospital of Philadelphia, Philadelphia, PA; Department of Pediatrics, University of Pennsylvania, Philadelphia, PA
| |
Collapse
|
6
|
Rahman J, Brankovic A, Tracy M, Khanna S. Exploring Computational Techniques in Preprocessing Neonatal Physiological Signals for Detecting Adverse Outcomes: Scoping Review. Interact J Med Res 2024; 13:e46946. [PMID: 39163610 PMCID: PMC11372324 DOI: 10.2196/46946] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/02/2023] [Revised: 03/27/2024] [Accepted: 06/26/2024] [Indexed: 08/22/2024] Open
Abstract
BACKGROUND Computational signal preprocessing is a prerequisite for developing data-driven predictive models for clinical decision support. Thus, identifying the best practices that adhere to clinical principles is critical to ensure transparency and reproducibility to drive clinical adoption. It further fosters reproducible, ethical, and reliable conduct of studies. This procedure is also crucial for setting up a software quality management system to ensure regulatory compliance in developing software as a medical device aimed at early preclinical detection of clinical deterioration. OBJECTIVE This scoping review focuses on the neonatal intensive care unit setting and summarizes the state-of-the-art computational methods used for preprocessing neonatal clinical physiological signals; these signals are used for the development of machine learning models to predict the risk of adverse outcomes. METHODS Five databases (PubMed, Web of Science, Scopus, IEEE, and ACM Digital Library) were searched using a combination of keywords and MeSH (Medical Subject Headings) terms. A total of 3585 papers from 2013 to January 2023 were identified based on the defined search terms and inclusion criteria. After removing duplicates, 2994 (83.51%) papers were screened by title and abstract, and 81 (0.03%) were selected for full-text review. Of these, 52 (64%) were eligible for inclusion in the detailed analysis. RESULTS Of the 52 articles reviewed, 24 (46%) studies focused on diagnostic models, while the remainder (n=28, 54%) focused on prognostic models. The analysis conducted in these studies involved various physiological signals, with electrocardiograms being the most prevalent. Different programming languages were used, with MATLAB and Python being notable. The monitoring and capturing of physiological data used diverse systems, impacting data quality and introducing study heterogeneity. Outcomes of interest included sepsis, apnea, bradycardia, mortality, necrotizing enterocolitis, and hypoxic-ischemic encephalopathy, with some studies analyzing combinations of adverse outcomes. We found a partial or complete lack of transparency in reporting the setting and the methods used for signal preprocessing. This includes reporting methods to handle missing data, segment size for considered analysis, and details regarding the modification of the state-of-the-art methods for physiological signal processing to align with the clinical principles for neonates. Only 7 (13%) of the 52 reviewed studies reported all the recommended preprocessing steps, which could have impacts on the downstream analysis. CONCLUSIONS The review found heterogeneity in the techniques used and inconsistent reporting of parameters and procedures used for preprocessing neonatal physiological signals, which is necessary to confirm adherence to clinical and software quality management system practices, usefulness, and choice of best practices. Enhancing transparency in reporting and standardizing procedures will boost study interpretation and reproducibility and expedite clinical adoption, instilling confidence in the research findings and streamlining the translation of research outcomes into clinical practice, ultimately contributing to the advancement of neonatal care and patient outcomes.
Collapse
Affiliation(s)
- Jessica Rahman
- Commonwealth Scientific and Industrial Research Organisation (CSIRO) Australian e-Health Research Centre, Australia, Sydney, Australia
| | - Aida Brankovic
- Commonwealth Scientific and Industrial Research Organisation (CSIRO) Australian e-Health Research Centre, Australia, Brisbane, Australia
| | - Mark Tracy
- Neonatal Intensive Care Unit, Westmead, Sydney, Australia
| | - Sankalp Khanna
- Commonwealth Scientific and Industrial Research Organisation (CSIRO) Australian e-Health Research Centre, Australia, Brisbane, Australia
| |
Collapse
|
7
|
Abda A, Panetta L, Blackburn J, Chevalier I, Lachance C, Ovetchkine P, Sicard M. Urinary tract infections in very premature neonates: the definition dilemma. J Perinatol 2024; 44:731-738. [PMID: 38553603 DOI: 10.1038/s41372-024-01951-1] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 05/29/2023] [Revised: 03/17/2024] [Accepted: 03/20/2024] [Indexed: 05/15/2024]
Abstract
BACKGROUND AND OBJECTIVES Data on urinary tract infections (UTIs) in very preterm neonates (VPTNs) are scarce. We aimed to (i) describe the characteristics of UTIs in VPTNs and (ii) compare the diagnostic practices of neonatal clinicians to established pediatric guidelines. METHODS All VPTNs (<29 weeks GA) with a suspected UTI at the CHU Sainte-Justine neonatal intensive care unit from January 1, 2014, and December 31, 2019, were included and divided into two definition categories: Possible UTI, and Definite UTI. RESULTS Most episodes were Possible UTI (87%). Symptoms of UTIs and pathogens varied based on the definition category. A positive urinalysis was obtained in 25%. Possible UTI episodes grew 2 organisms in 62% of cases and <50,000 CFU/mL in 62% of cases. CONCLUSION Characteristics of UTIs in VPTNs vary based on the definition category and case definitions used by clinicians differ from that of established pediatric guidelines.
Collapse
Affiliation(s)
- Assil Abda
- Department of Pediatrics, Division of Neonatology, CHU Sainte-Justine, University of Montreal, Montreal, QC, Canada.
| | - Luc Panetta
- Department of Pediatrics, Division of Infectious Diseases, CHU Sainte-Justine, University of Montreal, Montreal, QC, Canada
- Pediatric Emergency Department, Hospices Civils de Lyon, Hôpital Femme Mère Enfant, Lyon, France
| | - Julie Blackburn
- Department of Pediatrics, Division of Infectious Diseases, CHU Sainte-Justine, University of Montreal, Montreal, QC, Canada
- Research center, CHU Sainte-Justine, Montreal, QC, Canada
- Department of Microbiology, Infectious Diseases and Immunology, University of Montreal, Montreal, QC, Canada
| | - Isabelle Chevalier
- Department of Pediatrics, Division of General Pediatrics, CHU Sainte-Justine, University of Montreal, Montreal, QC, Canada
| | - Christian Lachance
- Department of Pediatrics, Division of Neonatology, CHU Sainte-Justine, University of Montreal, Montreal, QC, Canada
- Research center, CHU Sainte-Justine, Montreal, QC, Canada
| | - Philippe Ovetchkine
- Department of Pediatrics, Division of Infectious Diseases, CHU Sainte-Justine, University of Montreal, Montreal, QC, Canada
- Research center, CHU Sainte-Justine, Montreal, QC, Canada
| | - Melanie Sicard
- Department of Pediatrics, Division of Neonatology, CHU Sainte-Justine, University of Montreal, Montreal, QC, Canada
- Research center, CHU Sainte-Justine, Montreal, QC, Canada
- Department of Microbiology, Infectious Diseases and Immunology, University of Montreal, Montreal, QC, Canada
| |
Collapse
|
8
|
Patel HR, Traylor B, Ahamed MF, Darling G, Botchway A, Batton BJ, Majjiga VS. Impact of Physician Characteristics on Late-Onset Sepsis (LOS) Evaluation in the NICU. Healthcare (Basel) 2024; 12:845. [PMID: 38667607 PMCID: PMC11050479 DOI: 10.3390/healthcare12080845] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/24/2024] [Revised: 04/12/2024] [Accepted: 04/14/2024] [Indexed: 04/28/2024] Open
Abstract
The threshold for a late-onset sepsis (LOS) evaluation varies considerably across NICUs. This unexplained variability is probably related in part to physician bias regarding when sepsis should be "ruled out". The aim of this study is to determine if physician characteristics (race, gender, immigration status, years of experience and academic rank) effect LOS evaluation in the NICU. This study includes a retrospective chart review of all Level III NICU infants who had a LOS evaluation over 54 months. Physician characteristics were compared between positive and negative blood culture groups and whether CBC and CRP were obtained at LOS evaluations. There were 341 LOS evaluations performed during the study period. Two patients were excluded due to a contaminant. Patients in this study had a birth weight of [median (Q1, Q3)]+ 992 (720, 1820) grams and birth gestation of [median (Q1, Q3)] 276/7 (252/7, 330/7) weeks. There are 10 neonatologists in the group, 5/10 being female and 6/10 being immigrant physicians. Experienced physicians were more likely to obtain a CBC at the time of LOS evaluation. Physician characteristics of race, gender and immigration status impacted whether to include a CRP as part of a LOS evaluation but otherwise did not influence LOS evaluation, including the likelihood of bacteremia.
Collapse
Affiliation(s)
- Harshkumar R. Patel
- Department of Pediatrics, SIU School of Medicine, Springfield, IL 62794, USA; (H.R.P.); (G.D.); (B.J.B.)
| | | | - Mohamed Farooq Ahamed
- Department of Pediatrics, SIU School of Medicine, Springfield, IL 62794, USA; (H.R.P.); (G.D.); (B.J.B.)
| | - Ginger Darling
- Department of Pediatrics, SIU School of Medicine, Springfield, IL 62794, USA; (H.R.P.); (G.D.); (B.J.B.)
| | - Albert Botchway
- Center for Clinical Research, SIU School of Medicine, Springfield, IL 62702, USA;
| | - Beau J. Batton
- Department of Pediatrics, SIU School of Medicine, Springfield, IL 62794, USA; (H.R.P.); (G.D.); (B.J.B.)
| | - Venkata Sasidhar Majjiga
- Department of Pediatrics, SIU School of Medicine, Springfield, IL 62794, USA; (H.R.P.); (G.D.); (B.J.B.)
| |
Collapse
|
9
|
Kumar RS, Otero NA, Abubakar MO, Elliott MR, Wiggins JY, Sharif MM, Sullivan BA, Fairchild KD. Framework for Considering Abnormal Heart Rate Characteristics and Other Signs of Sepsis in Very Low Birth Weight Infants. Am J Perinatol 2024; 41:706-712. [PMID: 34875699 DOI: 10.1055/a-1715-3727] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Indexed: 10/19/2022]
Abstract
OBJECTIVE A heart rate characteristics index (HeRO score), incorporating low variability and superimposed decelerations, was developed as a sepsis risk indicator for preterm infants in the neonatal intensive care unit (NICU). A rise in the risk score should prompt consideration of other clinical changes that may be signs of sepsis to decide whether a workup and antibiotics are needed. We aimed to develop a framework to systematically consider signs potentially indicating sepsis in very low birth weight (VLBW) infants. STUDY DESIGN We developed easy-recall acronyms for 10 signs of sepsis in VLBW infants. Over 12 months in a level IV NICU, neonatology fellows completed a brief survey after each shift to document changes prompting sepsis workups. We analyzed associations between survey data, hourly heart rate characteristic data, and the diagnosis of the workup, grouped as culture-positive sepsis (CXSEP, positive blood or urine culture), clinical sepsis (CLINSEP, negative cultures treated with antibiotics ≥5 days), or sepsis ruled out (SRO, negative cultures and <3 days antibiotics). RESULTS We analyzed 93 sepsis workups in 48 VLBW infants (35 CXSEP, 20 CLINSEP, and 38 SRO). The most frequently cited changes prompting the workups were heart rate patterns and respiratory deterioration, which were common in all three categories. Low blood pressure and poor perfusion were uncommonly cited but were more likely to be associated with CXSEP than the other signs. A rise in the HeRO score ≥1 from 0 to 12 hours before compared with 12to 72 hours prior the blood culture occurred in 31% of workups diagnosed as CXSEP, 16% CLINSEP, and 31% SRO. CONCLUSION The HeRO score can alert clinicians to VLBW infants at high or increasing risk of a sepsis-like illness, but heart rate characteristic patterns are highly variable in individual babies. The easy-recall NeoSEP-10 framework can assist clinicians in considering other clinical changes when making decisions about sepsis workups and the duration of antibiotics. KEY POINTS · Abnormal heart rate characteristics can indicate sepsis or other pathologies in preterm infants.. · We developed a simple bedside tool to consider clinical signs potentially associated with sepsis.. · Considering vital sign trends together with clinical changes is a key to right-timing antibiotics..
Collapse
Affiliation(s)
- Rupin S Kumar
- Department of Pediatrics, University of Kentucky, Lexington, Kentucky
| | | | - Maryam O Abubakar
- Department of Pediatrics, University of Virginia, Charlottesville, Virginia
| | - Megan R Elliott
- Department of Pediatrics, University of Virginia, Charlottesville, Virginia
| | - Jaclyn Y Wiggins
- Department of Pediatrics, University of Virginia, Charlottesville, Virginia
| | - Misky M Sharif
- Department of Pediatrics, University of Virginia, Charlottesville, Virginia
| | - Brynne A Sullivan
- Department of Pediatrics, University of Virginia, Charlottesville, Virginia
| | - Karen D Fairchild
- Department of Pediatrics, University of Virginia, Charlottesville, Virginia
| |
Collapse
|
10
|
Meeus M, Beirnaert C, Mahieu L, Laukens K, Meysman P, Mulder A, Van Laere D. Clinical Decision Support for Improved Neonatal Care: The Development of a Machine Learning Model for the Prediction of Late-onset Sepsis and Necrotizing Enterocolitis. J Pediatr 2024; 266:113869. [PMID: 38065281 DOI: 10.1016/j.jpeds.2023.113869] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 07/20/2023] [Revised: 11/24/2023] [Accepted: 12/04/2023] [Indexed: 01/08/2024]
Abstract
OBJECTIVE To develop an artificial intelligence-based software system for predicting late-onset sepsis (LOS) and necrotizing enterocolitis (NEC) in infants admitted to the neonatal intensive care unit (NICU). STUDY DESIGN Single-center, retrospective cohort study, conducted in the NICU of the Antwerp University Hospital. Continuous monitoring data of 865 preterm infants born at <32 weeks gestational age, admitted to the NICU in the first week of life, were used to train an XGBoost machine learning (ML) algorithm for LOS and NEC prediction in a cross-validated setup. Afterward, the model's performance was assessed on an independent test set of 148 patients (internal validation). RESULTS The ML model delivered hourly risk predictions with an overall sensitivity of 69% (142/206) for all LOS/NEC episodes and 81% (67/83) for severe LOS/NEC episodes. The model showed a median time gain of ≤10 hours (IQR, 3.1-21.0 hours), compared with historical clinical diagnosis. On the complete retrospective dataset, the ML model made 721 069 predictions, of which 9805 (1.3%) depicted a LOS/NEC probability of ≥0.15, resulting in a total alarm rate of <1 patient alarm-day per week. The model reached a similar performance on the internal validation set. CONCLUSIONS Artificial intelligence technology can assist clinicians in the early detection of LOS and NEC in the NICU, which potentially can result in clinical and socioeconomic benefits. Additional studies are required to quantify further the effect of combining artificial and human intelligence on patient outcomes in the NICU.
Collapse
Affiliation(s)
- Marisse Meeus
- Department of Neonatal Intensive Care, Antwerp University Hospital, Edegem, Belgium; Laboratory of Experimental Medicine and Pediatrics, Faculty of Medicine and Health Sciences, University of Antwerp, Antwerpen, Belgium.
| | - Charlie Beirnaert
- Department of Neonatal Intensive Care, Antwerp University Hospital, Edegem, Belgium; Innocens BV, Antwerpen, Belgium; Department of Computer Science, University of Antwerp, Antwerpen, Belgium
| | - Ludo Mahieu
- Department of Neonatal Intensive Care, Antwerp University Hospital, Edegem, Belgium; Laboratory of Experimental Medicine and Pediatrics, Faculty of Medicine and Health Sciences, University of Antwerp, Antwerpen, Belgium
| | - Kris Laukens
- Department of Computer Science, University of Antwerp, Antwerpen, Belgium
| | - Pieter Meysman
- Department of Computer Science, University of Antwerp, Antwerpen, Belgium
| | - Antonius Mulder
- Department of Neonatal Intensive Care, Antwerp University Hospital, Edegem, Belgium; Laboratory of Experimental Medicine and Pediatrics, Faculty of Medicine and Health Sciences, University of Antwerp, Antwerpen, Belgium
| | - David Van Laere
- Department of Neonatal Intensive Care, Antwerp University Hospital, Edegem, Belgium; Laboratory of Experimental Medicine and Pediatrics, Faculty of Medicine and Health Sciences, University of Antwerp, Antwerpen, Belgium; Innocens BV, Antwerpen, Belgium
| |
Collapse
|
11
|
Weese-Mayer DE, Di Fiore JM, Lake DE, Hibbs AM, Claure N, Qiu J, Ambalavanan N, Bancalari E, Kemp JS, Zimmet AM, Carroll JL, Martin RJ, Krahn KN, Hamvas A, Ratcliffe SJ, Krishnamurthi N, Indic P, Dormishian A, Dennery PA, Moorman JR. Maturation of cardioventilatory physiological trajectories in extremely preterm infants. Pediatr Res 2024; 95:1060-1069. [PMID: 37857848 DOI: 10.1038/s41390-023-02839-0] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 04/13/2023] [Revised: 08/14/2023] [Accepted: 09/27/2023] [Indexed: 10/21/2023]
Abstract
BACKGROUND In extremely preterm infants, persistence of cardioventilatory events is associated with long-term morbidity. Therefore, the objective was to characterize physiologic growth curves of apnea, periodic breathing, intermittent hypoxemia, and bradycardia in extremely preterm infants during the first few months of life. METHODS The Prematurity-Related Ventilatory Control study included 717 preterm infants <29 weeks gestation. Waveforms were downloaded from bedside monitors with a novel sharing analytics strategy utilized to run software locally, with summary data sent to the Data Coordinating Center for compilation. RESULTS Apnea, periodic breathing, and intermittent hypoxemia events rose from day 3 of life then fell to near-resolution by 8-12 weeks of age. Apnea/intermittent hypoxemia were inversely correlated with gestational age, peaking at 3-4 weeks of age. Periodic breathing was positively correlated with gestational age peaking at 31-33 weeks postmenstrual age. Females had more periodic breathing but less intermittent hypoxemia/bradycardia. White infants had more apnea/periodic breathing/intermittent hypoxemia. Infants never receiving mechanical ventilation followed similar postnatal trajectories but with less apnea and intermittent hypoxemia, and more periodic breathing. CONCLUSIONS Cardioventilatory events peak during the first month of life but the actual postnatal trajectory is dependent on the type of event, race, sex and use of mechanical ventilation. IMPACT Physiologic curves of cardiorespiratory events in extremely preterm-born infants offer (1) objective measures to assess individual patient courses and (2) guides for research into control of ventilation, biomarkers and outcomes. Presented are updated maturational trajectories of apnea, periodic breathing, intermittent hypoxemia, and bradycardia in 717 infants born <29 weeks gestation from the multi-site NHLBI-funded Pre-Vent study. Cardioventilatory events peak during the first month of life but the actual postnatal trajectory is dependent on the type of event, race, sex and use of mechanical ventilation. Different time courses for apnea and periodic breathing suggest different maturational mechanisms.
Collapse
Affiliation(s)
- Debra E Weese-Mayer
- Department of Pediatrics, Northwestern University Feinberg School of Medicine, Chicago, IL, USA.
- Division of Autonomic Medicine, Department of Pediatrics, Ann & Robert H Lurie Children's Hospital of Chicago and Stanley Manne Children's Research Institute, Chicago, IL, USA.
| | - Juliann M Di Fiore
- Department of Pediatrics, Case Western Reserve University, School of Medicine, Cleveland, OH, USA.
- Department of Pediatrics, Division of Neonatology, UH Rainbow Babies & Children's Hospital, Cleveland, OH, USA.
| | - Douglas E Lake
- Division of Cardiovascular Medicine, Center for Advanced Medical Analytics and Department of Internal Medicine, University of Virginia School of Medicine, Charlottesville, VA, USA
| | - Anna Maria Hibbs
- Department of Pediatrics, Case Western Reserve University, School of Medicine, Cleveland, OH, USA
- Department of Pediatrics, Division of Neonatology, UH Rainbow Babies & Children's Hospital, Cleveland, OH, USA
| | - Nelson Claure
- Division of Neonatology, Department of Pediatrics, Holtz Children's Hospital - Jackson Memorial Medical Center, University of Miami Miller School of Medicine, Miami, FL, USA
| | - Jiaxing Qiu
- Division of Cardiovascular Medicine, Center for Advanced Medical Analytics and Department of Internal Medicine, University of Virginia School of Medicine, Charlottesville, VA, USA
| | - Namasivayam Ambalavanan
- Division of Neonatology, Department of Pediatrics, University of Alabama at Birmingham School of Medicine, Birmingham, AL, USA
| | - Eduardo Bancalari
- Division of Neonatology, Department of Pediatrics, Holtz Children's Hospital - Jackson Memorial Medical Center, University of Miami Miller School of Medicine, Miami, FL, USA
| | - James S Kemp
- Department of Pediatrics, Washington University School of Medicine in St. Louis, St. Louis, MO, USA
| | - Amanda M Zimmet
- Division of Cardiovascular Medicine, Center for Advanced Medical Analytics and Department of Internal Medicine, University of Virginia School of Medicine, Charlottesville, VA, USA
| | - John L Carroll
- Department of Pediatrics, University of Arkansas for Medical Sciences, Little Rock, AR, USA
| | - Richard J Martin
- Department of Pediatrics, Case Western Reserve University, School of Medicine, Cleveland, OH, USA
- Department of Pediatrics, Division of Neonatology, UH Rainbow Babies & Children's Hospital, Cleveland, OH, USA
| | - Katy N Krahn
- Division of Cardiovascular Medicine, Center for Advanced Medical Analytics and Department of Internal Medicine, University of Virginia School of Medicine, Charlottesville, VA, USA
| | - Aaron Hamvas
- Department of Pediatrics, Northwestern University Feinberg School of Medicine, Chicago, IL, USA
- Division of Neonatology, Department of Pediatrics, Ann & Robert H Lurie Children's Hospital of Chicago and Stanley Manne Children's Research Institute, Chicago, IL, USA
| | - Sarah J Ratcliffe
- Public Health Sciences, University of Virginia School of Medicine, Charlottesville, VA, USA
| | - Narayanan Krishnamurthi
- Division of Autonomic Medicine, Department of Pediatrics, Ann & Robert H Lurie Children's Hospital of Chicago and Stanley Manne Children's Research Institute, Chicago, IL, USA
| | - Premananda Indic
- Department of Electrical Engineering, University of Texas Tyler, Tyler, TX, USA
| | - Alaleh Dormishian
- Division of Neonatology, Department of Pediatrics, Holtz Children's Hospital - Jackson Memorial Medical Center, University of Miami Miller School of Medicine, Miami, FL, USA
| | - Phyllis A Dennery
- Hasbro Children's Hospital, Brown University, Warren Alpert School of Medicine, Providence, RI, USA
| | - J Randall Moorman
- Division of Cardiovascular Medicine, Center for Advanced Medical Analytics and Department of Internal Medicine, University of Virginia School of Medicine, Charlottesville, VA, USA
- Public Health Sciences, University of Virginia School of Medicine, Charlottesville, VA, USA
| |
Collapse
|
12
|
Zhang H, Wang C, Yang N. Diagnostic performance of machine-learning algorithms for sepsis prediction: An updated meta-analysis. Technol Health Care 2024; 32:4291-4307. [PMID: 38968031 PMCID: PMC11613038 DOI: 10.3233/thc-240087] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/10/2024] [Accepted: 03/02/2024] [Indexed: 07/07/2024]
Abstract
BACKGROUND Early identification of sepsis has been shown to significantly improve patient prognosis. OBJECTIVE Therefore, the aim of this meta-analysis is to systematically evaluate the diagnostic efficacy of machine-learning algorithms for sepsis prediction. METHODS Systematic searches were conducted in PubMed, Embase and Cochrane databases, covering literature up to December 2023. The keywords included machine learning, sepsis and prediction. After screening, data were extracted and analysed from studies meeting the inclusion criteria. Key evaluation metrics included sensitivity, specificity and the area under the curve (AUC) for diagnostic accuracy. RESULTS The meta-analysis included a total of 21 studies with a data sample size of 4,158,941. Overall, the pooled sensitivity was 0.82 (95% confidence interval [CI] = 0.70-0.90; P< 0.001; I2= 99.7%), the specificity was 0.91 (95% CI = 0.86-0.94; P< 0.001; I2= 99.9%), and the AUC was 0.94 (95% CI = 0.91-0.96). The subgroup analysis revealed that in the emergency department setting (6 studies), the pooled sensitivity was 0.79 (95% CI = 0.68-0.87; P< 0.001; I2= 99.6%), the specificity was 0.94 (95% CI 0.90-0.97; P< 0.001; I2= 99.9%), and the AUC was 0.94 (95% CI = 0.92-0.96). In the Intensive Care Unit setting (11 studies), the sensitivity was 0.91 (95% CI = 0.75-0.97; P< 0.001; I2= 98.3%), the specificity was 0.85 (95% CI = 0.75-0.92; P< 0.001; I2= 99.9%), and the AUC was 0.93 (95% CI = 0.91-0.95). Due to the limited number of studies in the in-hospital and mixed settings (n< 3), no pooled analysis was performed. CONCLUSION Machine-learning algorithms have demonstrated excellent diagnostic accuracy in predicting the occurrence of sepsis, showing potential for clinical application.
Collapse
Affiliation(s)
| | | | - Ning Yang
- Department of Pharmacy, Zhang Jiakou First Hospital, Zhangjiakou, Hebei, China
| |
Collapse
|
13
|
Speier RL, Cotten CM, Benjamin DK, Lewis K, Keeler K, Kidimbu G, Roberts W, Clark RH, Zimmerman KO, Stark A, Greenberg RG. Late-Onset Sepsis Evaluation and Empiric Therapy in Extremely Low Gestational Age Newborns. J Pediatric Infect Dis Soc 2023; 12:S37-S43. [PMID: 38146858 DOI: 10.1093/jpids/piad068] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 06/20/2023] [Accepted: 09/12/2023] [Indexed: 12/27/2023]
Abstract
BACKGROUND Little is known about late-onset sepsis (LOS) evaluations in extremely low gestational age newborns (ELGANs). We describe frequencies of LOS evaluation in ELGANs, infant characteristics, and empiric therapy choices during evaluations. METHODS Cohort study of infants 22-28 weeks gestational age (GA) discharged from 243 centers from 2009 to 2018, excluding infants with congenital anomalies, discharged or deceased prior to postnatal day (PND) 2, or admitted after PND 2. A new LOS evaluation was defined as the first blood culture obtained between PND 3 and 90, or one obtained ≥1 day following a negative culture and ≥10 days from prior positive cultures. We determined numbers of evaluations and percentage positive by GA, center, and over time. We described characteristics associated with positive evaluations, infants with LOS, and empiric antimicrobials. We calculated descriptive and comparative statistics using Wilcoxon rank sum, Fisher's exact, or Pearson chi-square tests, as appropriate. RESULTS Of 47,187 included infants, 67% had ≥1 LOS evaluation and 21% of evaluated infants had ≥1 LOS (culture positive) episode; 1.6 evaluations occurred per infant and 10% were positive. The percentage of infants evaluated and positive for LOS was higher at earlier GA. LOS was associated with inotrope support (15% vs. 9%; p < .001) and invasive mechanical ventilation (66% vs. 51%; p < .001). Infants with positive cultures were more likely than infants with negative cultures to receive empiric antimicrobials during the LOS evaluation (95% vs. 73%; p < .001). CONCLUSIONS Among ELGANs, earlier GA and postnatal age were associated with LOS evaluation and positive cultures. Most infants undergoing evaluation were started on empiric antimicrobials.
Collapse
Affiliation(s)
| | | | - Daniel K Benjamin
- Department of Medicine, Duke University School of Medicine
- Duke Clinical Research Institute, Durham, NC, USA
| | - Kelsey Lewis
- Duke Clinical Research Institute, Durham, NC, USA
| | | | | | | | - Reese H Clark
- Pediatrix Center for Research, Education, Quality, and Safety, Sunrise, FL, USA
| | - Kanecia O Zimmerman
- Department of Medicine, Duke University School of Medicine
- Duke Clinical Research Institute, Durham, NC, USA
| | - Ashley Stark
- Department of Medicine, Duke University School of Medicine
| | - Rachel G Greenberg
- Department of Medicine, Duke University School of Medicine
- Duke Clinical Research Institute, Durham, NC, USA
| |
Collapse
|
14
|
Sullivan BA, Hochheimer CJ, Chernyavskiy P, King WE, Fairchild KD. Impact of race on heart rate characteristics monitoring in very low birth weight infants. Pediatr Res 2023; 94:575-580. [PMID: 36650306 PMCID: PMC10350468 DOI: 10.1038/s41390-023-02470-z] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 06/13/2022] [Revised: 12/14/2022] [Accepted: 12/21/2022] [Indexed: 01/18/2023]
Abstract
BACKGROUND A multicenter RCT showed that displaying a heart rate characteristics index (HRCi) predicting late-onset sepsis reduced mortality for VLBW infants. We aimed to assess whether HRCi display had a differential impact for Black versus White infants. METHODS We performed secondary data analysis of Black and White infants enrolled in the HeRO RCT. We evaluated the predictive performance of the HRCi for infants with Black or White maternal race. Using models adjusted for birth weight, we assessed outcomes and interventions for a race × randomization interaction. RESULTS Among 2607 infants, Black infants had lower birth weight, gestational age, length of stay, and ventilator days, while sepsis and mortality were similar. The HRCi performed equally for sepsis prediction in Black and White infants. We found no differential effect of randomization by race on sepsis, mortality, antibiotic days, length of stay, or ventilator days. However, there was a differential randomization effect by race for blood cultures per patient: White RR 1.11 (95% CrI 1.04-1.18), Black RR 1.00 (0.93-1.07). CONCLUSIONS The HRCi performed similarly for sepsis prediction in Black and White infants. Randomization to HRCi display increased blood cultures in White but not in Black infants, while the impact on other outcomes or interventions was similar. IMPACT Predictive analytics, such as heart rate characteristics (HRC) monitoring for late-onset neonatal sepsis, should have equal impact among patients of different race. Infants with Black or White maternal race randomized to HRC display had similar outcomes, but randomization to the study arm increased a related clinical intervention, blood cultures, in White but not in Black infants. This study provides evidence of a differential effect of predictive models on clinical care by race. The work will promote consideration and analysis of equity in the implementation of predictive analytics.
Collapse
Affiliation(s)
- Brynne A Sullivan
- Division of Neonatology, Department of Pediatrics, University of Virginia School of Medicine, Charlottesville, VA, USA.
| | | | - Pavel Chernyavskiy
- Department of Public Health Sciences, University of Virginia School of Medicine, Charlottesville, VA, USA
| | - William E King
- Medical Predictive Sciences Corporation, Charlottesville, VA, USA
| | - Karen D Fairchild
- Division of Neonatology, Department of Pediatrics, University of Virginia School of Medicine, Charlottesville, VA, USA
| |
Collapse
|
15
|
Koppens HJ, Onland W, Visser DH, Denswil NP, van Kaam AH, Lutterman CA. Heart Rate Characteristics Monitoring for Late-Onset Sepsis in Preterm Infants: A Systematic Review. Neonatology 2023; 120:548-557. [PMID: 37379804 PMCID: PMC10614451 DOI: 10.1159/000531118] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 02/16/2023] [Accepted: 05/03/2023] [Indexed: 06/30/2023]
Abstract
BACKGROUND Early diagnosis of late-onset sepsis (LOS) and necrotizing enterocolitis (NEC) by monitoring heart rate characteristics (HRC) of preterm infants might reduce the risk of death and morbidities. We aimed to systematically assess the effects of HRC monitoring on death, LOS, and NEC. METHODS A systematic search was performed in MEDLINE, Embase, Cochrane Library, and Web of Science. RESULTS Fifteen papers were included in this review. Three of these papers reported results from the only identified randomized controlled trial (RCT). This RCT showed that HRC monitoring resulted in a small but significant reduction in mortality (absolute risk reduction 2.1% [95% confidence interval 0.01-4.14]) without any differences in neurodevelopmental impairment. The risk of bias was rated high due to performance and detection bias and failure to correct for multiple testing. Most diagnostic cohort studies showed high discriminating accuracy in predicting LOS but lacked sufficient quality and generalizability. No studies for the detection of NEC were identified. CONCLUSION Supported by multiple observational cohort studies, the RCT identified in this systematic review showed that HRC monitoring as an early warning system for LOS might reduce the risk of death in preterm infants. However, methodological weaknesses and limited generalizability do not justify implementation of HRC in clinical care. A large international RCT is warranted.
Collapse
Affiliation(s)
- Hugo J. Koppens
- Department of Neonatology, Emma Children’s Hospital, Amsterdam UMC location University of Amsterdam, Amsterdam, The Netherlands
- Amsterdam Reproduction and Development, Amsterdam, The Netherlands
| | - Wes Onland
- Department of Neonatology, Emma Children’s Hospital, Amsterdam UMC location University of Amsterdam, Amsterdam, The Netherlands
- Amsterdam Reproduction and Development, Amsterdam, The Netherlands
| | - Douwe H. Visser
- Department of Neonatology, Emma Children’s Hospital, Amsterdam UMC location University of Amsterdam, Amsterdam, The Netherlands
- Amsterdam Reproduction and Development, Amsterdam, The Netherlands
| | - Nerissa P. Denswil
- Amsterdam UMC Location University of Amsterdam, Medical Library, Amsterdam, The Netherlands
| | - Anton H. van Kaam
- Department of Neonatology, Emma Children’s Hospital, Amsterdam UMC location University of Amsterdam, Amsterdam, The Netherlands
- Amsterdam Reproduction and Development, Amsterdam, The Netherlands
| | | |
Collapse
|
16
|
Kausch SL, Brandberg JG, Qiu J, Panda A, Binai A, Isler J, Sahni R, Vesoulis ZA, Moorman JR, Fairchild KD, Lake DE, Sullivan BA. Cardiorespiratory signature of neonatal sepsis: development and validation of prediction models in 3 NICUs. Pediatr Res 2023; 93:1913-1921. [PMID: 36593281 PMCID: PMC10314957 DOI: 10.1038/s41390-022-02444-7] [Citation(s) in RCA: 17] [Impact Index Per Article: 8.5] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 08/17/2022] [Revised: 11/21/2022] [Accepted: 12/12/2022] [Indexed: 01/03/2023]
Abstract
BACKGROUND Heart rate characteristics aid early detection of late-onset sepsis (LOS), but respiratory data contain additional signatures of illness due to infection. Predictive models using cardiorespiratory data may improve early sepsis detection. We hypothesized that heart rate (HR) and oxygenation (SpO2) data contain signatures that improve sepsis risk prediction over HR or demographics alone. METHODS We analyzed cardiorespiratory data from very low birth weight (VLBW, <1500 g) infants admitted to three NICUs. We developed and externally validated four machine learning models to predict LOS using features calculated every 10 m: mean, standard deviation, skewness, kurtosis of HR and SpO2, and cross-correlation. We compared feature importance, discrimination, calibration, and dynamic prediction across models and cohorts. We built models of demographics and HR or SpO2 features alone for comparison with HR-SpO2 models. RESULTS Performance, feature importance, and calibration were similar among modeling methods. All models had favorable external validation performance. The HR-SpO2 model performed better than models using either HR or SpO2 alone. Demographics improved the discrimination of all physiologic data models but dampened dynamic performance. CONCLUSIONS Cardiorespiratory signatures detect LOS in VLBW infants at 3 NICUs. Demographics risk-stratify, but predictive modeling with both HR and SpO2 features provides the best dynamic risk prediction. IMPACT Heart rate characteristics aid early detection of late-onset sepsis, but respiratory data contain signatures of illness due to infection. Predictive models using both heart rate and respiratory data may improve early sepsis detection. A cardiorespiratory early warning score, analyzing heart rate from electrocardiogram or pulse oximetry with SpO2, predicts late-onset sepsis within 24 h across multiple NICUs and detects sepsis better than heart rate characteristics or demographics alone. Demographics risk-stratify, but predictive modeling with both HR and SpO2 features provides the best dynamic risk prediction. The results increase understanding of physiologic signatures of neonatal sepsis.
Collapse
Affiliation(s)
- Sherry L Kausch
- Department of Pediatrics, University of Virginia School of Medicine, Charlottesville, VA, USA.
| | - Jackson G Brandberg
- Department of Medicine, University of Virginia School of Medicine, Charlottesville, VA, USA
| | - Jiaxing Qiu
- Department of Medicine, University of Virginia School of Medicine, Charlottesville, VA, USA
| | - Aneesha Panda
- Department of Pediatrics, Washington University in St. Louis School of Medicine, St. Louis, MO, USA
| | - Alexandra Binai
- Department of Pediatrics, Washington University in St. Louis School of Medicine, St. Louis, MO, USA
| | - Joseph Isler
- Department of Pediatrics, Columbia University Vagelos College of Physicians and Surgeons, New York, NY, USA
| | - Rakesh Sahni
- Department of Pediatrics, Columbia University Vagelos College of Physicians and Surgeons, New York, NY, USA
| | - Zachary A Vesoulis
- Department of Pediatrics, Washington University in St. Louis School of Medicine, St. Louis, MO, USA
| | - J Randall Moorman
- Department of Medicine, University of Virginia School of Medicine, Charlottesville, VA, USA
| | - Karen D Fairchild
- Department of Pediatrics, University of Virginia School of Medicine, Charlottesville, VA, USA
| | - Douglas E Lake
- Department of Medicine, University of Virginia School of Medicine, Charlottesville, VA, USA
| | - Brynne A Sullivan
- Department of Pediatrics, University of Virginia School of Medicine, Charlottesville, VA, USA
| |
Collapse
|
17
|
Zeigler AC, Ainsworth JE, Fairchild KD, Wynn JL, Sullivan BA. Sepsis and Mortality Prediction in Very Low Birth Weight Infants: Analysis of HeRO and nSOFA. Am J Perinatol 2023; 40:407-414. [PMID: 33971672 PMCID: PMC8578589 DOI: 10.1055/s-0041-1728829] [Citation(s) in RCA: 18] [Impact Index Per Article: 9.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Indexed: 12/29/2022]
Abstract
OBJECTIVE Scores to predict sepsis or define sepsis severity could improve care for very low birth weight (VLBW) infants. The heart rate characteristics (HRC) index (HeRO score) was developed as an early warning system for late-onset sepsis (LOS), and also rises before necrotizing enterocolitis (NEC). The neonatal sequential organ failure assessment (nSOFA) was developed to predict sepsis-associated mortality using respiratory, hemodynamic, and hematologic data. The aim of this study was to analyze the HRC index and nSOFA near blood cultures in VLBW infants relative to diagnosis and sepsis-associated mortality. STUDY DESIGN Retrospective, single-center study of VLBW infants from 2011 to 2019. We analyzed HRC index and nSOFA around blood cultures diagnosed as LOS/NEC. In a subgroup of the cohort, we analyzed HRC and nSOFA near the first sepsis-like illness (SLI) or sepsis ruled-out (SRO) compared with LOS/NEC. We compared scores by diagnosis and mortality during treatment. RESULTS We analyzed 179 LOS/NEC, 93 SLI, and 96 SRO blood culture events. In LOS/NEC, the HRC index increased before the blood culture, while nSOFA increased at the time of culture. Both scores were higher in nonsurvivors compared with survivors and in LOS/NEC compared with SRO. The nSOFA 12 hours after the time of blood culture predicted mortality during treatment better than any other time point analyzed (area under the curve 0.91). CONCLUSION The HRC index provides earlier warning of imminent sepsis, whereas nSOFA after blood culture provides better prediction of mortality. KEY POINTS · The HRC index and nSOFA provide complementary information on sepsis risk and sepsis-related mortality risk.. · This study adds to existing literature evaluating these risk scores independently by analyzing them together and in cases of not only proven but also suspected infections.. · The impact of combining risk models could be improved outcomes for premature infants..
Collapse
Affiliation(s)
- Angela C. Zeigler
- University of Virginia School of Medicine, Charlottesville, Virginia
- Department of Pediatrics, Yale School of Medicine, New Haven, Connecticut
| | - John E. Ainsworth
- University of Virginia School of Medicine, Charlottesville, Virginia
| | - Karen D. Fairchild
- Division of Neonatology, Department of Pediatrics, University of Virginia School of Medicine, Charlottesville, Virginia
| | - James L. Wynn
- Division of Neonatology, Department of Pediatrics, University of Florida School of Medicine, Gainesville, Florida
| | - Brynne A. Sullivan
- Division of Neonatology, Department of Pediatrics, University of Virginia School of Medicine, Charlottesville, Virginia
| |
Collapse
|
18
|
Monfredi OJ, Moore CC, Sullivan BA, Keim-Malpass J, Fairchild KD, Loftus TJ, Bihorac A, Krahn KN, Dubrawski A, Lake DE, Moorman JR, Clermont G. Continuous ECG monitoring should be the heart of bedside AI-based predictive analytics monitoring for early detection of clinical deterioration. J Electrocardiol 2023; 76:35-38. [PMID: 36434848 PMCID: PMC10061545 DOI: 10.1016/j.jelectrocard.2022.10.011] [Citation(s) in RCA: 2] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/27/2022] [Revised: 10/01/2022] [Accepted: 10/22/2022] [Indexed: 11/24/2022]
Abstract
The idea that we can detect subacute potentially catastrophic illness earlier by using statistical models trained on clinical data is now well-established. We review evidence that supports the role of continuous cardiorespiratory monitoring in these predictive analytics monitoring tools. In particular, we review how continuous ECG monitoring reflects the patient and not the clinician, is less likely to be biased, is unaffected by changes in practice patterns, captures signatures of illnesses that are interpretable by clinicians, and is an underappreciated and underutilized source of detailed information for new mathematical methods to reveal.
Collapse
Affiliation(s)
- Oliver J Monfredi
- Center for Advanced Medical Analytics, University of Virginia, United States of America; Department of Medicine, University of Virginia, United States of America
| | - Christopher C Moore
- Center for Advanced Medical Analytics, University of Virginia, United States of America; Department of Medicine, University of Virginia, United States of America
| | - Brynne A Sullivan
- Center for Advanced Medical Analytics, University of Virginia, United States of America; Department of Pediatrics, University of Virginia, United States of America
| | - Jessica Keim-Malpass
- Center for Advanced Medical Analytics, University of Virginia, United States of America; School of Nursing, University of Virginia, United States of America
| | - Karen D Fairchild
- Center for Advanced Medical Analytics, University of Virginia, United States of America; Department of Pediatrics, University of Virginia, United States of America
| | - Tyler J Loftus
- Department of Surgery, University of Florida, United States of America
| | - Azra Bihorac
- Department of Medicine, University of Florida, United States of America
| | - Katherine N Krahn
- Center for Advanced Medical Analytics, University of Virginia, United States of America; Department of Medicine, University of Virginia, United States of America
| | - Artur Dubrawski
- Robotics Institute, Carnegie Mellon University, United States of America
| | - Douglas E Lake
- Center for Advanced Medical Analytics, University of Virginia, United States of America; Department of Medicine, University of Virginia, United States of America
| | - J Randall Moorman
- Center for Advanced Medical Analytics, University of Virginia, United States of America; Department of Medicine, University of Virginia, United States of America.
| | - Gilles Clermont
- Department of Critical Care, University of Pittsburgh, United States of America
| |
Collapse
|
19
|
Sullivan BA, Kausch SL, Fairchild KD. Artificial and human intelligence for early identification of neonatal sepsis. Pediatr Res 2023; 93:350-356. [PMID: 36127407 PMCID: PMC11749885 DOI: 10.1038/s41390-022-02274-7] [Citation(s) in RCA: 3] [Impact Index Per Article: 1.5] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 04/18/2022] [Revised: 07/29/2022] [Accepted: 08/05/2022] [Indexed: 11/09/2022]
Abstract
Artificial intelligence may have a role in the early detection of sepsis in neonates. Machine learning can identify patterns that predict high or increasing risk for clinical deterioration from a sepsis-like illness. In developing this potential addition to NICU care, careful consideration should be given to the data and methods used to develop, validate, and evaluate prediction models. When an AI system alerts clinicians to a change in a patient's condition that warrants a bedside evaluation, human intelligence and experience come into play to determine an appropriate course of action: evaluate and treat or wait and watch closely. With intelligently developed, validated, and implemented AI sepsis systems, both clinicians and patients stand to benefit. IMPACT: This narrative review highlights the application of AI in neonatal sepsis prediction. It describes issues in clinical prediction model development specific to this population. This article reviews the methods, considerations, and literature on neonatal sepsis model development and validation. Challenges of AI technology and potential barriers to using sepsis AI systems in the NICU are discussed.
Collapse
Affiliation(s)
- Brynne A Sullivan
- Department of Pediatrics, University of Virginia School of Medicine, Charlottesville, VA, USA.
| | - Sherry L Kausch
- Department of Pediatrics, University of Virginia School of Medicine, Charlottesville, VA, USA
| | - Karen D Fairchild
- Department of Pediatrics, University of Virginia School of Medicine, Charlottesville, VA, USA
| |
Collapse
|
20
|
Abstract
Neonatal late-onset sepsis (LOS) continues to threaten morbidity and mortality in the NICU and poses ongoing diagnostic and therapeutic challenges. Early recognition of clinical signs, rapid evaluation, and prompt initiation of treatment are critical to prevent life-threatening deterioration. Preterm infants-born at ever-decreasing gestational ages-are at particularly high risk for life-long morbidities and death. This changing NICU population necessitates continual reassessments of diagnostic and preventive measures and evidence-based treatment for LOS. The clinical presentation of LOS is varied and nonspecific. Despite ongoing research, reliable, specific laboratory biomarkers facilitating early diagnosis are lacking. These limitations drive an ongoing practice of liberal initiation of empiric antibiotics among infants with suspected LOS. Subsequent promotion of multidrug-resistant microorganisms threatens the future of antimicrobial therapy and puts preterm and chronically ill infants at even higher risk of nosocomial infection. Efforts to identify adjunctive therapies counteracting sepsis-driven hyperinflammation and sepsis-related functional immunosuppression are ongoing. However, most approaches have either failed to improve LOS prognosis or are not yet ready for clinical application. This article provides an overview of the epidemiology, risk factors, diagnostic tools, and treatment options of LOS in the context of increasing numbers of extremely preterm infants. It addresses the question of whether LOS could be identified earlier and more precisely to allow for earlier and more targeted therapy and discusses rational approaches to antibiotic therapy to avoid overuse. Finally, this review elucidates the necessity of long-term follow-up of infants with a history of LOS.
Collapse
Affiliation(s)
- Sarah A. Coggins
- Division of Neonatology, The Children’s Hospital of Philadelphia, Philadelphia, PA, USA
| | - Kirsten Glaser
- Division of Neonatology, Department of Women’s and Children’s Health, University of Leipzig Medical Center, Leipzig, Germany
| |
Collapse
|
21
|
Sullivan BA, Fairchild KD. Vital signs as physiomarkers of neonatal sepsis. Pediatr Res 2022; 91:273-282. [PMID: 34493832 DOI: 10.1038/s41390-021-01709-x] [Citation(s) in RCA: 21] [Impact Index Per Article: 7.0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 05/04/2021] [Revised: 07/24/2021] [Accepted: 07/27/2021] [Indexed: 02/08/2023]
Abstract
Neonatal sepsis accounts for significant morbidity and mortality, particularly among premature infants in the Neonatal Intensive Care Unit. Abnormal vital sign patterns serve as physiomarkers of sepsis and provide early warning of illness before overt clinical decompensation. The systemic inflammatory response to pathogens signals the autonomic nervous system, leading to changes in temperature, respiratory rate, heart rate, and blood pressure. In infants with comorbidities of prematurity, vital sign abnormalities often occur in the absence of infection, which confounds sepsis diagnosis. This review will cover the mechanisms of vital sign changes in neonatal sepsis, including the cholinergic anti-inflammatory pathway mediated by the vagus nerve, which is critical to the host response to infectious and inflammatory insults. We will also review the clinical implications of vital sign changes in neonatal sepsis, including their use in early warning scores and systems to direct clinicians to the bedside of infants with physiologic changes that might be due to sepsis. IMPACT: This manuscript summarizes and reviews the relevant literature on the physiological manifestations of neonatal sepsis and how we monitor and analyze these through vital signs and advanced analytics.
Collapse
Affiliation(s)
- Brynne A Sullivan
- Division of Neonatology, Department of Pediatrics, University of Virginia School of Medicine, Charlottesville, VA, USA.
| | - Karen D Fairchild
- Division of Neonatology, Department of Pediatrics, University of Virginia School of Medicine, Charlottesville, VA, USA
| |
Collapse
|
22
|
Knowledge gaps in late-onset neonatal sepsis in preterm neonates: a roadmap for future research. Pediatr Res 2022; 91:368-379. [PMID: 34497356 DOI: 10.1038/s41390-021-01721-1] [Citation(s) in RCA: 6] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 05/19/2021] [Revised: 08/13/2021] [Accepted: 08/17/2021] [Indexed: 12/16/2022]
Abstract
Late-onset neonatal sepsis (LONS) remains an important threat to the health of preterm neonates in the neonatal intensive care unit. Strategies to optimize care for preterm neonates with LONS are likely to improve survival and long-term neurocognitive outcomes. However, many important questions on how to improve the prevention, early detection, and therapy for LONS in preterm neonates remain unanswered. This review identifies important knowledge gaps in the management of LONS and describe possible methods and technologies that can be used to resolve these knowledge gaps. The availability of computational medicine and hypothesis-free-omics approaches give way to building bedside feedback tools to guide clinicians in personalized management of LONS. Despite advances in technology, implementation in clinical practice is largely lacking although such tools would help clinicians to optimize many aspects of the management of LONS. We outline which steps are needed to get possible research findings implemented on the neonatal intensive care unit and provide a roadmap for future research initiatives. IMPACT: This review identifies knowledge gaps in prevention, early detection, antibiotic, and additional therapy of late-onset neonatal sepsis in preterm neonates and provides a roadmap for future research efforts. Research opportunities are addressed, which could provide the means to fill knowledge gaps and the steps that need to be made before possible clinical use. Methods to personalize medicine and technologies feasible for bedside clinical use are described.
Collapse
|
23
|
Abstract
PURPOSE OF REVIEW Neonatal bloodstream infections (BSI) are a major contributor to morbidity and mortality within neonatal intensive care units. BSI, including central line-associated BSI, have decreased over the past 15 years but remain common in extremely preterm infants. The purpose of this review is to highlight recent advances in the causes, diagnosis, management, and prevention of neonatal BSI. RECENT FINDINGS Continued quality improvement efforts and bundles have reduced BSI incidence, and novel approaches are highlighted. An update of emerging pathogens as well as traditional pathogens with novel antimicrobial resistance, which are an increasingly common cause of neonatal BSI, is included. Finally, current and future investigations into serum or noninvasive biomarkers for neonatal BSI are reviewed. SUMMARY Neonatal BSIs continue to decrease due to enhanced infection control and prevention techniques. However, many challenges remain, including emerging bacterial and fungal resistance and the continued need for novel diagnostics that hasten time to pathogen identification and effective treatment. This review of the past 18 months highlights the rapid changes in this area. Ongoing efforts to reduce the morbidity and mortality caused by neonatal BSI must remain a priority.
Collapse
Affiliation(s)
| | - Joseph B Cantey
- Department of Pediatrics, Division of Neonatology
- Department of Pediatrics, Division of Pediatric Infectious Diseases, University of Texas Health San Antonio, San Antonio, Texas, USA
| |
Collapse
|