Simulation of Ventilator Allocation in Critically Ill Patients with COVID-19

Web-based Dashboard on ECMO Utilization in Germany: An Interactive Visualization, Analyses, and Prediction Based on Real-life Data

In Germany, a comprehensive reimbursement policy for extracorporeal membrane oxygenation (ECMO) results in the highest per capita use worldwide, although benefits remain controversial. Public ECMO data is unstructured and poorly accessible to healthcare professionals, researchers, and policymakers. In addition, there are no uniform policies for ECMO allocation which confronts medical personnel with ethical considerations during health crises such as respiratory virus outbreaks.Retrospective information on adult and pediatric ECMO support performed in German hospitals was extracted from publicly available reimbursement data and hospital quality reports and processed to create the web-based ECMO Dashboard built on Open-Source software. Patient-level and hospital-level data were merged resulting in a solid base for ECMO use analysis and ECMO demand forecasting with high spatial granularity at the level of 413 county and city districts in Germany.The ECMO Dashboard ( https://www.ecmo-dash.de/ ), an innovative visual platform, presents the retrospective utilization patterns of ECMO support in Germany. It features interactive maps, comprehensive charts, and tables, providing insights at the hospital, district, and national levels. This tool also highlights the high prevalence of ECMO support in Germany and emphasizes districts with ECMO surplus - where patients from other regions are treated, or deficit - origins from which ECMO patients are transferred to other regions. The dashboard will evolve iteratively to provide stakeholders with vital information for informed and transparent resource allocation and decision-making.Accessible public routine data could support evidence-informed, forward-looking resource management policies, which are urgently needed to increase the quality and prepare the critical care infrastructure for future pandemics.

An AI-based multiphase framework for improving the mechanical ventilation availability in emergency departments during respiratory disease seasons: a case study

BACKGROUND

Shortages of mechanical ventilation have become a constant problem in Emergency Departments (EDs), thereby affecting the timely deployment of medical interventions that counteract the severe health complications experienced during respiratory disease seasons. It is then necessary to count on agile and robust methodological approaches predicting the expected demand loads to EDs while supporting the timely allocation of ventilators. In this paper, we propose an integration of Artificial Intelligence (AI) and Discrete-event Simulation (DES) to design effective interventions ensuring the high availability of ventilators for patients needing these devices.

METHODS

First, we applied Random Forest (RF) to estimate the mechanical ventilation probability of respiratory-affected patients entering the emergency wards. Second, we introduced the RF predictions into a DES model to diagnose the response of EDs in terms of mechanical ventilator availability. Lately, we pretested two different interventions suggested by decision-makers to address the scarcity of this resource. A case study in a European hospital group was used to validate the proposed methodology.

RESULTS

The number of patients in the training cohort was 734, while the test group comprised 315. The sensitivity of the AI model was 93.08% (95% confidence interval, [88.46 - 96.26%]), whilst the specificity was 85.45% [77.45 - 91.45%]. On the other hand, the positive and negative predictive values were 91.62% (86.75 - 95.13%) and 87.85% (80.12 - 93.36%). Also, the Receiver Operator Characteristic (ROC) curve plot was 95.00% (89.25 - 100%). Finally, the median waiting time for mechanical ventilation was decreased by 17.48% after implementing a new resource capacity strategy.

CONCLUSIONS

Combining AI and DES helps healthcare decision-makers to elucidate interventions shortening the waiting times for mechanical ventilators in EDs during respiratory disease epidemics and pandemics.

Investigating Ethical Tradeoffs in Crisis Standards of Care through Simulation of Ventilator Allocation Protocols

Introduction

Arguments over the appropriate Crisis Standards of Care (CSC) for public health emergencies often assume that there is a tradeoff between saving the most lives, saving the most life-years, and preventing racial disparities. However, these assumptions have rarely been explored empirically. To quantitatively characterize possible ethical tradeoffs, we aimed to simulate the implementation of five proposed CSC protocols for rationing ventilators in the context of the COVID-19 pandemic.

Methods

A Monte Carlo simulation was used to estimate the number of lives saved and life-years saved by implementing clinical acuity-, comorbidity- and age-based CSC protocols under different shortage conditions. This model was populated with patient data from 3707 adult admissions requiring ventilator support in a New York hospital system between April 2020 and May 2021. To estimate lives and life-years saved by each protocol, we determined survival to discharge and estimated remaining life expectancy for each admission.

Results

The simulation demonstrated stronger performance for age- and comorbidity-sensitive protocols. For a capacity of 1 bed per 2 patients, ranking by age bands saves approximately 28.7 lives and 3408 life-years per thousand patients, while ranking by Sequential Organ Failure Assessment (SOFA) bands saved the fewest lives (13.2) and life-years (416). For all protocols, we observed a positive correlation between lives saved and life-years saved. For all protocols except lottery and the banded SOFA, significant disparities in lives saved and life-years saved were noted between White non-Hispanic, Black non-Hispanic, and Hispanic sub-populations.

Conclusion

While there is significant variance in the number of lives saved and life-years saved, we did not find a tradeoff between saving the most lives and saving the most life-years. Moreover, concerns about racial discrimination in triage protocols require thinking carefully about the tradeoff between enforcing equality of survival rates and maximizing the lives saved in each sub-population.

Simulation of New York City's Ventilator Allocation Guideline During the Spring 2020 COVID-19 Surge

Importance

The spring 2020 surge of COVID-19 unprecedentedly strained ventilator supply in New York City, with many hospitals nearly exhausting available ventilators and subsequently seriously considering enacting crisis standards of care and implementing New York State Ventilator Allocation Guidelines (NYVAG). However, there is little evidence as to how NYVAG would perform if implemented.

Objectives

To evaluate the performance and potential improvement of NYVAG during a surge of patients with respect to the length of rationing, overall mortality, and worsening health disparities.

Design, Setting, and Participants

This cohort study included intubated patients in a single health system in New York City from March through July 2020. A total of 20 000 simulations were conducted of ventilator triage (10 000 following NYVAG and 10 000 following a proposed improved NYVAG) during a crisis period, defined as the point at which the prepandemic ventilator supply was 95% utilized.

Exposures

The NYVAG protocol for triage ventilators.

Main Outcomes and Measures

Comparison of observed survival rates with simulations of scenarios requiring NYVAG ventilator rationing.

Results

The total cohort included 1671 patients; of these, 674 intubated patients (mean [SD] age, 63.7 [13.8] years; 465 male [69.9%]) were included in the crisis period, with 571 (84.7%) testing positive for COVID-19. Simulated ventilator rationing occurred for 163.9 patients over 15.0 days, 44.4% (95% CI, 38.3%-50.0%) of whom would have survived if provided a ventilator while only 34.8% (95% CI, 28.5%-40.0%) of those newly intubated patients receiving a reallocated ventilator survived. While triage categorization at the time of intubation exhibited partial prognostic differentiation, 94.8% of all ventilator rationing occurred after a time trial. Within this subset, 43.1% were intubated for 7 or more days with a favorable SOFA score that had not improved. An estimated 60.6% of these patients would have survived if sustained on a ventilator. Revising triage subcategorization, proposed improved NYVAG, would have improved this alarming ventilator allocation inefficiency (25.3% [95% CI, 22.1%-28.4%] of those selected for ventilator rationing would have survived if provided a ventilator). NYVAG ventilator rationing did not exacerbate existing health disparities.

Conclusions and Relevance

In this cohort study of intubated patients experiencing simulated ventilator rationing during the apex of the New York City COVID-19 2020 surge, NYVAG diverted ventilators from patients with a higher chance of survival to those with a lower chance of survival. Future efforts should be focused on triage subcategorization, which improved this triage inefficiency, and ventilator rationing after a time trial, when most ventilator rationing occurred.

Artificial intelligence-based clinical decision support in pediatrics

Machine learning models may be integrated into clinical decision support (CDS) systems to identify children at risk of specific diagnoses or clinical deterioration to provide evidence-based recommendations. This use of artificial intelligence models in clinical decision support (AI-CDS) may have several advantages over traditional "rule-based" CDS models in pediatric care through increased model accuracy, with fewer false alerts and missed patients. AI-CDS tools must be appropriately developed, provide insight into the rationale behind decisions, be seamlessly integrated into care pathways, be intuitive to use, answer clinically relevant questions, respect the content expertise of the healthcare provider, and be scientifically sound. While numerous machine learning models have been reported in pediatric care, their integration into AI-CDS remains incompletely realized to date. Important challenges in the application of AI models in pediatric care include the relatively lower rates of clinically significant outcomes compared to adults, and the lack of sufficiently large datasets available necessary for the development of machine learning models. In this review article, we summarize key concepts related to AI-CDS, its current application to pediatric care, and its potential benefits and risks. IMPACT: The performance of clinical decision support may be enhanced by the utilization of machine learning-based algorithms to improve the predictive performance of underlying models. Artificial intelligence-based clinical decision support (AI-CDS) uses models that are experientially improved through training and are particularly well suited toward high-dimensional data. The application of AI-CDS toward pediatric care remains limited currently but represents an important area of future research.

Differences in US Regional Healthcare Allocation Guidelines During the COVID-19 Pandemic

BACKGROUND

Hospitals faced unprecedented scarcity of resources without parallel in modern times during the COVID-19 pandemic. This scarcity led healthcare systems and states to develop or modify scarce resource allocation guidelines that could be implemented during "crisis standards of care" (CSC). CSC describes a significant change in healthcare operations and the level of care provided during a public health emergency.

OBJECTIVE

Our study provides a comprehensive examination of the latest CSC guidelines in the western region of the USA, where Alaska and Idaho declared CSC, focusing on ethical issues and health disparities.

DESIGN

Mixed-methods survey study of physicians and/or ethicists and review of healthcare system and state allocation guidelines.

PARTICIPANTS

Ten physicians and/or ethicists who participated in scarce resource allocation guideline development from seven healthcare systems or three state-appointed committees from the western region of the USA including Alaska, California, Idaho, Oregon, and California.

RESULTS

All sites surveyed developed allocation guidelines, but only four (40%) were operationalized either statewide or for specific scarce resources. Most guidelines included comorbidities (70%), and half included adjustments for socioeconomic disadvantage (50%), while only one included specific priority groups (10%). Allocation tiebreakers included the life cycle principle and random number generators. Six guidelines evolved over time, removing restrictions such as age, severity of illness, and comorbidities. Additional palliative care (20%) and ethics (50%) resources were planned by some guidelines.

CONCLUSIONS

Allocation guidelines are essential to support clinicians during public health emergencies; however, significant deficits and differences in guidelines were identified that may perpetuate structural inequities and racism. While a universal triage protocol that is equally accepted by all communities is unlikely, the lack of regional agreement on standards with justification and transparency has the potential to erode public trust and perpetuate inequity.

Catastrophic COVID-19 Delta Variant Surge in French West Indies: Report of an ICU Triage Policy

OBJECTIVES

Here, we report the management of a catastrophic COVID-19 Delta variant surge, which overloaded ICU capacity, using crisis standards of care (CSC) based on a multiapproach protocol.

DESIGN

Retrospective observational study.

SETTING

University Hospital of Guadeloupe.

PATIENTS

This study retrospectively included all patients who were hospitalized for COVID-19 pneumonia between August 11, 2021, and September 10, 2021, and were eligible for ICU admission.

INTERVENTION

Based on age, comorbidities, and disease severity, patients were assigned to three groups: Green (ICU admission as soon as possible), Orange (ICU admission after the admission of all patients in the Green group), and Red (no ICU admission).

MEASUREMENTS AND MAIN RESULTS

Among the 328 patients eligible for ICU admission, 100 (30%) were assigned to the Green group, 116 (35%) to the Orange group, and 112 (34%) to the Red group. No patient in the Green group died while waiting for an ICU bed, whereas 14 patients (12%) in the Orange group died while waiting for an ICU bed. The 90-day mortality rates were 24%, 37%, and 78% in the Green, Orange, and Red groups, respectively. A total of 130 patients were transferred to the ICU, including 79 from the Green group, 51 from the Orange group, and none from the Red group. Multivariate analysis revealed that among patients admitted to the ICU, death was independently associated with a longer time between ICU referral and ICU admission, the Sequential Organ Failure Assessment score, and the number of comorbidities, but not with triage group.

CONCLUSIONS

CSC based on a multiapproach protocol allowed admission of all patients with a good prognosis. Higher mortality was associated with late admission, rather than triage group.

Hospital trajectories and early predictors of clinical outcomes differ between SARS-CoV-2 and influenza pneumonia

BACKGROUND

A comparison of pneumonias due to SARS-CoV-2 and influenza, in terms of clinical course and predictors of outcomes, might inform prognosis and resource management. We aimed to compare clinical course and outcome predictors in SARS-CoV-2 and influenza pneumonia using multi-state modelling and supervised machine learning on clinical data among hospitalised patients.

METHODS

This multicenter retrospective cohort study of patients hospitalised with SARS-CoV-2 (March-December 2020) or influenza (Jan 2015-March 2020) pneumonia had the composite of hospital mortality and hospice discharge as the primary outcome. Multi-state models compared differences in oxygenation/ventilatory utilisation between pneumonias longitudinally throughout hospitalisation. Differences in predictors of outcome were modelled using supervised machine learning classifiers.

FINDINGS

Among 2,529 hospitalisations with SARS-CoV-2 and 2,256 with influenza pneumonia, the primary outcome occurred in 21% and 9%, respectively. Multi-state models differentiated oxygen requirement progression between viruses, with SARS-CoV-2 manifesting rapidly-escalating early hypoxemia. Highly contributory classifier variables for the primary outcome differed substantially between viruses.

INTERPRETATION

SARS-CoV-2 and influenza pneumonia differ in presentation, hospital course, and outcome predictors. These pathogen-specific differential responses in viral pneumonias suggest distinct management approaches should be investigated.

FUNDING

This project was supported by NIH/NCATS UL1 TR002345, NIH/NCATS KL2 TR002346 (PGL), the Doris Duke Charitable Foundation grant 2015215 (PGL), NIH/NHLBI R35 HL140026 (CSC), and a Big Ideas Award from the BJC HealthCare and Washington University School of Medicine Healthcare Innovation Lab and NIH/NIGMS R35 GM142992 (PS).

Development and Internal Validation of a New Prognostic Model Powered to Predict 28-Day All-Cause Mortality in ICU COVID-19 Patients-The COVID-SOFA Score

Background: The sequential organ failure assessment (SOFA) score has poor discriminative ability for death in severely or critically ill patients with Coronavirus disease 2019 (COVID-19) requiring intensive care unit (ICU) admission. Our aim was to create a new score powered to predict 28-day mortality. Methods: Retrospective, observational, bicentric cohort study including 425 patients with COVID-19 pneumonia, acute respiratory failure and SOFA score ≥ 2 requiring ICU admission for ≥72 h. Factors with independent predictive value for 28-day mortality were identified after stepwise Cox proportional hazards (PH) regression. Based on the regression coefficients, an equation was computed representing the COVID-SOFA score. Discriminative ability was tested using receiver operating characteristic (ROC) analysis, concordance statistics and precision-recall curves. This score was internally validated. Results: Median (Q1−Q3) age for the whole sample was 64 [55−72], with 290 (68.2%) of patients being male. The 28-day mortality was 54.58%. After stepwise Cox PH regression, age, neutrophil-to-lymphocyte ratio (NLR) and SOFA score remained in the final model. The following equation was computed: COVID-SOFA score = 10 × [0.037 × Age + 0.347 × ln(NLR) + 0.16 × SOFA]. Harrell’s C-index for the COVID-SOFA score was higher than the SOFA score alone for 28-day mortality (0.697 [95% CI; 0.662−0.731] versus 0.639 [95% CI: 0.605−0.672]). Subsequently, the prediction error rate was improved up to 16.06%. Area under the ROC (AUROC) was significantly higher for the COVID-SOFA score compared with the SOFA score for 28-day mortality: 0.796 [95% CI: 0.755−0.833] versus 0.699 [95% CI: 0.653−0.742, p < 0.001]. Better predictive value was observed with repeated measurement at 48 h after ICU admission. Conclusions: The COVID-SOFA score is better than the SOFA score alone for 28-day mortality prediction. Improvement in predictive value seen with measurements at 48 h after ICU admission suggests that the COVID-SOFA score can be used in a repetitive manner. External validation is required to support these results.

Comparison between machine learning methods for mortality prediction for sepsis patients with different social determinants

BACKGROUND

Sepsis is one of the most life-threatening circumstances for critically ill patients in the United States, while diagnosis of sepsis is challenging as a standardized criteria for sepsis identification is still under development. Disparities in social determinants of sepsis patients can interfere with the risk prediction performances using machine learning.

METHODS

We analyzed a cohort of critical care patients from the Medical Information Mart for Intensive Care (MIMIC)-III database. Disparities in social determinants, including race, sex, marital status, insurance types and languages, among patients identified by six available sepsis criteria were revealed by forest plots with 95% confidence intervals. Sepsis patients were then identified by the Sepsis-3 criteria. Sixteen machine learning classifiers were trained to predict in-hospital mortality for sepsis patients on a training set constructed by random selection. The performance was measured by area under the receiver operating characteristic curve (AUC). The performance of the trained model was tested on the entire randomly conducted test set and each sub-population built based on each of the following social determinants: race, sex, marital status, insurance type, and language. The fluctuations in performances were further examined by permutation tests.

RESULTS

We analyzed a total of 11,791 critical care patients from the MIMIC-III database. Within the population identified by each sepsis identification method, significant differences were observed among sub-populations regarding race, marital status, insurance type, and language. On the 5783 sepsis patients identified by the Sepsis-3 criteria statistically significant performance decreases for mortality prediction were observed when applying the trained machine learning model on Asian and Hispanic patients, as well as the Spanish-speaking patients. With pairwise comparison, we detected performance discrepancies in mortality prediction between Asian and White patients, Asians and patients of other races, as well as English-speaking and Spanish-speaking patients.

CONCLUSIONS

Disparities in proportions of patients identified by various sepsis criteria were detected among the different social determinant groups. The performances of mortality prediction for sepsis patients can be compromised when applying a universally trained model for each subpopulation. To achieve accurate diagnosis, a versatile diagnostic system for sepsis is needed to overcome the social determinant disparities of patients.

How Common SOFA and Ventilator Time Trial Criteria Would Have Performed During the COVID-19 Pandemic: An Observational Simulated Cohort Study

OBJECTIVES

To evaluate how key aspects of New York State Ventilator Allocation Guidelines (NYSVAG)-Sequential Organ Failure Assessment score criteria and ventilator time trials -might perform with respect to the frequency of ventilator reallocation and survival to hospital discharge in a simulated cohort of coronavirus disease (COVID-19) patients.

METHODS

Single center retrospective observational and simulation cohort study of 884 critically ill COVID-19 patients undergoing ventilator allocation per NYSVAG.

RESULTS

In total, 742 patients (83.9%) would have had their ventilator reallocated during the 11-day observation period, 280 (37.7%) of whom would have otherwise survived to hospital discharge if provided with a ventilator. Only 65 (18.1%) of the observed surviving patients would have survived by NYSVAG. Extending ventilator time trials from 2 to 5 days resulted in a 49.2% increase in simulated survival to discharge.

CONCLUSIONS

In the setting of a protracted respiratory pandemic, implementation of NYSVAG or similar protocols could lead to a high degree of ventilator reallocation, including withdrawal from patients who might otherwise survive. Longer ventilator time trials might lead to improved survival for COVID-19 patients given their protracted respiratory failure. Further studies are needed to understand the survival of patients receiving reallocated ventilators to determine whether implementation of NYSVAG would improve overall survival.

Sequential organ failure assessment, ventilator rationing and evolving triage guidance: new evidence underlines the need to recognise and revise, unjust allocation frameworks

We respond to recent comments on our proposal to improve justice in ventilator triage, in which we used as an example New Jersey's (NJ) publicly available and legally binding Directive Number 2020-03. We agree with Bernard Lo and Doug White that equity implications of triage frameworks should be continually reassessed, which is why we offered six concrete options for improvement, and called for monitoring the consequences of adopted triage models. We disagree with their assessment that we mis-characterised their Model Guidance, as included in the NJ Directive, in ways that undermine our conclusions. They suggest we erroneously described their model as a two-criterion allocation framework; that recognising other operant criterion reveals it 'likely mitigate[s] rather than exacerbate[s] racial disparities during triage', and allege that concerns about inequitable outcomes are 'without evidence'. We highlight two major studies robustly demonstrating why concerns about disparate outcomes are justified. We also show that White and Lo seek to retrospectively-and counterfactually-correct the version of the Model Guideline included in the NJ Directive. However, as our facsimile reproductions show, neither the alleged four-criteria form, nor other key changes, such as dropping the Sequential Organ Failure Assessment score, are found in the Directive. These points matter because (1) our conclusions hence stand, (2) because the public version of the Model Guidance had not been updated to reduce the risk of inequitable outcomes until June 2021 and (3) NJ's Directive still does not reflect these revisions, and, hence, represents a less equitable version, as acknowledged by its authors. We comment on broader policy implications and call for ways of ensuring accurate, transparent and timely updates for users of high-stakes guidelines.

Ethics of Extracorporeal Membrane Oxygenation under Conventional and Crisis Standards of Care

Extracorporeal membrane oxygenation (ECMO) is a form of life support for cardiac and/or pulmonary failure with unique ethical challenges compared to other forms of life support. Ethical challenges with ECMO exist when conventional standards of care apply, and are exacerbated during periods of absolute ECMO scarcity when "crisis standards of care" are instituted. When conventional standards of care apply, we propose that it is ethically permissible to withhold placing patients on ECMO for reasons of technical futility or when patients have terminal, short-term prognoses that are untreatable by ECMO. Under crisis standards of care, it is ethically permissible to broaden exclusionary criteria to also withhold ECMO from patients who have a low likelihood of recovery, to maximize the overall number of lives saved. Unilateral withdrawal of ECMO against a patient's preferences is unethical under conventional standards of care, but is ethical under crisis standards of care to increase access to ECMO to others in society. ECMO should only be rationed when true scarcity exists, and allocation protocols should be transparent to the public. When rationing must occur under crisis standards of care, it is imperative that oversight bodies assess for inequities in the allocation of ECMO and make frequent changes to improve any inequities.