External Validation of ASPECT (Algorithm for Suspected Pulmonary Embolism Confirmation and Treatment)

Trends in Initial Anticoagulation Among US Patients Hospitalized With Acute Pulmonary Embolism 2011-2020

STUDY OBJECTIVE

Guidelines recommend low-molecular-weight heparin (LMWH) and direct oral anticoagulants (DOACs) rather than unfractionated heparin (UFH) for treatment of acute pulmonary embolism (PE) given their efficacy and reduced risk of bleeding. Using data from a large consortium of US hospitals, we examined trends in initial anticoagulation among hospitalized patients diagnosed with acute PE.

METHODS

We conducted a retrospective study of inpatient and observation cases between January 1, 2011, and December 31, 2020, among individuals aged more than or equal to 18 years treated at acute care hospitals contributing data to the Premier Healthcare Database. Included cases received a diagnosis of acute PE, underwent imaging for PE, and received anticoagulation at the time of admission. The primary outcome was the initial anticoagulant selected for treatment.

RESULTS

Among 299,016 cases at 1,045 hospitals, similar proportions received initial treatment with UFH (47.4%) and LMWH (47.9%). Between 2011 and 2020, the proportion of patients initially treated with UFH increased from 41.9% to 56.3%. Over this period, use of LMWH as the initial anticoagulant was reduced from 58.1% in 2011 to 37.3% in 2020. The proportion of cases admitted to the ICU, treated with mechanical ventilation or vasopressors, and inpatient mortality were stable. Factors most strongly associated with receipt of UFH were admission to the ICU (odds ratio [OR] 6.90; 95% confidence interval [CI] 6.31 to 7.54) or step-down unit (OR 2.30; 95% CI 2.16 to 2.45), receipt of thrombolysis (OR 4.25; 95% CI 3.09 to 5.84) or vasopressors (OR 1.83; 95% CI 1.32 to 2.54), and chronic renal disease (OR 1.67; 95% CI 1.54 to 1.81).

CONCLUSIONS

Despite recommendations that LMWH and DOACs be considered first-line for most patients with acute PE, use of UFH is common and increasing. Further research is needed to elucidate factors associated with persistent use of UFH and opportunities for deimplementation of low-value care.

Derivation and external validation of a portable method to identify patients with pulmonary embolism from radiology reports: The READ-PE algorithm

BACKGROUND

Identification of pulmonary embolism (PE) across a cohort currently requires burdensome manual review. Previous approaches to automate capture of PE diagnosis have either been too complex for widespread use or have lacked external validation. We sought to develop and validate the Regular Expression Aided Determination of PE (READ-PE) algorithm, which uses a portable text-matching approach to identify PE in reports from computed tomography with angiography (CTA).

METHODS

We identified derivation and validation cohorts of final radiology reports for CTAs obtained on adults (≥ 18 years) at two independent, quaternary academic emergency departments (EDs) in the United States. All reports were in the English language. We manually reviewed CTA reports for PE as a reference standard. In the derivation cohort, we developed the READ-PE algorithm by iteratively combining regular expressions to identify PE. We validated the READ-PE algorithm in an independent cohort, and compared performance against three prior algorithms with sensitivity, specificity, positive-predictive-value (PPV), negative-predictive-value (NPV), and the F1 score.

RESULTS

Among 2948 CTAs in the derivation cohort 10.8 % had PE and the READ-PE algorithm reached 93 % sensitivity, 99 % specificity, 94 % PPV, 99 % NPV, and 0.93 F1 score, compared to F1 scores ranging from 0.50 to 0.85 for three prior algorithms. Among 1206 CTAs in the validation cohort 9.2 % had PE and the algorithm had 98 % sensitivity, 98 % specificity, 85 % PPV, 100 % NPV, and 0.91 F1 score.

CONCLUSIONS

The externally validated READ-PE algorithm identifies PE in English-language reports from CTAs obtained in the ED with high accuracy. This algorithm may be used in the electronic health record to accurately identify PE for research or surveillance. If implemented at other EDs, it should first undergo local validation and may require maintenance over time.

Validity of Routine Health Data To Identify Safety Outcomes of Interest For Covid-19 Vaccines and Therapeutics in the Context of the Emerging Pandemic: A Comprehensive Literature Review

Introduction

Regulatory guidance encourages transparent reporting of information on the quality and validity of electronic health record data being used to generate real-world benefit-risk evidence for vaccines and therapeutics. We aimed to provide an overview of the availability of validated diagnostic algorithms for selected safety endpoints for Coronavirus disease 2019 (COVID-19) vaccines and therapeutics in the context of the emerging pandemic prior to December 2020.

Methods

We reviewed the literature up to December 2020 to identify validation studies for various safety events of interest, including myocardial infarction, arrhythmia, myocarditis, acute cardiac injury, vasculitis/vasculopathy, venous thromboembolism, stroke, respiratory distress syndrome (RDS), pneumonitis, cytokine release syndrome (CRS), multiple organ dysfunction syndrome, and renal failure. We included studies published between 2015 and 2020 that were considered high quality assessed with QUADAS and that reported positive predictive values (PPVs).

Results

Out of 43 identified studies, we found that diagnostic algorithms for cardiovascular outcomes were supported by the highest number of validation studies (n=17). Accurate algorithms are available for myocardial infarction (median PPV 80%; IQR 22%), arrhythmia (PPV range >70%), venous thromboembolism (median PPV: 73%) and ischaemic stroke (PPV range ≥85%). We found a lack of validation studies for less common respiratory and cardiac safety outcomes of interest (eg, pneumonitis and myocarditis), as well as for COVID-specific complications (CRS, RDS).

Conclusion

There is a need for better understanding of barriers to conducting validation studies, including data governance restrictions. Regulatory guidance should promote embedding validation within real-world EHR research used for decision-making.

Accuracy of efficient data methods to determine the incidence of hospital-acquired thrombosis and major bleeding in medical and surgical inpatients: a multicentre observational cohort study in four UK hospitals

OBJECTIVES

We evaluated the accuracy of using routine health service data to identify hospital-acquired thrombosis (HAT) and major bleeding events (MBE) compared with a reference standard of case note review.

DESIGN

A multicentre observational cohort study.

SETTING

Four acute hospitals in the UK.

PARTICIPANTS

A consecutive unselective cohort of general medical and surgical patients requiring hospitalisation for a period of >24 hours during the calendar year 2021. We excluded paediatric, obstetric and critical care patients due to differential risk profiles.

INTERVENTIONS

We compared preidentified sources of routinely collected information (using hospital coding data and local contractually mandated thrombosis datasets) to data extracted from case notes using a predesigned workflow methodology.

PRIMARY AND SECONDARY OUTCOME MEASURES

We defined HAT as objectively confirmed venous thromboembolism occurring during hospital stay or within 90 days of discharge and MBE as per international consensus.

RESULTS

We were able to source all necessary routinely collected outcome data for 87% of 2008 case episodes reviewed. The sensitivity of hospital coding data (International Classification of Diseases 10th Revision, ICD-10) for the diagnosis of HAT and MBE was 62% (95% CI, 54 to 69) and 38% (95% CI, 27 to 50), respectively. Sensitivity improved to 81% (95% CI, 75 to 87) when using local thrombosis data sets.

CONCLUSIONS

Using routinely collected data appeared to miss a substantial proportion of outcome events, when compared with case note review. Our study suggests that currently available routine data collection methods in the UK are inadequate to support efficient study designs in venous thromboembolism research.

TRIAL REGISTRATION NUMBER

NIHR127454.

Developing and validating natural language processing algorithms for radiology reports compared to ICD-10 codes for identifying venous thromboembolism in hospitalized medical patients

BACKGROUND

Identifying venous thromboembolism (VTE) from large clinical and administrative databases is important for research and quality improvement.

OBJECTIVE

To develop and validate natural language processing (NLP) algorithms to identify VTE from radiology reports among general internal medicine (GIM) inpatients.

METHODS

This cross-sectional study included GIM hospitalizations between April 1, 2010 and March 31, 2017 at 5 hospitals in Toronto, Ontario, Canada. We developed NLP algorithms to identify pulmonary embolism (PE) and deep venous thrombosis (DVT) from radiologist reports of thoracic computed tomography (CT), extremity compression ultrasound (US), and nuclear ventilation-perfusion (VQ) scans in a training dataset of 1551 hospitalizations. We compared the accuracy of our NLP algorithms, the previously-published "simpleNLP" tool, and administrative discharge diagnosis codes (ICD-10-CA) for PE and DVT to the "gold standard" manual review in a separate random sample of 4000 GIM hospitalizations.

RESULTS

Our NLP algorithms were highly accurate for identifying DVT from US, with sensitivity 0.94, positive predictive value (PPV) 0.90, and Area Under the Receiver-Operating-Characteristic Curve (AUC) 0.96; and in identifying PE from CT, with sensitivity 0.91, PPV 0.89, and AUC 0.96. Administrative diagnosis codes and the simple NLP tool were less accurate for DVT (ICD-10-CA sensitivity 0.63, PPV 0.43, AUC 0.81; simpleNLP sensitivity 0.41, PPV 0.36, AUC 0.66) and PE (ICD-10-CA sensitivity 0.83, PPV 0.70, AUC 0.91; simpleNLP sensitivity 0.89, PPV 0.62, AUC 0.92).

CONCLUSIONS

Administrative diagnosis codes are unreliable in identifying VTE in hospitalized patients. We developed highly accurate NLP algorithms to identify VTE from radiology reports in a multicentre sample and have made the algorithms freely available to the academic community with a user-friendly tool (https://lks-chart.github.io/CHARTextract-docs/08-downloads/rulesets.html#venous-thromboembolism-vte-rulesets).

Effectiveness and safety of direct oral anticoagulants in patients with cancer associated venous thromboembolism

INTRODUCTION

Real-world evidence on the effects of direct oral anticoagulants (DOACs) in patients with cancer associated venous thromboembolism (VTE) is limited. Thus, our population-based cohort study aimed to assess the effectiveness and safety of DOACs compared to the standard of care low-molecular-weight heparin (LMWH) in this vulnerable population.

MATERIALS AND METHODS

Using linked administrative healthcare databases from the province of Québec, Canada, we identified patients with incident VTE from 2012 to 2015 and a cancer diagnosis in the year before the VTE, who initiated treatment with anticoagulants within 30 days after the VTE. Using an active comparator new-user design with an as-treated exposure definition, we compared use of DOACs with use of LMWH. Cox proportional hazards models estimated adjusted hazard ratios (HRs) and 95% confidence intervals (CIs) of recurrent VTE, major bleeding, and all-cause mortality. In secondary analyses, we stratified by age and sex.

RESULTS

Overall, 4438 patients with cancer associated VTE initiated treatment with anticoagulants (513 DOACs, 2698 LMWH). During a median follow-up of 0.3 years, and compared with LMWH, DOACs were associated with a decreased risk of recurrent VTE (HR, 0.54; 95% CI, 0.36-0.82) and major bleeding (HR, 0.54; 95% CI, 0.31-0.96). We also observed a decreased risk of all-cause mortality with DOACs compared with LMWH (HR, 0.14; 95% CI, 0.09-0.22). Age and sex did not modify the associations.

CONCLUSIONS

DOACs were associated with improved effectiveness and safety compared with LMWH in patients with cancer related VTE. Unmeasured confounding probably contributed to our findings on all-cause mortality.