Development and Validation of an Algorithm for Thrombosis with Thrombocytopenia Syndrome (TTS) at Unusual Sites

Introduction

Thrombosis with thrombocytopenia syndrome (TTS) has been reported following receipt of adenoviral vector-based COVID-19 vaccines. However, no validation studies evaluating the accuracy of International Classification of Diseases-10-Clinical Modification (ICD-10-CM)-based algorithm for unusual site TTS are available in the published literature.

Methods

The purpose of this study was to assess the performance of clinical coding to 1) leverage literature review and clinical input to develop an ICD-10-CM-based algorithm to identify unusual site TTS as a composite outcome and 2) validate the algorithm against the Brighton Collaboration's interim case definition using laboratory, pathology, and imaging reports in an academic health network electronic health record (EHR) within the US Food and Drug Administration (FDA) Biologics Effectiveness and Safety (BEST) Initiative. Validation of up to 50 cases per thrombosis site was conducted, with positive predictive values (PPV) and 95% confidence intervals (95% CI) calculated using pathology or imaging results as the gold standard.

Results

The algorithm identified 278 unusual site TTS cases, of which 117 (42.1%) were selected for validation. In both the algorithm-identified and validation cohorts, over 60% of patients were 56 years or older. The positive predictive value (PPV) for unusual site TTS was 76.1% (95% CI 67.2-83.2%) and at least 80% for all but one individual thrombosis diagnosis code. PPV for thrombocytopenia was 98.3% (95% CI 92.1-99.5%).

Discussion

This study represents the first report of a validated ICD-10-CM-based algorithm for unusual site TTS. A validation effort found that the algorithm performed at an intermediate-to-high PPV, suggesting that the algorithm can be used in observational studies including active surveillance of COVID-19 vaccines and other medical products.

Immune globulin usage trends in commercially insured and Medicare populations, 2009-2019

BACKGROUND

Longitudinal patterns of immune globulins (IG) use have not been described in large populations. Understanding IG usage is important given potential supply limitations impacting individuals for whom IG is the sole life-saving/health-preserving therapy. The study describes US IG utilization patterns from 2009 to 2019.

STUDY DESIGN AND METHODS

Using IBM MarketScan commercial and Medicare claims data, we examined four metrics overall and by condition-specific categories during 2009-2019: (1) IG administrations per 100,000 person-years, (2) IG recipients per 100,000 enrollees, (3) average annual administrations per recipient, and (4) average annual dose per recipient.

RESULTS

In the commercial and Medicare populations respectively: IG administrations per 100,000 person-years increased by 120% (213-470) and 144% (692-1693); IG recipients per 100,000 enrollees grew by 71% (24-42) and 102% (89-179); average annual administrations per recipient rose by 28% (8-10) and 19% (8-9); and average annual dose (grams) per recipient increased by 29% (384-497) and 34% (317-426). IG administrations associated with immunodeficiency (per 100,000 person-years) increased by 154% (from 127 to 321) and 176% (from 365 to 1007). Autoimmune and neurologic conditions were associated with higher annual average administrations and dose than other conditions.

DISCUSSION

IG use increased, coinciding with a growth in the IG recipient population in the United States. Several conditions contributed to the trend, with the largest increase observed among immunodeficient individuals. Future investigations should assess changes in the demand for IVIG by disease state or indication and consider treatment effectiveness.

Background rates of adverse events of special interest for COVID-19 vaccine safety monitoring in the United States, 2019-2020

BACKGROUND

The U.S. Food and Drug Administration (FDA) Biologics Effectiveness and Safety (BEST) Initiative conducts active surveillance of adverse events of special interest (AESI) after COVID-19 vaccination. Historical incidence rates (IRs) of AESI are comparators to evaluate safety.

METHODS

We estimated IRs of 17 AESI in six administrative claims databases from January 1, 2019, to December 11, 2020: Medicare claims for adults ≥ 65 years and commercial claims (Blue Health Intelligence®, CVS Health, HealthCore Integrated Research Database, IBM® MarketScan® Commercial Database, Optum pre-adjudicated claims) for adults < 65 years. IRs were estimated by sex, age, race/ethnicity (Medicare), and nursing home residency (Medicare) in 2019 and for specific periods in 2020.

RESULTS

The study included >100 million enrollees annually. In 2019, rates of most AESI increased with age. However, compared with commercially insured adults, Medicare enrollees had lower IRs of anaphylaxis (11 vs 12-19 per 100,000 person-years), appendicitis (80 vs 117-155), and narcolepsy (38 vs 41-53). Rates were higher in males than females for most AESI across databases and varied by race/ethnicity and nursing home status (Medicare). Acute myocardial infarction (Medicare) and anaphylaxis (all databases) IRs varied by season. IRs of most AESI were lower during March-May 2020 compared with March-May 2019 but returned to pre-pandemic levels after May 2020. However, rates of Bell's palsy, Guillain-Barré syndrome, narcolepsy, and hemorrhagic/non-hemorrhagic stroke remained lower in multiple databases after May 2020, whereas some AESI (e.g., disseminated intravascular coagulation) exhibited higher rates after May 2020 compared with 2019.

CONCLUSION

AESI background rates varied by database and demographics and fluctuated in March-December 2020, but most returned to pre-pandemic levels after May 2020. It is critical to standardize demographics and consider seasonal and other trends when comparing historical rates with post-vaccination AESI rates in the same database to evaluate COVID-19 vaccine safety.

Assessment of performance characteristics of COVID-19 ICD-10-CM diagnosis code U07.1 using SARS-CoV-2 nucleic acid amplification test results

The Food and Drug Administration’s Biologics Effectiveness and Safety Initiative conducts active surveillance to protect public health during the coronavirus disease 2019 (COVID-19) pandemic. This study evaluated performance of International Classification of Diseases, Tenth Revision, Clinical Modification (ICD-10-CM) diagnosis code U07.1 in identifying COVID-19 cases in claims compared with severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) nucleic acid amplification test results in linked electronic health records (EHRs). Care episodes in three populations were defined using COVID-19-related diagnoses (population 1), SARS-CoV-2 nucleic acid amplification test procedures (population 2), and all-cause hospitalizations (population 3) in two linked claims-EHR databases: IBM® MarketScan® Explorys® Claims-EMR Data Set (commercial) and OneFlorida Data Trust linked Medicaid-EHR. Positive and negative predictive values were calculated. Respectively, populations 1, 2, and 3 included 26,686, 26,095, and 2,564 episodes (commercial) and 29,117, 23,412, and 9,629 episodes (Florida Medicaid). The positive predictive value was >80% and the negative predictive value was >95% in each population, with the highest positive predictive value in population 3 (commercial: 91.9%; Medicaid: 93.1%). Findings did not vary substantially by patient age. Positive predictive values in populations 1 and 2 fluctuated during April–June 2020. They then stabilized in the commercial but not the Medicaid population. Negative predictive values were consistent over time in all populations and databases. Our findings indicate that U07.1 has high performance in identifying COVID-19 cases and noncases in claims databases. Performance may vary across populations and periods.

Characterization of test positivity among patients with coronavirus disease 2019 (COVID-19) in three electronic health records databases, February-November 2020

Background

Monitoring COVID-19 testing volumes and test positivity is an integral part of the response to the pandemic. We described the characteristics of individuals who were tested and tested positive for SARS-CoV-2 during the pre-vaccine phase of the pandemic in the United States (U.S.).

Methods

This descriptive study analyzed three U.S. electronic health record (EHR) databases (Explorys, Academic Health System, and OneFlorida) between February and November 2020, identifying patients who received an interpretable nucleic acid amplification test (NAAT) result. Test-level data were used to characterize the settings in which tests were administered. Patient-level data were used to calculate test positivity rates and characterize the demographics, comorbidities, and hospitalization rates of COVID-19-positive patients.

Results

Over 40% of tests were conducted in outpatient care settings, with a median time between test order and result of 0–1 day for most settings. Patients tested were mostly female (55.6–57.7%), 18–44 years of age (33.9–41.2%), and Caucasian (44.0–66.7%). The overall test positivity rate was 13.0% in Explorys, 8.0% in Academic Health System, and 8.9% in OneFlorida. The proportion of patients hospitalized within 14 days of a positive COVID-19 NAAT result was 24.2–33.1% across databases, with patients over 75 years demonstrating the highest hospitalization rates (46.7–69.7% of positive tests).

Conclusions

This analysis of COVID-19 testing volume and positivity patterns across three large EHR databases provides insight into the characteristics of COVID-19-tested, COVID-19-test-positive, and hospitalized COVID-19-test-positive patients during the early phase of the pandemic in the U.S.

Supplementary Information

The online version contains supplementary material available at 10.1186/s12889-022-13635-6.

Electronic phenotyping of health outcomes of interest using a linked claims-electronic health record database: Findings from a machine learning pilot project

OBJECTIVE

Claims-based algorithms are used in the Food and Drug Administration Sentinel Active Risk Identification and Analysis System to identify occurrences of health outcomes of interest (HOIs) for medical product safety assessment. This project aimed to apply machine learning classification techniques to demonstrate the feasibility of developing a claims-based algorithm to predict an HOI in structured electronic health record (EHR) data.

MATERIALS AND METHODS

We used the 2015-2019 IBM MarketScan Explorys Claims-EMR Data Set, linking administrative claims and EHR data at the patient level. We focused on a single HOI, rhabdomyolysis, defined by EHR laboratory test results. Using claims-based predictors, we applied machine learning techniques to predict the HOI: logistic regression, LASSO (least absolute shrinkage and selection operator), random forests, support vector machines, artificial neural nets, and an ensemble method (Super Learner).

RESULTS

The study cohort included 32 956 patients and 39 499 encounters. Model performance (positive predictive value [PPV], sensitivity, specificity, area under the receiver-operating characteristic curve) varied considerably across techniques. The area under the receiver-operating characteristic curve exceeded 0.80 in most model variations.

DISCUSSION

For the main Food and Drug Administration use case of assessing risk of rhabdomyolysis after drug use, a model with a high PPV is typically preferred. The Super Learner ensemble model without adjustment for class imbalance achieved a PPV of 75.6%, substantially better than a previously used human expert-developed model (PPV = 44.0%).

CONCLUSIONS

It is feasible to use machine learning methods to predict an EHR-derived HOI with claims-based predictors. Modeling strategies can be adapted for intended uses, including surveillance, identification of cases for chart review, and outcomes research.