How Reliable are Patient-Reported Rehospitalizations? Implications for the Design of Future Practical Clinical Studies

Clinical events classification (CEC) in clinical trials: Report on the current landscape and future directions - proceedings from the CEC Summit 2018

IMPORTANCE

Clinical events adjudication is pivotal for generating consistent and comparable evidence in clinical trials. The methodology of event adjudication is evolving, but research is needed to develop best practices and spur innovation.

OBSERVATIONS

A meeting of stakeholders from regulatory agencies, academic and contract research organizations, pharmaceutical and device companies, and clinical trialists convened in Chicago, IL, for Clinical Events Classification (CEC) Summit 2018 to discuss key topics and future directions. Formal studies are lacking on strategies to optimize CEC conduct, improve efficiency, minimize cost, and generally increase the speed and accuracy of the event adjudication process. Major challenges to CEC discussed included ensuring rigorous quality of the process, identifying safety events, standardizing event definitions, using uniform strategies for missing information, facilitating interactions between CEC members and other trial leadership, and determining the CEC's role in pragmatic trials or trials using real-world data. Consensus recommendations from the meeting include the following: (1) ensure an adequate adjudication infrastructure; (2) use negatively adjudicated events to identify important safety events reported only outside the scope of the primary endpoint; (3) conduct further research in the use of artificial intelligence and digital/mobile technologies to streamline adjudication processes; and (4) emphasize the importance of standardizing event definitions and quality metrics of CEC programs.

CONCLUSIONS AND RELEVANCE

As novel strategies for clinical trials emerge to generate evidence for regulatory approval and to guide clinical practice, a greater understanding of the role of the CEC process will be critical to optimize trial conduct and increase confidence in the data generated.

Development and Validation of a Risk Prediction Model for 1-Year Readmission Among Young Adults Hospitalized for Acute Myocardial Infarction

Background

Readmission over the first year following hospitalization for acute myocardial infarction (AMI) is common among younger adults (≤55 years). Our aim was to develop/validate a risk prediction model that considered a broad range of factors for readmission within 1 year.

Methods and Results

We used data from the VIRGO (Variation in Recovery: Role of Gender on Outcomes of Young AMI Patients) study, which enrolled young adults aged 18 to 55 years hospitalized with AMI across 103 US hospitals (N=2979). The primary outcome was ≥1 all‐cause readmissions within 1 year of hospital discharge. Bayesian model averaging was used to select the risk model. The mean age of participants was 47.1 years, 67.4% were women, and 23.2% were Black. Within 1 year of discharge for AMI, 905 (30.4%) of participants were readmitted and were more likely to be female, Black, and nonmarried. The final risk model consisted of 10 predictors: depressive symptoms (odds ratio [OR], 1.03; 95% CI, 1.01–1.05), better physical health (OR, 0.98; 95% CI, 0.97–0.99), in‐hospital complication of heart failure (OR, 1.44; 95% CI, 0.99–2.08), chronic obstructive pulmomary disease (OR, 1.29; 95% CI, 0.96–1.74), diabetes mellitus (OR, 1.23; 95% CI, 1.00–1.52), female sex (OR, 1.31; 95% CI, 1.05–1.65), low income (OR, 1.13; 95% CI, 0.89–1.42), prior AMI (OR, 1.47; 95% CI, 1.15–1.87), in‐hospital length of stay (OR, 1.13; 95% CI, 1.04–1.23), and being employed (OR, 0.88; 95% CI, 0.69–1.12). The model had excellent calibration and modest discrimination (C statistic=0.67 in development/validation cohorts).

Conclusions

Women and those with a prior AMI, increased depressive symptoms, longer inpatient length of stay and diabetes may be more likely to be readmitted. Notably, several predictors of readmission were psychosocial characteristics rather than markers of AMI severity. This finding may inform the development of interventions to reduce readmissions in young patients with AMI.

Ascertaining Nonfatal Endpoints in Clinical Trials: Central Adjudication Versus Patient Insurance Claims

BACKGROUND

The 21st Century Cures Act allows the US Food and Drug Administration (FDA) to utilize real-world data (RWD) to create real-world evidence (RWE) for new indications or post approval study requirements. We compared central adjudication with two insurance claims data sources to understand how endpoint accuracy differences impact RWE results.

METHODS

We developed a decision analytic model to compare differences in efficacy (all-cause death, stroke and myocardial infarction) and safety (bleeding requiring transfusion) results for a simulated acute coronary syndrome antiplatelet therapy clinical trial. Endpoint accuracy metrics were derived from previous studies that compared centrally-adjudicated and insurance claims-based clinical trial endpoints.

RESULTS

Efficacy endpoint results per 100 patients were similar for the central adjudication model (intervention event rate, 11.3; control, 13.7; difference, 2.4) and the prospective claims data collection model (intervention event rate, 11.2; control 13.6; difference, 2.3). However, the retrospective claims linking model's efficacy results were larger (intervention event rate, 14.6; control, 18.0; difference, 3.4). True positive event rate results (intervention, control and difference) for both insurance claims-based models were less than the central adjudication model due to false negative events. Differences in false positive event rates were responsible for differences in efficacy results for the two insurance claims-based models.

CONCLUSION

Efficacy endpoint results differed by data source. Investigators need guidance to determine which data sources produce regulatory-grade RWE.

Temporal relationship of computed and structured diagnoses in electronic health record data

Background

The electronic health record (EHR) holds the prospect of providing more complete and timely access to clinical information for biomedical research, quality assessments, and quality improvement compared to other data sources, such as administrative claims. In this study, we sought to assess the completeness and timeliness of structured diagnoses in the EHR compared to computed diagnoses for hypertension (HTN), hyperlipidemia (HLD), and diabetes mellitus (DM).

Methods

We determined the amount of time for a structured diagnosis to be recorded in the EHR from when an equivalent diagnosis could be computed from other structured data elements, such as vital signs and laboratory results. We used EHR data for encounters from January 1, 2012 through February 10, 2019 from an academic health system. Diagnoses for HTN, HLD, and DM were computed for patients with at least two observations above threshold separated by at least 30 days, where the thresholds were outpatient blood pressure of ≥ 140/90 mmHg, any low-density lipoprotein ≥ 130 mg/dl, or any hemoglobin A1c ≥ 6.5%, respectively. The primary measure was the length of time between the computed diagnosis and the time at which a structured diagnosis could be identified within the EHR history or problem list.

Results

We found that 39.8% of those with HTN, 21.6% with HLD, and 5.2% with DM did not receive a corresponding structured diagnosis recorded in the EHR. For those who received a structured diagnosis, a mean of 389, 198, and 166 days elapsed before the patient had the corresponding diagnosis of HTN, HLD, or DM, respectively, recorded in the EHR.

Conclusions

We found a marked temporal delay between when a diagnosis can be computed or inferred and when an equivalent structured diagnosis is recorded within the EHR. These findings demonstrate the continued need for additional study of the EHR to avoid bias when using observational data and reinforce the need for computational approaches to identify clinical phenotypes.

Rates and Predictors of Patient Underreporting of Hospitalizations During Follow-Up After Acute Myocardial Infarction: An Assessment From the TRIUMPH Study

BACKGROUND

Many clinical investigations depend on participant self-report as a principal method of identifying health care events. If self-report is used as the trigger to collect and adjudicate medical records, any event that is not reported by the patient will be missed by the investigators, reducing the power of the study and misrepresenting the risk of its participants. We sought to determine the rates and predictors of underreporting hospitalization events during the follow-up period of a prospective study of patients hospitalized with an acute myocardial infarction.

METHODS AND RESULTS

The TRIUMPH (Translational Research Investigating Underlying Disparities in Acute Myocardial Infarction Patients' Health Status) registry, a longitudinal multicenter cohort study of people with acute myocardial infarction in the United States, queried patients for hospitalization events during interviews at 1, 6, and 12 months. To validate these self-reports, medical records for all events at every hospital where the patient reported receiving care were acquired for adjudication, not just those for the reported events. Of the 4340 participants in TRIUMPH, 1209 (28%) reported at least one hospitalization. After medical records abstraction and adjudication, we identified 1086 hospitalizations from 639 participants (60.2±12 years of age, 38.2% women). Of these hospitalizations, 346 (31.9%) were underreported by the participants. Rates of underreporting ranged from 22.5% to 55.6% based on different patient characteristics. The odds of underreporting were highest for those not currently working (adjusted odds ratio, 1.66 [95% CI, 1.04-2.63]), lowest for those married (adjusted odds ratio, 0.50 [95% CI, 0.33-0.76]), and increased the longer the elapsed time between the admission and the patient's follow-up interview (adjusted odds ratio per month, 1.16 [95% CI, 1.08-1.24]). There was a substantial within-individual variation on the accuracy of reporting.

CONCLUSIONS

Hospitalizations after acute myocardial infarction are commonly underreported in interviews and should not be used alone to determine event rates in clinical studies.

Emotion regulation in patients with heart failure: Its relationship with depressive symptoms and rehospitalization

OBJECTIVE

To examine the role of emotion regulation and its relationship to mental and physical health in patients with heart failure (HF).

METHODS

Patients hospitalized with HF were screened for depressive symptoms with the two-item Patient Health Questionnaire (PHQ-2) and classified as screen-positive if endorsing ≥1 item and otherwise as screen-negative. One month after hospital discharge, the Emotion Regulation Questionnaire (ERQ) was administered to assess suppression and reappraisal as emotion regulation strategies. In all participants who completed the ERQ (N = 394), all-cause rehospitalization and depressive symptoms using the PHQ-9 were assessed at 1-, 3-, and 6-months after hospital discharge.

RESULTS

Overall, PHQ-9 scores decreased by 6-months (-0.13 points/month, p = .003), and although suppression showed a small association with depression, neither strategy modulated the slope of the decline in depressive symptoms. Multivariable-adjusted Cox models showed that reappraisal and suppression were not related to all-cause rehospitalization in the entire cohort. However, increasing reappraisal reduced rehospitalization risk by 24% for screen-positive patients (N = 311, HR = 0.76, p = .02), but increased it by 94% in screen-negative patients (N = 83, HR = 1.94, p = .009).

CONCLUSION

Suppression and reappraisal showed specific and divergent associations in patients with HF: Suppression may relate to depressive symptoms. Reappraisal was associated with rehospitalization, but differently for patients with a positive vs. negative depression screen. Further studies are needed to examine whether emotion-regulation skill training can improve mental and physical health in depressed patients with HF or ameliorate depression in those at-risk.

Accuracy of Medical Claims for Identifying Cardiovascular and Bleeding Events After Myocardial Infarction : A Secondary Analysis of the TRANSLATE-ACS Study

Importance

Pragmatic clinical trial designs have proposed the use of medical claims data to ascertain clinical events; however, the accuracy of billed diagnoses in identifying potential events is unclear.

Objectives

To compare the 1-year cumulative incidences of events when events were identified by medical claims vs by physician adjudication and to assess the accuracy of bill-identified events using physician adjudication as the criterion standard.

Design, Setting, and Participants

This post hoc analysis of a clinical trial assessed the medical claims forms and records for all rehospitalizations at 233 US hospitals within 1 year of the index acute myocardial infarction (MI) of 12 365 patients enrolled in the Treatment With Adenosine Diphosphate Receptor Inhibitors: Longitudinal Assessment of Treatment Patterns and Events After Acute Coronary Syndrome (TRANSLATE-ACS) study between April 1, 2010, and October 31, 2012. Fourteen patients (0.1%) died during the index hospitalization and were excluded from analysis. Recurrent MI, stroke, and bleeding events were identified per the International Classification of Diseases, Ninth Revision, Clinical Modification diagnosis and procedural codes in medical bills. These events were independently adjudicated by study physicians through medical record reviews using the prespecified criteria of recurrent MI and stroke and the bleeding definition by the Global Utilization of Streptokinase and Tissue Plasminogen Activator for Occluded Coronary Arteries (GUSTO) scale. Medical claims were reported on a Uniform Bill-04 claims form; claims were collected from all hospitals visited by patients enrolled in TRANSLATE-ACS. Agreement between medical claims-identified events and physician-adjudicated events over the 12 months after discharge was assessed with the κ statistic. Data were analyzed from January 30, 2015, to March 2, 2017.

Main Outcomes and Measures

Event rates within 1 year after MI.

Results

Among 12 365 patients with acute MI, 8890 (71.9%) were men and mean (SD) age was 60 (11.6) years. The cumulative 1-year incidence of events identified by medical claims was 4.3% for MI, 0.9% for stroke, and 5.0% for bleeding. Incidence rates based on physician adjudication were 4.7% for MI, 0.9% for stroke, and 5.4% for bleeding. Agreement between medical claims-identified and physician-adjudicated events was modest, with a κ of 0.76 (95% CI, 0.73 to 0.79) for MI and 0.55 (95% CI, 0.41 to 0.68) for stroke events. In contrast, agreement between medical claims-identified and physician-adjudicated bleeding events was poor, with a κ of 0.24 (95% CI, 0.19 to 0.30) for any hospitalized bleeding event and 0.15 (95% CI, 0.11 to 0.20) for moderate or severe bleeding on the GUSTO scale.

Conclusions and Relevance

Event rates at 1 year after MI were lower for MI, stroke, and bleeding when medical claims were used to identify events than when adjudicated by physicians. Medical claims diagnoses were only modestly accurate in identifying MI and stroke admissions but had limited accuracy for bleeding events. An alternative approach may be needed to ensure good safety surveillance in cardiovascular studies.

Trial Registration

clinicaltrials.gov Identifier: NCT01088503.

Administrative claims data to support pragmatic clinical trial outcome ascertainment on cardiovascular health

BACKGROUND/AIMS

Health plan administrative claims data present a cost-effective complement to traditional trial-specific ascertainment of clinical events typically conducted through patient report or a single health system electronic health record. We aim to demonstrate the value of health plan claims data in improving the capture of endpoints in longitudinal pragmatic clinical trials.

METHODS

This retrospective cohort study paralleled the design of the ADAPTABLE (Aspirin Dosing: A Patient-centric Trial Assessing Benefits and Long-Term Effectiveness) trial designed to compare the effectiveness of two doses of aspirin. We applied the ADAPTABLE identification query in claims data from Anthem, an American health insurance company, and identified health plan members who met the ADAPTABLE trial criteria. Among the ADAPTABLE eligible members, we selected overlapping members with PCORnet Clinical Data Research Networks in the 2 years prior to the index date (1 April 2014). PCORnet Clinical Data Research Networks consist of network partners (or healthcare systems) that store their electronic health record data in the same format to support multi-institutional research. ADAPTABLE outcome events-cardiovascular hospitalizations including admissions for myocardial infarction, stroke, or cardiac procedures; hospitalizations for major bleeding; and in-hospital deaths-were evaluated for a 2-year follow-up period. Events were classified as within or outside PCORnet Clinical Data Research Networks using facility identifiers affiliated with each hospital stay. Patient characteristics were examined with descriptive statistics, and incidence rates were reported for available Clinical Data Research Networks and claims data.

RESULTS

Among 884,311 ADAPTABLE eligible health plan members, 11,101 patients overlapped with PCORnet Clinical Data Research Networks. Average age was 70 years, 71% were male, and average follow-up was 20.7 months. Patients had 1521 cardiovascular hospitalizations (571 (37.5%) occurred outside PCORnet Clinical Data Research Networks), 710 for major bleeding (296 (41.7%) outside PCORnet Clinical Data Research Networks), and 196 in-hospital deaths (67 (34.2%) outside PCORnet Clinical Data Research Networks). Incidence rates (events per1000 patient-months) differed between available network partners and claims data: cardiovascular hospitalizations, 4.1 (95% confidence interval: 3.9, 4.4) versus 6.6 (95% confidence interval: 6.3, 7.0), major bleeding, 1.8 (95% confidence interval: 1.6, 2.0) versus 3.1 (95% confidence interval: 2.9, 3.3), and in-hospital death, 0.56 (95% confidence interval: 0.47, 0.67) versus 0.85 (95% confidence interval: 0.74, 0.98), respectively.

CONCLUSION

This study demonstrated the value of supplementing longitudinal site-based clinical studies with administrative claims data. Our results suggest that claims data together with network partner electronic health record data constitute an effective vehicle to capture patient outcomes since >30% of patients have non-fatal and fatal events outside of enrolling sites.

Management of Persistent Angina After Myocardial Infarction Treated With Percutaneous Coronary Intervention: Insights From the TRANSLATE-ACS Study

BACKGROUND

Angina has important implications for patients' quality of life and healthcare utilization. Angina management after acute myocardial infarction (MI) treated with percutaneous coronary intervention (PCI) is unknown.

METHODS AND RESULTS

TRANSLATE-ACS (Treatment With Adenosine Diphosphate Receptor Inhibitors: Longitudinal Assessment of Treatment Patterns and Events After Acute Coronary Syndrome) was a longitudinal study of MI patients treated with percutaneous coronary intervention at 233 US hospitals from 2010 to 2012. Among patients with self-reported angina at 6 weeks post-MI, we described patterns of angina and antianginal medication use through 1 year postdischarge. Of 10 870 percutaneous coronary intervention-treated MI patients, 3190 (29.3%) reported angina symptoms at 6 weeks post-MI; of these, 658 (20.6%) had daily/weekly angina while 2532 (79.4%) had monthly angina. Among patients with 6-week angina, 2936 (92.0%) received β-blockers during the 1 year post-MI, yet only 743 (23.3%) were treated with other antianginal medications. At 1 year, 1056 patients (33.1%) with 6-week angina reported persistent angina symptoms. Of these, only 31.2% had been prescribed non-β-blocker antianginal medications at any time in the past year. Among patients undergoing revascularization during follow-up, only 25.9% were on ≥1 non-β-blocker anti-anginal medication at the time of the procedure.

CONCLUSIONS

Angina is present in one third of percutaneous coronary intervention-treated MI patients as early as 6 weeks after discharge, and many of these patients have persistent angina at 1 year. Non-β-blocker antianginal medications are infrequently used in these patients, even among those with persistent angina and those undergoing revascularization.

Sex Differences in 1-Year All-Cause Rehospitalization in Patients After Acute Myocardial Infarction: A Prospective Observational Study

BACKGROUND

Compared with men, women are at higher risk of rehospitalization in the first month after discharge for acute myocardial infarction (AMI). However, it is unknown whether this risk extends to the full year and varies by age. Explanatory factors potentially mediating the relationship between sex and rehospitalization remain unexplored and are needed to reduce readmissions. The aim of this study was to assess sex differences and factors associated with 1-year rehospitalization rates after AMI.

METHODS

We recruited 3536 patients (33% women) ≥18 years of age hospitalized with AMI from 24 US centers into the TRIUMPH study (Translational Research Investigating Underlying Disparities in Acute Myocardial Infarction Patients' Health Status). Data were obtained by medical record abstraction and patient interviews, and a physician panel adjudicated hospitalizations within the first year after AMI. We compared sex differences in rehospitalization using a Cox proportional hazards model, following sequential adjustment for covariates and testing for an age-sex interaction.

RESULTS

One-year crude all-cause rehospitalization rates for women were significantly higher than men after AMI (hazard ratio, 1.29 for women; 95% confidence interval, 1.12-1.48). After adjustment for demographics and clinical factors, women had a persistent 26% higher risk of rehospitalization (hazard ratio, 1.26; 95% confidence interval, 1.08-1.47). However, after adjustment for health status and psychosocial factors (hazard ratio, 1.14; 95% confidence interval, 0.96-1.35), the association was attenuated. No significant age-sex interaction was found for 1-year rehospitalization, suggesting that the increased risk applied to both older and younger women.

CONCLUSIONS

Regardless of age, women have a higher risk of rehospitalization compared with men over the first year after AMI. Although the increased risk persisted after adjustment for clinical factors, the poorer health and psychosocial state of women attenuated the difference.