Validity of health insurance data to identify people with epilepsy

On the Dependence of the Critical Success Index (CSI) on Prevalence

The critical success index (CSI) is an established metric used in meteorology to verify the accuracy of weather forecasts. It is defined as the ratio of hits to the sum of hits, false alarms, and misses. Translationally, CSI has gained popularity as a unitary outcome measure in various clinical situations where large numbers of true negatives may influence the interpretation of other, more traditional, outcome measures, such as specificity (Spec) and negative predictive value (NPV), or when unified interpretation of positive predictive value (PPV) and sensitivity (Sens) is needed. The derivation of CSI from measures including PPV has prompted questions as to whether and how CSI values may vary with disease prevalence (P), just as PPV estimates are dependent on P, and hence whether CSI values are generalizable between studies with differing prevalences. As no detailed study of the relation of CSI to prevalence has been undertaken hitherto, the dataset of a previously published test accuracy study of a cognitive screening instrument was interrogated to address this question. Three different methods were used to examine the change in CSI across a range of prevalences, using both the Bayes formula and equations directly relating CSI to Sens, PPV, P, and the test threshold (Q). These approaches showed that, as expected, CSI does vary with prevalence, but the dependence differs according to the method of calculation that is adopted. Bayesian rescaling of both Sens and PPV generates a concave curve, suggesting that CSI will be maximal at a particular prevalence, which may vary according to the particular dataset.

Medication burden in epilepsy: Exploring the impact of non-epilepsy concomitant drugs load

PURPOSE

To determine the burden of non-epilepsy drugs on people with epilepsy, using administrative health care data.

METHODS

The Achmea Health Insurance Database (AHID) contains health claims data from 25 % of the Dutch population. From the AHID, we selected all policyholders with coverage for at least one full calendar year between 2006-2009. We included adults with diagnostic codes for epilepsy and randomly selected two frequency-matched controls per case. We labeled drugs dispensed at least twice per calendar year as chronic and excluded antiseizure medications. We estimated and compared the prevalence of chronic medication use, number of chronic medications used, number of prescriptions dispensed, Rx Risk comorbidity index, and drug burden index (DBI) between people with epilepsy and controls.

RESULTS

Non-epilepsy chronic medication use was more frequent in people with epilepsy than controls (67 % versus 59 %, p < 0.001). People with epilepsy had an increased DBI (average 0.19 versus 0.10, p < 0.001), used more chronic medications (median 2 versus 1, p < 0.001) and had more prescriptions dispensed (median 7 versus 3, p < 0.001). The DBI and number of unique chronic medications were higher among older (>60 years) than younger (<60 years) subjects in cases and controls. Non-epilepsy chronic medication use was more prevalent in people with epilepsy across all therapeutic drug classes and most comorbidities measured using the Rx Risk score.

CONCLUSION

Chronic non-epilepsy medication use is more prevalent among people with epilepsy. The medication burden is higher among elderly with epilepsy and could partially explain the lower quality of life of people with epilepsy with comorbidities.

Epilepsy Among Elderly Medicare Beneficiaries: A Validated Approach to Identify Prevalent and Incident Epilepsy

BACKGROUND

Uncertain validity of epilepsy diagnoses within health insurance claims and other large datasets have hindered efforts to study and monitor care at the population level.

OBJECTIVES

To develop and validate prediction models using longitudinal Medicare administrative data to identify patients with actual epilepsy among those with the diagnosis.

RESEARCH DESIGN, SUBJECTS, MEASURES

We used linked electronic health records and Medicare administrative data including claims to predict epilepsy status. A neurologist reviewed electronic health record data to assess epilepsy status in a stratified random sample of Medicare beneficiaries aged 65+ years between January 2012 and December 2014. We then reconstructed the full sample using inverse probability sampling weights. We developed prediction models using longitudinal Medicare data, then in a separate sample evaluated the predictive performance of each model, for example, area under the receiver operating characteristic curve (AUROC), sensitivity, and specificity.

RESULTS

Of 20,945 patients in the reconstructed sample, 2.1% had confirmed epilepsy. The best-performing prediction model to identify prevalent epilepsy required epilepsy diagnoses with multiple claims at least 60 days apart, and epilepsy-specific drug claims: AUROC=0.93 [95% confidence interval (CI), 0.90-0.96], and with an 80% diagnostic threshold, sensitivity=87.8% (95% CI, 80.4%-93.2%), specificity=98.4% (95% CI, 98.2%-98.5%). A similar model also performed well in predicting incident epilepsy (k=0.79; 95% CI, 0.66-0.92).

CONCLUSIONS

Prediction models using longitudinal Medicare data perform well in predicting incident and prevalent epilepsy status accurately.