Accuracy of claims-based algorithms for epilepsy research: Revealing the unseen performance of claims-based studies

Validation of ICD codes for the identification of patients with functional seizures

OBJECTIVE

To evaluate the validity of ICD-9-CM and ICD-10 codes for the identification of patients with functional seizures (FS).

METHODS

We evaluated the charts of 800 patients including 400 in an institution wide sample and 400 in an epilepsy monitoring unit (EMU) sample. Half of the patients from each sample came from 2012-2013 and 2022-2023 since ICD-9-CM codes and ICD-10 codes were exclusively used in these respective periods. The charts of each patient were manually reviewed and evaluated for the presence of epilepsy and FS. Based on the 2013 International League Against Epilepsy (ILAE) Nonepileptic Seizures Task Force guidelines we determined whether a patient had either presumptive (clinically established or documented) or probable/possible FS. We evaluated ICD-9-CM codes 300.11 (conversion disorder) and 780.39 (other convulsions) as well as ICD-10 codes F44.5 (conversion disorder or functional neurological disorder with seizures or convulsions) and R56.9 (unspecified convulsions). The positive predictive value (PPV) of each ICD code was calculated in the institution wide sample and the sensitivity, specificity, PPV and negative predictive values (NPV) were calculated in the EMU sample.

RESULTS

In the institution wide sample, F44.5 had a PPV of 74.0 % (64.6-81.6) for presumptive FS and 80.0 % (71.1-86.7) when probable/possible FS patients were included. The code 300.11 had a PPV of 52.0 % (42.3-61.5) for presumptive FS and 59.0 % (49.2-68.1) when probable/possible FS patients were included. Codes R56.9 and 780.39 had PPVs that were equal to or less than 20 %. In the EMU sample, the code F44.5 had a sensitivity, specificity, PPV and NPV of 67.1 % (56.3-76.3), 95.8 % (90.5-98.2), 91.7 % (81.9-96.4), and 80.7 % (73.4-86.4), respectively for presumptive FS. The code 300.11 had a sensitivity, specificity, PPV and NPV of 30.1 % (20.8-41.4), 95.3 % (90.1-97.8), 78.6 % (60.5-89.8), and 70.4 % (63.1-76.7), respectively for presumptive FS. The codes R56.9 and 780.39 performed poorly.

SIGNIFICANCE

ICD codes had a mixed performance when used to identify patients with FS. ICD-10 code F44.5 appeared to perform the best overall.

Seizure after flumazenil reversal for total intravenous anaesthesia with remimazolam versus propofol: a matched retrospective cohort analysis of a large Japanese nationwide inpatient database

BACKGROUND

Remimazolam is a novel anaesthetic and sedative agent that offers several advantages, including minimal adverse haemodynamic effects and availability of a specific antidote, flumazenil. Flumazenil can induce seizures as an adverse effect; however, the incidence of seizures after flumazenil reversal after total intravenous anaesthesia with remimazolam (remimazolam-flumazenil) remains unknown. We compared the risk of seizures between total i.v. anaesthesia with remimazolam-flumazenil or propofol.

METHODS

We retrospectively identified patients who underwent elective surgery (excluding brain surgery) with total i.v. anaesthesia in Japan between April 2020 and March 2022 using the Japanese Diagnosis Procedure Combination database. Patients were divided into remimazolam-flumazenil and propofol groups. Patients in the remimazolam-flumazenil group were matched to those in the propofol group at a variable ratio of 1:3 (maximum) based on age, sex, hospital, and type of surgery. We conducted conditional logistic regression analyses to assess the association between total i.v. anaesthesia with remimazolam-flumazenil and the incidence of perioperative seizures.

RESULTS

We identified 12 033 patients who underwent total i.v. anaesthesia with remimazolam-flumazenil and 432 275 patients with propofol, creating a matched cohort of 19 105. The crude incidence of seizures was 0.66% (95% confidence interval, 0.63-0.68%). There was no significant difference in seizures between the two groups (adjusted odds ratio, 1.08; 95% confidence interval, 0.49-2.37).

CONCLUSIONS

We observed no significant differences in perioperative seizures between remimazolam-flumazenil and propofol in patients undergoing non-neurological surgery. This suggests that remimazolam-flumazenil is a possible alternative to total i.v. anaesthesia with propofol.

Assessing the Risk for Falls in Older Adults After Initiating Gabapentin Versus Duloxetine

BACKGROUND

The evidence informing the harms of gabapentin use are at risk of bias from comparing users with nonusers.

OBJECTIVE

To describe the risk for fall-related outcomes in older adults starting treatment with gabapentin versus duloxetine.

DESIGN

New user, active comparator study using a target trial emulation framework.

SETTING

MarketScan (IBM) commercial claims between January 2014 and December 2021.

PARTICIPANTS

Adults aged 65 years or older with diabetic neuropathy, postherpetic neuralgia, or fibromyalgia and without depression, anxiety, seizures, or cancer in the 365 days before cohort entry.

INTERVENTION

New initiation of treatment with gabapentin or duloxetine (comparator).

MEASUREMENTS

The primary outcome was the hazard of experiencing any fall-related visit in the 6 months after initiating gabapentin or duloxetine until discontinuation of treatment. Secondary outcomes were hazard of severe fall-related events defined as a fall associated with hip fracture or emergency department visit or hospitalization associated with a fall. Stabilized inverse probability of treatment weighting was used to adjust for baseline characteristics.

RESULTS

Our analytic cohort included 57 086 older adults with a diagnosis of interest initiating treatment with gabapentin (n = 52 152) or duloxetine (n = 4934). Overall median follow-up duration was 30 days (IQR, 30 to 90 days). Weighted cumulative incidence of a fall-related visit per 1000 person-years at 30, 90, and 180 days was 103.60, 90.44, and 84.44 for gabapentin users and 203.43, 177.73, and 158.21 for duloxetine users, respectively. At 6-month follow-up, incident gabapentin users had lower hazard of falls (hazard ratio, 0.52 [95% CI, 0.43 to 0.64]), but there was no difference in the hazards of experiencing severe falls. Results were similar across sensitivity and subgroup analyses.

LIMITATION

Claims may contain fewer frail adults and undercount falls.

CONCLUSION

Compared with incident use of duloxetine, incident use of gabapentin was not associated with increased fall-related visits.

PRIMARY FUNDING SOURCE

None.

Association of first antiseizure medication with acute health care utilization in a cohort of adults with newly diagnosed epilepsy

OBJECTIVE

Epilepsy is primarily treated with antiseizure medications (ASMs). The recommendations for first ASM in newly diagnosed epilepsy are inconsistently followed, and we sought to examine whether nonrecommended first ASM was associated with acute care utilization.

METHODS

We conducted a retrospective cohort study of adults (≥18 years old) with newly diagnosed epilepsy (identified using validated epilepsy/convulsion International Classification of Diseases, Clinical Modification codes) in 2015-2019, sampled from Marketscan's Commercial and Medicare Databases. Exposure of interest was receipt of a non-guideline-recommended ASM, and the primary outcome was acute care utilization (an emergency department visit or hospitalization after the first ASM claim). Descriptive statistics characterized covariates, and multivariable negative binominal regression models were built adjusting for age, sex, Elixhauser Comorbidity Index, comorbid neurologic disease (e.g., stroke), and ASM polypharmacy.

RESULTS

Approximately 14 681 people with new epilepsy were prescribed an ASM within 1 year. The three most prescribed medications were levetiracetam (54%, n = 7912), gabapentin (10%, n = 1462), and topiramate (7%, n = 1022). Approximately 4% (n = 648) were prescribed an ASM that should be avoided, and ~74% of people with new epilepsy had an acute care visit during the follow-up period. Mean number of acute care visits during follow-up was 3.34 for "recommended" ASMs and 4.42 for ASMs that "should be avoided." Prescription of a recommended/neutral ASM as compared to an ASM that should be avoided was associated with reduced likelihood of acute care utilization (incidence rate ratio [IRR] = .85, 95% confidence interval [CI] = .77-.94). The recommended/neutral category of ASMs was not statistically significantly associated with seizure- or epilepsy-specific acute care utilization (IRR = .93, 95% CI = .79-1.09).

SIGNIFICANCE

Adults with new epilepsy are frequent users of acute care. There remain a proportion of persons with epilepsy prescribed ASMs that guidelines suggest avoiding, and these ASMs are associated with increased likelihood of emergency department visit or hospitalization. These findings reinforce the importance of optimizing the choice of first ASM in epilepsy.

Emergency department utilization among adults with epilepsy: A multi-state cross-sectional analysis, 2010-2019

OBJECTIVE

We described patterns and trends in ED use among adults with epilepsy in the United States.

METHODS

Utilizing inpatient and ED discharge data from seven states, we conducted a cross-sectional analysis to identify adult ED visits diagnosed with epilepsy or seizures from 2010 to 2019. Using ED visit counts and estimates of state-level epilepsy prevalence, we calculated ED visit rates overall and by payer, condition, and year.

RESULTS

Our data captured 304,935 ED visits with epilepsy as a primary or secondary diagnosis in 2019. Across the seven states, visit rates ranged between 366 and 726 per 1000 and were higher than rates for adults without epilepsy in all states but one. ED visit rates were highest among Medicare and Medicaid beneficiaries (vs commercial or self-pay). Adults with epilepsy were more likely to be admitted as inpatients. Visits for nervous system disorders were 6.3-8.2 times higher among people with epilepsy, and visits for mental health conditions were 1.2-2.6 times higher. Increases in ED visit rates from 2010 to 2019 among people with epilepsy exceeded increases among adults without by 6.0-27.3 percentage points.

CONCLUSION

Adults with epilepsy visit the ED frequently and visit rates have been increasing over time. These results underscore the importance of identifying factors contributing to ED use and designing tailored interventions to improve ambulatory care quality.

On the Dependence of the Critical Success Index (CSI) on Prevalence

The critical success index (CSI) is an established metric used in meteorology to verify the accuracy of weather forecasts. It is defined as the ratio of hits to the sum of hits, false alarms, and misses. Translationally, CSI has gained popularity as a unitary outcome measure in various clinical situations where large numbers of true negatives may influence the interpretation of other, more traditional, outcome measures, such as specificity (Spec) and negative predictive value (NPV), or when unified interpretation of positive predictive value (PPV) and sensitivity (Sens) is needed. The derivation of CSI from measures including PPV has prompted questions as to whether and how CSI values may vary with disease prevalence (P), just as PPV estimates are dependent on P, and hence whether CSI values are generalizable between studies with differing prevalences. As no detailed study of the relation of CSI to prevalence has been undertaken hitherto, the dataset of a previously published test accuracy study of a cognitive screening instrument was interrogated to address this question. Three different methods were used to examine the change in CSI across a range of prevalences, using both the Bayes formula and equations directly relating CSI to Sens, PPV, P, and the test threshold (Q). These approaches showed that, as expected, CSI does vary with prevalence, but the dependence differs according to the method of calculation that is adopted. Bayesian rescaling of both Sens and PPV generates a concave curve, suggesting that CSI will be maximal at a particular prevalence, which may vary according to the particular dataset.

Comparing the efficacy of anti-seizure medications using matched cohorts on a large insurance claims database

Epilepsy is a severe chronic neurological disease affecting 60 million people worldwide. Primary treatment is with anti-seizure medicines (ASMs), but many patients continue to experience seizures. We used retrospective insurance claims data on 280,587 patients with uncontrolled epilepsy (UE), defined as status epilepticus, need for a rescue medicine, or admission or emergency visit for an epilepsy code. We conducted a computational risk ratio analysis between pairs of ASMs using a causal inference method, in order to match 1034 clinical factors and simulate randomization. Data was extracted from the MarketScan insurance claims Research Database records from 2011 to 2015. The cohort consisted of individuals over 18 years old with a diagnosis of epilepsy who took one of eight ASMs and had more than a year of history prior to the filling of the drug prescription. Seven ASM exposures were analyzed: topiramate, phenytoin, levetiracetam, gabapentin, lamotrigine, valproate, and carbamazepine or oxcarbazepine (treated as the same exposure). We calculated the risk ratio of UE between pairs of ASM after controlling for bias with inverse propensity weighting applied to 1034 factors, such as demographics, confounding illnesses, non-epileptic conditions treated by ASMs, etc. All ASMs exhibited a significant reduction in the prevalence of UE, but three drugs showed pair-wise differences compared to other ASMs. Topiramate consistently was associated with a lower risk of UE, with a mean risk ratio range of 0.68-0.93 (average 0.82, CI: 0.56-1.08). Phenytoin and levetiracetam were consistently associated with a higher risk of UE with mean risk ratio ranges of 1.11 to 1.47 (average 1.13, CI 0.98-1.65) and 1.15 to 1.43 (average 1.2, CI 0.72-1.69), respectively. Large-scale retrospective insurance claims data - combined with causal inference analysis - provides an opportunity to compare the effect of treatments in real-world data in populations 1,000-fold larger than those in typical randomized trials. Our causal analysis identified the clinically unexpected finding of topiramate as being associated with a lower risk of UE; and phenytoin and levetiracetam as associated with a higher risk of UE (compared to other studied drugs, not to baseline). However, we note that our data set for this study only used insurance claims events, which does not comprise actual seizure frequencies, nor a clear picture of side effects. Our results do not advocate for any change in practice but demonstrate that conclusions from large databases may differ from and supplement those of randomized trials and clinical practice and therefore may guide further investigation.

Use of Healthcare Claims Data to Generate Real-World Evidence on Patients With Drug-Resistant Epilepsy: Practical Considerations for Research

Objectives: Regulatory bodies, health technology assessment agencies, payers, physicians, and other decision-makers increasingly recognize the importance of real-world evidence (RWE) to provide important and relevant insights on treatment patterns, burden/cost of illness, product safety, and long-term and comparative effectiveness. However, RWE generation requires a careful approach to ensure rigorous analysis and interpretation. There are limited examples of comprehensive methodology for the generation of RWE on patients who have undergone neuromodulation for drug-resistant epilepsy (DRE). This is likely due, at least in part, to the many challenges inherent in using real-world data to define DRE, neuromodulation (including type implanted), and related outcomes of interest. We sought to provide recommendations to enable generation of robust RWE that can increase knowledge of "real-world" patients with DRE and help inform the difficult decisions regarding treatment choices and reimbursement for this particularly vulnerable population. Methods: We drew upon our collective decades of experience in RWE generation and relevant disciplines (epidemiology, health economics, and biostatistics) to describe challenges inherent to this therapeutic area and to provide potential solutions thereto within healthcare claims databases. Several examples were provided from our experiences in DRE to further illustrate our recommendations for generation of robust RWE in this therapeutic area. Results: Our recommendations focus on considerations for the selection of an appropriate data source, development of a study timeline, exposure allotment (specifically, neuromodulation implantation for patients with DRE), and ascertainment of relevant outcomes. Conclusions: The need for RWE to inform healthcare decisions has never been greater and continues to grow in importance to regulators, payers, physicians, and other key stakeholders. However, as real-world data sources used to generate RWE are typically generated for reasons other than research, rigorous methodology is required to minimize bias and fully unlock their value.

Association between structural brain MRI abnormalities and epilepsy in older adults

OBJECTIVE

To determine the association between brain MRI abnormalities and incident epilepsy in older adults.

METHODS

Men and women (ages 45-64 years) from the Atherosclerosis Risk in Communities study were followed up from 1987 to 2018 with brain MRI performed between 2011 and 2013. We identified cases of incident late-onset epilepsy (LOE) with onset of seizures occurring after the acquisition of brain MRI. We evaluated the relative pattern of cortical thickness, subcortical volume, and white matter integrity among participants with incident LOE after MRI in comparison with participants without seizures. We examined the association between MRI abnormalities and incident LOE using Cox proportional hazards regression. Models were adjusted for demographics, hypertension, diabetes, smoking, stroke, and dementia status.

RESULTS

Among 1251 participants with brain MRI data, 27 (2.2%) developed LOE after MRI over a median of 6.4 years (25-75 percentile 5.8-6.9) of follow-up. Participants with incident LOE after MRI had higher levels of cortical thinning and white matter microstructural abnormalities before seizure onset compared to those without seizures. In longitudinal analyses, greater number of abnormalities was associated with incident LOE after controlling for demographic factors, risk factors for cardiovascular disease, stroke, and dementia (gray matter: hazard ratio [HR]: 2.3, 95% confidence interval [CI]: 1.0-4.9; white matter diffusivity: HR: 3.0, 95% CI: 1.2-7.3).

INTERPRETATION

This study demonstrates considerable gray and white matter pathology among individuals with LOE, which is present prior to the onset of seizures and provides important insights into the role of neurodegeneration, both of gray and white matter, and the risk of LOE.

Predicting seizure recurrence after an initial seizure-like episode from routine clinical notes using large language models: a retrospective cohort study

BACKGROUND

The evaluation and management of first-time seizure-like events in children can be difficult because these episodes are not always directly observed and might be epileptic seizures or other conditions (seizure mimics). We aimed to evaluate whether machine learning models using real-world data could predict seizure recurrence after an initial seizure-like event.

METHODS

This retrospective cohort study compared models trained and evaluated on two separate datasets between Jan 1, 2010, and Jan 1, 2020: electronic medical records (EMRs) at Boston Children's Hospital and de-identified, patient-level, administrative claims data from the IBM MarketScan research database. The study population comprised patients with an initial diagnosis of either epilepsy or convulsions before the age of 21 years, based on International Classification of Diseases, Clinical Modification (ICD-CM) codes. We compared machine learning-based predictive modelling using structured data (logistic regression and XGBoost) with emerging techniques in natural language processing by use of large language models.

FINDINGS

The primary cohort comprised 14 021 patients at Boston Children's Hospital matching inclusion criteria with an initial seizure-like event and the comparison cohort comprised 15 062 patients within the IBM MarketScan research database. Seizure recurrence based on a composite expert-derived definition occurred in 57% of patients at Boston Children's Hospital and 63% of patients within IBM MarketScan. Large language models with additional domain-specific and location-specific pre-training on patients excluded from the study (F1-score 0·826 [95% CI 0·817-0·835], AUC 0·897 [95% CI 0·875-0·913]) performed best. All large language models, including the base model without additional pre-training (F1-score 0·739 [95% CI 0·738-0·741], AUROC 0·846 [95% CI 0·826-0·861]) outperformed models trained with structured data. With structured data only, XGBoost outperformed logistic regression and XGBoost models trained with the Boston Children's Hospital EMR (logistic regression: F1-score 0·650 [95% CI 0·643-0·657], AUC 0·694 [95% CI 0·685-0·705], XGBoost: F1-score 0·679 [0·676-0·683], AUC 0·725 [0·717-0·734]) performed similarly to models trained on the IBM MarketScan database (logistic regression: F1-score 0·596 [0·590-0·601], AUC 0·670 [0·664-0·675], XGBoost: F1-score 0·678 [0·668-0·687], AUC 0·710 [0·703-0·714]).

INTERPRETATION

Physician's clinical notes about an initial seizure-like event include substantial signals for prediction of seizure recurrence, and additional domain-specific and location-specific pre-training can significantly improve the performance of clinical large language models, even for specialised cohorts.

FUNDING

UCB, National Institute of Neurological Disorders and Stroke (US National Institutes of Health).

Association of Plasma Aβ42/Aβ40 Ratio and Late-Onset Epilepsy: Results From the Atherosclerosis Risk in Communities Study

BACKGROUND AND OBJECTIVES

The objective of this study was to determine the relationship between plasma β-amyloid (Aβ), specifically the ratio of 2 Aβ peptides (the Aβ42/Aβ40 ratio, which correlates with increased accumulation of Aβ in the CNS), and late-onset epilepsy (LOE).

METHODS

We used Medicare fee-for-service claims codes from 1991 to 2018 to identify cases of LOE among 1,424 Black and White men and women enrolled in the Atherosclerosis Risk in Communities (ARIC) study cohort. The Aβ42/Aβ40 ratio was calculated from plasma samples collected from ARIC participants in 1993-1995 (age 50-71 years) and 2011-2013 (age 67-90 years). We used survival analysis accounting for the competing risk of death to determine the relationship between late-life plasma Aβ42/Aβ40, and its change from midlife to late life, and the subsequent development of epilepsy. We adjusted for demographics, the apolipoprotein e4 genotype, and comorbidities, including stroke, dementia, and head injury. A low plasma ratio of 2 Aβ peptides, the Aβ42/Aβ40 ratio, correlates with low CSF Aβ42/Aβ40 and with increased accumulation of Aβ in the CNS.

RESULTS

Decrease in plasma Aβ42/Aβ40 ratio from midlife to late life, but not an isolated measurement of Aβ42/Aβ40, was associated with development of epilepsy in later life. For every 50% reduction in Aβ42/Aβ40, there was a 2-fold increase in risk of epilepsy (adjusted subhazard ratio 2.30, 95% CI 1.27-4.17).

DISCUSSION

A reduction in plasma Aβ42/Aβ40 is associated with an increased risk of subsequent epilepsy. Our observations provide a further validation of the link between Aβ, hyperexcitable states, and LOE.

Association of first anti-seizure medication choice with injuries in older adults with newly diagnosed epilepsy

BACKGROUND

Epilepsy incidence increases exponentially in older adults, who are also at higher risk of adverse drug effects. Anti-seizure medications (ASM) may be associated with sedation and injuries, but discontinuation can result in seizures. We sought to determine whether there was an association between prescribing non-guideline concordant ASM and subsequent injury as this could inform care models.

METHODS

Retrospective cohort study of adults 50 years or older with newly-diagnosed epilepsy in 2015-16, sampled from the MarketScan Databases. The outcome of interest was injury within 1-year of ASM prescription (e.g., burns, falls) and the exposure of interest was ASM category (recommended vs. not recommended by clinical guidelines). Descriptive statistics characterized covariates and a multivariable Cox-regression model was built to examine the association between ASM category and subsequent injury.

RESULTS

5,931 people with newly diagnosed epilepsy were prescribed an ASM within 1-year. The three most common ASMs were: levetiracetam (62.86%), gabapentin (11.73%), and phenytoin (4.45%). Multivariable Cox-regression found that medication category was not associated with injury; however, older age (adjusted hazard ratio (AHR) 1.01/year), history of prior injury (AHR 1.77), traumatic brain injury (AHR 1.55) and ASM polypharmacy (AHR 1.32) were associated with increased hazard of injury.

CONCLUSIONS

Most older adults appear to be getting appropriate first prescriptions for epilepsy. However, a substantial proportion still receives medication that guidelines suggest avoiding. In addition, we show that ASM polypharmacy is associated with an increased hazard of injury within 1- year. Efforts to improve prescribing in older adults with epilepsy should consider how to reduce. both polypharmacy and exposure to medications that guidelines recommend avoiding.

Characterizing the treatment gap in the United States among adult patients with a new diagnosis of epilepsy

OBJECTIVE

Epilepsy is largely a treatable condition with antiseizure medication (ASM). Recent national administrative claims data suggest one third of newly diagnosed adult epilepsy patients remain untreated 3 years after diagnosis. We aimed to quantify and characterize this treatment gap within a large US academic health system leveraging the electronic health record for enriched clinical detail.

METHODS

This retrospective cohort study evaluated the proportion of adult patients in the health system from 2012 to 2020 who remained untreated 3 years after initial epilepsy diagnosis. To identify incident epilepsy, we applied validated administrative health data criteria of two encounters for epilepsy/seizures and/or convulsions, and we required no ASM prescription preceding the first encounter. Engagement with the health system at least 2 years before and at least 3 years after diagnosis was required. Among subjects who met administrative data diagnosis criteria, we manually reviewed medical records for a subset of 240 subjects to verify epilepsy diagnosis, confirm treatment status, and elucidate reason for nontreatment. These results were applied to estimate the proportion of the full cohort with untreated epilepsy.

RESULTS

Of 831 patients who were automatically classified as having incident epilepsy by inclusion criteria, 80 (10%) remained untreated 3 years after incident epilepsy diagnosis. Manual chart review of incident epilepsy classification revealed only 33% (78/240) had true incident epilepsy. We found untreated patients were more frequently misclassified (p < .001). Using corrected counts, we extrapolated to the full cohort (831) and estimated <1%-3% had true untreated epilepsy.

SIGNIFICANCE

We found a substantially lower proportion of patients with newly diagnosed epilepsy remained untreated compared to previous estimates from administrative data analysis. Manual chart review revealed patients were frequently misclassified as having incident epilepsy, particularly patients who were not treated with an ASM. Administrative data analyses utilizing only diagnosis codes may misclassify patients as having incident epilepsy.

Identification of patients with epilepsy using automated electronic health records phenotyping

OBJECTIVE

Unstructured data present in electronic health records (EHR) are a rich source of medical information; however, their abstraction is labor intensive. Automated EHR phenotyping (AEP) can reduce the need for manual chart review. We present an AEP model that is designed to automatically identify patients diagnosed with epilepsy.

METHODS

The ground truth for model training and evaluation was captured from a combination of structured questionnaires filled out by physicians for a subset of patients and manual chart review using customized software. Modeling features included indicators of the presence of keywords and phrases in unstructured clinical notes, prescriptions for antiseizure medications (ASMs), International Classification of Diseases (ICD) codes for seizures and epilepsy, number of ASMs and epilepsy-related ICD codes, age, and sex. Data were randomly divided into training (70%) and hold-out testing (30%) sets, with distinct patients in each set. We trained regularized logistic regression and an extreme gradient boosting models. Model performance was measured using area under the receiver operating curve (AUROC) and area under the precision-recall curve (AUPRC), with 95% confidence intervals (CI) estimated via bootstrapping.

RESULTS

Our study cohort included 3903 adults drawn from outpatient departments of nine hospitals between February 2015 and June 2022 (mean age = 47 ± 18 years, 57% women, 82% White, 84% non-Hispanic, 70% with epilepsy). The final models included 285 features, including 246 keywords and phrases captured from 8415 encounters. Both models achieved AUROC and AUPRC of 1 (95% CI = .99-1.00) in the hold-out testing set.

SIGNIFICANCE

A machine learning-based AEP approach accurately identifies patients with epilepsy from notes, ICD codes, and ASMs. This model can enable large-scale epilepsy research using EHR databases.

The epidemiology of epilepsy in older adults: A narrative review by the ILAE Task Force on Epilepsy in the Elderly

In an aging world, it is important to know the burden of epilepsy affecting populations of older persons. We performed a selective review of epidemiological studies that we considered to be most informative, trying to include data from all parts of the world. We emphasized primary reports rather than review articles. We reviewed studies reporting the incidence and prevalence of epilepsy that focused on an older population as well as studies that included a wider age range if older persons were tabulated as a subgroup. There is strong evidence that persons older than approximately 60 years incur an increasing risk of both acute symptomatic seizures and epilepsy. In wealthier countries, the incidence of epilepsy increases sharply after age 60 or 65 years. This phenomenon was not always observed among reports from populations with lower socioeconomic status. This discrepancy may reflect differences in etiologies, methods of ascertainment, or distribution of ages; this is an area for more research. We identified other areas for which there are inadequate data. Incidence data are scarcer than prevalence data and are missing for large areas of the world. Prevalence is lower than would be expected from cumulative incidence, possibly because of remissions, excess mortality, or misdiagnosis of acute symptomatic seizures as epilepsy. Segmentation by age, frailty, and comorbidities is desirable, because "epilepsy in the elderly" is otherwise too broad a concept. Data are needed on rates of status epilepticus and drug-resistant epilepsy using the newer definitions. Many more data are needed from low-income populations and from developing countries. Greater awareness of the high rates of seizures among older adults should lead to more focused diagnostic efforts for individuals. Accurate data on epilepsy among older adults should drive proper allocation of treatments for individuals and resources for societies.

Channeling of New Neuropsychiatric Drugs-Impact on Safety and Effectiveness Studies

This study aimed to examine differential prescribing due to channeling and propensity score non-overlap over time in new versus established treatments for common neurological conditions. We conducted cross-sectional analyses on a national sample of US commercially insured adults using 2005-2019 data. We compared new users of recently approved versus established medications for management of diabetic peripheral neuropathy (pregabalin versus gabapentin), Parkinson disease psychosis (pimavanserin versus quetiapine), and epilepsy (brivaracetam versus levetiracetam). Within these drug pairs, we compared demographic, clinical, and healthcare utilization characteristics of recipients of each drug. In addition, we fit yearly propensity score models for each condition and assessed propensity score non-overlap over time. For all three drug pairs, users of the more recently approved medications more frequently had prior treatment (pregabalin = 73.9%, gabapentin = 38.7%; pimavanserin = 41.1%, quetiapine = 14.0%; brivaracetam = 93.4%, levetiracetam = 32.1%). Propensity score non-overlap and its resulting sample loss after trimming were the greatest in the first year that the more recently approved medication was available (diabetic peripheral neuropathy, 12.4% non-overlap; Parkinson disease psychosis, 6.1%; epilepsy, 43.2%) and subsequently improved. Newer neuropsychiatric therapies appear to be channeled to individuals with refractory disease or intolerance to other treatments, leading to potential confounding and biased comparative effectiveness and safety study findings when compared to established treatments. Propensity score non-overlap should be reported in comparative studies that include newer medications. When studies comparing newer and established treatments are critically needed as soon as new treatments enter the market, investigators should recognize the potential for channeling bias and implement methodological approaches like those demonstrated in this study to understand and improve this issue in such studies.

PheValuator 2.0: Methodological improvements for the PheValuator approach to semi-automated phenotype algorithm evaluation

PURPOSE

Phenotype algorithms are central to performing analyses using observational data. These algorithms translate the clinical idea of a health condition into an executable set of rules allowing for queries of data elements from a database. PheValuator, a software package in the Observational Health Data Sciences and Informatics (OHDSI) tool stack, provides a method to assess the performance characteristics of these algorithms, namely, sensitivity, specificity, and positive and negative predictive value. It uses machine learning to develop predictive models for determining a probabilistic gold standard of subjects for assessment of cases and non-cases of health conditions. PheValuator was developed to complement or even replace the traditional approach of algorithm validation, i.e., by expert assessment of subject records through chart review. Results in our first PheValuator paper suggest a systematic underestimation of the PPV compared to previous results using chart review. In this paper we evaluate modifications made to the method designed to improve its performance.

METHODS

The major changes to PheValuator included allowing all diagnostic conditions, clinical observations, drug prescriptions, and laboratory measurements to be included as predictors within the modeling process whereas in the prior version there were significant restrictions on the included predictors. We also have allowed for the inclusion of the temporal relationships of the predictors in the model. To evaluate the performance of the new method, we compared the results from the new and original methods against results found from the literature using traditional validation of algorithms for 19 phenotypes. We performed these tests using data from five commercial databases.

RESULTS

In the assessment aggregating all phenotype algorithms, the median difference between the PheValuator estimate and the gold standard estimate for PPV was reduced from -21 (IQR -34, -3) in Version 1.0 to 4 (IQR -3, 15) using Version 2.0. We found a median difference in specificity of 3 (IQR 1, 4.25) for Version 1.0 and 3 (IQR 1, 4) for Version 2.0. The median difference between the two versions of PheValuator and the gold standard for estimates of sensitivity was reduced from -39 (-51, -20) to -16 (-34, -6).

CONCLUSION

PheValuator 2.0 produces estimates for the performance characteristics for phenotype algorithms that are significantly closer to estimates from traditional validation through chart review compared to version 1.0. With this tool in researcher's toolkits, methods, such as quantitative bias analysis, may now be used to improve the reliability and reproducibility of research studies using observational data.

Impact of COVID-19 pandemic on epilepsy care in Japan: A national-level multicenter retrospective cohort study

OBJECTIVE

The impact of the coronavirus disease 2019 (COVID-19) pandemic on epilepsy care across Japan was investigated by conducting a multicenter retrospective cohort study.

METHODS

This study included monthly data on the frequency of (1) visits by outpatients with epilepsy, (2) outpatient electroencephalography (EEG) studies, (3) telemedicine for epilepsy, (4) admissions for epilepsy, (5) EEG monitoring, and (6) epilepsy surgery in epilepsy centers and clinics across Japan between January 2019 and December 2020. We defined the primary outcome as epilepsy-center-specific monthly data divided by the 12-month average in 2019 for each facility. We determined whether the COVID-19 pandemic-related factors (such as year [2019 or 2020], COVID-19 cases in each prefecture in the previous month, and the state of emergency) were independently associated with these outcomes.

RESULTS

In 2020, the frequency of outpatient EEG studies (-10.7%, p<0.001) and cases with telemedicine (+2,608%, p=0.031) were affected. The number of COVID-19 cases was an independent associated factor for epilepsy admission (-3.75*10-3 % per case, p<0.001) and EEG monitoring (-3.81*10-3 % per case, p = 0.004). Further, the state of emergency was an independent factor associated with outpatient with epilepsy (-11.9%, p<0.001), outpatient EEG (-32.3%, p<0.001), telemedicine for epilepsy (+12,915%, p<0.001), epilepsy admissions (-35.3%; p<0.001), EEG monitoring (-24.7%: p<0.001), and epilepsy surgery (-50.3%, p<0.001).

SIGNIFICANCE

We demonstrated the significant impact that the COVID-19 pandemic had on epilepsy care. These results support those of previous studies and clarify the effect size of each pandemic-related factor on epilepsy care.

Severity of Epilepsy and Response to Antiseizure Medications in Individuals with Multiple Sclerosis: Analysis of a Real-World Dataset

Background and objectives:

Epilepsy is an important comorbidity that affects outcomes for people with multiple sclerosis (MS). However, it is unclear if seizure severity among individuals with coexistence of multiple sclerosis and epilepsy (MS+E) is higher than in those with other focal epilepsies. Our goal was to compare the overall severity of epilepsy in individuals with MS+E versus those with focal epilepsy without MS (E-MS) as defined by seizure-related healthcare utilization, frequency and duration of status epilepticus, and frequency of antiseizure medication (ASM) regimen changes.

Methods:

In this hypothesis-generating study, we analyzed a U.S. commercial nationwide de-identified claims dataset with > 86 million individuals between 1/1/2008 and 8/31/2019. Using validated algorithms, we identified adults with E-MS and those with MS+E. We compared the number and length of seizure-related hospital admissions, the number of claims and unique days with claims for status epilepticus, and the rates of ASM regimen changes between the MS+E and the E-MS groups.

Results:

During the study period, 66,708 individuals with E-MS and 537 with MS+E had ≥ 2 years of coverage after their initial diagnosis of epilepsy. There was no difference between the MS+E and E-MS groups in the percentage of individuals admitted for seizures and/or status epilepticus. However, MS+E with seizure-related admissions had more admissions and longer hospital stays than those with E-MS. MS+E who experienced status epilepticus had more unique days with status epilepticus claims compared to E-MS. MS+E were more likely to have ASM regimen changes in Year 2 after the initial diagnosis of epilepsy and had more ASM changes during Year 2 compared to E-MS. Among individuals with MS+E, there were no differences in our measures of seizure severity for those treated with sodium channel blockers/modulators versus other ASM classes.

Discussion:

This study supports the notion that individuals with MS+E can have more severe epilepsy than those with E-MS. Seizure severity among individuals with MS+E treated with sodium-channel blockers/modulators versus other ASM classes shows no significant differences.

Classification of evidence:

This study provides Class III evidence that individuals with MS+E can have more severe epilepsy than those with E-MS.

Patient characteristics and antiseizure medication pathways in newly diagnosed epilepsy: Feasibility and pilot results using the common data model in a single-center electronic medical record database

INTRODUCTION

Efforts to characterize variability in epilepsy treatment pathways are limited by the large number of possible antiseizure medication (ASM) regimens and sequences, heterogeneity of patients, and challenges of measuring confounding variables and outcomes across institutions. The Observational Health Data Science and Informatics (OHDSI) collaborative is an international data network representing over 1 billion patient records using common data standards. However, few studies have applied OHDSI's Common Data Model (CDM) to the population with epilepsy and none have validated relevant concepts. The goals of this study were to demonstrate the feasibility of characterizing adult patients with epilepsy and ASM treatment pathways using the CDM in an electronic health record (EHR)-derived database.

METHODS

We validated a phenotype algorithm for epilepsy in adults using the CDM in an EHR-derived database (2001-2020) against source records and a prospectively maintained database of patients with confirmed epilepsy. We obtained the frequency of all antecedent conditions and procedures for patients meeting the epilepsy phenotype criteria and characterized ASM exposure sequences over time and by age and sex.

RESULTS

The phenotype algorithm identified epilepsy with 73.0-85.0% positive predictive value and 86.3% sensitivity. Many patients had neurologic conditions and diagnoses antecedent to meeting epilepsy criteria. Levetiracetam incrementally replaced phenytoin as the most common first-line agent, but significant heterogeneity remained, particularly in second-line and subsequent agents. Drug sequences included up to 8 unique ingredients and a total of 1,235 unique pathways were observed.

CONCLUSIONS

Despite the availability of additional ASMs in the last 2 decades and accumulated guidelines and evidence, ASM use varies significantly in practice, particularly for second-line and subsequent agents. Multi-center OHDSI studies have the potential to better characterize the full extent of variability and support observational comparative effectiveness research, but additional work is needed to validate covariates and outcomes.

Integration of Real-World Data and Genetics to Support Target Identification and Validation

Even modest improvements in the probability of success of selecting drug targets which are ultimately approved can substantially reduce the costs of research and development. Drug targets with human genetic evidence of disease association are twice as likely to lead to approved drugs. A key enabler of identifying and validating these genetically validated targets is access to association results from genome-wide genotyping, whole-exome sequencing, and whole-genome sequencing studies with observable traits (often diseases) across large numbers of individuals. Today, linkage between genotype and real-world data (RWD) provides significant opportunities to not only increase the statistical power of genome-wide association studies by ascertaining additional cases for diseases of interest, but also to improve diversity and coverage of association studies across the disease phenome. As RWD-genetics linked resources continue to grow in diversity of participants, breadth of data captured, length of observation, and number of participants, there is a greater need to leverage the experience of RWD experts, clinicians, and highly experienced geneticists together to understand which lessons and frameworks from general research using RWD sources are relevant to improve genetics-driven drug discovery and development. This paper describes new challenges and opportunities for phenotypes enabled by diverse RWD sources, considerations in the use of RWD phenotypes for disease gene identification across the disease phenome, and challenges and opportunities in leveraging RWD phenotypes in target validation. The paper concludes with views on the future directions for phenotype development using RWD, and key questions requiring further research and development to advance this nascent field.

Definitions of Drug-Resistant Epilepsy for Administrative Claims Data Research

OBJECTIVE

To assess accuracy of definitions of drug-resistant epilepsy applied to administrative claims data.

METHODS

We randomly sampled 450 patients from a tertiary health system with >1 epilepsy/convulsion encounter and >2 distinct antiseizure medications (ASMs) from 2014-2020 and >2 years of electronic medical records (EMR) data. We established a drug-resistant epilepsy diagnosis at a specific visit by reviewing EMR data and employing a rubric based in the 2010 International League Against Epilepsy definition. We performed logistic regressions to assess clinically-relevant predictors of drug-resistant epilepsy and to inform claims-based definitions.

RESULTS

Of 450 patients reviewed, 150 were excluded for insufficient EMR data. Of the 300 patients included, 98 (33%) met criteria for current drug-resistant epilepsy. The strongest predictors of current drug-resistant epilepsy were drug-resistant epilepsy diagnosis code (OR 16.9, 95% CI 8.8-32.2), >2 ASMs in the prior two years (OR 13.0, 95% CI 5.1-33.3), >3 non-gabapentinoid ASMs (OR 10.3, 95% CI 5.4-19.6), neurosurgery visit (OR 45.2, 95% CI 5.9-344.3), and epilepsy surgery (OR 30.7, 95% CI 7.1-133.3). We created claims-based drug-resistant epilepsy definitions to: 1) maximize overall predictiveness (drug-resistant epilepsy diagnosis; sensitivity 0.86, specificity 0.74, area under the receiver operating characteristics curve [AUROC] 0.80), 2) maximize sensitivity (drug-resistant epilepsy diagnosis or >3 ASMs; sensitivity 0.98, specificity 0.47, AUROC 0.72), and 3) maximize specificity (drug-resistant epilepsy diagnosis and >3 non-gabapentinoid ASMs; sensitivity 0.42, specificity 0.98, AUROC 0.70).

CONCLUSIONS

Our findings provide validation for several claims-based definitions of drug-resistant epilepsy that can be applied to a variety of research questions.

The impact of COVID-19 pandemic on the utilization of ambulatory care for patients with chronic neurological diseases in Japan: Evaluation of an administrative claims database

The COVID-19 pandemic has affected not only the emergency medical system, but also patients' regular ambulatory care, as such decrease in the number of patients visiting outpatient clinics decreased in 2020 than in 2019, or the ban lifting of subsequent visits by telephone for outpatient clinics since March 2020 in lieu of ambulatory care for chronic diseases. In this context, we investigate the impact of the COVID-19 pandemic on ambulatory care at Japanese outpatient clinics for patients with chronic neurological diseases during 2020. We collected data from the administrative claims database (DeSC database) covering more than 1 million individuals. Serial changes in the frequency of subsequent outpatient visits to clinics or hospitals (excluding large hospitals) for chronic ambulatory care of epilepsy, migraine, Parkinson's disease (PD), and Alzheimer's disease (AD) in 2020 were measured. As a result, since April 2020, the monthly outpatient visits for epilepsy, PD, and AD decreased slightly but significantly (approximately 0.90 in relative risk [RR]) but visits for migraine increased (RR = 1.15). Telephone visit was most frequently used in April-May, in less than 5% of monthly outpatient clinic visits for the examined neurological diseases. Outpatient visits for migraine treatment were more likely to be done by telephone than in case of other diseases (adjusted Odds ratio = 2.08). These results suggest that the impact of COVID-19 pandemic on regular ambulatory care for several chronic neurological diseases yielded different effect depending on the disease, in terms of the frequency or type of outpatient visits.

Seizures and status epilepticus may be risk factor for cardiac arrhythmia or cardiac arrest across multiple time frames

OBJECTIVE

To determine if Emergency Department (ED) or inpatient encounters for epilepsy or status epilepticus are associated with increased odds of cardiac arrhythmia or cardiac arrest over successively longer time frames.

METHODS

The State Inpatient and ED Databases (from New York, Florida, and California) are statewide datasets containing data on 97% of hospitalizations and ED encounters from these states. In this retrospective, case-crossover study, we used International Classification of Diseases, Ninth Revision, Clinical Modification codes to identify index cardiac arrhythmia encounters. Exposures were inpatient or ED encounters for epilepsy or status epilepticus. The case-crossover analysis tested whether an epilepsy or status epilepticus encounter within various case periods (1, 3, 7, 30, 60, 90, and 180 days prior to index encounter) was associated with subsequent ED or inpatient encounter for cardiac arrhythmia, as compared to control periods of equal length one year prior.

RESULTS

The odds ratio (OR) for cardiac arrhythmia after an epilepsy encounter was significant at all time intervals (OR range 2.37-3.36), and highest at 1 day after epilepsy encounter (OR 3.63, 95% confidence interval [CI] 1.66-7.93, p = 0.0013). The OR after status epilepticus was significant at 7- to 180-day intervals (OR range 2.25-2.74), and highest at 60 days (OR 2.74, CI 2.09-3.61, p < 0.0001).

SIGNIFICANCE

Epilepsy and status epilepticus events are associated with increased odds of subsequent cardiac arrhythmia or cardiac arrest over multiple chronic timeframes. Increased cardiac surveillance may be warranted to minimize morbidity and mortality in patients with epilepsy.

Epilepsy in adulthood: Prevalence, incidence, and associated antiepileptic drug use in autistic adults in a state Medicaid system

LAY ABSTRACT

Epilepsy is more common in autistic children compared to children without autism, but we do not have good estimates of how many autistic adults have epilepsy. We used data from a full population of 7513 autistic adults who received Medicaid in Wisconsin to figure out the proportion of autistic adults who have epilepsy, as compared to 18,429 adults with intellectual disability. We also wanted to assess how often epilepsy is first diagnosed in adulthood. Finally, we wanted to see whether antiepileptic drugs are being used to treat epilepsy in autistic adults. We found that 34.6% of autistic adults with intellectual disability and 11.1% of autistic adults without intellectual disability had epilepsy, compared to 27.0% of adults with intellectual disability alone. Autistic women and autistic adults with intellectual disability were more likely than autistic men and autistic adults without intellectual disability to have both previous and new diagnoses of epilepsy. Finally, we found that antiepileptic medications are commonly prescribed to autistic people who do not have epilepsy potentially to treat mental health conditions or behavior problems, and that antiepileptic medications are not always prescribed to autistic people with epilepsy even though they are indicated as a first-line epilepsy treatment. The findings of this study highlight the need to effectively treat and prevent epilepsy in autistic adults.

Delays and disparities in diagnosis for adults with epilepsy: Findings from U.S. Medicaid data

OBJECTIVE

To identify disparities in care pathways and time from first seizure to epilepsy diagnosis, we examined 2010-2014 Medicaid claims (including pharmacy) data from 16 States for individuals with incident epilepsy.

METHODS

We identified adults (18-64) with an incident epilepsy diagnosis from 1/2012 through 6/2014. These individuals were enrolled in Medicaid for the entire study period and had no history of anti-epileptic drug (AED) use before their first seizure claim. We identified care pathways and calculated the duration from initial seizure to epilepsy diagnosis. We tested associations between these pathways and race/ethnicity, as well as time differences between care pathways using a Chi-squared and Kruskal-Wallis tests.

RESULTS

The 14,337 adults followed five different care pathways. Their overall median duration from first seizure code to epilepsy diagnosis code was 19.0 months (interquartile range: 4.6, 30.4), and 56.0% filled an AED prescription. Some minorities were more likely to follow pathways with increased durations and delay to diagnosis, and the duration to diagnosis varied significantly across the care pathways.

SIGNIFICANCE

The many different care pathways seen in people with epilepsy show substantial and significant time delays between first seizure diagnosis and epilepsy diagnosis, including significant racial/ethnic disparities that warrant attention.

Accuracy of ICD-10-CM claims-based definitions for epilepsy and seizure type

OBJECTIVE

To evaluate the accuracy of ICD-10-CM claims-based definitions for epilepsy and classifying seizure types in the outpatient setting.

METHODS

We reviewed electronic health records (EHR) for a cohort of adults aged 18+ years seen by six neurologists who had an outpatient visit at a level 4 epilepsy center between 01/2019-09/2019. The neurologists used a standardized documentation template to capture the diagnosis of epilepsy (yes/no/unsure), seizure type (focal/generalized/unknown), and seizure frequency in the EHR. Using linked ICD-10-CM codes assigned by the provider, we assessed the accuracy of claims-based definitions for epilepsy, focal seizure type, and generalized seizure type against the reference-standard EHR documentation by estimating sensitivity (Sn), specificity (Sp), positive predictive value (PPV), and negative predictive value (NPV).

RESULTS

There were 673 eligible outpatient encounters. After review of EHRs for standardized documentation, an analytic sample consisted of 520 encounters representing 402 unique patients. In the EHR documentation, 93.5 % (n = 486/520) of encounters were with patients with a diagnosis of epilepsy. Of those, 66.0 % (n = 321/486) had ≥1 focal seizure, 41.6 % (n = 202/486) had ≥1 generalized seizure, and 7% (n = 34/486) had ≥1 unknown seizure. An ICD-10-CM definition for epilepsy (i.e., ICD-10 G40.X) achieved Sn = 84.4 % (95 % CI 80.8-87.5%), Sp = 79.4 % (95 % CI 62.1-91.3%), PPV = 98.3 % (95 % CI 96.6-99.3%), and NPV = 26.2 % (95 % CI 18.0-35.8%). The classification of focal vs generalized/unknown seizures achieved Sn = 69.8 % (95 % CI 64.4-74.8%), Sp = 79.4 % (95 % CI 72.4-85.3%), PPV = 86.8 % (95 % CI 82.1-90.7%), and NPV = 57.5 % (95 % CI 50.8-64.0%).

CONCLUSIONS

Claims-based definitions using groups of ICD-10-CM codes assigned by neurologists in routine outpatient clinic visits at a level 4 epilepsy center performed well in discriminating between patients with and without a diagnosis of epilepsy and between seizure types.

Exacerbation of hepatic cirrhosis may trigger admission for epilepsy and status epilepticus

OBJECTIVE

To determine whether acute exacerbations of cirrhotic liver disease are associated with higher odds of readmission for epilepsy or status epilepticus.

METHODS

The New York State Inpatient Database is a statewide dataset containing data on 97% of hospitalizations for New York State. In this retrospective, case-crossover design study, we used International Classification of Diseases, Ninth Revision, Clinical Modification codes to identify index status epilepticus and epilepsy admissions. The primary exposure was defined as admission due to an acute exacerbation of cirrhotic liver disease. The case-crossover analysis tested whether exposure to a hepatic exacerbation within progressively longer case periods (14, 30, 60, 90, 120, 150, and 180 days before index admission), compared to control periods 1 year before the case period, was associated with readmission for epilepsy or status epilepticus.

RESULTS

The odds ratio for subsequent admission for epilepsy after exposure to an acute exacerbation of cirrhotic liver disease was significant in the 30-day window at 2.072 (95% confidence interval [CI] = 1.095-3.92, P = .0252) and peaked in the 150-day window at 2.742 (95% CI = 1.817-4.137, P < .0001). In the status epilepticus group, all case periods demonstrated significantly elevated odds of subsequent admission following hepatic exacerbation.

SIGNIFICANCE

Hepatic exacerbations are associated with increased odds for hospital admissions for epilepsy and status epilepticus across several timeframes.

Epilepsy Among Elderly Medicare Beneficiaries: A Validated Approach to Identify Prevalent and Incident Epilepsy

BACKGROUND

Uncertain validity of epilepsy diagnoses within health insurance claims and other large datasets have hindered efforts to study and monitor care at the population level.

OBJECTIVES

To develop and validate prediction models using longitudinal Medicare administrative data to identify patients with actual epilepsy among those with the diagnosis.

RESEARCH DESIGN, SUBJECTS, MEASURES

We used linked electronic health records and Medicare administrative data including claims to predict epilepsy status. A neurologist reviewed electronic health record data to assess epilepsy status in a stratified random sample of Medicare beneficiaries aged 65+ years between January 2012 and December 2014. We then reconstructed the full sample using inverse probability sampling weights. We developed prediction models using longitudinal Medicare data, then in a separate sample evaluated the predictive performance of each model, for example, area under the receiver operating characteristic curve (AUROC), sensitivity, and specificity.

RESULTS

Of 20,945 patients in the reconstructed sample, 2.1% had confirmed epilepsy. The best-performing prediction model to identify prevalent epilepsy required epilepsy diagnoses with multiple claims at least 60 days apart, and epilepsy-specific drug claims: AUROC=0.93 [95% confidence interval (CI), 0.90-0.96], and with an 80% diagnostic threshold, sensitivity=87.8% (95% CI, 80.4%-93.2%), specificity=98.4% (95% CI, 98.2%-98.5%). A similar model also performed well in predicting incident epilepsy (k=0.79; 95% CI, 0.66-0.92).

CONCLUSIONS

Prediction models using longitudinal Medicare data perform well in predicting incident and prevalent epilepsy status accurately.

Measuring diabetes guideline adherence with claims data: systematic construction of indicators and related challenges

OBJECTIVES

Indicators of guideline adherence are frequently used to examine the appropriateness of healthcare services. Only some potential indicators are actually usable for research with routine administrative claims data, potentially leading to a biased selection of research questions. This study aimed at developing a systematic approach to extract potential indicators from clinical practice guidelines (CPG), evaluate their feasibility for research with claims data and assess how the extracted set reflected different types of healthcare services. Diabetes mellitus (DM), Swiss national guidelines and health insurance claims data were analysed as a model case.

METHODS

CPG for diabetes patients were retrieved from the Swiss Endocrinology and Diabetes Society website. Recommendation statements involving a specific healthcare intervention for a defined patient population were translated into indicators of guideline adherence. Indicators were classified according to disease stage and healthcare service type. We assessed for all indicators whether they could be analysed with Swiss mandatory health insurance administrative claims data.

RESULTS

A total of 93 indicators were derived from 15 CPG, representing all sectors of diabetes care. For 63 indicators, the target population could not be identified using claims data only. For 67 indicators, the intervention could not be identified. Nine (10%) of all indicators were feasible for research with claims data (three addressed gestational diabetes and screening, five screening for complications and one glucose measurement). Some types of healthcare services, eg, management of risk factors, treatment of the disease and secondary prevention, lacked corresponding indicators feasible for research.

CONCLUSIONS

Our systematic approach could identify a number of indicators of healthcare service utilisation, feasible for DM research with Swiss claims data. Some areas of healthcare were covered less well. The approach could be applied to other diseases and countries, helping to identify the potential bias in the selection of indicators and optimise research.

Assessment and effect of a gap between new-onset epilepsy diagnosis and treatment in the US

Objective

To estimate the treatment gap between a new epilepsy diagnosis and antiepileptic drug (AED) initiation in the United States.

Methods

Retrospective claims-based cohort study using Truven Health MarketScan databases (commercial and supplemental Medicare, calendar years 2010–2015; Medicaid, 2010–2014) and a validation study using PharMetrics Plus Database linked to LRx claims database (2009–2014). Persons met epilepsy diagnostic criteria, had an index date (first epilepsy diagnosis) with a preceding 2-year baseline (1 year for persons aged 1 to <2 years; none for persons <1 year), and continuous medical and pharmacy enrollment without epilepsy/seizure diagnosis or AED prescription during baseline. Outcomes included percentage of untreated persons (no AED prescription) up to 3 years' follow-up and comparative outcomes (incidence rate ratio: untreated persons/treated persons), including medical events and health care resource utilization.

Results

In the primary study, 59,970 persons met selection (or inclusion) criteria; 36.7% of persons with newly diagnosed epilepsy remained untreated up to 3 years after diagnosis. In the validation study (N = 30,890), 31.8% of persons remained untreated up to 3 years after diagnosis. Lack of AED treatment was associated with an adjusted incidence rate ratio (95% confidence interval) of 1.2 (1.2–1.3) for medical events, 2.3 (2.2–2.3) for hospitalizations, and 2.8 (2.7–2.9) for emergency department visits.

Conclusions

One-third of newly diagnosed persons remain untreated up to 3 years after epilepsy diagnosis. The increased risk of medical events and health care utilization highlights the consequences of delayed treatment after epilepsy diagnosis, which might be preventable.

Medicare claims can identify post-stroke epilepsy

OBJECTIVE

There have been no validated Medicare claims-based algorithms available to identify epilepsy by discrete etiology of stroke (e.g., post-stroke epilepsy, PSE) in community-dwelling elderly individuals, despite the increasing availability of large datasets. Our objective was to validate algorithms that detect which patients have true PSE.

METHODS

We linked electronic health records (EHR) to Medicare claims from a Medicare Pioneer Accountable Care Organization (ACO) to identify PSE. A neurologist reviewed 01/2012-12/2014 EHR data from a stratified sample of Medicare patients aged 65+ years to adjudicate a reference-standard to develop an algorithm for identifying patients with PSE. Patient sampling strata included those with: A) epilepsy-related claims diagnosis (n = 534 [all]); B) no diagnosis but neurologist visit (n = 500 [randomly sampled from 4346]); C) all others (n = 500 [randomly sampled from 16,065]). We reconstructed the full sample using inverse probability sampling weights; then used half to derive algorithms and assess performance, and the remainder to confirm performance. We evaluated predictive performance across several measures, e.g., specificity, sensitivity, negative and positive predictive values (NPV, PPV). We selected our best performing algorithms based on the greatest specificity and sensitivity.

RESULTS

Of 20,943 patients in the reconstructed sample, 13.6% of patients with epilepsy had reference-standard PSE diagnosis, which represents a 3-year overall prevalence of 0.28% or 28/10,000, and a prevalence within the subpopulation with stroke of 3%. The best algorithm included three conditions: (a) at least one cerebrovascular claim AND one epilepsy-specific anticonvulsant OR (b) at least one cerebrovascular claim AND one electroencephalography claim (specificity 100.0% [95% CI 99.9%-100.0%], NPV 98.8% [98.6%-99.0%], sensitivity 20.6% [95% CI 14.6%-27.9%], PPV 86.5% [95% CI 71.2%-95.5%]).

CONCLUSION

Medicare claims can identify elderly Medicare beneficiaries with PSE with high accuracy. Future epidemiological surveillance of epilepsy could incorporate similar algorithms to accurately identify epilepsy by varying etiologies.

Claims data analyses unable to properly characterize the value of neurologists in epilepsy care

OBJECTIVE

To determine the association of a neurologist visit with health care use and cost outcomes for patients with incident epilepsy.

METHODS

Using health care claims data for individuals insured by United Healthcare from 2001 to 2016, we identified patients with incident epilepsy. The population was defined by an epilepsy/convulsion diagnosis code (ICD codes 345.xx/780.3x, G40.xx/R56.xx), an antiepileptic prescription filled within the succeeding 2 years, and neither criterion met in the 2 preceding years. Cases were defined as patients who had a neurologist encounter for epilepsy within 1 year after an incident diagnosis; a control cohort was constructed with propensity score matching. Primary outcomes were emergency room (ER) visits and hospitalizations for epilepsy. Secondary outcomes included measures of cost (epilepsy related, not epilepsy related, and antiepileptic drugs) and care escalation (including EEG evaluation and epilepsy surgery).

RESULTS

After participant identification and propensity score matching, there were 3,400 cases and 3,400 controls. Epilepsy-related ER visits were more likely for cases than controls (year 1: 5.9% vs 2.3%, p < 0.001), as were hospitalizations (year 1: 2.1% vs 0.7%, p < 0.001). Total medical costs for epilepsy care, nonepilepsy care, and antiepileptic drugs were greater for cases (p ≤ 0.001). EEG evaluation and epilepsy surgery occurred more commonly for cases (p ≤ 0.001).

CONCLUSIONS

Patients with epilepsy who visited a neurologist had greater subsequent health care use, medical costs, and care escalation than controls. This comparison using administrative claims is plausibly confounded by case disease severity, as suggested by higher nonepilepsy care costs. Linking patient-centered outcomes to claims data may provide the clinical resolution to assess care value within a heterogeneous population.

Predicting drug-resistant epilepsy - A machine learning approach based on administrative claims data

Patients with drug-resistant epilepsy (DRE) are at high risk of morbidity and mortality, yet their referral to specialist care is frequently delayed. The ability to identify patients at high risk of DRE at the time of treatment initiation, and to subsequently steer their treatment pathway toward more personalized interventions, has high clinical utility. Here, we aim to demonstrate the feasibility of developing algorithms for predicting DRE using machine learning methods. Longitudinal, intersected data sourced from US pharmacy, medical, and adjudicated hospital claims from 1,376,756 patients from 2006 to 2015 were analyzed; 292,892 met inclusion criteria for epilepsy, and 38,382 were classified as having DRE using a proxy measure for drug resistance. Patients were characterized using 1270 features reflecting demographics, comorbidities, medications, procedures, epilepsy status, and payer status. Data from 175,735 randomly selected patients were used to train three algorithms and from the remainder to assess the trained models' predictive power. A model with only age and sex was used as a benchmark. The best model, random forest, achieved an area under the receiver operating characteristic curve (95% confidence interval [CI]) of 0.764 (0.759, 0.770), compared with 0.657 (0.651, 0.663) for the benchmark model. Moreover, predicted probabilities for DRE were well-calibrated with the observed frequencies in the data. The model predicted drug resistance approximately 2 years before patients in the test dataset had failed two antiepileptic drugs (AEDs). Machine learning models constructed using claims data predicted which patients are likely to fail ≥3 AEDs and are at risk of developing DRE at the time of the first AED prescription. The use of such models can ensure that patients with predicted DRE receive specialist care with potentially more aggressive therapeutic interventions from diagnosis, to help reduce the serious sequelae of DRE.