Predicting outcome of epilepsy surgery in clinical practice: Prediction models vs. clinical acumen

Prediction tools and risk stratification in epilepsy surgery

OBJECTIVE

This study was undertaken to conduct external validation of previously published epilepsy surgery prediction tools using a large independent multicenter dataset and to assess whether these tools can stratify patients for being operated on and for becoming free of disabling seizures (International League Against Epilepsy stage 1 and 2).

METHODS

We analyzed a dataset of 1562 patients, not used for tool development. We applied two scales: Epilepsy Surgery Grading Scale (ESGS) and Seizure Freedom Score (SFS); and two versions of Epilepsy Surgery Nomogram (ESN): the original version and the modified version, which included electroencephalographic data. For the ESNs, we used calibration curves and concordance indexes. We stratified the patients into three tiers for assessing the chances of attaining freedom from disabling seizures after surgery: high (ESGS = 1, SFS = 3-4, ESNs > 70%), moderate (ESGS = 2, SFS = 2, ESNs = 40%-70%), and low (ESGS = 2, SFS = 0-1, ESNs < 40%). We compared the three tiers as stratified by these tools, concerning the proportion of patients who were operated on, and for the proportion of patients who became free of disabling seizures.

RESULTS

The concordance indexes for the various versions of the nomograms were between .56 and .69. Both scales (ESGS, SFS) and nomograms accurately stratified the patients for becoming free of disabling seizures, with significant differences among the three tiers (p < .05). In addition, ESGS and the modified ESN accurately stratified the patients for having been offered surgery, with significant difference among the three tiers (p < .05).

SIGNIFICANCE

ESGS and the modified ESN (at thresholds of 40% and 70%) stratify patients undergoing presurgical evaluation into three tiers, with high, moderate, and low chance for favorable outcome, with significant differences between the groups concerning having surgery and becoming free of disabling seizures. Stratifying patients for epilepsy surgery has the potential to help select the optimal candidates in underprivileged areas and better allocate resources in developed countries.

Predicting seizure outcome after epilepsy surgery: Do we need more complex models, larger samples, or better data?

OBJECTIVE

The accurate prediction of seizure freedom after epilepsy surgery remains challenging. We investigated if (1) training more complex models, (2) recruiting larger sample sizes, or (3) using data-driven selection of clinical predictors would improve our ability to predict postoperative seizure outcome using clinical features. We also conducted the first substantial external validation of a machine learning model trained to predict postoperative seizure outcome.

METHODS

We performed a retrospective cohort study of 797 children who had undergone resective or disconnective epilepsy surgery at a tertiary center. We extracted patient information from medical records and trained three models-a logistic regression, a multilayer perceptron, and an XGBoost model-to predict 1-year postoperative seizure outcome on our data set. We evaluated the performance of a recently published XGBoost model on the same patients. We further investigated the impact of sample size on model performance, using learning curve analysis to estimate performance at samples up to N = 2000. Finally, we examined the impact of predictor selection on model performance.

RESULTS

Our logistic regression achieved an accuracy of 72% (95% confidence interval [CI] = 68%-75%, area under the curve [AUC] = .72), whereas our multilayer perceptron and XGBoost both achieved accuracies of 71% (95% CIMLP  = 67%-74%, AUCMLP  = .70; 95% CIXGBoost own  = 68%-75%, AUCXGBoost own  = .70). There was no significant difference in performance between our three models (all p > .4) and they all performed better than the external XGBoost, which achieved an accuracy of 63% (95% CI = 59%-67%, AUC = .62; pLR  = .005, pMLP  = .01, pXGBoost own  = .01) on our data. All models showed improved performance with increasing sample size, but limited improvements beyond our current sample. The best model performance was achieved with data-driven feature selection.

SIGNIFICANCE

We show that neither the deployment of complex machine learning models nor the assembly of thousands of patients alone is likely to generate significant improvements in our ability to predict postoperative seizure freedom. We instead propose that improved feature selection alongside collaboration, data standardization, and model sharing is required to advance the field.

Short and long term prediction of seizure freedom in drug-resistant focal epilepsy surgery

BACKGROUND

The selection of candidates for drug-resistant focal epilepsy surgery is essential to achieve the best post-surgical outcomes.

OBJECTIVE

To develop two prediction models for seizure freedom in the short and long-term follow-up and from them to create a risk calculator in order to individualize the selection of candidates for surgery and future therapies in each patients.

METHODS

A sample of 64 consecutive patients who underwent epilepsy surgery at two Cuban tertiary health institutions between 2012 and 2020 constituted the basis for the prediction models. Two models were obtained through the novel methodology, based on biomarker selection reached by resampling methods, cross-validation and high-accuracy index measured through the area under the receiving operating curve (ROC) procedure.

RESULTS

The first, to pre-operative model included five predictors: epilepsy type, seizures per month, ictal pattern, interictal EEG topography and normal or abnormal magnetic resonance imaging,. it's precision was 0.77 at one year, and with four years and more 0.63. The second model including variables from the trans-surgical and post-surgical stages: the interictal discharges in the post-surgical EEG, incomplete or complete resection of the epileptogenic zone, the surgical techniques employed and disappearance of the discharge in post-resection electrocorticography; the precision of this model was 0.82 at one year, and with four years and more 0.97.

CONCLUSIONS

The introduction of trans-surgical and post-surgical variables increase the prediction of the pre-surgical model. A risk calculator was developed using these prediction models, which could be useful as an accurate tool to improve the prediction in epilepsy surgery.

Factors Predicting Outcome After Intracranial EEG Evaluation in Patients With Medically Refractory Epilepsy

BACKGROUND AND OBJECTIVES

The aim of this study was to identify predictors of a resective surgery and subsequent seizure freedom following intracranial EEG (ICEEG) for seizure-onset localization.

METHODS

This is a retrospective chart review of 178 consecutive patients with medically refractory epilepsy who underwent ICEEG monitoring from 2002 to 2015. Univariable and multivariable regression analysis identified independent predictors of resection vs other options. Stepwise Akaike information criteria with the aid of clinical consideration were used to select the best multivariable model for predicting resection and outcome. Discrete time survival analysis was used to analyze the factors predicting seizure-free outcome. Cumulative probability of seizure freedom was analyzed using Kaplan-Meier curves and compared between resection and nonresection groups. Additional univariate analysis was performed on 8 select clinical scenarios commonly encountered during epilepsy surgical evaluations.

RESULTS

Multivariable analysis identified the presence of a lesional MRI, presurgical hypothesis suggesting temporal lobe onset, and a nondominant hemisphere implant as independent predictors of resection (p < 0.0001, area under the receiver operating characteristic curve 0.80, 95% CI 0.73-0.87). Focal ICEEG onset and undergoing a resective surgery predicted absolute seizure freedom at the 5-year follow-up. Patients who underwent resective surgery were more likely to be seizure-free at 5 years compared with continued medical treatment or neuromodulation (60% vs 7%; p < 0.0001, hazard ratio 0.16, 95% CI 0.09-0.28). Even patients thought to have unfavorable predictors (nonlesional MRI or extratemporal lobe hypothesis or dominant hemisphere implant) had ≥50% chance of seizure freedom at 5 years if they underwent resection.

DISCUSSION

Unfavorable predictors, including having nonlesional extratemporal epilepsy, should not deter a thorough presurgical evaluation, including with invasive recordings in many cases. Resective surgery without functional impairment offers the best chance for sustained seizure freedom and should always be considered first.

CLASSIFICATION OF EVIDENCE

This study provides Class II evidence that the presence of a lesional MRI, presurgical hypothesis suggesting temporal lobe onset, and a nondominant hemisphere implant are independent predictors of resection. Focal ICEEG onset and undergoing resection are independent predictors of 5-year seizure freedom.

Automated Identification of Surgical Candidates and Estimation of Postoperative Seizure Freedom in Children - A Focused Review

Surgery is an effective but underused treatment for drug-resistant epilepsy in children. Algorithms to identify surgical candidates and estimate the likelihood of postoperative clinical improvement may be valuable to improve access to epilepsy surgery. We provide a focused review of these approaches. For adults with epilepsy, tools to identify surgical candidates and predict seizure and cognitive outcomes (Ie, Cases for Epilepsy (toolsforepilepsy.com) and Epilepsy Surgery Grading Scale) have been validated and are in use. Analogous tools for children need development. A promising approach is to apply statistical learning tools to clinical datasets, such as electroencephalogram tracings, imaging studies, and the text of clinician notes. Demonstration projects suggest these techniques have the potential to be highly accurate, and await further validation and clinical application.