Accuracy of operating neurosurgeons' prediction of functional levels after intracranial tumor surgery

Development and external validation of clinical prediction models for pituitary surgery

Introduction

Gross total resection (GTR), Biochemical Remission (BR) and restitution of a priorly disrupted hypothalamus pituitary axis (new improvement, IMP) are important factors in pituitary adenoma (PA) resection surgery. Prediction of these metrics using simple and preoperatively available data might help improve patient care and contribute to a more personalized medicine.

Research question

This study aims to develop machine learning models predicting GTR, BR, and IMP in PA resection surgery, using preoperatively available data.

Material and methods

With data from patients undergoing endoscopic transsphenoidal surgery for PAs machine learning models for prediction of GTR, BR and IMP were developed and externally validated. Development was carried out on a registry from Bologna, Italy while external validation was conducted using patient data from Zurich, Switzerland.

Results

The model development cohort consisted of 1203 patients. GTR was achieved in 207 (17.2%, 945 (78.6%) missing), BR in 173 (14.4%, 992 (82.5%) missing) and IMP in 208 (17.3%, 167 (13.9%) missing) cases. In the external validation cohort 206 patients were included and GTR was achieved in 121 (58.7%, 32 (15.5%) missing), BR in 46 (22.3%, 145 (70.4%) missing) and IMP in 42 (20.4%, 7 (3.4%) missing) cases. The AUC at external validation amounted to 0.72 (95% CI: 0.63-0.80) for GTR, 0.69 (0.52-0.83) for BR, as well as 0.82 (0.76-0.89) for IMP.

Discussion and conclusion

All models showed adequate generalizability, performing similarly in training and external validation, confirming the possible potentials of machine learning in helping to adapt surgical therapy to the individual patient.

Surgical Complexity and Complications: The Need for a Common Language

BACKGROUND

Quality measurement and outcome assessment have recently caught an attention of the neurosurgical community, but lack of standardized definitions and methodology significantly complicates these tasks.

OBJECTIVE

To identify a uniform definition of neurosurgical complications, to classify them according to etiology, and to evaluate them comprehensively in cases of intracranial tumor removal in order to establish a new, easy, and practical grading system capable of predicting the risk of postoperative clinical worsening of the patient condition.

METHODS

A retrospective analysis was conducted on all elective surgeries directed at removal of intracranial tumor in the authors' institution during 2-year study period. All sociodemographic, clinical, and surgical factors were extracted from prospectively compiled comprehensive patient registry. Data on all complications, defined as any deviation from the ideal postoperative course occurring within 30 days of the procedure, were collected with consideration of the required treatment and etiology. A logistic regression model was created for identification of independent factors associated with worsening of the Karnofsky Performance Scale (KPS) score at discharge after surgery in comparison with preoperative period. For each identified statistically significant independent predictor of the postoperative worsening, corresponding score was defined, and grading system, subsequently named Milan Complexity Scale (MCS), was formed.

RESULTS

Overall, 746 cases of surgeries for removal of intracranial tumor were analyzed. Postoperative complications of any kind were observed in 311 patients (41.7%). In 223 cases (29.9%), worsening of the KPS score at the time of discharge in comparison with preoperative period was noted. It was independently associated with 5 predictive factors-major brain vessel manipulation, surgery in the posterior fossa, cranial nerve manipulation, surgery in the eloquent area, tumor size >4 cm-which comprised MCS with a range of the total score from 0 to 8 (higher score indicates more complex clinical situations). Patients who demonstrated KPS worsening after surgery had significantly higher total MCS scores in comparison with individuals whose clinical status at discharge was improved or unchanged (3.24 ± 1.55 versus 1.47 ± 1.58; P < 0.001).

CONCLUSION

It is reasonable to define neurosurgical complication as any deviation from the ideal postoperative course occurring within 30 days of the procedure. Suggested MCS allows for standardized assessment of surgical complexity before intervention and for estimating the risk of clinical worsening after removal of intracranial tumor. Collection of data on surgical complexity, occurrence of complications, and postoperative outcomes, using standardized prospectively maintained comprehensive patient registries seems very important for quality measurement and should be attained in all neurosurgical centers.

Patient, Caregiver, and Oncologist Predictions of Quality of Life in Advanced Cancer: Accuracy and Associations with End-of-Life Care and Caregiver Bereavement

OBJECTIVE

Informed treatment decision-making necessitates accurate prognostication,including predictions about quality of life. We examined whether oncologists, patients with advanced cancer, and caregivers accurately predict patients' future quality of life and whether these predictions are prospectively associated with end-of-life care and bereavement.

METHODS

We secondary analyses of clinical trial data. Patients with advanced cancer (n=156), caregivers (n=156), and oncologists (n=38) predicted patient quality of life 3 months into the future. Patients subsequently rated their quality of life 3 months later. Medical record data documented chemotherapy and emergency department (ED)/inpatient visits in the 30 days before death (n=79 decedents). Caregivers self-reported on depression, anxiety, grief, purpose, 21 and regret 7-months post-mortem. In mixed-effects models, patient, caregiver, and oncologist quality-of-life predictions at study entry were used to predict end-of-life care and caregiver outcomes, controlling for patients' quality of life at 3-month follow-up, demographic and clinical characteristics, and nesting within oncologists.

RESULTS

Caregivers (P<.0001) and oncologists (P=.001) predicted lower quality of life than what patients actually experienced. Among decedents, 24.0% received chemotherapy and 54.5% had an ED/inpatient visit. When caregivers' predictions were more negative, patients were less likely to receive chemotherapy (P=.028) or have an ED/inpatient visit (P=.033), and caregivers reported worse depression (P=.002), anxiety (P=.019), and grief (P=.028) and less purpose in life (P<.001) 7-months post-mortem.

CONCLUSION

When caregivers have more negative expectations about patients' quality of life, patients receive less intensive end-of-life care, and caregivers report worse bereavement This article is protected by copyright. All rights reserved.

BOLD Coupling between Lesioned and Healthy Brain Is Associated with Glioma Patients' Recovery

Predicting functional outcomes after surgery and early adjuvant treatment is difficult due to the complex, extended, interlocking brain networks that underpin cognition. The aim of this study was to test glioma functional interactions with the rest of the brain, thereby identifying the risk factors of cognitive recovery or deterioration. Seventeen patients with diffuse non-enhancing glioma (aged 22-56 years) were longitudinally MRI scanned and cognitively assessed before and after surgery and during a 12-month recovery period (55 MRI scans in total after exclusions). We initially found, and then replicated in an independent dataset, that the spatial correlation pattern between regional and global BOLD signals (also known as global signal topography) was associated with tumour occurrence. We then estimated the coupling between the BOLD signal from within the tumour and the signal extracted from different brain tissues. We observed that the normative global signal topography is reorganised in glioma patients during the recovery period. Moreover, we found that the BOLD signal within the tumour and lesioned brain was coupled with the global signal and that this coupling was associated with cognitive recovery. Nevertheless, patients did not show any apparent disruption of functional connectivity within canonical functional networks. Understanding how tumour infiltration and coupling are related to patients' recovery represents a major step forward in prognostic development.

How well do neurosurgeons predict survival in patients with high-grade glioma?

Due to the lack of reliable prognostic tools, prognostication and surgical decisions largely rely on the neurosurgeons’ clinical prediction skills. The aim of this study was to assess the accuracy of neurosurgeons’ prediction of survival in patients with high-grade glioma and explore factors possibly associated with accurate predictions. In a prospective single-center study, 199 patients who underwent surgery for high-grade glioma were included. After surgery, the operating surgeon predicted the patient’s survival using an ordinal prediction scale. A survival curve was used to visualize actual survival in groups based on this scale, and the accuracy of clinical prediction was assessed by comparing predicted and actual survival. To investigate factors possibly associated with accurate estimation, a binary logistic regression analysis was performed. The surgeons were able to differentiate between patients with different lengths of survival, and median survival fell within the predicted range in all groups with predicted survival < 24 months. In the group with predicted survival > 24 months, median survival was shorter than predicted. The overall accuracy of surgeons’ survival estimates was 41%, and over- and underestimations were done in 34% and 26%, respectively. Consultants were 3.4 times more likely to accurately predict survival compared to residents (p = 0.006). Our findings demonstrate that although especially experienced neurosurgeons have rather good predictive abilities when estimating survival in patients with high-grade glioma on the group level, they often miss on the individual level. Future prognostic tools should aim to beat the presented clinical prediction skills.

Memory recovery in relation to default mode network impairment and neurite density during brain tumor treatment

OBJECTIVE

The aim of this study was to test brain tumor interactions with brain networks, thereby identifying protective features and risk factors for memory recovery after resection.

METHODS

Seventeen patients with diffuse nonenhancing glioma (ages 22-56 years) underwent longitudinal MRI before and after surgery, and during a 12-month recovery period (47 MRI scans in total after exclusion). After each scanning session, a battery of memory tests was performed using a tablet-based screening tool, including free verbal memory, overall verbal memory, episodic memory, orientation, forward digit span, and backward digit span. Using structural MRI and neurite orientation dispersion and density imaging (NODDI) derived from diffusion-weighted images, the authors estimated lesion overlap and neurite density, respectively, with brain networks derived from normative data in healthy participants (somatomotor, dorsal attention, ventral attention, frontoparietal, and default mode network [DMN]). Linear mixed-effect models (LMMs) that regressed out the effect of age, gender, tumor grade, type of treatment, total lesion volume, and total neurite density were used to test the potential longitudinal associations between imaging markers and memory recovery.

RESULTS

Memory recovery was not significantly associated with either the tumor location based on traditional lobe classification or the type of treatment received by patients (i.e., surgery alone or surgery with adjuvant chemoradiotherapy). Nonlocal effects of tumors were evident on neurite density, which was reduced not only within the tumor but also beyond the tumor boundary. In contrast, high preoperative neurite density outside the tumor but within the DMN was associated with better memory recovery (LMM, p value after false discovery rate correction [Pfdr] < 10-3). Furthermore, postoperative and follow-up neurite density within the DMN and frontoparietal network were also associated with memory recovery (LMM, Pfdr = 0.014 and Pfdr = 0.001, respectively). Preoperative tumor and postoperative lesion overlap with the DMN showed a significant negative association with memory recovery (LMM, Pfdr = 0.002 and Pfdr < 10-4, respectively).

CONCLUSIONS

Imaging biomarkers of cognitive recovery and decline can be identified using NODDI and resting-state networks. Brain tumors and their corresponding treatment affecting brain networks that are fundamental for memory functioning such as the DMN can have a major impact on patients' memory recovery.

Development and external validation of a clinical prediction model for functional impairment after intracranial tumor surgery

OBJECTIVE

Decision-making for intracranial tumor surgery requires balancing the oncological benefit against the risk for resection-related impairment. Risk estimates are commonly based on subjective experience and generalized numbers from the literature, but even experienced surgeons overestimate functional outcome after surgery. Today, there is no reliable and objective way to preoperatively predict an individual patient's risk of experiencing any functional impairment.

METHODS

The authors developed a prediction model for functional impairment at 3 to 6 months after microsurgical resection, defined as a decrease in Karnofsky Performance Status of ≥ 10 points. Two prospective registries in Switzerland and Italy were used for development. External validation was performed in 7 cohorts from Sweden, Norway, Germany, Austria, and the Netherlands. Age, sex, prior surgery, tumor histology and maximum diameter, expected major brain vessel or cranial nerve manipulation, resection in eloquent areas and the posterior fossa, and surgical approach were recorded. Discrimination and calibration metrics were evaluated.

RESULTS

In the development (2437 patients, 48.2% male; mean age ± SD: 55 ± 15 years) and external validation (2427 patients, 42.4% male; mean age ± SD: 58 ± 13 years) cohorts, functional impairment rates were 21.5% and 28.5%, respectively. In the development cohort, area under the curve (AUC) values of 0.72 (95% CI 0.69-0.74) were observed. In the pooled external validation cohort, the AUC was 0.72 (95% CI 0.69-0.74), confirming generalizability. Calibration plots indicated fair calibration in both cohorts. The tool has been incorporated into a web-based application available at https://neurosurgery.shinyapps.io/impairment/.

CONCLUSIONS

Functional impairment after intracranial tumor surgery remains extraordinarily difficult to predict, although machine learning can help quantify risk. This externally validated prediction tool can serve as the basis for case-by-case discussions and risk-to-benefit estimation of surgical treatment in the individual patient.

Outcome prediction in brain tumor surgery: a literature review on the influence of nonmedical factors

The purpose of the present study was to review the existing data on preoperative nonmedical factors that are predictive of outcome in brain tumor surgery. Our hypothesis was that also the individual characteristics (e.g., emotional state, cognitive status, social relationships) could influence the postoperative course in addition to clinical factors usually investigated in brain tumor surgery. PubMed, Embase, and Scopus were searched from 2008 to 2018 using terms relating to brain tumors, craniotomy, and predictors. All types of outcome were considered: clinical, cognitive, and psychological. Out of 6.288 records identified, 16 articles were selected for analysis and a qualitative synthesis of the prognostic factors was performed. The following nonmedical factors were found to be predictive of surgical outcomes: socio-demographic (age, marital status, type of insurance, gender, socio-economic status, type of hospital), cognitive (preoperative language and cognitive deficits, performance at TMT-B test), and psychological (preoperative depressive symptoms, personality traits, autonomy for daily activities, altered mental status). This review showed that nonmedical predictors of outcome exist in brain tumor surgery. Consequently, individual characteristics (e.g., emotional state, cognitive status, social relationships) can influence the postoperative course in addition to clinical factors.

Advancements in predicting outcomes in patients with glioma: a surgical perspective

Introduction: Diffuse glioma is a challenging neurosurgical entity. Although surgery does not provide a cure, it may greatly influence survival, brain function, and quality of life. Surgical treatment is by nature highly personalized and outcome prediction is very complex. To engage and succeed in this balancing act it is important to make best use of the information available to the neurosurgeon.Areas covered: This narrative review provides an update on advancements in predicting outcomes in patients with glioma that are relevant to neurosurgeons.Expert opinion: The classical 'gut feeling' is notoriously unreliable and better prediction strategies for patients with glioma are warranted. There are numerous tools readily available for the neurosurgeon in predicting tumor biology and survival. Predicting extent of resection, functional outcome, and quality of life remains difficult. Although machine-learning approaches are currently not readily available in daily clinical practice, there are several ongoing efforts with the use of big data sets that are likely to create new prediction models and refine the existing models.

Systematic review and narrative synthesis of surgeons' perception of postoperative outcomes and risk

BACKGROUND

The accuracy with which surgeons can predict outcomes following surgery has not been explored in a systematic way. The aim of this review was to determine how accurately a surgeon's 'gut feeling' or perception of risk correlates with patient outcomes and available risk scoring systems.

METHODS

A systematic review was undertaken in accordance with PRISMA guidelines. A narrative synthesis was performed in accordance with the Guidance on the Conduct of Narrative Synthesis In Systematic Reviews. Studies comparing surgeons' preoperative or postoperative assessment of patient outcomes were included. Studies that made comparisons with risk scoring tools were also included. Outcomes evaluated were postoperative mortality, general and operation-specific morbidity and long-term outcomes.

RESULTS

Twenty-seven studies comprising 20 898 patients undergoing general, gastrointestinal, cardiothoracic, orthopaedic, vascular, urology, endocrine and neurosurgical operations were included. Surgeons consistently overpredicted mortality rates and were outperformed by existing risk scoring tools in six of seven studies comparing area under receiver operating characteristic (ROC) curves (AUC). Surgeons' prediction of general morbidity was good, and was equivalent to, or better than, pre-existing risk prediction models. Long-term outcomes were poorly predicted by surgeons, with AUC values ranging from 0·51 to 0·75. Four of five studies found postoperative risk estimates to be more accurate than those made before surgery.

CONCLUSION

Surgeons consistently overestimate mortality risk and are outperformed by pre-existing tools; prediction of longer-term outcomes is also poor. Surgeons should consider the use of risk prediction tools when available to inform clinical decision-making.

Patients' reported outcome measures and clinical scales in brain tumor surgery: results from a prospective cohort study

BACKGROUND

This study aims to assess surgical outcome in brain tumor surgery using patient reported outcome measures (PROMs) and to compare their results with traditional clinical outcome measurements.

METHOD

Neuro-oncological patients undergoing surgical removal for the lesion were enrolled; MOCA test, PROMs (EUROHIS-QoL, PGWB-S, WHODAS-12), and the clinical scale Karnofsky Performance Status (KPS) were administered to evaluate respectively cognitive status, quality of life, well-being, disability, and functional status before surgery and at 3-month follow-up. Wilcoxon test was performed to evaluate the longitudinal change of test scores, the smallest detectable difference to classify the change of patients in PROMs, the Cohen kappa to investigate the concordance between KPS and PROMs in classifying the patients' change, and Mann-Whitney U test to compare patients with complications and no complications.

RESULTS

A total of 101 patients were enrolled (54 woman, mean age 50.2 ± 14.1, range 20-85): psychological well-being improved at follow-up; 95 patients (94.1%) were improved/unchanged and 6 (5.9%) were worsened according to PROMs; functional status measured with KPS had a slight agreement with quality of life and disability and no agreement with psychological well-being questionnaires; patients with complications had a greater worsening in KPS.

CONCLUSIONS

According to PROMs measuring QoL, disability, and psychological well-being, most of the patients were improved/unchanged after surgery. Since PROMs and KPS detect different aspects of the patients' health status, PROMs should be integrated in surgical outcome evaluation. Furthermore, their association with complications and with other clinical and subjective variables that could influence patient's perception of health status should be investigated.