Predicting Future Elective Colon Resection for Diverticulitis Using Patterns of Health Care Utilization

A cross-institutional evaluation on breast cancer phenotyping NLP algorithms on electronic health records

Objective

Transformer-based language models are prevailing in the clinical domain due to their excellent performance on clinical NLP tasks. The generalizability of those models is usually ignored during the model development process. This study evaluated the generalizability of CancerBERT, a Transformer-based clinical NLP model, along with classic machine learning models, i.e., conditional random field (CRF), bi-directional long short-term memory CRF (BiLSTM-CRF), across different clinical institutes through a breast cancer phenotype extraction task.

Materials and methods

Two clinical corpora of breast cancer patients were collected from the electronic health records from the University of Minnesota (UMN) and Mayo Clinic (MC), and annotated following the same guideline. We developed three types of NLP models (i.e., CRF, BiLSTM-CRF and CancerBERT) to extract cancer phenotypes from clinical texts. We evaluated the generalizability of models on different test sets with different learning strategies (model transfer vs locally trained). The entity coverage score was assessed with their association with the model performances.

Results

We manually annotated 200 and 161 clinical documents at UMN and MC. The corpora of the two institutes were found to have higher similarity between the target entities than the overall corpora. The CancerBERT models obtained the best performances among the independent test sets from two clinical institutes and the permutation test set. The CancerBERT model developed in one institute and further fine-tuned in another institute achieved reasonable performance compared to the model developed on local data (micro-F1: 0.925 vs 0.932).

Conclusions

The results indicate the CancerBERT model has superior learning ability and generalizability among the three types of clinical NLP models for our named entity recognition task. It has the advantage to recognize complex entities, e.g., entities with different labels.

Postoperative Use of Biologics was Less Common among Patients with Crohn's Disease With Emergent/Urgent Versus Elective Intestinal Resection

Background & Aims

Given the risk of intestinal resection for Crohn's disease, postoperative treatment may be informed by several risk factors, including resection type. We compared postoperative treatment strategies for Crohn's disease between emergent/urgent versus elective resection.

Methods

We identified patients with intestinal resection for Crohn's disease between 2002-2018 using the MarketScan databases. We classified emergent/urgent resections as those occurring after emergency department admission or after the second day of admission. We estimated adjusted risk differences for the association between resection type (emergent/urgent versus elective) and 6-month postoperative medication strategy (biologic monotherapy, biologic combination therapy with an immunomodulator, immunomodulator monotherapy, other non-biologic medication for Crohn's [5-aminosalicylates, antibiotics, corticosteroids], or no medications for Crohn's).

Results

During 6 months after resection among 4,187 patients, 23% received biologic monotherapy, 6% received combination therapy, 16% received immunomodulator monotherapy, and 36% received other non-biologics. Compared to elective resection, emergent/urgent resection was associated with more common use of "other non-biologic" medications (risk difference 6.4%; 95% confidence interval [CI] 2.8%, 10.0%), but less common use of biologic monotherapy (risk difference -3.2%; 95% CI -6.2%, -0.1%) and no medications (risk difference -3.6%; 95% CI -6.6%, -0.6%).

Conclusions

Although patients with emergent/urgent resection may benefit from more aggressive postoperative therapy, there was evidence that emergent/urgent resection was more associated than elective resection with postoperative use of non-biologics for Crohn's disease. Future studies of treatment patterns and comparative effectiveness of postoperative treatment strategies for Crohn's patients should consider these differences between resection types, which may be important drivers of longer-term outcomes.

Predicting outcomes of pelvic exenteration using machine learning

AIM

We aim to compare machine learning with neural network performance in predicting R0 resection (R0), length of stay > 14 days (LOS), major complication rates at 30 days postoperatively (COMP) and survival greater than 1 year (SURV) for patients having pelvic exenteration for locally advanced and recurrent rectal cancer.

METHOD

A deep learning computer was built and the programming environment was established. The PelvEx Collaborative database was used which contains anonymized data on patients who underwent pelvic exenteration for locally advanced or locally recurrent colorectal cancer between 2004 and 2014. Logistic regression, a support vector machine and an artificial neural network (ANN) were trained. Twenty per cent of the data were used as a test set for calculating prediction accuracy for R0, LOS, COMP and SURV. Model performance was measured by plotting receiver operating characteristic (ROC) curves and calculating the area under the ROC curve (AUROC).

RESULTS

Machine learning models and ANNs were trained on 1147 cases. The AUROC for all outcome predictions ranged from 0.608 to 0.793 indicating modest to moderate predictive ability. The models performed best at predicting LOS > 14 days with an AUROC of 0.793 using preoperative and operative data. Visualized logistic regression model weights indicate a varying impact of variables on the outcome in question.

CONCLUSION

This paper highlights the potential for predictive modelling of large international databases. Current data allow moderate predictive ability of both complex ANNs and more classic methods.