Development and Assessment of a Machine Learning Model to Help Predict Survival Among Patients With Oral Squamous Cell Carcinoma

Prediction models applying machine learning to oral cavity cancer outcomes: A systematic review

OBJECTIVES

Machine learning platforms are now being introduced into modern oncological practice for classification and prediction of patient outcomes. To determine the current status of the application of these learning models as adjunctive decision-making tools in oral cavity cancer management, this systematic review aims to summarize the accuracy of machine-learning based models for disease outcomes.

METHODS

Electronic databases including PubMed, Scopus, EMBASE, Cochrane Library, LILACS, SciELO, PsychINFO, and Web of Science were searched up until December 21, 2020. Pertinent articles detailing the development and accuracy of machine learning prediction models for oral cavity cancer outcomes were selected in a two-stage process. Quality assessment was conducted using the Quality in Prognosis Studies (QUIPS) tool and results of base studies were qualitatively synthesized by all authors. Outcomes of interest were malignant transformation of precancer lesions, cervical lymph node metastasis, as well as treatment response, and prognosis of oral cavity cancer.

RESULTS

Twenty-seven articles out of 950 citations identified from electronic and manual searching were included in this study. Five studies had low bias concerns on the QUIPS tool. Prediction of malignant transformation, cervical lymph node metastasis, treatment response, and prognosis were reported in three, six, eight, and eleven articles respectively. Accuracy of these learning models on the internal or external validation sets ranged from 0.85 to 0.97 for malignant transformation prediction, 0.78-0.91 for cervical lymph node metastasis prediction, 0.64-1.00 for treatment response prediction, and 0.71-0.99 for prognosis prediction. In general, most trained algorithms predicting these outcomes performed better than alternate methods of prediction. We also found that models including molecular markers in training data had better accuracy estimates for malignant transformation, treatment response, and prognosis prediction.

CONCLUSION

Machine learning algorithms have a satisfactory to excellent accuracy for predicting three of four oral cavity cancer outcomes i.e., malignant transformation, nodal metastasis, and prognosis. However, considering the training approach of many available classifiers, these models may not be streamlined enough for clinical application currently.

[Artificial intelligence in otorhinolaryngology]

Hintergrund

Die fortschreitende Digitalisierung ermöglicht zunehmend den Einsatz von künstlicher Intelligenz (KI). Sie wird Gesellschaft und Medizin in den nächsten Jahren maßgeblich beeinflussen.

Ziel der Arbeit

Darstellung des gegenwärtigen Einsatzspektrums von KI in der Hals-Nasen-Ohren-Heilkunde und Skizzierung zukünftiger Entwicklungen bei der Anwendung dieser Technologie.

Material und Methoden

Es erfolgte die Auswertung und Diskussion wissenschaftlicher Studien und Expertenanalysen.

Ergebnisse

Durch die Verwendung von KI kann der Nutzen herkömmlicher diagnostischer Werkzeuge in der Hals-Nasen-Ohren-Heilkunde gesteigert werden. Zudem kann der Einsatz dieser Technologie die chirurgische Präzision in der Kopf-Hals-Chirurgie weiter erhöhen.

Schlussfolgerungen

KI besitzt ein großes Potenzial zur weiteren Verbesserung diagnostischer und therapeutischer Verfahren in der Hals-Nasen-Ohren-Heilkunde. Allerdings ist die Anwendung dieser Technologie auch mit Herausforderungen verbunden, beispielsweise im Bereich des Datenschutzes.

Automatic Classification of the Korean Triage Acuity Scale in Simulated Emergency Rooms Using Speech Recognition and Natural Language Processing: a Proof of Concept Study

BACKGROUND

Rapid triage reduces the patients' stay time at an emergency department (ED). The Korean Triage Acuity Scale (KTAS) is mandatorily applied at EDs in South Korea. For rapid triage, we studied machine learning-based triage systems composed of a speech recognition model and natural language processing-based classification.

METHODS

We simulated 762 triage cases that consisted of 18 classes with six types of the main symptom (chest pain, dyspnea, fever, stroke, abdominal pain, and headache) and three levels of KTAS. In addition, we recorded conversations between emergency patients and clinicians during the simulation. We used speech recognition models to transcribe the conversation. Bidirectional Encoder Representation from Transformers (BERT), support vector machine (SVM), random forest (RF), and k-nearest neighbors (KNN) were used for KTAS and symptom classification. Additionally, we evaluated the Shapley Additive exPlanations (SHAP) values of features to interpret the classifiers.

RESULTS

The character error rate of the speech recognition model was reduced to 25.21% through transfer learning. With auto-transcribed scripts, support vector machine (area under the receiver operating characteristic curve [AUROC], 0.86; 95% confidence interval [CI], 0.81-0.9), KNN (AUROC, 0.89; 95% CI, 0.85-0.93), RF (AUROC, 0.86; 95% CI, 0.82-0.9) and BERT (AUROC, 0.82; 95% CI, 0.75-0.87) achieved excellent classification performance. Based on SHAP, we found "stress", "pain score point", "fever", "breath", "head" and "chest" were the important vocabularies for determining KTAS and symptoms.

CONCLUSION

We demonstrated the potential of an automatic KTAS classification system using speech recognition models, machine learning and BERT-based classifiers.

Application and Performance of Artificial Intelligence Technology in Oral Cancer Diagnosis and Prediction of Prognosis: A Systematic Review

Oral cancer (OC) is a deadly disease with a high mortality and complex etiology. Artificial intelligence (AI) is one of the outstanding innovations in technology used in dental science. This paper intends to report on the application and performance of AI in diagnosis and predicting the occurrence of OC. In this study, we carried out data search through an electronic search in several renowned databases, which mainly included PubMed, Google Scholar, Scopus, Embase, Cochrane, Web of Science, and the Saudi Digital Library for articles that were published between January 2000 to March 2021. We included 16 articles that met the eligibility criteria and were critically analyzed using QUADAS-2. AI can precisely analyze an enormous dataset of images (fluorescent, hyperspectral, cytology, CT images, etc.) to diagnose OC. AI can accurately predict the occurrence of OC, as compared to conventional methods, by analyzing predisposing factors like age, gender, tobacco habits, and bio-markers. The precision and accuracy of AI in diagnosis as well as predicting the occurrence are higher than the current, existing clinical strategies, as well as conventional statistics like cox regression analysis and logistic regression.

A review of AI and Data Science support for cancer management

INTRODUCTION

Thanks to improvement of care, cancer has become a chronic condition. But due to the toxicity of treatment, the importance of supporting the quality of life (QoL) of cancer patients increases. Monitoring and managing QoL relies on data collected by the patient in his/her home environment, its integration, and its analysis, which supports personalization of cancer management recommendations. We review the state-of-the-art of computerized systems that employ AI and Data Science methods to monitor the health status and provide support to cancer patients managed at home.

OBJECTIVE

Our main objective is to analyze the literature to identify open research challenges that a novel decision support system for cancer patients and clinicians will need to address, point to potential solutions, and provide a list of established best-practices to adopt.

METHODS

We designed a review study, in compliance with the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) guidelines, analyzing studies retrieved from PubMed related to monitoring cancer patients in their home environments via sensors and self-reporting: what data is collected, what are the techniques used to collect data, semantically integrate it, infer the patient's state from it and deliver coaching/behavior change interventions.

RESULTS

Starting from an initial corpus of 819 unique articles, a total of 180 papers were considered in the full-text analysis and 109 were finally included in the review. Our findings are organized and presented in four main sub-topics consisting of data collection, data integration, predictive modeling and patient coaching.

CONCLUSION

Development of modern decision support systems for cancer needs to utilize best practices like the use of validated electronic questionnaires for quality-of-life assessment, adoption of appropriate information modeling standards supplemented by terminologies/ontologies, adherence to FAIR data principles, external validation, stratification of patients in subgroups for better predictive modeling, and adoption of formal behavior change theories. Open research challenges include supporting emotional and social dimensions of well-being, including PROs in predictive modeling, and providing better customization of behavioral interventions for the specific population of cancer patients.

Machine learning in oral squamous cell carcinoma: Current status, clinical concerns and prospects for future-A systematic review

BACKGROUND

Oral cancer can show heterogenous patterns of behavior. For proper and effective management of oral cancer, early diagnosis and accurate prediction of prognosis are important. To achieve this, artificial intelligence (AI) or its subfield, machine learning, has been touted for its potential to revolutionize cancer management through improved diagnostic precision and prediction of outcomes. Yet, to date, it has made only few contributions to actual medical practice or patient care.

OBJECTIVES

This study provides a systematic review of diagnostic and prognostic application of machine learning in oral squamous cell carcinoma (OSCC) and also highlights some of the limitations and concerns of clinicians towards the implementation of machine learning-based models for daily clinical practice.

DATA SOURCES

We searched OvidMedline, PubMed, Scopus, Web of Science, and Institute of Electrical and Electronics Engineers (IEEE) databases from inception until February 2020 for articles that used machine learning for diagnostic or prognostic purposes of OSCC.

ELIGIBILITY CRITERIA

Only original studies that examined the application of machine learning models for prognostic and/or diagnostic purposes were considered.

DATA EXTRACTION

Independent extraction of articles was done by two researchers (A.R. & O.Y) using predefine study selection criteria. We used the Preferred Reporting Items for Systematic Review and Meta-Analysis (PRISMA) in the searching and screening processes. We also used Prediction model Risk of Bias Assessment Tool (PROBAST) for assessing the risk of bias (ROB) and quality of included studies.

RESULTS

A total of 41 studies were published to have used machine learning to aid in the diagnosis/or prognosis of OSCC. The majority of these studies used the support vector machine (SVM) and artificial neural network (ANN) algorithms as machine learning techniques. Their specificity ranged from 0.57 to 1.00, sensitivity from 0.70 to 1.00, and accuracy from 63.4 % to 100.0 % in these studies. The main limitations and concerns can be grouped as either the challenges inherent to the science of machine learning or relating to the clinical implementations.

CONCLUSION

Machine learning models have been reported to show promising performances for diagnostic and prognostic analyses in studies of oral cancer. These models should be developed to further enhance explainability, interpretability, and externally validated for generalizability in order to be safely integrated into daily clinical practices. Also, regulatory frameworks for the adoption of these models in clinical practices are necessary.

Machine learning predicts unpredicted deaths with high accuracy following hepatopancreatic surgery

Background

Machine learning to predict morbidity and mortality-especially in a population traditionally considered low risk-has not been previously examined. We sought to characterize the incidence of death among patients with a low estimated morbidity and mortality risk based on the National Surgical Quality Improvement Program (NSQIP) estimated probability (EP), as well as develop a machine learning model to identify individuals at risk for "unpredicted death" (UD) among patients undergoing hepatopancreatic (HP) procedures.

Methods

The NSQIP database was used to identify patients who underwent elective HP surgery between 2012-2017. The risk of morbidity and mortality was stratified into three tiers (low, intermediate, or high estimated) using a k-means clustering method with bin sorting. A machine learning classification tree and multivariable regression analyses were used to predict 30-day mortality with a 10-fold cross validation. C statistics were used to compare model performance.

Results

Among 63,507 patients who underwent an HP procedure, median patient age was 63 (IQR: 54-71) years. Patients underwent either pancreatectomy (n=38,209, 60.2%) or hepatic resection (n=25,298, 39.8%). Patients were stratified into three tiers of predicted morbidity and mortality risk based on the NSQIP EP: low (n=36,923, 58.1%), intermediate (n=23,609, 37.2%) and high risk (n=2,975, 4.7%). Among 36,923 patients with low estimated risk of morbidity and mortality, 237 patients (0.6%) experienced a UD. According to the classification tree analysis, age was the most important factor to predict UD (importance 16.9) followed by preoperative albumin level (importance: 10.8), disseminated cancer (importance: 6.5), preoperative platelet count (importance: 6.5), and sex (importance 5.9). Among patients deemed to be low risk, the c-statistic for the machine learning derived prediction model was 0.807 compared with an AUC of only 0.662 for the NSQIP EP.

Conclusions

A prognostic model derived using machine learning methodology performed better than the NSQIP EP in predicting 30-day UD among low risk patients undergoing HP surgery.

Interpretable Machine Learning Model for Locoregional Relapse Prediction in Oropharyngeal Cancers

BACKGROUND

There is no evidence to support surgery or radiotherapy as the best treatment for resectable oropharyngeal cancers with a negative HPV status. Predictive algorithms may help to decide which strategy to choose, but they will only be accepted by caregivers and European authorities if they are interpretable. As a proof of concept, we developed a predictive and interpretable algorithm to predict locoregional relapse at 18 months for oropharyngeal cancers as a first step towards that goal.

METHODS

The model was based on clinical and Pyradiomics features extracted from the dosimetric CT scan. Intraclass correlation was used to filter out features dependant on delineation. Correlated redundant features were also removed. An XGBoost model was cross-validated and optimised on the HN1 cohort (79 patients), and performances were assessed on the ART ORL cohort (45 patients). The Shapley Values were used to provide an overall and local explanation of the model.

RESULTS

On the ART ORL cohort, the model trained on HN1 yielded a precision-or predictive positive value-of 0.92, a recall of 0.42, an area under the curve of the receiver operating characteristic of 0.68 and an accuracy of 0.64. The most contributory features were shape Voxel Volume, grey level size zone matrix Small Area Emphasis (glszmSAE), gldm Dependence Non Uniformity Normalized (gldmDNUN), Sex and Age.

CONCLUSIONS

We developed an interpretable and generalizable model that could yield a good precision-positive predictive value-for relapse at 18 months on a different test cohort.

Comparison of machine learning algorithms for the prediction of five-year survival in oral squamous cell carcinoma

BACKGROUND/AIM

Machine learning analyses of cancer outcomes for oral cancer remain sparse compared to other types of cancer like breast or lung. The purpose of the present study was to compare the performance of machine learning algorithms in the prediction of global, recurrence-free five-year survival in oral cancer patients based on clinical and histopathological data.

METHODS

Data were gathered retrospectively from 416 patients with oral squamous cell carcinoma. The data set was divided into training and test data set (75:25 split). Training performance of five machine learning algorithms (Logistic regression, K-nearest neighbours, Naïve Bayes, Decision tree and Random forest classifiers) for prediction was assessed by k-fold cross-validation. Variables used in the machine learning models were age, sex, pain symptoms, grade of lesion, lymphovascular invasion, extracapsular extension, perineural invasion, bone invasion and type of treatment. Variable importance was assessed and model performance on the testing data was assessed using receiver operating characteristic curves, accuracy, sensitivity, specificity and F1 score.

RESULTS

The best performing model was the Decision tree classifier, followed by the Logistic Regression model (accuracy 76% and 60%, respectively). The Naïve Bayes model did not display any predictive value with 0% specificity.

CONCLUSIONS

Machine learning presents a promising and accessible toolset for improving prediction of oral cancer outcomes. Our findings add to a growing body of evidence that Decision tree models are useful in models in predicting OSCC outcomes. We would advise that future similar studies explore a variety of machine learning models including Logistic regression to help evaluate model performance.

Evolution of Risk Calculators and the Dawn of Artificial Intelligence in Predicting Patient Complications

Risk calculators are an underused tool for surgeons and trainees when determining and communicating surgical risk. We summarize some of the more common risk calculators and discuss their evolution and limitations. We also describe artificial intelligence models, which have the potential to help clinicians better understand and use risk assessment.

Recent Advances in the Application of Artificial Intelligence in Otorhinolaryngology-Head and Neck Surgery

This study presents an up-to-date survey of the use of artificial intelligence (AI) in the field of otorhinolaryngology, considering opportunities, research challenges, and research directions. We searched PubMed, the Cochrane Central Register of Controlled Trials, Embase, and the Web of Science. We initially retrieved 458 articles. The exclusion of non-English publications and duplicates yielded a total of 90 remaining studies. These 90 studies were divided into those analyzing medical images, voice, medical devices, and clinical diagnoses and treatments. Most studies (42.2%, 38/90) used AI for image-based analysis, followed by clinical diagnoses and treatments (24 studies). Each of the remaining two subcategories included 14 studies. Machine learning and deep learning have been extensively applied in the field of otorhinolaryngology. However, the performance of AI models varies and research challenges remain.

Comparison of nomogram with machine learning techniques for prediction of overall survival in patients with tongue cancer

BACKGROUND

The prediction of overall survival in tongue cancer is important for planning of personalized care and patient counselling.

OBJECTIVES

This study compares the performance of a nomogram with a machine learning model to predict overall survival in tongue cancer. The nomogram and machine learning model were built using a large data set from the Surveillance, Epidemiology, and End Results (SEER) program database. The comparison is necessary to provide the clinicians with a comprehensive, practical, and most accurate assistive system to predict overall survival of this patient population.

METHODS

The data set used included the records of 7596 tongue cancer patients. The considered machine learning algorithms were logistic regression, support vector machine, Bayes point machine, boosted decision tree, decision forest, and decision jungle. These algorithms were mainly evaluated in terms of the areas under the receiver-operating characteristic (ROC) curve (AUC) and accuracy values. The performance of the algorithm that produced the best result was compared with a nomogram to predict overall survival in tongue cancer patients.

RESULTS

The boosted decision-tree algorithm outperformed other algorithms. When compared with a nomogram using external validation data, the boosted decision tree produced an accuracy of 88.7% while the nomogram showed an accuracy of 60.4%. In addition, it was found that age of patient, T stage, radiotherapy, and the surgical resection were the most prominent features with significant influence on the machine learning model's performance to predict overall survival.

CONCLUSION

The machine learning model provides more personalized and reliable prognostic information of tongue cancer than the nomogram. However, the level of transparency offered by the nomogram in estimating patients' outcomes seems more confident and strengthened the principle of shared decision making between the patient and clinician. Therefore, a combination of a nomogram - machine learning (NomoML) predictive model may help to improve care, provides information to patients, and facilitates the clinicians in making tongue cancer management-related decisions.

Development of a Machine Learning Model for Survival Risk Stratification of Patients With Advanced Oral Cancer

IMPORTANCE

A tool for precisely stratifying postoperative patients with advanced oral cancer is crucial for the treatment plan, such as intensifying or deintensifying the regimen to improve their quality of life and prognosis.

OBJECTIVE

To develop and validate a machine learning-based algorithm that can provide survival risk stratification for patients with advanced oral cancer who have comprehensive clinicopathologic and genetic data.

DESIGN, SETTING, AND PARTICIPANTS

In this prognostic cohort study, the elastic net penalized Cox proportional hazards regression-based risk stratification model was developed and validated using single-center data collected between January 1, 1996, and December 31, 2011. In total, comprehensive clinicopathologic and genetic data (including clinical, pathologic, and 44 cancer-related gene variant profiles) of 334 patients with stage III or IV oral squamous cell carcinoma were used to develop and validate the algorithm in this 15-year cohort study. Data analysis was conducted between February 1, 2018, and May 6, 2020.

MAIN OUTCOMES AND MEASURES

The main outcomes were cancer-specific survival, distant metastasis-free survival, and locoregional recurrence-free survival. Model performance was compared in terms of the Akaike information criterion and the Harrell concordance index (C index).

RESULTS

Complete data were available for 334 patients (315 men; median age at onset, 48 years [interquartile range, 42-56 years]). The predictive models using comprehensive clinicopathologic and genetic data outperformed those using clinicopathologic data alone. In the groups of postoperative patients receiving adjuvant concurrent chemoradiotherapy, the models demonstrated higher classification performance than those using clinicopathologic data alone in cancer-specific survival (mean [SD] C index, 0.689 [0.050] vs 0.673 [0.051]; P = .02) and locoregional recurrence-free survival (mean [SD] C index, 0.693 [0.039] vs 0.678 [0.035]; P = .004). The classification performance in distant metastasis-free survival was not different (mean [SD] C index, 0.702 [0.056] vs 0.688 [0.048]; P = .09).

CONCLUSIONS AND RELEVANCE

A risk stratification model using comprehensive clinicopathologic and genetic data accurately differentiated the high-risk group from the low-risk group in cancer-specific survival and locoregional recurrence-free survival for postoperative patients with advanced oral cancer. This algorithm could be used through an online calculator to provide additional personalized information for postoperative management of patients with advanced oral squamous cell carcinoma.

Utilizing Artificial Intelligence for Head and Neck Cancer Outcomes Prediction From Imaging

Artificial intelligence (AI)-based models have become a growing area of interest in predictive medicine and have the potential to aid physician decision-making to improve patient outcomes. Imaging and radiomics play an increasingly important role in these models. This review summarizes recent developments in the field of radiomics for AI in head and neck cancer. Prediction models for oncologic outcomes, treatment toxicity, and pathological findings have all been created. Exploratory studies are promising; however, validation studies that demonstrate consistency, reproducibility, and prognostic impact remain uncommon. Prospective clinical trials with standardized procedures are required for clinical translation.

Applying nomograms based on the surveillance, epidemiology and end results database to predict long-term overall survival and cancer-specific survival in patients with oropharyngeal squamous cell carcinomas: A case-control research

Few models regarding to the individualized prognosis assessment of oropharyngeal squamous cell carcinoma (OPSCC) patients were documented. The purpose of this study was to establish nomogram model to predict the long-term overall survival (OS) and cancer-specific survival (CSS) of OPSCC patients. The detailed clinical data for the 10,980 OPSCC patients were collected from the surveillance, epidemiology and end results (SEER) database. Furthermore, we applied a popular and reasonable random split-sample method to divide the total 10,980 patients into 2 groups, including 9881 (90%) patients in the modeling cohort and 1099 (10%) patients in the external validation cohort. Among the modeling cohort, 3084 (31.2%) patients were deceased at the last follow-up date. Of those patients, 2188 (22.1%) patients died due to OPSCC. In addition, 896 (9.1%) patients died due to other causes. The median follow-up period was 45 months (1-119 months). We developed 2 nomograms to predict 5- and 8- year OS and CSS using Cox Proportional Hazards model. The nomograms' accuracy was evaluated through the concordance index (C-index) and calibration curves by internal and external validation. The C-indexes of internal validation on the 5- and 8-year OS and CSS were 0.742 and 0.765, respectively. Moreover, the C-indexes of external validation were 0.740 and 0.759, accordingly. Based on a retrospective cohort from the SEER database, we succeeded in constructing 2 nomograms to predict long-term OS and CSS for OPSCC patients, which provides reference for surgeons to develop a treatment plan and individual prognostic evaluations.

The use of artificial intelligence, machine learning and deep learning in oncologic histopathology

BACKGROUND

Recently, there has been a momentous drive to apply advanced artificial intelligence (AI) technologies to diagnostic medicine. The introduction of AI has provided vast new opportunities to improve health care and has introduced a new wave of heightened precision in oncologic pathology. The impact of AI on oncologic pathology has now become apparent, and its use with respect to oral oncology is still in the nascent stage.

DISCUSSION

A foundational overview of AI classification systems used in medicine and a review of common terminology used in machine learning and computational pathology will be presented. This paper provides a focused review on the recent advances in AI and deep learning in oncologic histopathology and oral oncology. In addition, specific emphasis on recent studies that have applied these technologies to oral cancer prognostication will also be discussed.

CONCLUSION

Machine and deep learning methods designed to enhance prognostication of oral cancer have been proposed with much of the work focused on prediction models on patient survival and locoregional recurrences in patients with oral squamous cell carcinomas (OSCC). Few studies have explored machine learning methods on OSCC digital histopathologic images. It is evident that further research at the whole slide image level is needed and future collaborations with computer scientists may progress the field of oral oncology.