Corrected ROC analysis for misclassified binary outcomes

Notification of locoregional breast cancer recurrence based on pathology reports: A nationwide validation study with the Netherlands Cancer Registry

BACKGROUND

Usually, data on locoregional recurrent breast cancer (LRR) are collected by reviewing all patient files of a specific cohort, despite only few patients actually have a LRR. We describe and validate a new procedure in which notifications of LRRs are obtained via pathology reports, which could improve efficiency.

METHODS

Patients diagnosed with nonmetastatic invasive breast cancer between 2012 and 2016 were identified from the Netherlands Cancer Registry (NCR) and linked to the Dutch Nationwide Pathology Databank (Palga). LRRs were identified using a complex algorithm based on codes and text in pathology reports, whereafter only files from patients with a notification - i.e. patients who were suspected of having had a LRR - were consulted for confirmation and additional information. To validate this procedure, patients diagnosed between January-March 2012 - of whom data on LRRs were previously collected manually by registrars from the Netherlands Cancer Registry - were used as the gold standard. Subsequently, patients with LRRs not notified by the new method were identified and original pathology reports and clinical reports were evaluated to find reasons for the lack of notification.

RESULTS

In total, 88,257 patients were linked to Palga, and 5069 patients were labelled with a notification. In patients diagnosed between January-March 2012 (validation cohort, n = 3092), 270 patients were labelled with a notification. Of these patients, 82 (2.7 %) were diagnosed with a LRR. The notification method identified 63 patients (77 %) with LRRs. Missed notifications were due to clinical diagnoses (not available in Palga, 53 %) or incomplete/incorrect pathological reporting (47 %). The notification method resulted in cost savings of €2.949.127,- as compared to the manual scenario.

CONCLUSION

Using the notification method, almost 80 % of the patients with LRRs were identified, with huge reductions in registration burden and costs. The incompleteness should be considered in future analyses. Improvement in pathology reporting could increase completeness.

The INFLUENCE 3.0 model: Updated predictions of locoregional recurrence and contralateral breast cancer, now also suitable for patients treated with neoadjuvant systemic therapy

BACKGROUND

Individual risk prediction of 5-year locoregional recurrence (LRR) and contralateral breast cancer (CBC) supports decisions regarding personalised surveillance. The previously developed INFLUENCE tool was rebuild, including a recent population and patients who received neoadjuvant systemic therapy (NST).

METHODS

Women, surgically treated for nonmetastatic breast cancer, diagnosed between 2012 and 2016, were selected from the Netherlands Cancer Registry. Cox regression with restricted cubic splines was compared to Random Survival Forest (RSF) to predict five-year LRR and CBC risks. Separate models were developed for NST patients. Discrimination and calibration were assessed by 100x bootstrap resampling.

RESULTS

In the non-NST and NST group, 49,631 and 10,154 patients were included, respectively. Age, mode of detection, histology, sublocalisation, grade, pT, pN, hormonal receptor status ± endocrine treatment, HER2 status ± targeted treatment, surgery ± immediate reconstruction ± radiation therapy, and chemotherapy were significant predictors for LRR and/or CBC in non-NST patients. For NST patients this was similar, but excluding (y)pT and (y)pN status, and including presence of ductal carcinoma in situ, axillary lymph node dissection and pathologic complete response. For non-NST patients, the Cox and RSF models were integrated in the online tool with 5-year AUCs of 0.77 (95%CI:0.77-0.77) and 0.68 (95%CI:0.67-0.68)] for LRR and CBC prediction, respectively. For NST patients, the RSF model performed best (AUCs 0.77 (95%CI:0.76-0.78) and 0.73 (95%CI:0.69-0.76) for LRR and CBC, respectively). Regarding calibration, observed-predicted differences were all <1 %.

CONCLUSION

This INFLUENCE 3.0 models showed moderate performance in LRR and CBC prediction. The models have been made available as online tool to enable clinical decision support regarding personalised follow-up.

BAYESIAN ANALYSIS FOR IMBALANCED POSITIVE-UNLABELLED DIAGNOSIS CODES IN ELECTRONIC HEALTH RECORDS

With the increasing availability of electronic health records (EHR), significant progress has been made on developing predictive inference and algorithms by health data analysts and researchers. However, the EHR data are notoriously noisy due to missing and inaccurate inputs despite the information is abundant. One serious problem is that only a small portion of patients in the database has confirmatory diagnoses while many other patients remain undiagnosed because they did not comply with the recommended examinations. The phenomenon leads to a so-called positive-unlabelled situation and the labels are extremely imbalanced. In this paper, we propose a model-based approach to classify the unlabelled patients by using a Bayesian finite mixture model. We also discuss the label switching issue for the imbalanced data and propose a consensus Monte Carlo approach to address the imbalance issue and improve computational efficiency simultaneously. Simulation studies show that our proposed model-based approach outperforms existing positive-unlabelled learning algorithms. The proposed method is applied on the Cerner EHR for detecting diabetic retinopathy (DR) patients using laboratory measurements. With only 3% confirmatory diagnoses in the EHR database, we estimate the actual DR prevalence to be 25% which coincides with reported findings in the medical literature.

Have you had bleeding from your gums? Self-report to identify giNGival inflammation (The SING diagnostic accuracy and diagnostic model development study)

AIM

To assess the diagnostic performance of self-reported oral health questions and develop a diagnostic model with additional risk factors to predict clinical gingival inflammation in systemically healthy adults in the United Kingdom.

METHODS

Gingival inflammation was measured by trained staff and defined as bleeding on probing (present if bleeding sites ≥ 30%). Sensitivity and specificity of self-reported questions were calculated; a diagnostic model to predict gingival inflammation was developed and its performance (calibration and discrimination) assessed.

RESULTS

We included 2853 participants. Self-reported questions about bleeding gums had the best performance: the highest sensitivity was 0.73 (95% CI 0.70, 0.75) for a Likert item and the highest specificity 0.89 (95% CI 0.87, 0.90) for a binary question. The final diagnostic model included self-reported bleeding, oral health behaviour, smoking status, previous scale and polish received. Its area under the curve was 0.65 (95% CI 0.63-0.67).

CONCLUSION

This is the largest assessment of diagnostic performance of self-reported oral health questions and the first diagnostic model developed to diagnose gingival inflammation. A self-reported bleeding question or our model could be used to rule in gingival inflammation since they showed good sensitivity, but are limited in identifying healthy individuals and should be externally validated.

Prediction of Target-Drug Therapy by Identifying Gene Mutations in Lung Cancer With Histopathological Stained Image and Deep Learning Techniques

Lung cancer is a kind of cancer with high morbidity and mortality which is associated with various gene mutations. Individualized targeted-drug therapy has become the optimized treatment of lung cancer, especially benefit for patients who are not qualified for lung lobectomy. It is crucial to accurately identify mutant genes within tumor region from stained pathological slice. Therefore, we mainly focus on identifying mutant gene of lung cancer by analyzing the pathological images. In this study, we have proposed a method by identifying gene mutations in lung cancer with histopathological stained image and deep learning to predict target-drug therapy, referred to as DeepIMLH. The DeepIMLH algorithm first downloaded 180 hematoxylin-eosin staining (H&E) images of lung cancer from the Cancer Gene Atlas (TCGA). Then deep convolution Gaussian mixture model (DCGMM) was used to perform color normalization. Convolutional neural network (CNN) and residual network (Res-Net) were used to identifying mutated gene from H&E stained imaging and achieved good accuracy. It demonstrated that our method can be used to choose targeted-drug therapy which might be applied to clinical practice. More studies should be conducted though.

Deep learning enables robust assessment and selection of human blastocysts after in vitro fertilization

Visual morphology assessment is routinely used for evaluating of embryo quality and selecting human blastocysts for transfer after in vitro fertilization (IVF). However, the assessment produces different results between embryologists and as a result, the success rate of IVF remains low. To overcome uncertainties in embryo quality, multiple embryos are often implanted resulting in undesired multiple pregnancies and complications. Unlike in other imaging fields, human embryology and IVF have not yet leveraged artificial intelligence (AI) for unbiased, automated embryo assessment. We postulated that an AI approach trained on thousands of embryos can reliably predict embryo quality without human intervention. We implemented an AI approach based on deep neural networks (DNNs) to select highest quality embryos using a large collection of human embryo time-lapse images (about 50,000 images) from a high-volume fertility center in the United States. We developed a framework (STORK) based on Google’s Inception model. STORK predicts blastocyst quality with an AUC of >0.98 and generalizes well to images from other clinics outside the US and outperforms individual embryologists. Using clinical data for 2182 embryos, we created a decision tree to integrate embryo quality and patient age to identify scenarios associated with pregnancy likelihood. Our analysis shows that the chance of pregnancy based on individual embryos varies from 13.8% (age ≥41 and poor-quality) to 66.3% (age <37 and good-quality) depending on automated blastocyst quality assessment and patient age. In conclusion, our AI-driven approach provides a reproducible way to assess embryo quality and uncovers new, potentially personalized strategies to select embryos.

One-carbon metabolite ratios as functional B-vitamin markers and in relation to colorectal cancer risk

One‐carbon metabolism biomarkers are easily measured in plasma, but analyzing them one at a time in relation to disease does not take into account the interdependence of the many factors involved. The relative dynamics of major one‐carbon metabolism branches can be assessed by relating the functional B‐vitamin marker total homocysteine (tHcy) to transsulfuration (total cysteine) and methylation (creatinine) outputs. We validated the ratios of tHcy to total cysteine (Hcy:Cys), tHcy to creatinine (Hcy:Cre) and tHcy to cysteine to creatinine (Hcy:Cys:Cre) as functional markers of B‐vitamin status. We also calculated the associations of these ratios to colorectal cancer (CRC) risk. Furthermore, the relative contribution of potential confounders to the variance of the ratio‐based B‐vitamin markers was calculated by linear regression in a nested case–control study of 613 CRC cases and 1,190 matched controls. Total B‐vitamin status was represented by a summary score comprising Z‐standardized plasma concentrations of folate, cobalamin, betaine, pyridoxal 5′‐phosphate and riboflavin. Associations with CRC risk were estimated using conditional logistic regression. We found that the ratio‐based B‐vitamin markers all outperformed tHcy as markers of total B‐vitamin status, in both CRC cases and controls. In addition, associations with CRC risk were similar for the ratio‐based B‐vitamin markers and total B‐vitamin status (approximately 25% lower risk for high vs. low B‐vitamin status). In conclusion, ratio‐based B‐vitamin markers were good predictors of total B‐vitamin status and displayed similar associations as total B‐vitamin status with CRC risk. Since tHcy and creatinine are routinely clinically analyzed, Hcy:Cre could be easily implemented in clinical practice.

What's new?

While total homocysteine (tHcy) levels are an important biomarker of B‐vitamin status and may be predictive for colorectal cancer (CRC) risk, they are influenced by a variety of factors, such as age, sex, and lifestyle. Here, tHcy was compared to ratio‐based biomarkers of total B‐vitamin status to assess functionality and relation to CRC risk. In CRC patients and controls, the ratio‐based markers outperformed tHcy as indicators of total B‐vitamin status. Their association with CRC risk was similar to that of total B‐vitamin status. Ratio‐based biomarkers could fill a valuable role in assessments of functional B‐vitamin levels and disease risk.

Deep Convolutional Neural Networks Enable Discrimination of Heterogeneous Digital Pathology Images

Pathological evaluation of tumor tissue is pivotal for diagnosis in cancer patients and automated image analysis approaches have great potential to increase precision of diagnosis and help reduce human error. In this study, we utilize several computational methods based on convolutional neural networks (CNN) and build a stand-alone pipeline to effectively classify different histopathology images across different types of cancer. In particular, we demonstrate the utility of our pipeline to discriminate between two subtypes of lung cancer, four biomarkers of bladder cancer, and five biomarkers of breast cancer. In addition, we apply our pipeline to discriminate among four immunohistochemistry (IHC) staining scores of bladder and breast cancers. Our classification pipeline includes a basic CNN architecture, Google's Inceptions with three training strategies, and an ensemble of two state-of-the-art algorithms, Inception and ResNet. Training strategies include training the last layer of Google's Inceptions, training the network from scratch, and fine-tunning the parameters for our data using two pre-trained version of Google's Inception architectures, Inception-V1 and Inception-V3. We demonstrate the power of deep learning approaches for identifying cancer subtypes, and the robustness of Google's Inceptions even in presence of extensive tumor heterogeneity. On average, our pipeline achieved accuracies of 100%, 92%, 95%, and 69% for discrimination of various cancer tissues, subtypes, biomarkers, and scores, respectively. Our pipeline and related documentation is freely available at https://github.com/ih-_lab/CNN_Smoothie.