Deep radial basis function networks with subcategorization for mitosis detection in breast histopathology images

Due to the intra-class diversity of mitotic cells and the morphological overlap with similarly looking imposters, automatic mitosis detection in histopathology slides is still a challenging task. In this paper, we propose a novel mitosis detection model in a weakly supervised way, which consists of a candidate proposal network and a verification network. The candidate proposal network based on patch learning aims to separate both mitotic cells and their mimics from the background as candidate objects, which substantially reduces missed detections in the screening process of candidates. These obtained candidate results are then fed into the verification network for mitosis refinement. The verification network adopts an RBF-based subcategorization scheme to deal with the problems of high intra-class variability of mitosis and the mimics with similar appearance. We utilize the RBF centers to define subcategories containing mitotic cells with similar properties and capture representative RBF center locations through joint training of classification and clustering. Due to the lower intra-class variation within a subcategory, the localized feature space at subcategory level can better characterize a certain type of mitotic figures and can provide a better similarity measurement for distinguishing mitotic cells from nonmitotic cells. Our experiments manifest that this subcategorization scheme helps improve the performance of mitosis detection and achieves state-of-the-art results on the publicly available mitosis datasets using only weak labels.

A systematic comparison of deep learning methods for Gleason grading and scoring

Prostate cancer is the second most frequent cancer in men worldwide after lung cancer. Its diagnosis is based on the identification of the Gleason score that evaluates the abnormality of cells in glands through the analysis of the different Gleason patterns within tissue samples. The recent advancements in computational pathology, a domain aiming at developing algorithms to automatically analyze digitized histopathology images, lead to a large variety and availability of datasets and algorithms for Gleason grading and scoring. However, there is no clear consensus on which methods are best suited for each problem in relation to the characteristics of data and labels. This paper provides a systematic comparison on nine datasets with state-of-the-art training approaches for deep neural networks (including fully-supervised learning, weakly-supervised learning, semi-supervised learning, Additive-MIL, Attention-Based MIL, Dual-Stream MIL, TransMIL and CLAM) applied to Gleason grading and scoring tasks. The nine datasets are collected from pathology institutes and openly accessible repositories. The results show that the best methods for Gleason grading and Gleason scoring tasks are fully supervised learning and CLAM, respectively, guiding researchers to the best practice to adopt depending on the task to solve and the labels that are available.

Comprehensive mining of information in Weakly Supervised Semantic Segmentation: Saliency semantics and edge semantics

In the studies of Weakly Supervised Semantic Segmentation (WSSS) with image-level labels, there is an issue of incomplete semantic information, which we summarize as insufficient saliency semantic mining and neglected edge semantics. We proposes a two-stage framework, Saliency Semantic Full Mining-Edge Semantic Mining (SSFM-ESM), which views WSSS from the perspective of comprehensive information mining. In the first stage, we rely on SSFM to address the insufficient saliency semantic mining. The network learns feature representations consistent with salient regions via the proposed pixel-level class-agnostic distance loss. Then, the full saliency semantic information is mined by explicitly receiving pixel-level feedback. The initial pseudo-label with complete saliency semantic information can be obtained after the first stage. In the second stage, we focus on the mining of edge semantic information through the proposed edge semantic mining module. Specifically, we guide the initial pseudo-label avoid learning about false semantic information and obtain high-confidence edge semantics. The self-correction ability of the segmentation network is also fully utilized to obtain more edge semantic information. Extensive experiments are conducted on the PASCAL VOC 2012 and MS COCO 2014 datasets to verify the feasibility and superiority of this approach.

Evaluation of mediastinal lymph node segmentation of heterogeneous CT data with full and weak supervision

Accurate lymph node size estimation is critical for staging cancer patients, initial therapeutic management, and assessing response to therapy. Current standard practice for quantifying lymph node size is based on a variety of criteria that use uni-directional or bi-directional measurements. Segmentation in 3D can provide more accurate evaluations of the lymph node size. Fully convolutional neural networks (FCNs) have achieved state-of-the-art results in segmentation for numerous medical imaging applications, including lymph node segmentation. Adoption of deep learning segmentation models in clinical trials often faces numerous challenges. These include lack of pixel-level ground truth annotations for training, generalizability of the models on unseen test domains due to the heterogeneity of test cases and variation of imaging parameters. In this paper, we studied and evaluated the performance of lymph node segmentation models on a dataset that was completely independent of the one used to create the models. We analyzed the generalizability of the models in the face of a heterogeneous dataset and assessed the potential effects of different disease conditions and imaging parameters. Furthermore, we systematically compared fully-supervised and weakly-supervised methods in this context. We evaluated the proposed methods using an independent dataset comprising 806 mediastinal lymph nodes from 540 unique patients. The results show that performance achieved on the independent test set is comparable to that on the training set. Furthermore, neither the underlying disease nor the heterogeneous imaging parameters impacted the performance of the models. Finally, the results indicate that our weakly-supervised method attains 90%- 91% of the performance achieved by the fully supervised training.

Diagnosis and detection of pneumonia using weak-label based on X-ray images: a multi-center study

PURPOSE

Development and assessment the deep learning weakly supervised algorithm for the classification and detection pneumonia via X-ray.

METHODS

This retrospective study analyzed two publicly available dataset that contain X-ray images of pneumonia cases and normal cases. The first dataset from Guangzhou Women and Children's Medical Center. It contains a total of 5,856 X-ray images, which are divided into training, validation, and test sets with 8:1:1 ratio for algorithm training and testing. The deep learning algorithm ResNet34 was employed to build diagnostic model. And the second public dataset were collated by researchers from Qatar University and the University of Dhaka along with collaborators from Pakistan and Malaysia and some medical doctors. A total of 1,300 images of COVID-19 positive cases, 1,300 normal images and 1,300 images of viral pneumonia for external validation. Class activation map (CAM) were used to location the pneumonia lesions.

RESULTS

The ResNet34 model for pneumonia detection achieved an AUC of 0.9949 [0.9910-0.9981] (with an accuracy of 98.29% a sensitivity of 99.29% and a specificity of 95.57%) in the test dataset. And for external validation dataset, the model obtained an AUC of 0.9835[0.9806-0.9864] (with an accuracy of 94.62%, a sensitivity of 92.35% and a specificity of 99.15%). Moreover, the CAM can accurately locate the pneumonia area.

CONCLUSION

The deep learning algorithm can accurately detect pneumonia and locate the pneumonia area based on weak supervision information, which can provide potential value for helping radiologists to improve their accuracy of detection pneumonia patients through X-ray images.

CholecTriplet2022: Show me a tool and tell me the triplet - An endoscopic vision challenge for surgical action triplet detection

Formalizing surgical activities as triplets of the used instruments, actions performed, and target anatomies is becoming a gold standard approach for surgical activity modeling. The benefit is that this formalization helps to obtain a more detailed understanding of tool-tissue interaction which can be used to develop better Artificial Intelligence assistance for image-guided surgery. Earlier efforts and the CholecTriplet challenge introduced in 2021 have put together techniques aimed at recognizing these triplets from surgical footage. Estimating also the spatial locations of the triplets would offer a more precise intraoperative context-aware decision support for computer-assisted intervention. This paper presents the CholecTriplet2022 challenge, which extends surgical action triplet modeling from recognition to detection. It includes weakly-supervised bounding box localization of every visible surgical instrument (or tool), as the key actors, and the modeling of each tool-activity in the form of ‹instrument, verb, target› triplet. The paper describes a baseline method and 10 new deep learning algorithms presented at the challenge to solve the task. It also provides thorough methodological comparisons of the methods, an in-depth analysis of the obtained results across multiple metrics, visual and procedural challenges; their significance, and useful insights for future research directions and applications in surgery.

Large-scale investigation of weakly-supervised deep learning for the fine-grained semantic indexing of biomedical literature

OBJECTIVE

Semantic indexing of biomedical literature is usually done at the level of MeSH descriptors with several related but distinct biomedical concepts often grouped together and treated as a single topic. This study proposes a new method for the automated refinement of subject annotations at the level of MeSH concepts.

METHODS

Lacking labelled data, we rely on weak supervision based on concept occurrence in the abstract of an article, which is also enhanced by dictionary-based heuristics. In addition, we investigate deep learning approaches, making design choices to tackle the particular challenges of this task. The new method is evaluated on a large-scale retrospective scenario, based on concepts that have been promoted to descriptors.

RESULTS

In our experiments concept occurrence was the strongest heuristic achieving a macro-F1 score of about 0.63 across several labels. The proposed method improved it further by more than 4pp.

CONCLUSION

The results suggest that concept occurrence is a strong heuristic for refining the coarse-grained labels at the level of MeSH concepts and the proposed method improves it further.

Cross-modal hashing with missing labels

Hashing-based cross-modal retrieval methods have become increasingly popular due to their advantages in storage and speed. While current methods have demonstrated impressive results, there are still several issues that have not been addressed. Specifically, many of these approaches assume that labels are perfectly assigned, despite the fact that in real-world scenarios, labels are often incomplete or partially missing. There are two reasons for this, as manual labeling can be a complex and time-consuming task, and annotators may only be interested in certain objects. As such, cross-modal retrieval with missing labels is a significant challenge that requires further attention. Moreover, the similarity between labels is frequently ignored, which is important for exploring the high-level semantics of labels. To address these limitations, we propose a novel method called Cross-Modal Hashing with Missing Labels (CMHML). Our method consists of several key components. First, we introduce Reliable Label Learning to preserve reliable information from the observed labels. Next, to infer the uncertain part of the predicted labels, we decompose the predicted labels into latent representations of labels and samples. The representation of samples is extracted from different modalities, which assists in inferring missing labels. We also propose Label Correlation Preservation to enhance the similarity between latent representations of labels. Hash codes are then learned from the representation of samples through Global Approximation Learning. We also construct a similarity matrix according to predicted labels and embed it into hash codes learning to explore the value of labels. Finally, we train linear classifiers to map original samples to a low-dimensional Hamming space. To evaluate the efficacy of CMHML, we conduct extensive experiments on four publicly available datasets. Our method is compared to other state-of-the-art methods, and the results demonstrate that our model performs competitively even when most labels are missing.

Ontology-driven and weakly supervised rare disease identification from clinical notes

BACKGROUND

Computational text phenotyping is the practice of identifying patients with certain disorders and traits from clinical notes. Rare diseases are challenging to be identified due to few cases available for machine learning and the need for data annotation from domain experts.

METHODS

We propose a method using ontologies and weak supervision, with recent pre-trained contextual representations from Bi-directional Transformers (e.g. BERT). The ontology-driven framework includes two steps: (i) Text-to-UMLS, extracting phenotypes by contextually linking mentions to concepts in Unified Medical Language System (UMLS), with a Named Entity Recognition and Linking (NER+L) tool, SemEHR, and weak supervision with customised rules and contextual mention representation; (ii) UMLS-to-ORDO, matching UMLS concepts to rare diseases in Orphanet Rare Disease Ontology (ORDO). The weakly supervised approach is proposed to learn a phenotype confirmation model to improve Text-to-UMLS linking, without annotated data from domain experts. We evaluated the approach on three clinical datasets, MIMIC-III discharge summaries, MIMIC-III radiology reports, and NHS Tayside brain imaging reports from two institutions in the US and the UK, with annotations.

RESULTS

The improvements in the precision were pronounced (by over 30% to 50% absolute score for Text-to-UMLS linking), with almost no loss of recall compared to the existing NER+L tool, SemEHR. Results on radiology reports from MIMIC-III and NHS Tayside were consistent with the discharge summaries. The overall pipeline processing clinical notes can extract rare disease cases, mostly uncaptured in structured data (manually assigned ICD codes).

CONCLUSION

The study provides empirical evidence for the task by applying a weakly supervised NLP pipeline on clinical notes. The proposed weak supervised deep learning approach requires no human annotation except for validation and testing, by leveraging ontologies, NER+L tools, and contextual representations. The study also demonstrates that Natural Language Processing (NLP) can complement traditional ICD-based approaches to better estimate rare diseases in clinical notes. We discuss the usefulness and limitations of the weak supervision approach and propose directions for future studies.

Exploiting weak supervision to facilitate segmentation, classification, and analysis of microplastics (<100 μm) using Raman microspectroscopy images

Reliable quantification and characterization of microplastics are necessary for large-scale and long-term monitoring of their behaviors and evolution in the environment. This is especially true in recent times because of the increase in the production and use of plastics during the pandemic. However, because of the myriad of microplastic morphologies, dynamic environmental forces, and time-consuming and expensive methods to characterize microplastics, it is challenging to understand microplastic transport in the environment. This paper describes a novel approach that compares unsupervised, weakly-supervised, and supervised approaches to facilitate segmentation, classification, and the analysis of <100 μm-sized microplastics without the use of pixel-wise human-labeled data. The secondary aim of this work is to provide insight into what can be accomplished when no human annotations are available, using the segmentation and classification tasks as use cases. In particular, the weakly-supervised segmentation performance surpasses the baseline performance set by the unsupervised approach. Consequently, feature extraction (derived from the segmentation results) provides objective parameters describing microplastic morphologies that will result in better standardization and comparisons of microplastic morphology across future studies. The weakly-supervised performance for microplastic morphology classification (e.g., fiber, spheroid, shard/fragment, irregular) also exceeds the performance of the supervised analogue. Moreover, in contrast to the supervised method, our weakly-supervised approach provides the benefit of pixel-wise detection of microplastic morphology. Pixel-wise detection is used further to improve shape classifications. We also demonstrate a proof-of-concept for distinguishing microplastic particles from non-microplastic particles using verification data from Raman microspectroscopy. As the automation of microplastic monitoring progresses, robust and scalable identification of microplastics based on their morphology may be achievable.

Localization of lung abnormalities on chest X-rays using self-supervised equivariant attention

Chest X-Ray (CXR) images provide most anatomical details and the abnormalities on a 2D plane. Therefore, a 2D view of the 3D anatomy is sometimes sufficient for the initial diagnosis. However, close to fourteen commonly occurring diseases are sometimes difficult to identify by visually inspecting the images. Therefore, there is a drift toward developing computer-aided assistive systems to help radiologists. This paper proposes a deep learning model for the classification and localization of chest diseases by using image-level annotations. The model consists of a modified Resnet50 backbone for extracting feature corpus from the images, a classifier, and a pixel correlation module (PCM). During PCM training, the network is a weight-shared siamese architecture where the first branch applies the affine transform to the image before feeding to the network, while the second applies the same transform to the network output. The method was evaluated on CXR from the clinical center in the ratio of 70:20 for training and testing. The model was developed and tested using the cloud computing platform Google Colaboratory (NVidia Tesla P100 GPU, 16 GB of RAM). A radiologist subjectively validated the results. Our model trained with the configurations mentioned in this paper outperformed benchmark results.

Supplementary Information

The online version contains supplementary material available at 10.1007/s13534-022-00249-5.

Weakly Supervised Learning using Attention gates for colon cancer histopathological image segmentation

Recently, Artificial Intelligence namely Deep Learning methods have revolutionized a wide range of domains and applications. Besides, Digital Pathology has so far played a major role in the diagnosis and the prognosis of tumors. However, the characteristics of the Whole Slide Images namely the gigapixel size, high resolution and the shortage of richly labeled samples have hindered the efficiency of classical Machine Learning methods. That goes without saying that traditional methods are poor in generalization to different tasks and data contents. Regarding the success of Deep learning when dealing with Large Scale applications, we have resorted to the use of such models for histopathological image segmentation tasks. First, we review and compare the classical UNet and Att-UNet models for colon cancer WSI segmentation in a sparsely annotated data scenario. Then, we introduce novel enhanced models of the Att-UNet where different schemes are proposed for the skip connections and spatial attention gates positions in the network. In fact, spatial attention gates assist the training process and enable the model to avoid irrelevant feature learning. Alternating the presence of such modules namely in our Alter-AttUNet model adds robustness and ensures better image segmentation results. In order to cope with the lack of richly annotated data in our AiCOLO colon cancer dataset, we suggest the use of a multi-step training strategy that also deals with the WSI sparse annotations and unbalanced class issues. All proposed methods outperform state-of-the-art approaches but Alter-AttUNet generates the best compromise between accurate results and light network. The model achieves 95.88% accuracy with our sparse AiCOLO colon cancer datasets. Finally, to evaluate and validate our proposed architectures we resort to publicly available WSI data: the NCT-CRC-HE-100K, the CRC-5000 and the Warwick colon cancer histopathological dataset. Respective accuracies of 99.65%, 99.73% and 79.03% were reached. A comparison with state-of-art approaches is established to view and compare the key solutions for histopathological image segmentation.

Leveraging weak supervision to perform named entity recognition in electronic health records progress notes to identify the ophthalmology exam

OBJECTIVE

To develop deep learning models to recognize ophthalmic examination components from clinical notes in electronic health records (EHR) using a weak supervision approach.

METHODS

A corpus of 39,099 ophthalmology notes weakly labeled for 24 examination entities was assembled from the EHR of one academic center. Four pre-trained transformer-based language models (DistilBert, BioBert, BlueBert, and ClinicalBert) were fine-tuned to this named entity recognition task and compared to a baseline regular expression model. Models were evaluated on the weakly labeled test dataset, a human-labeled sample of that set, and a human-labeled independent dataset.

RESULTS

On the weakly labeled test set, all transformer-based models had recall > 0.93, with precision varying from 0.815 to 0.843. The baseline model had lower recall (0.769) and precision (0.682). On the human-annotated sample, the baseline model had high recall (0.962, 95 % CI 0.955-0.067) with variable precision across entities (0.081-0.999). Bert models had recall ranging from 0.771 to 0.831, and precision >=0.973. On the independent dataset, precision was 0.926 and recall 0.458 for BlueBert. The baseline model had better recall (0.708, 95 % CI 0.674-0.738) but worse precision (0.399, 95 % CI -0.352-0.451).

CONCLUSION

We developed the first deep learning system to recognize eye examination components from clinical notes, leveraging a novel opportunity for weak supervision. Transformer-based models had high precision on human-annotated labels, whereas the baseline model had poor precision but higher recall. This system may be used to improve cohort and feature identification using free-text notes.Our weakly supervised approach may help amass large datasets of domain-specific entities from EHRs in many fields.

A data-centric weak supervised learning for highway traffic incident detection

Using the data from loop detector sensors for near-real-time detection of traffic incidents on highways is crucial to averting major traffic congestion. While recent supervised machine learning methods offer solutions to incident detection by leveraging human-labeled incident data, the false alarm rate is often too high to be used in practice. Specifically, the inconsistency in the human labeling of the incidents significantly affects the performance of supervised learning models. To that end, we focus on a data-centric approach to improve the accuracy and reduce the false alarm rate of traffic incident detection on highways. We develop a weak supervised learning workflow to generate high-quality training labels for the incident data without the ground truth labels, and we use those generated labels in the supervised learning setup for final detection. This approach comprises three stages. First, we introduce a data preprocessing and curation pipeline that processes traffic sensor data to generate high-quality training data through leveraging labeling functions, which can be domain knowledge-related or simple heuristic rules. Second, we evaluate the training data generated by weak supervision using three supervised learning models-random forest, k-nearest neighbors, and a support vector machine ensemble-and long short-term memory classifiers. The results show that the accuracy of all of the models improves significantly after using the training data generated by weak supervision. Third, we develop an online real-time incident detection approach that leverages the model ensemble and the uncertainty quantification while detecting incidents. Overall, we show that our proposed weak supervised learning workflow achieves a high incident detection rate (0.90) and low false alarm rate (0.08).

TBGA: a large-scale Gene-Disease Association dataset for Biomedical Relation Extraction

Background

Databases are fundamental to advance biomedical science. However, most of them are populated and updated with a great deal of human effort. Biomedical Relation Extraction (BioRE) aims to shift this burden to machines. Among its different applications, the discovery of Gene-Disease Associations (GDAs) is one of BioRE most relevant tasks. Nevertheless, few resources have been developed to train models for GDA extraction. Besides, these resources are all limited in size—preventing models from scaling effectively to large amounts of data.

Results

To overcome this limitation, we have exploited the DisGeNET database to build a large-scale, semi-automatically annotated dataset for GDA extraction. DisGeNET stores one of the largest available collections of genes and variants involved in human diseases. Relying on DisGeNET, we developed TBGA: a GDA extraction dataset generated from more than 700K publications that consists of over 200K instances and 100K gene-disease pairs. Each instance consists of the sentence from which the GDA was extracted, the corresponding GDA, and the information about the gene-disease pair.

Conclusions

TBGA is amongst the largest datasets for GDA extraction. We have evaluated state-of-the-art models for GDA extraction on TBGA, showing that it is a challenging and well-suited dataset for the task. We made the dataset publicly available to foster the development of state-of-the-art BioRE models for GDA extraction.

Supplementary Information

The online version contains supplementary material available at 10.1186/s12859-022-04646-6.

Using weak supervision to generate training datasets from social media data: a proof of concept to identify drug mentions

Twitter has been a remarkable resource for research in pharmacovigilance in the last decade. Traditionally, rule- or lexicon-based methods have been utilized for automatically extracting drug tweets for human annotation. The process of human annotation to create labeled sets for machine learning models is laborious, time consuming and not scalable. In this work, we demonstrate the feasibility of applying weak supervision (noisy labeling) to select drug data, and build machine learning models using large amounts of noisy labeled data instead of limited gold standard labelled sets. Our results demonstrate the models built with large amounts of noisy data achieve similar performance than models trained on limited gold standard datasets, hence demonstrating that weak supervision helps reduce the need to rely on manual annotation, allowing more data to be easily labeled and useful for downstream machine learning applications, in this case drug mention identification.

Relation extraction from DailyMed structured product labels by optimally combining crowd, experts and machines

The effectiveness of machine learning models to provide accurate and consistent results in drug discovery and clinical decision support is strongly dependent on the quality of the data used. However, substantive amounts of open data that drive drug discovery suffer from a number of issues including inconsistent representation, inaccurate reporting, and incomplete context. For example, databases of FDA-approved drug indications used in computational drug repositioning studies do not distinguish between treatments that simply offer symptomatic relief from those that target the underlying pathology. Moreover, drug indication sources often lack proper provenance and have little overlap. Consequently, new predictions can be of poor quality as they offer little in the way of new insights. Hence, work remains to be done to establish higher quality databases of drug indications that are suitable for use in drug discovery and repositioning studies. Here, we report on the combination of weak supervision (i.e., programmatic labeling and crowdsourcing) and deep learning methods for relation extraction from DailyMed text to create a higher quality drug-disease relation dataset. The generated drug-disease relation data shows a high overlap with DrugCentral, a manually curated dataset. Using this dataset, we constructed a machine learning model to classify relations between drugs and diseases from text into four categories; treatment, symptomatic relief, contradiction, and effect, exhibiting an improvement of 15.5% with Bi-LSTM (F1 score of 71.8%) over the best performing discrete method. Access to high quality data is crucial to building accurate and reliable drug repurposing prediction models. Our work suggests how the combination of crowds, experts, and machine learning methods can go hand-in-hand to improve datasets and predictive models.

Automatic left ventricle volume calculation with explainability through a deep learning weak-supervision methodology

BACKGROUND AND OBJECTIVE

Magnetic resonance imaging is the most reliable imaging technique to assess the heart. More specifically there is great importance in the analysis of the left ventricle, as the main pathologies directly affect this region. In order to characterize the left ventricle, it is necessary to extract its volume. In this work we present a neural network architecture that is capable of directly estimating the left ventricle volume in short axis cine Magnetic Resonance Imaging in the end-diastolic frame and provide a segmentation of the region which is the basis of the volume calculation, thus offering explainability to the estimated value.

METHODS

The network was designed to directly target the volumes to estimate, not requiring any labeled segmentation on the images. The network was based on a 3D U-net with extra layers defined in a scanning module that learned features like the circularity of the objects and the volumes to estimate in a weakly-supervised manner. The only targets defined were the left ventricle volumes and the circularity of the object detected through the estimation of the π value derived from its shape. We had access to 397 cases corresponding to 397 different subjects. We randomly selected 98 cases to use as test set.

RESULTS

The results show a good match between the real and estimated volumes in the test set, with a mean relative error of 8% and a mean absolute error of 9.12 ml with a Pearson correlation coefficient of 0.95. The derived segmentations obtained by the network achieved Dice coefficients with a mean value of 0.79.

CONCLUSIONS

The proposed method is capable of obtaining the left ventricle volume biomarker in the end-diastole and offer an explanation of how it obtains the result in the form of a segmentation mask without the need of segmentation labels to train the algorithm, making it a potentially more trustworthy method for clinicians and a way to train neural networks more easily when segmentation labels are not readily available.

Combining weakly and strongly supervised learning improves strong supervision in Gleason pattern classification

BACKGROUND

One challenge to train deep convolutional neural network (CNNs) models with whole slide images (WSIs) is providing the required large number of costly, manually annotated image regions. Strategies to alleviate the scarcity of annotated data include: using transfer learning, data augmentation and training the models with less expensive image-level annotations (weakly-supervised learning). However, it is not clear how to combine the use of transfer learning in a CNN model when different data sources are available for training or how to leverage from the combination of large amounts of weakly annotated images with a set of local region annotations. This paper aims to evaluate CNN training strategies based on transfer learning to leverage the combination of weak and strong annotations in heterogeneous data sources. The trade-off between classification performance and annotation effort is explored by evaluating a CNN that learns from strong labels (region annotations) and is later fine-tuned on a dataset with less expensive weak (image-level) labels.

RESULTS

As expected, the model performance on strongly annotated data steadily increases as the percentage of strong annotations that are used increases, reaching a performance comparable to pathologists ([Formula: see text]). Nevertheless, the performance sharply decreases when applied for the WSI classification scenario with [Formula: see text]. Moreover, it only provides a lower performance regardless of the number of annotations used. The model performance increases when fine-tuning the model for the task of Gleason scoring with the weak WSI labels [Formula: see text].

CONCLUSION

Combining weak and strong supervision improves strong supervision in classification of Gleason patterns using tissue microarrays (TMA) and WSI regions. Our results contribute very good strategies for training CNN models combining few annotated data and heterogeneous data sources. The performance increases in the controlled TMA scenario with the number of annotations used to train the model. Nevertheless, the performance is hindered when the trained TMA model is applied directly to the more challenging WSI classification problem. This demonstrates that a good pre-trained model for prostate cancer TMA image classification may lead to the best downstream model if fine-tuned on the WSI target dataset. We have made available the source code repository for reproducing the experiments in the paper: https://github.com/ilmaro8/Digital_Pathology_Transfer_Learning.

Using weak supervision and deep learning to classify clinical notes for identification of current suicidal ideation

Mental health concerns, such as suicidal thoughts, are frequently documented by providers in clinical notes, as opposed to structured coded data. In this study, we evaluated weakly supervised methods for detecting "current" suicidal ideation from unstructured clinical notes in electronic health record (EHR) systems. Weakly supervised machine learning methods leverage imperfect labels for training, alleviating the burden of creating a large manually annotated dataset. After identifying a cohort of 600 patients at risk for suicidal ideation, we used a rule-based natural language processing approach (NLP) approach to label the training and validation notes (n = 17,978). Using this large corpus of clinical notes, we trained several statistical machine learning models-logistic classifier, support vector machines (SVM), Naive Bayes classifier-and one deep learning model, namely a text classification convolutional neural network (CNN), to be evaluated on a manually-reviewed test set (n = 837). The CNN model outperformed all other methods, achieving an overall accuracy of 94% and a F1-score of 0.82 on documents with "current" suicidal ideation. This algorithm correctly identified an additional 42 encounters and 9 patients indicative of suicidal ideation but missing a structured diagnosis code. When applied to a random subset of 5,000 clinical notes, the algorithm classified 0.46% (n = 23) for "current" suicidal ideation, of which 87% were truly indicative via manual review. Implementation of this approach for large-scale document screening may play an important role in point-of-care clinical information systems for targeted suicide prevention interventions and improve research on the pathways from ideation to attempt.

Impact of hybrid supervision approaches on the performance of artificial intelligence for the classification of chest radiographs

PURPOSE

To evaluate the impact of different supervision regimens on the training of artificial intelligence (AI) in the classification of chest radiographs as normal or abnormal in a moderately sized cohort of individuals more likely to be outpatients.

MATERIALS AND METHODS

In a retrospective study, 7000 consecutive two-view chest radiographs obtained from 2012 to 2015 were labeled as normal or abnormal based on clinical reports. A convolutional neural network (CNN) was trained on this dataset and then evaluated with an unseen subset of 500 radiographs. Five different training approaches were tested: (1) weak supervision and four hybrid approaches combining weak supervision and extra supervision with annotation in (2) an unbalanced set of normal and abnormal cases, (3) a set of only abnormal cases, (4) a set of only normal cases, and (5) a balanced set of normal and abnormal cases. Standard binary classification metrics were assessed.

RESULTS

The weakly supervised model achieved an accuracy of 82%, but yielded 75 false negative cases, at a sensitivity of 70.0% and a negative predictive value (NPV) of 75.5%. Extra supervision increased NPV at the expense of the false positive rate and overall accuracy. Extra supervision with training using a balance of abnormal and normal radiographs resulted in the greatest increase in NPV (87.2%), improved sensitivity (92.8%), and reduced the number of false negatives by more than fourfold (18 compared to 75 cases).

CONCLUSION

Extra supervision using a balance of annotated normal and abnormal cases applied to a weakly supervised model can minimize the number of false negative cases when classifying two-view chest radiographs. Further refinement of such hybrid training approaches for AI is warranted to refine models for practical clinical applications.

Active learning with point supervision for cost-effective panicle detection in cereal crops

BACKGROUND

Panicle density of cereal crops such as wheat and sorghum is one of the main components for plant breeders and agronomists in understanding the yield of their crops. To phenotype the panicle density effectively, researchers agree there is a significant need for computer vision-based object detection techniques. Especially in recent times, research in deep learning-based object detection shows promising results in various agricultural studies. However, training such systems usually requires a lot of bounding-box labeled data. Since crops vary by both environmental and genetic conditions, acquisition of huge amount of labeled image datasets for each crop is expensive and time-consuming. Thus, to catalyze the widespread usage of automatic object detection for crop phenotyping, a cost-effective method to develop such automated systems is essential.

RESULTS

We propose a point supervision based active learning approach for panicle detection in cereal crops. In our approach, the model constantly interacts with a human annotator by iteratively querying the labels for only the most informative images, as opposed to all images in a dataset. Our query method is specifically designed for cereal crops which usually tend to have panicles with low variance in appearance. Our method reduces labeling costs by intelligently leveraging low-cost weak labels (object centers) for picking the most informative images for which strong labels (bounding boxes) are required. We show promising results on two publicly available cereal crop datasets-Sorghum and Wheat. On Sorghum, 6 variants of our proposed method outperform the best baseline method with more than 55% savings in labeling time. Similarly, on Wheat, 3 variants of our proposed methods outperform the best baseline method with more than 50% of savings in labeling time.

CONCLUSION

We proposed a cost effective method to train reliable panicle detectors for cereal crops. A low cost panicle detection method for cereal crops is highly beneficial to both breeders and agronomists. Plant breeders can obtain quick crop yield estimates to make important crop management decisions. Similarly, obtaining real time visual crop analysis is valuable for researchers to analyze the crop's response to various experimental conditions.

Contextual Deep Regression Network for Volume Estimation in Orbital CT

Diseases of the optic nerve cause structural changes observable through clinical computed tomography (CT) imaging. Previous work has shown that multi-atlas methods can be used to segment and extract volumetric measurements from the optic nerve, which are associated with visual disability and disease. In this work, we trained a weakly supervised convolutional neural network to learn optic nerve volumes directly, without segmentation. Furthermore, we explored the role of contextual electronic medical record (EMR) information, specifically ICD-9 codes, to improve optic nerve volume estimation. We constructed a merged network to combine data from imaging as well as EMR and demonstrated that context improved volume prediction, with a 15% increase in explained-variance ( R ²). Finally, we compared disease prediction models using volumes learned from multi-atlas, CNN, and contextual-CNN. We observed that the predicted optic nerve volume from merge-CNN had an AUC of 0.74 for classification of disease, as compared to an AUC of 0.54 using the multi-atlas metric. This is the first work to show that a contextually derived volume biomarker is more accurate than volume estimations through multi-atlas or weakly supervised image CNN. These results highlight the potential for image processing improvements by incorporating non-imaging data.

Snorkel: rapid training data creation with weak supervision

Labeling training data is increasingly the largest bottleneck in deploying machine learning systems. We present Snorkel, a first-of-its-kind system that enables users to train state-of-the-art models without hand labeling any training data. Instead, users write labeling functions that express arbitrary heuristics, which can have unknown accuracies and correlations. Snorkel denoises their outputs without access to ground truth by incorporating the first end-to-end implementation of our recently proposed machine learning paradigm, data programming. We present a flexible interface layer for writing labeling functions based on our experience over the past year collaborating with companies, agencies, and research laboratories. In a user study, subject matter experts build models 2.8 × faster and increase predictive performance an average 45.5 % versus seven hours of hand labeling. We study the modeling trade-offs in this new setting and propose an optimizer for automating trade-off decisions that gives up to 1.8 × speedup per pipeline execution. In two collaborations, with the US Department of Veterans Affairs and the US Food and Drug Administration, and on four open-source text and image data sets representative of other deployments, Snorkel provides 132 % average improvements to predictive performance over prior heuristic approaches and comes within an average 3.60 % of the predictive performance of large hand-curated training sets.

Weakly supervised convolutional LSTM approach for tool tracking in laparoscopic videos

PURPOSE

Real-time surgical tool tracking is a core component of the future intelligent operating room (OR), because it is highly instrumental to analyze and understand the surgical activities. Current methods for surgical tool tracking in videos need to be trained on data in which the spatial positions of the tools are manually annotated. Generating such training data is difficult and time-consuming. Instead, we propose to use solely binary presence annotations to train a tool tracker for laparoscopic videos.

METHODS

The proposed approach is composed of a CNN + Convolutional LSTM (ConvLSTM) neural network trained end to end, but weakly supervised on tool binary presence labels only. We use the ConvLSTM to model the temporal dependencies in the motion of the surgical tools and leverage its spatiotemporal ability to smooth the class peak activations in the localization heat maps (Lh-maps).

RESULTS

We build a baseline tracker on top of the CNN model and demonstrate that our approach based on the ConvLSTM outperforms the baseline in tool presence detection, spatial localization, and motion tracking by over [Formula: see text], [Formula: see text], and [Formula: see text], respectively.

CONCLUSIONS

In this paper, we demonstrate that binary presence labels are sufficient for training a deep learning tracking model using our proposed method. We also show that the ConvLSTM can leverage the spatiotemporal coherence of consecutive image frames across a surgical video to improve tool presence detection, spatial localization, and motion tracking.

A clinical text classification paradigm using weak supervision and deep representation

BACKGROUND

Automatic clinical text classification is a natural language processing (NLP) technology that unlocks information embedded in clinical narratives. Machine learning approaches have been shown to be effective for clinical text classification tasks. However, a successful machine learning model usually requires extensive human efforts to create labeled training data and conduct feature engineering. In this study, we propose a clinical text classification paradigm using weak supervision and deep representation to reduce these human efforts.

METHODS

We develop a rule-based NLP algorithm to automatically generate labels for the training data, and then use the pre-trained word embeddings as deep representation features for training machine learning models. Since machine learning is trained on labels generated by the automatic NLP algorithm, this training process is called weak supervision. We evaluat the paradigm effectiveness on two institutional case studies at Mayo Clinic: smoking status classification and proximal femur (hip) fracture classification, and one case study using a public dataset: the i2b2 2006 smoking status classification shared task. We test four widely used machine learning models, namely, Support Vector Machine (SVM), Random Forest (RF), Multilayer Perceptron Neural Networks (MLPNN), and Convolutional Neural Networks (CNN), using this paradigm. Precision, recall, and F1 score are used as metrics to evaluate performance.

RESULTS

CNN achieves the best performance in both institutional tasks (F1 score: 0.92 for Mayo Clinic smoking status classification and 0.97 for fracture classification). We show that word embeddings significantly outperform tf-idf and topic modeling features in the paradigm, and that CNN captures additional patterns from the weak supervision compared to the rule-based NLP algorithms. We also observe two drawbacks of the proposed paradigm that CNN is more sensitive to the size of training data, and that the proposed paradigm might not be effective for complex multiclass classification tasks.

CONCLUSION

The proposed clinical text classification paradigm could reduce human efforts of labeled training data creation and feature engineering for applying machine learning to clinical text classification by leveraging weak supervision and deep representation. The experimental experiments have validated the effectiveness of paradigm by two institutional and one shared clinical text classification tasks.