Neural Models for Generating Natural Language Summaries from Temporal Personal Health Data

With an increased interest in the production of personal health technologies designed to track user data (e.g., nutrient  intake, step counts), there is now more opportunity than ever to surface meaningful behavioral insights to everyday users in the form of natural language. This knowledge can increase their behavioral awareness and allow them to take action to meet their health goals. It can also bridge the gap between the vast collection of personal health data and the summary generation required to describe an individual's behavioral tendencies. Previous work has focused on rule-based time-series data summarization methods designed to generate natural language summaries of interesting patterns found within temporal personal health data. We examine recurrent, convolutional, and Transformer-based encoder-decoder models to automatically generate natural language summaries from numeric temporal personal health data. We showcase the effectiveness of our models on real user health data logged in MyFitnessPal (Weber and Achananuparp 2016) and show that we can automatically generate high-quality natural language summaries. Our work serves as a first step towards the ambitious goal of automatically generating novel and meaningful temporal summaries from personal health data.

Learnable DoG convolutional filters for microcalcification detection

Difference of Gaussians (DoG) convolutional filters are one of the earliest image processing methods employed for detecting microcalcifications on mammogram images before machine and deep learning methods became widespread. DoG is a blob enhancement filter that consists in subtracting one Gaussian-smoothed version of an image from another less Gaussian-smoothed version of the same image. Smoothing with a Gaussian kernel suppresses high-frequency spatial information, thus DoG can be regarded as a band-pass filter. However, due to their small size and overimposed breast tissue, microcalcifications vary greatly in contrast-to-noise ratio and sharpness. This makes it difficult to find a single DoG configuration that enhances all microcalcifications. In this work, we propose a convolutional network, named DoG-MCNet, where the first layer automatically learns a bank of DoG filters parameterized by their associated standard deviations. We experimentally show that when employed for microcalcification detection, our DoG layer acts as a learnable bank of band-pass preprocessing filters and improves detection performance by 4.86% AUFROC over baseline MCNet and 1.53% AUFROC over state-of-the-art multicontext ensemble of CNNs.

HybridBranchNet: A novel structure for branch hybrid convolutional neural networks architecture

ConvNet deep neural networks are developed with a consistent structure. The availability of abundant resources helps these structures to be scaled and redesigned in different sizes so that they can be optimized for different applications. By increasing one or more dimensions of the network, such as depth, resolution and width, the number of trainable network parameters will increase and, as a result, the accuracy and performance It should be noted that the backtracking of the convolutional neural network will improve. However, but increasing the number of network parameters increases the complexity of the network, which is not desirable. Therefore, adjusting the structure of the network, increasing the speed, and reducing the number of network parameters along with ensuring accuracy optimization will be important. This study aims to examine a branch network structure systematically, which can lead to better performance. In this study, in order to increase the speed, to reduce the size of the convolutional network model, and to increase the accuracy optimization, a new scaling method, which optimally designs all dimensions of depth, width, and resolution, is proposed based on a branch neural network. A family of HybridBranchNet networks, which is more accurate and efficient than ConvNets, has been created along with this design. HybridBranchNet3 has a classification accuracy of 83.1%. The proposed model was compared with a family of EfficientNet convolutional networks. The comparison results revealed that the proposed network exceeded the mentioned models in terms of accuracy and speed by 1.03% and 39%, respectively. They also showed that the number of trainable parameters is 13% less than that of the EfficientNet network. The proposed method has an accuracy of 92.3% in the CIFAR-100 dataset and 98.8% in the Flowers-102 dataset. Although the architectures such as CoAtNet have slightly higher classification accuracy than the proposed method, they have a greater number of parameters that cannot be used in a conventional system.

RODAN: a fully convolutional architecture for basecalling nanopore RNA sequencing data

Background

Despite recent progress in basecalling of Oxford nanopore DNA sequencing data, its wide adoption is still being hampered by its relatively low accuracy compared to short read technologies. Furthermore, very little of the recent research was focused on basecalling of RNA data, which has different characteristics than its DNA counterpart.

Results

We fill this gap by benchmarking a fully convolutional deep learning basecalling architecture with improved performance compared to Oxford nanopore’s RNA basecallers.

Availability

The source code for our basecaller is available at: https://github.com/biodlab/RODAN.

Supplementary Information

The online version contains supplementary material available at 10.1186/s12859-022-04686-y.

Acral melanoma detection using dermoscopic images and convolutional neural networks

Acral melanoma (AM) is a rare and lethal type of skin cancer. It can be diagnosed by expert dermatologists, using dermoscopic imaging. It is challenging for dermatologists to diagnose melanoma because of the very minor differences between melanoma and non-melanoma cancers. Most of the research on skin cancer diagnosis is related to the binary classification of lesions into melanoma and non-melanoma. However, to date, limited research has been conducted on the classification of melanoma subtypes. The current study investigated the effectiveness of dermoscopy and deep learning in classifying melanoma subtypes, such as, AM. In this study, we present a novel deep learning model, developed to classify skin cancer. We utilized a dermoscopic image dataset from the Yonsei University Health System South Korea for the classification of skin lesions. Various image processing and data augmentation techniques have been applied to develop a robust automated system for AM detection. Our custom-built model is a seven-layered deep convolutional network that was trained from scratch. Additionally, transfer learning was utilized to compare the performance of our model, where AlexNet and ResNet-18 were modified, fine-tuned, and trained on the same dataset. We achieved improved results from our proposed model with an accuracy of more than 90 % for AM and benign nevus, respectively. Additionally, using the transfer learning approach, we achieved an average accuracy of nearly 97 %, which is comparable to that of state-of-the-art methods. From our analysis and results, we found that our model performed well and was able to effectively classify skin cancer. Our results show that the proposed system can be used by dermatologists in the clinical decision-making process for the early diagnosis of AM.

Fully-channel regional attention network for disease-location recognition with tongue images

OBJECTIVE

Using the deep learning model to realize tongue image-based disease location recognition and focus on solving two problems: 1. The ability of the general convolution network to model detailed regional tongue features is weak; 2. Ignoring the group relationship between convolution channels, which caused the high redundancy of the model.

METHODS

To enhance the convolutional neural networks. In this paper, a stochastic region pooling method is proposed to gain detailed regional features. Also, an inner-imaging channel relationship modeling method is proposed to model multi-region relations on all channels. Moreover, we combine it with the spatial attention mechanism.

RESULTS

The tongue image dataset with the clinical disease-location label is established. Abundant experiments are carried out on it. The experimental results show that the proposed method can effectively model the regional details of tongue image and improve the performance of disease location recognition.

CONCLUSION

In this paper, we construct the tongue image dataset with disease-location labels to mine the relationship between tongue images and disease locations. A novel fully-channel regional attention network is proposed to model the local detail tongue features and improve the modeling efficiency.

SIGNIFICANCE

The applications of deep learning in tongue image disease-location recognition and the proposed innovative models have guiding significance for other assistant diagnostic tasks. The proposed model provides an example of efficient modeling of detailed tongue features, which is of great guiding significance for other auxiliary diagnosis applications.

Automated whole breast segmentation for hand-held ultrasound with position information: Application to breast density estimation

BACKGROUND AND OBJECTIVE

Women with higher breast densities have a relatively higher risk to be diagnosed with breast cancer. Hand-held ultrasound (HHUS) can provide precise screening results and detect masses in dense breasts. However, its lack of position information and automatic extraction of breast area hinder the implementation of density estimation. To facilitate reliable breast density evaluation, this study proposed an upgraded version of our whole-breast ultrasound (WBUS) system, which not only can provide precise position information, but also can extract precise breast area automatically based on deep learning method.

METHODS

WBUS images with probe position information were collected from 117 women. For each case, an automatic breast region segmentation by DeepResUnet was conducted, then fibroglandular tissues were extracted from breast region using fuzzy c-mean (FCM) classifier. Finally, the percentage of breast density and breast area of the DeepResUnet predicted region and the breast region of the ground truth were calculated and compared.

RESULTS

The average and standard deviation of each breast case for DeepResUnet predicted breast region of 10-fold in Accuracy (ACC) was 0.963±0.054. Sensitivity (SENS) was 0.928±0.11. Specificity (SPEC) was 0.967±0.054. Dice coefficient (Dice) was 0.916±0.98. Region intersection over union (IoU) was 0.856±0.134. Significant and very high correlations of breast density, fibroglandular tissue area and breast area (R = 0.843, R= 0.822 and R = 0.984, all p values < 0.001) were found between the ground truth and the result of the proposed method for ultrasound images.

CONCLUSIONS

Breast density, fibroglandular tissue, and breast volume evaluated based on the proposed method and WBUS system have significant correlations with ground truth, indicating that the proposed method and WBUS system has the potential to be an alternative modality for breast screening and density estimation in clinical use.

Keras R-CNN: library for cell detection in biological images using deep neural networks

BACKGROUND

A common yet still manual task in basic biology research, high-throughput drug screening and digital pathology is identifying the number, location, and type of individual cells in images. Object detection methods can be useful for identifying individual cells as well as their phenotype in one step. State-of-the-art deep learning for object detection is poised to improve the accuracy and efficiency of biological image analysis.

RESULTS

We created Keras R-CNN to bring leading computational research to the everyday practice of bioimage analysts. Keras R-CNN implements deep learning object detection techniques using Keras and Tensorflow ( https://github.com/broadinstitute/keras-rcnn ). We demonstrate the command line tool's simplified Application Programming Interface on two important biological problems, nucleus detection and malaria stage classification, and show its potential for identifying and classifying a large number of cells. For malaria stage classification, we compare results with expert human annotators and find comparable performance.

CONCLUSIONS

Keras R-CNN is a Python package that performs automated cell identification for both brightfield and fluorescence images and can process large image sets. Both the package and image datasets are freely available on GitHub and the Broad Bioimage Benchmark Collection.

A deep learning-based hybrid approach for the solution of multiphysics problems in electrosurgery

Multiphysics modeling of evolving topology in the electrosurgical dissection of soft hydrated tissues is a challenging problem, requiring heavy computational resources. In this paper, we propose a hybrid approach that leverages the regressive capabilities of deep convolutional neural networks (CNN) with the precision of conventional solvers to accelerate Multiphysics computations. The electro-thermal problem is solved using a finite element method (FEM) with a Krylov subspace-based iterative solver and a deflation-based block preconditioner. The mechanical deformation induced by evaporation of intra- and extracellular water is obtained using a CNN model. The CNN is trained using a supervised learning framework that maps the nonlinear relationship between the micropore pressure and deformation field for a given tissue topology. The simulation results show that the hybrid approach is significantly more computationally efficient than a FEM-based solution approach using a block-preconditioned Krylov subspace solver and a parametric solution approach using a proper generalized decomposition (PGD) based reduced order model. The accuracy of the hybrid approach is comparable to the ground truth obtained using a standard multiphysics solver. The hydrid approach overcomes the limitations of end-to-end learning including the need for massive datasets for training the network.

Measurement-oriented deep-learning workflow for improved segmentation of myelin and axons in high-resolution images of human cerebral white matter

BACKGROUND

Standard segmentation of high-contrast electron micrographs (EM) identifies myelin accurately but does not translate easily into measurements of individual axons and their myelin, even in cross-sections of parallel fibers. We describe automated segmentation and measurement of each myelinated axon and its sheath in EMs of arbitrarily oriented human white matter from autopsies.

NEW METHODS

Preliminary segmentation of myelin, axons and background by machine learning, using selected filters, precedes automated correction of systematic errors. Final segmentation is done by a deep neural network (DNN). Automated measurement of each putative fiber rejects measures encountering pre-defined artifacts and excludes fibers failing to satisfy pre-defined conditions.

RESULTS

Improved segmentation of three sets of 30 annotated images each (two sets from human prefrontal white matter and one from human optic nerve) is achieved with a DNN trained only with a subset of the first set from prefrontal white matter. Total number of myelinated axons identified by the DNN differed from expert segmentation by 0.2%, 2.9%, and -5.1%, respectively. G-ratios differed by 2.96%, 0.74% and 2.83%. Intraclass correlation coefficients between DNN and annotated segmentation were mostly >0.9, indicating nearly interchangeable performance.

COMPARISON WITH EXISTING METHOD(S)

Measurement-oriented studies of arbitrarily oriented fibers from central white matter are rare. Published methods are typically applied to cross-sections of fascicles and measure aggregated areas of myelin sheaths and axons, allowing estimation only of average g-ratio.

CONCLUSIONS

Automated segmentation and measurement of axons and myelin is complex. We report a feasible approach that has so far proven comparable to manual segmentation.

Flowers, leaves or both? How to obtain suitable images for automated plant identification

BACKGROUND

Deep learning algorithms for automated plant identification need large quantities of precisely labelled images in order to produce reliable classification results. Here, we explore what kind of perspectives and their combinations contain more characteristic information and therefore allow for higher identification accuracy.

RESULTS

We developed an image-capturing scheme to create observations of flowering plants. Each observation comprises five in-situ images of the same individual from predefined perspectives (entire plant, flower frontal- and lateral view, leaf top- and back side view). We collected a completely balanced dataset comprising 100 observations for each of 101 species with an emphasis on groups of conspecific and visually similar species including twelve Poaceae species. We used this dataset to train convolutional neural networks and determine the prediction accuracy for each single perspective and their combinations via score level fusion. Top-1 accuracies ranged between 77% (entire plant) and 97% (fusion of all perspectives) when averaged across species. Flower frontal view achieved the highest accuracy (88%). Fusing flower frontal, flower lateral and leaf top views yields the most reasonable compromise with respect to acquisition effort and accuracy (96%). The perspective achieving the highest accuracy was species dependent.

CONCLUSIONS

We argue that image databases of herbaceous plants would benefit from multi organ observations, comprising at least the front and lateral perspective of flowers and the leaf top view.

Survey on deep learning for radiotherapy

More than 50% of cancer patients are treated with radiotherapy, either exclusively or in combination with other methods. The planning and delivery of radiotherapy treatment is a complex process, but can now be greatly facilitated by artificial intelligence technology. Deep learning is the fastest-growing field in artificial intelligence and has been successfully used in recent years in many domains, including medicine. In this article, we first explain the concept of deep learning, addressing it in the broader context of machine learning. The most common network architectures are presented, with a more specific focus on convolutional neural networks. We then present a review of the published works on deep learning methods that can be applied to radiotherapy, which are classified into seven categories related to the patient workflow, and can provide some insights of potential future applications. We have attempted to make this paper accessible to both radiotherapy and deep learning communities, and hope that it will inspire new collaborations between these two communities to develop dedicated radiotherapy applications.

Deep MRI brain extraction: A 3D convolutional neural network for skull stripping

Brain extraction from magnetic resonance imaging (MRI) is crucial for many neuroimaging workflows. Current methods demonstrate good results on non-enhanced T1-weighted images, but struggle when confronted with other modalities and pathologically altered tissue. In this paper we present a 3D convolutional deep learning architecture to address these shortcomings. In contrast to existing methods, we are not limited to non-enhanced T1w images. When trained appropriately, our approach handles an arbitrary number of modalities including contrast-enhanced scans. Its applicability to MRI data, comprising four channels: non-enhanced and contrast-enhanced T1w, T2w and FLAIR contrasts, is demonstrated on a challenging clinical data set containing brain tumors (N=53), where our approach significantly outperforms six commonly used tools with a mean Dice score of 95.19. Further, the proposed method at least matches state-of-the-art performance as demonstrated on three publicly available data sets: IBSR, LPBA40 and OASIS, totaling N=135 volumes. For the IBSR (96.32) and LPBA40 (96.96) data set the convolutional neuronal network (CNN) obtains the highest average Dice scores, albeit not being significantly different from the second best performing method. For the OASIS data the second best Dice (95.02) results are achieved, with no statistical difference in comparison to the best performing tool. For all data sets the highest average specificity measures are evaluated, whereas the sensitivity displays about average results. Adjusting the cut-off threshold for generating the binary masks from the CNN's probability output can be used to increase the sensitivity of the method. Of course, this comes at the cost of a decreased specificity and has to be decided application specific. Using an optimized GPU implementation predictions can be achieved in less than one minute. The proposed method may prove useful for large-scale studies and clinical trials.