ImageGCN: Multi-Relational Image Graph Convolutional Networks for Disease Identification With Chest X-Rays

Development and validation of a graph convolutional network (GCN)-based automatic superimposition method for maxillary digital dental models (MDMs)

OBJECTIVES

To validate the accuracy and reliability of a graph convolutional network (GCN)-based superimposition method of a maxillary digital dental model (MDM) by comparing it with manual superimposition and quantifying the clinical error from this method.

MATERIALS AND METHODS

Based on a GCN, learning the features from 100 three-dimensional digital occlusal models under supervision of the palatal stable structure labels that were manually annotated by senior specialists, the palatal stable structure was automatically segmented. The average Hausdorff distance was calculated to assess the difference between automatic and manual segmentations. Tooth position and angulation, including rotation, tip, and torque, of bilateral upper first molars and central incisors were obtained to measure the clinical error of automatic superimposition. Reliability was calculated by intraclass correlation coefficient (ICC).

RESULTS

The average Hausdorff distance was 0.36 mm between automatic and manual segmentations of the palatal stable region and was larger than the intraexaminer and interexaminer deviations. The tooth position deviation was <0.32 mm, and the tooth angulation difference was <0.26° for tip and torque, and 0.46-0.61° in rotation. ICCs, used for assessment of reliability, ranged from 0.82 to 0.99 in all variables.

CONCLUSIONS

The GCN-based MDM superimposition is an efficient method for the assessment of tooth movement in adults. The clinical error in tooth position and angulation induced by the method was clinically acceptable. Reliability was as high as manual segmentation.

DiffMIC-v2: Medical Image Classification via Improved Diffusion Network

Recently, Denoising Diffusion Models have achieved outstanding success in generative image modeling and attracted significant attention in the computer vision community. Although a substantial amount of diffusion-based research has focused on generative tasks, few studies apply diffusion models to medical diagnosis. In this paper, we propose a diffusion-based network (named DiffMIC-v2) to address general medical image classification by eliminating unexpected noise and perturbations in image representations. To achieve this goal, we first devise an improved dual-conditional guidance strategy that conditions each diffusion step with multiple granularities to enhance step-wise regional attention. Furthermore, we design a novel Heterologous diffusion process that achieves efficient visual representation learning in the latent space. We evaluate the effectiveness of our DiffMIC-v2 on four medical classification tasks with different image modalities, including thoracic diseases classification on chest X-ray, placental maturity grading on ultrasound images, skin lesion classification using dermatoscopic images, and diabetic retinopathy grading using fundus images. Experimental results demonstrate that our DiffMIC-v2 outperforms state-of-the-art methods by a significant margin, which indicates the universality and effectiveness of the proposed model on multi-class and multi-label classification tasks. DiffMIC-v2 can use fewer iterations than our previous DiffMIC to obtain accurate estimations, and also achieves greater runtime efficiency with superior results. The code will be publicly available at https://github.com/scott-yjyang/DiffMICv2.

Advancement in medical report generation: current practices, challenges, and future directions

The correct analysis of medical images requires the medical knowledge and expertise of radiologists to understand, clarify, and explain complex patterns and diagnose diseases. After analyzing, radiologists write detailed and well-structured reports that contribute to the precise and timely diagnosis of patients. However, manually writing reports is often expensive and time-consuming, and it is difficult for radiologists to analyze medical images, particularly images with multiple views and perceptions. It is challenging to accurately diagnose diseases, and many methods are proposed to help radiologists, both traditional and deep learning-based. Automatic report generation is widely used to tackle this issue as it streamlines the process and lessens the burden of manual labeling of images. This paper introduces a systematic literature review with a focus on analyses and evaluating existing research on medical report generation. This SLR follows a proper protocol for the planning, reviewing, and reporting of the results. This review recognizes that the most commonly used deep learning models are encoder-decoder frameworks (45 articles), which provide an accuracy of around 92-95%. Transformers-based models (20 articles) are the second most established method and achieve an accuracy of around 91%. The remaining articles explored in this SLR are attention mechanisms (10), RNN-LSTM (10), Large language models (LLM-10), and graph-based methods (4) with promising results. However, these methods also face certain limitations such as overfitting, risk of bias, and high data dependency that impact their performance. The review not only highlights the strengths and challenges of these methods but also suggests ways to handle them in the future to increase the accuracy and timely generation of medical reports. The goal of this review is to direct radiologists toward methods that lessen their workload and provide precise medical diagnoses.

Deep learning paradigms in lung cancer diagnosis: A methodological review, open challenges, and future directions

Lung cancer is the leading cause of global cancer-related deaths, which emphasizes the critical importance of early diagnosis in enhancing patient outcomes. Deep learning has demonstrated significant promise in lung cancer diagnosis, excelling in nodule detection, classification, and prognosis prediction. This methodological review comprehensively explores deep learning models' application in lung cancer diagnosis, uncovering their integration across various imaging modalities. Deep learning consistently achieves state-of-the-art performance, occasionally surpassing human expert accuracy. Notably, deep neural networks excel in detecting lung nodules, distinguishing between benign and malignant nodules, and predicting patient prognosis. They have also led to the development of computer-aided diagnosis systems, enhancing diagnostic accuracy for radiologists. This review follows the specified criteria for article selection outlined by PRISMA framework. Despite challenges such as data quality and interpretability limitations, this review emphasizes the potential of deep learning to significantly improve the precision and efficiency of lung cancer diagnosis, facilitating continued research efforts to overcome these obstacles and fully harness neural network's transformative impact in this field.

MVASA-HGN: multi-view adaptive semantic-aware heterogeneous graph network for KRAS mutation status prediction

Background

In the treatment of advanced non-small cell lung cancer (NSCLC), the mutation status of the Kirsten rat sarcoma virus oncogene homolog (KRAS) gene has been shown to be a key factor affecting the efficacy of immune checkpoint inhibitors (ICIs), which is an important guideline for physicians to develop personalized treatment strategies. However, existing mutation prediction studies have primarily focused on the feature representation of individual patient medical data, ignoring the complex semantic relationships among patients in diverse clinical features. This study aimed to accurately identify KRAS gene status, which will not only assist physicians in accurately screening the patient population most likely to benefit from immunotherapy, but also reduce patient burden by avoiding unnecessary treatment attempts.

Methods

A multi-view adaptive semantics-aware heterogeneous graph framework (MVASA-HGN) based on multimodal medical data was developed to accurately predict KRAS mutation status in NSCLC patients. The framework first parses the relational semantics through clinical feature clustering and constructs a heterogeneous graph by combining computed tomography (CT) image and clinical features. In the second step, the heterogeneous graph is split into relational subgraphs under multiple views, and the node representations are constructed and updated gradually through a two-stage strategy of single-view graph representation learning and multi-view heterogeneous information fusion. In the single-view phase, we enhance the node self-embedding and construct the adjacency embedding of neighbors with the same type of relationship to ensure that the relational subgraph under each semantic preserves the complete local structure. Two attention mechanisms are introduced in the multi-view fusion phase to capture the enriched semantics preserved in nodes and heterogeneous relations, respectively. Finally, a comprehensive node representation is obtained through adaptive aggregation of different view neighborhood information and enhanced node embedding without predefined meta-paths.

Results

The classification results were evaluated on cooperative hospitals and The Cancer Imaging Archive (TCIA) datasets, and ablation experiments and comparison experiments were performed on the components of the framework, while exploring the framework's rationality and interpretability. Accuracy reached 85.29% and specificity reached 89.67% on the test set, indicating that our framework has significant advantages in deeply modeling complex heterogeneous semantics in local structures and fully exploiting and utilizing the rich semantic information preserved in heterogeneous relationships. The source code of MVASA-HGN is available at https://github.com/Yangwanter37/MVASA-HGN.

Conclusions

Our proposed MVASA-HGN framework provides a new perspective for multimodal information fusion and creates a new avenue to explore the potential link between images and genes, and the framework provides a non-invasive and cost-effective solution for identifying KRAS mutation status, which has a broad application prospect.

A New Benchmark: Clinical Uncertainty and Severity Aware Labeled Chest X-Ray Images With Multi-Relationship Graph Learning

Chest radiography, commonly known as CXR, is frequently utilized in clinical settings to detect cardiopulmonary conditions. However, even seasoned radiologists might offer different evaluations regarding the seriousness and uncertainty associated with observed abnormalities. Previous research has attempted to utilize clinical notes to extract abnormal labels for training deep-learning models in CXR image diagnosis. However, these methods often neglected the varying degrees of severity and uncertainty linked to different labels. In our study, we initially assembled a comprehensive new dataset of CXR images based on clinical textual data, which incorporated radiologists' assessments of uncertainty and severity. Using this dataset, we introduced a multi-relationship graph learning framework that leverages spatial and semantic relationships while addressing expert uncertainty through a dedicated loss function. Our research showcases a notable enhancement in CXR image diagnosis and the interpretability of the diagnostic model, surpassing existing state-of-the-art methodologies. The dataset address of disease severity and uncertainty we extracted is: https://physionet.org/content/cad-chest/1.0/.

Leveraging anatomical constraints with uncertainty for pneumothorax segmentation

Background

Pneumothorax is a medical emergency caused by the abnormal accumulation of air in the pleural space-the potential space between the lungs and chest wall. On 2D chest radiographs, pneumothorax occurs within the thoracic cavity and outside of the mediastinum, and we refer to this area as "lung + space." While deep learning (DL) has increasingly been utilized to segment pneumothorax lesions in chest radiographs, many existing DL models employ an end-to-end approach. These models directly map chest radiographs to clinician-annotated lesion areas, often neglecting the vital domain knowledge that pneumothorax is inherently location-sensitive.

Methods

We propose a novel approach that incorporates the lung + space as a constraint during DL model training for pneumothorax segmentation on 2D chest radiographs. To circumvent the need for additional annotations and to prevent potential label leakage on the target task, our method utilizes external datasets and an auxiliary task of lung segmentation. This approach generates a specific constraint of lung + space for each chest radiograph. Furthermore, we have incorporated a discriminator to eliminate unreliable constraints caused by the domain shift between the auxiliary and target datasets.

Results

Our results demonstrated considerable improvements, with average performance gains of 4.6%, 3.6%, and 3.3% regarding intersection over union, dice similarity coefficient, and Hausdorff distance. These results were consistent across six baseline models built on three architectures (U-Net, LinkNet, or PSPNet) and two backbones (VGG-11 or MobileOne-S0). We further conducted an ablation study to evaluate the contribution of each component in the proposed method and undertook several robustness studies on hyper-parameter selection to validate the stability of our method.

Conclusions

The integration of domain knowledge in DL models for medical applications has often been underemphasized. Our research underscores the significance of incorporating medical domain knowledge about the location-specific nature of pneumothorax to enhance DL-based lesion segmentation and further bolster clinicians' trust in DL tools. Beyond pneumothorax, our approach is promising for other thoracic conditions that possess location-relevant characteristics.

Exponential Pixelating Integral transform with dual fractal features for enhanced chest X-ray abnormality detection

The heightened prevalence of respiratory disorders, particularly exacerbated by a significant upswing in fatalities due to the novel coronavirus, underscores the critical need for early detection and timely intervention. This imperative is paramount, possessing the potential to profoundly impact and safeguard numerous lives. Medically, chest radiography stands out as an essential and economically viable medical imaging approach for diagnosing and assessing the severity of diverse Respiratory Disorders. However, their detection in Chest X-Rays is a cumbersome task even for well-trained radiologists owing to low contrast issues, overlapping of the tissue structures, subjective variability, and the presence of noise. To address these issues, a novel analytical model termed Exponential Pixelating Integral is introduced for the automatic detection of infections in Chest X-Rays in this work. Initially, the presented Exponential Pixelating Integral enhances the pixel intensities to overcome the low-contrast issues that are then polar-transformed followed by their representation using the locally invariant Mandelbrot and Julia fractal geometries for effective distinction of structural features. The collated features labeled Exponential Pixelating Integral with dually characterized fractal features are then classified by the non-parametric multivariate adaptive regression splines to establish an ensemble model between each pair of classes for effective diagnosis of diverse diseases. Rigorous analysis of the proposed classification framework on large medical benchmarked datasets showcases its superiority over its peers by registering a higher classification accuracy and F1 scores ranging from 98.46 to 99.45 % and 96.53-98.10 % respectively, making it a precise and interpretable automated system for diagnosing respiratory disorders.

Diagnosing Necrotizing Enterocolitis via Fine-Grained Visual Classification

Necrotizing Enterocolitis (NEC) is a devastating condition affecting prematurely born neonates. Reviewing Abdominal X-rays (AXRs) is a key step in NEC diagnosis, staging and treatment decision-making, but poses significant challenges due to the subtle, difficult-to-identify radiological signs of the disease. In this paper, we propose AIDNEC - AI Diagnosis of NECrotizing enterocolitis, a deep learning method to automatically detect and stratify the severity (surgical or medical) of NEC from no pathology in AXRs. The model is trainable end-to-end and integrates a Detection Transformer and Graph Convolution modules for localizing discriminative areas in AXRs, used to formulate subtle local embeddings. These are then combined with global image features to perform Fine-Grained Visual Classification (FGVC). We evaluate AIDNEC on our GOSH NEC dataset of 1153 images from 334 patients, achieving 79.7% accuracy in classifying NEC against No Pathology. AIDNEC outperforms the backbone by 2.6%, FGVC models by 2.5% and CheXNet by 4.2%, with statistically significant (two-tailed p 0.05) improvements, while providing meaningful discriminative regions to support the classification decision. Additional validation in the publicly available Chest X-ray14 dataset yields comparable performance to state-of-the-art methods, illustrating AIDNEC's robustness in a different X-ray classification task.

Development of message passing-based graph convolutional networks for classifying cancer pathology reports

BACKGROUND

Applying graph convolutional networks (GCN) to the classification of free-form natural language texts leveraged by graph-of-words features (TextGCN) was studied and confirmed to be an effective means of describing complex natural language texts. However, the text classification models based on the TextGCN possess weaknesses in terms of memory consumption and model dissemination and distribution. In this paper, we present a fast message passing network (FastMPN), implementing a GCN with message passing architecture that provides versatility and flexibility by allowing trainable node embedding and edge weights, helping the GCN model find the better solution. We applied the FastMPN model to the task of clinical information extraction from cancer pathology reports, extracting the following six properties: main site, subsite, laterality, histology, behavior, and grade.

RESULTS

We evaluated the clinical task performance of the FastMPN models in terms of micro- and macro-averaged F1 scores. A comparison was performed with the multi-task convolutional neural network (MT-CNN) model. Results show that the FastMPN model is equivalent to or better than the MT-CNN.

CONCLUSIONS

Our implementation revealed that our FastMPN model, which is based on the PyTorch platform, can train a large corpus (667,290 training samples) with 202,373 unique words in less than 3 minutes per epoch using one NVIDIA V100 hardware accelerator. Our experiments demonstrated that using this implementation, the clinical task performance scores of information extraction related to tumors from cancer pathology reports were highly competitive.

Northwestern University resource and education development initiatives to advance collaborative artificial intelligence across the learning health system

Introduction

The rapid development of artificial intelligence (AI) in healthcare has exposed the unmet need for growing a multidisciplinary workforce that can collaborate effectively in the learning health systems. Maximizing the synergy among multiple teams is critical for Collaborative AI in Healthcare.

Methods

We have developed a series of data, tools, and educational resources for cultivating the next generation of multidisciplinary workforce for Collaborative AI in Healthcare. We built bulk-natural language processing pipelines to extract structured information from clinical notes and stored them in common data models. We developed multimodal AI/machine learning (ML) tools and tutorials to enrich the toolbox of the multidisciplinary workforce to analyze multimodal healthcare data. We have created a fertile ground to cross-pollinate clinicians and AI scientists and train the next generation of AI health workforce to collaborate effectively.

Results

Our work has democratized access to unstructured health information, AI/ML tools and resources for healthcare, and collaborative education resources. From 2017 to 2022, this has enabled studies in multiple clinical specialties resulting in 68 peer-reviewed publications. In 2022, our cross-discipline efforts converged and institutionalized into the Center for Collaborative AI in Healthcare.

Conclusions

Our Collaborative AI in Healthcare initiatives has created valuable educational and practical resources. They have enabled more clinicians, scientists, and hospital administrators to successfully apply AI methods in their daily research and practice, develop closer collaborations, and advanced the institution-level learning health system.

Chest x-ray diagnosis via spatial-channel high-order attention representation learning

Objective. Chest x-ray image representation and learning is an important problem in computer-aided diagnostic area. Existing methods usually adopt CNN or Transformers for feature representation learning and focus on learning effective representations for chest x-ray images. Although good performance can be obtained, however, these works are still limited mainly due to the ignorance of mining the correlations of channels and pay little attention on the local context-aware feature representation of chest x-ray image.Approach. To address these problems, in this paper, we propose a novel spatial-channel high-order attention model (SCHA) for chest x-ray image representation and diagnosis. The proposed network architecture mainly contains three modules, i.e. CEBN, SHAM and CHAM. To be specific, firstly, we introduce a context-enhanced backbone network by employing multi-head self-attention to extract initial features for the input chest x-ray images. Then, we develop a novel SCHA which contains both spatial and channel high-order attention learning branches. For the spatial branch, we develop a novel local biased self-attention mechanism which can capture both local and long-range global dependences of positions to learn rich context-aware representation. For the channel branch, we employ Brownian Distance Covariance to encode the correlation information of channels and regard it as the image representation. Finally, the two learning branches are integrated together for the final multi-label diagnosis classification and prediction.Main results. Experiments on the commonly used datasets including ChestX-ray14 and CheXpert demonstrate that our proposed SCHA approach can obtain better performance when comparing many related approaches.Significance. This study obtains a more discriminative method for chest x-ray classification and provides a technique for computer-aided diagnosis.

Immunotherapy Efficacy Prediction for Non-Small Cell Lung Cancer Using Multi-View Adaptive Weighted Graph Convolutional Networks

Immunotherapy is an effective way to treat non-small cell lung cancer (NSCLC). The efficacy of immunotherapy differs from person to person and may cause side effects, making it important to predict the efficacy of immunotherapy before surgery. Radiomics based on machine learning has been successfully used to predict the efficacy of NSCLC immunotherapy. However, most studies only considered the radiomic features of the individual patient, ignoring the inter-patient correlations. Besides, they usually concatenated different features as the input of a single-view model, failing to consider the complex correlation among features of multiple types. To this end, we propose a multi-view adaptive weighted graph convolutional network (MVAW-GCN) for the prediction of NSCLC immunotherapy efficacy. Specifically, we group the radiomic features into several views according to the type of the fitered images they extracted from. We construct a graph in each view based on the radiomic features and phenotypic information. An attention mechanism is introduced to automatically assign weights to each view. Considering the view-shared and view-specific knowledge of radiomic features, we propose separable graph convolution that decomposes the output of the last convolution layer into two components, i.e., the view-shared and view-specific outputs. We maximize the consistency and enhance the diversity among different views in the learning procedure. The proposed MVAW-GCN is evaluated on 107 NSCLC patients, including 52 patients with valid efficacy and 55 patients with invalid efficacy. Our method achieved an accuracy of 77.27% and an area under the curve (AUC) of 0.7780, indicating its effectiveness in NSCLC immunotherapy efficacy prediction.

Deep reinforcement learning framework for thoracic diseases classification via prior knowledge guidance

The chest X-ray is commonly employed in the diagnosis of thoracic diseases. Over the years, numerous approaches have been proposed to address the issue of automatic diagnosis based on chest X-rays. However, the limited availability of labeled data for related diseases remains a significant challenge in achieving accurate diagnoses. This paper focuses on the diagnostic problem of thorax diseases and presents a novel deep reinforcement learning framework. This framework incorporates prior knowledge to guide the learning process of diagnostic agents, and the model parameters can be continually updated as more data becomes available, mimicking a person's learning process. Specifically, our approach offers two key contributions: (1) prior knowledge can be acquired from pre-trained models using old data or similar data from other domains, effectively reducing the dependence on target domain data; and (2) the reinforcement learning framework enables the diagnostic agent to be as exploratory as a human, leading to improved diagnostic accuracy through continuous exploration. Moreover, this method effectively addresses the challenge of learning models with limited data, enhancing the model's generalization capability. We evaluate the performance of our approach using the well-known NIH ChestX-ray 14 and CheXpert datasets, and achieve competitive results. More importantly, in clinical application, we make considerable progress. The source code for our approach can be accessed at the following URL: https://github.com/NeaseZ/MARL.

Multi-Label Local to Global Learning: A Novel Learning Paradigm for Chest X-Ray Abnormality Classification

Deep neural network (DNN) approaches have shown remarkable progress in automatic Chest X-rays classification. However, existing methods use a training scheme that simultaneously trains all abnormalities without considering their learning priority. Inspired by the clinical practice of radiologists progressively recognizing more abnormalities and the observation that existing curriculum learning (CL) methods based on image difficulty may not be suitable for disease diagnosis, we propose a novel CL paradigm, named multi-label local to global (ML-LGL). This approach iteratively trains DNN models on gradually increasing abnormalities within the dataset, i,e, from fewer abnormalities (local) to more ones (global). At each iteration, we first build the local category by adding high-priority abnormalities for training, and the abnormality's priority is determined by our three proposed clinical knowledge-leveraged selection functions. Then, images containing abnormalities in the local category are gathered to form a new training set. The model is lastly trained on this set using a dynamic loss. Additionally, we demonstrate the superiority of ML-LGL from the perspective of the model's initial stability during training. Experimental results on three open-source datasets, PLCO, ChestX-ray14 and CheXpert show that our proposed learning paradigm outperforms baselines and achieves comparable results to state-of-the-art methods. The improved performance promises potential applications in multi-label Chest X-ray classification.

Predicting Recurrence in Pancreatic Ductal Adenocarcinoma after Radical Surgery Using an AX-Unet Pancreas Segmentation Model and Dynamic Nomogram

This study aims to investigate the reliability of radiomic features extracted from contrast-enhanced computer tomography (CT) by AX-Unet, a pancreas segmentation model, to analyse the recurrence of pancreatic ductal adenocarcinoma (PDAC) after radical surgery. In this study, we trained an AX-Unet model to extract the radiomic features from preoperative contrast-enhanced CT images on a training set of 205 PDAC patients. Then we evaluated the segmentation ability of AX-Unet and the relationship between radiomic features and clinical characteristics on an independent testing set of 64 patients with clear prognoses. The lasso regression analysis was used to screen for variables of interest affecting patients' post-operative recurrence, and the Cox proportional risk model regression analysis was used to screen for risk factors and create a nomogram prediction model. The proposed model achieved an accuracy of 85.9% for pancreas segmentation, meeting the requirements of most clinical applications. Radiomic features were found to be significantly correlated with clinical characteristics such as lymph node metastasis, resectability status, and abnormally elevated serum carbohydrate antigen 19-9 (CA 19-9) levels. Specifically, variance and entropy were associated with the recurrence rate (p < 0.05). The AUC for the nomogram predicting whether the patient recurred after surgery was 0.92 (95% CI: 0.78-0.99) and the C index was 0.62 (95% CI: 0.48-0.78). The AX-Unet pancreas segmentation model shows promise in analysing recurrence risk factors after radical surgery for PDAC. Additionally, our findings suggest that a dynamic nomogram model based on AX-Unet can provide pancreatic oncologists with more accurate prognostic assessments for their patients.

Graph representation learning in biomedicine and healthcare

Networks-or graphs-are universal descriptors of systems of interacting elements. In biomedicine and healthcare, they can represent, for example, molecular interactions, signalling pathways, disease co-morbidities or healthcare systems. In this Perspective, we posit that representation learning can realize principles of network medicine, discuss successes and current limitations of the use of representation learning on graphs in biomedicine and healthcare, and outline algorithmic strategies that leverage the topology of graphs to embed them into compact vectorial spaces. We argue that graph representation learning will keep pushing forward machine learning for biomedicine and healthcare applications, including the identification of genetic variants underlying complex traits, the disentanglement of single-cell behaviours and their effects on health, the assistance of patients in diagnosis and treatment, and the development of safe and effective medicines.

Machine Learning for Lung Cancer Diagnosis, Treatment, and Prognosis

The recent development of imaging and sequencing technologies enables systematic advances in the clinical study of lung cancer. Meanwhile, the human mind is limited in effectively handling and fully utilizing the accumulation of such enormous amounts of data. Machine learning-based approaches play a critical role in integrating and analyzing these large and complex datasets, which have extensively characterized lung cancer through the use of different perspectives from these accrued data. In this review, we provide an overview of machine learning-based approaches that strengthen the varying aspects of lung cancer diagnosis and therapy, including early detection, auxiliary diagnosis, prognosis prediction, and immunotherapy practice. Moreover, we highlight the challenges and opportunities for future applications of machine learning in lung cancer.

AX-Unet: A Deep Learning Framework for Image Segmentation to Assist Pancreatic Tumor Diagnosis

Image segmentation plays an essential role in medical imaging analysis such as tumor boundary extraction. Recently, deep learning techniques have dramatically improved performance for image segmentation. However, an important factor preventing deep neural networks from going further is the information loss during the information propagation process. In this article, we present AX-Unet, a deep learning framework incorporating a modified atrous spatial pyramid pooling module to learn the location information and to extract multi-level contextual information to reduce information loss during downsampling. We also introduce a special group convolution operation on the feature map at each level to achieve information decoupling between channels. In addition, we propose an explicit boundary-aware loss function to tackle the blurry boundary problem. We evaluate our model on two public Pancreas-CT datasets, NIH Pancreas-CT dataset, and the pancreas part in medical segmentation decathlon (MSD) medical dataset. The experimental results validate that our model can outperform the state-of-the-art methods in pancreas CT image segmentation. By comparing the extracted feature output of our model, we find that the pancreatic region of normal people and patients with pancreatic tumors shows significant differences. This could provide a promising and reliable way to assist physicians for the screening of pancreatic tumors.

MedGCN: Medication recommendation and lab test imputation via graph convolutional networks

Laboratory testing and medication prescription are two of the most important routines in daily clinical practice. Developing an artificial intelligence system that can automatically make lab test imputations and medication recommendations can save costs on potentially redundant lab tests and inform physicians of a more effective prescription. We present an intelligent medical system (named MedGCN) that can automatically recommend the patients' medications based on their incomplete lab tests, and can even accurately estimate the lab values that have not been taken. In our system, we integrate the complex relations between multiple types of medical entities with their inherent features in a heterogeneous graph. Then we model the graph to learn a distributed representation for each entity in the graph based on graph convolutional networks (GCN). By the propagation of graph convolutional networks, the entity representations can incorporate multiple types of medical information that can benefit multiple medical tasks. Moreover, we introduce a cross regularization strategy to reduce overfitting for multi-task training by the interaction between the multiple tasks. In this study, we construct a graph to associate 4 types of medical entities, i.e., patients, encounters, lab tests, and medications, and applied a graph neural network to learn node embeddings for medication recommendation and lab test imputation. we validate our MedGCN model on two real-world datasets: NMEDW and MIMIC-III. The experimental results on both datasets demonstrate that our model can outperform the state-of-the-art in both tasks. We believe that our innovative system can provide a promising and reliable way to assist physicians to make medication prescriptions and to save costs on potentially redundant lab tests.