DRCNNTLe: A deep recurrent convolutional neural network with transfer learning through pre-trained embeddings for automated ICD coding

Automatic ICD-10-CM coding via Lambda-Scaled attention based deep learning model

The International Classification of Diseases (ICD) serves as a global healthcare administration standard, with one of its editions being ICD-10-CM, an enhanced diagnostic classification system featuring numerous new codes for specific anatomic sites, co-morbidities, and causes. These additions facilitate conveying the complexities of various diseases. Currently, ICD-10 coding is widely adopted worldwide. However, public hospitals in Pakistan have yet to implement it and automate the coding process. In this research, we implemented ICD-10-CM coding for a private database and named it Clinical Pool of Liver Transplant (CPLT). Additionally, we proposed a novel deep learning model called Deep Recurrent-Convolution Neural Network with a lambda-scaled Attention module (DRCNN-ATT) using the CPLT database to achieve automatic ICD-10-CM coding. DRCNN-ATT combines a bi-directional long short-term memory network (bi-LSTM), a multi-scale convolutional neural network (MS-CNN), and a lambda-scaled attention module. Experimental results demonstrate that deep recurrent convolutional neural network (DRCNN) faces attention score vanishing problem with a standard attention module for automatic ICD coding. However, adding a lambda-scaled attention module resolves this issue. We evaluated DRCNN-ATT model using two distinct datasets: a private CPLT dataset and a public MIMIC III top 50 dataset. The results indicate that the DRCNN-ATT model outperformed various baselines by generating 0.862 micro F1 and 0.25 macro F1 scores on CPLT dataset and 0.705 micro F1 and 0.655 macro F1 scores on MIMIC III top 50 dataset. Furthermore, we also deployed our model for automatic ICD-10-CM coding using ngrok and the Flask APIs, which receives input, processes it, and then returns the results.

Retrieve and rerank for automated ICD coding via Contrastive Learning

Automated ICD coding is a multi-label prediction task aiming at assigning patient diagnoses with the most relevant subsets of disease codes. In the deep learning regime, recent works have suffered from large label set and heavy imbalance distribution. To mitigate the negative effect in such scenarios, we propose a retrieve and rerank framework that introduces the Contrastive Learning (CL) for label retrieval, allowing the model to make more accurate prediction from a simplified label space. Given the appealing discriminative power of CL, we adopt it as the training strategy to replace the standard cross-entropy objective and retrieve a small subset by taking the distance between clinical notes and ICD codes into account. After properly training, the retriever could implicitly capture the code co-occurrence, which makes up for the deficiency of cross-entropy assigning each label independently of the others. Further, we evolve a powerful model via a Transformer variant for refining and reranking the candidate set, which can extract semantically meaningful features from long clinical sequences. Applying our method on well-known models, experiments show that our framework provides more accurate results guaranteed by preselecting a small subset of candidates before fine-level reranking. Relying on the framework, our proposed model achieves 0.590 and 0.990 in terms of Micro-F1 and Micro-AUC on benchmark MIMIC-III.