Constructing tongue coating recognition model using deep transfer learning to assist syndrome diagnosis and its potential in noninvasive ethnopharmacological evaluation

Tongue feature recognition to monitor rehabilitation: deep neural network with visual attention mechanism

Objective

We endeavor to develop a novel deep learning architecture tailored specifically for the analysis and classification of tongue features, including color, shape, and coating. Unlike conventional methods based on architectures like VGG or ResNet, our proposed method aims to address the challenges arising from their extensive size, thereby mitigating the overfitting problem. Through this research, we aim to contribute to the advancement of techniques in tongue feature recognition, ultimately leading to more precise diagnoses and better patient rehabilitation in Traditional Chinese Medicine (TCM).

Methods

In this study, we introduce TGANet (Tongue Feature Attention Network) to enhance model performance. TGANet utilizes the initial five convolutional blocks of pre-trained VGG16 as the backbone and integrates an attention mechanism into this backbone. The integration of the attention mechanism aims to mimic human cognitive attention, emphasizing model weights on pivotal regions of the image. During the learning process, the allocation of attention weights facilitates the interpretation of causal relationships in the model's decision-making.

Results

Experimental results demonstrate that TGANet outperforms baseline models, including VGG16, ResNet18, and TSC-WNet, in terms of accuracy, precision, F1 score, and AUC metrics. Additionally, TGANet provides a more intuitive and meaningful understanding of tongue feature classification models through the visualization of attention weights.

Conclusion

In conclusion, TGANet presents an effective approach to tongue feature classification, addressing challenges associated with model size and overfitting. By leveraging the attention mechanism and pre-trained VGG16 backbone, TGANet achieves superior performance metrics and enhances the interpretability of the model's decision-making process. The visualization of attention weights contributes to a more intuitive understanding of the classification process, making TGANet a promising tool in tongue diagnosis and rehabilitation.

Development and Application of Traditional Chinese Medicine Using AI Machine Learning and Deep Learning Strategies

Traditional Chinese medicine (TCM) has been used for thousands of years and has been proven to be effective at treating many complicated illnesses with minimal side effects. The application and advancement of TCM are, however, constrained by the absence of objective measuring standards due to its relatively abstract diagnostic methods and syndrome differentiation theories. Ongoing developments in machine learning (ML) and deep learning (DL), specifically in computer vision (CV) and natural language processing (NLP), offer novel opportunities to modernize TCM by exploring the profound connotations of its theory. This review begins with an overview of the ML and DL methods employed in TCM; this is followed by practical instances of these applications. Furthermore, extensive discussions emphasize the mature integration of ML and DL in TCM, such as tongue diagnosis, pulse diagnosis, and syndrome differentiation treatment, highlighting their early successful application in the TCM field. Finally, this study validates the accomplishments and addresses the problems and challenges posed by the application and development of TCM powered by ML and DL. As ML and DL techniques continue to evolve, modern technology will spark new advances in TCM.

Integrating artificial intelligence into the modernization of traditional Chinese medicine industry: a review

Traditional Chinese medicine (TCM) is the practical experience and summary of the Chinese nation for thousands of years. It shows great potential in treating various chronic diseases, complex diseases and major infectious diseases, and has gradually attracted the attention of people all over the world. However, due to the complexity of prescription and action mechanism of TCM, the development of TCM industry is still in a relatively conservative stage. With the rise of artificial intelligence technology in various fields, many scholars began to apply artificial intelligence technology to traditional Chinese medicine industry and made remarkable progress. This paper comprehensively summarizes the important role of artificial intelligence in the development of traditional Chinese medicine industry from various aspects, including new drug discovery, data mining, quality standardization and industry technology of traditional Chinese medicine. The limitations of artificial intelligence in these applications are also emphasized, including the lack of pharmacological research, database quality problems and the challenges brought by human-computer interaction. Nevertheless, the development of artificial intelligence has brought new opportunities and innovations to the modernization of traditional Chinese medicine. Integrating artificial intelligence technology into the comprehensive application of Chinese medicine industry is expected to overcome the major problems faced by traditional Chinese medicine industry and further promote the modernization of the whole traditional Chinese medicine industry.

In-depth metaproteomics analysis of tongue coating for gastric cancer: a multicenter diagnostic research study

BACKGROUND

Our previous study revealed marked differences in tongue images between individuals with gastric cancer and those without gastric cancer. However, the biological mechanism of tongue images as a disease indicator remains unclear. Tongue coating, a major factor in tongue appearance, is the visible layer on the tongue dorsum that provides a vital environment for oral microorganisms. While oral microorganisms are associated with gastric and intestinal diseases, the comprehensive function profiles of oral microbiota remain incompletely understood. Metaproteomics has unique strength in revealing functional profiles of microbiota that aid in comprehending the mechanism behind specific tongue coating formation and its role as an indicator of gastric cancer.

METHODS

We employed pressure cycling technology and data-independent acquisition (PCT-DIA) mass spectrometry to extract and identify tongue-coating proteins from 180 gastric cancer patients and 185 non-gastric cancer patients across 5 independent research centers in China. Additionally, we investigated the temporal stability of tongue-coating proteins based on a time-series cohort. Finally, we constructed a machine learning model using the stochastic gradient boosting algorithm to identify individuals at high risk of gastric cancer based on tongue-coating microbial proteins.

RESULTS

We measured 1432 human-derived proteins and 13,780 microbial proteins from 345 tongue-coating samples. The abundance of tongue-coating proteins exhibited high temporal stability within an individual. Notably, we observed the downregulation of human keratins KRT2 and KRT9 on the tongue surface, as well as the downregulation of ABC transporter COG1136 in microbiota, in gastric cancer patients. This suggests a decline in the defense capacity of the lingual mucosa. Finally, we established a machine learning model that employs 50 microbial proteins of tongue coating to identify individuals at a high risk of gastric cancer, achieving an area under the curve (AUC) of 0.91 in the independent validation cohort.

CONCLUSIONS

We characterized the alterations in tongue-coating proteins among gastric cancer patients and constructed a gastric cancer screening model based on microbial-derived tongue-coating proteins. Tongue-coating proteins are shown as a promising indicator for identifying high-risk groups for gastric cancer. Video Abstract.

ODQN-Net: Optimized Deep Q Neural Networks for Disease Prediction Through Tongue Image Analysis Using Remora Optimization Algorithm

Tongue analysis plays the major role in disease type prediction and classification according to Indian ayurvedic medicine. Traditionally, there is a manual inspection of tongue image by the expert ayurvedic doctor to identify or predict the disease. However, this is time-consuming and even imprecise. Due to the advancements in recent machine learning models, several researchers addressed the disease prediction from tongue image analysis. However, they have failed to provide enough accuracy. In addition, multiclass disease classification with enhanced accuracy is still a challenging task. Therefore, this article focuses on the development of optimized deep q-neural network (DQNN) for disease identification and classification from tongue images, hereafter referred as ODQN-Net. Initially, the multiscale retinex approach is introduced for enhancing the quality of tongue images, which also acts as a noise removal technique. In addition, a local ternary pattern is used to extract the disease-specific and disease-dependent features based on color analysis. Then, the best features are extracted from the available features set using the natural inspired Remora optimization algorithm with reduced computational time. Finally, the DQNN model is used to classify the type of diseases from these pretrained features. The obtained simulation performance on tongue imaging data set proved that the proposed ODQN-Net resulted in superior performance compared with state-of-the-art approaches with 99.17% of accuracy and 99.75% and 99.84% of F1-score and Mathew's correlation coefficient, respectively.

A Multi-stage Segmentation Method for Tongue Ecchymosis. ANNUAL INTERNATIONAL CONFERENCE OF THE IEEE ENGINEERING IN MEDICINE AND BIOLOGY SOCIETY. IEEE ENGINEERING IN MEDICINE AND BIOLOGY SOCIETY. ANNUAL INTERNATIONAL CONFERENCE 2023;2023:1-4. [PMID: 38083193 DOI: 10.1109/embc40787.2023.10340439] [Citation(s) in RCA: 0] [Impact Index Per Article: 0]

Tongue diagnosis is an important component of traditional Chinese medicine (TCM), in which tongue ecchymosis is the main diagnostic basis for the blood stasis syndrome of TCM. Most of the existing methods are unsupervised and cannot accurately segment tongue ecchymosis. In this paper, we propose a multi-stage segmentation method for tongue ecchymosis. We first employ an object detection model for rough localization of tongue ecchymosis, and then use the unsupervised clustering and the watershed transform for rough segmentation and fine segmentation of tongue ecchymosis respectively. To the best of our knowledge, we are the first to combine machine learning and deep learning to segment tongue ecchymosis. Experimental results show that the tongue ecchymoses obtained by our method are more similar to the real tongue ecchymoses compared with the existing methods, and the Intersection-over-Union (IoU) is improved by 0.12 compared with the latest method.Clinical Relevance-Tongue ecchymosis obtained by this paper is the main diagnostic basis for the blood stasis syndrome of TCM.

A survey of artificial intelligence in tongue image for disease diagnosis and syndrome differentiation

The rapid development of artificial intelligence technology has gradually extended from the general field to all walks of life, and intelligent tongue diagnosis is the product of a miraculous connection between this new discipline and traditional disciplines. We reviewed the deep learning methods and machine learning applied in tongue image analysis that have been studied in the last 5 years, focusing on tongue image calibration, detection, segmentation, and classification of diseases, syndromes, and symptoms/signs. Introducing technical evolutions or emerging technologies were applied in tongue image analysis; as we have noticed, attention mechanism, multiscale features, and prior knowledge were successfully applied in it, and we emphasized the value of combining deep learning with traditional methods. We also pointed out two major problems concerned with data set construction and the low reliability of performance evaluation that exist in this field based on the basic essence of tongue diagnosis in traditional Chinese medicine. Finally, a perspective on the future of intelligent tongue diagnosis was presented; we believe that the self-supervised method, multimodal information fusion, and the study of tongue pathology will have great research significance.

Deep learning and machine intelligence: New computational modeling techniques for discovery of the combination rules and pharmacodynamic characteristics of Traditional Chinese Medicine

It has been increasingly accepted that Multi-Ingredient-Based interventions provide advantages over single-target therapy for complex diseases. With the growing development of Traditional Chinese Medicine (TCM) and continually being refined of a holistic view, "multi-target" and "multi-pathway" integration characteristics of which are being accepted. However, its effector substances, efficacy targets, especially the combination rules and mechanisms remain unclear, and more powerful strategies to interpret the synergy are urgently needed. Artificial intelligence (AI) and computer vision lead to a rapidly expanding in many fields, including diagnosis and treatment of TCM. AI technology significantly improves the reliability and accuracy of diagnostics, target screening, and new drug research. While all AI techniques are capable of matching models to biological big data, the specific methods are complex and varied. Retrieves literature by the keywords such as "artificial intelligence", "machine learning", "deep learning", "traditional Chinese medicine" and "Chinese medicine". Search the application of computer algorithms of TCM between 2000 and 2021 in PubMed, Web of Science, China National Knowledge Infrastructure (CNKI), Elsevier and Springer. This review concentrates on the application of computational in herb quality evaluation, drug target discovery, optimized compatibility and medical diagnoses of TCM. We describe the characteristics of biological data for which different AI techniques are applicable, and discuss some of the best data mining methods and the problems faced by deep learning and machine learning methods applied to Chinese medicine.

Microbiological characteristics of different tongue coatings in adults

Background

Tongue coating is an important health indicator in traditional Chinese medicine (TCM). The tongue coating microbiome can distinguish disease patients from healthy controls. To study the relationship between different types of tongue coatings and health, we analyzed the species composition of different types of tongue coatings and the co-occurrence relationships between microorganisms in Chinese adults.

From June 2019 to October 2020, 158 adults from Hangzhou and Shaoxing City, Zhejiang Province, were enrolled. We classified the TCM tongue coatings into four different types: thin white tongue fur (TWF), thin yellow tongue fur (TYF), white greasy tongue fur (WGF), and yellow greasy tongue fur (YGF). Tongue coating specimens were collected and used for 16S rRNA gene sequencing using the Illumina MiSeq system. Wilcoxon rank-sum and permutational multivariate analysis of variance tests were used to analyze the data. The microbial networks in the four types of tongue coatings were inferred independently using sparse inverse covariance estimation for ecological association inference.

Results

The microbial composition was similar among the different tongue coatings; however, the abundance of microorganisms differed. TWF had a higher abundance of Fusobacterium periodonticum and Neisseria mucosa, the highest α-diversity, and a highly connected community (average degree = 3.59, average closeness centrality = 0.33). TYF had the lowest α-diversity, but the most species in the co-occurrence network diagram (number of nodes = 88). The platelet-to-lymphocyte ratio (PLR) was associated with tongue coating (P = 0.035), and the YGF and TYF groups had higher PLR values. In the co-occurrence network, Aggregatibacter segnis was the “driver species” of the TWF and TYF groups and correlated with C-reactive protein (P < 0.05). Streptococcus anginosus was the “driver species” in the YGF and TWF groups and was positively correlated with body mass index and weight (P < 0.05).

Conclusion

Different tongue coatings have similar microbial compositions but different abundances of certain bacteria. The co-occurrence of microorganisms in the different tongue coatings also varies. The significance of different tongue coatings in TCM theory is consistent with the characteristics and roles of the corresponding tongue-coating microbes. This further supports considering tongue coating as a risk factor for disease.

Weakly Supervised Deep Learning for Tooth-Marked Tongue Recognition

The recognition of tooth-marked tongues has important value for clinical diagnosis of traditional Chinese medicine. Tooth-marked tongue is often related to spleen deficiency, cold dampness, sputum, effusion, and blood stasis. The clinical manifestations of patients with tooth-marked tongue include loss of appetite, borborygmus, gastric distention, and loose stool. Traditional clinical tooth-marked tongue recognition is conducted subjectively based on the doctor’s visual observation, and its performance is affected by the doctor’s subjectivity, experience, and environmental lighting changes. In addition, the tooth marks typically have various shapes and colors on the tongue, which make it very challenging for doctors to identify tooth marks. The existing methods based on deep learning have made great progress for tooth-marked tongue recognition, but there are still shortcomings such as requiring a large amount of manual labeling of tooth marks, inability to detect and locate the tooth marks, and not conducive to clinical diagnosis and interpretation. In this study, we propose an end-to-end deep neural network for tooth-marked tongue recognition based on weakly supervised learning. Note that the deep neural network only requires image-level annotations of tooth-marked or non-tooth marked tongues. In this method, a deep neural network is trained to classify tooth-marked tongues with the image-level annotations. Then, a weakly supervised tooth-mark detection network (WSTDN) as an architecture variant of the pre-trained deep neural network is proposed for the tooth-marked region detection. Finally, the WSTDN is re-trained and fine-tuned using only the image-level annotations to simultaneously realize the classification of the tooth-marked tongue and the positioning of the tooth-marked region. Experimental results of clinical tongue images demonstrate the superiority of the proposed method compared with previously reported deep learning methods for tooth-marked tongue recognition. The proposed tooth-marked tongue recognition model may provide important syndrome diagnosis and efficacy evaluation methods, and contribute to the understanding of ethnopharmacological mechanisms.