Noninvasive Diagnostic Technique for Nonalcoholic Fatty Liver Disease Based on Features of Tongue Images

OBJECTIVE

To investigate a new noninvasive diagnostic model for nonalcoholic fatty liver disease (NAFLD) based on features of tongue images.

METHODS

Healthy controls and volunteers confirmed to have NAFLD by liver ultrasound were recruited from China-Japan Friendship Hospital between September 2018 and May 2019, then the anthropometric indexes and sampled tongue images were measured. The tongue images were labeled by features, based on a brief protocol, without knowing any other clinical data, after a series of corrections and data cleaning. The algorithm was trained on images using labels and several anthropometric indexes for inputs, utilizing machine learning technology. Finally, a logistic regression algorithm and a decision tree model were constructed as 2 diagnostic models for NAFLD.

RESULTS

A total of 720 subjects were enrolled in this study, including 432 patients with NAFLD and 288 healthy volunteers. Of them, 482 were randomly allocated into the training set and 238 into the validation set. The diagnostic model based on logistic regression exhibited excellent performance: in validation set, it achieved an accuracy of 86.98%, sensitivity of 91.43%, and specificity of 80.61%; with an area under the curve (AUC) of 0.93 [95% confidence interval (CI) 0.68-0.98]. The decision tree model achieved an accuracy of 81.09%, sensitivity of 91.43%, and specificity of 66.33%; with an AUC of 0.89 (95% CI 0.66-0.92) in validation set.

CONCLUSIONS

The features of tongue images were associated with NAFLD. Both the 2 diagnostic models, which would be convenient, noninvasive, lightweight, rapid, and inexpensive technical references for early screening, can accurately distinguish NAFLD and are worth further study.

Classifying Chinese Medicine Constitution Using Multimodal Deep-Learning Model

OBJECTIVE

To develop a multimodal deep-learning model for classifying Chinese medicine constitution, i.e., the balanced and unbalanced constitutions, based on inspection of tongue and face images, pulse waves from palpation, and health information from a total of 540 subjects.

METHODS

This study data consisted of tongue and face images, pulse waves obtained by palpation, and health information, including personal information, life habits, medical history, and current symptoms, from 540 subjects (202 males and 338 females). Convolutional neural networks, recurrent neural networks, and fully connected neural networks were used to extract deep features from the data. Feature fusion and decision fusion models were constructed for the multimodal data.

RESULTS

The optimal models for tongue and face images, pulse waves and health information were ResNet18, Gate Recurrent Unit, and entity embedding, respectively. Feature fusion was superior to decision fusion. The multimodal analysis revealed that multimodal data compensated for the loss of information from a single mode, resulting in improved classification performance.

CONCLUSIONS

Multimodal data fusion can supplement single model information and improve classification performance. Our research underscores the effectiveness of multimodal deep learning technology to identify body constitution for modernizing and improving the intelligent application of Chinese medicine.

Study on the difference and regularity of tongue images in 309 patients with different pathological stages of non-small cell lung cancer

BACKGROUND

Tongue diagnosis is a crucial traditional Chinese medicine (TCM) inspection method for TCM syndrome differentiation and treatment.

OBJECTIVE

The primary research focus was on tongue image characteristic parameters of patients with non-small cell lung cancer (NSCLC). Analysis of the tongue image parameters of various pathological stages of NSCLC provides technical support for establishing an integrated Chinese and Western auxiliary diagnosis and efficacy evaluation medicine system for lung cancer that integrates tongue image features.

METHODS

Tongue image characteristics of 309 patients with NSCLC and 206 controls were collected and analyzed clinically. The T-test or rank sum test and logistic regression analysis were applied to analyze the characteristics of tongue image indicators of different pathological stages of NSCLC.

RESULTS

There were differences in tongue image characteristics in the NSCLC group compared to the control group. The tongue quality and brightness of the tongue coating in the NSCLC group increased, the red component was reduced, the tongue coating thickened, and the yellow component increased compared to the healthy control group. A comparison of tongue image indexes of NSCLC in different pathological stages showed that stage IV had lower TB-b and higher TB-a than stage I. In addition, stage IV had lower TB-b than stage II + III, showing an increase in the blue and red components of the tongue in stage IV and the appearance of cyanotic tongue features.

CONCLUSION

The tongue image characteristics of NSCLC patients differed from those of the control group. Tongue imaging indicators can reflect the characteristics of tongue images of patients with NSCLC. The tongue image characteristics of patients with stage IV lung cancer are bluish and purple compared with those with stage I, II, and III. It is suggested that the tongue's image characteristics can be used as a reference for the pathological classification of NSCLC and judgment of the disease process.

A lung cancer risk warning model based on tongue images

Objective: To investigate the tongue image features of patients with lung cancer and benign pulmonary nodules and to construct a lung cancer risk warning model using machine learning methods. Methods: From July 2020 to March 2022, we collected 862 participants including 263 patients with lung cancer, 292 patients with benign pulmonary nodules, and 307 healthy subjects. The TFDA-1 digital tongue diagnosis instrument was used to capture tongue images, using feature extraction technology to obtain the index of the tongue images. The statistical characteristics and correlations of the tongue index were analyzed, and six machine learning algorithms were used to build prediction models of lung cancer based on different data sets. Results: Patients with benign pulmonary nodules had different statistical characteristics and correlations of tongue image data than patients with lung cancer. Among the models based on tongue image data, the random forest prediction model performed the best, with a model accuracy of 0.679 ± 0.048 and an AUC of 0.752 ± 0.051. The accuracy for the logistic regression, decision tree, SVM, random forest, neural network, and naïve bayes models based on both the baseline and tongue image data were 0.760 ± 0.021, 0.764 ± 0.043, 0.774 ± 0.029, 0.770 ± 0.050, 0.762 ± 0.059, and 0.709 ± 0.052, respectively, while the corresponding AUCs were 0.808 ± 0.031, 0.764 ± 0.033, 0.755 ± 0.027, 0.804 ± 0.029, 0.777 ± 0.044, and 0.795 ± 0.039, respectively. Conclusion: The tongue diagnosis data under the guidance of traditional Chinese medicine diagnostic theory was useful. The performance of models built on tongue image and baseline data was superior to that of the models built using only the tongue image data or the baseline data. Adding objective tongue image data to baseline data can significantly improve the efficacy of lung cancer prediction models.

Construction of Tongue Image-Based Machine Learning Model for Screening Patients with Gastric Precancerous Lesions

Screening patients with precancerous lesions of gastric cancer (PLGC) is important for gastric cancer prevention. The accuracy and convenience of PLGC screening could be improved with the use of machine learning methodologies to uncover and integrate valuable characteristics of noninvasive medical images related to PLGC. In this study, we therefore focused on tongue images and for the first time constructed a tongue image-based PLGC screening deep learning model (AITongue). The AITongue model uncovered potential associations between tongue image characteristics and PLGC, and integrated canonical risk factors, including age, sex, and Hp infection. Five-fold cross validation analysis on an independent cohort of 1995 patients revealed the AITongue model could screen PLGC individuals with an AUC of 0.75, 10.3% higher than that of the model with only including canonical risk factors. Of note, we investigated the value of the AITongue model in predicting PLGC risk by establishing a prospective PLGC follow-up cohort, reaching an AUC of 0.71. In addition, we developed a smartphone-based app screening system to enhance the application convenience of the AITongue model in the natural population from high-risk areas of gastric cancer in China. Collectively, our study has demonstrated the value of tongue image characteristics in PLGC screening and risk prediction.

A survey of artificial intelligence in tongue image for disease diagnosis and syndrome differentiation

The rapid development of artificial intelligence technology has gradually extended from the general field to all walks of life, and intelligent tongue diagnosis is the product of a miraculous connection between this new discipline and traditional disciplines. We reviewed the deep learning methods and machine learning applied in tongue image analysis that have been studied in the last 5 years, focusing on tongue image calibration, detection, segmentation, and classification of diseases, syndromes, and symptoms/signs. Introducing technical evolutions or emerging technologies were applied in tongue image analysis; as we have noticed, attention mechanism, multiscale features, and prior knowledge were successfully applied in it, and we emphasized the value of combining deep learning with traditional methods. We also pointed out two major problems concerned with data set construction and the low reliability of performance evaluation that exist in this field based on the basic essence of tongue diagnosis in traditional Chinese medicine. Finally, a perspective on the future of intelligent tongue diagnosis was presented; we believe that the self-supervised method, multimodal information fusion, and the study of tongue pathology will have great research significance.

Intra-Rater and Inter-Rater Reliability of Tongue Coating Diagnosis in Traditional Chinese Medicine Using Smartphones: Quasi-Delphi Study

Background

There is a growing trend in the use of mobile health (mHealth) technologies in traditional Chinese medicine (TCM) and telemedicine, especially during the coronavirus disease (COVID-19) outbreak. Tongue diagnosis is an important component of TCM, but also plays a role in Western medicine, for example in dermatology. However, the procedure of obtaining tongue images has not been standardized and the reliability of tongue diagnosis by smartphone tongue images has yet to be evaluated.

Objective

The first objective of this study was to develop an operating classification scheme for tongue coating diagnosis. The second and main objective of this study was to determine the intra-rater and inter-rater reliability of tongue coating diagnosis using the operating classification scheme.

Methods

An operating classification scheme for tongue coating was developed using a stepwise approach and a quasi-Delphi method. First, tongue images (n=2023) were analyzed by 2 groups of assessors to develop the operating classification scheme for tongue coating diagnosis. Based on clinicians’ (n=17) own interpretations as well as their use of the operating classification scheme, the results of tongue diagnosis on a representative tongue image set (n=24) were compared. After gathering consensus for the operating classification scheme, the clinicians were instructed to use the scheme to assess tongue features of their patients under direct visual inspection. At the same time, the clinicians took tongue images of the patients with smartphones and assessed tongue features observed in the smartphone image using the same classification scheme. The intra-rater agreements of these two assessments were calculated to determine which features of tongue coating were better retained by the image. Using the finalized operating classification scheme, clinicians in the study group assessed representative tongue images (n=24) that they had taken, and the intra-rater and inter-rater reliability of their assessments was evaluated.

Results

Intra-rater agreement between direct subject inspection and tongue image inspection was good to very good (Cohen κ range 0.69-1.0). Additionally, when comparing the assessment of tongue images on different days, intra-rater reliability was good to very good (κ range 0.7-1.0), except for the color of the tongue body (κ=0.22) and slippery tongue fur (κ=0.1). Inter-rater reliability was moderate for tongue coating (Gwet AC2 range 0.49-0.55), and fair for color and other features of the tongue body (Gwet AC2=0.34).

Conclusions

Taken together, our study has shown that tongue images collected via smartphone contain some reliable features, including tongue coating, that can be used in mHealth analysis. Our findings thus support the use of smartphones in telemedicine for detecting changes in tongue coating.