Rapid extraction of the hottest or coldest regions of medical thermographic images

A Novel Machine Learning Approach for Severity Classification of Diabetic Foot Complications Using Thermogram Images

Diabetes mellitus (DM) is one of the most prevalent diseases in the world, and is correlated to a high index of mortality. One of its major complications is diabetic foot, leading to plantar ulcers, amputation, and death. Several studies report that a thermogram helps to detect changes in the plantar temperature of the foot, which may lead to a higher risk of ulceration. However, in diabetic patients, the distribution of plantar temperature does not follow a standard pattern, thereby making it difficult to quantify the changes. The abnormal temperature distribution in infrared (IR) foot thermogram images can be used for the early detection of diabetic foot before ulceration to avoid complications. There is no machine learning-based technique reported in the literature to classify these thermograms based on the severity of diabetic foot complications. This paper uses an available labeled diabetic thermogram dataset and uses the k-mean clustering technique to cluster the severity risk of diabetic foot ulcers using an unsupervised approach. Using the plantar foot temperature, the new clustered dataset is verified by expert medical doctors in terms of risk for the development of foot ulcers. The newly labeled dataset is then investigated in terms of robustness to be classified by any machine learning network. Classical machine learning algorithms with feature engineering and a convolutional neural network (CNN) with image-enhancement techniques are investigated to provide the best-performing network in classifying thermograms based on severity. It is found that the popular VGG 19 CNN model shows an accuracy, precision, sensitivity, F1-score, and specificity of 95.08%, 95.08%, 95.09%, 95.08%, and 97.2%, respectively, in the stratification of severity. A stacking classifier is proposed using extracted features of the thermogram, which is created using the trained gradient boost classifier, XGBoost classifier, and random forest classifier. This provides a comparable performance of 94.47%, 94.45%, 94.47%, 94.43%, and 93.25% for accuracy, precision, sensitivity, F1-score, and specificity, respectively.

Automatic Detection of Human Maxillofacial Tumors by Using Thermal Imaging: A Preliminary Study

Traditional computed tomography (CT) delivers a relatively high dose of radiation to the patient and cannot be used as a method for screening of pathologies. Instead, infrared thermography (IRT) might help in the detection of pathologies, but interpreting thermal imaging (TI) is difficult even for the expert. The main objective of this work is to present a new, automated IRT method capable to discern the absence or presence of tumor in the orofacial/maxillofacial region of patients. We evaluated the use of a special feature vector extracted from face and mouth cavity thermograms in classifying TIs against the absence/presence of tumor (n = 23 patients per group). Eight statistical features extracted from TI were used in a k-nearest neighbor (kNN) classifier. Classification accuracy of kNN was evaluated by CT, and by creating a vector with the true class labels for TIs. The presented algorithm, constructed from a training data set, gives good results of classification accuracy of kNN: sensitivity of 77.9%, specificity of 94.9%, and accuracy of 94.1%. The new algorithm exhibited almost the same accuracy in detecting the absence/presence of tumor as CT, and is a proof-of-principle that IRT could be useful as an additional reliable screening tool for detecting orofacial/maxillofacial tumors.

Facial Paralysis Detection in Infrared Thermal Images Using Asymmetry Analysis of Temperature and Texture Features

Facial temperature distribution in healthy people shows contralateral symmetry, which is generally disrupted by facial paralysis. This study aims to develop a quantitative thermal asymmetry analysis method for early diagnosis of facial paralysis in infrared thermal images. First, to improve the reliability of thermal image analysis, the facial regions of interest (ROIs) were segmented using corner and edge detection. A new temperature feature was then defined using the maximum and minimum temperature, and it was combined with the texture feature to represent temperature distribution of facial ROIs. Finally, Minkowski distance was used to measure feature symmetry of bilateral ROIs. The feature symmetry vectors were input into support vector machine to evaluate the degree of facial thermal symmetry. The results showed that there were significant differences in thermal symmetry between patients with facial paralysis and healthy people. The accuracy of the proposed method for early diagnosis of facial paralysis was 0.933, and the area under the ROC curve was 0.947. In conclusion, temperature and texture features can effectively quantify thermal asymmetry caused by facial paralysis, and the application of machine learning in early detection of facial paralysis in thermal images is feasible.

A machine learning model for early detection of diabetic foot using thermogram images

Diabetes foot ulceration (DFU) and amputation are a cause of significant morbidity. The prevention of DFU may be achieved by the identification of patients at risk of DFU and the institution of preventative measures through education and offloading. Several studies have reported that thermogram images may help to detect an increase in plantar temperature prior to DFU. However, the distribution of plantar temperature may be heterogeneous, making it difficult to quantify and utilize to predict outcomes. We have compared a machine learning-based scoring technique with feature selection and optimization techniques and learning classifiers to several state-of-the-art Convolutional Neural Networks (CNNs) on foot thermogram images and propose a robust solution to identify the diabetic foot. A comparatively shallow CNN model, MobilenetV2 achieved an F1 score of ∼95% for a two-feet thermogram image-based classification and the AdaBoost Classifier used 10 features and achieved an F1 score of 97%. A comparison of the inference time for the best-performing networks confirmed that the proposed algorithm can be deployed as a smartphone application to allow the user to monitor the progression of the DFU in a home setting.

Non-obstructive monitoring of muscle fatigue for low intensity dynamic exercise with infrared thermography technique

Surface electromyography (sEMG) has been widely used in evaluating muscle fatigue among athletes where electrodes are attached on the skin during the activity. Recently, infrared thermography technique (IRT) has gain popularity and shown to be another preferred method in monitoring and predicting muscle fatigue non-obstructively. This paper investigates the correlation between surface temperature and muscle activation parameters obtained using both IRT and sEMG methods simultaneously. Twenty healthy subjects were required to perform a repetitive calf raise exercise with various loads attached around their ankle for 3 min to induce fatigue on the targeted gastrocnemius muscles. Average temperature and temperature difference information were extracted from thermal images, while root mean square (RMS) and median frequency (MF) were extracted from sEMG signals. Spearman statistical analysis performed shows that there is a significant correlation between average temperature with RMS and between temperature difference with MF values at p<0.05. While ANOVA test conducted shows that there is significant impact of loads on RMS and MF where F=12.61 and 3.59, respectively, at p< 0.05. This study suggested that skin surface temperature can be utilized in monitoring and predicting muscle fatigue in low intensity dynamic exercise and can be extended to other dynamic exercises.

Deep Instance Segmentation of Laboratory Animals in Thermal Images

In this paper we focus on the role of deep instance segmentation of laboratory rodents in thermal images. Thermal imaging is very suitable to observe the behaviour of laboratory animals, especially in low light conditions. It is an non-intrusive method allowing to monitor the activity of animals and potentially observe some physiological changes expressed in dynamic thermal patterns. The analysis of the recorded sequence of thermal images requires smart algorithms for automatic processing of millions of thermal frames. Instance image segmentation allows to extract each animal from a frame and track its activity and thermal patterns. In this work, we adopted two instance segmentation algorithms, i.e., Mask R-CNN and TensorMask. Both methods in different configurations were applied to a set of thermal sequences, and both achieved high results. The best results were obtained for the TensorMask model, initially pre-trained on visible light images and finally trained on thermal images of rodents. The achieved mean average precision was above 90 percent, which proves that model pre-training on visible images can improve results of thermal image segmentation.

Deep Learning Classification for Diabetic Foot Thermograms

According to the World Health Organization (WHO), Diabetes Mellitus (DM) is one of the most prevalent diseases in the world. It is also associated with a high mortality index. Diabetic foot is one of its main complications, and it comprises the development of plantar ulcers that could result in an amputation. Several works report that thermography is useful to detect changes in the plantar temperature, which could give rise to a higher risk of ulceration. However, the plantar temperature distribution does not follow a particular pattern in diabetic patients, thereby making it difficult to measure the changes. Thus, there is an interest in improving the success of the analysis and classification methods that help to detect abnormal changes in the plantar temperature. All this leads to the use of computer-aided systems, such as those involved in artificial intelligence (AI), which operate with highly complex data structures. This paper compares machine learning-based techniques with Deep Learning (DL) structures. We tested common structures in the mode of transfer learning, including AlexNet and GoogleNet. Moreover, we designed a new DL-structure, which is trained from scratch and is able to reach higher values in terms of accuracy and other quality measures. The main goal of this work is to analyze the use of AI and DL for the classification of diabetic foot thermograms, highlighting their advantages and limitations. To the best of our knowledge, this is the first proposal of DL networks applied to the classification of diabetic foot thermograms. The experiments are conducted over thermograms of DM and control groups. After that, a multi-level classification is performed based on a previously reported thermal change index. The high accuracy obtained shows the usefulness of AI and DL as auxiliary tools to aid during the medical diagnosis.

Breast Cancer Identification via Thermography Image Segmentation with a Gradient Vector Flow and a Convolutional Neural Network

Breast cancer is the most common cancer among women worldwide with about half a million cases reported each year. Mammary thermography can offer early diagnosis at low cost if adequate thermographic images of the breasts are taken. The identification of breast cancer in an automated way can accelerate many tasks and applications of pathology. This can help complement diagnosis. The aim of this work is to develop a system that automatically captures thermographic images of breast and classifies them as normal and abnormal (without cancer and with cancer). This paper focuses on a segmentation method based on a combination of the curvature function k and the gradient vector flow, and for classification, we proposed a convolutional neural network (CNN) using the segmented breast. The aim of this paper is to compare CNN results with other classification techniques. Thus, every breast is characterized by its shape, colour, and texture, as well as left or right breast. These data were used for training as well as to compare the performance of CNN with three classification techniques: tree random forest (TRF), multilayer perceptron (MLP), and Bayes network (BN). CNN presents better results than TRF, MLP, and BN.

Breast Cancer Detection in Thermal Infrared Images Using Representation Learning and Texture Analysis Methods

Nowadays, breast cancer is one of the most common cancers diagnosed in women. Mammography is the standard screening imaging technique for the early detection of breast cancer. However, thermal infrared images (thermographies) can be used to reveal lesions in dense breasts. In these images, the temperature of the regions that contain tumors is warmer than the normal tissue. To detect that difference in temperature between normal and cancerous regions, a dynamic thermography procedure uses thermal infrared cameras to generate infrared images at fixed time steps, obtaining a sequence of infrared images. In this paper, we propose a novel method to model the changes on temperatures in normal and abnormal breasts using a representation learning technique called learning-to-rank and texture analysis methods. The proposed method generates a compact representation for the infrared images of each sequence, which is then exploited to differentiate between normal and cancerous cases. Our method produced competitive (AUC = 0.989) results when compared to other studies in the literature.