Nasal skin temperature reveals changes in arousal levels due to time on task: An experimental thermal infrared imaging study

Driver drowsiness is associated with altered facial thermal patterns: Machine learning insights from a thermal imaging approach

Driver drowsiness is a significant factor in road accidents. Thermal imaging has emerged as an effective tool for detecting drowsiness by enabling the analysis of facial thermal patterns. However, it is not clear which facial areas are most affected and correlate most strongly with drowsiness. This study examines the variations and importance of various facial areas and proposes an approach for detecting driver drowsiness. Twenty participants underwent tests in a driving simulator, and temperature changes in various facial regions were measured. The random forest method was employed to evaluate the importance of each facial region. The results revealed that temperature changes in the nasal area exhibited the highest value, while the eyes had the most correlated changes with drowsiness. Furthermore, drowsiness was classified with an accuracy of 88 % utilizing thermal variations in the facial region identified as the most important regions by the random forest feature importance model. These findings provide a comprehensive overview of facial thermal imaging for detecting driver drowsiness and introduce eye temperature as a novel and effective measure for investigating cognitive activities.

IoT-Based Assessment of a Driver's Stress Level

Driver Monitoring Systems (DMSs) play a key role in preventing hazardous events (e.g., road accidents) by providing prompt assistance when anomalies are detected while driving. Different factors, such as traffic and road conditions, might alter the psycho-physiological status of a driver by increasing stress and workload levels. This motivates the development of advanced monitoring architectures taking into account psycho-physiological aspects. In this work, we propose a novel in-vehicle Internet of Things (IoT)-oriented monitoring system to assess the stress status of the driver. In detail, the system leverages heterogeneous components and techniques to collect driver (and, possibly, vehicle) data, aiming at estimating the driver's arousal level, i.e., their psycho-physiological response to driving tasks. In particular, a wearable sensorized bodice and a thermal camera are employed to extract physiological parameters of interest (namely, the heart rate and skin temperature of the subject), which are processed and analyzed with innovative algorithms. Finally, experimental results are obtained both in simulated and real driving scenarios, demonstrating the adaptability and efficacy of the proposed system.

Your blush gives you away: detecting hidden mental states with remote photoplethysmography and thermal imaging

Multimodal emotion recognition techniques are increasingly essential for assessing mental states. Image-based methods, however, tend to focus predominantly on overt visual cues and often overlook subtler mental state changes. Psychophysiological research has demonstrated that heart rate (HR) and skin temperature are effective in detecting autonomic nervous system (ANS) activities, thereby revealing these subtle changes. However, traditional HR tools are generally more costly and less portable, while skin temperature analysis usually necessitates extensive manual processing. Advances in remote photoplethysmography (r-PPG) and automatic thermal region of interest (ROI) detection algorithms have been developed to address these issues, yet their accuracy in practical applications remains limited. This study aims to bridge this gap by integrating r-PPG with thermal imaging to enhance prediction performance. Ninety participants completed a 20-min questionnaire to induce cognitive stress, followed by watching a film aimed at eliciting moral elevation. The results demonstrate that the combination of r-PPG and thermal imaging effectively detects emotional shifts. Using r-PPG alone, the prediction accuracy was 77% for cognitive stress and 61% for moral elevation, as determined by a support vector machine (SVM). Thermal imaging alone achieved 79% accuracy for cognitive stress and 78% for moral elevation, utilizing a random forest (RF) algorithm. An early fusion strategy of these modalities significantly improved accuracies, achieving 87% for cognitive stress and 83% for moral elevation using RF. Further analysis, which utilized statistical metrics and explainable machine learning methods including SHapley Additive exPlanations (SHAP), highlighted key features and clarified the relationship between cardiac responses and facial temperature variations. Notably, it was observed that cardiovascular features derived from r-PPG models had a more pronounced influence in data fusion, despite thermal imaging's higher predictive accuracy in unimodal analysis.

A hybrid approach for driver drowsiness detection utilizing practical data to improve performance system and applicability

BACKGROUND

Numerous systems for detecting driver drowsiness have been developed; however, these systems have not yet been widely used in real-time.

OBJECTIVE

The purpose of this study was to investigate at the feasibility of detecting alert and drowsy states in drivers using an integration of features from respiratory signals, vehicle lateral position, and reaction time and out-of-vehicle ways of data collection in order to improve the system's performance and applicability in the real world.

METHODS

Data was collected from 25 healthy volunteers in a driving simulator-based study. Their respiratory activity was recorded using a wearable belt and their reaction time and vehicle lateral position were measured using tests developed on the driving simulator. To induce drowsiness, a monotonous driving environment was used. Different time domain features have been extracted from respiratory signals and combined with the reaction time and lateral position of the vehicle for modeling. The observer of rating drowsiness (ORD) scale was used to label the driver's actual states. The t-tests and Man-Whitney test was used to select only statistically significant features (p < 0.05), that can differentiate between the alert and drowsy states effectively. Significant features then combined to investigate the improvement in performance using the Multilayer Perceptron (MLP), the Support Vector Machines (SVMs), the Decision Trees (DTs), and the Long Short Term Memory (LSTM) classifiers. The models were implemented in Python library 3.6.

RESULTS

The experimental results illustrate that the support vector machine classifier achieved accuracy of 88%, precision of 85%, recall of 83%, and F1 score of 84% using selected features.

CONCLUSION

These results indicate the possibility of very accurate detection of driver drowsiness and a viable solution for a practical driver drowsiness system based on combined measurement using less-intrusive and out-of-vehicle recording methods.

Facial infrared thermography as an index of social anxiety

Previous research on physiological indices of social anxiety has offered unclear results. In this study, participants with low and high social anxiety performed five social interaction tasks while being recorded with a thermal camera. Each task was associated with a dimension assessed by the Social Anxiety Questionnaire for Adults (1 = Interactions with strangers. 2 = Speaking in public/Talking with people in authority, 3 = Criticism and embarrassment, 4 = Assertive expression of annoyance, disgust or displeasure, 5 = Interactions with the opposite sex). Mixed-effects models revealed that the temperature of the tip of the nose decreased significantly in participants with low (vs. high) social anxiety (p < 0.001), while no significant differences were found in other facial regions of interest: forehead (p = 0.999) and cheeks (p = 0.999). Furthermore, task 1 was the most effective at discriminating between the thermal change of the nose tip and social anxiety, with a trend for a higher nose temperature in participants with high social anxiety and a lower nose temperature for the low social anxiety group. We emphasize the importance of corroborating thermography with specific tasks as an ecological method, and tip of the nose thermal change as a psychophysiological index associated with social anxiety.

Thermal Balance in Male Water Buffaloes Transported by Long and Short Journeys

Transport is a stressor that can cause physiological and metabolic imbalances in livestock, resulting in stress-induced hyperthermia. In water buffaloes, studies regarding the thermal state of animals during mobilization are scarce. Therefore, this study aimed to compare the thermal response of 1516 water buffaloes using infrared thermography (IRT) during 15 short trips (783 animals, 60,291 records, average duration = 50.33 min ± 5.48 min) and 14 long trips (733 animals, 56,441 records, average duration = 13.31 h ± 47.32 min). The surface temperature was assessed in 11 regions (periocular, lacrimal caruncle, nasal, lower eyelid, auricular, frontal-parietal, pelvic limb, torso, abdominal, lumbar, and thoracic) during seven phases from pasture to post-transport. It was found that the surface temperature of the periocular, lacrimal caruncle, nasal, auricular, frontal-parietal, pelvic limb, torso, abdominal, lumbar, and thoracic regions was significantly higher during SJs (+3 °C) when compared to LJs (p < 0.0001). In particular, the frontal-parietal region had a significant increase of 10 °C during the post-transport phase (p < 0.0001) in both groups, recording the highest temperatures during this phase. Likewise, a strong positive significant correlation between the different regions was found (r = 0.90, p < 0.0001). It is worth mentioning that the herding, loading, pre-, and post-transport phases were the ones where the greatest thermal response was recorded, possibly due to the influence of human interaction. Finally, a strong positive correlation (r above 0.9, p > 0.001) between the periocular, lacrimal caruncle, pinna, and pelvic limb was found. According to the results, SJ could be considered a stressful event that hinders thermal generation, contrarily to LJ.

Facial thermal response to non-painful stressor in premature and term neonates

BACKGROUND

This study is a preliminary clinical investigation with the objective to evaluate the facial thermal response of premature and term neonates to a non-painful stressor (hunger) using infrared thermography (IRT). The development of objective and reliable parameters to monitor pain and stress is of relevance for optimal neonatal outcome and achieving a better management of patient comfort.

METHODS

We enrolled 12 neonates ranging from 27 to 39 weeks gestation (median: 34) and aged 3-79 days (median: 13). Recordings were performed before and after feeding, with and without hunger. Six regions of interest were chosen for evaluation (nose tip, periorbital and corrugator region, forehead, perioral and chin region).

RESULTS

There was an increase in the facial temperature in infants immediately prior to their next feed relative to infants who were not hungry, with the nasal tip being the facial evaluation site with the greatest temperature change.

CONCLUSIONS

The IRT appears to be a feasible and suitable method to detect changes in the neonatal patient. The thermal variations observed seem to reflect an arousal mediated by the parasympathetic nervous system, which has been described in existing infant stress research.

IMPACT

This is the first study to examine the use of infrared thermography (IRT) in monitoring the facial thermal response to a mild stressor (hunger) in premature and term neonates. Hunger as a mild, non-pain-associated stressor showed a significant effect on the facial temperature. The thermal signature of the regions of interest chosen showed hunger-related thermal variations. Results suggest the feasibility and suitability of IRT as an objective diagnostic tool to approach stress and changes in the condition of the neonatal patient.

Thermal Imaging to Assess the Health Status in Wildlife Animals under Human Care: Limitations and Perspectives

Promoting animal welfare in wildlife species under human care requires the implementation of techniques for continuously monitoring their health. Infrared thermography is a non-invasive tool that uses the radiation emitted from the skin of animals to assess their thermal state. However, there are no established thermal windows in wildlife species because factors such as the thickness or color of the skin, type/length of coat, or presence of fur can influence the readings taken to obtain objective, sensitive values. Therefore, this review aims to discuss the usefulness and application of the ocular, nasal, thoracic, abdominal, and podal anatomical regions as thermal windows for evaluating zoo animals' thermal response and health status. A literature search of the Web of Science, Science Direct, and PubMed databases was performed to identify relevant studies that used IRT with wild species as a complementary diagnostic tool. Implementing IRT in zoos or conservation centers could also serve as a method for determining and monitoring optimal habitat designs to meet the needs of specific animals. In addition, we analyze the limitations of using IRT with various wildlife species under human care to understand better the differences among animals and the factors that must be considered when using infrared thermography.

Towards a Positive Welfare Protocol for Cattle: A Critical Review of Indicators and Suggestion of How We Might Proceed

Current animal welfare protocols focus on demonstrating the absence (or at least low levels) of indicators of poor welfare, potentially creating a mismatch between what is expected by society (an assurance of good animal welfare) and what is actually being delivered (an assurance of the absence of welfare problems). This paper explores how far we have come, and what work still needs to be done, if we are to develop a protocol for use on commercial dairy farms where the aim is to demonstrate the presence of positive welfare. Following conceptual considerations around a perceived “ideal” protocol, we propose that a future protocol should be constructed (i) of animal-based measures, (ii) of indicators of affective state, and (iii) be structured according to indicators of short-term emotion, medium-term moods and long-term cumulative assessment of negative and positive experiences of an animal's life until now (in contrast to the current focus on indicators that represent different domains/criteria of welfare). These three conditions imposed the overall structure within which we selected our indicators. The paper includes a critical review of the literature on potential indicators of positive affective states in cattle. Based on evidence about the validity and reliability of the different indicators, we select ear position, play, allogrooming, brush use and QBA as candidate indicators that we suggest could form a prototype positive welfare protocol. We emphasise that this prototype protocol has not been tested in practice and so it is perhaps not the protocol itself that is the main outcome of this paper, but the process of trying to develop it. In a final section of this paper, we reflect on some of the lessons learnt from this exercise and speculate on future perspectives. For example, while we consider we have moved towards a prototype positive welfare protocol for short-term affective states, future research energy should be directed towards valid indicators for the medium and long-term.

A cross-cultural comparison of visual search strategies and response times in road hazard perception testing

Road hazard perception is considered the most prominent higher-order cognitive skill related to traffic-accident involvement. Regional cultures and social rules that govern acceptable behavior may influence drivers' interpretation of a traffic situation and, consequently, the correct identification of potentially hazardous situations. Here, we aimed to compare hazard perception skills among four European countries that differ in their traffic culture, policies to reduce traffic risks, and fatal crashes: Ukraine, Italy, Spain, and Sweden. We developed a static hazard perception test in which driving scenes with different levels of braking affordance were presented while drivers' gaze was recorded. The test required drivers to indicate the action they would undertake: to brake vs. to keep driving. We assessed 218 young adult drivers. Multilevel models revealed that the scenes' levels of braking affordance (i.e., road hazard) modulated drivers' behavior. As the levels of braking affordance increased, drivers' responses became faster and their gaze entropy decreased (i.e., visual search strategy became less erratic). The country of origin influenced these effects. Ukrainian drivers were the fastest and Swedish drivers were the slowest to respond. For all countries, the decrement in response times was less marked in the case of experienced drivers. Also, Spanish drivers showed the most structured (least erratic) visual search strategy, whereas the Italians had the most rigid (most constant) one. These results suggest that road hazard perception can be defined cross-culturally, with cultural factors (e.g., traffic climate, legislation) modulating response times and visual search strategies. Our results also support the idea that a multimodal assessment methodology is possible for mass testing of road hazard perception and its outcomes would be relevant to understand how different traffic cultures shape driving behavior.

Thermal Infrared Imaging-Based Affective Computing and Its Application to Facilitate Human Robot Interaction: A Review

Over recent years, robots are increasingly being employed in several aspects of modern society. Among others, social robots have the potential to benefit education, healthcare, and tourism. To achieve this purpose, robots should be able to engage humans, recognize users’ emotions, and to some extent properly react and "behave" in a natural interaction. Most robotics applications primarily use visual information for emotion recognition, which is often based on facial expressions. However, the display of emotional states through facial expression is inherently a voluntary controlled process that is typical of human–human interaction. In fact, humans have not yet learned to use this channel when communicating with a robotic technology. Hence, there is an urgent need to exploit emotion information channels not directly controlled by humans, such as those that can be ascribed to physiological modulations. Thermal infrared imaging-based affective computing has the potential to be the solution to such an issue. It is a validated technology that allows the non-obtrusive monitoring of physiological parameters and from which it might be possible to infer affective states. This review is aimed to outline the advantages and the current research challenges of thermal imaging-based affective computing for human–robot interaction.

EEG Theta Power Activity Reflects Workload among Army Combat Drivers: An Experimental Study

We aimed to evaluate the effects of mental workload variations, as a function of the road environment, on the brain activity of army drivers performing combat and non-combat scenarios in a light multirole vehicle dynamic simulator. Forty-one non-commissioned officers completed three standardized driving exercises with different terrain complexities (low, medium, and high) while we recorded their electroencephalographic (EEG) activity. We focused on variations in the theta EEG power spectrum, a well-known index of mental workload. We also assessed performance and subjective ratings of task load. The theta EEG power spectrum in the frontal, temporal, and occipital areas were higher during the most complex scenarios. Performance (number of engine stops) and subjective data supported these findings. Our findings strengthen previous results found in civilians on the relationship between driver mental workload and the theta EEG power spectrum. This suggests that EEG activity can give relevant insight into mental workload variations in an objective, unbiased fashion, even during real training and/or operations. The continuous monitoring of the warfighter not only allows instantaneous detection of over/underload but also might provide online feedback to the system (either automated equipment or the crew) to take countermeasures and prevent fatal errors.

Monitoring core temperature of firefighters to validate a wearable non-invasive core thermometer in different types of protective clothing: Concurrent in-vivo validation

This study aims (1) to test the validity of a new non-invasive core thermometer, Cosinuss°, in rest and (2) during firefighting simulation tasks, against invasive temperature pill and inner-ear temperature and (3) to compare the change in core temperature of firefighters when working in two types of protective clothing (traditional turnout gear versus new concept). 11 active firefighters performed twice a selection of tasks during their periodic preventive medical examination and a fire-extinguishing task. Without correction no correlation between the Cosinuss° and thermometer pill (ICC≤0.09, p ≥ 0.154, LoA≥1.37) and a moderate correlation between Cosinuss° and inner-ear infrared (ICC = 0.40, p = 0.044, LoA±1.20) was observed. With individual correction both correlations were excellent (ICC≥0.84, p = 0.000, LoA≤0.30). However, during and after working all correlations were poor and non-significant (ICC≤0.38, p ≥ 0.091, LoA≥1.71). During firefighting tasks, the Cosinuss° is invalid for measuring the core temperature. No differences in heat development in the two types of protective clothing was proven.

Validation of Electroencephalographic Recordings Obtained with a Consumer-Grade, Single Dry Electrode, Low-Cost Device: A Comparative Study

The functional validity of the signal obtained with low-cost electroencephalography (EEG) devices is still under debate. Here, we have conducted an in-depth comparison of the EEG-recordings obtained with a medical-grade golden-cup electrodes ambulatory device, the SOMNOwatch + EEG-6, vs those obtained with a consumer-grade, single dry electrode low-cost device, the NeuroSky MindWave, one of the most affordable devices currently available. We recorded EEG signals at Fp1 using the two different devices simultaneously on 21 participants who underwent two experimental phases: a 12-minute resting state task (alternating two cycles of closed/open eyes periods), followed by 60-minute virtual-driving task. We evaluated the EEG recording quality by comparing the similarity between the temporal data series, their spectra, their signal-to-noise ratio, the reliability of EEG measurements (comparing the closed eyes periods), as well as their blink detection rate. We found substantial agreement between signals: whereas, qualitatively, the NeuroSky MindWave presented higher levels of noise and a biphasic shape of blinks, the similarity metric indicated that signals from both recording devices were significantly correlated. While the NeuroSky MindWave was less reliable, both devices had a similar blink detection rate. Overall, the NeuroSky MindWave is noise-limited, but provides stable recordings even through long periods of time. Furthermore, its data would be of adequate quality compared to that of conventional wet electrode EEG devices, except for a potential calibration error and spectral differences at low frequencies.