Gaze entropy measures detect alcohol-induced driver impairment

A semi-naturalistic open-label study examining the effect of prescribed medical cannabis use on simulated driving performance

BACKGROUND

Despite increasing medical cannabis use, research has yet to establish whether and to what extent products containing delta-9-tetrahydrocannabinol (THC) impact driving performance among patients. Stable doses of prescribed cannabinoid products during long-term treatment may alleviate clinical symptoms affecting cognitive and psychomotor performance.

AIM

To examine the effects of open-label prescribed medical cannabis use on simulated driving performance among patients.

METHODS

In a semi-naturalistic laboratory study, 40 adults (55% male) aged between 23 and 80 years, consumed their own prescribed medical cannabis product. Driving performance outcomes including standard deviation of lateral position (SDLP), the standard deviation of speed (SDS), mean speed and steering variability were evaluated using the Forum8 driving simulator at baseline (pre-dosing), 2.5 h and 5 -h (post-dosing). Perceived driving effort (PDE) was self-reported after each drive. Oral fluid and whole blood samples were collected at multiple timepoints and analysed for THC via liquid chromatography-mass spectrometry.

RESULTS

A significant main effect of time was observed for mean speed (p = 0.014) and PDE (p = 0.020), with patients displaying modest stabilisation of vehicle control, increased adherence to speed limits and reductions in PDE post-dosing, relative to baseline. SDLP (p = 0.015) and PDE (p = 0.043) were elevated for those who consumed oil relative to flower-based products. Detectable THC concentrations were observed in oral fluid at 6-h post-dosing (range = 0-24 ng/mL).

CONCLUSIONS

This semi-naturalistic study suggests that the consumption of medical cannabis containing THC (1.13-39.18 mg/dose) has a negligible impact on driving performance when used as prescribed.

What can entropy metrics tell us about the characteristics of ocular fixation trajectories?

In this study, we provide a detailed analysis of entropy measures calculated for fixation eye movement trajectories from the three different datasets. We employed six key metrics (Fuzzy, Increment, Sample, Gridded Distribution, Phase, and Spectral Entropies). We calculate these six metrics on three sets of fixations: (1) fixations from the GazeCom dataset, (2) fixations from what we refer to as the "Lund" dataset, and (3) fixations from our own research laboratory ("OK Lab" dataset). For each entropy measure, for each dataset, we closely examined the 36 fixations with the highest entropy and the 36 fixations with the lowest entropy. From this, it was clear that the nature of the information from our entropy metrics depended on which dataset was evaluated. These entropy metrics found various types of misclassified fixations in the GazeCom dataset. Two entropy metrics also detected fixation with substantial linear drift. For the Lund dataset, the only finding was that low spectral entropy was associated with what we call "bumpy" fixations. These are fixations with low-frequency oscillations. For the OK Lab dataset, three entropies found fixations with high-frequency noise which probably represent ocular microtremor. In this dataset, one entropy found fixations with linear drift. The between-dataset results are discussed in terms of the number of fixations in each dataset, the different eye movement stimuli employed, and the method of eye movement classification.

Driving impairment and altered ocular activity under the effects of alprazolam and alcohol: A randomized, double-blind, placebo-controlled study

BACKGROUND

Alprazolam, also known by trade-name Xanax, is regularly detected along with alcohol in blood samples of drivers injured or killed in traffic collisions. While their co-consumption is principally legal, policy guidelines concerning fitness-to-drive are lacking and methods to index impairment are underdeveloped.

METHODS

In this randomized, double-blind, placebo-controlled, crossover trial, we examined whether legally permissible levels of alcohol [target 0.04% blood alcohol concentration (BAC)], alprazolam (1mg), and their combination impacts driving performance, and whether driving impairment can be indexed by ocular activity. Participants completed a test battery consisting of a 40-minute simulated highway drive with ocular parameters assessed simultaneously, the Karolinska Sleepiness Scale, and a confidence to drive assessment following four separate treatment combinations. The predictive efficacy of ocular parameters to identify alcohol and alprazolam-related driving impairment was also examined.

RESULTS

Among 21 healthy, fully licensed drivers (37% female, mean age 28.43, SD ± 3.96), driving performance was significantly impacted by alprazolam, alcohol, and their combination. Linear regression models revealed that the odds of an out-of-lane event occurring increased five-fold under the influence alprazolam alone and when combined with alcohol. An increase in gaze transition entropy (GTE) demonstrated the strongest association with the odds of an out-of-lane event occurring in the same minute, with both microsleeps and fixation rate achieving moderate accuracy across treatments.

CONCLUSIONS

Alprazolam and alcohol, alone and in combination, impaired select aspects of vehicle control over time. GTE, microsleeps, and fixation rate show potential as real-time indicators of driving impairment and crash risk associated with alcohol and alprazolam consumption.

Effect of CannEpil® on simulated driving performance and co-monitoring of ocular activity: A randomised controlled trial

BACKGROUND

Medicinal cannabis products containing Δ9-tetrahydrocannabinol (THC) are increasingly accessible. Yet, policy guidelines regarding fitness to drive are lacking, and cannabinoid-specific indexations of impairment are underdeveloped.

AIMS

To determine the impact of a standardised 1 mL sublingual dose of CannEpil^®, a medicinal cannabis oil containing 100 mg cannabidiol (CBD) and 5 mg THC on simulated driving performance, relative to placebo and whether variations in vehicle control can be indexed by ocular activity.

METHODS

A double-blind, within-subjects, randomised, placebo-controlled, crossover trial assessed 31 healthy fully licensed drivers (15 male, 16 female) aged between 21 and 58 years (M = 38.0, SD = 10.78). Standard deviation of lateral position (SDLP), standard deviation of speed (SDS) and steering variability were assessed over time and as a function of treatment during a 40 min simulated drive, with oculomotor parameters assessed simultaneously. Oral fluid and plasma were collected at 30 min and 2.5 h.

RESULTS

CannEpil did not significantly alter SDLP across the full drive, although increased SDLP was observed between 20 and 30 min (p < 0.05). CannEpil increased SDS across the full drive (p < 0.05), with variance greatest at 20-30 min (p < 0.001). CannEpil increased fixation duration (p < 0.05), blink rate (trend p = 0.051) and decreased blink duration (p < 0.001) during driving. No significant correlations were observed between biological matrices and performance outcomes.

CONCLUSIONS

CannEpil impairs select aspects of vehicle control (speed and weaving) over time. Alterations to ocular behaviour suggest that eye tracking may assist in determining cannabis-related driver impairment or intoxication. Australian and New Zealand Clinician Trials Registry, https://anzctr.org.au(ACTRN12619000932167).

Convergent validity of video-based observer rating of drowsiness, against subjective, behavioral, and physiological measures

Driver drowsiness is a widely recognized cause of motor vehicle accidents. Therefore, a reduction in drowsy driving crashes is required. Many studies evaluating the crash risk of drowsy driving and developing drowsiness detection systems, have used observer rating of drowsiness (ORD) as a reference standard (i.e. ground truth) of drowsiness. ORD is a method of human raters evaluating the levels of driver drowsiness, by visually observing a driver. Despite the widespread use of ORD, concerns remain regarding its convergent validity, which is supported by the relationship between ORD and other drowsiness measures. The objective of the present study was to validate video-based ORD, by examining correlations between ORD levels and other drowsiness measures. Seventeen participants performed eight sessions of a simulated driving task, verbally responding to Karolinska sleepiness scale (KSS), while infra-red face video, lateral position of the participant's car, eye closure, electrooculography (EOG), and electroencephalography (EEG) were recorded. Three experienced raters evaluated the ORD levels by observing facial videos. The results showed significant positive correlations between the ORD levels and all other drowsiness measures (i.e., KSS, standard deviation of the lateral position of the car, percentage of time occupied by slow eye movement calculated from EOG, EEG alpha power, and EEG theta power). The results support the convergent validity of video-based ORD as a measure of driver drowsiness. This suggests that ORD might be suitable as a ground truth for drowsiness.

Audio-based Deep Learning Algorithm to Identify Alcohol Inebriation (ADLAIA)

BACKGROUND

Acute alcohol intoxication impairs cognitive and psychomotor abilities leading to various public health hazards such as road traffic accidents and alcohol-related violence. Intoxicated individuals are usually identified by measuring their blood alcohol concentration (BAC) using breathalysers that are expensive and labour-intensive. In this paper, we developed the Audio-based Deep Learning Algorithm to Identify Alcohol Inebriation (ADLAIA) that can instantly predict an individual's intoxication status based on a 12-second recording of their speech.

METHODS

ADLAIA was trained on a publicly available German Alcohol Language Corpus that comprises a total of 12,360 audio clips of inebriated and sober speakers (total of 162, aged 21-64, 47.7% female). ADLAIA's performance was determined by computing the unweighted average recall (UAR) and accuracy of inebriation prediction.

RESULTS

ADLAIA was able to identify inebriated speakers-with BAC of 0.05% or higher-with an UAR of 68.09% and accuracy of 67.67%. ADLAIA had a higher performance (UAR of 75.7%) in identifying intoxicated speakers (BAC > 0.12%).

CONCLUSION

Being able to identify intoxicated individuals solely based on their speech, ADLAIA could be integrated in mobile applications and used in environments (such as bars, sports stadiums) to get instantaneous results about inebriation status of individuals.

Multimodal driver state modeling through unsupervised learning

Naturalistic driving data (NDD) can help understand drivers' reactions to each driving scenario and provide personalized context to driving behavior. However, NDD requires a high amount of manual labor to label certain driver's state and behavioral patterns. Unsupervised analysis of NDD can be used to automatically detect different patterns from the driver and vehicle data. In this paper, we propose a methodology to understand changes in driver's physiological responses within different driving patterns. Our methodology first decomposes a driving scenario by using a Bayesian Change Point detection model. We then apply the Latent Dirichlet Allocation method on both driver state and behavior data to detect patterns. We present two case studies in which vehicles were equipped to collect exterior, interior, and driver behavioral data. Four patterns of driving behaviors (i.e., harsh brake, normal brake, curved driving, and highway driving), as well as two patterns of driver's heart rate (HR) (i.e., normal vs. abnormal high HR), and gaze entropy (i.e., low versus high), were detected in these two case studies. The findings of these case studies indicated that among our participants, the drivers' HR had a higher fraction of abnormal patterns during harsh brakes, accelerating and curved driving. Additionally, free-flow driving with close to zero accelerations on the highway was accompanied by more fraction of normal HR as well as a lower gaze entropy pattern. With the proposed methodology we can better understand variations in driver's psychophysiological states within different driving scenarios. The findings of this work, has the potential to guide future autonomous vehicles to take actions that are fit to each specific driver.

The Static and Dynamic Analyses of Drivers’ Gaze Movement Using VR Driving Simulator

Drivers collect information of road and traffic conditions through a visual search while driving to avoid any potential hazards they perceive. Novice drivers with lack of driving experience may be involved in a car accident as they misjudge the information obtained by insufficient visual search with a narrower field of vision than experienced drivers do. In this regard, the current study compared and identified the gap between novice and experienced drivers in regard to the information they obtained in a visual search of gaze movement and visual attention. A combination of a static analysis, based on the dwell time, fixation duration, the number of fixations and stationary gaze entropy in visual search, and a dynamic analysis using gaze transition entropy was applied. The static analysis on gaze indicated that the group of novice drivers showed a longer dwell time on the traffic lights, pedestrians, and passing vehicles, and a longer fixation duration on the navigation system and the dashboard than the experienced ones. Also, the novice had their eyes fixed on the area of interests straight ahead more frequently while driving at an intersection. In addition, the novice group demonstrated less information at 2.60 bits out of the maximum stationary gaze entropy of 3.32 bits that a driver can exhibit, which indicated that their gaze fixations were concentrated. Meanwhile, the experienced group displayed approx. 3.09 bits, showing that their gaze was not narrowed on a certain area of interests, but was relatively evenly distributed. The dynamic analysis results showed that the novice group conducted the most gaze transitions between traffic lights, pedestrians and passing vehicles, whereas experienced drivers displayed the most transitions between the right- and left-side mirrors, passing vehicles, pedestrians, and traffic lights to find more out about the surrounding traffic conditions. In addition, the experienced group (3.04 bits) showed a higher gaze transition entropy than the novice group (2.21 bits). This indicated that a larger entropy was required to understand the visual search data because visual search strategies changed depending on the situations.

A driving simulation study on the effects of different wine types on the performance of young drivers

BACKGROUND

Alcohol consumption is responsible for a significant number of road fatalities. To contrast this phenomenon, a more responsible attitude to the wine consumption, especially among young, inexperienced drivers prone to risky behaviour on the road must be promoted.

METHOD

This is a simplified single-blind, placebo-controlled experiment aimed at evaluating 44 young drivers monitored during a driving simulation following the consumption of natural and conventional wines, with a reference blood alcohol concentration (BAC) of 0.5 g/l. Two hypotheses are tested: (1) the legal consumption of wine has no significant impact on young drivers' performance in both ordinary and unusual road events; (2) natural and conventional wines are expected to produce negligible and acceptable impairments in young drivers the same BAC. Two reference groups (BAC = 0 g/l), one a placebo-controlled group with drivers treated with a dealcoholized wine, were included.

RESULTS AND CONCLUSIONS

Significant differences between the groups in terms of perception and reaction times (PRT) to visual and auditory stimuli, and to speeding were observed, with young drivers treated with conventional wine displaying more aggressive behaviours. In contrast, participants treated with natural wine showed PRT which were not significantly different from those belonging to control groups. The gaze attention levels of wine treated drivers were found to be dose dependant, with young drivers of the two control groups and those of the treated ones with BAC < 0.3 g/l able to focus on wider area ahead and, thereby, collect more information from the road environment.

Driver monitoring systems (DMS): The future of impaired driving management?

OBJECTIVE

Driver monitoring systems (DMS) are the next generation of vehicle safety technology. Broadly, these refer to the embedded, aftermarket wearable or vehicle-mounted devices that collect observable information about the operator to make real-time assessment of their capacity to perform the driving task. Integrating biobehavioral monitoring (primarily ocular metrics) with driving performance assessments, these systems function to infer driver state in real time to identify operator conditions that negatively affect driving (such as fatigue, inattention, or distraction).

METHOD

We review available methods used to infer driver state, as referenced against accepted models for optimal performance. Modeling our observations on deviation from predetermined performance thresholds used to trigger graded safety alerts, we suggest that many psychoactive substances produce alterations to biobehavioral processes including attentional and motor control, which affect performance indices in a manner already arguably captured by these technologies.

RESULTS

Using these existing frameworks, there is considerable potential to similarly catalogue the effect of many common intoxicants known to negatively affect driving ability. This will provide safety-relevant and practical biological models for the development of next-generation multimodal DMS that integrate ocular and physiological variables sensitive to the effects of common and emergent psychoactive substances.

CONCLUSION

These devices have tangible potential application across all areas of transportation, including aviation, rail, and all commercial and private vehicle systems.

Measuring inter- and intra-individual differences in visual scan patterns in a driving simulator experiment using active information storage

Scan pattern analysis has been discussed as a promising tool in the context of real-time gaze-based applications. In particular, information-theoretic measures of scan path predictability, such as the gaze transition entropy (GTE), have been proposed for detecting relevant changes in user state or task demand. These measures model scan patterns as first-order Markov chains, assuming that only the location of the previous fixation is predictive of the next fixation in time. However, this assumption may not be sufficient in general, as recent research has shown that scan patterns may also exhibit more long-range temporal correlations. Thus, we here evaluate the active information storage (AIS) as a novel information-theoretic approach to quantifying scan path predictability in a dynamic task. In contrast to the GTE, the AIS provides means to statistically test and account for temporal correlations in scan path data beyond the previous last fixation. We compare AIS to GTE in a driving simulator experiment, in which participants drove in a highway scenario, where trials were defined based on an experimental manipulation that encouraged the driver to start an overtaking maneuver. Two levels of difficulty were realized by varying the time left to complete the task. We found that individual observers indeed showed temporal correlations beyond a single past fixation and that the length of the correlation varied between observers. No effect of task difficulty was observed on scan path predictability for either AIS or GTE, but we found a significant increase in predictability during overtaking. Importantly, for participants for which the first-order Markov chain assumption did not hold, this was only shown using AIS but not GTE. We conclude that accounting for longer time horizons in scan paths in a personalized fashion is beneficial for interpreting gaze pattern in dynamic tasks.

Quantifying the Predictability of Visual Scanpaths Using Active Information Storage

Entropy-based measures are an important tool for studying human gaze behavior under various conditions. In particular, gaze transition entropy (GTE) is a popular method to quantify the predictability of a visual scanpath as the entropy of transitions between fixations and has been shown to correlate with changes in task demand or changes in observer state. Measuring scanpath predictability is thus a promising approach to identifying viewers' cognitive states in behavioral experiments or gaze-based applications. However, GTE does not account for temporal dependencies beyond two consecutive fixations and may thus underestimate the actual predictability of the current fixation given past gaze behavior. Instead, we propose to quantify scanpath predictability by estimating the active information storage (AIS), which can account for dependencies spanning multiple fixations. AIS is calculated as the mutual information between a processes' multivariate past state and its next value. It is thus able to measure how much information a sequence of past fixations provides about the next fixation, hence covering a longer temporal horizon. Applying the proposed approach, we were able to distinguish between induced observer states based on estimated AIS, providing first evidence that AIS may be used in the inference of user states to improve human-machine interaction.

Eye-head coordination and dynamic visual scanning as indicators of visuo-cognitive demands in driving simulator

Driving is an everyday task involving a complex interaction between visual and cognitive processes. As such, an increase in the cognitive and/or visual demands can lead to a mental overload which can be detrimental for driving safety. Compiling evidence suggest that eye and head movements are relevant indicators of visuo-cognitive demands and attention allocation. This study aims to investigate the effects of visual degradation on eye-head coordination as well as visual scanning behavior during a highly demanding task in a driving simulator. A total of 21 emmetropic participants (21 to 34 years old) performed dual-task driving in which they were asked to maintain a constant speed on a highway while completing a visual search and detection task on a navigation device. Participants did the experiment with optimal vision and with contact lenses that introduced a visual perturbation (myopic defocus). The results indicate modifications of eye-head coordination and the dynamics of visual scanning in response to the visual perturbation induced. More specifically, the head was more involved in horizontal gaze shifts when the visual needs were not met. Furthermore, the evaluation of visual scanning dynamics, based on time-based entropy which measures the complexity and randomness of scanpaths, revealed that eye and gaze movements became less explorative and more stereotyped when vision was not optimal. These results provide evidence for a reorganization of both eye and head movements in response to increasing visual-cognitive demands during a driving task. Altogether, these findings suggest that eye and head movements can provide relevant information about visuo-cognitive demands associated with complex tasks. Ultimately, eye-head coordination and visual scanning dynamics may be good candidates to estimate drivers' workload and better characterize risky driving behavior.

Amphetamine-induced alteration to gaze parameters: A novel conceptual pathway and implications for naturalistic behavior

Amphetamine produces a multiplicity of well-documented end-order biochemical, pharmacological and biobehavioural effects. Mechanistically, amphetamine downregulates presynaptic and postsynaptic striatal monoamine (primarily dopaminergic) systems, producing alterations to key brain regions which manifest as stereotyped ridged behaviour which occurs under both acute and chronic dosing schedules and persists beyond detoxification. Despite evidence of amphetamine-induced visual attentional dysfunction, no conceptual synthesis has yet captured how characteristic pharmaco-behavioural processes are critically implicated via these pathways, nor described the potential implications for safety-sensitive behaviours. Drawing on known pathomechanisms, we propose a cross-disciplinary, novel conceptual functional system framework for delineating the biobehavioural consequences of amphetamine use on visual attentional capacity and discuss the implications for functional and behavioural outcomes. Specifically, we highlight the manifest implications for behaviours that are conceptually driven and highly dependent on visual information processing for timely execution of visually-guided movements. Following this, we highlight the potential impact on safety-sensitive, but common behaviours, such as driving a motor vehicle. The close pathophysiological relationship between oculomotor control and higher-order cognitive processes further suggests that dynamic measurement of movement related to the motion of the eye (gaze behaviour) may be a simple, effective and direct measure of behavioural performance capabilities in naturalistic settings. Consequently, we discuss the potential efficacy of ocular monitoring for the detection and monitoring of driver states for this drug user group, and potential wider application. Significance statement: We propose a novel biochemical-physiological-behavioural pathway which delineates how amphetamine use critically alters oculomotor function, visual-attentional performance and information processing capabilities. Given the manifest implications for behaviours that are conceptually driven and highly dependent on these processes, we recommend oculography as a novel means of detecting and monitoring gaze behaviours during naturalistic tasks such as driving. Real-word examination of gaze behaviour therefore present as an effective means to detect driver impairment and prevent performance degradation due to these drugs.

Behavioral Activity Recognition Based on Gaze Ethograms

Noninvasive behavior observation techniques allow more natural human behavior assessment experiments with higher ecological validity. We propose the use of gaze ethograms in the context of user interaction with a computer display to characterize the user's behavioral activity. A gaze ethogram is a time sequence of the screen regions the user is looking at. It can be used for the behavioral modeling of the user. Given a rough partition of the display space, we are able to extract gaze ethograms that allow discrimination of three common user behavioral activities: reading a text, viewing a video clip, and writing a text. A gaze tracking system is used to build the gaze ethogram. User behavioral activity is modeled by a classifier of gaze ethograms able to recognize the user activity after training. Conventional commercial gaze tracking for research in the neurosciences and psychology science are expensive and intrusive, sometimes impose wearing uncomfortable appliances. For the purposes of our behavioral research, we have developed an open source gaze tracking system that runs on conventional laptop computers using their low quality cameras. Some of the gaze tracking pipeline elements have been borrowed from the open source community. However, we have developed innovative solutions to some of the key issues that arise in the gaze tracker. Specifically, we have proposed texture-based eye features that are quite robust to low quality images. These features are the input for a classifier predicting the screen target area, the user is looking at. We report comparative results of several classifier architectures carried out in order to select the classifier to be used to extract the gaze ethograms for our behavioral research. We perform another classifier selection at the level of ethogram classification. Finally, we report encouraging results of user behavioral activity recognition experiments carried out over an inhouse dataset.