A head mounted augmented reality design practice for maintenance assembly: Toward meeting perceptual and cognitive needs of AR users

Head Mounted Display (HMD) based Augmented Reality (AR) is being increasingly used in manufacturing and maintenance. However, limited research has been done to understand user interaction with AR interfaces, which may lead to poor usability, risk of occupational hazards, and low acceptance of AR systems. This paper uses a theoretically-driven approach to interaction design to investigate the impact of different AR modalities in terms of information mode (i.e. video vs. 3D animation) and interaction modality (i.e. hand-gesture vs. voice command) on user performance, workload, eye gaze behaviours, and usability during a maintenance assembly task. The results show that different information modes have distinct impacts compared to paper-based maintenance, in particular, 3D animation led to a 14% improvement over the video instructions in task completion time. Moreover, insights from eye gaze behaviours such as number of fixations and transition between Areas of Interest (AOIs) revealed the differences in attention switching and task comprehension difficulty with the choice of AR modalities. While, subjective user perceptions highlight some ergonomic issues such as misguidance and overreliance, which must be considered and addressed from the joint cognitive systems' (JCSs) perspective and in line with the predictions derived from the Multiple Resources Model.

Assessing the Role of Gaze Tracking in Optimizing Humans-In-The-Loop Telerobotic Operation Using Multimodal Feedback

A key challenge in achieving effective robot teleoperation is minimizing teleoperators’ cognitive workload and fatigue. We set out to investigate the extent to which gaze tracking data can reveal how teleoperators interact with a system. In this study, we present an analysis of gaze tracking, captured as participants completed a multi-stage task: grasping and emptying the contents of a jar into a container. The task was repeated with different combinations of visual, haptic, and verbal feedback. Our aim was to determine if teleoperation workload can be inferred by combining the gaze duration, fixation count, task completion time, and complexity of robot motion (measured as the sum of robot joint steps) at different stages of the task. Visual information of the robot workspace was captured using four cameras, positioned to capture the robot workspace from different angles. These camera views (aerial, right, eye-level, and left) were displayed through four quadrants (top-left, top-right, bottom-left, and bottom-right quadrants) of participants’ video feedback computer screen, respectively. We found that the gaze duration and the fixation count were highly dependent on the stage of the task and the feedback scenario utilized. The results revealed that combining feedback modalities reduced the cognitive workload (inferred by investigating the correlation between gaze duration, fixation count, task completion time, success or failure of task completion, and robot gripper trajectories), particularly in the task stages that require more precision. There was a significant positive correlation between gaze duration and complexity of robot joint movements. Participants’ gaze outside the areas of interest (distractions) was not influenced by feedback scenarios. A learning effect was observed in the use of the controller for all participants as they repeated the task with different feedback combination scenarios. To design a system for teleoperation, applicable in healthcare, we found that the analysis of teleoperators’ gaze can help understand how teleoperators interact with the system, hence making it possible to develop the system from the teleoperators’ stand point.

Dual Arm Co-Manipulation Architecture with Enhanced Human-Robot Communication for Large Part Manipulation

The emergence of collaborative robotics has had a great impact on the development of robotic solutions for cooperative tasks nowadays carried out by humans, especially in industrial environments where robots can act as assistants to operators. Even so, the coordinated manipulation of large parts between robots and humans gives rise to many technical challenges, ranging from the coordination of both robotic arms to the human-robot information exchange. This paper presents a novel architecture for the execution of trajectory driven collaborative tasks, combining impedance control and trajectory coordination in the control loop, as well as adding mechanisms to provide effective robot-to-human feedback for a successful and satisfactory task completion. The obtained results demonstrate the validity of the proposed architecture as well as its suitability for the implementation of collaborative robotic systems.

Effects of an Unexpected and Expected Event on Older Adults' Autonomic Arousal and Eye Fixations During Autonomous Driving

Driving cessation for some older adults can exacerbate physical, cognitive, and mental health challenges due to loss of independence and social isolation. Fully autonomous vehicles may offer an alternative transport solution, increasing social contact and encouraging independence. However, there are gaps in understanding the impact of older adults’ passive role on safe human–vehicle interaction, and on their well-being. 37 older adults (mean age ± SD = 68.35 ± 8.49 years) participated in an experiment where they experienced fully autonomous journeys consisting of a distinct stop (an unexpected event versus an expected event). The autonomous behavior of the vehicle was achieved using the Wizard of Oz approach. Subjective ratings of trust and reliability, and driver state monitoring including visual attention strategies (fixation duration and count) and physiological arousal (skin conductance and heart rate), were captured during the journeys. Results revealed that subjective trust and reliability ratings were high after journeys for both types of events. During an unexpected stop, overt visual attention was allocated toward the event, whereas during an expected stop, visual attention was directed toward the human–machine interface (HMI) and distributed across the central and peripheral driving environment. Elevated skin conductance level reflecting increased arousal persisted only after the unexpected event. These results suggest that safety-critical events occurring during passive fully automated driving may narrow visual attention and elevate arousal mechanisms. To improve in-vehicle user experience for older adults, a driver state monitoring system could examine such psychophysiological indices to evaluate functional state and well-being. This information could then be used to make informed decisions on vehicle behavior and offer reassurance during elevated arousal during unexpected events.

Left Amygdala and Putamen Activation Modulate Emotion Driven Decisions in the Iterated Prisoner's Dilemma Game

Although economic decision-making is commonly characterized as a purely rational phenomenon, it is clear that real-world decision-making is influenced by emotions. Yet, relatively little is known about the neural correlates of this process. To explore this issue, 20 participants underwent fMRI scanning while engaged in the Prisoner's Dilemma game under partner-directed sympathy, anger and neutral emotion conditions. Participants were most and least likely to cooperate after sympathy and anger induction, respectively, with the neutral condition eliciting intermediate cooperation rates. Moreover, the sympathy condition elicited quicker responses for cooperation than defection choices, whereas this pattern was reversed in the anger and neutral conditions. Left amygdala activation showed a positive correlation with cooperation rates and self-reports of partner directed sympathy in the sympathy condition. In the anger condition, left putamen activation was positively correlated with cooperation rates and negatively correlated with self-reports of partner directed anger strength. These findings indicate that while the left amygdala activation may be indicative of emotion enhancement and increase of cooperative behavior, the left putamen may help to suppress an emotion to overcome anger and engage in cooperation.

Differential roles of polar orbital prefrontal cortex and parietal lobes in logical reasoning with neutral and negative emotional content

To answer the question of how brain pathology affects reasoning about negative emotional content, we administered a disjunctive logical reasoning task involving arguments with neutral content (e.g. Either there are tigers or women in NYC, but not both; There are no tigers in NYC; There are women in NYC) and emotionally laden content (e.g. Either there are pedophiles or politicians in Texas, but not both; There are politicians in Texas; There are no pedophiles in Texas) to 92 neurological patients with focal lesions to various parts of the brain. A Voxel Lesion Symptom Mapping (VLSM) analysis identified 16 patients, all with lesions to the orbital polar prefrontal cortex (BA 10 & 11), as being selectively impaired in the emotional reasoning condition. Another 17 patients, all with lesions to the parietal cortex, were identified as being impaired in the neutral content condition. The reasoning scores of these two patient groups, along with 23 matched normal controls, underwent additional analysis to explore the effect of belief bias. This analysis revealed that the differences identified above were largely driven by trials where there was an incongruency between the believability of the conclusion and the validity of the argument (i.e. valid argument/false conclusion or invalid argument/true conclusion). Patients with lesions to polar orbital prefrontal cortex underperformed in incongruent emotional content trials and over performed in incongruent neutral content trials (compared to both normal controls and patients with parietal lobe lesions). Patients with lesions to parietal lobes underperformed normal controls (at a trend level) in neutral trials where there was a congruency between the believability of the conclusion and the validity of the argument (i.e. valid argument/true conclusion or invalid argument/false conclusion). We conclude that lesions to the polar orbital prefrontal cortex (i) prevent these patients from enjoying any emotionally induced cognitive boost, and (ii) block the belief bias processing route in the neutral condition. Lesions to parietal lobes result in a generalized impairment in logical reasoning with neutral content.

•

Polar/orbital PFC lesions result in impaired reasoning in emotional content reasoning.

•

Polar/orbital PFC lesion patients overperform in neutral content reasoning trials.

•

These differences were driven by incongruent reasoning trials.

•

Parietal lesions result in impaired reasoning with neutral but not emotional content.

Fear, Anger, and Risk Preference Reversals: An Experimental Study on a Chinese Sample

Fear and anger are basic emotions of the same valence which differ in terms of their certainty and control dimensions according to the Appraisal Tendency Framework, a theory addressing the relationship between specific emotions, and judgments and choices. Past research based on the Appraisal Theory revealed contradictory results for risky choice decision-making. However, these conclusions were drawn from Western samples (e.g., North American). Considering potential cultural differences, the present study aims to investigate whether the Appraisal Tendency hypothesis yields the same results in a Chinese sample. Our first study explores how dispositional fear and anger influence risk preferences through a classic virtual “Asia Disease Problem” task and the second study investigates how induced fear and anger influence risk preferences through an incentive-compatible task. Consistent with previous research, our results reveal that induced fear and anger have differential effects on risky decisions: angry participants prefer the risk-seeking option, whereas fearful participants prefer a risk-averse option. However, we find no associations between dispositional fear (or anger) and risky decisions.

The effect of partner-directed emotion in social exchange decision-making

Despite the prevalence of studies examining economic decision-making as a purely rational phenomenon, common sense suggests that emotions affect our decision-making particularly in a social context. To explore the influence of emotions on economic decision-making, we manipulated opponent-directed emotions prior to engaging participants in two social exchange decision-making games (the Trust Game and the Prisoner's Dilemma). Participants played both games with three different (fictional) partners and their tendency to defect was measured. Prior to playing each game, participants exchanged handwritten "essays" with their partners, and subsequently exchanged evaluations of each essay. The essays and evaluations, read by the participant, were designed to induce either anger, sympathy, or a neutral emotional response toward the confederate with whom they would then play the social exchange games. Galvanic skin conductance level (SCL) showed enhanced physiological arousal during anger induction compared to both the neutral and sympathy conditions. In both social exchange games, participants were most likely to defect against their partner after anger induction and least likely to defect after sympathy induction, with the neutral condition eliciting intermediate defection rates. This pattern was found to be strongest in participants exhibiting low cognitive control (as measured by a Go/no-Go task). The findings indicate that emotions felt toward another individual alter how one chooses to interact with them, and that this influence depends both on the specific emotion induced and the cognitive control of the individual.