Enhancing the effectiveness of human-robot teaming with a closed-loop system

An Adaptive Human-Robotic Interaction Architecture for Augmenting Surgery Performance Using Real-Time Workload Sensing-Demonstration of a Semi-autonomous Suction Tool

OBJECTIVE

This study developed and evaluated a mental workload-based adaptive automation (MWL-AA) that monitors surgeon cognitive load and assist during cognitively demanding tasks and assists surgeons in robotic-assisted surgery (RAS).

BACKGROUND

The introduction of RAS makes operators overwhelmed. The need for precise, continuous assessment of human mental workload (MWL) states is important to identify when the interventions should be delivered to moderate operators' MWL.

METHOD

The MWL-AA presented in this study was a semi-autonomous suction tool. The first experiment recruited ten participants to perform surgical tasks under different MWL levels. The physiological responses were captured and used to develop a real-time multi-sensing model for MWL detection. The second experiment evaluated the effectiveness of the MWL-AA, where nine brand-new surgical trainees performed the surgical task with and without the MWL-AA. Mixed effect models were used to compare task performance, objective- and subjective-measured MWL.

RESULTS

The proposed system predicted high MWL hemorrhage conditions with an accuracy of 77.9%. For the MWL-AA evaluation, the surgeons' gaze behaviors and brain activities suggested lower perceived MWL with MWL-AA than without. This was further supported by lower self-reported MWL and better task performance in the task condition with MWL-AA.

CONCLUSION

A MWL-AA systems can reduce surgeons' workload and improve performance in a high-stress hemorrhaging scenario. Findings highlight the potential of utilizing MWL-AA to enhance the collaboration between the autonomous system and surgeons. Developing a robust and personalized MWL-AA is the first step that can be used do develop additional use cases in future studies.

APPLICATION

The proposed framework can be expanded and applied to more complex environments to improve human-robot collaboration.

Nonlinear dynamics of EEG responses to unmanned vehicle visual detection with different levels of task difficulty

The main objective of this study was to examine the presence of chaos in the EEG recordings of brain activity under simulated unmanned ground vehicle visual detection scenarios with different levels of task difficulty. One hundred and fifty people participated in the experiment and completed four visual detection task scenarios: (1) change detection, (2) a threat detection task, (3) a dual-task with different change detection task rates, and (4) a dual-task with different threat detection task rates. We used the largest Lyapunov exponent and correlation dimension of the EEG data and performed 0-1 tests on the EEG data. The results revealed a change in the level of nonlinearity in the EEG data corresponding to different levels of cognitive task difficulty. The differences in EEG nonlinearity measures among the studied levels of task difficulty, as well as between a single task scenario and a dual-task scenario, have also been assessed. The results increase our understanding of the nature of unmanned systems' operational requirements.

Development and evaluation of design guidelines for cognitive ergonomics in human-robot collaborative assembly systems

Industry 4.0 is the concept used to summarize the ongoing fourth industrial revolution, which is profoundly changing the manufacturing systems and business models all over the world. Collaborative robotics is one of the most promising technologies of Industry 4.0. Human-robot interaction and human-robot collaboration will be crucial for enhancing the operator's work conditions and production performance. In this regard, this enabling technology opens new possibilities but also new challenges. There is no doubt that safety is of primary importance when humans and robots interact in industrial settings. Nevertheless, human factors and cognitive ergonomics (i.e. cognitive workload, usability, trust, acceptance, stress, frustration, perceived enjoyment) are crucial, even if they are often underestimated or ignored. Therefore, this work refers to cognitive ergonomics in the design of human-robot collaborative assembly systems. A set of design guidelines has been developed according to the analysis of the scientific literature. Their effectiveness has been evaluated through multiple experiments based on a laboratory case study where different participants interacted with a low-payload collaborative robotic system for the joint assembly of a manufacturing product. The main assumption to be tested is that it is possible to improve the operator's experience and efficiency by manipulating the system features and interaction patterns according to the proposed design guidelines. Results confirmed that participants improved their cognitive response to human-robot interaction as well as the assembly performance with the enhancement of workstation features and interaction conditions by implementing an increasing number of guidelines.

Influence of task decision autonomy on physical ergonomics and robot performances in an industrial human–robot collaboration scenario

Adoption of human–robot collaboration is hindered by barriers in collaborative task design. A new approach for solving these problems is to empower operators in the design of their tasks. However, how this approach may affect user welfare or performance in industrial scenarios has not yet been studied. Therefore, in this research, the results of an experiment designed to identify the influences of the operator’s self-designed task on physical ergonomics and task performance are presented. At first, a collaborative framework able to accept operator task definition via parts’ locations and monitor the operator’s posture is presented. Second, the framework is used to tailor a collaborative experience favoring decision autonomy using the SHOP4CF architecture. Finally, the framework is used to investigate how this personalization influences collaboration through a user study with untrained personnel on physical ergonomics. The results from this study are twofold. On one hand, a high degree of decision autonomy was felt by the operators when they were allowed to allocate the parts. On the other hand, high decision autonomy was not found to vary task efficiency nor the MSD risk level. Therefore, this study emphasizes that allowing operators to choose the position of the parts may help task acceptance and does not vary operators’ physical ergonomics or task efficiency. Unfortunately, the test was limited to 16 participants and the measured risk level was medium. Therefore, this study also stresses that operators should be allowed to choose their own work parameters, but some guidelines should be followed to further reduce MSD risk levels.

Let's Work Together: A Meta-Analysis on Robot Design Features That Enable Successful Human-Robot Interaction at Work

OBJECTIVE

This meta-analysis reviews robot design features of interface, controller, and appearance and statistically summarizes their effect on successful human-robot interaction (HRI) at work (that is, task performance, cooperation, satisfaction, acceptance, trust, mental workload, and situation awareness).

BACKGROUND

Robots are becoming an integral part of many workplaces. As interactions with employees increase, ensuring success becomes ever more vital. Even though many studies investigated robot design features, an overview on general and specific effects is missing.

METHOD

Systematic selection of literature and structured coding led to 81 included experimental studies containing 380 effect sizes. Mean effects were calculated using a three-level meta-analysis to handle dependencies of multiple effect sizes in one study.

RESULTS

Sufficient feedback through the interface, clear visibility of affordances, and adaptability and autonomy of the controller significantly affect successful HRI, whereas appearance does not. The features of the interface and controller affect performance and satisfaction but do not affect situation awareness and trust. Specific effects of adaptability on cooperation and acceptance, as well as autonomy on mental workload, could be shown.

CONCLUSION

Robot design at work needs to cover multiple features of interface and controller to achieve successful HRI that covers not only performance and satisfaction, but also cooperation, acceptance, and mental workload. More empirical research is needed to investigate mediating mechanisms and underrepresented design features' effects.

APPLICATION

Robot designers should carefully choose design features to balance specific effects and implementation costs with regard to tasks, work design aims, and employee needs in the specific work context.

Cross-Task Consistency of Electroencephalography-Based Mental Workload Indicators: Comparisons Between Power Spectral Density and Task-Irrelevant Auditory Event-Related Potentials

Mental workload (MWL) estimators based on ongoing electroencephalography (EEG) and event-related potentials (ERPs) have shown great potentials to build adaptive aiding systems for human-machine systems by estimating MWL in real time. However, extracting EEG features which are consistent in indicating MWL across different tasks is still one of the critical challenges. This study attempts to compare the cross-task consistency in indexing MWL variations between two commonly used EEG-based MWL indicators, power spectral density (PSD) of ongoing EEG and task-irrelevant auditory ERPs (tir-aERPs). The verbal N-back and the multi-attribute task battery (MATB), both with two difficulty levels, were employed in the experiment, along with task-irrelevant auditory probes. EEG was recorded from 17 subjects when they were performing the tasks. The tir-aERPs elicited by the auditory probes and the relative PSDs of ongoing EEG between two consecutive auditory probes were extracted and statistically analyzed to reveal the effects of MWL and task type. Discriminant analysis and support vector machine were employed to examine the generalization of tir-aERP and PSD features in indexing MWL variations across different tasks. The results showed that the amplitudes of tir-aERP components, N1, early P3a, late P3a, and the reorienting negativity, significantly decreased with the increasing MWL in both N-back and MATB. Task type had no obvious influence on the amplitudes and topological layout of the MWL-sensitive tir-aERP features. The relative PSDs in θ, α, and low β bands were also sensitive to MWL variations. However, the MWL-sensitive PSD features and their topological patterns were significantly affected by task type. The cross-task classification results based on tir-aERP features also significantly outperformed the PSD features. These results suggest that the tir-aERPs should be potentially more consistent MWL indicators across very different task types when compared to PSD. The current study may provide new insights to our understanding of the common and distinctive neuropsychological essences of MWL across different tasks.

Ergonomic intervention on a packing workstation with robotic aid -case study at a furniture manufacturing industry

BACKGROUND

Several risk factors among packing lines workers can lead to Work-related Musculoskeletal Disorders (WRMSD) occurrence. Foreseeing WRMSD prevention and productivity increase, some furniture manufacturing industries have been investing in the adoption of robotic solutions. In this field, ergonomics plays an important role to verify if automation implementation has been successful.

OBJECTIVE

This study aims to address the general impact and effectiveness from an ergonomics point of view of the implementation of a robotic aid in a packing workstation.

METHODS

The Nordic Musculoskeletal Questionnaire (NMQ) was applied to 14 workers of semi-automated packing lines. Some additional questions about occupational conditions were included. In order to assess the ergonomic impact of the robotic aid, Rapid Upper Limb Assessment (RULA) was also applied by trained ergonomists, by analyzing the considered packing workstations before and after the adoption of the robotic aid proposed solution.

RESULTS

The results showed that trunk torsion was the most highlighted WRMSD risk factor by all workers, associating it with the lumbar pain. The obtained RULA scores demonstrated that the adoption of a robotic aid eliminated this risk factor and, consequently, reduced the corresponding WRMSD risk.

CONCLUSIONS

The adoption of robotic aids can be instrumental in reducing WRMSD risk in furniture manufacturing industries. Ergonomic studies with workers' participatory approaches seem to be an appropriate strategy to enable the validation and development of industrial robotic solutions.

Workload and Performance: Associations, Insensitivities, and Dissociations

OBJECTIVE

The aim of this study was to distill and define those influences under which change in objective performance level and the linked cognitive workload reflections of subjective experience and physiological variation either associate, dissociate, or are insensitive, one to another.

BACKGROUND

Human factors/ergonomics frequently employs users' self-reports of their own conscious experience, as well as their physiological reactivity, to augment the understanding of changing performance capacity. Under some circumstances, these latter workload responses are the only available assessment information to hand. How such perceptions and physiological responses match, fail to match, or are insensitive to the change in primary-task performance can prove critical to operational success. The reasons underlying these associations, dissociations, and insensitivities are central to the success of future effective human-machine interaction.

METHOD

Using extant research on the relations between differing methods of workload assessment, factors influencing their association, dissociation, and insensitivity are identified.

RESULTS

Dissociations and insensitivities occur more frequently than extant explanatory theories imply. Methodological and conceptual reasons for these patterns of incongruity are identified and evaluated.

APPLICATION

We often seek convergence of results in order to provide coherent explanations as bases for future prediction and practical design implementation. Identifying and understanding the causes as to why different reflections of workload diverge can help practitioners toward operational success.