Psychometric Properties of NASA-TLX and Index of Cognitive Activity as Measures of Cognitive Workload in Older Adults

Validity and Impact of Methods for Collecting Training Data for Myoelectric Prosthetic Control Algorithms

Intuitive regression control of prostheses relies on training algorithms to correlate biological recordings to motor intent. The quality of the training dataset is critical to run-time regression performance, but accurately labeling intended hand kinematics after hand amputation is challenging. In this study, we quantified the accuracy and precision of labeling hand kinematics using two common training paradigms: 1) mimic training, where participants mimic predetermined motions of a prosthesis, and 2) mirror training, where participants mirror their contralateral intact hand during synchronized bilateral movements. We first explored this question in healthy non-amputee individuals where the ground-truth kinematics could be readily determined using motion capture. Kinematic data showed that mimic training fails to account for biomechanical coupling and temporal changes in hand posture. Additionally, mirror training exhibited significantly higher accuracy and precision in labeling hand kinematics. These findings suggest that the mirror training approach generates a more faithful, albeit more complex, dataset. Accordingly, mirror training resulted in significantly better offline regression performance when using a large amount of training data and a non-linear neural network. Next, we explored these different training paradigms online, with a cohort of unilateral transradial amputees actively controlling a prosthesis in real-time to complete a functional task. Overall, we found that mirror training resulted in significantly faster task completion speeds and similar subjective workload. These results demonstrate that mirror training can potentially provide more dexterous control through the utilization of task-specific, user-selected training data. Consequently, these findings serve as a valuable guide for the next generation of myoelectric and neuroprostheses leveraging machine learning to provide more dexterous and intuitive control.

Quantifying patient experiences with therapeutic neurorehabilitation technologies: a scoping review

PURPOSE

Neurorehabilitation technologies are a novel approach to providing rehabilitation for patients with neurological conditions. There is a need to explore patient experiences. This study aimed; 1) To identify available questionnaires that assess patients' experiences with neurorehabilitation technologies, and 2) where reported, to document the psychometric properties of the identified questionnaires.

MATERIALS AND METHODS

Four databases were searched (Medline, Embase, Emcare and PsycInfo). The inclusion criteria were all types of primary data collection that included neurological patients of all ages who had experienced therapy with neurorehabilitation technologies and completed questionnaires to assess these experiences.

RESULTS

Eighty-eight publications were included. Fifteen different questionnaires along with many self-developed scales were identified. These were categorised as; 1) self-developed tools, 2) specific questionnaire for a particular technology, and 3) generic questionnaires originally developed for a different purpose. The questionnaires were used to assess various technologies, including virtual reality, robotics, and gaming systems. Most studies did not report any psychometric properties.

CONCLUSION

Many tools have been used to evaluate patient experiences, but few were specifically developed for neurorehabilitation technologies and psychometric data was limited. A preliminary recommendation would be use of the User Satisfaction Evaluation Questionnaire to evaluate patient experience with virtual reality systems.

Comparison of two pupillometric measures as indicators of cognitive strain and under the influence of screen luminance

The pupil diameter has been shown to provide insight to a person's experienced cognitive strain. Pupillary light responses, however, make this measure unreliable in uncontrolled settings. Two derived indicators-Index of Cognitive Activity (ICA) and Index of Pupillary Activity (IPA)-aim to 'eliminate' lighting influences, changing based only on the perceived cognitive strain. The IPA potentially offers a valuable alternative to the ICA through its fully transparent calculation, which lifts the restrictions to proprietary software and supported eye trackers. The measures are examined and compared based on two experimental studies; (i) as indicators of cognitive strain during mental arithmetic tasks and (ii) under different conditions of computer screen luminance. Results indicate that neither indicator differentiates between the increasing levels of cognitive strain. Differences in screen luminance are reflected in both indicators, although differently between the conditions. Both results contradict the claims of the indicators and further investigations are thus required.

A Combined Assessment Tool of Teamwork, Communication, and Workload in Hospital Procedural Units

Teamwork, communication, and workload issues continue to contribute to patient safety events. The authors developed a diagnostic mixed methods toolkit combining a behavior observation tool, semistructured interview guide, and surveys to proactively identify relevant gaps. Applied across 14 units at three hospitals, this toolkit yielded 344 findings with 156 associated recommendations and took, on average, four days of observation. On a scale from 1 (not at all helpful) to 6 (substantially helpful), leaders indicated that the assessment and its recommendations were very helpful (median 5, interquartile range 5-6, 34 survey respondents, 47.9% individual-level response rate, 85.7% unit-level response rate). Integrating this tool into a broader safety strategy can help inform organizational improvement efforts.

Structured Multidisciplinary work Evaluation Tool (SMET) questionnaire: Translation, cultural adaptation and psychometric evaluation of the Persian version

BACKGROUND

To assess relevant environmental conditions in any work-setting requires a multidisciplinary perspective that is practical, valid, and reliable. This includes the physical, environmental, and psychosocial risk-factors. The Structured Multidisciplinary work Evaluation Tool (SMET) questionnaire simultaneously considers multiple work-related demands.

OBJECTIVE

This study translated and culturally adapted the SMET into Persian and evaluated its psychometric properties in Persian industrial workers.

METHODS

Cross-sectional translation and cross-cultural adaptation in five standardized phases. A convenience sample (n = 211) recruited from an industrial-occupation setting completed: the SMET; and the Rapid Upper Limb Assessment (RULA), and National Aeronautics and Space Administration Task Load Index (NASA-TLX) criteria; plus light and noise levels were concurrently assessed. Psychometric properties included: validity, with face (from confirmed language clarity, simplicity, and readability), content (via the content validity index, CVI, for equivalency and relevancy), criterion (through Pearson's r correlation with relevant criteria), and construct (through known group validity between participants with/without work-related musculoskeletal disorders, WMSDs); internal consistency (Cronbach's α); and test-retest reliability (intraclass correlation coefficient, ICC2.1).

RESULTS

Validity was confirmed with: face through the adaptation; content from suitable CVI values for items (CVI range = 0.78-1.0) and scale-total (CVI = 0.86); criterion from SMET associations with levels for light (r = - 0.42) and noise (r = 0.21), plus RULA (r = 0.42) and NASA-TLX (r = 0.39); and construct through participants with WMSD having significantly higher SMET total-scores (p = 0.01). Internal consistency (α= 0.89) and reliability (ICC2.1 = 0.87) were acceptable and strong.

CONCLUSION

This study indicated that the SMET Persian version had acceptable psychometric properties in an industrial occupational setting. Further investigation in longitudinal populations is recommended.

The effects of distraction on younger drivers: A neurophysiological perspective

Distracted driving remains a significant cause of traffic accidents globally, including in Australia. However, many younger drivers still admit to using a phone while driving. A simulated driving study investigated the neurophysiological effects of visual, auditory, and higher-order cognitive (i.e., requiring the use of executive functions) distraction on young drivers. In total, 24 young adults aged 18-25 years completed four 8 min simulated driving sessions while concurrently engaging in various distractor tasks. Neurophysiological arousal was measured via EEG. Additionally, subjective workload and objective driving performance were assessed. Frontal beta and gamma power exhibited their highest levels during tasks involving higher-order cognitive and visual demands. The higher-order cognitive condition was rated as the most mentally demanding. In comparison, the visual condition had the most significant impact on both the standard deviation of speed and standard deviation of lateral positioning. This study has significant implications for all road users, particularly those aged 18-25 years, and it reinforces the importance of not using a phone while driving.

Standardized circuit class group training versus individualized goal-directed group training to improve upper limb function in stroke survivors during in-patient rehabilitation: a pragmatic trial

PURPOSE

To investigate the usability and effectiveness of standardized circuit class group training (CCT) compared to individualized goal-directed group training (GDT) in subacute stroke survivors.

MATERIALS AND METHODS

This study consists of three parts. Part 1 involved a pragmatic, non-randomized controlled trial with subacute participants and their therapists, who participated in four weeks either CCT or GDT. Superiority of the intervention was defined as significantly larger improvement on the Motor Activity Log for patients and lower workload for therapists. In Part 2, six additional workstations were developed for CCT. Part 3 replicated the study of Part 1 with the expanded CCT.

RESULTS

Part 1 showed no difference in effectiveness between training methods. CCT did not match the rehabilitation goals of the patient sufficiently, however mental workload seemed lower for therapists. An expansion of CCT could improve the match between the patient's goals and the training (Part 2). Results of Part 3 showed again no difference in effectiveness between methods. CCT was however perceived as less engaging compared to GDT, but mental load for therapists remained lower.

CONCLUSIONS

A standardized training could reduce the mental workload for therapists, but patients seemed less engaged. A combination of both might be most beneficial.Trial registration: Dutch Trial Register: NL8844 and NL9471IMPLICATIONS FOR REHABILITATIONProviding a standardized training program after stroke reduces therapists' mental workload.Individualized goal-directed group training results in the best achievement of rehabilitation goals.A combination of standardized and individual training would use best of both intervention modalities.Patients should be involved in the co-creation developing process of training programs.

Ergonomic Analysis of Otologic Surgery: Comparison of Endoscope and Microscope

OBJECTIVE

The comparative postural health of surgeons performing endoscopic and microscopic otologic surgeries has been a topic of active debate, with many nascent or anecdotal reports suggesting the latter encourages suboptimal ergonomics. Using inertial body sensors to measure joint angles, this study sought to objectively evaluate and compare the ergonomics of surgeons during endoscopic and microscopic otologic surgeries.

STUDY DESIGN

Prospective pilot trial.

SETTING

Large, multicenter, academic hospital system. Performed 21 otologic operations (10 endoscopic and 11 microscopic) in November 2020 and January 2021. All attendings were fellowship trained in otology/neurotology.

SUBJECTS

Eight otolaryngologists (four attendings and four residents) performing 21 otologic surgeries (11 microscopic and 10 endoscopic).

INTERVENTION

Approach to otologic surgery: endoscope or microscope.

MAIN OUTCOME MEASURES

Surgeons' neck and back angles while wearing ergonomic sensors affixed to either side of each major joint, mental and physical burdens and pain after each surgery (via modified NASA Task Load Index).

RESULTS

Residents' necks (9.54° microscopic vs. -4.79° endoscopic, p = 0.04) and backs (16.48° microscopic vs. 3.66° endoscopic, p = 0.01) were significantly more flexed when performing microscopic surgery than when performing endoscopic surgery, although attending neck and back flexion were comparable during microscopic and endoscopic surgeries. Attendings reported significantly higher pain levels after operating microscopically than after operating endoscopically (0.13 vs. 2.76, p = 0.01).

CONCLUSIONS

Residents were found to operate with significantly higher risk back and neck postures (as defined by the validated ergonomic tool, Rapid Entire Body Assessment) when operating microscopically. Attendings reported significantly higher levels of pain after operating microscopically versus endoscopically, suggesting that the suboptimal microscopic postures adopted earlier in training may pose an indelible risk later in a surgeon's career.

Effect of Cognitive Reserve on Physiological Measures of Cognitive Workload in Older Adults with Cognitive Impairments

BACKGROUND

Cognitive reserve may protect against cognitive decline.

OBJECTIVE

This cross-sectional study investigated the association between cognitive reserve and physiological measures of cognitive workload in older adults with cognitive impairment.

METHODS

29 older adults with cognitive impairment (age: 75±6, 11 (38%) women, MoCA: 20±7) and 19 with normal cognition (age: 74±6; 11 (58%) women; MoCA: 28±2) completed a working memory test of increasing task demand (0-, 1-, 2-back). Cognitive workload was indexed using amplitude and latency of the P3 event-related potential (ERP) at electrode sites Fz, Cz, and Pz, and changes in pupillary size, converted to an index of cognitive activity (ICA). The Cognitive Reserve Index questionnaire (CRIq) evaluated Education, Work Activity, and Leisure Time as a proxy of cognitive reserve. Linear mixed models evaluated the main effects of cognitive status, CRIq, and the interaction effect of CRIq by cognitive status on ERP and ICA.

RESULTS

The interaction effect of CRIq total score by cognitive status on P3 ERP and ICA was not significant. However, higher CRIq total scores were associated with lower ICA (p = 0.03). The interaction effects of CRIq subscores showed that Work Activity affected P3 amplitude (p = 0.03) and ICA (p = 0.03) differently between older adults with and without cognitive impairments. Similarly, Education affected ICA (p = 0.02) differently between the two groups. No associations were observed between CRIq and P3 latency.

CONCLUSION

Specific components of cognitive reserve affect cognitive workload and neural efficiency differently in older adults with and without cognitive impairments.

Did the general practice residents well adapt to real public health prevention --a study from the COVID-19 prevention training in China

OBJECT

GP residents's has the obligation to take task with the public health prevetion. GP residents receive the public health training during their college study period and the residents training. The sudden outbreak of the COVID-19 epidemic, highlight the importance and competence of the community prevention as the front line of epidemic prevention and control, pushing the general practitioner (GP) residents into the front team of epidemic prevention and control. Residents' participation in epidemic prevention and control is not only a field workload participation in public health disease prevention and control, but also a rare and value-oriented training experience. This study aims to explores the research on the training content, ability improvement and cognitive load of the resident, and to demonstrate past and future training effects of epidemic prevention and control.

METHODS

Object cognitive load scale (NASA-TLX scale) and self-developed questionnaires were adopted to conduct a questionnaire survey on resident doctors who were in GP training program from West China Hospital of Sichuan University, and finally 190 questionnaires were collected. SPSS 23.0 statistical software for statistical analysis of data.

RESULT

Most indicators of cognitive load NASA scale are at a "moderate" level by the GP residents, generally indicating that the intensity of on-site epidemic prevention and control (training) can be tolerated. The chi-square test is used to study the status of "How responsible you are for epidemic prevention and control in a community in the future", the residents grade shows no significant difference while " how many months after the outbreak when you participated in the epidemic prevention" shows significant difference, the result show that GP residents already have konwledge and skills for the public health prevetion, they need more attitude and mental preparation. Continuing education will have a significant positive relationship with the GP residents's confidence of the prevetion ofpublic health prevetion.

CONCLUSION

Former medical school education and training at the regulatory training stage have a good effect for residents to master the ability of epidemic prevention and control, and to prepare for the needs of epidemic prevention and control physically and mentally. After this stage, the epidemic prevention and control training under the real situation will make a great contribution to the self-assessment and performance improvement of the final general practitioners.

Development of a Vision-Guided Shared-Control System for Assistive Robotic Manipulators

Assistive robotic manipulators (ARMs) provide a potential solution to mitigating the difficulties and lost independence associated with manipulation deficits in individuals with upper-limb impairments. However, achieving efficient control of an ARM can be a challenge due to the multiple degrees of freedom (DoFs) of an ARM that need to be controlled. This study describes the development of a vision-guided shared-control (VGS) system and how it is applied to a multi-step drinking task. The VGS control allows the user to control the gross motion of the ARM via teleoperation and commands the ARM to autonomously perform fine manipulation. A bench-top test of the autonomous actions showed that success rates for different subtasks ranged from 80% to 100%. An evaluation with three test pilots showed that the overall task performance, in terms of success rate, task completion time, and joystick mode-switch frequency, was better with VGS than with teleoperation. Similar trends were observed with a case participant with a spinal cord injury. While his performance was better and he perceived a smaller workload with VGS, his perceived usability for VGS and teleoperation was similar. More work is needed to further improve and test VGS on participants with disabilities.

Development of a patient-centered preference tool for patients with hematologic malignancies: protocol for a mixed methods study (Preprint)

Background

The approval of novel therapies for patients diagnosed with hematologic malignancies have improved survival outcomes but increased the challenge of aligning chemotherapy choices with patient preferences. We previously developed paper versions of a discrete choice experiment (DCE) and a best-worst scaling (BWS) instrument to quantify the treatment outcome preferences of patients with hematologic malignancies to inform shared decision making.

Objective

We aim to develop an electronic health care tool (EHT) to guide clinical decision making that uses either a BWS or DCE instrument to capture patient preferences. The primary objective of this study is to use both qualitative and quantitative methods to evaluate the perceived usability, cognitive workload (CWL), and performance of electronic prototypes that include the DCE and BWS instrument.

Methods

This mixed methods study includes iterative co-design methods that will involve healthy volunteers, patient-caregiver pairs, and health care workers to evaluate the perceived usability, CWL, and performance of tasks within distinct prototypes. Think-aloud sessions and semistructured interviews will be conducted to collect qualitative data to develop an affinity diagram for thematic analysis. Validated assessments (Post-Study System Usability Questionnaire [PSSUQ] and the National Aeronautical and Space Administration’s Task Load Index [NASA-TLX]) will be used to evaluate the usability and CWL required to complete tasks within the prototypes. Performance assessments of the DCE and BWS will include the evaluation of tasks using the Single Easy Questionnaire (SEQ), time to complete using the prototype, and the number of errors. Additional qualitative assessments will be conducted to gather participants’ feedback on visualizations used in the Personalized Treatment Preferences Dashboard that provides a representation of user results after completing the choice tasks within the prototype.

Results

Ethical approval was obtained in June 2021 from the Institutional Review Board of the University of North Carolina at Chapel Hill. The DCE and BWS instruments were developed and incorporated into the PRIME (Preference Reporting to Improve Management and Experience) prototype in early 2021 and prototypes were completed by June 2021. Heuristic evaluations were conducted in phase 1 and completed by July 2021. Recruitment of healthy volunteers began in August 2021 and concluded in September 2021. In December 2021, our findings from phase 2 were accepted for publication. Phase 3 recruitment began in January 2022 and is expected to conclude in September 2022. The data analysis from phase 3 is expected to be completed by November 2022.

Conclusions

Our findings will help differentiate the usability, CWL, and performance of the DCE and BWS within the prototypes. These findings will contribute to the optimization of the prototypes, leading to the development of an EHT that helps facilitate shared decision making. This evaluation will inform the development of EHTs to be used clinically with patients and health care workers.

International Registered Report Identifier (IRRID)

DERR1-10.2196/39586

Augmented Reality and Intraoperative Navigation in Sinonasal Malignancies: A Preclinical Study

Objective

To report the first use of a novel projected augmented reality (AR) system in open sinonasal tumor resections in preclinical models and to compare the AR approach with an advanced intraoperative navigation (IN) system.

Methods

Four tumor models were created. Five head and neck surgeons participated in the study performing virtual osteotomies. Unguided, AR, IN, and AR + IN simulations were performed. Statistical comparisons between approaches were obtained. Intratumoral cut rate was the main outcome. The groups were also compared in terms of percentage of intratumoral, close, adequate, and excessive distances from the tumor. Information on a wearable gaze tracker headset and NASA Task Load Index questionnaire results were analyzed as well.

Results

A total of 335 cuts were simulated. Intratumoral cuts were observed in 20.7%, 9.4%, 1.2,% and 0% of the unguided, AR, IN, and AR + IN simulations, respectively (p < 0.0001). The AR was superior than the unguided approach in univariate and multivariate models. The percentage of time looking at the screen during the procedures was 55.5% for the unguided approaches and 0%, 78.5%, and 61.8% in AR, IN, and AR + IN, respectively (p < 0.001). The combined approach significantly reduced the screen time compared with the IN procedure alone.

Conclusion

We reported the use of a novel AR system for oncological resections in open sinonasal approaches, with improved margin delineation compared with unguided techniques. AR improved the gaze-toggling drawback of IN. Further refinements of the AR system are needed before translating our experience to clinical practice.

Special Issue: Neural Bases of Cognitive Processing

The main aim of Cognitive Neuroscience is investigating how brain functions lead to mental processes and behavior [...].

Ergonomic Analysis of Functional Endoscopic Sinus Surgery Using Novel Inertial Sensors

OBJECTIVES/HYPOTHESIS

Suboptimal ergonomics during endoscopic sinus surgery can lead to considerable physical discomfort and fatigue for the surgeon. The purpose of this pilot study is to objectively evaluate the ergonomic positions of trainee and attending surgeons while performing functional endoscopic sinus surgery (FESS).

STUDY DESIGN

Pilot prospective trial.

METHODS

Six surgeons (two attendings and four trainees) performed FESS while wearing 11 inertial measurement units (IMUs) affixed to either side of each major joint. Screen placement was standardized to be 1 m directly in front of the surgeon and on the patient's left, 0-15° declined from the surgeons' eyes. Bed height was standardized such that the workspace was 0 to 10 cm below the elbows. IMU data were analyzed to calculate joint angles. Ideal joint angles (i.e., <10° for neck and trunk) were determined by the validated Rapid Entire Body Assessment tool. Subjects subsequently completed a modified National Aeronautics and Space Administration Task Load Index to assess cognitive and physical burden and pain. Student's t-test was employed to detect differences between groups.

RESULTS

Trainees adopted positions involving significantly greater neck flexion (9.90° vs. -6.48°, P = .03) and reported significantly higher frustration levels (3.04 vs. 1.33, P = .02) while operating than attendings. For both cohorts, increased operative time was significantly correlated with greater back flexion (r = 0.90, P = .02; r = 0.55, P = .04, respectively).

CONCLUSIONS

Our data suggest that trainees operate with higher risk neck postures than do attendings. These data indicate high-risk operative postures may be borne of inexperience and present an opportunity for postural interventions at an early stage of training.

LEVEL OF EVIDENCE

N/A Laryngoscope, 2021.

Neuroergonomics: A Perspective from Neuropsychology, with a Proposal about Workload

In a brief overview of neuroergonomics, including some personal reminiscences of Raja Parasuraman, it is recognized that the field of human factors and ergonomics has benefitted greatly from the inclusion and integration of neuroscientific methods and theory. It is argued that such synergistic success can work in the other direction as well with the inclusion of methods and theory of human factors by a neuro field, in this case, neuropsychology. More specifically, it is proposed that neuropsychology can benefit from the inclusion of workload measures and theory. Preliminary studies on older adults, persons living with HIV, and patients with a traumatic brain injury or multiple sclerosis, are reviewed. As an adjunct measure to neuropsychological tests, the construct of workload seems perfectly suited to provide an additional vector of information on patient status, capturing some of the large individual differences evident in clinical populations and facilitating the early detection of cognitive change.