A Novel Gesture Recognition System for Intelligent Interaction with a Nursing-Care Assistant Robot

An empirical study on the collaborative usability of age-appropriate smart home interface design

Introduction

The smart home has become a popular product, but with the development of the aging population, the differentiated characteristics of the elderly smart home products in terms of demand and use are becoming more and more significant. The existing smart products are complicated to operate and cumbersome to interact with, which increases the cognitive load of the elderly group and hinders the daily use and user experience feeling of the elderly. The purpose of this paper is to study the interface data information and interface visual design starting from hardware and software, interface interaction, to explore the better interface data information and interface visual design, and to output, a new prototype of the operating interface of smart home system for the elderly, so that the smart products can be better used by the elderly.

Methods

Thirty-two participants aged 55-75 were invited to conduct the test, and subjective evaluation was conducted at the end of the test. Through the tests, the operability of the prototype structure for smart furniture systems for the elderly was demonstrated.

Results

In terms of functionality a new task based on a combination of icons and text is proposed. In the control of devices, the switching status of devices, etc., needs to be clearly distinguished visually, eye-protective bright colors are used, paired with low saturation to highlight the focus, and high bright colors with gray to distinguish the device status. In terms of the density of the content, an appropriate proportion of images and text were used to make the information less dense. ln the arrangement of web content, information content relevant to users was placed first as much as possible.

Discussion

Based on this, a secondary optimal design was carried out to improve the interactive design of the smart home for the elderly and output it as a prototype interactive interface. Thus, the operability, rationality, and aesthetic comfort of the prototype design of smart home interaction in an age-friendly scenario are improved, allowing the elderly to have a better experience when using the smart home.

Hybrid Target Selections by "Hand Gestures + Facial Expression" for a Rehabilitation Robot

In this study we propose a "hand gesture + face expression" human machine interaction technique, and apply this technique to bedridden rehabilitation robot. "Hand gesture + Facial expression" interactive technology combines the input mode of gesture and facial expression perception. It involves seven basic facial expressions that can be used to determine a target selecting task, while hand gestures are used to control a cursor's location. A controlled experiment was designed and conducted to evaluate the effectiveness of the proposed hybrid technology. A series of target selecting tasks with different target widths and layouts were designed to examine the recognition accuracy of hybrid control gestures. An interactive experiment applied to a rehabilitation robot is designed to verify the feasibility of this interactive technology applied to rehabilitation robots. The experimental results show that the "hand + facial expression" interactive gesture has strong robustness, which can provide a novel guideline for designing applications in VR interfaces, and it can be applied to the rehabilitation robots.

An Empirical Study on the Influence of Smart Home Interface Design on the Interaction Performance of the Elderly

The concept of the smart home has been widely recognized and accepted, but the differentiated characteristics of elderly smart products in terms of demand and use are becoming more and more prominent. The lack of an efficient navigation design of the smart product interface increases the cognitive burden of elderly users, and how to better meet the needs of the elderly with smart products gradually becomes the focus of attention. This study was conducted for the elderly group, using the scenario-based design method to analyze the needs of elderly users, combining the research results of scenario theory with the smart home interaction design research method, focusing on how to make the style of interface navigation, sliding layout and button size more suitable for the cognitive behavior of elderly users. The purpose of this research is to realize an age-friendly smart home interaction design in terms of functional design and interface design. The experiment is divided into two stages: in stage 1, two different layouts and operation methods are commonly used for the age-friendly smart home interface: up and down sliding and left and right sliding; in stage 2, the functional buttons are square, where 4 styles are selected, and the side lengths are set to 10 mm, 12 mm, 15 mm, 18 mm and 22 mm. The sliding and retrieval test and retrieval and click test results show that for different sliding layout methods, the interactive performance and subjective evaluation of the interface with the up-and-down sliding layout are better. Among all functional button styles, the interaction performance and subjective evaluation of the simple button style with lines are the best. Among the function keys with a size of 10–22 mm, the interaction performance is better from 12 mm to 18 mm. The conclusion of the better interface data information obtained from this experiment improves the rationality of the age-friendly smart home interface and makes the smart home interface better for the age-friendly scenario.

GuLiM: A Hybrid Motion Mapping Technique for Teleoperation of Medical Assistive Robot in Combating the COVID-19 Pandemic

Driven by the demand to largely mitigate nosocomial infection problems in combating the coronavirus disease 2019 (COVID-19) pandemic, the trend of developing technologies for teleoperation of medical assistive robots is emerging. However, traditional teleoperation of robots requires professional training and sophisticated manipulation, imposing a burden on healthcare workers, taking a long time to deploy, and conflicting the urgent demand for a timely and effective response to the pandemic. This paper presents a novel motion synchronization method enabled by the hybrid mapping technique of hand gesture and upper-limb motion (GuLiM). It tackles a limitation that the existing motion mapping scheme has to be customized according to the kinematic configuration of operators. The operator awakes the robot from any initial pose state without extra calibration procedure, thereby reducing operational complexity and relieving unnecessary pre-training, making it user-friendly for healthcare workers to master teleoperation skills. Experimenting with robotic grasping tasks verifies the outperformance of the proposed GuLiM method compared with the traditional direct mapping method. Moreover, a field investigation of GuLiM illustrates its potential for the teleoperation of medical assistive robots in the isolation ward as the Second Body of healthcare workers for telehealthcare, avoiding exposure of healthcare workers to the COVID-19.

Human Motion State Recognition Based on Flexible, Wearable Capacitive Pressure Sensors

Human motion state recognition technology based on flexible, wearable sensor devices has been widely applied in the fields of human-computer interaction and health monitoring. In this study, a new type of flexible capacitive pressure sensor is designed and applied to the recognition of human motion state. The electrode layers use multi-walled carbon nanotubes (MWCNTs) as conductive materials, and polydimethylsiloxane (PDMS) with microstructures is embedded in the surface as a flexible substrate. A composite film of barium titanate (BaTiO3) with a high dielectric constant and low dielectric loss and PDMS is used as the intermediate dielectric layer. The sensor has the advantages of high sensitivity (2.39 kPa-1), wide pressure range (0-120 kPa), low pressure resolution (6.8 Pa), fast response time (16 ms), fast recovery time (8 ms), lower hysteresis, and stability. The human body motion state recognition system is designed based on a multi-layer back propagation neural network, which can collect, process, and recognize the sensor signals of different motion states (sitting, standing, walking, and running). The results indicate that the overall recognition rate of the system for the human motion state reaches 94%. This proves the feasibility of the human motion state recognition system based on the flexible wearable sensor. Furthermore, the system has high application potential in the field of wearable motion detection.

A Gesture-Controlled Rehabilitation Robot to Improve Engagement and Quantify Movement Performance

Rehabilitation requires repetitive and coordinated movements for effective treatment, which are contingent on patient compliance and motivation. However, the monotony, intensity, and expense of most therapy routines do not promote engagement. Gesture-controlled rehabilitation has the potential to quantify performance and provide engaging, cost-effective treatment, leading to better compliance and mobility. We present the design and testing of a gesture-controlled rehabilitation robot (GC-Rebot) to assess its potential for monitoring user performance and providing entertainment while conducting physical therapy. Healthy participants (n = 11) completed a maze with GC-Rebot for six trials. User performance was evaluated through quantitative metrics of movement quality and quantity, and participants rated the system usability with a validated survey. For participants with self-reported video-game experience (n = 10), wrist active range of motion across trials (mean ± standard deviation) was 41.6 ± 13° and 76.8 ± 16° for pitch and roll, respectively. In the course of conducting a single trial with a time duration of 68.3 ± 19 s, these participants performed 27 ± 8 full wrist motion repetitions (i.e., flexion/extension), with a dose-rate of 24.2 ± 5 reps/min. These participants also rated system usability as excellent (score: 86.3 ± 12). Gesture-controlled therapy using the GC-Rebot demonstrated the potential to be an evidence-based rehabilitation tool based on excellent user ratings and the ability to monitor at-home compliance and performance.

Teleoperation of Collaborative Robot for Remote Dementia Care in Home Environments

As a senile chronic, progressive and currently incurable disease, dementia has an enormous impact on society and life quality of the elderly. The development of teleoperation technology has changed the traditional way of care delivery and brought a variety of novel applications for dementia care. In this paper, a telerobotic system is presented which gives the caregivers the capability of assisting dementia elderly remotely. The proposed system is composed of a dual-arm collaborative robot (YuMi) and a wearable motion capture device. The communication architecture is achieved by the robot operation system (ROS). The position-orientation data of the operator's hand are obtained and used to control the YuMi robot. Besides, a path-constrained mapping method is designed for motion trajectory tracking between the robot and the operator in the progress of teleoperation. Meanwhile, corresponding experiments are conducted to verify the performance of the trajectory tracking using the path-constrained mapping method. Results show that the position tracking deviation between the trajectory of the operator and the robot measured by dynamic time warping distance is 1.05 mm at the sampling frequency of 7.5 Hz. Moreover, the practicability of the proposed system was verified by teleoperating the YuMi robot to pick up a medicine bottle and further demonstrated by assisting an elderly woman in picking up a cup remotely. The proposed telerobotic system has potential utility for improving the life quality of dementia elderly and the care effect of their caregivers.

Homecare Robotic Systems for Healthcare 4.0: Visions and Enabling Technologies

Powered by the technologies that have originated from manufacturing, the fourth revolution of healthcare technologies is happening (Healthcare 4.0). As an example of such revolution, new generation homecare robotic systems (HRS) based on the cyber-physical systems (CPS) with higher speed and more intelligent execution are emerging. In this article, the new visions and features of the CPS-based HRS are proposed. The latest progress in related enabling technologies is reviewed, including artificial intelligence, sensing fundamentals, materials and machines, cloud computing and communication, as well as motion capture and mapping. Finally, the future perspectives of the CPS-based HRS and the technical challenges faced in each technical area are discussed.

A Low-Cost Soft Robotic Hand Exoskeleton for Use in Therapy of Limited Hand–Motor Function

We present the design and validation of a low-cost, customizable and 3D-printed anthropomorphic soft robotic hand exoskeleton for rehabilitation of hand injuries using remotely administered physical therapy regimens. The design builds upon previous work done on cable actuated exoskeleton designs by implementing the same kinematic functionality, but with the focus shifted to ease of assembly and cost effectiveness as to allow patients and physicians to manufacture and assemble the hardware necessary to implement treatment. The exoskeleton was constructed solely from 3D-printed and widely available off-the-shelf components. Control of the actuators was realized using an Arduino microcontroller, with a custom-designed shield to facilitate ease of wiring. Tests were conducted to verify that the range of motion of the digits and the forces exerted at the fingertip coincided with those of a healthy human hand.

Navigating a Service Robot for Indoor Complex Environments

This paper investigates the use of an autonomous service robot in an indoor complex environment, such as a hospital ward or a retirement home. This type of service robot not only needs to plan and find paths around obstacles, but must also interact with caregivers or patients. This study presents a type of service robot that combines the image from a 3D depth camera with infrared sensors, and the inputs from multiple sonar sensors in an Adaptive Neuro-Fuzzy Inference System (ANFIS)-based approach in path planning. In personal contacts, facial features are used to perform person recognition in order to discriminate between staff, patients, or a stranger. In the case of staff, the service robot can perform a follow-me function if requested. The robot can also use an additional feature which is to classify the person’s gender. The purpose of facial and gender recognition includes helping to present choices for suitable destinations to the user. Experiments were done in cramped but open spaces, as well as confined passages scenarios, and in almost all cases, the autonomous robots were able to reach their destinations.