Comparison of proactive and reactive interaction modes in a mobile robotic telecare study

Mobile robotic telepresence systems require that information about the environment, the task, and the robot be presented to a remotely located user (operator) who controls the robot for a specific task. In this study, two interaction modes, proactive and reactive, that differ in the way the user receives information from the robot, were compared in an experimental system simulating a healthcare setting. The users controlled a mobile telepresence robot that delivered and received items (medication, food, or drink), and also obtained metrics (vital signs) from a simulated patient while the users performed a secondary healthcare-related task (they compiled health records which were displayed to them on the screen and answered related questions). The effect of the two interaction modes on overall performance and user perception was evaluated through a within-participant study design conducted with 50 participants belonging to two different types of populations (with and without a technological background). Efficiency, effectiveness, understanding, satisfaction, and situation awareness were defined as the dependent variables measured both objectively and subjectively. The proactive mode increased user performance, and understanding of the system and reduced the workload compared to the reactive mode. However, several of the users valued the option of increased user control experienced in the reactive mode. We, therefore, proposed design suggestions to highlight some of the benefits of factoring the reactive mode into the design as a hybrid mode.

Assimilation of socially assistive robots' by older adults: an interplay of uses, constraints and outcomes

By supporting autonomy, aging in place, and wellbeing in later life, Socially Assistive Robots are expected to help humanity face the challenges posed by the rapid aging of the world's population. For the successful acceptance and assimilation of SARs by older adults, it is necessary to understand the factors affecting their Quality Evaluations Previous studies examining Human-Robot Interaction in later life indicated that three aspects shape older adults' overall QEs of robots: uses, constraints, and outcomes. However, studies were usually limited in duration, focused on acceptance rather than assimilation, and typically explored only one aspect of the interaction. In the present study, we examined uses, constraints, and outcomes simultaneously and over a long period. Nineteen community-dwelling older adults aged 75-97 were given a SAR for physical training for 6 weeks. Their experiences were documented via in-depth interviews conducted before and after the study period, short weekly telephone surveys, and reports produced by the robots. Analysis revealed two distinct groups: (A) The 'Fans' - participants who enjoyed using the SAR, attributed added value to it, and experienced a successful assimilation process; and (B) The 'Skeptics' - participants who did not like it, negatively evaluated its use, and experienced a disappointing assimilation process. Despite the vast differences between the groups, both reported more positive evaluations of SARs at the end of the study than before it began. Overall, the results indicated that the process of SARs' assimilation is not homogeneous and provided a profound understanding of the factors shaping older adults' QE of SARs following actual use. Additionally, the findings demonstrated the theoretical and practical usefulness of a holistic approach in researching older SARs users.

Automatic Root Length Estimation from Images Acquired In Situ without Segmentation

Image-based root phenotyping technologies, including the minirhizotron (MR), have expanded our understanding of the in situ root responses to changing environmental conditions. The conventional manual methods used to analyze MR images are time-consuming, limiting their implementation. This study presents an adaptation of our previously developed convolutional neural network-based models to estimate the total (cumulative) root length (TRL) per MR image without requiring segmentation. Training data were derived from manual annotations in Rootfly, commonly used software for MR image analysis. We compared TRL estimation with 2 models, a regression-based model and a detection-based model that detects the annotated points along the roots. Notably, the detection-based model can assist in examining human annotations by providing a visual inspection of roots in MR images. The models were trained and tested with 4,015 images acquired using 2 MR system types (manual and automated) and from 4 crop species (corn, pepper, melon, and tomato) grown under various abiotic stresses. These datasets are made publicly available as part of this publication. The coefficients of determination (R2), between the measurements made using Rootfly and the suggested TRL estimation models were 0.929 to 0.986 for the main datasets, demonstrating that this tool is accurate and robust. Additional analyses were conducted to examine the effects of (a) the data acquisition system and thus the image quality on the models' performance, (b) automated differentiation between images with and without roots, and (c) the use of the transfer learning technique. These approaches can support precision agriculture by providing real-time root growth information.

Evaluating levels of automation with different feedback modes in an assistive robotic table clearing task for eldercare

This paper focuses on how the autonomy level of an assistive robot that offers support for older adults in a daily task and its feedback affect the interaction. Identifying the level of automation (LOA) that prioritizes older adults' preferences while avoiding passiveness and sedentariness is challenging. The feedback mode should match the cognitive and perceptual capabilities of older adults and the LOA. We characterized three LOAs and paired them with two modes of feedback in a human-robot collaborative task. Twenty-seven older adults participated in evaluating the LOA-feedback variations in a mixed experimental design, utilizing an experimental setup of an assistive robot in a table clearing task. The quality of the interaction was evaluated with objective and subjective measures. The combination of high LOA with voice feedback improved the overall interaction when compared to other LOA and feedback combinations. This study emphasizes the importance of appropriate coupling of LOA and feedback for successful interaction of the older adults with an assistive robot.

Levels of Automation for a Mobile Robot Teleoperated by a Caregiver

Caregivers in eldercare can benefit from telepresence robots that allow them to perform a variety of tasks remotely. In order for such robots to be operated effectively and efficiently by non-technical users, it is important to examine if and how the robotic system’s level of automation (LOA) impacts their performance. The objective of this work was to develop suitable LOA modes for a mobile robotic telepresence (MRP) system for eldercare and assess their influence on users’ performance, workload, awareness of the environment, and usability at two different levels of task complexity. For this purpose, two LOA modes were implemented on the MRP platform: assisted teleoperation (low LOA mode) and autonomous navigation (high LOA mode). The system was evaluated in a user study with 20 participants, who, in the role of the caregiver, navigated the robot through a home-like environment to perform control and perception tasks. Results revealed that performance improved in the high LOA when task complexity was low. However, when task complexity increased, lower LOA improved performance. This opposite trend was also observed in the results for workload and situation awareness. We discuss the results in terms of the LOAs’ impact on users’ attitude towards automation and implications on usability.

Human Preferences for Robot Eye Gaze in Human-to-Robot Handovers

This paper investigates human’s preferences for a robot’s eye gaze behavior during human-to-robot handovers. We studied gaze patterns for all three phases of the handover process: reach, transfer, and retreat, as opposed to previous work which only focused on the reaching phase. Additionally, we investigated whether the object’s size or fragility or the human’s posture affect the human’s preferences for the robot gaze. A public data-set of human-human handovers was analyzed to obtain the most frequent gaze behaviors that human receivers perform. These were then used to program the robot’s receiver gaze behaviors. In two sets of user studies (video and in-person), a collaborative robot exhibited these gaze behaviors while receiving an object from a human. In the video studies, 72 participants watched and compared videos of handovers between a human actor and a robot demonstrating each of the three gaze behaviors. In the in-person studies, a different set of 72 participants physically performed object handovers with the robot and evaluated their perception of the handovers for the robot’s different gaze behaviors. Results showed that, for both observers and participants in a handover, when the robot exhibited Face-Hand-Face gaze (gazing at the giver’s face and then at the giver’s hand during the reach phase and back at the giver’s face during the retreat phase), participants considered the handover to be more likable, anthropomorphic, and communicative of timing \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$(p < 0.0001)$$\end{document}(p<0.0001). However, we did not find evidence of any effect of the object’s size or fragility or the giver’s posture on the gaze preference.

Robotic System for Physical Training of Older Adults

Physical exercise has many physical, psychological and social health benefits leading to improved life quality. This paper presents a robotic system developed as a personal coach for older adults aiming to motivate older adults to participate in physical activities. The robot instructs the participants, demonstrates the exercises and provides real-time corrective and positive feedback according to the participant’s performance as monitored by an RGB-D camera. Two robotic systems based on two different humanoid robots (Nao, toy-like and Poppy, mechanical-like) were developed and implemented using the Python programming language. Experimental studies with 32 older adults were conducted, to determine the preferable mode and timing of the feedback provided to the user to accommodate user preferences, motivate the users and improve their interaction with the system. Additionally, user preferences with regards to the two different humanoid robots used were explored. The results revealed that the system motivated the older adults to engage more in physical exercises. The type and timing of feedback influenced this engagement. Most of these older adults also perceived the system as very useful, easy to use, had a positive attitude towards the system and noted their intention to use it. Most users preferred the more mechanical looking robot (Poppy) over the toy-like robot (Nao).

Viewpoint Analysis for Maturity Classification of Sweet Peppers

The effect of camera viewpoint and fruit orientation on the performance of a sweet pepper maturity level classification algorithm was evaluated. Image datasets of sweet peppers harvested from a commercial greenhouse were collected using two different methods, resulting in 789 RGB—Red Green Blue (images acquired in a photocell) and 417 RGB-D—Red Green Blue-Depth (images acquired by a robotic arm in the laboratory), which are published as part of this paper. Maturity level classification was performed using a random forest algorithm. Classifications of maturity level from different camera viewpoints, using a combination of viewpoints, and different fruit orientations on the plant were evaluated and compared to manual classification. Results revealed that: (1) the bottom viewpoint is the best single viewpoint for maturity level classification accuracy; (2) information from two viewpoints increases the classification by 25 and 15 percent compared to a single viewpoint for red and yellow peppers, respectively, and (3) classification performance is highly dependent on the fruit’s orientation on the plant.

Robotic gaming prototype for upper limb exercise: Effects of age and embodiment on user preferences and movement

Background:

Effective human-robot interactions in rehabilitation necessitates an understanding of how these should be tailored to the needs of the human. We report on a robotic system developed as a partner on a 3-D everyday task, using a gamified approach.

Objectives:

To: (1) design and test a prototype system, to be ultimately used for upper-limb rehabilitation; (2) evaluate how age affects the response to such a robotic system; and (3) identify whether the robot’s physical embodiment is an important aspect in motivating users to complete a set of repetitive tasks.

Methods:

62 healthy participants, young (<30 yo) and old (>60 yo), played a 3D tic-tac-toe game against an embodied (a robotic arm) and a non-embodied (a computer-controlled lighting system) partner. To win, participants had to place three cups in sequence on a physical 3D grid. Cup picking-and-placing was chosen as a functional task that is often practiced in post-stroke rehabilitation. Movement of the participants was recorded using a Kinect camera.

Results:

The timing of the participants’ movement was primed by the response time of the system: participants moved slower when playing with the slower embodied system (p = 0.006). The majority of participants preferred the robot over the computer-controlled system. Slower response time of the robot compared to the computer-controlled one only affected the young group’s motivation to continue playing.

Conclusion:

We demonstrated the feasibility of the system to encourage the performance of repetitive 3D functional movements, and track these movements. Young and old participants preferred to interact with the robot, compared with the non-embodied system. We contribute to the growing knowledge concerning personalized human-robot interactions by (1) demonstrating the priming of the human movement by the robotic movement – an important design feature, and (2) identifying response-speed as a design variable, the importance of which depends on the age of the user.

SIT LESS: A prototype home-based system for monitoring older adults sedentary behavior

This paper presents the overall design of a prototype home-based system aimed to reduce sedentary behavior of older adults. Quantitative performance indicators were developed to measure the sedentary behavior and daily activities of an older adult. The sedentary behavior is monitored by identifying individual positions (standing, sitting, and lying) within the field of view of a Microsoft Kinect sensor, using a custom designed algorithm. The physical activity of the older adult when outside the field of view of the Microsoft Kinect sensor is monitored by counting the number of steps using a Fitbit Charge HR watch, which the older adult is equipped with. A user interface was developed on a PC platform to interact with the older adult. The user interface is automatically operated and includes several modules. It displays the activity level, and provides feedbacks, alerts, and reminders to reduce sedentary behavior. Evaluations using a mixed methods approach that included a focus group, interviews, and observations were conducted to examine the integrated system, evaluate the users' experience with the system, and compare different types of feedbacks and alerts. The analyses indicated the feasibility of the proposed SIT LESS system along with recommendations for improving the system in future research.

HRI usability evaluation of interaction modes for a teleoperated agricultural robotic sprayer

Teleoperation of an agricultural robotic system requires effective and efficient human-robot interaction. This paper investigates the usability of different interaction modes for agricultural robot teleoperation. Specifically, we examined the overall influence of two types of output devices (PC screen, head mounted display), two types of peripheral vision support mechanisms (single view, multiple views), and two types of control input devices (PC keyboard, PS3 gamepad) on observed and perceived usability of a teleoperated agricultural sprayer. A modular user interface for teleoperating an agricultural robot sprayer was constructed and field-tested. Evaluation included eight interaction modes: the different combinations of the 3 factors. Thirty representative participants used each interaction mode to navigate the robot along a vineyard and spray grape clusters based on a 2 × 2 × 2 repeated measures experimental design. Objective metrics of the effectiveness and efficiency of the human-robot collaboration were collected. Participants also completed questionnaires related to their user experience with the system in each interaction mode. Results show that the most important factor for human-robot interface usability is the number and placement of views. The type of robot control input device was also a significant factor in certain dependents, whereas the effect of the screen output type was only significant on the participants' perceived workload index. Specific recommendations for mobile field robot teleoperation to improve HRI awareness for the agricultural spraying task are presented.

Comparison of three different techniques for camera and motion control of a teleoperated robot

This research aims to evaluate new methods for robot motion control and camera orientation control through the operator's head orientation in robot teleoperation tasks. Specifically, the use of head-tracking in a non-invasive way, without immersive virtual reality devices was combined and compared with classical control modes for robot movements and camera control. Three control conditions were tested: 1) a condition with classical joystick control of both the movements of the robot and the robot camera, 2) a condition where the robot movements were controlled by a joystick and the robot camera was controlled by the user head orientation, and 3) a condition where the movements of the robot were controlled by hand gestures and the robot camera was controlled by the user head orientation. Performance, workload metrics and their evolution as the participants gained experience with the system were evaluated in a series of experiments: for each participant, the metrics were recorded during four successive similar trials. Results shows that the concept of robot camera control by user head orientation has the potential of improving the intuitiveness of robot teleoperation interfaces, specifically for novice users. However, more development is needed to reach a margin of progression comparable to a classical joystick interface.

Statistical Evaluation Method for Comparing Grid Map Based Sensor Fusion Algorithms

In this paper we present a method for evaluating sensor fusion algorithms based on a quantitative comparison, which is independent of the data acquired and the sensors used. The sensor fusion performance measures and performance analysis procedure provide a basis for modeling, analyzing, experimenting, and comparing different sensor fusion algorithms. The capability to compare different algorithms creates a ranking basis, making it possible to select the best algorithm. The statistical evaluation method defines the experimental design and statistical analysis. The numbers of experiments and repetitions required are derived from the statistical characteristics and the desired confidence level. Since procedures are defined to ensure that the experiments are indeed conducted differently, the results are not specific for either the evaluated test cases or the sensor characteristics. The statistical analysis provides a systematic method for comparing sensor fusion algorithms. Although this method requires experimentation, it offers the ability to compare actual performances in the real world. Quantitative procedures are developed to ensure that specific environmental conditions evaluated do not influence the evaluation. To demonstrate the statistical evaluation method it is applied to a case study that compared five different sensor fusion algorithms in a mobile robot experiment.

Distance-Dependent Multimodal Image Registration for Agriculture Tasks

Image registration is the process of aligning two or more images of the same scene taken at different times; from different viewpoints; and/or by different sensors. This research focuses on developing a practical method for automatic image registration for agricultural systems that use multimodal sensory systems and operate in natural environments. While not limited to any particular modalities; here we focus on systems with visual and thermal sensory inputs. Our approach is based on pre-calibrating a distance-dependent transformation matrix (DDTM) between the sensors; and representing it in a compact way by regressing the distance-dependent coefficients as distance-dependent functions. The DDTM is measured by calculating a projective transformation matrix for varying distances between the sensors and possible targets. To do so we designed a unique experimental setup including unique Artificial Control Points (ACPs) and their detection algorithms for the two sensors. We demonstrate the utility of our approach using different experiments and evaluation criteria.

Relation between perceived effort and the electromyographic signal in localized low-effort activities

Hand-based human–machine interfaces are complex tasks that involve repetitive or sustained movements and postures of the hands that can lead to overuse syndromes of the musculoskeletal system. Consequently, it is important to minimize the physical effort that occurs at these interfaces. The evaluation of physical effort can be performed either by subjective evaluation of the relative perceived effort (e.g., Borg scale) or by objective physiological measurements (e.g., electromyography – EMG). However, the relation between these two measures has not been sufficiently studied for localized low-effort activities. This study investigated the relation between EMG and Borg ratings, as well as the issue of gender differences during low-effort activity of forearm muscles. Nine females and nine males performed eight different hand gestures (localized low-effort activity), during which EMG signals were recorded from six forearm muscles and Borg ratings were obtained. On average, the female subjects rated the gestures as less effortful than the male subjects, and also demonstrated a higher positive correlation between the EMG and Borg ratings. Furthermore, the linear model that was fitted for predicting the Borg ratings based on gender and the combined activity of muscles provided an R-squared value of approximately 0.3.

Advanced methods for displays and remote control of robots

An in-depth evaluation of the usability and situation awareness performance of different displays and destination controls of robots are presented. In two experiments we evaluate the way information is presented to the operator and assess different means for controlling the robot. Our study compares three types of displays: a "blocks" display, a HUD (head-up display), and a radar display, and two types of controls: touch screen and hand gestures. The HUD demonstrated better performance when compared to the blocks display and was perceived to have greater usability compared to the radar display. The HUD was also found to be more useful when the operation of the robot was more difficult, i.e., when using the hand-gesture method. The experiments also pointed to the importance of using a wide viewing angle to minimize distortion and for easier coping with the difficulties of locating objects in the field of view margins. The touch screen was found to be superior in terms of both objective performance and its perceived usability. No differences were found between the displays and the controllers in terms of situation awareness. This research sheds light on the preferred display type and controlling method for operating robots from a distance, making it easier to cope with the challenges of operating such systems.

A gesture-based tool for sterile browsing of radiology images

The use of doctor-computer interaction devices in the operation room (OR) requires new modalities that support medical imaging manipulation while allowing doctors' hands to remain sterile, supporting their focus of attention, and providing fast response times. This paper presents "Gestix," a vision-based hand gesture capture and recognition system that interprets in real-time the user's gestures for navigation and manipulation of images in an electronic medical record (EMR) database. Navigation and other gestures are translated to commands based on their temporal trajectories, through video capture. "Gestix" was tested during a brain biopsy procedure. In the in vivo experiment, this interface prevented the surgeon's focus shift and change of location while achieving a rapid intuitive reaction and easy interaction. Data from two usability tests provide insights and implications regarding human-computer interaction based on nonverbal conversational modalities.

Predicting Feed Intake of the Individual Dairy Cow

The voluntary feed intake of the dairy cow is an important variable in dairy operation but is impossible to measure individually when cows are kept in groups or grazing. Existing formulas that calculate dry matter intake (DMI) from ration and performance variables are not applicable to an individual cow for online decision-making, such as daily ration density adjustment by computerized feeders in a milking robot. This led to a new DMI modeling approach of using only animal factors that are measurable online on an individual basis. In 1997 we published a small-scale attempt of this approach using milk yield (MY) and body weight (BW). In 2001, this approach was adopted by the National Research Council (NRC), using 4% fat-corrected milk rather than MY together with BW and time after calving. In the present study, we increased the number of cows. The model is a multiple regression, where the descriptive variables are the interrelation MY/BW, daily BW change, and milk fat including the effect of previous 2 d. The coefficients are calculated on daily basis, i.e., each day has its own coefficients. Our model differs from that of the NRC by: 1) the descriptive variable, 2) using daily coefficients to deal with the ever-changing physiological state of lactation, and 3) considering previous performance. Two data sets (60 cows together) acquired in 2 intervals of the Volcani Center herd were used to calibrate (18 cows) and test (42 cows) the model. Model validity was statistically tested, compared to that of the NRC, and was not rejected with 99.5% confidence.

Regularity of dairy cow feeding behavior with computer-controlled feeders

The regularity with which cows visit a computerized concentrate feeding station was investigated under two fixed-time feeding routines with four and six equal diurnal feeding intervals (feeding windows). The experiment was conducted using an Israeli commercial dairy herd, and space was sufficient for cow traffic related to feeding. In both trials, a regular diurnal pattern of feeding in accordance with programmed feed availability could be observed within feeding windows. Concentrates were consumed in the same feeding window in which they were allotted in 95 to 97% of the cases. Patterns of feeding events and visits to a feeding station within a feeding window were identical for both feeding routines investigated. Distributions of feeding intervals for individual cows point to a definite feeding order in the herd rather than a random sequence of the cows feeding within a feeding window. Number and timing of visits to the feeding station were quickly reoriented to the new system of feeding windows. Cow behavior was based on an understanding of and adaptation to the proposed feeding routines, rather than being a result of periodic visits to feeding station, regardless of the feeding program, thus indicating ability to control this behavior.