1
|
Hägglund S, Andtfolk M, Rosenberg S, Wingren M, Andersson S, Nyholm L. Do you wanna dance? Tales of trust and driving trust factors in robot medication counseling in the pharmacy context. Front Robot AI 2024; 11:1332110. [PMID: 39170902 PMCID: PMC11336249 DOI: 10.3389/frobt.2024.1332110] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/02/2023] [Accepted: 07/12/2024] [Indexed: 08/23/2024] Open
Abstract
Introduction: The sustainable implementation of socially assistive robots in a pharmacy setting requires that customers trust the robot. Our aim was to explore young adults' anticipations of and motives for trusting robot medication counseling in a high-stakes scenario. Methods: Through a co-creation approach, we co-designed a prototype application for the Furhat platform together with young adults. In-lab testing of a pharmacy scenario, where the robot provides medication counseling related to emergency contraceptive pills, was conducted to deepen our understanding of some factors driving young adults' initial trust establishment and anticipations of interacting with a robot in a high-stakes scenario. Qualitative data from interviews with six study participants were analyzed using inductive, reflexive thematic analysis and are presented through a narrative approach. Results: We outline five tales of trust characterized by personas. A continuum of different anticipations for consulting a robot in medication counseling is presented, ranging from low to high expectations of use. Driving factors in the initial trust establishment process are position, autonomy, boundaries, shame, gaze, and alignment. Discussion: The article adds to the understanding of the dimensions of the multifaceted trust concept, of driving trust factors, and of the subsequent anticipation to trust robots in a high-stakes pharmacy context.
Collapse
Affiliation(s)
- Susanne Hägglund
- Faculty of Education and Welfare Studies, Åbo Akademi University, Vaasa, Finland
| | - Malin Andtfolk
- Faculty of Education and Welfare Studies, Åbo Akademi University, Vaasa, Finland
| | - Sara Rosenberg
- Pharmaceutical Sciences Laboratory, Faculty of Science and Engineering, Åbo Akademi University, Turku, Finland
| | - Mattias Wingren
- Faculty of Education and Welfare Studies, Åbo Akademi University, Vaasa, Finland
| | - Sören Andersson
- Faculty of Education and Welfare Studies, Åbo Akademi University, Vaasa, Finland
| | - Linda Nyholm
- Department of Caring and Ethics, Faculty of Health Sciences, University of Stavanger, Stavanger, Norway
| |
Collapse
|
2
|
Szabo DA, Neagu N, Teodorescu S, Apostu M, Predescu C, Pârvu C, Veres C. The Role and Importance of Using Sensor-Based Devices in Medical Rehabilitation: A Literature Review on the New Therapeutic Approaches. SENSORS (BASEL, SWITZERLAND) 2023; 23:8950. [PMID: 37960649 PMCID: PMC10648494 DOI: 10.3390/s23218950] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 08/31/2023] [Revised: 10/22/2023] [Accepted: 10/31/2023] [Indexed: 11/15/2023]
Abstract
Due to the growth of sensor technology, more affordable integrated circuits, and connectivity technologies, the usage of wearable equipment and sensing devices for monitoring physical activities, whether for wellness, sports monitoring, or medical rehabilitation, has exploded. The current literature review was performed between October 2022 and February 2023 using PubMed, Web of Science, and Scopus in accordance with P.R.I.S.M.A. criteria. The screening phase resulted in the exclusion of 69 articles that did not fit the themes developed in all subchapters of the study, 41 articles that dealt exclusively with rehabilitation and orthopaedics, 28 articles whose abstracts were not visible, and 10 articles that dealt exclusively with other sensor-based devices and not medical ones; the inclusion phase resulted in the inclusion of 111 articles. Patients who utilise sensor-based devices have several advantages due to rehabilitating a missing component, which marks the accomplishment of a fundamental goal within the rehabilitation program. As technology moves faster and faster forward, the field of medical rehabilitation has to adapt to the time we live in by using technology and intelligent devices. This means changing every part of rehabilitation and finding the most valuable and helpful gadgets that can be used to regain lost functions, keep people healthy, or prevent diseases.
Collapse
Affiliation(s)
- Dan Alexandru Szabo
- Department of Human Movement Sciences, George Emil Palade University of Medicine, Pharmacy, Science, and Technology of Targu Mures, 540139 Targu Mures, Romania;
- Department ME1, Faculty of Medicine in English, George Emil Palade University of Medicine, Pharmacy, Science, and Technology of Targu Mures, 540139 Targu Mures, Romania
| | - Nicolae Neagu
- Department of Human Movement Sciences, George Emil Palade University of Medicine, Pharmacy, Science, and Technology of Targu Mures, 540139 Targu Mures, Romania;
| | - Silvia Teodorescu
- Department of Doctoral Studies, National University of Physical Education and Sports, 060057 Bucharest, Romania;
| | - Mihaela Apostu
- Department of Special Motor and Rehabilitation Medicine, National University of Physical Education and Sports, 060057 Bucharest, Romania; (M.A.); (C.P.)
| | - Corina Predescu
- Department of Special Motor and Rehabilitation Medicine, National University of Physical Education and Sports, 060057 Bucharest, Romania; (M.A.); (C.P.)
| | - Carmen Pârvu
- Faculty of Physical Education and Sports, “Dunărea de Jos” University, 63-65 Gării Street, 337347 Galati, Romania;
| | - Cristina Veres
- Department of Industrial Engineering and Management, University of Medicine, Pharmacy, Science and Technology of Targu Mures, 540142 Targu Mures, Romania;
| |
Collapse
|
3
|
Chiriatti G, Carbonari L, Ceravolo MG, Andrenelli E, Millevolte M, Palmieri G. A Robot-Assisted Framework for Rehabilitation Practices: Implementation and Experimental Results. SENSORS (BASEL, SWITZERLAND) 2023; 23:7652. [PMID: 37688108 PMCID: PMC10563072 DOI: 10.3390/s23177652] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 07/05/2023] [Revised: 08/07/2023] [Accepted: 08/31/2023] [Indexed: 09/10/2023]
Abstract
One of the most interesting characteristics of collaborative robots is their ability to be used in close cooperation scenarios. In industry, this facilitates the implementation of human-in-loop workflows. However, this feature can also be exploited in different fields, such as healthcare. In this paper, a rehabilitation framework for the upper limbs of neurological patients is presented, consisting of a collaborative robot that helps users perform three-dimensional trajectories. Such a practice is aimed at improving the coordination of patients by guiding their motions in a preferred direction. We present the mechatronic setup, along with a preliminary experimental set of results from 19 volunteers (patients and control subjects) who provided positive feedback on the training experience (52% of the subjects would return and 44% enjoyed performing the exercise). Patients were able to execute the exercise, with a maximum deviation from the trajectory of 16 mm. The muscular effort required was limited, with average maximum forces recorded at around 50 N.
Collapse
Affiliation(s)
- Giorgia Chiriatti
- Department of Industrial Engineering and Mathematical Sciences, Polytechnic University of Marche, 60131 Ancona, Italy; (G.C.); (G.P.)
| | - Luca Carbonari
- Department of Industrial Engineering and Mathematical Sciences, Polytechnic University of Marche, 60131 Ancona, Italy; (G.C.); (G.P.)
| | - Maria Gabriella Ceravolo
- Department of Experimental and Clinical Medicine, Polytechnic University of Marche, 60131 Ancona, Italy; (M.G.C.); (E.A.)
| | - Elisa Andrenelli
- Department of Experimental and Clinical Medicine, Polytechnic University of Marche, 60131 Ancona, Italy; (M.G.C.); (E.A.)
| | - Marzia Millevolte
- Neurorehabilitation Clinic, Ancona University Hospital, 60131 Ancona, Italy;
| | - Giacomo Palmieri
- Department of Industrial Engineering and Mathematical Sciences, Polytechnic University of Marche, 60131 Ancona, Italy; (G.C.); (G.P.)
| |
Collapse
|
4
|
Giansanti D. Synergizing Intelligence and Building a Smarter Future: Artificial Intelligence Meets Bioengineering. Bioengineering (Basel) 2023; 10:691. [PMID: 37370622 DOI: 10.3390/bioengineering10060691] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/23/2023] [Accepted: 06/01/2023] [Indexed: 06/29/2023] Open
Abstract
Smart Engineering (SE) describes the methods, processes, and IT tools for the interdisciplinary, system-oriented development of innovative, intelligent, networked products, production plants, and infrastructures [...].
Collapse
|
5
|
Zhu Y, Wang C, Li J, Zeng L, Zhang P. Effect of different modalities of artificial intelligence rehabilitation techniques on patients with upper limb dysfunction after stroke-A network meta-analysis of randomized controlled trials. Front Neurol 2023; 14:1125172. [PMID: 37139055 PMCID: PMC10150552 DOI: 10.3389/fneur.2023.1125172] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/16/2022] [Accepted: 03/21/2023] [Indexed: 05/05/2023] Open
Abstract
Background This study aimed to observe the effects of six different types of AI rehabilitation techniques (RR, IR, RT, RT + VR, VR and BCI) on upper limb shoulder-elbow and wrist motor function, overall upper limb function (grip, grasp, pinch and gross motor) and daily living ability in subjects with stroke. Direct and indirect comparisons were drawn to conclude which AI rehabilitation techniques were most effective in improving the above functions. Methods From establishment to 5 September 2022, we systematically searched PubMed, EMBASE, the Cochrane Library, Web of Science, CNKI, VIP and Wanfang. Only randomized controlled trials (RCTs) that met the inclusion criteria were included. The risk of bias in studies was evaluated using the Cochrane Collaborative Risk of Bias Assessment Tool. A cumulative ranking analysis by SUCRA was performed to compare the effectiveness of different AI rehabilitation techniques for patients with stroke and upper limb dysfunction. Results We included 101 publications involving 4,702 subjects. According to the results of the SUCRA curves, RT + VR (SUCRA = 84.8%, 74.1%, 99.6%) was most effective in improving FMA-UE-Distal, FMA-UE-Proximal and ARAT function for subjects with upper limb dysfunction and stroke, respectively. IR (SUCRA = 70.5%) ranked highest in improving FMA-UE-Total with upper limb motor function amongst subjects with stroke. The BCI (SUCRA = 73.6%) also had the most significant advantage in improving their MBI daily living ability. Conclusions The network meta-analysis (NMA) results and SUCRA rankings suggest RT + VR appears to have a greater advantage compared with other interventions in improving upper limb motor function amongst subjects with stroke in FMA-UE-Proximal and FMA-UE-Distal and ARAT. Similarly, IR had shown the most significant advantage over other interventions in improving the FMA-UE-Total upper limb motor function score of subjects with stroke. The BCI also had the most significant advantage in improving their MBI daily living ability. Future studies should consider and report on key patient characteristics, such as stroke severity, degree of upper limb impairment, and treatment intensity/frequency and duration. Systematic review registration www.crd.york.ac.uk/prospero/#recordDetail, identifier: CRD42022337776.
Collapse
Affiliation(s)
- Yu Zhu
- School of Sports Medicine and Rehabilitation, Beijing Sport University, Beijing, China
- Linfen Central Hospital, Linfen, Shanxi, China
| | - Chen Wang
- School of Sports Medicine and Rehabilitation, Beijing Sport University, Beijing, China
| | - Jin Li
- School of Sports Medicine and Rehabilitation, Beijing Sport University, Beijing, China
| | - Liqing Zeng
- School of Sports Medicine and Rehabilitation, Beijing Sport University, Beijing, China
| | - Peizhen Zhang
- School of Sports Medicine and Rehabilitation, Beijing Sport University, Beijing, China
- *Correspondence: Peizhen Zhang
| |
Collapse
|
6
|
Marcos-Pablos S, García-Peñalvo FJ. More than surgical tools: a systematic review of robots as didactic tools for the education of professionals in health sciences. ADVANCES IN HEALTH SCIENCES EDUCATION : THEORY AND PRACTICE 2022; 27:1139-1176. [PMID: 35771316 PMCID: PMC9244888 DOI: 10.1007/s10459-022-10118-6] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Figures] [Subscribe] [Scholar Register] [Received: 04/29/2021] [Accepted: 04/23/2022] [Indexed: 06/15/2023]
Abstract
Within the field of robots in medical education, most of the work done during the last years has focused on surgeon training in robotic surgery, practicing surgery procedures through simulators. Apart from surgical education, robots have also been widely employed in assistive and rehabilitation procedures, where education has traditionally focused in the patient. Therefore, there has been extensive review bibliography in the field of medical robotics focused on surgical and rehabilitation and assistive robots, but there is a lack of survey papers that explore the potential of robotics in the education of healthcare students and professionals beyond their training in the use of the robotic system. The scope of the current review are works in which robots are used as didactic tools for the education of professionals in health sciences, investigating the enablers and barriers that affect the use of robots as learning facilitators. Systematic literature searches were conducted in WOS and Scopus, yielding a total of 3812 candidate papers. After removing duplicates, inclusion criteria were defined and applied, resulting in 171 papers. An in-depth quality assessment was then performed leading to 26 papers for qualitative synthesis. Results show that robots in health sciences education are still developed with a roboticist mindset, without clearly incorporating aspects of the teaching/learning process. However, they have proven potential to be used in health sciences as they allow to parameterize procedures, autonomously guide learners to achieve greater engagement, or enable collective learning including patients and instructors "in the loop". Although there exist documented added-value benefits, further research and efforts needs to be done to foster the inclusion of robots as didactic tools in the curricula of health sciences professionals. On the one hand, by analyzing how robotic technology should be developed to become more flexible and usable to support both teaching and learning processes in health sciences education, as final users are not necessarily well-versed in how to use it. On the other, there continues to be a need to develop effective and standard robotic enhanced learning evaluation tools, as well good quality studies that describe effective evaluation of robotic enhanced education for professionals in health sciences. As happens with other technologies when applied to the health sciences field, studies often fail to provide sufficient detail to support transferability or direct future robotic health care education programs.
Collapse
Affiliation(s)
- Samuel Marcos-Pablos
- GRIAL Research Group, University of Salamanca, IUCE, Paseo de Canalejas 169, 37008 Salamanca, Spain
| | | |
Collapse
|
7
|
Zolfagharian A, Khosravani MR, Duong Vu H, Nguyen MK, Kouzani AZ, Bodaghi M. AI-Based Soft Module for Safe Human-Robot Interaction towards 4D Printing. Polymers (Basel) 2022; 14:polym14163302. [PMID: 36015560 PMCID: PMC9416509 DOI: 10.3390/polym14163302] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/27/2022] [Revised: 08/09/2022] [Accepted: 08/10/2022] [Indexed: 11/16/2022] Open
Abstract
Soft robotic modules have potential use for therapeutic and educational purposes. To do so, they need to be safe, soft, smart, and customizable to serve individuals' different preferences and personalities. A safe modular robotic product made of soft materials, particularly silicon, programmed by artificial intelligence algorithms and developed via additive manufacturing would be promising. This study focuses on the safe tactile interaction between humans and robots by means of soft material characteristics for translating physical communication to auditory. The embedded vibratory sensors used to stimulate touch senses transmitted through soft materials are presented. The soft module was developed and verified successfully to react to three different patterns of human-robot contact, particularly users' touches, and then communicate the type of contact with sound. The study develops and verifies a model that can classify different tactile gestures via machine learning algorithms for safe human-robot physical interaction. The system accurately recognizes the gestures and shapes of three-dimensional (3D) printed soft modules. The gestures used for the experiment are the three most common, including slapping, squeezing, and tickling. The model builds on the concept of how safe human-robot physical interactions could help with cognitive and behavioral communication. In this context, the ability to measure, classify, and reflect the behavior of soft materials in robotic modules represents a prerequisite for endowing robotic materials in additive manufacturing for safe interaction with humans.
Collapse
Affiliation(s)
- Ali Zolfagharian
- School of Engineering, Deakin University, Geelong, VIC 3216, Australia
- Correspondence:
| | - Mohammad Reza Khosravani
- Chair of Product Development, University of Siegen, Paul-Bonatz-Str. 9–11, 57068 Siegen, Germany
| | - Hoang Duong Vu
- School of Engineering, Deakin University, Geelong, VIC 3216, Australia
| | - Minh Khoi Nguyen
- School of Engineering, Deakin University, Geelong, VIC 3216, Australia
| | - Abbas Z. Kouzani
- School of Engineering, Deakin University, Geelong, VIC 3216, Australia
| | - Mahdi Bodaghi
- Department of Engineering, School of Science and Technology, Nottingham Trent University, Nottingham NG11 8NS, UK
| |
Collapse
|
8
|
Geva N, Hermoni N, Levy-Tzedek S. Interaction Matters: The Effect of Touching the Social Robot PARO on Pain and Stress is Stronger When Turned ON vs. OFF. Front Robot AI 2022; 9:926185. [PMID: 35875704 PMCID: PMC9305613 DOI: 10.3389/frobt.2022.926185] [Citation(s) in RCA: 5] [Impact Index Per Article: 1.7] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/22/2022] [Accepted: 06/17/2022] [Indexed: 11/13/2022] Open
Abstract
Social touch between humans, as well as between humans and animals, was previously found to reduce pain and stress. We previously reported that touching a social robot can also induce a reduction in pain ratings. However, it is unclear if the effect that touching a robot has on pain perception is due to its appearance and its pleasant touch, or due to its ability to socially interact with humans. In the current experiment, we aimed to assess the contribution of the interactive quality to pain perception. We assessed the effect of touching the social robot PARO on mild and strong pain ratings and on stress perception, on a total of 60 healthy young participants. The robot either interacted with participants (ON group, n = 30) or was turned off (OFF group, n = 30). Touching the robot induced a decrease in mild pain ratings (compared to baseline) only in the ON group while strong pain ratings decreased similarly in both the ON and the OFF groups. The decrease in mild pain ratings in the ON group was significantly greater in participants with a higher positive perception of the interaction with PARO. We conclude that part of the effect that touching the robot has on pain stems from its interactive features.
Collapse
Affiliation(s)
- Nirit Geva
- Recanati School for Community Health Professions, Department of Physical Therapy, Ben-Gurion University of the Negev, Beer-Sheva, Israel
| | - Netta Hermoni
- Department of Biomedical Engineering, Ben-Gurion University of the Negev, Beer-Sheva, Israel
| | - Shelly Levy-Tzedek
- Recanati School for Community Health Professions, Department of Physical Therapy, Ben-Gurion University of the Negev, Beer-Sheva, Israel
- Zlotowski Center for Neuroscience, Ben-Gurion University of the Negev, Beer-Sheva, Israel
- Freiburg Institute for Advanced Studies (FRIAS), University of Freiburg, Freiburg, Germany
- *Correspondence: Shelly Levy-Tzedek,
| |
Collapse
|
9
|
Assistive Technologies, Robotics, Automatic Machines: Perspectives of Integration in the Health Domain. Healthcare (Basel) 2022; 10:healthcare10061080. [PMID: 35742131 PMCID: PMC9222886 DOI: 10.3390/healthcare10061080] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Download PDF] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/24/2022] [Accepted: 06/01/2022] [Indexed: 11/29/2022] Open
|
10
|
Manipulability Optimization of a Rehabilitative Collaborative Robotic System. MACHINES 2022. [DOI: 10.3390/machines10060452] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 11/16/2022]
Abstract
The use of collaborative robots (or cobots) in rehabilitation therapies is aimed at assisting and shortening the patient’s recovery after neurological injuries. Cobots are inherently safe when interacting with humans and can be programmed in different working modalities based on the patient’s needs and the level of the injury. This study presents a design optimization of a robotic system for upper limb rehabilitation based on the manipulability ellipsoid method. The human–robot system is modeled as a closed kinematic chain in which the human hand grasps a handle attached to the robot’s end effector. The manipulability ellipsoids are determined for both the human and the robotic arm and compared by calculating an index that quantifies the alignment of the principal axes. The optimal position of the robot base with respect to the patient is identified by a first global optimization and by a further local refinement, seeking the best alignment of the manipulability ellipsoids in a series of points uniformly distributed within the shared workspace.
Collapse
|
11
|
Simeoni R, Colonnelli F, Eutizi V, Marchetti M, Paolini E, Papalini V, Punturo A, Salvò A, Scipinotti N, Serpente C, Barbini E, Troscia R, Maccioni G, Giansanti D. The Social Robot and the Digital Physiotherapist: Are We Ready for the Team Play? Healthcare (Basel) 2021; 9:1454. [PMID: 34828501 PMCID: PMC8618922 DOI: 10.3390/healthcare9111454] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/12/2021] [Revised: 10/20/2021] [Accepted: 10/23/2021] [Indexed: 11/17/2022] Open
Abstract
Motivation: We are witnessing two phenomena. The first is that the physiotherapist is increasingly becoming a figure that must interact with Digital Health. On the other hand, social robots through research are improving more and more in the aspects of social interaction thanks also to artificial intelligence and becoming useful in rehabilitation processes. It begins to become strategic to investigate the intersections between these two phenomena. Objective: Therefore, we set ourselves the goal of investigating the consensus and opinion of physiotherapists around the introduction of social robots in clinical practice both in rehabilitation and assistance. Procedure: An electronic survey has been developed focused on social robot-based rehabilitation and assistance and has been submitted to subjects focused on physiotherapy sciences to investigate their opinion and their level of consent regarding the use of the social robot in rehabilitation and assistance. Two samples of subjects were recruited: the first group (156 participating subjects, 79 males, 77 females, mean age 24.3 years) was in the training phase, and the second (167 participating subjects, 86 males, 81 females, mean age 42.4 years) group was involved in the work processes. An electronic feedback form was also submitted to investigate the acceptance of the proposed methodology. Results: The survey showed a consistency of the results between the two samples from which interesting considerations emerge. Contrary to stereotypes that report how AI-based devices put jobs at risk, physiotherapists are not afraid of these devices. The subjects involved in the study believe the following: (a) social robots can be reliable co-workers but will remain a complementary device; (b) their role will be of the utmost importance as an operational manager in their use and in performance monitoring; (c) these devices will allow an increase in working capacity and facilitate integration. All those involved in the study believe that the proposed electronic survey has proved to be a useful and effective tool that can be useful as a periodic monitoring tool and useful for scientific societies. Conclusions: The evolution of social robots represents an unstoppable process as does the increase in the aging of the population. Stakeholders must not look with suspicion toward these devices, which can represent an important resource, but rather invest in monitoring and consensus training initiatives.
Collapse
Affiliation(s)
- Rossella Simeoni
- Faculty of Medicine and Surgery, Università Cattolica del Sacro Cuore, San Martino al Cimino, 01100 Viterbo, Italy; (R.S.); (F.C.); (V.E.); (M.M.); (E.P.); (V.P.); (A.P.); (A.S.); (N.S.); (C.S.); (E.B.); (R.T.)
| | - Federico Colonnelli
- Faculty of Medicine and Surgery, Università Cattolica del Sacro Cuore, San Martino al Cimino, 01100 Viterbo, Italy; (R.S.); (F.C.); (V.E.); (M.M.); (E.P.); (V.P.); (A.P.); (A.S.); (N.S.); (C.S.); (E.B.); (R.T.)
| | - Veronica Eutizi
- Faculty of Medicine and Surgery, Università Cattolica del Sacro Cuore, San Martino al Cimino, 01100 Viterbo, Italy; (R.S.); (F.C.); (V.E.); (M.M.); (E.P.); (V.P.); (A.P.); (A.S.); (N.S.); (C.S.); (E.B.); (R.T.)
| | - Matteo Marchetti
- Faculty of Medicine and Surgery, Università Cattolica del Sacro Cuore, San Martino al Cimino, 01100 Viterbo, Italy; (R.S.); (F.C.); (V.E.); (M.M.); (E.P.); (V.P.); (A.P.); (A.S.); (N.S.); (C.S.); (E.B.); (R.T.)
| | - Elena Paolini
- Faculty of Medicine and Surgery, Università Cattolica del Sacro Cuore, San Martino al Cimino, 01100 Viterbo, Italy; (R.S.); (F.C.); (V.E.); (M.M.); (E.P.); (V.P.); (A.P.); (A.S.); (N.S.); (C.S.); (E.B.); (R.T.)
| | - Valentina Papalini
- Faculty of Medicine and Surgery, Università Cattolica del Sacro Cuore, San Martino al Cimino, 01100 Viterbo, Italy; (R.S.); (F.C.); (V.E.); (M.M.); (E.P.); (V.P.); (A.P.); (A.S.); (N.S.); (C.S.); (E.B.); (R.T.)
| | - Alessio Punturo
- Faculty of Medicine and Surgery, Università Cattolica del Sacro Cuore, San Martino al Cimino, 01100 Viterbo, Italy; (R.S.); (F.C.); (V.E.); (M.M.); (E.P.); (V.P.); (A.P.); (A.S.); (N.S.); (C.S.); (E.B.); (R.T.)
| | - Alice Salvò
- Faculty of Medicine and Surgery, Università Cattolica del Sacro Cuore, San Martino al Cimino, 01100 Viterbo, Italy; (R.S.); (F.C.); (V.E.); (M.M.); (E.P.); (V.P.); (A.P.); (A.S.); (N.S.); (C.S.); (E.B.); (R.T.)
| | - Nicoletta Scipinotti
- Faculty of Medicine and Surgery, Università Cattolica del Sacro Cuore, San Martino al Cimino, 01100 Viterbo, Italy; (R.S.); (F.C.); (V.E.); (M.M.); (E.P.); (V.P.); (A.P.); (A.S.); (N.S.); (C.S.); (E.B.); (R.T.)
| | - Christian Serpente
- Faculty of Medicine and Surgery, Università Cattolica del Sacro Cuore, San Martino al Cimino, 01100 Viterbo, Italy; (R.S.); (F.C.); (V.E.); (M.M.); (E.P.); (V.P.); (A.P.); (A.S.); (N.S.); (C.S.); (E.B.); (R.T.)
| | - Emanuele Barbini
- Faculty of Medicine and Surgery, Università Cattolica del Sacro Cuore, San Martino al Cimino, 01100 Viterbo, Italy; (R.S.); (F.C.); (V.E.); (M.M.); (E.P.); (V.P.); (A.P.); (A.S.); (N.S.); (C.S.); (E.B.); (R.T.)
| | - Riccardo Troscia
- Faculty of Medicine and Surgery, Università Cattolica del Sacro Cuore, San Martino al Cimino, 01100 Viterbo, Italy; (R.S.); (F.C.); (V.E.); (M.M.); (E.P.); (V.P.); (A.P.); (A.S.); (N.S.); (C.S.); (E.B.); (R.T.)
| | | | | |
Collapse
|
12
|
Facial Emotion Recognition from an Unmanned Flying Social Robot for Home Care of Dependent People. ELECTRONICS 2021. [DOI: 10.3390/electronics10070868] [Citation(s) in RCA: 4] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 11/16/2022]
Abstract
This work is part of an ongoing research project to develop an unmanned flying social robot to monitor dependants at home in order to detect the person’s state and bring the necessary assistance. In this sense, this paper focuses on the description of a virtual reality (VR) simulation platform for the monitoring process of an avatar in a virtual home by a rotatory-wing autonomous unmanned aerial vehicle (UAV). This platform is based on a distributed architecture composed of three modules communicated through the message queue telemetry transport (MQTT) protocol: the UAV Simulator implemented in MATLAB/Simulink, the VR Visualiser developed in Unity, and the new emotion recognition (ER) system developed in Python. Using a face detection algorithm and a convolutional neural network (CNN), the ER System is able to detect the person’s face in the image captured by the UAV’s on-board camera and classify the emotion among seven possible ones (surprise; fear; happiness; sadness; disgust; anger; or neutral expression). The experimental results demonstrate the correct integration of this new computer vision module within the VR platform, as well as the good performance of the designed CNN, with around 85% in the F1-score, a mean of the precision and recall of the model. The developed emotion detection system can be used in the future implementation of the assistance UAV that monitors dependent people in a real environment, since the methodology used is valid for images of real people.
Collapse
|