Haptic Object Recognition in Underwater and Deep-sea Environments

Bioinspired soft robots for deep-sea exploration

The deep ocean, Earth's untouched expanse, presents immense challenges for exploration due to its extreme pressure, temperature, and darkness. Unlike traditional marine robots that require specialized metallic vessels for protection, deep-sea species thrive without such cumbersome pressure-resistant designs. Their pressure-adaptive forms, unique propulsion methods, and advanced senses have inspired innovation in designing lightweight, compact soft machines. This perspective addresses challenges, recent strides, and design strategies for bioinspired deep-sea soft robots. Drawing from abyssal life, it explores the actuation, sensing, power, and pressure resilience of multifunctional deep-sea soft robots, offering game-changing solutions for profound exploration and operation in harsh conditions.

Visual-Tactile Fused Graph Learning for Object Clustering

Object clustering has received considerable research attention most recently. However, 1) most existing object clustering methods utilize visual information while ignoring important tactile modality, which would inevitably lead to model performance degradation and 2) simply concatenating visual and tactile information via multiview clustering method can make complementary information to not be fully explored, since there are many differences between vision and touch. To address these issues, we put forward a graph-based visual-tactile fused object clustering framework with two modules: 1) a modality-specific representation learning module M_R and 2) a unified affinity graph learning module M_U . Specifically, M_R focuses on learning modality-specific representations for visual-tactile data, where deep non-negative matrix factorization (NMF) is adopted to extract the hidden information behind each modality. Meanwhile, we employ an autoencoder-like structure to enhance the robustness of the learned representations, and two graphs to improve its compactness. Furthermore, M_U highlights how to mitigate the differences between vision and touch, and further maximize the mutual information, which adopts a minimizing disagreement scheme to guide the modality-specific representations toward a unified affinity graph. To achieve ideal clustering performance, a Laplacian rank constraint is imposed to regularize the learned graph with ideal connected components, where noises that caused wrong connections are removed and clustering labels can be obtained directly. Finally, we propose an efficient alternating iterative minimization updating strategy, followed by a theoretical proof to prove framework convergence. Comprehensive experiments on five public datasets demonstrate the superiority of the proposed framework.

Soft Multi-Directional Force Sensor for Underwater Robotic Application

Tactile information is crucial for recognizing physical interactions, manipulation of an object, and motion planning for a robotic gripper; however, concurrent tactile technologies have certain limitations over directional force sensing. In particular, they are expensive, difficult to fabricate, and mostly unsuitable for underwater use. Here, we present a facile and cost-effective synthesis technique of a flexible multi-directional force sensing system, which is also favorable to be utilized in underwater environments. We made use of four flex sensors within a silicone-made hemispherical shell structure. Each sensor was placed 90° apart and aligned with the curve of the hemispherical shape. If the force is applied on the top of the hemisphere, all the flex sensors would bend uniformly and yield nearly identical readings. When force is applied from a different direction, a set of flex sensors would characterize distinctive output patterns to localize the point of contact as well as the direction and magnitude of the force. The deformation of the fabricated soft sensor due to applied force was simulated numerically and compared with the experimental results. The fabricated sensor was experimentally calibrated and tested for characterization including an underwater demonstration. This study would widen the scope of identification of multi-directional force sensing, especially for underwater soft robotic applications.

Advances and future outlooks in soft robotics for minimally invasive marine biology

This Viewpoint describes interdisciplinary research that aims to maximize understanding of deep marine life, while concurrently being minimally invasive. We describe the synthesis of multiple modern approaches (spanning robotics, biology, biomechanics, engineering, imaging, and genomic sequencing) and present future directions that hold the potential for a paradigm shift in marine biology.

Compliant Underwater Manipulator with Integrated Tactile Sensor for Nonlinear Force Feedback Control of an SMA Actuation System

Design, sensing, and control of underwater gripping systems remain challenges for soft robotic manipulators. Our study investigates these critical issues by designing a shape memory alloy (SMA) actuation system for a soft robotic finger with a directly 3D-printed stretchable skin-like tactile sensor. SMA actuators were thermomechanically trained to assume a curved finger-like shape when Joule heated, and the flexible multi-layered tactile sensor was directly 3D-printed onto the surface of the fingertip. A nonlinear controller was developed to enable precise fingertip force control using feedback from the compliant tactile sensor. Underwater experiments were conducted using closed-loop force feedback from the directly 3D-printed tactile sensor with the SMA actuators, showing satisfactory force tracking ability. Furthermore, a 3D finite element model was developed to more deeply understand the shape memory thermal-fluidic-structural multi-physics simulation of the manipulator underwater. An application for human control via electromyogram (EMG) signals also demonstrated an intuitive way for a person to operate the submerged robotic finger. Together, these results suggested that the soft robotic finger could be used to carefully manipulate fragile objects underwater.

Hands in the Real World

Robots face a rapidly expanding range of potential applications beyond controlled environments, from remote exploration and search-and-rescue to household assistance and agriculture. The focus of physical interaction is typically delegated to end-effectors-fixtures, grippers or hands-as these machines perform manual tasks. Yet, effective deployment of versatile robot hands in the real world is still limited to few examples, despite decades of dedicated research. In this paper we review hands that found application in the field, aiming to discuss open challenges with more articulated designs, discussing novel trends and perspectives. We hope to encourage swift development of capable robotic hands for long-term use in varied real world settings. The first part of the paper centers around progress in artificial hand design, identifying key functions for a variety of environments. The final part focuses on the overall trends in hand mechanics, sensors and control, and how performance and resiliency are qualified for real world deployment.

Feel-Good Robotics: Requirements on Touch for Embodiment in Assistive Robotics

The feeling of embodiment, i.e., experiencing the body as belonging to oneself and being able to integrate objects into one's bodily self-representation, is a key aspect of human self-consciousness and has been shown to importantly shape human cognition. An extension of such feelings toward robots has been argued as being crucial for assistive technologies aiming at restoring, extending, or simulating sensorimotor functions. Empirical and theoretical work illustrates the importance of sensory feedback for the feeling of embodiment and also immersion; we focus on the the perceptual level of touch and the role of tactile feedback in various assistive robotic devices. We critically review how different facets of tactile perception in humans, i.e., affective, social, and self-touch, might influence embodiment. This is particularly important as current assistive robotic devices – such as prostheses, orthoses, exoskeletons, and devices for teleoperation–often limit touch low-density and spatially constrained haptic feedback, i.e., the mere touch sensation linked to an action. Here, we analyze, discuss, and propose how and to what degree tactile feedback might increase the embodiment of certain robotic devices, e.g., prostheses, and the feeling of immersion in human-robot interaction, e.g., in teleoperation. Based on recent findings from cognitive psychology on interactive processes between touch and embodiment, we discuss technical solutions for specific applications, which might be used to enhance embodiment, and facilitate the study of how embodiment might alter human-robot interactions. We postulate that high-density and large surface sensing and stimulation are required to foster embodiment of such assistive devices.

Design, Analysis and Experiment of a Tactile Force Sensor for Underwater Dexterous Hand Intelligent Grasping

This paper proposes a novel underwater dexterous hand structure whose fingertip is equipped with underwater tactile force sensor (UTFS) array to realize the grasping sample location determination and force perception. The measurement structure, theoretical analysis, prototype development and experimental verification of the UTFS are purposefully studied in order to achieve accurate measurement under huge water pressure influence. The UTFS is designed as capsule shape type with differential pressure structure, and the external water pressure signal is separately transmitted to the silicon cup bottom which is considered to be an elastomer with four strain elements distribution through the upper and lower flexible contacts and the silicone oil filled in the upper and lower cavities of UTFS. The external tactile force information can be obtained by the vector superposition between the upper and lower of silicon cup bottom to counteract the water pressure influence. The analytical solution of deformation and stress of the bottom of the square silicon cup bottom is analyzed with the use of elasticity and shell theory, and compared with the Finite Element Analysis results, which provides theoretical support for the distribution design of four strain elements at the bottom of the silicon cup. At last, the UTFS zero drift experiment without force applying under different water depths, the output of the standard force applying under different water depth and the test of the standard force applying under conditions of different 0 ∘C–30 ∘C temperature with 0.1 m water depth are carried out to verify the performance of the sensor. The experiments show that the UTFS has a high linearity and sensitivity, and which has a regular zero drift and temperature drift which can be eliminated by calibration algorithm.

Feel-Good Robotics: Requirements on Touch for Embodiment in Assistive Robotics

The feeling of embodiment, i.e., experiencing the body as belonging to oneself and being able to integrate objects into one's bodily self-representation, is a key aspect of human self-consciousness and has been shown to importantly shape human cognition. An extension of such feelings toward robots has been argued as being crucial for assistive technologies aiming at restoring, extending, or simulating sensorimotor functions. Empirical and theoretical work illustrates the importance of sensory feedback for the feeling of embodiment and also immersion; we focus on the the perceptual level of touch and the role of tactile feedback in various assistive robotic devices. We critically review how different facets of tactile perception in humans, i.e., affective, social, and self-touch, might influence embodiment. This is particularly important as current assistive robotic devices - such as prostheses, orthoses, exoskeletons, and devices for teleoperation-often limit touch low-density and spatially constrained haptic feedback, i.e., the mere touch sensation linked to an action. Here, we analyze, discuss, and propose how and to what degree tactile feedback might increase the embodiment of certain robotic devices, e.g., prostheses, and the feeling of immersion in human-robot interaction, e.g., in teleoperation. Based on recent findings from cognitive psychology on interactive processes between touch and embodiment, we discuss technical solutions for specific applications, which might be used to enhance embodiment, and facilitate the study of how embodiment might alter human-robot interactions. We postulate that high-density and large surface sensing and stimulation are required to foster embodiment of such assistive devices.

Object recognition combining vision and touch

This paper explores ways of combining vision and touch for the purpose of object recognition. In particular, it focuses on scenarios when there are few tactile training samples (as these are usually costly to obtain) and when vision is artificially impaired. Whilst machine vision is a widely studied field, and machine touch has received some attention recently, the fusion of both modalities remains a relatively unexplored area. It has been suggested that, in the human brain, there exist shared multi-sensorial representations of objects. This provides robustness when one or more senses are absent or unreliable. Modern robotics systems can benefit from multi-sensorial input, in particular in contexts where one or more of the sensors perform poorly. In this paper, a recently proposed tactile recognition model was extended by integrating a simple vision system in three different ways: vector concatenation (vision feature vector and tactile feature vector), object label posterior averaging and object label posterior product. A comparison is drawn in terms of overall accuracy of recognition and in terms of how quickly (number of training samples) learning occurs. The conclusions reached are: (1) the most accurate system is “posterior product”, (2) multi-modal recognition has higher accuracy to either modality alone if all visual and tactile training data are pooled together, and (3) in the case of visual impairment, multi-modal recognition “learns faster”, i.e. requires fewer training samples to achieve the same accuracy as either other modality.

The Use of Vibrotactile Feedback to Restore Texture Recognition Capabilities, and the Effect of Subject Training

This paper presents a vibrotactile haptic feedback system for use under dynamic conditions, verifies its functionality, and shows how results may be affected by the amount of training that subjects receive. We hope that by using vibrotactile feedback to distinguish between different textures, upper-limb amputees may be able to partially regain the sense of touch. During a previous experiment (Motamedi et al., 2015) we noticed a correlation between how familiar the subjects were with haptic systems, and how well they were able to use the haptic system to accurately identify textures. This observation lead us to conduct a second experiment, the results of which are the main focus of this paper. We began with a group of subjects who were completely unfamiliar with haptic systems, and tracked the improvements in their accuracy over a period of four weeks. Although the subjects showed a 16% improvement in their ability to recognize textures, going from a 64% success rate after the first week to 80% after the fourth, perfect accuracy was not attained. A subsequent experiment, however, shows that this result should not diminish our perception of the haptic system's effectiveness. When we asked the same subjects to identify the textures using only their fingertips, we found that even humans cannot distinguish between near-identical textures with complete accuracy. The subjects' overall success rate when using their own hands was 91%, demonstrating that the proposed haptic system is not far from achieving the same texture recognition capabilities as the human sense of touch.

Haptic Perception of Unknown Object by Robot Hand: Exploration Strategy and Recognition Approach

In this paper, the exploration and recognition in unknown object perception by robot hand is discussed. Inspired by the touch and exploration of human hand, a haptic exploration strategy for multi-fingered robot hand is proposed. Based on the observations from human experiments, the proposed strategy can be used to guide the robot hand to plan a series of movements to get tactile information from different unknown objects, with the precondition of avoiding unexpected collisions with the objects. A recognition approach is then presented to recognize object shapes based on the tactile point data collected by the strategy. Geometric feature vectors are extracted from tactile point locations and normal vectors after clustering, and the object shapes are recognized by the random forests classifier. Simulations and experiments results show that the exploration strategy can be used to guide the robot to gather tactile information from unknown object automatically, and the recognition approach is effective and robust in object shape recognition work. This framework provides a sensible solution for robot unknown object perception problem, which is suitable for the multi-fingered robot hand with low-resolution tactile sensors.