Reaction Force Mapping by 3-Axis Tactile Sensing With Arbitrary Angles for Tissue Hard-Inclusion Localization

A piezoresistive-based 3-axial MEMS tactile sensor and integrated surgical forceps for gastrointestinal endoscopic minimally invasive surgery

In robotic-assisted surgery (RAS), traditional surgical instruments without sensing capability cannot perceive accurate operational forces during the task, and such drawbacks can be largely intensified when sophisticated tasks involving flexible and slender arms with small end-effectors, such as in gastrointestinal endoscopic surgery (GES). In this study, we propose a microelectromechanical system (MEMS) piezoresistive 3-axial tactile sensor for GES forceps, which can intuitively provide surgeons with online force feedback during robotic surgery. The MEMS fabrication process facilitates sensor chips with miniaturized dimensions. The fully encapsulated tactile sensors can be effortlessly integrated into miniature GES forceps, which feature a slender diameter of just 3.5 mm and undergo meticulous calibration procedures via the least squares method. Through experiments, the sensor's ability to accurately measure directional forces up to 1.2 N in the Z axis was validated, demonstrating an average relative error of only 1.18% compared with the full-scale output. The results indicate that this tactile sensor can provide effective 3-axial force sensing during surgical operations, such as grasping and pulling, and in ex vivo testing with a porcine stomach. The compact size, high precision, and integrability of the sensor establish solid foundations for clinical application in the operating theater.

TransVFS: A spatio-temporal local-global transformer for vision-based force sensing during ultrasound-guided prostate biopsy

Robot-assisted prostate biopsy is a new technology to diagnose prostate cancer, but its safety is influenced by the inability of robots to sense the tool-tissue interaction force accurately during biopsy. Recently, vision based force sensing (VFS) provides a potential solution to this issue by utilizing image sequences to infer the interaction force. However, the existing mainstream VFS methods cannot realize the accurate force sensing due to the adoption of convolutional or recurrent neural network to learn deformation from the optical images and some of these methods are not efficient especially when the recurrent convolutional operations are involved. This paper has presented a Transformer based VFS (TransVFS) method by leveraging ultrasound volume sequences acquired during prostate biopsy. The TransVFS method uses a spatio-temporal local-global Transformer to capture the local image details and the global dependency simultaneously to learn prostate deformations for force estimation. Distinctively, our method explores both the spatial and temporal attention mechanisms for image feature learning, thereby addressing the influence of the low ultrasound image resolution and the unclear prostate boundary on the accurate force estimation. Meanwhile, the two efficient local-global attention modules are introduced to reduce 4D spatio-temporal computation burden by utilizing the factorized spatio-temporal processing strategy, thereby facilitating the fast force estimation. Experiments on prostate phantom and beagle dogs show that our method significantly outperforms existing VFS methods and other spatio-temporal Transformer models. The TransVFS method surpasses the most competitive compared method ResNet3dGRU by providing the mean absolute errors of force estimation, i.e., 70.4 ± 60.0 millinewton (mN) vs 123.7 ± 95.6 mN, on the transabdominal ultrasound dataset of dogs.

An intelligent grasper to provide real-time force feedback to shorten the learning curve in laparoscopic training

BACKGROUND

A lack of force feedback in laparoscopic surgery often leads to a steep learning curve to the novices and traditional training system equipped with force feedback need a high educational cost. This study aimed to use a laparoscopic grasper providing force feedback in laparoscopic training which can assist in controlling of gripping forces and improve the learning processing of the novices.

METHODS

Firstly, we conducted a pre-experiment to verify the role of force feedback in gripping operations and establish the safe gripping force threshold for the tasks. Following this, we proceeded with a four-week training program. Unlike the novices without feedback (Group A2), the novices receiving feedback (Group B2) underwent training that included force feedback. Finally, we completed a follow-up period without providing force feedback to assess the training effect under different conditions. Real-time force parameters were recorded and compared.

RESULTS

In the pre-experiment, we set the gripping force threshold for the tasks based on the experienced surgeons' performance. This is reasonable as the experienced surgeons have obtained adequate skill of handling grasper. The thresholds for task 1, 2, and 3 were set as 0.731 N, 1.203 N and 0.938 N, respectively. With force feedback, the gripping force applied by the novices with feedback (Group B1) was lower than that of the novices without feedback (Group A1) (p < 0.005). During the training period, the Group B2 takes 6 trails to achieve gripping force of 0.635 N, which is lower than the threshold line, whereas the Group A2 needs 11 trails, meaning that the learning curve of Group B2 was significantly shorter than that of Group A2. Additionally, during the follow-up period, there was no significant decline in force learning, and Group B2 demonstrated better control of gripping operations. The training with force feedback received positive evaluations.

CONCLUSION

Our study shows that using a grasper providing force feedback in laparoscopic training can help to control the gripping force and shorten the learning curve. It is anticipated that the laparoscopic grasper equipped with FBG sensor is promising to provide force feedback during laparoscopic training, which ultimately shows great potential in laparoscopic surgery.

Visualizing tactile feedback: an overview of current technologies with a focus on ultrasound elastography

Tissue elasticity remains an essential biomarker of health and is indicative of irregularities such as tumors or infection. The timely detection of such abnormalities is crucial for the prevention of disease progression and complications that arise from late-stage illnesses. However, at both the bedside and the operating table, there is a distinct lack of tactile feedback for deep-seated tissue. As surgical techniques advance toward remote or minimally invasive options to reduce infection risk and hasten healing time, surgeons lose the ability to manually palpate tissue. Furthermore, palpation of deep structures results in decreased accuracy, with the additional barrier of needing years of experience for adequate confidence of diagnoses. This review delves into the current modalities used to fulfill the clinical need of quantifying physical touch. It covers research efforts involving tactile sensing for remote or minimally invasive surgeries, as well as the potential of ultrasound elastography to further this field with non-invasive real-time imaging of the organ's biomechanical properties. Elastography monitors tissue response to acoustic or mechanical energy and reconstructs an image representative of the elastic profile in the region of interest. This intuitive visualization of tissue elasticity surpasses the tactile information provided by sensors currently used to augment or supplement manual palpation. Focusing on common ultrasound elastography modalities, we evaluate various sensing mechanisms used for measuring tactile information and describe their emerging use in clinical settings where palpation is insufficient or restricted. With the ongoing advancements in ultrasound technology, particularly the emergence of micromachined ultrasound transducers, these devices hold great potential in facilitating early detection of tissue abnormalities and providing an objective measure of patient health.

A BTO/PVDF/PDMS Piezoelectric Tangential and Normal Force Sensor Inspired by a Wind Chime

There is a growing demand for flexible pressure sensors in environmental monitoring and human-robot interaction robotics. A flexible and susceptible sensor can discriminate multidirectional pressure, thus effectively detecting signals of small environmental changes and providing solutions for personalized medicine. This paper proposes a multidimensional force detection sensor inspired by a wind chime structure with a three-dimensional force structure to detect and analyze normal and shear forces in real time. The force-sensing structure of the sensor consists of an upper and lower membrane on a polydimethylsiloxane substrate and four surrounding cylinders. A piezoelectric hemisphere is made of BTO/PVDF/PDMS composite material. The sensor columns in the wind chime structure surround the piezoelectric layer in the middle. When pressure is applied externally, the sensor columns are connected to the piezoelectric layer with a light touch. The piezoelectric hemisphere generates a voltage signal. Due to the particular structure of the sensor, it can accurately capture multidimensional forces and identify the direction of the external force by analyzing the position of the sensor and the output voltage amplitude. The development of such sensors shows excellent potential for self-powered wearable sensors, human-computer interaction, electronic skin, and soft robotics applications.

Progress in Probe-Based Sensing Techniques for In Vivo Diagnosis

Advancements in robotic surgery help to improve the endoluminal diagnosis and treatment with minimally invasive or non-invasive intervention in a precise and safe manner. Miniaturized probe-based sensors can be used to obtain information about endoluminal anatomy, and they can be integrated with medical robots to augment the convenience of robotic operations. The tremendous benefit of having this physiological information during the intervention has led to the development of a variety of in vivo sensing technologies over the past decades. In this paper, we review the probe-based sensing techniques for the in vivo physical and biochemical sensing in China in recent years, especially on in vivo force sensing, temperature sensing, optical coherence tomography/photoacoustic/ultrasound imaging, chemical sensing, and biomarker sensing.

Origami-Inspired Structure with Pneumatic-Induced Variable Stiffness for Multi-DOF Force-Sensing

With the emerging need for human-machine interactions, multi-modal sensory interaction is gradually pursued rather than satisfying common perception forms (visual or auditory), so developing flexible, adaptive, and stiffness-variable force-sensing devices is the key to further promoting human-machine fusion. However, current sensor sensitivity is fixed and nonadjustable after fabrication, limiting further development. To solve this problem, we propose an origami-inspired structure to achieve multiple degrees of freedom (DoFs) motions with variable stiffness for force-sensing, which combines the ductility and flexibility of origami structures. In combination with the pneumatic actuation, the structure can achieve and adapt the compression, pitch, roll, diagonal, and array motions (five motion modes), which significantly increase the force adaptability and sensing diversity. To achieve closed-loop control and avoid excessive gas injection, the ultra-flexible microfiber sensor is designed and seamlessly embedded with an approximately linear sensitivity of ∼0.35 Ω/kPa at a relative pressure of 0-100 kPa, and an exponential sensitivity at a relative pressure of 100-350 kPa, which can render this device capable of working under various conditions. The final calibration experiment demonstrates that the pre-pressure value can affect the sensor's sensitivity. With the increasing pre-pressure of 65-95 kPa, the average sensitivity curve shifts rightwards around 9 N intervals, which highly increases the force-sensing capability towards the range of 0-2 N. When the pre-pressure is at the relatively extreme air pressure of 100 kPa, the force sensitivity value is around 11.6 Ω/N. Therefore, our proposed design (which has a low fabrication cost, high integration level, and a suitable sensing range) shows great potential for applications in flexible force-sensing development.

A Piezoelectric Tactile Sensor for Tissue Stiffness Detection with Arbitrary Contact Angle

In this paper, a piezoelectric tactile sensor for detecting tissue stiffness in robot-assisted minimally invasive surgery (RMIS) is proposed. It can detect the stiffness not only when the probe is normal to the tissue surface, but also when there is a contact angle between the probe and normal direction. It solves the problem that existing sensors can only detect in the normal direction to ensure accuracy when the degree of freedom (DOF) of surgical instruments is limited. The proposed senor can distinguish samples with different stiffness and recognize lump from normal tissue effectively when the contact angle varies within [0°, 45°]. These are achieved by establishing a new detection model and sensor optimization. It deduces the influence of contact angle on stiffness detection by sensor parameters design and optimization. The detection performance of the sensor is confirmed by simulation and experiment. Five samples with different stiffness (including lump and normal samples with close stiffness) are used. Through blind recognition test in simulation, the recognition rate is 100% when the contact angle is randomly selected within 30°, 94.1% within 45°, which is 38.7% higher than the unoptimized sensor. Through blind classification test and automatic k-means clustering in experiment, the correct rate is 92% when the contact angle is randomly selected within 45°. We can get the proposed sensor can easily recognize samples with different stiffness with high accuracy which has broad application prospects in the medical field.