Rib region detection for scanning path planning for fully automated robotic abdominal ultrasonography

PURPOSE

Scanning path planning is an essential technology for fully automated ultrasound (US) robotics. During biliary scanning, the subcostal boundary is critical body surface landmarks for scanning path planning but are often invisible, depending on the individual. This study developed a method of estimating the rib region for scanning path planning toward fully automated robotic US systems.

METHODS

We proposed a method for determining the rib region using RGB-D images and respiratory variation. We hypothesized that detecting the rib region would be possible based on changes in body surface position due to breathing. We generated a depth difference image by finding the difference between the depth image taken at the resting inspiratory position and the depth image taken at the maximum inspiratory position, which clearly shows the rib region. The boundary position of the subcostal was then determined by applying training using the YOLOv5 object detection model to this depth difference image.

RESULTS

In the experiments with healthy subjects, the proposed method of rib detection using the depth difference image marked an intersection over union (IoU) of 0.951 and average confidence of 0.77. The average error between the ground truth and predicted positions was 16.5 mm in 3D space. The results were superior to rib detection using only the RGB image.

CONCLUSION

The proposed depth difference imaging method, which measures respiratory variation, was able to accurately estimate the rib region without contact and physician intervention. It will be useful for planning the scan path during the biliary imaging.

Collaborative robots can augment human cognition in regret-sensitive tasks

Despite theoretical benefits of collaborative robots, disappointing outcomes are well documented by clinical studies, spanning rehabilitation, prostheses, and surgery. Cognitive load theory provides a possible explanation for why humans in the real world are not realizing the benefits of collaborative robots: high cognitive loads may be impeding human performance. Measuring cognitive availability using an electrocardiogram, we ask 25 participants to complete a virtual-reality task alongside an invisible agent that determines optimal performance by iteratively updating the Bellman equation. Three robots assist by providing environmental information relevant to task performance. By enabling the robots to act more autonomously-managing more of their own behavior with fewer instructions from the human-here we show that robots can augment participants' cognitive availability and decision-making. The way in which robots describe and achieve their objective can improve the human's cognitive ability to reason about the task and contribute to human-robot collaboration outcomes. Augmenting human cognition provides a path to improve the efficacy of collaborative robots. By demonstrating how robots can improve human cognition, this work paves the way for improving the cognitive capabilities of first responders, manufacturing workers, surgeons, and other future users of collaborative autonomy systems.

Locating and sizing tumor nodules in human prostate using instrumented probing - computational framework and experimental validation

Detection of tumor nodules is key to early cancer diagnosis. This study investigates the potential of using the mechanical data, acquired from probing the prostate for detecting the existence, and, more importantly, characterizing the size and depth, from the posterior surface, of the prostate cancer (PCa) nodules. A computational approach is developed to quantify the uncertainty of nodule detectability and is based on identifying stiffness anomalies in the profiles of point force measurements across transverse sections of the prostate. The capability of the proposed method was assessed firstly using a 'training' dataset of in silico models including PCa nodules with random size, depth and location, followed by a clinical feasibility study, involving experimental data from 13 ex vivo prostates from patients who had undergone radical prostatatectomy. Promising levels of sensitivity and specificity were obtained for detecting the PCa nodules in a total of 44 prostate sections. This study has shown that the proposed methods could be a useful complementary tool to exisiting diagnostic methods of PCa. The future study will involve implementing the proposed measurement and detection strategies in vivo, with the help of a miniturized medical device.

Mechanism of Anti-Inflammatory Drugs in the Early Treatment of Oral Gingival Mucosa and Soft Tissue Trauma

Traumatic tissue develops an uncontrolled inflammatory response that causes secondary damage to the injured tissue and other parts of the body. Therefore, preventing wound infection, reducing inflammatory response, and reducing secondary tissue damage are the keys to early treatment of tissue trauma. In the treatment of gingival soft tissue trauma, anti-inflammatory and analgesic drugs are reasonably selected according to the condition, which can effectively reduce inflammation, and they help periodontal tissue regeneration and healing. However, there are few studies on the mechanism of anti-inflammatory drugs in the early treatment of oral gingival mucosal soft tissue trauma, and the specific mechanism is unknown. Therefore, this paper explored the mechanism of anti-inflammatory drugs in the early treatment of oral gingival mucosa and soft tissue trauma through experiments, which provided theoretical support for the clinical treatment of gingival mucosa and soft tissue trauma repair. In this paper, two anti-inflammatory drugs, levofloxacin and metronidazole, were selected to measure their release properties in vitro and in vivo. Then, the white-eared rabbits were treated with gingival wound treatment experiments, and the physiological characteristics, intratissue pressure, tissue partial pressure of oxygen, IL-6 content, and PGE2 content were determined at each postinjury period, and the mechanism of action of anti-inflammatory drugs was determined. Research results have shown that anti-inflammatory drugs can significantly inhibit the content of IL-6 and PGE2 in gingival soft tissue after injury, reduce the local inflammatory response, and accelerate tissue healing.

Action Augmentation of Tactile Perception for Soft-Body Palpation

Medical palpation is a diagnostic technique in which physicians use the sense of touch to manipulate the soft human tissue. This can be done to enable the diagnosis of possibly life-threatening conditions, such as cancer. Palpation is still poorly understood because of the complex interaction dynamics between the practitioners' hands and the soft human body. To understand this complex of soft body interactions, we explore robotic palpation for the purpose of diagnosing the presence of abnormal inclusions, or tumors. Using a Bayesian framework for training and classification, we show that the exploration of soft bodies requires complex, multi-axis, palpation trajectories. We also find that this probabilistic approach is capable of rapidly searching the large action space of the robot. This work progresses “robotic” palpation, and it provides frameworks for understanding and exploiting soft body interactions.

Conditioned haptic perception for 3D localization of nodules in soft tissue palpation with a variable stiffness probe

This paper provides a solution for fast haptic information gain during soft tissue palpation using a Variable Lever Mechanism (VLM) probe. More specifically, we investigate the impact of stiffness variation of the probe to condition likelihood functions of the kinesthetic force and tactile sensors measurements during a palpation task for two sweeping directions. Using knowledge obtained from past probing trials or Finite Element (FE) simulations, we implemented this likelihood conditioning in an autonomous palpation control strategy. Based on a recursive Bayesian inferencing framework, this new control strategy adapts the sweeping direction and the stiffness of the probe to detect abnormal stiff inclusions in soft tissues. This original control strategy for compliant palpation probes shows a sub-millimeter accuracy for the 3D localization of the nodules in a soft tissue phantom as well as a 100% reliability detecting the existence of nodules in a soft phantom.

Share control of surgery robot master manipulator guiding tool along the standard path

Recently, minimally invasive surgery (MIS) robotics enters the phase of autonomous operation. However, because of the high variability of the environment, conducting a fully autonomous surgery is still extremely difficult. This paper presents a share control system, the objective of which is to suggest the optimum path of tool guidance through the use of force on the master manipulator (hereinafter as master), meaning the surgeon's hand. Owing to this type of control, the surgeon has full control over the position of the tool the entire time and is supported by the system to better and faster guide the tool during surgery. The force should be felt by the surgeon but, simultaneously, must not hinder or impact the surgical process. Furthermore, the share control system presented in the paper can be turned on or off at any moment during surgery.

Enhancing Perception with Tactile Object Recognition in Adaptive Grippers for Human-Robot Interaction

The use of tactile perception can help first response robotic teams in disaster scenarios, where visibility conditions are often reduced due to the presence of dust, mud, or smoke, distinguishing human limbs from other objects with similar shapes. Here, the integration of the tactile sensor in adaptive grippers is evaluated, measuring the performance of an object recognition task based on deep convolutional neural networks (DCNNs) using a flexible sensor mounted in adaptive grippers. A total of 15 classes with 50 tactile images each were trained, including human body parts and common environment objects, in semi-rigid and flexible adaptive grippers based on the fin ray effect. The classifier was compared against the rigid configuration and a support vector machine classifier (SVM). Finally, a two-level output network has been proposed to provide both object-type recognition and human/non-human classification. Sensors in adaptive grippers have a higher number of non-null tactels (up to 37% more), with a lower mean of pressure values (up to 72% less) than when using a rigid sensor, with a softer grip, which is needed in physical human–robot interaction (pHRI). A semi-rigid implementation with 95.13% object recognition rate was chosen, even though the human/non-human classification had better results (98.78%) with a rigid sensor.