Artificial intelligence meets medical robotics

Artificial intelligence (AI) applications in medical robots are bringing a new era to medicine. Advanced medical robots can perform diagnostic and surgical procedures, aid rehabilitation, and provide symbiotic prosthetics to replace limbs. The technology used in these devices, including computer vision, medical image analysis, haptics, navigation, precise manipulation, and machine learning (ML) , could allow autonomous robots to carry out diagnostic imaging, remote surgery, surgical subtasks, or even entire surgical procedures. Moreover, AI in rehabilitation devices and advanced prosthetics can provide individualized support, as well as improved functionality and mobility (see the figure). The combination of extraordinary advances in robotics, medicine, materials science, and computing could bring safer, more efficient, and more widely available patient care in the future. -Gemma K. Alderton.

What we look for at Science Robotics

Science Robotics welcomes papers demonstrating technical and scientific advances, with potential for influence beyond robotics.

Five years of Science Robotics

[Figure: see text].

Sequential robot imitation learning from observations

This paper presents a framework to learn the sequential structure in the demonstrations for robot imitation learning. We first present a family of task-parameterized hidden semi-Markov models that extracts invariant segments (also called sub-goals or options) from demonstrated trajectories, and optimally follows the sampled sequence of states from the model with a linear quadratic tracking controller. We then extend the concept to learning invariant segments from visual observations that are sequenced together for robot imitation. We present Motion2Vec that learns a deep embedding space by minimizing a metric learning loss in a Siamese network: images from the same action segment are pulled together while being pushed away from randomly sampled images of other segments, and a time contrastive loss is used to preserve the temporal ordering of the images. The trained embeddings are segmented with a recurrent neural network, and subsequently used for decoding the end-effector pose of the robot. We first show its application to a pick-and-place task with the Baxter robot while avoiding a moving obstacle from four kinesthetic demonstrations only, followed by suturing task imitation from publicly available suturing videos of the JIGSAWS dataset with state-of-the-art [Formula: see text]% segmentation accuracy and [Formula: see text] cm error in position per observation on the test set.

Getting a grip on reality

The ability to reliably grasp and manipulate novel objects is a grand challenge for robotics.

Deep learning can accelerate grasp-optimized motion planning

Robots for picking in e-commerce warehouses require rapid computing of efficient and smooth robot arm motions between varying configurations. Recent results integrate grasp analysis with arm motion planning to compute optimal smooth arm motions; however, computation times on the order of tens of seconds dominate motion times. Recent advances in deep learning allow neural networks to quickly compute these motions; however, they lack the precision required to produce kinematically and dynamically feasible motions. While infeasible, the network-computed motions approximate the optimized results. The proposed method warm starts the optimization process by using the approximate motions as a starting point from which the optimizing motion planner refines to an optimized and feasible motion with few iterations. In experiments, the proposed deep learning-based warm-started optimizing motion planner reduces compute and motion time when compared to a sampling-based asymptotically optimal motion planner and an optimizing motion planner. When applied to grasp-optimized motion planning, the results suggest that deep learning can reduce the computation time by two orders of magnitude (300×), from 29 s to 80 ms, making it practical for e-commerce warehouse picking.

Dynamic regret convergence analysis and an adaptive regularization algorithm for on-policy robot imitation learning

On-policy imitation learning algorithms such as DAgger evolve a robot control policy by executing it, measuring performance (loss), obtaining corrective feedback from a supervisor, and generating the next policy. As the loss between iterations can vary unpredictably, a fundamental question is under what conditions this process will eventually achieve a converged policy. If one assumes the underlying trajectory distribution is static (stationary), it is possible to prove convergence for DAgger. However, in more realistic models for robotics, the underlying trajectory distribution is dynamic because it is a function of the policy. Recent results show it is possible to prove convergence of DAgger when a regularity condition on the rate of change of the trajectory distributions is satisfied. In this article, we reframe this result using dynamic regret theory from the field of online optimization and show that dynamic regret can be applied to any on-policy algorithm to analyze its convergence and optimality. These results inspire a new algorithm, Adaptive On-Policy Regularization (Aor), that ensures the conditions for convergence. We present simulation results with cart–pole balancing and locomotion benchmarks that suggest Aor can significantly decrease dynamic regret and chattering as the robot learns. To the best of the authors’ knowledge, this is the first application of dynamic regret theory to imitation learning.

Combating COVID-19—The role of robotics in managing public health and infectious diseases

COVID-19 may drive sustained research in robotics to address risks of infectious diseases.

Applying machine learning to predict future adherence to physical activity programs

BACKGROUND

Identifying individuals who are unlikely to adhere to a physical exercise regime has potential to improve physical activity interventions. The aim of this paper is to develop and test adherence prediction models using objectively measured physical activity data in the Mobile Phone-Based Physical Activity Education program (mPED) trial. To the best of our knowledge, this is the first to apply Machine Learning methods to predict exercise relapse using accelerometer-recorded physical activity data.

METHODS

We use logistic regression and support vector machine methods to design two versions of a Discontinuation Prediction Score (DiPS), which uses objectively measured past data (e.g., steps and goal achievement) to provide a numerical quantity indicating the likelihood of exercise relapse in the upcoming week. The respective prediction accuracy of these two versions of DiPS are compared, and then numerical simulation is performed to explore the potential of using DiPS to selectively allocate financial incentives to participants to encourage them to increase physical activity.

RESULTS

we had access to a physical activity trial data that were continuously collected every 60 sec every day for 9 months in 210 participants. By using the first 15 weeks of data as training and test on weeks 16-30, we show that both versions of DiPS have a test AUC of 0.9 with high sensitivity and specificity in predicting the probability of exercise adherence. Simulation results assuming different intervention regimes suggest the potential benefit of using DiPS as a score to allocate resources in physical activity intervention programs in reducing costs over other allocation schemes.

CONCLUSIONS

DiPS is capable of making accurate and robust predictions for future weeks. The most predictive features are steps and physical activity intensity. Furthermore, the use of DiPS scores can be a promising approach to determine when or if to provide just-in-time messages and step goal adjustments to improve compliance. Further studies on the use of DiPS in the design of physical activity promotion programs are warranted.

TRIAL REGISTRATION

ClinicalTrials.gov NCT01280812 Registered on January 21, 2011.

Learning ambidextrous robot grasping policies

Universal picking (UP), or reliable robot grasping of a diverse range of novel objects from heaps, is a grand challenge for e-commerce order fulfillment, manufacturing, inspection, and home service robots. Optimizing the rate, reliability, and range of UP is difficult due to inherent uncertainty in sensing, control, and contact physics. This paper explores “ambidextrous” robot grasping, where two or more heterogeneous grippers are used. We present Dexterity Network (Dex-Net) 4.0, a substantial extension to previous versions of Dex-Net that learns policies for a given set of grippers by training on synthetic datasets using domain randomization with analytic models of physics and geometry. We train policies for a parallel-jaw and a vacuum-based suction cup gripper on 5 million synthetic depth images, grasps, and rewards generated from heaps of three-dimensional objects. On a physical robot with two grippers, the Dex-Net 4.0 policy consistently clears bins of up to 25 novel objects with reliability greater than 95% at a rate of more than 300 mean picks per hour.

SWIRL: A sequential windowed inverse reinforcement learning algorithm for robot tasks with delayed rewards

We present sequential windowed inverse reinforcement learning (SWIRL), a policy search algorithm that is a hybrid of exploration and demonstration paradigms for robot learning. We apply unsupervised learning to a small number of initial expert demonstrations to structure future autonomous exploration. SWIRL approximates a long time horizon task as a sequence of local reward functions and subtask transition conditions. Over this approximation, SWIRL applies Q-learning to compute a policy that maximizes rewards. Experiments suggest that SWIRL requires significantly fewer rollouts than pure reinforcement learning and fewer expert demonstrations than behavioral cloning to learn a policy. We evaluate SWIRL in two simulated control tasks, parallel parking and a two-link pendulum. On the parallel parking task, SWIRL achieves the maximum reward on the task with 85% fewer rollouts than Q-learning, and one-eight of demonstrations needed by behavioral cloning. We also consider physical experiments on surgical tensioning and cutting deformable sheets using a da Vinci surgical robot. On the deformable tensioning task, SWIRL achieves a 36% relative improvement in reward compared with a baseline of behavioral cloning with segmentation.

Personalizing Mobile Fitness Apps using Reinforcement Learning

Despite the vast number of mobile fitness applications (apps) and their potential advantages in promoting physical activity, many existing apps lack behavior-change features and are not able to maintain behavior change motivation. This paper describes a novel fitness app called CalFit, which implements important behavior-change features like dynamic goal setting and self-monitoring. CalFit uses a reinforcement learning algorithm to generate personalized daily step goals that are challenging but attainable. We conducted the Mobile Student Activity Reinforcement (mSTAR) study with 13 college students to evaluate the efficacy of the CalFit app. The control group (receiving goals of 10,000 steps/day) had a decrease in daily step count of 1,520 (SD ± 740) between baseline and 10-weeks, compared to an increase of 700 (SD ± 830) in the intervention group (receiving personalized step goals). The difference in daily steps between the two groups was 2,220, with a statistically significant p = 0.039.

Objectively Measured Baseline Physical Activity Patterns in Women in the mPED Trial: Cluster Analysis

BACKGROUND

Determining patterns of physical activity throughout the day could assist in developing more personalized interventions or physical activity guidelines in general and, in particular, for women who are less likely to be physically active than men.

OBJECTIVE

The aims of this report are to identify clusters of women based on accelerometer-measured baseline raw metabolic equivalent of task (MET) values and a normalized version of the METs ≥3 data, and to compare sociodemographic and cardiometabolic risks among these identified clusters.

METHODS

A total of 215 women who were enrolled in the Mobile Phone Based Physical Activity Education (mPED) trial and wore an accelerometer for at least 8 hours per day for the 7 days prior to the randomization visit were analyzed. The k-means clustering method and the Lloyd algorithm were used on the data. We used the elbow method to choose the number of clusters, looking at the percentage of variance explained as a function of the number of clusters.

RESULTS

The results of the k-means cluster analyses of raw METs revealed three different clusters. The unengaged group (n=102) had the highest depressive symptoms score compared with the afternoon engaged (n=65) and morning engaged (n=48) groups (overall P<.001). Based on a normalized version of the METs ≥3 data, the moderate-to-vigorous physical activity (MVPA) evening peak group (n=108) had a higher body mass index (P=.03), waist circumference (P=.02), and hip circumference (P=.03) than the MVPA noon peak group (n=61).

CONCLUSIONS

Categorizing physically inactive individuals into more specific activity patterns could aid in creating timing, frequency, duration, and intensity of physical activity interventions for women. Further research is needed to confirm these cluster groups using a large national dataset.

TRIAL REGISTRATION

ClinicalTrials.gov NCT01280812; https://clinicaltrials.gov/ct2/show/NCT01280812 (Archived by WebCite at http://www.webcitation.org/6vVyLzwft).

Evaluating Machine Learning-Based Automated Personalized Daily Step Goals Delivered Through a Mobile Phone App: Randomized Controlled Trial

Background

Growing evidence shows that fixed, nonpersonalized daily step goals can discourage individuals, resulting in unchanged or even reduced physical activity.

Objective

The aim of this randomized controlled trial (RCT) was to evaluate the efficacy of an automated mobile phone–based personalized and adaptive goal-setting intervention using machine learning as compared with an active control with steady daily step goals of 10,000.

Methods

In this 10-week RCT, 64 participants were recruited via email announcements and were required to attend an initial in-person session. The participants were randomized into either the intervention or active control group with a one-to-one ratio after a run-in period for data collection. A study-developed mobile phone app (which delivers daily step goals using push notifications and allows real-time physical activity monitoring) was installed on each participant’s mobile phone, and participants were asked to keep their phone in a pocket throughout the entire day. Through the app, the intervention group received fully automated adaptively personalized daily step goals, and the control group received constant step goals of 10,000 steps per day. Daily step count was objectively measured by the study-developed mobile phone app.

Results

The mean (SD) age of participants was 41.1 (11.3) years, and 83% (53/64) of participants were female. The baseline demographics between the 2 groups were similar (P>.05). Participants in the intervention group (n=34) had a decrease in mean (SD) daily step count of 390 (490) steps between run-in and 10 weeks, compared with a decrease of 1350 (420) steps among control participants (n=30; P=.03). The net difference in daily steps between the groups was 960 steps (95% CI 90-1830 steps). Both groups had a decrease in daily step count between run-in and 10 weeks because interventions were also provided during run-in and no natural baseline was collected.

Conclusions

The results showed the short-term efficacy of this intervention, which should be formally evaluated in a full-scale RCT with a longer follow-up period.

Trial Registration

ClinicalTrials.gov: NCT02886871; https://clinicaltrials.gov/ct2/show/NCT02886871 (Archived by WebCite at http://www.webcitation.org/6wM1Be1Ng).

Transition state clustering: Unsupervised surgical trajectory segmentation for robot learning

Demonstration trajectories collected from a supervisor in teleoperation are widely used for robot learning, and temporally segmenting the trajectories into shorter, less-variable segments can improve the efficiency and reliability of learning algorithms. Trajectory segmentation algorithms can be sensitive to noise, spurious motions, and temporal variation. We present a new unsupervised segmentation algorithm, transition state clustering (TSC), which leverages repeated demonstrations of a task by clustering segment endpoints across demonstrations. TSC complements any motion-based segmentation algorithm by identifying candidate transitions, clustering them by kinematic similarity, and then correlating the kinematic clusters with available sensory and temporal features. TSC uses a hierarchical Dirichlet process Gaussian mixture model to avoid selecting the number of segments a priori. We present simulated results to suggest that TSC significantly reduces the number of false-positive segments in dynamical systems observed with noise as compared with seven probabilistic and non-probabilistic segmentation algorithms. We additionally compare algorithms that use piecewise linear segment models, and find that TSC recovers segments of a generated piecewise linear trajectory with greater accuracy in the presence of process and observation noise. At the maximum noise level, TSC recovers the ground truth 49% more accurately than alternatives. Furthermore, TSC runs 100× faster than the next most accurate alternative autoregressive models, which require expensive Markov chain Monte Carlo (MCMC)-based inference. We also evaluated TSC on 67 recordings of surgical needle passing and suturing. We supplemented the kinematic recordings with manually annotated visual features that denote grasp and penetration conditions. On this dataset, TSC finds 83% of needle passing transitions and 73% of the suturing transitions annotated by human experts.

D-space and Deform Closure Grasps of Deformable Parts

Building on the well-established form closure framework for holding rigid parts, in this paper we propose a new framework for holding deformable parts. We consider the class of deformable parts that can be modeled as linearly elastic polygons with a triangular finite element mesh and given stiffness matrix. We define the D-space (deformation-space) of a part as the C-space of all its mesh nodes. Free space is the intersection of the set of topology-preserving mesh configurations with the complement of the union of all D-obstacles that represent collisions of the part with finger bodies. Consider a given set of finger bodies in frictionless contact with a part. When positive work is needed to release the part, we say that it is in deform closure. We present numerical examples and prove two results: (1) if a contact set holds a rigid part in form closure, it will hold the equivalent deformable part in deform closure and (2) deform closure is frame invariant.

We then consider frictionless deform closure grasps with two contact points. We define a measure of grasp quality based on balancing the potential energy needed to release the part against the potential energy that would result in plastic deformation. Given two jaw contacts at the perimeter nodes, we develop numerical algorithms to determine the optimal jaw separation based on this metric. For a part with n mesh nodes and p perimeter nodes, we give an algorithm that computes the optimal separation in time O(n3 p2 +p6 log p) and an approximation algorithm that runs in time O(n3 p2 +(p2 /ε)log p) .

Trap Design for Vibratory Bowl Feeders

The vibratory bowl feeder is the oldest and still most common approach to the automated feeding (orienting) of industrial parts. In this paper, the authors consider a class of vibratory bowl filters that can be described by removing polygonal sections from the track; this class of filters is referred to as traps. For an n-sided polygonal part and an m-sided polygonal trap, an O(n2m log n) algorithm is given to decide whether the part in a specific orientation will safely move across the trap or will fall through the trap and thus be filtered out. For an n-sided convex polygonal part and m-sided convex polygonal trap, this bound is improved to O((n+m) log n). Furthermore, the authors show how to design various trap shapes, ranging from simple traps to general polygons, which will filter out all but one of the different stable orientations of a given part. Although the runtimes of the design algorithms are exponential in the number of trap parameters, many industrial part feeders use few-parameter traps (balconies, canyons, slots); in these cases, the running times of the algorithms range from linear to low-degree polynomial.

On the Existence of Solutions in Modular Fixturing

A fixture is a device that locates and holds parts during ma chining or assembly. A modular fixture employs reusable components on a regular lattice. Given a part, machinists combine intuition with trial and error to design an appropriate fixture. When a machinist is unable to find a design, it may be that (1) a feasible design was overlooked or (2) no feasible design exists. Complete algorithms for modular fixturing, such as those in Brost and Goldberg (1994), ensure that no fixture design is overlooked. But the question remains: are there Parts for which no modular fixture exists?

For the class of modular fixtures using three locators and a clamp, we show that there exists a class of polygonal parts that cannot be fixtured. We believe that this is the first negative re sult in the area of fixturing. We also show two positive results, namely, that a modular fixture always exists when we broaden the class of fixtures to include a T-slot and narrow the class of parts. We show that one class of fixtures strictly dominates the other. These results raise a number of open problems concern ing the existence of solutions for other classes of fixtures and parts and suggest a hierarchy of fixturing models.

1. Nguyen used the term "force-torque closure" to describe what is more commonly called form closure (Trinkle 1992).

2. Nguyen used the term independent regions of contact.

Evaluation of PC-ISO for customized, 3D Printed, gynecologic 192-Ir HDR brachytherapy applicators

The purpose of this study was to evaluate the radiation attenuation properties of PC-ISO, a commercially available, biocompatible, sterilizable 3D printing material, and its suitability for customized, single-use gynecologic (GYN) brachytherapy applicators that have the potential for accurate guiding of seeds through linear and curved internal channels. A custom radiochromic film dosimetry apparatus was 3D-printed in PC-ISO with a single catheter channel and a slit to hold a film segment. The apparatus was designed specifically to test geometry pertinent for use of this material in a clinical setting. A brachytherapy dose plan was computed to deliver a cylindrical dose distribution to the film. The dose plan used an 192Ir source and was normalized to 1500 cGy at 1 cm from the channel. The material was evaluated by comparing the film exposure to an identical test done in water. The Hounsfield unit (HU) distributions were computed from a CT scan of the apparatus and compared to the HU distribution of water and the HU distribution of a commercial GYN cylinder applicator. The dose depth curve of PC-ISO as measured by the radiochromic film was within 1% of water between 1 cm and 6 cm from the channel. The mean HU was -10 for PC-ISO and -1 for water. As expected, the honeycombed structure of the PC-ISO 3D printing process created a moderate spread of HU values, but the mean was comparable to water. PC-ISO is sufficiently water-equivalent to be compatible with our HDR brachytherapy planning system and clinical workflow and, therefore, it is suitable for creating custom GYN brachytherapy applicators. Our current clinical practice includes the use of custom GYN applicators made of commercially available PC-ISO when doing so can improve the patient's treatment. 

Motion planning with sequential convex optimization and convex collision checking

We present a new optimization-based approach for robotic motion planning among obstacles. Like CHOMP (Covariant Hamiltonian Optimization for Motion Planning), our algorithm can be used to find collision-free trajectories from naïve, straight-line initializations that might be in collision. At the core of our approach are (a) a sequential convex optimization procedure, which penalizes collisions with a hinge loss and increases the penalty coefficients in an outer loop as necessary, and (b) an efficient formulation of the no-collisions constraint that directly considers continuous-time safety Our algorithm is implemented in a software package called TrajOpt.

We report results from a series of experiments comparing TrajOpt with CHOMP and randomized planners from OMPL, with regard to planning time and path quality. We consider motion planning for 7 DOF robot arms, 18 DOF full-body robots, statically stable walking motion for the 34 DOF Atlas humanoid robot, and physical experiments with the 18 DOF PR2. We also apply TrajOpt to plan curvature-constrained steerable needle trajectories in the SE(3) configuration space and multiple non-intersecting curved channels within 3D-printed implants for intracavitary brachytherapy. Details, videos, and source code are freely available at: http://rll.berkeley.edu/trajopt/ijrr .

Risk factors for osteonecrosis of the jaws: a case-control study from the CONDOR Dental PBRN

Case reports and cohort studies have linked bisphosphonate therapy and osteonecrosis of the jaws (ONJ), but neither causality nor specific risks for lesion development have been clearly established. We conducted a 1:3 case-control study with 3 dental practice-based research networks, using dentist questionnaires and patient interviews for collection of data on bisphosphonate therapy, demographics, co-morbidities, and dental and medical treatments. Multivariable logistic regression analyses tested associations between bisphosphonate use and other risk factors with ONJ. We enrolled 191 ONJ cases and 573 controls in 119 dental practices. Bisphosphonate use was strongly associated with ONJ (odds ratios [OR] 299.5 {95% CI 70.0-1282.7} for intravenous [IV] use and OR = 12.2 {4.3-35.0} for oral use). Risk markers included local suppuration (OR = 7.8 {1.8-34.1}), dental extraction (OR = 7.6 {2.4-24.7}), and radiation therapy (OR = 24.1 {4.9-118.4}). When cancer patients (n = 143) were excluded, bisphosphonate use (OR = 7.2 {2.1-24.7}), suppuration (OR = 11.9 {2.0-69.5}), and extractions (OR = 6.6 {1.6-26.6}) remained associated with ONJ. Higher risk of ONJ began within 2 years of bisphosphonate initiation and increased 4-fold after 2 years. Both IV and oral bisphosphonate use were strongly associated with ONJ. Duration of treatment >2 years; suppuration and dental extractions were independent risk factors for ONJ.

NPIP: A skew line needle configuration optimization system for HDR brachytherapy

PURPOSE

In this study, the authors introduce skew line needle configurations for high dose rate (HDR) brachytherapy and needle planning by integer program (NPIP), a computational method for generating these configurations. NPIP generates needle configurations that are specific to the anatomy of the patient, avoid critical structures near the penile bulb and other healthy structures, and avoid needle collisions inside the body.

METHODS

NPIP consisted of three major components: a method for generating a set of candidate needles, a needle selection component that chose a candidate needle subset to be inserted, and a dose planner for verifying that the final needle configuration could meet dose objectives. NPIP was used to compute needle configurations for prostate cancer data sets from patients previously treated at our clinic. NPIP took two user-parameters: a number of candidate needles, and needle coverage radius, δ. The candidate needle set consisted of 5000 needles, and a range of δ values was used to compute different needle configurations for each patient. Dose plans were computed for each needle configuration. The number of needles generated and dosimetry were analyzed and compared to the physician implant.

RESULTS

NPIP computed at least one needle configuration for every patient that met dose objectives, avoided healthy structures and needle collisions, and used as many or fewer needles than standard practice. These needle configurations corresponded to a narrow range of δ values, which could be used as default values if this system is used in practice. The average end-to-end runtime for this implementation of NPIP was 286 s, but there was a wide variation from case to case.

CONCLUSIONS

The authors have shown that NPIP can automatically generate skew line needle configurations with the aforementioned properties, and that given the correct input parameters, NPIP can generate needle configurations which meet dose objectives and use as many or fewer needles than the current HDR brachytherapy workflow. Combined with robot assisted brachytherapy, this system has the potential to reduce side effects associated with treatment. A physical trial should be done to test the implant feasibility of NPIP needle configurations.

Assessment of Infant Movement With a Compact Wireless Accelerometer System

There is emerging data that patterns of motor activity early in neonatal life can predict impairments in neuromotor development. However, current techniques to monitor infant movement mainly rely on observer scoring, a technique limited by skill, fatigue, and inter-rater reliability. Consequently, we tested the use of a lightweight, wireless, accelerometer system that measures movement and can be worn by premature babies without interfering with routine care. We hypothesized that this system would be useful in assessing motor activity, in identifying abnormal movement, and in reducing the amount of video that a clinician would need to review for abnormal movements. Ten preterm infants in the NICU were monitored for 1 h using both the accelerometer system and video. A physical therapist trained to recognize cramped-synchronized general movements scored all of the video data by labeling each abnormal movement observed. The parameters of three different computer models were then optimized based on correlating features computed from accelerometer data and the observer’s annotations. The annotations were compared to the model’s prediction on unseen data. The trained observer identified cramped-synchronized general movements in 6 of the 10 infants. The computer models attained between 70% and 90% accuracy when predicting the same observer label for each data point. Our study suggests that mini-accelerometers may prove useful as a clinical tool assessing patterns of movement in preterm infants.

A geometric approach to robotic laundry folding

We consider the problem of autonomous robotic laundry folding, and propose a solution to the perception and manipulation challenges inherent to the task. At the core of our approach is a quasi-static cloth model which allows us to neglect the complex dynamics of cloth under significant parts of the state space, allowing us to reason instead in terms of simple geometry. We present an algorithm which, given a 2D cloth polygon and a desired sequence of folds, outputs a motion plan for executing the corresponding manipulations, deemed g-folds, on a minimal number of robot grippers. We define parametrized fold sequences for four clothing categories: towels, pants, short-sleeved shirts, and long-sleeved shirts, each represented as polygons. We then devise a model-based optimization approach for visually inferring the class and pose of a spread-out or folded clothing article from a single image, such that the resulting polygon provides a parse suitable for these folding primitives. We test the manipulation and perception tasks individually, and combine them to implement an autonomous folding system on the Willow Garage PR2. This enables the PR2 to identify a clothing article spread out on a table, execute the computed folding sequence, and visually track its progress over successive folds.

Robot-Assisted Needle Steering

Needle insertion is a critical aspect of many medical treatments, diagnostic methods, and scientific studies, and is considered to be one of the simplest and most minimally invasive medical procedures. Robot-assisted needle steering has the potential to improve the effectiveness of existing medical procedures and enable new ones by allowing increased accuracy through more dexterous control of the needle tip path and acquisition of targets not accessible by straight-line trajectories. In this article, we describe a robot-assisted needle steering system that uses three integrated controllers: a motion planner concerned with guiding the needle around obstacles to a target in a desired plane, a planar controller that maintains the needle in the desired plane, and a torsion compensator that controls the needle tip orientation about the axis of the needle shaft. Experimental results from steering an asymmetric-tip needle in artificial tissue demonstrate the effectiveness of the system and its sensitivity to various environmental and control parameters. In addition, we show an example of needle steering in ex vivo biological tissue to accomplish a clinically relevant task, and highlight challenges of practical needle steering implementation.

IPIP: A new approach to inverse planning for HDR brachytherapy by directly optimizing dosimetric indices

PURPOSE

Many planning methods for high dose rate (HDR) brachytherapy require an iterative approach. A set of computational parameters are hypothesized that will give a dose plan that meets dosimetric criteria. A dose plan is computed using these parameters, and if any dosimetric criteria are not met, the process is iterated until a suitable dose plan is found. In this way, the dose distribution is controlled by abstract parameters. The purpose of this study is to develop a new approach for HDR brachytherapy by directly optimizing the dose distribution based on dosimetric criteria.

METHODS

The authors developed inverse planning by integer program (IPIP), an optimization model for computing HDR brachytherapy dose plans and a fast heuristic for it. They used their heuristic to compute dose plans for 20 anonymized prostate cancer image data sets from patients previously treated at their clinic database. Dosimetry was evaluated and compared to dosimetric criteria.

RESULTS

Dose plans computed from IPIP satisfied all given dosimetric criteria for the target and healthy tissue after a single iteration. The average target coverage was 95%. The average computation time for IPIP was 30.1 s on an Intel(R) Core 2 Duo CPU 1.67 GHz processor with 3 Gib RAM.

CONCLUSIONS

IPIP is an HDR brachytherapy planning system that directly incorporates dosimetric criteria. The authors have demonstrated that IPIP has clinically acceptable performance for the prostate cases and dosimetric criteria used in this study, in both dosimetry and runtime. Further study is required to determine if IPIP performs well for a more general group of patients and dosimetric criteria, including other cancer sites such as GYN.

LQG-MP: Optimized path planning for robots with motion uncertainty and imperfect state information

In this paper we present LQG-MP (linear-quadratic Gaussian motion planning), a new approach to robot motion planning that takes into account the sensors and the controller that will be used during the execution of the robot’s path. LQG-MP is based on the linear-quadratic controller with Gaussian models of uncertainty, and explicitly characterizes in advance (i.e. before execution) the a priori probability distributions of the state of the robot along its path. These distributions can be used to assess the quality of the path, for instance by computing the probability of avoiding collisions. Many methods can be used to generate the required ensemble of candidate paths from which the best path is selected; in this paper we report results using rapidly exploring random trees (RRT). We study the performance of LQG-MP with simulation experiments in three scenarios: (A) a kinodynamic car-like robot, (B) multi-robot planning with differential-drive robots, and (C) a 6-DOF serial manipulator. We also present a method that applies Kalman smoothing to make paths Ck-continuous and apply LQG-MP to precomputed roadmaps using a variant of Dijkstra’s algorithm to efficiently find high-quality paths.

Risk factors for osteonecrosis of the jaws: a case-control study from the CONDOR dental PBRN

Case reports and cohort studies have linked bisphosphonate therapy and osteonecrosis of the jaws (ONJ), but neither causality nor specific risks for lesion development have been clearly established. We conducted a 1:3 case-control study with three dental Practice-based Research Networks, using dentist questionnaires and patient interviews for collection of data on bisphosphonate therapy, demographics, co-morbidities, and dental and medical treatments. Multivariable logistic regression analyses tested associations between bisphosphonate use and other risk factors with ONJ. We enrolled 191 ONJ cases and 573 controls in 119 dental practices. Bisphosphonate use was strongly associated with ONJ (odds ratios [OR] 299.5 {95%CI 70.0-1282.7} for intravenous [IV] use and OR = 12.2 {4.3-35.0} for oral use). Risk markers included local suppuration (OR = 7.8 {1.8-34.1}), dental extraction (OR = 7.6 {2.4-24.7}), and radiation therapy (OR = 24.1 {4.9-118.4}). When cancer patients (n = 143) were excluded, bisphosphonate use (OR = 7.2 {2.1-24.7}), suppuration (OR = 11.9 {2.0-69.5}), and extractions (OR = 6.6 {1.6-26.6}) remained associated with ONJ. Higher risk of ONJ began within 2 years of bisphosphonate initiation and increased four-fold after 2 years. Both IV and oral bisphosphonate use were strongly associated with ONJ. Duration of treatment > 2 years; suppuration and dental extractions were independent risk factors for ONJ.

Surgical Retraction of Non-Uniform Deformable Layers of Tissue: 2D Robot Grasping and Path Planning

Robotic surgical assistants (RSAs) have the potential to facilitate surgeries and reduce human fatigue. In this paper we focus on surgical retraction, the common surgical primitive of grasping and lifting a thin layer of tissue to expose an underlying area. Given a 2D cross-sectional model of heterogeneous tissue with embedded structures (such as veins) and a desired underlying exposure region, we present an algorithm that computes a set of stable and secure grasp-and-retract trajectories, and runs a 3D finite element (FEM) simulation to certify the quality of each trajectory. To choose secure candidate grasp locations, we introduce the continuous spring method and combine it with the Deformation Space (D-Space) approach to grasping deformable objects with a linearized potential energy model based on the locations of embedded bodies. Experiments show that this method produces many of the same grasps as an exhaustive computation with an FEM mesh, but is orders of magnitude cheaper: our method runs in O(υ log υ) time, when υ is the number of veins, while the FEM computation takes O(pn 3) time, where n is the number of nodes in the FEM mesh and p is the number of nodes on its perimeter. Furthermore, we present a constant tissue curvature (CTC) retraction trajectory that distributes strain uniformly around the medial axis of the tissue, by moving the gripper such that the tissue follows a constant-curvature, constant-length arc. 3D FEM simulations show that the CTC achieves retractions with lower tissue strain than circular and linear trajectories. Overall, our algorithm computes and certifies a high-quality retraction in about one minute on a PC.