Model predictive manipulation of compliant objects with multi-objective optimizer and adversarial network for occlusion compensation

BACKGROUND

The manipulation of compliant objects by robotic systems remains a challenging task, largely due to their variable shapes and the complex, high-dimensional nature of their interaction dynamics. Traditional robotic manipulation strategies struggle with the accurate modeling and control necessary to handle such materials, especially in the presence of visual occlusions that frequently occur in dynamic environments. Meanwhile, for most unstructured environments, robots are required to have autonomous interactions with their surroundings.

METHODS

To solve the shape manipulation of compliant objects in an unstructured environment, we begin by exploring the regression-based algorithm of representing the high-dimensional configuration space of deformable objects in a compressed form that enables efficient and effective manipulation. Simultaneously, we address the issue of visual occlusions by proposing the integration of an adversarial network, enabling guiding the shaping task even with partial observations of the object. Afterwards, we propose a receding-time estimator to coordinate the robot action with the computed shape features while satisfying various performance criteria. Finally, model predictive controller is utilized to compute the robot's shaping motions subject to safety constraints. Detailed experiments are presented to evaluate the proposed manipulation framework.

SIGNIFICANT FINDINGS

Our MPC framework utilizes the compressed representation and occlusion-compensated information to predict the object's behavior, while the multi-objective optimizer ensures that the resulting control actions meet multiple performance criteria. Through rigorous experimental validation, our approach demonstrates superior manipulation capabilities in scenarios with visual obstructions, outperforming existing methods in terms of precision and operational reliability. The findings highlight the potential of our integrated approach to significantly enhance the manipulation of compliant objects in real-world robotic applications.

Model-Based Manipulation of Linear Flexible Objects: Task Automation in Simulation and Real World

Manipulation of deformable objects is a desired skill in making robots ubiquitous in manufacturing, service, healthcare, and security. Common deformable objects (e.g., wires, clothes, bed sheets, etc.) are significantly more difficult to model than rigid objects. In this research, we contribute to the model-based manipulation of linear flexible objects such as cables. We propose a 3D geometric model of the linear flexible object that is subject to gravity and a physical model with multiple links connected by revolute joints and identified model parameters. These models enable task automation in manipulating linear flexible objects both in simulation and real world. To bridge the gap between simulation and real world and build a close-to-reality simulation of flexible objects, we propose a new strategy called Simulation-to-Real-to-Simulation (Sim2Real2Sim). We demonstrate the feasibility of our approach by completing the Plug Task used in the 2015 DARPA Robotics Challenge Finals both in simulation and real world, which involves unplugging a power cable from one socket and plugging it into another. Numerical experiments are implemented to validate our approach.