Sim-Grasp: Learning 6-DOF Grasp Policies for Cluttered Environments Using a Synthetic Benchmark

In this paper, we present Sim-Grasp, a robust 6-DOF two-finger grasping system that integrates advanced language models for enhanced object manipulation in cluttered environments. We introduce the Sim-Grasp-Dataset, which includes 1,550 objects across 500 scenarios with 7.9 million annotated labels, and develop Sim-GraspNet to generate grasp poses from point clouds. The Sim-Grasp-Polices achieve grasping success rates of 97.14% for single objects and 87.43% and 83.33% for mixed clutter scenarios of Levels 1-2 and Levels 3-4 objects, respectively. By incorporating language models for target identification through text and box prompts, Sim-Grasp enables both object-agnostic and target picking, pushing the boundaries of intelligent robotic systems.

Shape-programmable Adaptive Multi-material Microrobots for Biomedical Applications

Flagellated microorganisms can swim at low Reynolds numbers and adapt to changes in their environment. Specifically, the flagella can switch their shapes or modes through gene expression. In the past decade, efforts have been made to fabricate and investigate rigid types of microrobots without any adaptation to the environments. More recently, obtaining adaptive microrobots mimicking real microorganisms is getting more attention. However, even though some adaptive microrobots achieved by hydrogels have emerged, the swimming behaviors of the microrobots before and after the environment-induced deformations are not predicted in a systematic standardized way. In this work, experiments, finite element analysis, and dynamic modeling are presented together to realize a complete understanding of these adaptive microrobots. The above three parts are cross-verified proving the success of using such methods, facilitating the bio-applications with shape-programmable and even swimming performance-programmable microrobots. Moreover, an application of targeted object delivery using the proposed microrobot has been successfully demonstrated. Finally, cytotoxicity tests are performed to prove the potential for using the proposed microrobot for biomedical applications.

Responsive Hydrogel-based Modular Microrobots for Multi-functional Micromanipulation

Microrobots show great potential in biomedical applications such as drug delivery and cell manipulations. However, current microrobots are mostly fabricated as a single entity and type and the tasks they can perform are limited. In this paper, modular microrobots, with an overall size of 120 $\mu$m $\times$ 200 $\mu$m, are proposed with responsive mating components, made from stimuli-responsive hydrogels, and application specific end-effectors for microassembly tasks. The modular microrobots are fabricated based on photolithography and two-photon polymerization together or separately. Two types of modular microrobots are created based on the location of the responsive mating component. The first type of modular microrobot has a mating component that can shrink upon stimulation while the second type has a double bilayer structure that can realize an open and close motion. The exchange of end-effectors with an identical actuation base is demonstrated for both types of microrobots. Finally, different manipulation tasks are performed with different types of end-effectors.

Sim-Suction: Learning a Suction Grasp Policy for Cluttered Environments Using a Synthetic Benchmark

This paper presents Sim-Suction, a robust object-aware suction grasp policy for mobile manipulation platforms with dynamic camera viewpoints, designed to pick up unknown objects from cluttered environments. Suction grasp policies typically employ data-driven approaches, necessitating large-scale, accurately-annotated suction grasp datasets. However, the generation of suction grasp datasets in cluttered environments remains underexplored, leaving uncertainties about the relationship between the object of interest and its surroundings. To address this, we propose a benchmark synthetic dataset, Sim-Suction-Dataset, comprising 500 cluttered environments with 3.2 million annotated suction grasp poses. The efficient Sim-Suction-Dataset generation process provides novel insights by combining analytical models with dynamic physical simulations to create fast and accurate suction grasp pose annotations. We introduce Sim-Suction-Pointnet to generate robust 6D suction grasp poses by learning point-wise affordances from the Sim-Suction-Dataset, leveraging the synergy of zero-shot text-to-segmentation. Real-world experiments for picking up all objects demonstrate that Sim-Suction-Pointnet achieves success rates of 96.76%, 94.23%, and 92.39% on cluttered level 1 objects (prismatic shape), cluttered level 2 objects (more complex geometry), and cluttered mixed objects, respectively. The Sim-Suction policies outperform state-of-the-art benchmarks tested by approximately 21% in cluttered mixed scenes.

Sim-MEES: Modular End-Effector System Grasping Dataset for Mobile Manipulators in Cluttered Environments

In this paper, we present Sim-MEES: a large-scale synthetic dataset that contains 1,550 objects with varying difficulty levels and physics properties, as well as 11 million grasp labels for mobile manipulators to plan grasps using different gripper modalities in cluttered environments. Our dataset generation process combines analytic models and dynamic simulations of the entire cluttered environment to provide accurate grasp labels. We provide a detailed study of our proposed labeling process for both parallel jaw grippers and suction cup grippers, comparing them with state-of-the-art methods to demonstrate how Sim-MEES can provide precise grasp labels in cluttered environments.

Dynamic Simulation-Guided Design of Tumbling Magnetic Microrobots

Design of robots at the small scale is a trial-and-error based process, which is costly and time-consuming. There are few dynamic simulation tools available to accurately predict the motion or performance of untethered microrobots as they move over a substrate. At smaller length scales, the influence of adhesion and friction, which scales with surface area, becomes more pronounced. Thus, rigid body dynamic simulators, which implicitly assume that contact between two bodies can be modeled as point contact are not suitable. In this paper, we present techniques for simulating the motion of microrobots where there can be intermittent and non-point contact between the robot and the substrate. We use these techniques to study the motion of tumbling microrobots of different shapes and select shapes that are optimal for improving locomotion performance. Simulation results are verified using experimental data on linear velocity, maximum climbable incline angle, and microrobot trajectory. Microrobots with improved geometry were fabricated, but limitations in the fabrication process resulted in unexpected manufacturing errors and material/size scale adjustments. The developed simulation model is able to incorporate these limitations and emulate their effect on the microrobot's motion, reproducing the experimental behavior of the tumbling microrobots, further showcasing the effectiveness of having such a dynamic model.

Towards Dynamic Simulation Guided Optimal Design of Tumbling Microrobots

Design of robots at the small scale is a trial-and-error based process, which is costly and time-consuming. There are no good dynamic simulation tools to predict the motion or performance of a microrobot as it moves against a substrate. At smaller length scales, the influence of adhesion and friction, which scales with surface area, becomes more pronounced. Thus, rigid body dynamic simulators, which implicitly assume that contact between two bodies can be modeled as point contact are not suitable. In this paper, we present techniques for simulating the motion of microrobots where there can be intermittent and non-point contact between the robot and the substrate. We use this simulator to study the motion of microrobots of different shapes and select shapes that are most promising for performing a given task.

Control of Magnetic Microrobot Teams for Temporal Micromanipulation Tasks

In this paper, we present a control framework that allows magnetic microrobot teams to accomplish complex micromanipulation tasks captured by global Linear Temporal Logic (LTL) formulas. To address this problem, we propose an optimal control synthesis method that constructs discrete plans for the robots that satisfy both the assigned tasks as well as proximity constraints between the robots due to the physics of the problem. Our proposed algorithm relies on an existing optimal control synthesis approach combined with a novel sampling-based technique to reduce the state-space of the product automaton that is associated with the LTL specifications. The synthesized discrete plans are executed by the microrobots independently using local magnetic fields. Simulation studies show that the proposed algorithm can address large-scale planning problems that cannot be solved using existing optimal control synthesis approaches. Moreover, we present experimental results that also illustrate the potential of our method in practice. To the best of our knowledge, this is the first control framework that allows independent control of teams of magnetic microrobots for temporal micromanipulation tasks.