Task-Aware Robotic Grasping by evaluating Quality Diversity Solutions through Foundation Models

Task-aware robotic grasping is a challenging problem that requires the integration of semantic understanding and geometric reasoning. Traditional grasp planning approaches focus on stable or feasible grasps, often disregarding the specific tasks the robot needs to accomplish. This paper proposes a novel framework that leverages Large Language Models (LLMs) and Quality Diversity (QD) algorithms to enable zero-shot task-conditioned grasp selection. The framework segments objects into meaningful subparts and labels each subpart semantically, creating structured representations that can be used to prompt an LLM. By coupling semantic and geometric representations of an object's structure, the LLM's knowledge about tasks and which parts to grasp can be applied in the physical world. The QD-generated grasp archive provides a diverse set of grasps, allowing us to select the most suitable grasp based on the task. We evaluate the proposed method on a subset of the YCB dataset, where a Franka Emika robot is assigned to perform various actions based on object-specific task requirements. We created a ground truth by conducting a survey with six participants to determine the best grasp region for each task-object combination according to human intuition. The model was evaluated on 12 different objects across 4--7 object-specific tasks, achieving a weighted intersection over union (IoU) of 76.4% when compared to the survey data.

Flying Hydraulically Amplified Electrostatic Gripper System for Aerial Object Manipulation

Rapid and versatile object manipulation in air is an open challenge. An energy-efficient and adaptive soft gripper combined with an agile aerial vehicle could revolutionize aerial robotic manipulation in areas such as warehousing. This paper presents a bio-inspired gripper powered by hydraulically amplified electrostatic actuators mounted to a quadcopter that can interact safely and naturally with its environment. Our gripping concept is motivated by an eagle's talon. Our custom multi-actuator type is inspired by a previous scorpion tail design (consisting of a base electrode and pouches stacked adjacently) and spider-inspired joints (classic pouch motors with a flexible hinge layer). A fusion of these two concepts realizes a higher force output than single-actuator types under considerable deflections of up to 25{\deg}. By adding a sandwich hinge layer structure to the classic pouch motor concept we improve the overall robustness of the gripper. We show, for the first time, that soft manipulation in air is possible using electrostatic actuation. This study demonstrates the high potential of untethered hydraulically amplified actuators for the future of robotic manipulation. Our lightweight and bio-inspired system opens up the use of hydraulic electrostatic actuators in aerial mobile systems.