Unified Task and Motion Planning using Object-centric Abstractions of Motion Constraints

In task and motion planning (TAMP), the ambiguity and underdetermination of abstract descriptions used by task planning methods make it difficult to characterize physical constraints needed to successfully execute a task. The usual approach is to overlook such constraints at task planning level and to implement expensive sub-symbolic geometric reasoning techniques that perform multiple calls on unfeasible actions, plan corrections, and re-planning until a feasible solution is found. We propose an alternative TAMP approach that unifies task and motion planning into a single heuristic search. Our approach is based on an object-centric abstraction of motion constraints that permits leveraging the computational efficiency of off-the-shelf AI heuristic search to yield physically feasible plans. These plans can be directly transformed into object and motion parameters for task execution without the need of intensive sub-symbolic geometric reasoning.

Long-Horizon Task and Motion Planning with Functional Object-Oriented Networks

Following work on joint object-action representations, functional object-oriented networks (FOON) were introduced as a knowledge graph representation for robots. Taking the form of a bipartite graph, a FOON contains symbolic (high-level) concepts useful to a robot's understanding of tasks and its environment for object-level planning. Prior to this paper, little has been done to demonstrate how task plans acquired from FOON via task tree retrieval can be executed by a robot, as the concepts in a FOON are too abstract for immediate execution. We propose a hierarchical task planning approach that translates a FOON graph into a PDDL-based representation of domain knowledge for manipulation planning. As a result of this process, a task plan can be acquired that a robot can execute from start to end, leveraging the use of action contexts and skills in the form of dynamic movement primitives (DMP). We demonstrate the entire pipeline from planning to execution using CoppeliaSim and show how learned action contexts can be extended to never-before-seen scenarios.

A Road-map to Robot Task Execution with the Functional Object-Oriented Network

Following work on joint object-action representations, the functional object-oriented network (FOON) was introduced as a knowledge graph representation for robots. Taking the form of a bipartite graph, a FOON contains symbolic or high-level information that would be pertinent to a robot's understanding of its environment and tasks in a way that mirrors human understanding of actions. In this work, we outline a road-map for future development of FOON and its application in robotic systems for task planning as well as knowledge acquisition from demonstration. We propose preliminary ideas to show how a FOON can be created in a real-world scenario with a robot and human teacher in a way that can jointly augment existing knowledge in a FOON and teach a robot the skills it needs to replicate the demonstrated actions and solve a given manipulation problem.

Efficient State Abstraction using Object-centered Predicates for Manipulation Planning

The definition of symbolic descriptions that consistently represent relevant geometrical aspects in manipulation tasks is a challenging problem that has received little attention in the robotic community. This definition is usually done from an observer perspective of a finite set of object relations and orientations that only satisfy geometrical constraints to execute experiments in laboratory conditions. This restricts the possible changes with manipulation actions in the object configuration space to those compatible with that particular external reference definitions, which greatly limits the spectrum of possible manipulations. To tackle these limitations we propose an object-centered representation that permits characterizing a much wider set of possible changes in configuration spaces than the traditional observer perspective counterpart. Based on this representation, we define universal planning operators for picking and placing actions that permits generating plans with geometric and force consistency in manipulation tasks. This object-centered description is directly obtained from the poses and bounding boxes of objects using a novel learning mechanisms that permits generating signal-symbols relations without the need of handcrafting these relations for each particular scenario.