Towards Tangible Immersion for Cobot Programming-by-Demonstration: Visual, Tactile and Haptic Interfaces for Mixed-Reality Cobot Automation in Semiconductor Manufacturing

Sensor-based reactive and hybrid approaches have proven a promising line of study to address imperfect knowledge in grasping and manipulation. However the reactive approaches are usually tightly coupled to a particular embodiment making transfer of knowledge difficult. This paper proposes a paradigm for modeling and execution of reactive manipulation actions, which makes knowledge transfer to different embodiments possible while retaining the reactive capabilities of the embodiments. The proposed approach extends the idea of control primitives coordinated by a state machine by introducing an embodiment independent layer of abstraction. Abstract manipulation primitives constitute a vocabulary of atomic, embodiment independent actions, which can be coordinated using state machines to describe complex actions. To obtain embodiment specific models, the abstract state machines are automatically translated to embodiment specific models, such that full capabilities of each platform can be utilized. The strength of the manipulation primitives paradigm is demonstrated by developing a set of corresponding embodiment specific primitives for object transport, including a complex reactive grasping primitive. The robustness of the approach is experimentally studied in emptying of a box filled with several unknown objects. The embodiment independence is studied by performing a manipulation task on two different platforms using the same abstract description.

Intuitive and Efficient Human-robot Collaboration via Real-time Approximate Bayesian Inference

The combination of collaborative robots and end-to-end AI, promises flexible automation of human tasks in factories and warehouses. However, such promise seems a few breakthroughs away. In the meantime, humans and cobots will collaborate helping each other. For these collaborations to be effective and safe, robots need to model, predict and exploit human's intents for responsive decision making processes. Approximate Bayesian Computation (ABC) is an analysis-by-synthesis approach to perform probabilistic predictions upon uncertain quantities. ABC includes priors conveniently, leverages sampling algorithms for inference and is flexible to benefit from complex models, e.g. via simulators. However, ABC is known to be computationally too intensive to run at interactive frame rates required for effective human-robot collaboration tasks. In this paper, we formulate human reaching intent prediction as an ABC problem and describe two key performance innovations which allow computations at interactive rates. Our real-world experiments with a collaborative robot set-up, demonstrate the viability of our proposed approach. Experimental evaluations convey the advantages and value of human intent prediction for packing cooperative tasks. Qualitative results show how anticipating human's reaching intent improves human-robot collaboration without compromising safety. Quantitative task fluency metrics confirm the qualitative claims.