StereoTac: a Novel Visuotactile Sensor that Combines Tactile Sensing with 3D Vision

Combining 3D vision with tactile sensing could unlock a greater level of dexterity for robots and improve several manipulation tasks. However, obtaining a close-up 3D view of the location where manipulation contacts occur can be challenging, particularly in confined spaces, cluttered environments, or without installing more sensors on the end effector. In this context, this paper presents StereoTac, a novel vision-based sensor that combines tactile sensing with 3D vision. The proposed sensor relies on stereoscopic vision to capture a 3D representation of the environment before contact and uses photometric stereo to reconstruct the tactile imprint generated by an object during contact. To this end, two cameras were integrated in a single sensor, whose interface is made of a transparent elastomer coated with a thin layer of paint with a level of transparency that can be adjusted by varying the sensor's internal lighting conditions. We describe the sensor's fabrication and evaluate its performance for both tactile perception and 3D vision. Our results show that the proposed sensor can reconstruct a 3D view of a scene just before grasping and perceive the tactile imprint after grasping, allowing for monitoring of the contact during manipulation.

Going In Blind: Object Motion Classification using Distributed Tactile Sensing for Safe Reaching in Clutter

Robotic manipulators navigating cluttered shelves or cabinets may find it challenging to avoid contact with obstacles. Indeed, rearranging obstacles may be necessary to access a target. Rather than planning explicit motions that place obstacles into a desired pose, we suggest allowing incidental contacts to rearrange obstacles while monitoring contacts for safety. Bypassing object identification, we present a method for categorizing object motions from tactile data collected from incidental contacts with a capacitive tactile skin on an Allegro Hand. We formalize tactile cues associated with categories of object motion, demonstrating that they can determine with $>90$% accuracy whether an object is movable and whether a contact is causing the object to slide stably (safe contact) or tip (unsafe).