Abstract: This work studies the problem of shape reconstruction and object localization using a vision-based tactile sensor, GelSlim. The main contributions are the recovery of local shapes from contact, an approach to reconstruct the tactile shape of objects from tactile imprints, and an accurate method for object localization of previously reconstructed objects. The algorithms can be applied to a large variety of 3D objects and provide accurate tactile feedback for in-hand manipulation.
Results show that we can reconstruct the shape of objects with high accuracy and do real time object identification and localization, opening the door to more sophisticated and reactive manipulations guided by tactile sensing.
Our camera-based tactile sensor, GelSlim, provides high-resolution tactile imprints.
We provide some basic code that computes given a tactile image, the resulting local contact as a heightmap, and the 3D models of the objects used during training. Code and 3D models .
Data collection for the flaslight.
Localization and tracking in-hand of the flashlight.