Previous Projects

How much should we trust contact models and how to improve them? Our reseach explores contact datasets, data-reinforced models of frictional dynamics and stochastic simulation and plannign with data-reinforced models.

Data-Driven and Data-Augmented Contact Models



NeRF works better than RGB-D cameras or multi-view stereo when learning object descriptors.

Learning Dense Object Descriptors from NeRF



We formulate a method for estimating shape and pose of a planar object from a stream of tactile measurements.

Tactile SLAM





This work studies the problem of shape reconstruction and object localization using a vision-based tactile sensor, GelSlim.

Tactile Mapping and Localization



iNeRF performs differentiable rendering with Neural Radiance Fields (NeRF) for object pose estimation and camera tracking.

iNeRF: Inverting Neural Radiance Fields



We leverage an existing task and motion planning system to solve complex multi-step manipulation problems that involve forceful operations between the robot and the environment.

Planning Forceful Manipulation



We are interested in building abstract knowledge of physical interactions that can facilitate decision making for manipulation.

Jenga Robot





We develop an approach to robotic manipulation of unknown objects by directly estimating and controlling the forces and motions transmitted through the contact interfaces.

Contact Configuration Regulation



GelSlim 3.0 is the latest version of GelSight-type tactile-sensing fingers that integrates the ability to sense high-resolution shape, force, and slip in a more compact form factor than previous implementations, and is designed specifically for cluttered bin-picking scenarios.

GelSlim 3.0





We have considered the problem of adapting the grasp on an object by pushing it against the environment, i.e. prehensile pushing.

Prehensile Pushing





This work studies the problem of object localization from the first touch, without requiring prior contact or external perception systems.

Tactile Localization From The First Touch



In this work, we develop a perception and control framework to enable robots to accomplish the task of following a cable. We rely on a vision-based tactile sensor, GelSight, to estimate the pose of the cable in the grip as well as the friction forces during cable sliding.

Cable Manipulation





How should robots use sensed contact information? Our research explores high-resolution tactile sensing, real-time integration of vision and touch and identification of physical properties from contact.

On-line state estimation through contact



This work presents a framework for jointly learning grasping and throwing policies that enable TossingBot to pick-andthrow arbitrary objects from an unstructured bin into boxes.

TossingBot: Learning to Throw



How do we structure manipulation planning so that tactile feedback is not only relevant, but also convenient?

Tactile Dexterity





A novel representation that models objects as 3D neural fields of descriptors. We apply this representation to enable pick-and-place on unseen objects in out-of-distribution poses from a small handful of demonstrations.

Neural Descriptor Fields





We propose a method that actively estimates the location of contact between a grasped rigid object and its environment, and uses this as input to a peg-in-hole insertion policy.

Active Extrinsic Contact Sensing



The experimental materials for our tool use project. Specifically, we include the printable models of our tools and environment pieces, along with several other task descriptors.

Forceful Manipulation for Tool Use



This project covers the design of the GelSlim tactile sensor, the methods to extract useful low-level features (force and slip) from the tactile sensor and how the tactile signals are being helpful for grasping and packing tasks.

GelSlim





How can we build a fully autonomous robot system capable of warehouse autonomous, specifically warehouse fulfillment? We have explore this application of manipulation with the Amazon Robotics Challenge (2015-2017).

Amazon Robotics Challenge



How to plan/execute complex sequences of manipulation actions? Our research explores reactive control through contact.

Closed-loop Planar Pushing





In this work, we present a framework for learning pushing and grasping policies in a mutually supportive way. We show that the synergy between planning non-prehensile (pushing) and prehensile (grasping) actions can be learned from experience

Learning Synergies between Pushing and Grasping



How do we simultaneously optimize for morphology and control? Our research explores shape and motion planning for dynamic planar manipulation.

Shape & Motion Optimization for Planar Manipulation



This work focuses on developing a fast manipulation planners that can both re-plan mode sequences and recover from large perturbations.

Control for Hybrid Manipulation Primitives