Partially Observable Task and Motion Planning with Uncertainty and Risk Awareness

Aidan Curtis, George Matheos, Nishad Gothoskar, Vikash Mansinghka, Joshua Tenenbaum,, Tomás Lozano-Pérez, Leslie Pack Kaelbling
MIT EECS
Paper arXiv Code (Coming soon)

TAMPURA efficiently searching for a hidden object among many occluders

TAMPURA completing a long-horizon task while avoiding a human in the workspace

Abstract

Integrated task and motion planning (TAMP) has proven to be a valuable approach to generalizable long-horizon robotic manipulation and navigation problems. However, the typical TAMP problem formulation assumes full observability and deterministic action effects. These assumptions limit the ability of the planner to gather information and make decisions that are risk-aware. We propose a strategy for TAMP with Uncertainty and Risk Awareness (TAMPURA) that is capable of efficiently solving long-horizon planning problems with initial-state and action outcome uncertainty, including problems that require information gathering and avoiding undesirable and irreversible outcomes. Our planner reasons under uncertainty at both the abstract task level and continuous controller level. Given a set of closed-loop goal-conditioned controllers operating in the primitive action space and a description of their preconditions and potential capabilities, we learn a high-level abstraction that can be solved efficiently and then refined to continuous actions for execution. We demonstrate our approach on several robotics problems where uncertainty is a crucial factor and show that reasoning under uncertainty in these problems outperforms previously proposed determinized planning, direct search, and reinforcement learning strategies. Lastly, we demonstrate our planner on two real-world robotics problems using recent advancements in probabilistic perception.

Joint phyiscal and perceptual reasoning

The robot first looks behind the cracker box, sees the object, and then picks it up.

The cracker box is now too far away to look behind directly, so the robot must move the box out of the way to look behind it.

The block is now too close to the box to be picked without collision, so the robot must move the box before picking the block.

Probabilistically Efficient Search

The robot looks behind the largest object first, which is the most likely location of the hidden object.

The Bayes3D perception system recognizes the target object (banana) could only be behind the book.

The robot first looks behind the target objects and the underneath them until it finds the target object.

Saftey-Aware Planning

The robot must place all blocks in the bowl, but waits for the human to move away from the bowl when it expects to collide with it.

The robot must place all blocks in the bowl, but it prioritizes parts of the task that would not lead to collision with a human in its workspace.



Simulated Experiments

Physical uncertainty

In this task, the robot plays a game of shuffleboard with a puck of unknown friction. It must first push the puck around the shuffleboard to estimate its friction before attempting the shot.



Pose Uncertainty

The robot has to stack three blocks, but is uncertain about their exact pose. Directly picking the objects would result in a low probability of success, so it uses the puck to reduce pose uncertainty before stacking the blocks.



Partial Observability

The robot is searching for a small object (a die) in a cluttered scene. It can look behind objects or pick and place potential obstructors, but must avoid dropping breakable objects. What is the most efficient search strategy?



Localization and Manipulation

The robot most navigate through this 2d workspace and collect all the yellow blocks. Over time, it loses track of where it is and has to relocalize at the blue beacons.


BibTeX


@misc{curtis2024partially,
  title={Partially Observable Task and Motion Planning with Uncertainty and Risk Awareness}, 
  author={Aidan Curtis and George Matheos and Nishad Gothoskar and Vikash Mansinghka and Joshua Tenenbaum and Tomás Lozano-Pérez and Leslie Pack Kaelbling},
  year={2024},
  eprint={2403.10454},
  archivePrefix={arXiv},
  primaryClass={cs.RO}
}