Sense, Plan, Improvise: Learning to Act Efficiently and Robustly within Bilevel Planning

Abstract

Robot manipulation is challenging due to long horizons, sparse rewards, and continuous state and action spaces. Bilevel planning is a powerful framework for its ability to decompose the long-horizon task into tractable subproblems, reason about subtask interdependencies, and find zero-shot solutions with a predefined skill library by sampling parameters that satisfy the geometric and kinematic constraints for the downstream task. The predefined skill library is often engineered or learned in factory with simplifying assumptions about the world and the robot, which can lead to failures or inefficiency at deployment. This thesis tackles two assumptions in particular: (1) the assumption about robot contact modes, and (2) the assumption of full observability and deterministic transitions. Both can be addressed by learning and adapting on the job. In the works presented in this thesis, the robot uses reinforcement learning to learn shortcuts in the abstract planning graph induced by predefined skills to relax (1) and enhance its execution efficiency. It makes closed-loop decisions about when and how to perform information gathering interleaved with compliant goal-directed manipulation to relax (2) and enhance success rates without the computational cost of belief-space planning. This thesis is an attempt to get one step closer to general-purpose robot manipulation by refining the skill library from factory via learning on the job.