SLIP and SLIDE: Tackling Robot Manipulation Tasks with Iterative Shortcut Learning

Abstract

Robotic manipulation has made great strides in recent years, with imitation learning on vast datasets producing generalizable state and action abstractions, and planning algorithms stitching them together to accomplish diverse tasks, a paradigm known as Task and Motion Planning (TAMP). The hierarchical composition of abstract actions can be used to open doors and place blocks, yet the over-reliance on these fixed skills learned from potentially subpar demonstrations limits the robots of today from performing manipulations that are as fast, smooth, and complex as those done by humans. Recent work has proposed learning such "shortcuts" between abstract states via environment experience rather than imitation, but the method is limited to adding relatively short and simple skills to the TAMP graph. This thesis will investigate a novel approach to shortcut learning which has the potential for higher sample-efficiency and better generalization to long-horizon tasks, due to its innovative use of goal-conditioned reinforcement learning to estimate important state-space distances that iteratively guide skill acquisition toward greater complexity. The resulting algorithms -- Shortcut Learning with Intelligent Pruning (SLIP) and Shortcut Learning via Iterative Distance Estimation (SLIDE) -- are tested on toy problems and simulated robot manipulation tasks, exploring their effectiveness and their ability to tackle the real-world challenges that robots face. Additionally, a novel "Contrastive TAMP" method for learning new state and action abstractions from demonstrations is investigated, showing that it may be able to improve the limited applicability of iterative shortcut learning for robot manipulation tasks. The code for all methods is made publicly available, providing a jumping-off point for future research.