Adversarial Reinforcement Learning for Safe Autonomous Vehicle Motion Planning

Abstract

Creating a safe autonomous vehicle motion planner which is robust to rare, dangerous road hazards is a critical requirement for the deployment of an autonomous vehicle system. Current learned motion planners are primarily trained on nominal datasets, which have few to none of these rare scenarios, making them vulnerable to disturbances at test time. This thesis proposes an adversarial finetuning framework for learned autonomous vehicle motion planning that produces safer driving policies by exposing these policies to generated adversarial scenarios during training. Beyond improving the safety of driving policies, this framework also admits a range of uses from offline evaluation to online decision-making. For example, the adversarial scenario generation learned through the framework can enable validation of future autonomous systems, while the framework-produced safety value function can be used to monitor safety in real-time driving scenarios and actively defend against potential road hazards. In developing this framework, this thesis also produces several pretrained transformer-based behavior prediction models, analogous to large language models in the field of natural language processing, which perform well in a variety of nominal urban driving scenarios.