Leveraging Language to Enhance Decision-making AI

Abstract

A key feature of human intelligence is the ability to leverage language to solve tasks in completely unknown settings. Furthermore, language is the most intuitive medium for communication between humans and artificial decision-making agents. However, most learnable decision-making agents lack a natural language communication interface, limiting their generalization ability to observations in their training distribution. In this thesis, we propose several methods that learn a natural language interface for decision-making AI, enabling these agents to interact with human feedback and solve tasks by reading from text. In the first half, we present Language-guided World Models (LWMs), a model-based reinforcement learning method that learns a language-editable world model of its environment, enabling it to solve unknown environments by reading natural language manuals. In the second half, we study the capabilities of large language models (LLMs) as agents on text games, beating previous attempts to use LLMs on text games and showing that these models struggle in partially observable domains that require exploration. To this end, we augment these agents with exploration techniques (e.g. XTX) and show that combining these methods yields sample-efficient yet highly performant language agents. Finally, we provide a simple example of influencing the policy of a model-free reinforcement learning agent through iterative reward shaping with an LLM. Our work proposes several research directions for building decision-making AI that can interact with the world and communicate through natural language.