Safety Monitoring for Autonomous Systems using Introspective LLMs

Abstract

As autonomous systems become increasingly deployed in real-world environments, ensuring their safety under anomalous conditions remains a critical challenge. This thesis investigates the use of large language models (LLMs) as reasoning agents for selecting appropriate fallback strategies in response to hazardous scenarios. We develop a prompting and evaluation methodology to assess an LLM agent's performance in three tasks: classifying scenarios as safe or hazardous, selecting an appropriate fallback for a hazardous observation, and predicting a set of multiple feasible fallbacks for a scenario. To improve decision quality, we apply introspective planning techniques that ground the LLM's reasoning in prior knowledge. We compare zero-shot reasoning with the introspective planning approach, demonstrating that the latter improves both accuracy and safe fallback selection. This work takes a step toward the integration of LLMs in the safety monitoring pipelines of autonomous systems.