PATCH: Program-Aided Tuning of Control Heuristics

Abstract

Large language models are being continuously used to create executable control policies from natural language task descriptions. However, these policies often require a lot of numerical tuning and structural correction specific to a task or environment before they become reliable. This project studies a program synthesis driven workflow in which LLMs propose parametric policy structures and classical optimization refines their parameters. We introduce and test five approaches on several environments of ranging complexity: a human written expert controller, a single prompt LLM policy, a iterative LLM refinement pipeline, a LLM synthesized policy using a 1-D grid search, and an iterative LLM refinement and grid search policy. Together, these experiments investigate how program synthesis, parameter tuning, and feedback driven revision interact to create interpretable control policies.