Towards Memory Augmented Coding Agents

Abstract

Traditional language agents, built upon large language models, suffer from limited context windows and inefficiency in utilizing long contexts, thereby hindering long-term knowledge retention and utility. To address these challenges, we propose the augmentation of language agents with a read-write memory store that filters task-relevant information in both the writing and retrieval phases. We focus our scope on coding agents, specifically for competitive programming, developing and employing a challenging new benchmark (USACO) consisting of 307 problems from the USA Computing Olympiad. Standard models like GPT-4 exhibit modest performance on this benchmark, only achieving a 8.7% pass@1 score. Our memory-augmented coding agents demonstrate significant performance improvements, well over doubling the baseline solve rates, from a pass@1 rate of 8.7% to 20.2%. Further analysis reveals that the quality and the type of the stored content are crucial, with practical examples of code combined with reasoning being particularly impactful. Retrieved memories serve to ground models in snippet of verified code + reasoning, greatly improving performance and underscoring the advantage of memory stores that prioritize operational knowledge over theoretical content. In sum, we hope to not only establish the feasibility and efficacy of external memory stores for language agents in coding applications, but also sets a precedent for future explorations into memory-augmented cognitive models for a range of tasks, potentially transforming how agents learn and interact with information over long durations. A complete codebase can be found at https://github.com/princeton-nlp/USACO.