Toward the Automation of Scientific Discovery: An Agentic AI Tab Complete VSCode Extension

Abstract

Artificial Intelligence (AI) and Large Language Models (LLMs) have entrenched themselves among the most important tools in contemporary life. AI and LLMs wield tremendous power and can be harnessed to perform the duties typically restricted to highly trained individuals. While the extent to which these tools should be incorporated into our lives is up for debate, many professionals and students now rely on these tools in their day-to-day lives. One particular area where AI stands to play a critical role is in the automation of scientific research. Skills that are required to conduct research at the highest standard, like surveying all the relevant literature, designing effective experiments, and conducting rigorous data analysis, all fall within the wheelhouse of LLMs. Although questions can be raised as to whether or not AI has gotten to the point where it can rival human researchers, human researchers can leverage AI to improve the rigor of their own experiments. This thesis aims to construct a VSCode extension that enables human researchers to invoke an AI agent to generate test cases on their code. The agent uses retrieval-augmented generation (RAG) to aid in generative test suits designed for implementations of bespoke statistical tests. Acceptance or rejection of the proposed test suit is made simple using the tab complete mechanism common to AI coding aids. By designing a user-friendly extension to generate test cases, this thesis aims to promote the adoption of thorough testing suites by ensuring researchers can easily delegate the task to agentic AI.