Employing Mechanistic Interpretability to Understand Alignment in LLMs

Abstract

Why do LLMs answer questions incorrectly, even those that seem trivial to the human mind? This thesis explores an approach to understanding Machine Learning success and failure that is centered around using Mechanistic Interpretability (MI) techniques to peer inside of LLMs, in order to shed light on their internal decision-making processes. Employing a curated dataset of simple grammatical and mathematical questions, as well as Logit Lens and Residual-Stream and Attention-Head Activation Patching, the intermediary outputs of the components of openai-community/gpt2, Qwen/Qwen2.5-1.5B, and meta-llama/Llama-3.1-8B are analyzed in-depth. Our results combine accuracy, top-1 confidence, entropy, and patched logit difference to explicate how our models generate tokens, as well as the roles of different model components in overall model performance.