Identifying Concepts Used by Human-Like Neural Network Chess Engines

Abstract

As neural networks reach expert-level performance in many domains, there is growing interest in understanding the internal representations that guide their decisions. Models trained to mimic human decisions, rather than strive for optimal performance, offer a particularly interesting testbed for interpretability. Identifying what information these networks encode can generate hypotheses about what information humans rely on when performing similar tasks. Chess presents a well-suited domain to investigate this as neural network-based chess engines have achieved superhuman performance in the game, and previous work has demonstrated that human-interpretable chess concepts are decodable from their latent representations. In this thesis, I extend this line of work by applying the technique of linear probing to Maia chess engines, a family of neural network-based chess engines that are trained to emulate human play, and Leela Chess Zero, a superhuman chess engine trained through self-play. I find that human-interpretable chess concepts are linearly decodable from the internal activations. In the Maia models, decodability increases with human-emulated skill level for most concepts and also varies with network depth. Preliminary evidence supports the idea that concept complexity influences the relationship between layer depth and decodability: the simpler the concept, the earlier it peaks in the network’s activation. These results are mirrored in Leela despite it never having seen a human game. Altogether, these results suggest that human understandable chess concepts are useful representations that are employed by neural network-based chess engines and how task representations may shift with expertise.