The Role of Hippocampal Place Cells as a Potential Representation of State in Reinforcement Learning for a Spatial Navigation Task

Abstract

This study examined whether local hippocampal place cell activity could function as an effective state representation in reinforcement learning for a spatial navigation task. In order to address this question, this study first investigated how hippocampal place cells would potentially represent state. While it was predicted that place cells would encode distance geodesically, as this would reflect how rats behaviorally take map distance into account during value computation (Krausz et al., 2023), no evidence was found that place cells represent map distance. This does not necessarily mean that place cells cannot function as an effective neural correlate of state. Indeed, it may be possible that place cells simply reflect important properties of value computation in another way. Beyond examining place cell properties, this study directly tested the use of local hippocampal place cell activity as the state representation in a reinforcement learning model. Specifically, the "Dual-component hex-value learner" (Krausz et al., 2023), which originally relies on manually designed bins (i.e. hexes) to represent the state, was adapted by this study. Contrary to expectations, adapted models incorporating place cell activity either as a partial or full state representation did not perform significantly differently than the original Krausz et al. (2023) hex state model. However, the fact that such adapted models performed comparably to the original model suggests that local hippocampal place cell activity could function as an alternative state representation without reducing the effectiveness of the reinforcement learning model. Moreover, there were certain limitations to the data and methodology of this study that, when addressed, could potentially result in better performance from adapted models which use hippocampal place cell activity to represent the state.