Dynamics of Recurrent Neural Network Models of Working Memory

Abstract

How do populations of neurons encode information over very short time scales? Given extensive training, neurons are able to change the weighted connections between them in order to encode information. However, very short timescales of only a few seconds are far too short to change neural weights. Still, humans and higher functioning animals pos- sess the ability to encode and maintain small amounts of information presented over very short timescales. This is the problem of working memory, the transient holding, processing, and manipulation of information used in higher cognitive functioning. Previous computa- tional models of working memory have typically been constructed with a strict, hand-tuned architecture. Here, we attempted to train a relatively simple, unconstrained neural network on complex working memory tasks and analyzed the natural solution space found by the network. Through a range of analyses, it is clear that even a simple, single layer recurrent network is capable of dynamic, generalized solutions without deliberate solution paths presented [8].