Learning With Memory in Neural Networks

Abstract

This paper explores how different parameters of the memory of a recurrent neural network affect learning performance. The size of memory and the effects of long term dependencies are studied in depth for simple, yet interesting problems in which memory plays a central role. Different types of recurrent neural network architectures and structured recurrent neural networks are also explored.