How the Brain Represents Narrative

Abstract

In this thesis, we asked about the transformation of stimuli with linguistic content into patterns of activity in the brain. To what extent does this transformation generalize across people, both spatially and temporally? By what technical means might we expect to learn about or to approximate this mapping from data? Here we experimented with (a) a simple pipeline of simple techniques, (b) convolutional nets, both pre-trained and not, and (c) recurrent neural nets. We formulated this as a classification problem -- given a set of voxel time-courses corresponding with 45 seconds of recording, can we predict the label of the section of stimulus that evoked it? We find that classification accuracy is relatively invariant across methods. We achieve 88 percent accuracy on a 5-class problem, 57 percent accuracy on a 10-class problem, 46 percent accuracy on a 20-class problem, and 23 percent accuracy on a 50-class problem.