Spatiotemporal Neural Processing of Naturalistic Stimulus Properties

Abstract

Humans possess the ability to process, store, and recall naturalistic, narrative stimuli such as audio-visual movies, spoken stories, and written stories. Neural processing of such narrative stimuli has generally been shown to occur in a hierarchical manner — both temporally and spatially — promoting further research into the precise spatiotemporal extent of this processing hierarchy. In this study, we investigate the possibility of using encoding models of various stimulus properties, including vectorized embeddings from machine learning models, in conjunction with electrocorticography (capable of recording neural activity with high temporal and spatial resolution) to identify neural representations of naturalistic stimulus properties and trace the hierarchical flow of information processed from a single stimulus (movie clips) on a fine-grained timescale. We find clear evidence of a hierarchical auditory processing pathway as well as limited evidence of hierarchical visual processing. The results of this research provide insight into narrative-induced neural functioning and, in conjunction with future work evaluating the implications of hierarchical stimulus processing on episodic memory, have the potential for broad implications on teaching and learning.