Reward Prediction Errors Shape Memory during Reinforcement Learning

Abstract

In this dissertation, I characterize the role or reward prediction errors (RPEs) in shaping episodic memory across three series of behavioral experiments and computational modeling of learning and memory behavior. In Chapter 1, I show that large unsigned RPEs increase learning for those outcomes (i.e., learning rate) as well as memory for those outcome events. However, I do not find these effects to be correlated, suggesting distinct underlying mechanisms. In Chapter 2, I further test whether depressive symptoms modulate unsigned-RPE effects on learning and memory. I do not find depressive symptoms to lead to overall differences in learning and memory. Instead, I find that symptom group predicts opposite biases in the unsigned-RPE modulation of memory: in depressive participants, unsigned RPEs increased memory more for negative- versus positive-RPE events, whereas in non-depressive participants, unsigned RPEs increased memory more for positive- versus negative-RPE events. In Chapter 3, I dissociate the effects of RPEs experienced at reward cue from those at outcome on learning and memory for those events. I show, in line with classic associative models of attention, that signed RPEs at reward cue and unsigned RPEs at reward outcome modulate a dynamic learning rate in reinforcement learning models fit to behavior. When characterizing RPE effects on memory, I replicate previous results and find that unsigned RPEs at outcome enhance memory throughout learning, especially for outcome events. In addition to this, memory for cue events increases as a function of learning wherein a signed RPE at cue boosts memory for events associated with more valued reward categories. Finally, in Chapter 4, I investigate the computational mechanism supporting better memory for large unsigned-RPE events by testing whether they create event boundaries in memory. Large-RPE events are more strongly encoded and show intact associative links with their predecessors; nevertheless, they consistently disrupt the integration of events that occur across them, thereby creating event boundaries in memory. I capture these effects in a computational model of memory modified to incorporate RPEs into the encoding process. To conclude, I link my findings to interactions between reinforcement learning and memory systems, offering targets for future neuroscientific research.