Emotes to E-Notes: Constructing a Real-Time Emotion-Conditioned Generative Music System using Facial Recognition

Abstract

This paper presents a real-time emotion-conditioned generative music system that uses facial recognition to continuously adapt musical output to a user’s detected affective state. Emotion is represented using the valence-arousal framework derived from Russell’s circumplex model of affect. A Conditional Variational AutoEncoder (cVAE) combined with a Gated Recurrent Unit (GRU) serves as the generative backbone. This paper will focus on the construction of the real-time adaptive model as well as the literature that led to its conversation.