EyeTunes: A Music Visualization Framework Inspired by Chromesthesia

Abstract

Across different cultures, backgrounds, and experiences, music brings humanity together to march to the same tune. Our multimedia interactions with it blend the senses and can even enhance our understanding of other intertwined disciplines. EyeTunes creates a visualization of music inspired by chromesthesia, a unique condition with which some people see shapes and colors evoked by auditory stimuli. Compiling testimonies, experimental findings, and artistic expressions from people with chromesthesia, this thesis builds a framework that maps audio input from Musical Instrument Digital Interface (MIDI) representations of music to corresponding visual output from a Java animation library and syncs them together in real time. Participants were presented with video output from EyeTunes and consistently identified matches between the animations and their corresponding songs, demonstrating that EyeTunes successfully replicates the strong and distinct associations between audio and visual. Future exploration and applications of EyeTunes include implementing it in introductory music and computer science classrooms to simultaneously augment students’ learning and comprehension of both disciplines through their interplay.