When Art Finds Its Words: Enhancing Aesthetic Experience with Computational Pairing

Abstract

This thesis explores the benefits and applications of pairing different modalities of art, specifically focusing on computational approaches to match paintings with poetry. I began by building a theoretical framework, investigating how art pairing might make art appreciation more accessible by creating multiple pathways for engagement. To evaluate this claim, I developed and tested three distinct computational pairing frameworks: CLIP scoring based pairing, emotional similarity based matching, and object-based correlation. Through a survey-based experiment with 54 participants, I compared these frameworks against random pairing to measure changes in emotional response and engagement quality. Results indicate that each framework generally outperformed random pairing, offering modest increases in appreciation. Qualitative analysis of responses showed a shift from perceptual analysis to interpretive meaning-making when viewing pairings, suggesting that pairing creates more reflective engagement. Based on these findings, I developed a gallery-style application that demonstrates these methodologies. This research aims to contribute to under- standing how computational approaches can support aesthetic engagement, doing so through pairing.