TO PERTURB THE MACHINE: A Perturbation-Based Theory of the Creation of Visual Art Generative Models as a Proxy to Understanding Cognitive Visual Creativity

Abstract

Art and its purpose stands as a pillar of humanity, embodying invaluable expression throughout history and across peoples. The advent of artificial intelligence evidences some ability to generate art, however its outputs are contested as true to cognitive accounts and art theory. This thesis presents a unified cognitive framework on the basis of visual art creativity by synthesizing standing theories on creative art cognition, neuroscientific understanding of the visual-perceptual system and semantic networks, and artificial model attempts to emulate these behaviors as a tool. Altogether, we posit a hierarchy of representations towards visual creation starting with semantics and features, then form and shape, texture and pattern, and finally mark-making and line. These representational stages are derived from a refined reversal of the visual ventral pathway and artificial vision network architecture, and are grounded in correlates in both models. A secondary argument within this framework is that features and texture representations are grounded in semantic meaning, but form and mark-making are disconnected. This study then considers Janik’s (2023) perturbation of the BigGAN network and its curious output of artistic, essential renditions. We thus develop a perturbation-based theory where a neural correlate of perturbation of the representational stages in this novel framework, is responsible for the creation of visual art across forms and styles. A case study grounds these theories, employing layer-specific and representation-specific perturbations, which are assessed on perceptual manifestation of the four representations in this framework. Altogether, these findings scaffold the gap in explaining visual creativity, and push the interdisciplinary frontier towards an objective, functional understanding of subjective, artistic experience.