SPADE: A Synthetic Paired Dataset for Specular-Diffuse Video Decomposition

Abstract

Computer vision systems struggle with specular highlights—bright spots that obscure underlying visual information—yet video-based removal methods remain unexplored due to the absence of temporally consistent training data. This thesis demonstrates that incorporating temporal information significantly improves highlight removal quality and consistency, addressing a critical gap in computational photography. I introduce SPADE, the first dataset of paired specular-diffuse video sequences, created through controlled synthetic rendering of 250 objects under varied conditions. An ablation study comparing frame-based and sequence-based neural architectures quantifies temporal processing benefits: the temporal model achieves 16.2% higher PSNR, 10.2% better SSIM, and 2.0% improved temporal consistency. Material analysis reveals these improvements are most pronounced for metallic surfaces and moderate camera movements. Beyond highlight removal, this work establishes a paradigm for leveraging temporal information in appearance decomposition tasks, with applications in augmented reality, film production, and medical imaging.