Billboard Hot 100 Chart-Toppers Understood: A Comprehensive Analysis of Popular Music in the 21st Century

Abstract

This paper delves into the audio and lyrical features of popular music in the 21st century, primarily focusing on hit songs in the United States that charted on the Billboard Hot 100. Historical Billboard Hot 100 charts, lyric data from Genius Lyrics, and Spotify audio feature are the three primary datasets that construct a snapshot of contemporary popular music. Exploratory data analysis and clustering techniques highlight changes and continuities within the data, while latent Dirichlet allocation (LDA) is utilized to discover the thematic topics of hit and non-hit music. The overarching goal is to classify songs as hits and non-hits based on their underlying audio and lyrical features. To achieve this, a support vector machine model (SVM) is trained and optimized. The SVM achieves an accuracy rate of 82%, mirroring the successes of other papers in the field, while adding a new dimension to the data. Beyond the core features of the project, this paper contributes to the field of hit song science (HSS) and offers a new framework to study Billboard Hot 100 hits.