Uncovering Market Signals from Insider Expressions: Leveraging Machine Learning in Congressional Testimonies

Abstract

Advancements in machine learning architectures have unlocked new ways to process multimodal data traditionally inaccessible to economic analysis. We leverage deep neural networks to analyze high-profile congressional testimonies of large market cap public companies, determining if there exists correlation between facial expression, sentiment, and short-term stock price movement. We particularly use modern machine learning libraries leveraging transformer and convolutional neural network architectures, and we build full end-to-end data preprocessing pipelines to systemically analyze hours of congressional footage. Ultimately, we find a positive correlation between facial expression and sentiment, a negative correlation between facial expression and stock price movement, and no correlation between sentiment and stock price movement. Select interpretations of these unexpected results imply that perhaps more data to avoid overfitting is necessary to prove a generalizable correlation, and that perhaps off-the-shelf pretrained models are not fine-grained enough for inference on nuanced facial expression and sentiment analysis in their current states. This work lays foundations for future research into public congressional testimony analysis and demonstrates use cases for novel machine learning methods in the field of economics.