Using Large Language Models to Measure Investor Sentiment: Implications for Cross-Sectional Return Predictability

Abstract

We leverage recent advances in natural language processing through large language models (LLMs) to improve upon widely used measures of investor sentiment. Using a dataset containing the full texts of over 27,000 newspaper articles on the U.S. equity markets and macroeconomic conditions, we systematically prompt an LLM to extract a sentiment score from each article and aggregate these scores to construct an index of news sentiment towards the stock market. We combine this news sentiment information with traditional market-level sentiment factors to create a single, refined sentiment index that incorporates multiple information channels and is more comprehensive than traditional indices that do not incorporate financial media outlook. Using this index, we explore the implications of sentiment information for the predictability of cross-sectional returns for speculative stocks that are more susceptible to sentiment-driven mispricing, as well as across a series of long-short portfolio specifications. We find that news sentiment is a noisy metric when considered on a standalone basis. However, when we integrate news sentiment with the market-level sentiment framework presented in Baker and Wurgler (2006), we generally achieve better model fit than their original baseline approach in a long-short setting. These findings demonstrate that LLMs can extract economically meaningful information from unstructured text data and, more broadly, contribute to the growing literature on LLMs in asset pricing.