The Price is (Almost) Right: A Calibration Analysis of Polymarket Pricing Data

Abstract

Prediction markets have emerged as a prominent mechanism for aggregating crowd beliefs into probability forecasts for real-world events. The world’s largest prediction market, Polymarket, explicitly claims that its trading prices are probabilities, a claim that invites rigorous empirical scrutiny. This thesis presents a dedicated, multi-domain, multi-horizon calibration study of Polymarket, analyzing 188,509 resolved binary markets and 1,875,570 daily price observations spanning November 2022 through December 2025. Calibration is assessed through reliability diagrams, Brier Score decomposition, and logistic recalibration across both market domain and time-to-resolution horizon. Uncertainty quantification employs an event-level clustered bootstrap approach that corrects for the non-independence of Polymarket’s correlated multi-market event structure, a methodological gap identified in comparable recent literature. We find that Polymarket prices demonstrate genuine forecasting skill overall, achieving a Brier Skill Score of 0.398. However, this magnitude varies substantially by domain, ranging from 0.264 in Sports (worst) to 0.565 in Politics (best). The fitted overall logistic recalibration slope b = 1.112 confirms broad underconfidence consistent with the well-documented Favorite-Longshot Bias (FLB): prices are systematically compressed toward 0.50, with longshots overpriced and favorites underpriced. This bias is universally present across all market domains and worsens at longer time horizons. Clustered bootstrapping significantly widens confidence intervals relative to naive inference: while the core miscalibration finding proves robust, two horizon bands lose statistical significance, showing that ignoring clustering would produce incorrect inferences. These findings replicate and extend FLB patterns documented on prior prediction market platforms, and suggest that domain- and horizon-specific recalibration could improve the interpretation of raw Polymarket prices.