Egregiously Expensive Electricity: Bringing Your Bill Back to Earth Using Weather Data in a Deterministic Weighted Linear Program to Forecast Day-Ahead Prices

Abstract

Renewable energy is beginning to power the world. In 2024, 23% of the United States' electricity came from wind, hydropower, solar, and geothermal sources (EIA). This transition away from coal and natural gas has introduced challenges, particularly the unreliability of weather-dependent generation. Wind and solar rely on stochastic natural processes to produce power, making their output difficult to predict in advance. When these assets underperform their forecasted production, local grid operators must purchase emergency electricity in the spot market from fossil-fuel-powered peaker plants—at significant financial and environmental cost. These costs are passed on to consumers, raising electricity bills and introducing instability into the system. A solution to this problem would be to have grid operators probabilistcally account for the uncertainty of renewable energy production. There exists a robust body of academic research offering stochastic alternatives to the current deterministic mixed-integer linear program (MILP) used by Independent System Grid Operators (ISOs). Unfortunately, federal regulatory agencies have rejected these proposals, citing concerns that overhauling the grid's optimization algorithm could cause unacceptable blackouts to essential public infrastructure like hospitals, water treatment facilities, and emergency response networks. This disconnect between academia, industry, and grid operators has created a critical gap in public-domain research: the need for an interpretable forecasting model that accounts for stochasticity while retaining the deterministic structure required for real-time system deployment. My thesis bridges this gap by enhancing the predictive power of deterministic day-ahead electricity pricing models by incorporating relevant weather features. My ultimate goal is to reduce consumer electricity prices and stabilize the renewable energy transition. To address this challenge, I created three deterministic optimization models that map day-ahead electricity prices to their realizations. I began by constructing a baseline model that minimizes absolute error between day-ahead and real-time prices. I then extend this model to include weather-based features such as temperature, dewpoint, and relative humidity. These variables are selected based on domain knowledge of their influence on renewable generation variability. Using data from PJM's PSEG node with the greatest price volatility, I fit a linear program that minimizes mean absolute error (MAE) between forecasted and realized prices. My final reduced model includes only the most relevant weather predictors and demonstrates improved predictive accuracy without sacrificing interpretability. My results show that incorporating weather features into a deterministic framework improves forecasting accuracy, reducing the MAE from 9.16 to 9.04. While this reduction is modest, it validates my hypothesis that deterministic models can be enhanced without requiring probabilistic and stochastic components. More importantly, my approach lays the groundwork for real-world integration, because I maintain compatibility with the current deterministic MILP structure used by ISO.