Short-Term Stock Price Prediction Using High-Dimensional Techniques

Abstract

Stock price prediction has been the focus of many studies. In the past, researchers such as Fama and French (1992) and Kothari and Shanken (1997) have identified various fundamental and technical indicators that explain stock returns. Others have used machine learning methods to produce their forecasts. For example, Chinco et al. (2017) uses the lasso across the lagged returns of all NYSE-listed stocks to find sparse predictors. In this paper, we will take a big data approach and focus on short-term prediction. In particular, we use the lasso and SVM to make rolling one-minute ahead stock return forecasts and directional predictions. Our training data is the market data from the past three minutes taken at one-minute intervals of all NYSE stocks. In our initial data set, like Chinco et al. (2017) we include the lagged returns, and in our expanded data set, we introduce the lagged volatility, volume, and bid/ask imbalance. We want to determine if there is a significant change in out-of-sample prediction performance after adding the additional variables. We find that the lasso makes a unique prediction from OLS benchmarks. Furthermore, adding the bid/ask imbalance data leads to an improvement in out-of-sample fit. With the SVM, we find that the L1 penalty performs better than the L2 penalty. Through Recursive Feature Elimination on the L2 model, we can improve the accuracy and average F1 score. Both the lasso and SVM select the most recent lag of the stock's bid/ask imbalance with higher frequency than other variables.