Modelling the Volatility of the JSE Top40 Index Through GAS Models and the Hybrid GARCH–XGBoost Approach

Abstract

Modelling volatility is one of the most significant issues in financial risk management and forecasting, especially for market indices such as the JSE Top40, which are used as benchmarks for the South African stock market. This dissertation explored the volatility process of the log-returns of the JSE Top40 Index from 2011 to 2025 by combining both statistical and machine learning methods. The conditional mean was modelled with an ARMA(3,2) process, and various GARCH variants (sGARCH(1,1), GJR-GARCH(1,1), and EGARCH(1,1)) were fitted under different conditional error distributions (STD, SSTD, GED, SGED, and GHD), with model selection based on AIC, BIC, HQIC, and log-likelihood(LL). ARMA(3,2) – EGARCH(1,1) emerged as the best standalone model, capturing autocorrelation, conditional heteroskedasticity, asymmetry, and volatility persistence. Standardised residuals from this model were then used as lagged features in an XGBoost framework, producing a hybrid ARMA(3,2)– EGARCH(1,1)–XGBoost model that improved short-term volatility forecasts, outperformed the standalone model across all forecast accuracy measures (MASE, RMSE, MAE, MAPE, sMAPE) and prediction interval evaluations (PICP, PINAW, PICAW, PINAD). In parallel, a univariate GAS model with time-varying location, scale, and shape parameters (identity score scaling) was estimated on 3515 daily observations, with the Student-t GAS model (GAS–STD) achieving the lowest information criteria (AIC = 10,188.142; BIC = 10,243.626) and statistically significant persistence in location and scale. GAS-STD model provided strong density and risk forecasting, producing accurate 5% VaR and ES paths and passing coverage backtests, whereas the hybrid ARMA(3,2)-EGARCH(1,1)-XGBoost model excelled in short-horizon point forecasts (RMSE = 0.1386), confirmed by the full Diebold-Mariano (DM) test and the Model Confidence Set (MCS) procedure. Simulation experiments highlighted the sensitivity of tail behaviour to the Student-t degrees-of-freedom, with ν = 5, yielding total kurtosis of 7.32, indicating heavier tails than the Gaussian distribution. In general, this study demonstrates the complementary advantages of statistical and machine learning methods, with GAS-STD providing robust density and risk forecasts and the hybrid model yielding superior short-term volatility predictions for the JSE Top40 Index.