Sigmoid calibration: MAE -42%, autocorrelation eliminated
MAE dropped 42% in one experiment. Autocorrelation went from 0.1651 to -0.001 : consecutive pricing errors are now statistically independent. These are the two largest single-experiment improvements in the entire v13-v17 chain, and they happened in the same run.
Context
After AR error correction (v14), the model was making better individual-step forecasts, but there was a subtler problem: the forecasting errors were correlated over time. An autocorrelation of 0.1651 means that if the model overestimates today, it’s likely to overestimate tomorrow. This is a sign that the model is missing a structural feature : some persistent market signal it’s not capturing : rather than making random errors. Correlated errors compound in multi-step forecasts.
The second problem was binary blending: the model switched between its Monte Carlo output and Polymarket market prices based on a hard threshold. A binary switch creates discontinuities in the blended probability that don’t reflect how markets actually resolve.
What Changed
Sigmoid probability blending replaced the binary threshold. Instead of snapping between model output and Polymarket prices at a hard cutoff, the blend now transitions smoothly based on a sigmoid function of the confidence differential. This eliminates the discontinuity artifact and produces more coherent probability distributions across the forecast horizon.
Simultaneously, the validation dataset was expanded from a few hundred observations to 10,114 daily WTI crude oil prices spanning 1986 to 2026 : 40 years of price history including all major shock events. This expanded dataset gives the calibration procedure far more statistical power to identify and correct structural biases.
Impact
Before: MAE $1.633, residual autocorrelation 0.1651. After: MAE $0.951 (-42%), autocorrelation -0.001.
The 42% MAE reduction is the largest single-experiment forecast accuracy gain across all v13-v17 experiments. The autocorrelation result is arguably more important: -0.001 is statistically indistinguishable from zero, meaning errors are now independent across time steps. A model with correlated errors has a structural problem that compounds as forecast horizons extend. This experiment eliminated that problem.