Breakthrough Preferences oil

Realtime consensus pipeline: autonomous hourly model updates, cumulative MAE -49%

breakthroughoilquant-finance
Key Metric
Before
Manual parameter updates, 2-4 hours per cycle, 1-2x daily...
↓
After
Hourly autonomous updates. Cumulative v13-v17: MAE -49%, ...

A 2-4 hour manual process now runs autonomously every hour. The v13-v17 chain produced a cumulative MAE reduction of 49%. This is what the pipeline looks like when it’s finished: autonomous, continuous, and measurably better than what a human can do by hand.

Context

Even after the v13-v16 experiments produced a highly accurate pricing model, the update cycle was a bottleneck. Parameter recalibration required manual intervention: pull new price data, run the calibration scripts, evaluate the output, update the active parameters, verify no regression. This process took 2-4 hours per cycle and could only be done 1-2 times per day at best. Markets move faster than that. A model that’s only updated twice daily is operating on stale calibration during the intervening hours.

The question was whether the entire parameter update cycle could be fully automated with safety gates to prevent degradation.

What Changed

An 8-layer geopolitical consensus pipeline with 6 parallel sub-agents now runs every hour at :17. The layers address: data ingestion, geopolitical signal aggregation (6 sub-agents running in parallel), parameter candidate generation, forward sandbox gate (tests candidates against forward-looking validation data), backward sandbox gate (tests against historical backtests), anti-oscillation logic (prevents parameters from oscillating between values), tier-based parameter classification (some parameters update hourly, others only on significant signal changes), and final parameter commit.

The forward and backward sandbox gates are the safety mechanism: parameter updates are only committed if they pass both validation gates. The anti-oscillation logic prevents the system from chasing noise in short-term price movements.

Impact

Cumulative v13-v17 results across all 5 experiments: MAE reduced 49% ($1.633 to $0.833), OOS R² improved +6.24pp, Brier improved +9.93pp vs Polymarket.

Before: manual parameter updates, 2-4 hours per cycle, maximum 1-2x daily. After: fully autonomous hourly updates. Cycle 3 alone produced R² +0.0159, MAPE -0.59pp. Update cadence: 24x daily instead of 2x.

This is the culmination of the entire v13-v17 chain. Each prior experiment improved accuracy; this one removed the human constraint on how often that accuracy is maintained. The combination is an oil pricing model that continuously improves its own calibration.

Source