Ablation Study
Remove one piece, measure the drop. Tells you which component is actually doing the work.
An ablation study removes one component at a time from a working system and measures how much performance drops. The bigger the drop, the more important that component was. It’s reverse engineering by subtraction : you bake the cake six times, each time leaving out one ingredient, to find out which ones actually matter. The name comes from medicine, where ablation means surgically removing tissue to study its function. Ablation studies justify complexity: every surviving component has proven it earns its place by degrading performance when removed.
How It Works
Run full-system baseline → for each component, run the system without it → compare each ablated score to the baseline → rank components by performance delta. Watch for synergistic pairs: removing A and B together may cause a bigger drop than removing each alone.
Example
The oil model used ablation to evaluate its AR (autoregressive) correction component. Full model: R² 0.913, MAPE 2.26%. Remove AR correction: R² drops to 0.871, MAPE rises to 3.14%. That 4.2pp gap proved the AR correction earned its complexity. Removing the news-sentiment feature caused only 0.4% change : flagged for simplification. Analysis in Oil v16.