I spent months trying to build a profitable gold trading bot. The most valuable output was the falsification framework that proved it didn't work.
After testing multiple systematic strategies on XAUUSD M5 (trend following, mean reversion, Asian session liquidity sweeps), every approach collapsed on held-out data.
What survived the research wasn't a strategy. It was the validation tooling I built to test them:
- IS/OOS temporal splits with proper regime coverage
- Monte Carlo sequence reshuffling (1,000 simulations)
- Regime classification across macro eras (2016-2026)
- Statistical significance testing with BH-FDR correction
- ATR-normalized execution modeling
- Signal pipeline attrition diagnostics
- Robustness scoring with deployment tier classification
Most backtesting tools are built to find edge. Almost none are built to falsify it.
Wondering if there's genuine demand for a tool that systematically tries to prove your strategy is fake before you deploy capital.
Has anyone here deployed something that looked solid in backtest and failed live? What do you think would have caught it?
u/Existing_Ad_3487 — 16 days ago