u/lexicalmaze

been building an RL agent that trades a cointegrated pair. walk forward, 14 out of sample folds, average OOS sharpe came out to 3.45. that's high enough that my default assumption is i broke something, so before i get excited i want people who've done this to tell me what i'm missing.

setup:

• PPO agent, three actions: long the spread, short the spread, flat
• trained on the in-sample window of each fold, scored only on the held-out window right after it
• 14 folds, non-overlapping, roughly 8 years of daily data
• entry and exit are the agent's call, not a fixed z-score band
• costs modeled at 5 bps per side including slippage
• fixed position sizing
• around 40 trades per fold, so it isn't one lucky trade carrying the whole thing

stuff i've already checked: features only use data up to time t, no future info in the state, folds don't overlap so nothing leaks across them, and costs aren't zero. still holds up.

what i keep coming back to:

• maybe 3.45 is just what a clean cointegrated pair gives you right until the relationship breaks, and it dies the moment the spread decoheres
• maybe i'm overfitting the pair selection itself across folds
• maybe the reward is quietly leaking something i haven't spotted

code's on my profile if you want to tear it apart. genuinely trying to find the flaw, not flex a number. where would you look first?

I've been building a backtesting framework and wanted to actually stress-test the classic strategies everyone talks about — mean reversion (Bollinger+RSI), momentum (dual EMA+ATR), breakout (Donchian+volume) — on SPY and AAPL.

Full-history results first: every single strategy underperformed buy-and-hold. SPY returned 3.3x over the period. Best strategy did 1.5x.

Then I ran proper walk-forward — tune parameters on a past window, test on the next unseen one, roll forward. Momentum on SPY: mean out-of-sample Sharpe of 0.46 across 8 yearly windows, only 5 of 8 profitable. The 2022 window trained to a 1.54 Sharpe then posted -2.05 out-of-sample.

That gap is the whole problem with most backtests you see posted online.

My question for this sub: how do you handle the parameter stability problem across regimes? Fixed windows, anchored, or expanding? And do you find walk-forward actually predictive or just another form of overfitting?

I see a lot of posts here with beautiful equity curves and no out-of-sample testing, so I ran an experiment and want to share the humbling result.

I took three textbook strategies — mean reversion (Bollinger+RSI), momentum (dual EMA), breakout (Donchian) — and tested them on SPY and AAPL. Full-history first, then proper walk-forward (tune on a past window, test on the next unseen one, roll forward).

Full-history: every strategy underperformed buy-and-hold. SPY buy-and-hold returned 3.3x; best strategy did 1.5x.

Walk-forward on momentum/SPY was the real lesson: mean out-of-sample Sharpe of 0.46, only 5 of 8 yearly windows profitable. The 2022 window trained to a 1.54 Sharpe then posted -2.05 live. That gap between in-sample confidence and out-of-sample reality is the whole game.

Takeaway I keep relearning: in-sample performance is almost meaningless. If a strategy can't survive walk-forward, it won't survive your money. Curious how others here handle the train/test split — fixed, anchored, or expanding?