I backtested 100+ retail trading strategies (~700 versions) with real costs and dead companies included. Here is what survived
For the last few weeks I've been testing basically every strategy idea I came across with a custom backtesting engine built in Python with the help of Claude Opus 5.5. Chart patterns, SMC, insider buying, options flow, ML models, paper replications, AI reading news. Around 100+ ideas and ~700 versions in total. Long only, US stocks and ETFs, cash account.
Most of it didn't survive. Writing this up because the reasons they died were more useful to me than the few that worked.
Setup (so you can tell me where im wrong)
- Signal on the close, fill at the next open. No same-bar fills
- Costs 3-10 bp per side depending on the test, and every result rechecked at double costs
- Every setup compared against random entries on the same stocks with the same holding period, stop and target
- Split into a design period and an untouched period, plus walk-forward for anything with fitted parameters
- Point-in-time universe, plus a separate check that includes delisted stocks
1. Dead companies change everything
My main portfolio is 50% SPY + 50% residual momentum stocks. Using only stocks listed today, it did 20.7%/yr vs SPY 15.3% (2015-2026).
With delisted stocks kept in, the same thing did 18.1% vs 13.8% from 2015 to mid 2026. And over 1999-2019 the edge shrank to 7.9% vs 6.3%, with a worse max drawdown (-59% vs -55%).
So the edge is real-ish but a big chunk of the backtest was just the dead stocks missing.
2. Costs decide more than the signal
A ML ranking model (walk-forward, every year trained only on earlier years) had the top 20 beating the universe by 13.2%/yr before costs. At 2 bp per side 11.0%, at 5 bp 7.7%, at 10 bp 2.4%. Same predictions, the only thing that changed was the fee.
Later a bigger version (129 features, gradient boosting) had a really strong out-of-sample IC (t 7.0) and still lost to plain momentum once turnover costs were in.
3. High win rate means almost nothing
Tested SMC (order block bounce, FVG, 50% mitigation) on the 4H chart, 477 stocks, 2023-2026. The win rate looked good. But random entries with the same stop and target got the same win rate. The setup was 3.8 to 16.5 bp per trade worse than random. The win rate comes from where you put the stop and target, not from the setup. Worst trade was -43%, the stock gapped down 22% overnight.
Same story with the classic RSI(2) SPY rule: 69% winners over 10 years, +16.9% total vs +214.9% for just holding SPY.
4. Insider buying is too late for a normal person
Insider clusters (Form 4) look great in backtests that buy at the filing date. But 69% of filings are accepted after the close, and the stock has usually gapped up by the next open. Buying at the first price you can actually get: +21 bp vs the universe, t 0.7. Nothing.
5. AI reads the news right, still too slow
Gave an LLM 8-K filings published after its training cutoff so it couldnt cheat. It understood them fine. But by the first tradable price the move had already happened. Being correct about news is not the same as being early.
6. The open is expensive exists at the opening auction.
Limit orders to save the spread didn't help either. In one test ~20% of limit orders never filled, and the ones that didn't fill went on to make +1.95% on average. You save the spread on the losers and miss the winners. Net gain was about +1.4 bp, basically nothing.
7. A lot of the money is made overnight
84% of my dip-buying edge over random entries happened between the close and the next open. And over 2023-2026 SPY from 10:30am to the close lost ~3.9% a year while SPY overall was up ~22% a year. Day-trading-only strategies are swimming against that.
8. Stop losses don't protect you from gaps
On small caps (4 strategy families), trades with stops still had worst trades of -59% to -91% because the stock gapped through the stop overnight. Stops on my momentum portfolio also lowered returns and didn't really reduce drawdowns.
9. Even the best thing I found isn't proven
Once you count how many things you tried, the bar goes way up. One of my best results, judged against ~320 tries, failed Bonferroni and had a deflated Sharpe probability of 0.086, so it could be luck. The main portfolio beats SPY with t 1.8, which is suggestive, not significant. So now it runs on a forward paper record and I don't trust any backtest number until that has months behind it.
What actually held up (small stuff)
- Buy-the-dip on SPY/QQQ in uptrends: SPY 1993-2026, 804 trades, ~70% winners, ~+0.25% a trade, worst -10.9%. Youre in cash most days so on its own its only 6-9% a year
- A month-end SPY trade: 202 trades since 2002, 66% winners, ~+0.86% a trade, worst -8.1%
- Residual momentum + SPY blend above, with all the caveats
Also dead: chart pattern breakouts (231 instruments, none beat random), FVG / order blocks / S&R / candlesticks / MA crosses (1,441 stocks, ~2.4M signals, no |t| > 2.3), options flow as a stock signal (597 stocks), sector rotation, volume spikes (7 versions), earnings gap continuation, opening range breakouts, and 12 strategies from published papers. Most of the paper ones only worked with short selling, pre-2015 numbers or tiny stocks.
Happy to share more detail on any of these. And if you think a test is wrong, tell me. I'd rather find the mistake now than with real money.
What strategy do you trade that you think would survive this kind of testing?
Edit: as a commenter pointed out, the two runs in point 1 cover different windows (sep 2015-sep 2026 vs jan 2014-mid 2026), so the 20.7 vs 18.1 drop isn't all survivorship. same-date rerun coming.