Built a walk-forward signal calibration engine from scratch. Here's what surprised me after 272 million candles
About 18 months ago I started building a rules-based signal research system for crypto. What I thought would take a few weeks turned into one of the most technically humbling projects I've ever worked on.
A few things that genuinely surprised me:
Your scoring system can be completely broken and look fine.
For weeks every signal scored exactly 55. Not 54. Not 56. Exactly 55 every time. Turns out the composite scorer had a single early return that fired whenever the local per-symbol cohort had fewer than 12 resolved trades. Instead of falling through to the family-level calibration data — which had 45,000+ resolved trades — it stamped 55 and moved on. The feed floor was 65. Nothing could ever clear it. The pipeline looked healthy. Zero users saw anything.
One function. One early return. Weeks of silent failure.
Your delivery pipeline can accept signals and still deliver nothing.
After fixing scoring, signals started clearing the floor. Users still weren't seeing them. A race condition in the fan-out process was partially completing, hitting a 500, then the retry got a duplicate signal response and returned early — without checking whether delivery rows had actually been created. Signal accepted. Row in the database. Zero deliveries. The pipeline reported success at every stage.
Gates that look protective are often just expensive.
I built a volume confirmation gate that seemed reasonable — require elevated volume before emitting a signal. After 272 million candles of walk-forward research the gate suppression audit showed this single gate was suppressing 158,000R of positive expectancy across 33,000+ missed winners. Every gate has a cost. Most people never measure it.
A 99.4% rejection rate is a feature not a bug.
The system generates roughly 26 million candidate setups lifetime. It emits about 146,000. That 99.4% rejection rate is the whole point. The hard part is knowing which gates are earning their keep and which are just creating noise.
Regime matters more than the signal.
I just ran a popular YouTube EMA crossover strategy through the same walk-forward engine. 152 resolved trades. Overall win rate 25.7%, expected R -0.72. But broken down by regime: trendUp showed 53% win rate and positive expected R. trendDown showed 8.4% win rate. Same strategy. Same rules. Completely different outcomes depending on market conditions. Regime filtering is the most underrated component in any signal system.
Currently at 272 million candles processed, 50 symbols, 4 timeframes, running 24/7. Happy to answer questions about the architecture — the gate suppression audit design and walk-forward calibration engine are the parts I find most interesting.