I backtested a Poisson model across 22 European leagues. It failed in five of them.
I backtested a Poisson model across 22 European leagues. It failed in five of them.
I've been building a match projection model and wanted to know where it actually works rather than assuming it works everywhere. Sharing the results because the failures turned out more interesting than the successes.
Setup
Standard Poisson approach — attack and defence strength from each team's recent matches, normalised against league scoring average, with per-league home advantage. Shrinkage toward neutral for teams with thin sample.
I ran a rolling backtest over 2025–26: replay matches in chronological order, and for each fixture the model only sees results from before that kickoff. Roughly 6,700 predictions across 22 leagues.
Overall
- 48.2% correct on 1X2
- Log loss 1.029 (random is 1.099)
- Brier 0.618 (random 0.667)
Modest, and roughly what you'd expect from a goals-only model. Calibration held up well — the 60–70% bucket landed at 62%, the 80–90% bucket at 87.7%.
Where it broke down
Five leagues came out materially worse. The pattern that surprised me: split-season formats. Austria's Bundesliga was the worst — below random. Belgium's Pro League similar. Both split into championship and relegation groups partway through, which resets the competitive structure the model assumes.
The second-tier leagues also underperformed (Championship 44%, La Liga 2 44.5%), which I'd guess is squad churn and rotation making recent form less predictive.
What I'm still stuck on
Draws. Calibration is fine in aggregate — the model says 27% and about 27% of matches draw — but there's no discrimination at the top end. The 24–27%, 27–30% and 30%+ buckets all landed within a point of each other. So I can tell you how many draws a league will have, but not which matches. Dixon-Coles is the obvious next step; hasn't been tested yet.
Also unsure whether shrinkage at k=6 is right. A parameter sweep picked 14-match windows over 8 or 20, but the k value was less clearly separated.
Happy to share the per-league breakdown if useful. Curious whether anyone else has seen the split-season effect, or found something that handles it.