Testing schedule and travel features on an NHL prediction model: accuracy improved, calibration didn't
I've been building an NHL prediction model in Python and recently tested whether adding schedule and travel information would improve its performance.
I evaluated the changes on 2,614 historical NHL games.
The baseline model had:
Accuracy: 55.09%
Brier score: 0.244975
The combined feature version had:
Accuracy: 56.27%
Brier score: 0.245381
So the new version classified more winners correctly, but its probability calibration was slightly worse.
I found that pretty interesting because I'd originally assumed incorporating more context would improve the model overall.
The features I tested included rest days, back-to-backs, travel/time-zone changes and home/away performance.
For now I've left the baseline in place rather than treating the higher accuracy as an automatic improvement.
I've also been collecting live predictions separately. Across the first 100 published predictions, 89 had decisive outcomes at the time of the audit, with 46 wins and 43 losses.
Still a small sample, and I haven't demonstrated a profitable betting edge.
For anyone working on sports prediction models, would you prioritize improving probability calibration first, or classification accuracy?
I'm leaning toward calibration, especially if the eventual goal is comparing model probabilities against market prices.