u/Small-Yogurtcloset57

I run PunditBench, a public benchmark that asks models for exact scores before kickoff. It publishes SHA-256 hashes and git history, so changing a miss later would invalidate the record and leave public evidence.

La Liga starts today. The Matchday 1 batch finished and was hash-locked at 06:59 UTC on 14 August, before the first match. All 42 eligible models returned predictions for all ten fixtures.

For the two games tonight:

- Alavés vs Getafe: 0 predict an Alavés win, 32 predict a draw and 10 predict a Getafe win. The modal score is 1-1 (26/42).

- Sevilla vs Rayo Vallecano: 19 predict Sevilla, 20 predict a draw and 3 predict Rayo. Here too, 1-1 is the mode (20/42); another 15 choose 2-1.

The separate preseason-table track is even more concentrated. Forty models produced valid 20-team tables; three other models still had no valid table after three attempts each, so they remain absent. Twenty-five make Real Madrid champion and 15 choose Barcelona. Every valid table has the same top-four membership: Real Madrid, Barcelona, Atlético Madrid and Villarreal. Racing Santander lands in the bottom three in 37/40 and last in 23/40.

The obvious concern is that these are not 40-plus independent forecasts. The models share training data and received the same prompt, so a consensus can just be a herd.

I put the readable summary, raw files and lock evidence in the first comment. What would you use as the fairest non-LLM baseline over a full season: Elo, a Poisson model using prior results, or a simple home/draw/away frequency model?

Follow-up to my pre-final post.

Spain won 1–0 (a.e.t.). 4 of 40 models had them as champion.

The consensus aged badly. Brazil and Germany (their top two) out before the quarters. Roughly half the field backed a team gone by the last eight.

Funny bit: the two tracks crown different winners. The pre-tournament bracket champ (Claude Fable 5) actually picked Argentina. The round-by-round winner is a $0.003 model. The $8 one is mid-table.

All public if you want to dig in (link in comments).

Real signal, or just a pricey way to copy the bookies? Might do the big leagues next season vs closing odds.

Follow-up to my June post: 40 LLMs predicted the whole World Cup before kickoff, all hashed up front so I couldn't fudge it.

The consensus aged badly — Brazil (their #1 pick) out in the R16, Germany out in the R32, France (semi favourite, 31/39) lost 0–2 to Spain. ~Half the field backed a team gone before the quarters.

Tonight, Spain v Argentina, they're deadlocked: 20 Spain, 19 Argentina — neither even a favourite.

All open — picks, raw outputs, hashes (link in comments). Is a 40-model consensus a real signal, or just a pricey way to reproduce the bookies' favourites?

follow-up to the 40-LLMs-predict-the-world-cup post. picks are all locked, so i looked at how each one bets — goals, draws, follows the crowd or not, backs favs or loves an upset.

gpt-4 is the most attacking of all 40 and basically never calls a draw. hunyuan draws almost half its games and goes against everyone. opus 4.8 just backs the favourite every single time.

it's on each model's page. which one would you trust with your bracket

Before kickoff I had 40 LLMs (GPT-3.5-era to 2026 flagships) predict all 72 group games + their own brackets — locked and SHA-256-hashed beforehand, so no test-set contamination. 20 games in:

Favorites → sharp. All 40 picked Mexico over South Africa; 26 nailed the exact 2-0.

Draws → blind. 8 of the first 20 games were draws; the models went 25/320 on them (under 8%). All 40 had Spain beating Cape Verde — it finished 0-0, nobody scored a point.

Leaderboard surprise: Claude Fable 5 is tied for 1st with Gemma 2 27B, a small 2024 open model. Llama 4 is dead last.

Open prompts, raw logs, scoring code + live leaderboard. Non-profit, no ads.

Link in comments.

Each of the 40 models (19 vendors, GPT-3.5 to 2026 flagships) predicted all 72 group matches; from its own scorelines its knockout bracket was computed, which it predicted through to its own champion. Everything — prompts, raw API logs, scoring engine, data — is in the repo; the live leaderboard is at punditbench.com. Non-profit hobby project, no ads.

Hi! I'm an old nhl games "veteran" starting from NHL 96 on pc, and played a lot of the PS2 and PS3 versions throughout games 2001-2014 or so.

The most I've enjoyed the game was the early 2010s PS3 versions GM mode.

My question is that are the current games good on that side of the game? Does it make sense to buy PS5 and the newest games just for offline gaming? If there's any oldies there who can compare the 2010s games to current ones on that sense, I'd really appreciate that!

Thanks!