Le Chonk scorecard from AA and Vals: beats Opus 5.5 on legal agents, loses to GLM-5.3 on both big indexes
**TL;DR:** independent evals put Le Chonk at 38 on the AA index (pic 1), level with GPT-6 Luna and behind GLM-5.3, Kimi K3 and even DeepSeek V4.1 Flash. real wins in cyber and legal agents. preview only, weights promised by end of month.
Official post: https://mistral.ai/news/mistral-large-4/
plenty of release threads already, so i sorted the day one numbers into independent evals (Artificial Analysis, Vals) vs Mistral's own charts.
where it actually wins:
* CyberGym E2E (AA): 81.7%, #1, ahead of MiMo-V2.6-Pro 78.6 and GPT-6 Luna 77.9
* Harvey legal agent (Vals): 15.83%, #6/75. GLM-5.3 gets 8.33, Opus 5.5 gets 3.75
* Finance Agent v2 (Vals): 54.68%, just above GPT-6 Astra 53.54, still under Opus 5.5 58.59
* AA-LCR long context: 81.3%, basically tied with GPT-6 Astra 80.7
where it doesn't:
* Terminal-Bench 4 (AA): 26.8% vs GLM-5.3 41.9 and GPT-6 Astra 59.1
* Vals Index: 48.05%, #32/44, tied with Qwen 3.8 Max and behind GLM-5.3 53.51
the cyber W needs an asterisk. Opus 5.5 refused 98.5% of CyberGym tasks and GPT-6 Astra refused all of them, so "beats Opus at cyber" is mostly a refusal stat. beating Luna and MiMo there is legit tho.
the AA index in pic 1 is the part that hurts. Le Chonk costs $1.13 per index task vs $0.07 for Luna at the same 38, partly because it spat out 200M output tokens (median is 81M). a 1T model trading blows with flash tier stuff is lowkey cooked on value. huge jump from Large 3 at 9 though.
pic 2 is Mistral's own DeepSWE chart: Kimi K3 68, Le Chonk 62. the blog text only quotes the wins. the same set of charts has GLM-5.3 ahead on Terminal-Bench 4 (40 vs 28), Kimi K3 ahead on Finch (77.3 vs 67.4), and three open models above it on SciCode-Verified, where the text claims open-weight SOTA.
pricing: list is $1.36 in / $4.18 out per 1M. docs and OpenRouter show 50% off right now ($0.68 / $2.09), no end date given.
caveats: it's a preview and Mistral says the RL run is "still in flight", so scores can move. no HF repo or license yet, press says weights Oct 27. blog says 49B active, docs say 52B, either way it's ~1T total so it's not running on your 3090. gguf when
anyone running it on actual legal, finance or long doc work? do the Harvey and LCR numbers hold up for you, or does it feel like a 38?