u/Relevant_Boss3271

**TL;DR:** independent evals put Le Chonk at 38 on the AA index (pic 1), level with GPT-6 Luna and behind GLM-5.3, Kimi K3 and even DeepSeek V4.1 Flash. real wins in cyber and legal agents. preview only, weights promised by end of month.

Official post: https://mistral.ai/news/mistral-large-4/

plenty of release threads already, so i sorted the day one numbers into independent evals (Artificial Analysis, Vals) vs Mistral's own charts.

where it actually wins:

* CyberGym E2E (AA): 81.7%, #1, ahead of MiMo-V2.6-Pro 78.6 and GPT-6 Luna 77.9

* Harvey legal agent (Vals): 15.83%, #6/75. GLM-5.3 gets 8.33, Opus 5.5 gets 3.75

* Finance Agent v2 (Vals): 54.68%, just above GPT-6 Astra 53.54, still under Opus 5.5 58.59

* AA-LCR long context: 81.3%, basically tied with GPT-6 Astra 80.7

where it doesn't:

* Terminal-Bench 4 (AA): 26.8% vs GLM-5.3 41.9 and GPT-6 Astra 59.1

* Vals Index: 48.05%, #32/44, tied with Qwen 3.8 Max and behind GLM-5.3 53.51

the cyber W needs an asterisk. Opus 5.5 refused 98.5% of CyberGym tasks and GPT-6 Astra refused all of them, so "beats Opus at cyber" is mostly a refusal stat. beating Luna and MiMo there is legit tho.

the AA index in pic 1 is the part that hurts. Le Chonk costs $1.13 per index task vs $0.07 for Luna at the same 38, partly because it spat out 200M output tokens (median is 81M). a 1T model trading blows with flash tier stuff is lowkey cooked on value. huge jump from Large 3 at 9 though.

pic 2 is Mistral's own DeepSWE chart: Kimi K3 68, Le Chonk 62. the blog text only quotes the wins. the same set of charts has GLM-5.3 ahead on Terminal-Bench 4 (40 vs 28), Kimi K3 ahead on Finch (77.3 vs 67.4), and three open models above it on SciCode-Verified, where the text claims open-weight SOTA.

pricing: list is $1.36 in / $4.18 out per 1M. docs and OpenRouter show 50% off right now ($0.68 / $2.09), no end date given.

caveats: it's a preview and Mistral says the RL run is "still in flight", so scores can move. no HF repo or license yet, press says weights Oct 27. blog says 49B active, docs say 52B, either way it's ~1T total so it's not running on your 3090. gguf when

anyone running it on actual legal, finance or long doc work? do the Harvey and LCR numbers hold up for you, or does it feel like a 38?

I benchmarked FreeToken 0.1.2 on an RTX 5070 Ti Laptop with 12 GB VRAM, i9-13900HX and 32 GB RAM under WSL2.

The goal was to measure how FreeToken behaves as GPU expert-cache capacity is reduced, rather than just reporting a single tok/s number.

I tested:

- Qwen3-4B BF16 fully fitting in VRAM

- GPT-OSS-20B with FreeToken's automatic expert cache

- GPT-OSS-20B with a heavily restricted 32-slot GPU expert cache

- five prompt sizes from ~1K to ~28.7K tokens

- a separate expert-cache sweep from 32 to 482 GPU slots

Main result: with the automatic 482-slot cache, GPT-OSS-20B stayed around 61–71 tok/s across the tested prompt lengths. With only 32 expert slots on GPU, decode dropped to about 8.7 tok/s, while TTFT increased significantly.

The cache-size sweep also shows how quickly throughput falls as more experts have to be streamed from system memory.

All runs were sequential, with one model server and one request active at a time. Each measured point was repeated three times after a warm-up.

I tried to keep the report purely measurement-based rather than adding a subjective review.

Feedback on the methodology is welcome, especially from people testing FreeToken on 12–16 GB GPUs.