DeepSeek releases DSpark for V4 Flash & Pro, a new speculative decoding method boosting throughput by 51% to 400% vs single MTP!

DeepSeek also showed DSpark works well for other OSS models like Gemma & Qwen in their research paper as well.

They also compared to Eagle3 and DFlash, and showed DSpark performs better as well!

• Github: https://github.com/deepseek-ai/DeepSpec
• Paper: https://github.com/deepseek-ai/DeepSpec/blob/main/DSpark_paper.pdf
• Hugging Face Model: https://huggingface.co/deepseek-ai/DeepSeek-V4-Pro-DSpark

Gemma 4 now runs 2x faster with MTP GGUFs! Run locally on just 6GB RAM. ⚡️

MTP enables Google Gemma 4 run ~1.4–2.2× faster with no accuracy loss.

Gemma 4 12B MTP can run at 162 t/s vs. 52 t/s without MTP. 31B reaches 101 t/s.

GGUFs + Guide: https://unsloth.ai/docs/models/mtp

Gemma 4 MTP now runs automatically in Unsloth Studio when you download the original Gemma 4 GGUFs. Toggle speculative decoding settings if needed, though Unsloth should auto-adjust to your hardware. See the guide above for details, and make sure you’re on the latest Unsloth version.

Hey all! Google just released Gemma-4 QAT quants for all 5 model variants (E2B, E4B, 12B, 26B-A4B, 31B) - they're trained in Q4_0 via QAT.

• We found converting BF16 QAT Q4_0 to lose some accuracy if naively doing it (most providers will be doing this)
• If you do it correctly, then E2B has a mean KLD of 0.00173 vs 0.05109 (29x better relatively) for the naive Q4_0 quantization, and the correct one is even 22% smaller!
• We found the lattice structure in llama.cpp conversion to be different from the true QAT Q4_0 in BF16 - we applied our methods to recover most of the accuracy for all Gemma-4 QAT GGUFs!
• Below is a table showing how our method makes the quant smaller yet more accurate as well!

Model	Method	Disk (GB)	99.9% KLD	Mean KLD	Top-1 %
E2B	Unsloth	2.62	0.0557	0.00173	98.16
E2B	Q4_0	3.35	1.0513	0.05109	89.29
E4B	Unsloth	4.22	0.0536	0.00121	98.54
E4B	Q4_0	5.15	0.6722	0.03778	90.94
26B	Unsloth	14.25	2.7087	0.09788	85.63
26B	Q4_0	14.44	4.5420	0.36094	70.20
31B	Unsloth	17.29	1.3659	0.01403	96.67
31B	Q4_0	17.65	3.0030	0.09349	87.91
12B	Unsloth	6.72	9.2740	0.13288	88.76
12B	Q4_0	6.98	14.7323	0.50702	74.08

We also converted the Mobile quants to GGUFs as well. We used TQ2_0 for the 2-bit layers and did a negative scaler. We made UD-Q2_K_XL quants for both E2B and E4B.

E2B mobile	E4B mobile
Size	2.19 GB
2-bit (TQ2_0) tensors	61 (incl. deep MLP)
Mean KLD vs BF16	0.00409
Top-1 %	97.82%
Base PPL	~103

See https://huggingface.co/collections/unsloth/gemma-4-qat for all QAT GGUFs!

For more details, results, we have some analysis in https://unsloth.ai/docs/models/gemma-4/qat

Hey folks! We did some KLD benchmarks of NVIDIA's latest Nemotron-3-Ultra model for our dynamic GGUFs at https://huggingface.co/unsloth/NVIDIA-Nemotron-3-Ultra-550B-A55B-GGUF

We provided both log scale and linear scale mean perplexity as per suggestions from last time!

Guide to run Nemotron-3-Ultra: https://unsloth.ai/docs/models/nemotron-3-ultra

Hey folks! We did some KLD benchmarks for StepFun's new model!

Dynamic quants are at https://huggingface.co/unsloth/Step-3.7-Flash-GGUF. Vision also works well!

The plot shows:

1. MXFP4 is known to be much worse in general - use Q4_K_XL. This is because MXFP4 tensors do not yet have imatrix support, so the calibration process doesn't work on them
2. StepFun's official GGUFs are ok, but Unsloth ones are better overall for disk space vs KLD.
3. AesSedai's ones are also reasonably good!

Hey guys, I’ll be speaking and hosting two RL workshops at Microsoft Build next week! 🚀

I’ll cover 2026 RL fundamentals and show how to RL on your local laptop with AMD and @UnslothAI.

I’ll also be in a panel discussion discussing inference & open-source on Wed, Jun 3.

RL Workshop #1 - Tue, Jun 2: https://build.microsoft.com/en-US/sessions/LABSP585?source=sessions

RL Workshop #2 - Wed, Jun 3: https://build.microsoft.com/en-US/sessions/LABSP585-R1?source=sessions

Hope to see you there, we'll also have lots of merch. 🥰

Qwen3.6 MTP Unsloth GGUFs now run **1.8x faster, increased from 1.4x just two days ago!**This is due to llama.cpp adding --spec-draft-p-min 0.75!

Args have also changed from
--spec-type mtp
to
--spec-type draft-mtp

Also increase --spec-draft-n-max 2 to 6

We also released Qwen3.5-0.8B, 2B, 4B, 9B MTP GGUFs! We'll be providing more soon!

For folks who find the new updated branch to have some perf regression, set --spec-draft-p-min to 0.0 to get the old behavior - we provided a plot of the old branch (red) vs the new branch (blue / green) as well.

Also you can use 2 speculative decoding algos - you can add ngram via --spec-type ngram-mod,draft-mtp - the perf isn't yet optimized so I'll do more benchmarks to find better numbers - see https://github.com/ggml-org/llama.cpp/pull/22673

Guide for MTP: https://unsloth.ai/docs/models/qwen3.6#mtp-guide

Hello everyone - you may have seen https://tanstack.com/blog/npm-supply-chain-compromise-postmortem

Unsloth Core & Unsloth Studio are NOT affected

Our studio/frontend/package-lock.json is pinned to versions OLDER than the malicious publications. Cross-checked against the official advisory table in GHSA-g7cv-rxg3-hmpx:

Package	Our lockfile	Compromised versions	Safe version	Status
@tanstack/history	1.161.6	1.161.9, 1.161.12	1.161.13	clean
@tanstack/react-router	1.169.2	1.169.5, 1.169.8	1.169.9	clean
@tanstack/router-core	1.169.2	1.169.5, 1.169.8	1.169.9	clean
@tanstack/react-store	0.9.3	not in advisory	--	clean
@tanstack/store	0.9.3	store family not affected	--	clean
@tanstack/react-table	8.21.3	table family not affected	--	clean
@tanstack/table-core	8.21.3	table family not affected	--	clean

Why we weren't exposed:

1. Our lockfile resolved versions are below the compromise floor. The malicious publications happened on 2026-05-11 19:20-19:26 UTC. Our lockfile was generated against package versions published BEFORE that window, so npm ci only ever pulls our pre-compromise pins.
2. All Studio CI uses npm ci, not npm install. npm ci is lockfile-strict, refuses to mutate package-lock.json, and validates every downloaded tarball against its integrity SHA. A tampered tarball with a different SHA than the lockfile would be rejected.
3. No traces of any compromised namespace anywhere. Grepped package-lock.json and confirmed zero matches for @squawk, @uipath, @tallyui, @beproduct, @mistralai, @draftlab, @draftauth, @taskflow-corp, @tolka, router_init.js, tanstack_runner.js, router_runtime.js, @tanstack/setup, the specific worm commit hash, or getsession.org.

This attack is related to https://www.reddit.com/r/unsloth/comments/1s2gxsr/unsloth_studio_not_affected_by_litellm_compromise/ LiteLLM, https://www.reddit.com/r/unsloth/comments/1t06uhk/unsloth_does_not_use_pytorch_lightning/ Lightning AI compromise

Unsloth is NOT affected by LiteLLM, Lightning AI compromises

Going forward, we are further locking down our security scans on our CI to make it even more secure for future proofing:

• We use lockfiles for ALL packages
• We auto scan pypi and npm packages in our CI which can detect these issues (AST / regex checks NOT executing code)
• CI will run on published pypi packages and published npm packages

Hey guys, we at Unsloth collaborated with NVIDIA to teach you how we made LLM training ~25% faster! 🚀

Learn how our 3 optimizations help your home GPU train models faster:

1. Packed-sequence metadata caching

2. Double-buffered checkpoint reloads

3. Faster MoE routing

Guide: https://unsloth.ai/blog/nvidia-collab

Training code GitHub: https://github.com/unslothai/unsloth

Let us know what you'd like to see next!