Hey everyone! We’re releasing new Qwen3.8-27B GGUFs with 10% higher accuracy for the same size. This uses a new version of Dynamic v3.0

Unsloth Dynamic V3 outperforms others by >10% on Div-300, KLD & more benchmarks.

We also release 1-bit quants that retain 77% accuracy. Run on 8GB RAM.

Some of you already saw we updated our quants a few hours ago. No, nothing was broken, nothing needed fixes (I don't know why people even said this since it's a complete fabricated story). This was purely an update to make them EVEN BETTER.

We do not train on the imatrix calibration dataset, and we do NOT use QAT or QAD. Everything is done through post-training quantization. Our imatrix file used is available for the community to test, evaluate, and use. We encourage researchers and developers to create variations and fine-tunes of Qwen3.8 using our Unsloth quants/imatrix. You can read our over fitting analysis as well.

Blog with all details and more benchmarks: https://unsloth.ai/docs/basics/dynamic-3.0-ggufs

GGUF: https://huggingface.co/unsloth/Qwen3.8-27B-GGUF

Enjoy! We also will be doing a new Unsloth Desktop update today: https://github.com/unslothai/unsloth

We had A LOT of updates and will be introducing auto compaction, allowing external APIs to do tool calling and more.

Qwen3.8-27B GGUF has reached 1,000 likes in less than 24 hours!

It's now the #3 trending model on Hugging Face with 1M overall downloads.

Run on 17GB RAM/VRAM setups via Unsloth! Thanks so much for your support everyone!

Model: https://huggingface.co/unsloth/Qwen3.8-27B-GGUF

Unsloth GitHub to run them: https://github.com/unslothai/unsloth

Hi LocalLlama, we're super excited to release Unsloth Desktop today! 🦥
It's the first desktop app that enables you to run and train models locally.

Open-source. Available on Mac, Windows, and Linux

• Supports MLX, diffusion image/video models, audio models, and GGUF
• You can run MiniMax-H3, Muse Glimmer, soon Qwen 3.8 etc
• Connect Claude Code and Codex to local LLMs
• 50% more accurate with self-healing tool calls and sandboxed code execution
• Supports CPU and multi-GPU setups across NVIDIA, AMD, Intel, and Mac
• Train models 2× faster while using 70% less VRAM
• Includes private web search, deep research, RAG, MCP, and exports (NVFP4, GGUF)
• Use Unsloth’s OpenAI-compatible API with OpenAI and Anthropic cloud models
• Securely deploy LLMs remotely and access them anywhere via Cloudflare HTTPS

We do not collect any telemetry or data.

Unsloth Desktop is now available on unsloth.ai and GitHub.

• GitHub: https://github.com/unslothai/unsloth
• Blog & Guide: https://unsloth.ai/docs/desktop

Thanks for the support as always and we're here to answer any questions! 💗

As per https://www.aikido.dev/blog/keyv-and-friends-compromised-in-npm-supply-chain-attack, Unsloth uses ESLint which uses keyv==4.5.4, flat-cache==4.0.1, file-entry-cache==8.0.0.

The compromised versions are keyv==6.0.0, flat-cache==6.1.24, file-entry-cache==11.1.6

Unsloth DOES NOT use the compromised versions, so you do NOT need to take any action.

However please check your own systems and other non Unsloth packages for any compromise - see https://github.com/wiz-sec-public/wiz-research-iocs/blob/main/reports/keyv-packages.csv for compromised packages

Hey guys, we signed the Open Weights letter because we believe the future of AI should be shaped by everyone, not controlled by a select few.

That belief has always been at the heart of Unsloth: everyone should be able to train and run models on their own local device.

We wouldn't be here without you guys and the wonderful open-source community! 🙏🦥

Hey r/LocalLLaMA folks! Unsloth now officially supports AMD hardware for local inference, fine-tuning, reinforcement learning, and deployment! It's been in the works for quite some time, but it works on Windows, Linux & WSL devices (+ technically Mac) with AMD GPUs!

Unsloth Studio is fully open source and free, and supports:

• Radeon RX 9000 and 7000 series
• Instinct MI350 and MI300 GPUs
• Strix Halo / Ryzen AI Max systems
• AMD CPUs for GPU-free inference

You can train models with up to 70% less VRAM, run reinforcement learning with up to 80% less VRAM, and use optimized ROCm, Triton, bitsandbytes, PyTorch, and llama.cpp builds - all installed automatically.

Linux, WSL, and macOS:

curl -fsSL https://unsloth.ai/install.sh | sh

Windows PowerShell:

irm https://unsloth.ai/install.ps1 | iex

Unsloth supports inference and training for nearly all models, including Qwen, Gemma, DeepSeek, GLM, Kimi, MiniMax, and DiffusionGemma.

You can also:

• Export models as GGUF, safetensors, or LoRA adapters
• Connect local models to Claude Code, Codex, Hermes Agent, OpenClaw, Pi, OpenCode!
• Track RAM and VRAM usage during training - remotely and locally
• Access Unsloth remotely through secure Cloudflare HTTPS tunneling - like a "LM Link"!
• Update with daily AMD-optimized llama.cpp ROCm prebuilts to reduce compilation time!

For plain pip installation:

uv pip install "unsloth[amd]"

Huge thanks to the AMD team for collaborating with us on this release! Let us know what AMD hardware you’re using and share any feedback - we'll try to make AMD much better!

More details on the release blog: https://unsloth.ai/docs/basics/amd

Hey r/LocalLLaMA folks! We made NVFP4 quants 2.5x faster for Qwen3.6 27B and also 1.56x to 1.79x faster for 35B-A3B vs NVIDIA's NVFP4 quants without any accuracy degradation! We used W4A4 so actual 4bit tensor cores for matmuls, whilst NVIDIA's ones uses W4A16.

FP8 KV Cache calibration is also provided, auto allowing 2x longer contexts. For accuracy we conducted MMLU-Pro, AIME 2025, GPQA for FP8, BF16, NVIDIA's NVFP4 and our NVFP4s. It also has MTP pre-embedded.

We also provided 2 35B versions NVFP4-Fast (1.79x faster) and NVFP4 (1.56x faster) where NVFP4-Fast fully uses W4A4 whilst NVFP4 normal uses a mixture to stay a little bit more accurate.

NVFP4 links:
Qwen3.6-35B-A3B-NVFP4 (1.56x Faster)
Qwen3.6-35B-A3B-NVFP4-Fast (1.79x Faster)
Qwen3.6-27B-NVFP4 (2.5x Faster)

Qwen3.6-27B

Provider	MMLU-Pro	GPQA	AIME 2025
Unsloth	86.25	86.34	93.12
NVIDIA	85.96	86.87	93.12
FP8	86.11	86.87	93.75
BF16	85.96	88.13	93.33

Qwen3.6-35B-A3B

Provider	MMLU-Pro	GPQA	AIME 2025
Unsloth	85.85	86.74	92.29
Unsloth Fast	85.58	87.75	91.67
NVIDIA	85.60	87.12	91.88
FP8	85.75	86.74	93.12
BF16	85.75	86.36	92.50

We have more analysis and benchmarks in our NVFP4 Qwen3.6 blog: https://unsloth.ai/docs/models/qwen3.6#nvfp4

Have a nice weekend folks!

Also for DGX Spark folks - use the flashinfer backend or you will get 2x slower inference! Our blog has more details

Hey folks! We did KLD benchmarks for our new Unsloth DeepSeek-V4-Flash GGUFs for dynamic 1-bit all the way until lossless Q8_K_XL and also Antirez's quants + Bartowski!

UD-Q8_K_XL is fully lossless as we use FP8 -> BF16 conversion and not Q8_0 - we noticed converting to Q8_0 losses a tiny bit of accuracy.

UD-Q4_K_XL is the next closest.

UD-IQ3_XXS looks to be the sweet spot for 128GB machines! We tried making the quants as useable and small as possible! You will also need to install https://github.com/ggml-org/llama.cpp/pull/25402 which fixes some issues for DS4 or use Unsloth Studio which comes with prebuilt binaries with fixes!

It fixes multi turn chat:

Engine	Tool Call score	Multi Turn tools
Official DS4 code	15/15	3/3
Any provider (Unsloth, Bartowsk, antirez) + llama.cpp	4/15	0/3
After our fix	15/15	3/3

See https://unsloth.ai/docs/models/deepseek-v4#llama.cpp-deepseek-v4-implementation-fixes for more details on the issues + benchmarks and how to run DS4!

DeepSeek releases DSpark for V4 Flash & Pro, a new speculative decoding method boosting throughput by 51% to 400% vs single MTP!

DeepSeek also showed DSpark works well for other OSS models like Gemma & Qwen in their research paper as well.

They also compared to Eagle3 and DFlash, and showed DSpark performs better as well!

• Github: https://github.com/deepseek-ai/DeepSpec
• Paper: https://github.com/deepseek-ai/DeepSpec/blob/main/DSpark_paper.pdf
• Hugging Face Model: https://huggingface.co/deepseek-ai/DeepSeek-V4-Pro-DSpark

Gemma 4 now runs 2x faster with MTP GGUFs! Run locally on just 6GB RAM. ⚡️

MTP enables Google Gemma 4 run ~1.4–2.2× faster with no accuracy loss.

Gemma 4 12B MTP can run at 162 t/s vs. 52 t/s without MTP. 31B reaches 101 t/s.

GGUFs + Guide: https://unsloth.ai/docs/models/mtp

Gemma 4 MTP now runs automatically in Unsloth Studio when you download the original Gemma 4 GGUFs. Toggle speculative decoding settings if needed, though Unsloth should auto-adjust to your hardware. See the guide above for details, and make sure you’re on the latest Unsloth version.

Hey all! Google just released Gemma-4 QAT quants for all 5 model variants (E2B, E4B, 12B, 26B-A4B, 31B) - they're trained in Q4_0 via QAT.

• We found converting BF16 QAT Q4_0 to lose some accuracy if naively doing it (most providers will be doing this)
• If you do it correctly, then E2B has a mean KLD of 0.00173 vs 0.05109 (29x better relatively) for the naive Q4_0 quantization, and the correct one is even 22% smaller!
• We found the lattice structure in llama.cpp conversion to be different from the true QAT Q4_0 in BF16 - we applied our methods to recover most of the accuracy for all Gemma-4 QAT GGUFs!
• Below is a table showing how our method makes the quant smaller yet more accurate as well!

Model	Method	Disk (GB)	99.9% KLD	Mean KLD	Top-1 %
E2B	Unsloth	2.62	0.0557	0.00173	98.16
E2B	Q4_0	3.35	1.0513	0.05109	89.29
E4B	Unsloth	4.22	0.0536	0.00121	98.54
E4B	Q4_0	5.15	0.6722	0.03778	90.94
26B	Unsloth	14.25	2.7087	0.09788	85.63
26B	Q4_0	14.44	4.5420	0.36094	70.20
31B	Unsloth	17.29	1.3659	0.01403	96.67
31B	Q4_0	17.65	3.0030	0.09349	87.91
12B	Unsloth	6.72	9.2740	0.13288	88.76
12B	Q4_0	6.98	14.7323	0.50702	74.08

We also converted the Mobile quants to GGUFs as well. We used TQ2_0 for the 2-bit layers and did a negative scaler. We made UD-Q2_K_XL quants for both E2B and E4B.

E2B mobile	E4B mobile
Size	2.19 GB
2-bit (TQ2_0) tensors	61 (incl. deep MLP)
Mean KLD vs BF16	0.00409
Top-1 %	97.82%
Base PPL	~103

See https://huggingface.co/collections/unsloth/gemma-4-qat for all QAT GGUFs!

For more details, results, we have some analysis in https://unsloth.ai/docs/models/gemma-4/qat

Hey folks! We did some KLD benchmarks of NVIDIA's latest Nemotron-3-Ultra model for our dynamic GGUFs at https://huggingface.co/unsloth/NVIDIA-Nemotron-3-Ultra-550B-A55B-GGUF

We provided both log scale and linear scale mean perplexity as per suggestions from last time!

Guide to run Nemotron-3-Ultra: https://unsloth.ai/docs/models/nemotron-3-ultra

Hey folks! We did some KLD benchmarks for StepFun's new model!

Dynamic quants are at https://huggingface.co/unsloth/Step-3.7-Flash-GGUF. Vision also works well!

The plot shows:

1. MXFP4 is known to be much worse in general - use Q4_K_XL. This is because MXFP4 tensors do not yet have imatrix support, so the calibration process doesn't work on them
2. StepFun's official GGUFs are ok, but Unsloth ones are better overall for disk space vs KLD.
3. AesSedai's ones are also reasonably good!

Hey guys, I’ll be speaking and hosting two RL workshops at Microsoft Build next week! 🚀

I’ll cover 2026 RL fundamentals and show how to RL on your local laptop with AMD and @UnslothAI.

I’ll also be in a panel discussion discussing inference & open-source on Wed, Jun 3.

RL Workshop #1 - Tue, Jun 2: https://build.microsoft.com/en-US/sessions/LABSP585?source=sessions

RL Workshop #2 - Wed, Jun 3: https://build.microsoft.com/en-US/sessions/LABSP585-R1?source=sessions

Hope to see you there, we'll also have lots of merch. 🥰

Qwen3.6 MTP Unsloth GGUFs now run **1.8x faster, increased from 1.4x just two days ago!**This is due to llama.cpp adding --spec-draft-p-min 0.75!

Args have also changed from
--spec-type mtp
to
--spec-type draft-mtp

Also increase --spec-draft-n-max 2 to 6

We also released Qwen3.5-0.8B, 2B, 4B, 9B MTP GGUFs! We'll be providing more soon!

For folks who find the new updated branch to have some perf regression, set --spec-draft-p-min to 0.0 to get the old behavior - we provided a plot of the old branch (red) vs the new branch (blue / green) as well.

Also you can use 2 speculative decoding algos - you can add ngram via --spec-type ngram-mod,draft-mtp - the perf isn't yet optimized so I'll do more benchmarks to find better numbers - see https://github.com/ggml-org/llama.cpp/pull/22673

Guide for MTP: https://unsloth.ai/docs/models/qwen3.6#mtp-guide

Hello everyone - you may have seen https://tanstack.com/blog/npm-supply-chain-compromise-postmortem

Unsloth Core & Unsloth Studio are NOT affected

Our studio/frontend/package-lock.json is pinned to versions OLDER than the malicious publications. Cross-checked against the official advisory table in GHSA-g7cv-rxg3-hmpx:

Package	Our lockfile	Compromised versions	Safe version	Status
@tanstack/history	1.161.6	1.161.9, 1.161.12	1.161.13	clean
@tanstack/react-router	1.169.2	1.169.5, 1.169.8	1.169.9	clean
@tanstack/router-core	1.169.2	1.169.5, 1.169.8	1.169.9	clean
@tanstack/react-store	0.9.3	not in advisory	--	clean
@tanstack/store	0.9.3	store family not affected	--	clean
@tanstack/react-table	8.21.3	table family not affected	--	clean
@tanstack/table-core	8.21.3	table family not affected	--	clean

Why we weren't exposed:

1. Our lockfile resolved versions are below the compromise floor. The malicious publications happened on 2026-05-11 19:20-19:26 UTC. Our lockfile was generated against package versions published BEFORE that window, so npm ci only ever pulls our pre-compromise pins.
2. All Studio CI uses npm ci, not npm install. npm ci is lockfile-strict, refuses to mutate package-lock.json, and validates every downloaded tarball against its integrity SHA. A tampered tarball with a different SHA than the lockfile would be rejected.
3. No traces of any compromised namespace anywhere. Grepped package-lock.json and confirmed zero matches for @squawk, @uipath, @tallyui, @beproduct, @mistralai, @draftlab, @draftauth, @taskflow-corp, @tolka, router_init.js, tanstack_runner.js, router_runtime.js, @tanstack/setup, the specific worm commit hash, or getsession.org.

This attack is related to https://www.reddit.com/r/unsloth/comments/1s2gxsr/unsloth_studio_not_affected_by_litellm_compromise/ LiteLLM, https://www.reddit.com/r/unsloth/comments/1t06uhk/unsloth_does_not_use_pytorch_lightning/ Lightning AI compromise

Unsloth is NOT affected by LiteLLM, Lightning AI compromises

Going forward, we are further locking down our security scans on our CI to make it even more secure for future proofing:

• We use lockfiles for ALL packages
• We auto scan pypi and npm packages in our CI which can detect these issues (AST / regex checks NOT executing code)
• CI will run on published pypi packages and published npm packages

Hey guys, we at Unsloth collaborated with NVIDIA to teach you how we made LLM training ~25% faster! 🚀

Learn how our 3 optimizations help your home GPU train models faster:

1. Packed-sequence metadata caching

2. Double-buffered checkpoint reloads

3. Faster MoE routing

Guide: https://unsloth.ai/blog/nvidia-collab

Training code GitHub: https://github.com/unslothai/unsloth

Let us know what you'd like to see next!