u/stereohype

Dense 27B, meet DFlash2. On my Flow Z13 (Ryzen AI Max+ 395, Radeon 8060S, 128GB), Qwen3.8-27B hits 31 t/s burst at 80W and runs 26-29 t/s through an entire 32k-token generation, with every token paying for all 27B parameters. No MoE shortcuts this time.

Three things got it there: Inco AI's DFlash2 drafter, n-max 4, and a chat template that stops the model from drowning in its own reasoning. The last one surprised me most. The stock template's runaway thinking didn't just waste tokens, it collapsed draft acceptance at depth; fix the style and the speed decay disappears.

Last week's DeepSeek V4 Flash guide (https://www.reddit.com/r/LocalLLaMA/comments/1vlmh0b) covered the MoE side of this hardware. This post is the dense half.

Note: the writing is AI-assisted editing; the research, debugging, and every number are from my own runs on this machine.

Edit (Aug 20): the recommended config was revised after the burst table below was measured. Three findings landed on top of the initial numbers: n-max 4 beats 7 by 25% on long generations, the Sharp chat template fixes a runaway-reasoning failure mode and lifts sustained decode another 26%, and q8_0 drafter KV is a free +1.6% with bit-identical output. The launch command reflects all three; sustained expectations are now 26-29 t/s from 1k to 32k output tokens.

Caveat: these are early benchmarks. The sustained matrix (both quants to 8k, Q5 to 64k), the Unsloth v2/v3 A/B, and full-corpus PPL have landed; more workloads and Q4 long-horizon runs are still in progress.

TL;DR

• Daily driver: UD-Q5_K_XL + incoai DFlash2 Q4_K_M drafter (n-max 4) + Nathan's v0.6.5, f16 KV, drafter KV q8_0, ubatch 4096, Sharp chat template. ~30 t/s burst, 26-29 t/s sustained out to 32k tokens.
• DFlash2 over the built-in MTP is +40% burst (30.2 vs 21.6 t/s at 70W) and +60% sustained (25.9 vs 16.3 t/s at 8k tokens). Lossless: output is bit-identical to unspec decode at temp 0, verified.
• Q5_K_XL decodes faster than Q4_K_XL with DFlash2 on short generations (31.4 vs 28.5 t/s at 80W burst; Q5's 55.9% vs 44.9% acceptance pays for the extra bandwidth). On long generations the ranking flips: at 8k tokens Q4 + Sharp beats Q5 + Sharp (28.8 vs 27.5 t/s). I run Q5 for the quality at ~1-2 t/s sustained cost; all-long-form-code workflows should take Q4 + Sharp.
• Pin your Unsloth revision: every number here is the pre-Aug-19 v2 file. Unsloth re-uploaded all UD quants as Dynamic v3 under the same filenames. v3 is better quality (PPL 13.84 vs 14.51) and faster on burst, but collapses at depth: 19.7 vs 27.5 t/s at 8k under the recommended config. The v2 download link is in Sources.
• n-max 4, not 7. At depth, deep drafts get rejected and waste verify bandwidth: n4 is +25% over n7 on long generations, costs ~1.5% on short ones. Nathan's v0.6.5 independently landed on the same number, and my re-sweep under Sharp (where higher acceptance could have shifted the optimum) still says 4.
• The Sharp template is the hidden lever. The stock template burned entire 4k and 8k budgets on reasoning with zero output on my agentic test prompt. With Sharp: the model answers, and sustained decode runs 25.7-29.0 t/s from 1k to 32k tokens, peaking at 32k.
• Skip the FP4 builds: a full PPL tier worse than Q4_K_XL, slower than this stack anyway, and riding a one-off engine (gotcha #3). Tested --spec-draft-p-min 0.7 too: -3% here, an MTP-era trick that doesn't transfer to block drafting.

Quick start

1. Grab Nathan's v0.6.5: Nathanw1014/strix-halo-llamacpp. I benchmarked the identical payload when it was a dev tag (dev-20260819-0b0f35d); v0.6.5 is the same code, now gfx1151-validated.
2. Download the target: Qwen3.8-27B-UD-Q5_K_XL (20.6GB). Pin revision 408fcc1807ab (pre-Aug-19 v2, the file I benched): plain main now serves Dynamic v3, which trades sustained speed for quality (details in the note below).
3. Download the drafter: incoai/Qwen3.8-27B-DFlash2-GGUF, Q4_K_M (1.1GB). Caveat: the upstream DFlash2 PR is still in review, so if the drafter format changes before merge you may need to re-download.
4. Download the template: chat_template.jinja from the Sharp repo, save as sharp.jinja.
5. Launch:

llama-server \
  -m Qwen3.8-27B-UD-Q5_K_XL.gguf \
  -md Qwen3.8-27B-DFlash2-Q4_K_M.gguf \
  -ngl all -ngld all -fa on \
  -ctk f16 -ctv f16 -ctkd q8_0 -ctvd q8_0 \
  -c 65536 -np 1 \
  -b 4096 -ub 4096 \
  -t 16 -tb 32 \
  --spec-type draft-dflash --spec-draft-n-max 4 \
  --chat-template-file sharp.jinja \
  --jinja --host 127.0.0.1 --port 8080 --metrics

The -ctkd/-ctvd q8_0 flags quantize only the drafter's KV cache: +1.6% t/s, and I verified the output stays bit-identical. No boot params needed; the model is small enough that GTT defaults just work.

Results

Config-comparison table: 4-task streaming bench (python, math, JSON, long-prompt; short 100-200 token generations, warm runs), stock template, same engine build throughout.

Spec method	n-max	@70W	@80W	Acceptance
plain (no spec)	-	~14 t/s	-	-
MTP n5/p0.50	5	21.6 t/s	-	66.4%
DFlash2, Q4_K_XL	7	27.3 t/s	28.5 t/s	44.9%
DFlash2, Q5_K_XL	7	30.2 t/s	31.4 t/s	55.9%

Read those as burst rates. The number that matters for real work is sustained generation, measured with a fixed agentic coding prompt at n4 with drafter-KV q8_0, 70W:

quant + template	2k tokens	8k tokens	32k tokens
Q5_K_XL, stock	20.6 t/s	19.8-20.2 t/s	-
Q5_K_XL, Sharp	25.9 t/s	27.5 t/s	29.0 t/s
Q4_K_XL, stock	23.4 t/s	20.4 t/s	-
Q4_K_XL, Sharp	26.6 t/s	28.8 t/s	not tested

Two lessons in that table. Burst belongs to Q5 (its distribution sits closer to the drafter's training, so acceptance climbs and pays for the extra weight bytes); at 8k the ranking flips to Q4, because acceptance converges between the quants and Q4's ~2.3GB fewer weight bytes per pass wins. Q5 stays my daily driver for the PPL and burst; Q4 + Sharp is the pick for long-form code.

Second, and bigger: with the Sharp template, decode gains speed with depth instead of decaying.

The "spec decode collapses at long context" story turned out to be half a template problem. Stock-template output drifts into sprawling reasoning that the drafter can't predict, so acceptance falls from ~56% to ~24% and t/s follows.

Sharp's terse style keeps acceptance climbing as code accumulates: 0.57 at 8k, 0.72 by 32k, and the rate curve rises the whole way (25.7 at 1k up to 29.0 t/s at 32k). Past ~40k the classic long-context decay does return: a full 64k generation averages 24.8 t/s with the trailing window near 16, even though acceptance still holds 0.62. So Sharp moved the horizon from 8k to ~32k tokens; it did not make it infinite, and the late decay is attention cost, not draft mismatch.

Acceptance percentages are not comparable across methods (DFlash2 drafts blocks in parallel, MTP sequentially), so judge by t/s. Per-task burst spans 21 to 40 t/s depending on workload; that spread is content, not noise.

Prefill: what to expect at depth

Shallow (3k prompt): ~298-302 t/s at ubatch 4096, both spec methods identical. Ubatch 4096 is the knee: 8192 measured flat, 2048 leaves ~5% on the table.

Deep context, f16 KV, llama-bench:

Model	pp32768	pp65536
Q4_K_XL	265.1 t/s	222.1 t/s
Q5_K_XL	251.5 t/s	212.2 t/s

Why the deep numbers hold up: Nathan's fork carries a flash-attention prefill series (dequant-once scratch, KV contiguize, coopmat1 tuning), and the gains grow with depth. I ported that series onto the ROCmFPX fork to A/B it properly: +59% at 32k with f16 KV (165 to 263 t/s). At 64k the unpatched engine didn't just slow down, it threw vk::DeviceLostError on every model I tried, while the ported one ran 199-219 t/s.

The port is here with full numbers, and the findings went upstream.

Why not FP4? The reddit hype, checked

The ROCmFP4 builds were everywhere last week with 30+ t/s headlines, so I ran the whole stack. What holds up: the FP4 engine's Vulkan path genuinely is fast, and it beat the MTP-era configs (23 vs 21.6 t/s). What doesn't: it's slower than Q5+DFlash2, it measures a full PPL tier worse than Q4_K_XL, and the engine is a dead end.

Same-engine, same-method short-corpus perplexity (90KB wikitext-2 slice, c8192, n512, 2 chunks, f16 KV):

Quant	Short PPL
Q5_K_XL	53.85
Q4_K_XL	57.26
ROCmFP4_FAST	59.06

The engine is also a one-off: the pinned build the benchmarks ran against was force-pushed out of the repo's history, and the prebuilt crashes at 64k f16 KV. Fun demo, not a daily driver.

Dynamic v3 note (updated Aug 20): Unsloth re-uploaded every UD quant with Dynamic v3 under the same filenames on Aug 19, so all numbers in this post are the v2 files. I A/B'd both Q5s under the recommended Sharp config, same session, 70W. v3 wins two of three: PPL 13.84 vs 14.51 (wikitext-2 full corpus, ~4x the error bars) and burst 30.6 vs 29.2 t/s at 73.7% acceptance.

The third column is why I still recommend v2: sustained 8k collapses to 19.7 t/s vs 27.5 (acceptance decays to 0.40 at depth). For agentic coding that's the whole game; for short generations and maximum quality, take v3 knowingly.

Reproducing my numbers means pinning the revision. The v2 files live at commit 408fcc1807ab (Aug 14, last before the re-upload): Q5 direct is huggingface.co/unsloth/Qwen3.8-27B-GGUF/resolve/408fcc1807ab/Qwen3.8-27B-UD-Q5_K_XL.gguf, or hf download unsloth/Qwen3.8-27B-GGUF Qwen3.8-27B-UD-Q5_K_XL.gguf --revision 408fcc1807ab. My Q4_K_XL numbers are v2-era too; current main serves a different Q4.

Gotchas (learned the hard way)

1. The stock template can burn your entire budget on thinking. On my agentic test prompt it produced zero content tokens at both 4k and 8k max_tokens: 8192 reasoning tokens, no answer, six minutes of GPU time. That failure mode, not hardware, is the biggest speed killer this model ships with.

The Sharp template fixes it and lifted sustained decode 26% on top.

2. n-max is a depth knob. 4/7/8 are within ~1.5% on short generations, but at 8k tokens n4 beats n7 by 25% (19.8 vs 15.8 t/s). Deep drafts get rejected as acceptance decays, so the long tail just costs verify bandwidth. Nathan's v0.6.5 release notes independently measured the same crossover (width 4: 21.1 t/s vs width 7: 16.3 at 32k), and re-sweeping under Sharp confirmed 4: n5 is par on burst and -2% at 8k.

3. DSpark is broken for this model in llama.cpp right now. The community drafter gives 15-16% acceptance and no speedup, and it's not the drafter's fault: I verified the GGUF metadata and bisected the failure into llama.cpp's DSpark draft generation itself. Details in ggml-org/llama.cpp#25618; use MTP or DFlash2 until it lands.

4. p-min does not transfer. A commenter reported --spec-draft-p-min 0.7 lifting DS4+MTP acceptance dramatically; on DFlash2 here it cost 3%. Block drafting changes the economics, pruning just shrinks the block.

5. First bench run is always cold. First DFlash2 pass read 24 t/s; warm runs 27+. The drafter warms up across requests, so run the workload twice before believing a number.

6. Dense vs MoE is the real choice. Qwen3-Coder-30B-A3B (3B active) prefills at ~1900 t/s on this box vs ~300 here; that's arithmetic, not tuning, so huge-prompt short-answer workloads should use the A3B. For agentic coding where answer quality is the product, this stack is the strongest thing I've run locally.

Sources

• Engine: Nathanw1014/strix-halo-llamacpp v0.6.5 (DFlash2 validated on gfx1151; I benched the identical payload as dev-20260819-0b0f35d)
• DFlash2: https://inco.ai/blog/dflash2/
• Drafter: incoai/Qwen3.8-27B-DFlash2-GGUF
• Template: peculiar-ragdoll/Qwen-Sharp-Chat-Templates
• Target (v2, pinned): unsloth/Qwen3.8-27B-GGUF @ 408fcc1807ab UD-Q5_K_XL + UD-Q4_K_XL; current main = Dynamic v3, see the note above
• DSpark broken draft path: ggml-org/llama.cpp#25618
• My FA-prefill port + ROCmFPX findings: aic0d3r/ROCmFPX port-nathan-prefill, charlie12345/ROCmFPX#86
• Prior work on this machine: DeepSeek V4 Flash guide: https://www.reddit.com/r/LocalLLaMA/comments/1vlmh0b (full guide: https://www.reddit.com/r/LocalAiCore/comments/1vkq5kj)

Happy to answer questions. More Strix Halo benchmarks and guides at r/LocalAiCore.

I've spent a week benchmarking DSv4 Flash 0731 on a Flow Z13 (Ryzen AI MAX+ 395, Radeon 8060S / gfx1151, 128GB LPDDR5X). This is what actually works on this hardware and what doesn't.

Note: the writing is AI-assisted editing; the research, debugging, and every number are from my own runs on this machine.

Edit (Aug 13): since this went up, Nathan shipped the v0.6.3 beta with sparse-attention prefill, and I ran it on my Flow Z13 — the deeper your context, the bigger the win (+39% at 32k, +78% at 64k vs dense attention), and decode stays untouched. Long-context prefill no longer tanks like it used to — full numbers in the prefill section.

TL;DR

• You can run a ~300B MoE locally on a from-$2,920 AMD APU mini PC or laptop — roughly half the price of a DGX Spark — at 27+ t/s decode and ~285 t/s prefill. DSpark is the decode lever, Nathan's Vulkan fork is the prefill lever.
• Both levers come from the same place: MoE kernel tuning. On ROCm, with identical hardware and model, prefill drops ~30% and DSpark goes from +46% to +0%. Supported upgrades don't fix kernel tuning.
• Two config picks matter: q8_0 KV (doubles context to 131k, small quality tail, measured in gotcha #1) and the Q2K-Q8 drafter (frees ~4GB VRAM, no speed cost, gotcha #2). Everything else is table stakes.

Quick start

1. Download the model + drafter from unsloth/DeepSeek-V4-Flash-0731-GGUF: the 4-file UD-IQ3_XXS split (~98GB) and a DSpark drafter, Q2K-Q8 (~7GB) or bf16 (~11GB).
2. Patch the Q2K-Q8 drafter's header with the bf16 drafter's tokenizer if you use it (gotcha #2). Skip this step with bf16.
3. Add the boot params amd_iommu=off amdgpu.gttsize=126976 ttm.pages_limit=32505856 ttm.page_pool_size=32505856 to your kernel cmdline (how depends on GRUB vs systemd-boot). The model won't load without these.
4. Install Nathan's build — v0.6.1 for these exact numbers, or v0.6.3-beta1 (adds sparse-attention prefill, see the prefill section): Nathanw1014/strix-halo-llamacpp. Bundled RADV, no ROCm.
5. Run the command block below with your model and drafter paths.
6. Connect a client to http://127.0.0.1:8080 (Open WebUI, LM Studio, or any OpenAI-compatible frontend).

Setup

How I measured. All numbers are from one machine, and every comparison is same-session so thermals and clocks match. I used a streaming client bench with the same prompt, 3 trials per config, and I report ranges, not just bests, wherever the run-to-run spread is real (gotcha #7). Isolated prefill/decode numbers come from llama-bench, and DGX Spark numbers are cited from external sources at the bottom.

Hardware and config.

• Target: Unsloth UD-IQ3_XXS, ~98GB, 4-file split
• Draft: DSpark drafter, Q2K-Q8 (~7GB, header-patched, gotcha #2) or bf16 (~11GB)
• KV: q8_0
• Backend: Vulkan via Nathan's v0.6.1 build with bundled RADV, no ROCm. It's a pinned release you install and run as-is, all flags are in this script, nothing to tune day to day. The v0.6.3 beta adds sparse-attention prefill (see the prefill section); these numbers are the validated v0.6.1 line.
• OS: CachyOS, kernel 7.1.6, Mesa 26.1.6

llama-server \
  -m DeepSeek-V4-Flash-0731-UD-IQ3_XXS-00001-of-00004.gguf \
  -md DSV4-Flash-DSpark-draft-bf16.gguf \
  -ngl all -ngld all \
  -fa on \
  -ctk q8_0 -ctv q8_0 \
  -c 131072 -np 1 \
  -b 2048 -ub 2048 \
  --spec-type draft-dspark \
  --spec-draft-n-max 64 \
  --jinja --host 127.0.0.1 --port 8080

Boot params: amd_iommu=off amdgpu.gttsize=126976 ttm.pages_limit=32505856 ttm.page_pool_size=32505856 (without these the model won't load).

Power: z13ctl+ profile, CPU boost off with min freq pinned. Decode is bandwidth-bound, so the downclock costs nothing and keeps the tablet cool.

VRAM: 4GB BIOS carve-out, GTT covers the rest. Committed footprint is ~105GB (98GB model + 7GB Q8 draft), leaving ~23GB for KV and compute. The bf16 draft pushes that to ~109GB / ~19GB.

Results

Best client-side observation on 4096-token generation. Server-side timing for similar runs consistently shows 23-24 t/s. See gotcha #7 for the variance.

Metric	Value
Decode (best client-side, 4096 tok)	27.13 t/s
Decode (server-side typical)	23-24 t/s
Decode (peak 3s window)	35.27 t/s
Plain decode (no spec, same engine)	18.55 t/s (llama-bench)
Plain prefill (no spec, same engine)	284.98 t/s (llama-bench, ub2048)
Max prefill (f16 KV, 65k ctx, no DSpark)	293.33 t/s (llama-bench, ub2048)
DSpark acceptance	0.7209 best, mean accepted len 4.02
Context	131,072 (q8_0 KV)
GPU util / CPU util	~92% / ~1%

131k context runs on q8_0 KV, which doubles context but costs a little quality. Full measurement of that tradeoff is in gotcha #1.

Cross-platform comparison

Both are unified-memory APUs with similar LPDDR5X bandwidth (~256 vs ~273 GB/s). The Strix Vulkan rows are Nathan's fork, not upstream llama.cpp. Its hand-tuned MoE kernels (the GGML_VK_MMID_* flags) are most of why Vulkan leads here, not the API itself (gotcha #3).

The ROCm row is stock mainline llama.cpp.

Platform	Engine	Backend	Spec	Decode	Prefill @2K
Strix Halo	Nathan fork v0.6.1	Vulkan	none	18.55 t/s	284.98 t/s
Strix Halo	llama.cpp (mainline)	ROCm 7.14	none	13.35 t/s	191 t/s
Strix Halo	ds4 (upstream)	ROCm	none	12.5 t/s	122 t/s
DGX Spark	ds4 (upstream)	CUDA	none	14.2 t/s	392 t/s
DGX Spark	Entrpi/ds4 fork	CUDA	none	20.0 t/s	~960 t/s
Strix Halo	Nathan fork v0.6.1	Vulkan	DSpark	27.13 t/s	~285 t/s
DGX Spark	Entrpi/ds4 fork v0.5.6	CUDA	DSpark	27.3 t/s	~960 t/s

Notes: ds4 is ROCm/HIP-only, no Vulkan backend. Its 122 t/s figure predates ROCm 7.14 and used the Q2 quant. Prefill is unaffected by speculation, and decode is bandwidth-bound, so context depth barely moves it.

Strix numbers use the header-patched Q8 drafter where noted (gotcha #2).

Decode is a dead heat. Plain decode clusters in the 12-20 t/s range everywhere, DSpark brings both platforms to ~27 t/s. The same-engine boost is 1.46x on Strix (27.13 / 18.55) vs 1.37x on DGX Spark (27.3 / 20.0). Speculation is the only lever on bandwidth-bound decode, and it behaves the same on both platforms.

Prefill is where platforms diverge. No engine runs optimally on both platforms, so the honest comparison is the best of each: llama.cpp Vulkan at 284.98 t/s on Strix vs the D2R CUDA fork at 960 t/s on DGX, a 3.37x gap. The fork's D2R pass is a clean 2.45x over its own CUDA baseline on identical hardware, which suggests most of the gap is closeable with better shaders.

I measured the Strix side with GPU perf counters (amdgpu gpu_metrics via amdgpu_top, 1 Hz during llama-bench pp2048, same v0.6.1 build and q8_0/ub2048 config; 274–285 t/s across sessions). During prefill the shader array is ~94% busy while DRAM traffic is only ~74 GB/s — about 30% of the ~256 GB/s bus. Strix prefill is shader-bound, not bandwidth-bound.

Effective compute works out to ~4.6 TFLOPS, under 10% of gfx1151's FP16 MMA peak. At ub2048 each routed expert sees exactly 48 tokens (2048 × 6 active experts ÷ 256 total = 48), so the M=48 tiles can't fill the tensor cores. The ~26 GB/s of writes is the dequant-to-fp16 staging pass — real traffic, but the bus absorbs it, so fusing it away wouldn't move prefill much.

What Strix would actually score with D2R-quality shaders remains a projection, so it stays parked in the footnote.

Since writing this, Nathan shipped the v0.6.3 sparse-prefill line (alpha1 → beta1, prefill numbers unchanged), and I validated it on this 128GB box: the raw/selected split is +26% over the single coopmat kernel, and vs dense attention prefill is +39% at 32k depth and +78% at 64k, because the split flattens the depth penalty (−8% vs −28% from 32k to 64k). That confirms the shader-bound diagnosis from the attention side too — a full 3x was on the table in the FA stage at depth, now realized. It's attention-stage work, so the MoE-GEMM lever (D2R-style shaders) in the footnote is still unclaimed, and decode is untouched.

Footnote (prefill math): MoE prefill doesn't scale with raw FP16 TFLOPS. The compute ratio is 4.2x but the measured gap is 3.37x, and expert GEMMs are tiny (M=48 at ub2048: 2048 tokens × 6 active experts ÷ 256 total), so tensor cores stay underutilized — on Strix now measured directly (94% shader busy at ~30% of DRAM peak, ~4.6 effective TFLOPS). If Vulkan's MoE shaders matched D2R quality, Strix would project to ~698 t/s (284.98 × 2.45), leaving a residual gap of ~1.4x. The 698 t/s figure remains a projection, not a measurement.

Gotchas (learned the hard way)

1. q8_0 KV: faster + more context, but it costs quality. The speed crossover first, measured same engine and prompt shape on single runs per config:

KV type	Context	1024 tok	4096 tok
f16	65k	21.99 t/s	19.32 t/s
q8_0	131k	20.70 t/s	22.74 t/s

At short context q8_0 is ~6% slower because per-element dequant overhead dominates a small cache. At long context it's +18% faster because it halves bytes-per-step, and it doubles context (131k vs 65k). This is a crossover observation, not a precision delta, so treat the percentages as directional.

The real cost is quality. I ran llama-perplexity --kl-divergence (bf16 baseline vs q8_0) on ~50k tokens of prose with the same model and Nathan's build:

Metric	q8_0 vs bf16 (this work, Moby Dick)	erazortt's DS4F (wikitext)
Same top-p	97.84% (2.16% flip)	87.19% (13% flip)
KLD mean	0.0126	0.1459
KLD 99.9%	0.81	4.535
RMS Δp	5.12%	11.88%

Mean PPL barely moves (1.0015x), but the tail is real: the 99.9th-percentile KLD is 64x the mean, with ±40% probability swings on rare tokens. The average stays clean while a thin slice of tokens goes somewhere else. My numbers look ~10x better than erazortt's, but that's corpus: Moby Dick (PPL 1.54) is far more predictable than wikitext (PPL ~5.8).

On harder text, expect degradation closer to his figures, so treat mine as a lower bound. Root cause: DS4F's MLA is built around fp8 KV natively, which llama.cpp doesn't expose, so it's f16 or q8_0 (worse dynamic range than the model expects). That's a tooling gap, not a choice.

For precision or agentic work, stay on f16 KV at 65k context. Methodology thread: https://www.reddit.com/r/LocalLLaMA/comments/1vduxth

2. The draft doesn't need to be bf16. Use the Q2K-Q8 drafter and keep ~4GB VRAM. The ~7GB quantized drafter crashed at first with invalid token = -1, but that was a broken conversion, not a precision problem. The file had its tokenizer metadata stripped and declared tokenizer.ggml.model = no_vocab, under which llama.cpp nulls every special-token id (including the draft mask token) to -1. Patching the header with the bf16 drafter's tokenizer fixes it.

Same-session, same q8_0 KV config, 4096-token runs: decode 25.3-27.8 t/s (best 27.78, mean ~26.3 vs bf16 ~24) and draft acceptance 0.657-0.740 (mean 0.687 vs bf16 0.606). Both differences sit inside the run-to-run noise band from gotcha #7, so count them as "no speed or acceptance cost", not wins. The drafter is a few billion params in a bandwidth-bound loop, so Q8 quantization costs nothing measurable.

The 4GB of freed VRAM is the real take.

3. Vulkan beats ROCm on gfx1151, but "supported ≠ tuned". Fair pushback in the comments that I'd judged ROCm on the ds4 engine and a pre-7.14 build. So I benched mainline llama.cpp + ROCm 7.14 properly: gfx1151 went officially supported on 2026-07-16 and loads natively now, no HSA_OVERRIDE or TheRock hacks. Same model, same session, q8_0 KV, -fa on, -ub 2048, 80W sustained:

metric	Vulkan v0.6.1	ROCm 7.14 (mainline)	ROCm
prefill pp2048	284.98 t/s	191.28 t/s	0.67x
prefill pp8192	265.26 t/s	163.47 t/s	0.62x
plain decode	18.55 t/s	13.35 t/s	0.72x
DSpark decode	27.13 t/s @ 0.7209	13.35 t/s @ 0.613	0.49x
DSpark speedup vs plain	+46%	+0%	n/a
GPU load	~99% steady	~53-96% (bursty)
CPU spillover	~1%	~15%
temp	81-84°C	82-90°C (spiky)

ROCm 7.14 fixed support. It did not fix kernel tuning for DSv4's MoE: about 30% behind on prefill, 28% on decode, and the gap widens with context. From pp2048 to pp8192, ROCm drops 14.5% while the fork drops 6.9%, so the "ROCm wins long context" story doesn't hold on this APU.

DSpark acceptance is a dead heat (means 0.606 vs 0.613), so the drafter produces equally good tokens on both, but on ROCm the draft runs on the same slow MoE path and its cost cancels the ~4 accepted tokens per pass. Net: +0% on ROCm, +46% on Vulkan.

The mechanism is concrete. ROCm spills ~15% of work to the CPU, which on a unified-memory APU steals bandwidth from the GPU it's trying to help. That's the bursty load and the temperature spikes.

The fork's MoE kernels keep everything on-GPU and back to back. So this is stock-mainline-ROCm's generic gfx1151 kernels vs a hand-tuned MoE Vulkan path, not "ROCm vs Vulkan" as APIs. On a dense model, or any MoE where those flags don't fire, ROCm can absolutely win.

The broader ask: the 8060S has the bandwidth to be a real local-inference answer to the DGX Spark, and decode is already a dead heat. What's missing isn't hardware or a support checkbox, it's kernel tuning. Right now one community member's Vulkan fork beats AMD's official ROCm backend on AMD's flagship APU by ~30%, and turns DSpark from a +46% win into 0%.

If AMD wants Strix Halo taken seriously against CUDA, the MoE kernels in ROCm need the same attention the Vulkan path got from one person. Supported ≠ tuned.

4. Ubatch 2048 is the VRAM ceiling with DSpark. At 109GB committed (98 + 11GB bf16 draft), ubatch 4096 OOMs. The Q2K-Q8 draft (gotcha #2) moves that ceiling to ~105GB / ~11GB free, but since ubatch >2048 isn't a prefill win on this MoE, there's no reason to chase it. Measured same-session on the ngram config (f16 KV): pp2048 at ub2048 = 293.33 t/s and at ub8192 = 292.77 t/s, identical within noise.

Prefill saturates at ub2048 here. The ngram config does edge DSpark on prefill (~293 vs ~285), but that ~3% comes from f16 KV (which halves context to 65k), not from ub8192 or dropping the draft. It's a context-vs-prefill tradeoff, not a free speed win.

Decode drops to 18.12 t/s without DSpark, and ngram's cache from a 2.3K-token prompt is too sparse for general prose (3 speculation attempts over 4096 tokens). Ngram is a capacity lever, not a speed lever.

5. v0.6 regression: upgrade to v0.6.1. Nathan's v0.6 has a stride bug in TENSOR_ALLOW_RESHAPE that miscomputes strides for block-quantized tensors. Vulkan silently falls back 43 attention layers to CPU, showing 75% GPU and 50% CPU while running at half speed. v0.6.1 fixes it. Bug report: https://github.com/Nathanw1014/strix-halo-llamacpp/issues/2

6. GPU util drops to ~92% on v0.6.1 (99% on the 0.4 build, same config). The new MoE shaders (ROWLISTS, SMALLN, BM64, M128, F16B, FA_WAVE32) finish each kernel faster, so fixed dispatch overhead becomes a bigger fraction. Lower utilization, higher throughput (27.13 vs 20.88 t/s). Don't let the number fool you.

7. DSpark decode varies run-to-run on the same prompt. Best client-side observation: 27.13 t/s at 0.7209 acceptance. The same session's three trials spanned 20.96–27.13 t/s and 0.507–0.721 acceptance, the drafter warming up across requests — variance tracks GPU thermal/clock state and swings with output content. If you benchmark anything here, run the same prompt 5+ times and report the range, not a single number.

Sources

• Engine: Nathanw1014/strix-halo-llamacpp v0.6.1
• Sparse prefill (v0.6.3): v0.6.3-beta1 release (alpha1 → beta1, prefill numbers unchanged)
• Model: unsloth/DeepSeek-V4-Flash-0731-GGUF
• DGX Spark DSpark numbers: Entrpi/ds4-on-spark v0.5.6 (2026-08-08)
• Model specs: deepseek4 arch, 43 layers, 256 experts (6 active + 1 shared), 1M native context, MIT license. Technical report: https://arxiv.org/abs/2606.19348

More Strix Halo benchmarks and guides at r/LocalAiCore if you're interested in this hardware.

Following the Z13CTL+ and Z13GUI+ releases, I have released the third part of the stack: z13-tablet-kit.

https://github.com/aic0d3r/z13-tablet-kit

It adds posture-aware tablet support for the AMD Strix Halo Flow Z13 (GZ302) on KDE Plasma 6 / Wayland.

The kit automatically distinguishes between three modes:

- Desktop: dock or external display connected

- Laptop: keyboard folio attached

- Tablet: neither attached

It then adapts the system for each posture:

- Enables touchscreen and auto-rotate in tablet mode

- Uses 175% display scaling in tablet mode and 150% otherwise

- Enables the on-screen keyboard only in tablet mode

- Prevents the on-screen keyboard from appearing over the physical keyboard

- Restarts the keyboard after posture changes to prevent clipped layouts

- Provides two-finger scrolling in terminals, browsers, document viewers and other applications

- Optionally maps a stationary two-finger hold to a real right-click for context menus

- Creates the missing tablet-mode switch needed by Plasma’s “rotate only in tablet mode” option

- Disables the unreliable folio touchpad by default, with a guarded opt-in toggle

It uses Plasma’s native rotation support and does not require kernel patches or a root daemon. Only two udev rules require root access; the posture and gesture services run in the active user session.

The full stack consists of:

z13-tablet-kit

Posture detection, tablet mode and touch input

https://github.com/aic0d3r/z13-tablet-kit

Z13CTL+

Display coordination, persistent settings and hardware control

https://github.com/aic0d3r/z13ctl-plus

Z13GUI+

Optional touchscreen overlay with live posture status and tablet controls

https://github.com/aic0d3r/z13gui-plus

The kit currently targets Arch/CachyOS with KDE Plasma 6 on Wayland. z13ctl-plus is required, while z13gui-plus is optional.

Installation instructions and technical details are in the README. Testing, bug reports and contributions are welcome, especially from other GZ302 owners.

I have released Z13CTL+ and Z13GUI+, an expanded Linux control and telemetry stack for the Strix Halo-based 2025 ROG Flow Z13.

Before anything else, special thanks to dahui, who created the original z13ctl and z13gui projects. These releases build on that foundation and would not exist without the work already done there.

My goal was to add more automation, tuning and telemetry while preserving compatibility with the original commands, services, paths and API.

Z13CTL+
https://github.com/aic0d3r/z13ctl-plus

Z13GUI+
https://github.com/aic0d3r/z13gui-plus

Z13CTL+ is the daemon and CLI. It adds:

• Automatic switching between plugged-in and battery configurations
• Independent TDP, fan-curve and undervolt overrides
• AMD P-State controls for minimum CPU frequency, EPP and boost
• CPU, GPU, NPU, memory, VRAM, temperature, fan, battery and power telemetry
• KDE display posture, rotation, scale and refresh-rate coordination
• Recovery for lighting and firmware fan behavior
• CLI and API access for scripting and automation

Z13GUI+ is a GTK4 interface designed around the Z13 touchscreen. It includes:

• A live hardware and power overview
• Separate AC and battery profiles
• CPU policy controls
• 60 Hz and 180 Hz refresh-rate switching
• Battery information
• Fan presets and tuning controls
• Clearer feedback when changing tuning values

The tools are currently focused on the 2025 Flow Z13, but the telemetry side may also be interesting to other Strix Halo users, especially the CPU, GPU, NPU, unified-memory, VRAM, power, clock and temperature monitoring.

Packages are available through the AUR, along with DEB, RPM and portable release archives.

I would be interested in hearing what telemetry and power-management controls are exposed on other Strix Halo systems under Linux, and whether parts of this could eventually be generalized beyond the Z13.

Testing, bug reports and contributions are welcome.