r/DGX_Spark

After 2 days of testing and benchmarking every single recipe I could find on the forums and twitter, this this the current best I have put together for single spark (i tested vllm, sglang, etc). This is tuned also for use with Hermes agent at 256k context, 5 lanes.

PLEASE POST YOURS, with your results.

https://github.com/styles01/sparkrun-recipes/blob/main/recipes/qwen-38-27b.yaml

recipe_version: "4"
name: qwen-38-27b
description: "Qwen3.8 27B NVFP4 — triton_attn, MTP k=2, fp8 KV (auto from checkpoint), 256K context, vision. Mia Lab config."
model: unsloth/Qwen3.8-27B-NVFP4
runtime: vllm
container: vllm/vllm-openai:nightly-aarch64
maintainer: styles01

metadata:
  author: styles01
  created: "2026-08-14"
  updated: "2026-08-14"
  tags:
    - qwen
    - qwen3.8
    - 27b
    - nvfp4
    - mtp
    - triton-attn
    - vision
  runbook: runbooks/qwen-38-27b.md

solo_only: false
cluster_only: false

min_nodes: 1
max_nodes: 1

defaults:
  port: 8000
  host: 0.0.0.0
  tensor_parallel: 1
  pipeline_parallel: 1
  gpu_memory_utilization: 0.84
  max_model_len: 262144
  max_num_seqs: 4
  max_num_batched_tokens: 8192
  kv_cache_dtype: fp8
  attention_backend: triton_attn
  load_format: fastsafetensors
  tool_call_parser: qwen3_coder
  reasoning_parser: qwen3
  served_model_name: "qwen3.8-27b unsloth/Qwen3.8-27B-NVFP4"
  speculative_config: '{"method":"mtp","num_speculative_tokens":2}'
  async_scheduling: true
  enable_prefix_caching: true
  enable_chunked_prefill: true
  trust_remote_code: true
  enable_auto_tool_choice: true
  quantization: compressed-tensors

env:
  CUTE_DSL_ARCH: sm_121a

executor_config:
  entrypoint: ""
  auto_remove: false
  user: root

# NO Mamba patch needed — vLLM nightly-aarch64 has native MTP support for Qwen 3.8.
# NO enforce-eager — CUDA graphs work with triton_attn + NVFP4 checkpoint.
# The NVFP4 checkpoint auto-applies FP8 KV cache scheme (calibrated), doubling KV capacity.
# Using FP8 checkpoint (Qwen/Qwen3.8-27B-FP8) instead of NVFP4 causes OOM during MTP — that was our root crash cause.
# NOTE: max_num_batched_tokens=8192 is REQUIRED for GDN/Mamba cache alignment (32768 crashes under concurrency).
# See runbooks/qwen-38-27b.md for alternative configs (erdaltoprak GMU 0.50, SGLang FP8, Radix DSpark).

benchmark:
  framework: llama-benchy
  pp: [2048]
  tg: [128]
  depth: [0, 4096, 8192, 16384, 32768, 65535, 100000]
  concurrency: [1, 2, 5, 10]
  prefix_caching: true
  runs: 3

command: |
  vllm serve {model} \
    --host {host} \
    --port {port} \
    --served-model-name {served_model_name} \
    --max-model-len {max_model_len} \
    --max-num-seqs {max_num_seqs} \
    --max-num-batched-tokens {max_num_batched_tokens} \
    --trust-remote-code \
    --gpu-memory-utilization {gpu_memory_utilization} \
    --enable-auto-tool-choice \
    --tool-call-parser {tool_call_parser} \
    --reasoning-parser {reasoning_parser} \
    --quantization {quantization} \
    --load-format {load_format} \
    --attention-backend {attention_backend} \
    --async-scheduling \
    --enable-prefix-caching \
    --enable-chunked-prefill \
    --speculative-config '{speculative_config}' \
    -tp {tensor_parallel} \
    -pp {pipeline_parallel}

Hi all,
with this post I want to talk again of AudioMuse-AI, a free and open source selfhostable software to analyze your song and automatically create playlist on your supported music server like Jellyfin, Navidrome (or open subsonic api based), Emby and Lyrion:

• https://github.com/NeptuneHub/AudioMuse-AI

With this post I want to celebrate two big things, first of all AudioMuse-Ai born on May 2025, so it's stil live and fully mantained after 1 years, 217 issue closed and 182 PR closed !

We also want to celebrate the new AudioMuse-AI v1.1.0 release that introduce Lyrics Semanthics similarity throug different functionality.

I'm very proud of this release because multiple time we heard that yes the mood is similar but totally different lyrics, now you can search your song also semathically with:

• Axis-based search: Explore songs across 5 defined semantic axes, selecting one or more values that best describe the target mood or meaning.
• Text search: Simple natural language queries (e.g., “love”, “run”) focused on lyrical meaning, not musical groove (distinct from DCLAP search).
• Song similarity search: Use a reference track to find similar songs, weighted by default as 75% lyrical meaning and 25% audio similarity to preserve genre consistency.

Lyrics functionality off course need lyrics, the best way is to have already them in your music server OR configure in AudioMuse-AI your favourite API in the setup wizard:

Example API formats supported in Setup Wizard:

https://api.example.com/get?artist={artist}&title={title}
https://api.example.com/v1/{artist}/{title}

Anyway as a fallback is also supported the transcription with Whisper Small and if needed can be disabled in the setup wizard by setting LYRICS_ENABLED=true

Important: after the update a new analysis will do the Lyrics analysis on the already analyzed song (if enabled, enabled by default) or a full analysis (Musicnn + Clap + Lyrics) for new song. This new analysis is mandatory to use the new functionality.

I hope you will like both of this milestone and as usual, if you want to support AudioMuse-AI, please add a start on the github repository.
Thanks to be with us for our first year!