Opinions on DGPP ? Inference engine for DGX Spark
Carelessly swapped SGLang for DGPP ( https://github.com/HawkBearPig/dgpp ) using an example config ( https://raw.githubusercontent.com/HawkBearPig/dgpp/refs/heads/master/deploy/cluster_qwen-3.8-flash-next_nvfp4_w1.example.json ) on my single DGX Spark setup with Qwen3.8-Flash-Next and I'm amazed by the solid performance and surprised that I never read about it before. It even does mixed chunking while SGLang couldn't.
Anybody here is using it and/or did accuracy tests?
EDIT Oct, 3rd: I find out the the stock recipe is very conservative about memory. I use DGX Spark as a headless server, so I bumped both KV Cache and prompt cache.
Here's my current config:
{
"model": "nvidia/Qwen3.8-Flash-Next-NVFP4",
"world_size": 1,
"engine": {
"max_concurrency": 4,
"kv_capacity": 524288,
"kv_dtype": "bf16",
"bf16_weights": "bf12+bf16",
"ngram_table": "mmap",
"default_max_tokens": 32768,
"queue_limit": 8,
"max_connections": 64,
"decode_graph": true,
"mtp": true,
"mtp_depth": 1,
"prefix_cache_gib": 6,
"admission": "grow",
"admission_window": 256,
"model_alias": "Qwen3.8-Flash-Next",
"sampling_candidates": 128,
"stats_interval_s": 10,
"dense_weights": "fp8",
"fp8_head": "mma",
"prefill_budget_tokens": 1024,
"prefill_idle_budget_tokens": 4096
},
"paths": {
"resident_cache": ""
},
"http": {
"bind_host": "0.0.0.0",
"port": 8000
}
}