New Chesp overkill build
Rocky Linux 9, I5 3330 4/4 8gb DDR3, Mi50 "Radeon Pro VII" bios, still cracking some numbers.
llama serve \
-m Qwen3.5-9B-Uncensored-HauhauCS-Aggressive-Q4_K_M.gguf \
--mmproj mmproj-Qwen3.5-9B-Uncensored-HauhauCS-Aggressive-BF16.gguf \
-c 65536 -ctk q4_0 -ctv q4_0 -ngl 99 --host 0.0.0.0 --port 8080
[INIT & LOAD]
- Server initialized (host: 0.0.0.0:8080)
- Multimodal model loaded successfully
- Slots: 4, Ctx/slot: 65536, KV-unified: true
[PERFORMANCE SUMMARY]
• Task 0 | Prompt: 289.12 t/s (333 tok) | Eval: 46.76 t/s (52 tok) | Total: 2.24s
• Task 55 | Prompt: 571.71 t/s (1159 tok) | Eval: 49.89 t/s (704 tok) | Total: 16.12s
• Task 763 | Prompt: 262.88 t/s (539 tok) | Eval: 44.77 t/s (650 tok) | Total: 16.55s
• Task 1417 | Prompt: 32.81 t/s (4 tok) | Eval: 44.35 t/s (522 tok) | Total: 11.87s
• Task 1940 | Prompt: 35.17 t/s (4 tok) | Eval: 50.75 t/s (587 tok) | Total: 11.66s
(45-50 tokens/sec generation)
[GPU STATUS & PROCESS]
Model: AMD Radeon Pro VII (ROCm 7.2.4)
Power: 186.0W / 190.0W (Cap)
Temp: 44.0°C | Fan: 23.5%
Clocks: SCLK 1700MHz | MCLK 1000MHz
Usage: GPU 99% | VRAM 46% (7595 MB / 16368 MB)