Easy ROCm + llama.cpp build script for AMD GPUs
I made a setup script that handles ROCm installation and building llama.cpp from source with ROCm + Vulkan support. It lets you pick your GPU target and ROCm version interactively, then clones the latest llama.cpp master, configures automatically and build quickly. I have attached my current config to run server where i am getting following bench.
Bench (RX 6700 XT - gfx1030):
Ornith-1.0-35B-MTP-APEX-I-Compact (17GB Q4_K)
- pp3000: 786 tok/s
- tg128: 40 tok/s
Note: Use latest llama.cpp master. My old 2-month-old build got 25-30 tps, updated build now 40-42 tps on same model. Big difference from recent commits.
Repo in comment