I take it all the buddies heard of the fight (between two adults who should know better) outside the Last Post on Tuesday night?
The folk on Paisley Crowd and Renfrewshire Gazzette comments said it was no surprise....
That got me thinking: what would you say is the roughest pub in Paisley, and why?
I'll agree with the Last Post, but the Patter Bar and Argyll Bars are up there (although I do love the Argyll)
I use llama.cpp with Windows at the moment, but I'm seriously contemplating setting up Ubuntu - as long as my tok/s will go up significantly.
I asked Claude if it was worth it, and Claude said "ROCM on Windows is far behind Linux"
As with all AI output I took it with a pinch of salt - but is it true?
With ROCM on Linux are you exceeding 50 tok/s with a high context (say, 128K) using a dense model? I'm getting that with Vulkan on windows atm with Qwen 27B.
TL:DR feelings about Windows aside, is installing Ubuntu worthwhile to get superior token generation with ROCM?
Hardware:
9900X
64GB DDR5 6000 @ CL30
AMD 9070XT 16GB
AMD R9700 AI PRO 32GB
SSDs - 14900k/s Samsung x2 (not RAID)
Our AI evangelist says coding is SOLVED. Yes really. Interesting - he's not a developer, he hasn't even used Git, so how can he say such things?
(Bear with me here, there is a question at the end)
I don't think he realises that, without the following our code would be shit:
TL:DR
I'm trying to convince management IT TAKES A LOT OF WORK to keep the slop away. And that if other dev teams in the enterprise adopt AI it won't automagically generate software fit for purpose.
They've got the dollar signs in their eyes (ie: sack the lot of us and let AI do the work)
So I would like to ask the devs who read this sub:
Could you give me concrete examples of slop code that AI has generated in your enterprise apps (naturally omitting details that could identify your company)? Or things that you KNOW go wrong when using AI to generate code, guided or not.
Where does it fail the most?
I've Googled for examples, but obviously Microsoft, Amazon etc don't release details of how their AI generated code failed.
If I had some actual evidence or steps to reproduce, I could maybe stop this madness.
EDIT: I'm looking for concrete evidence or steps to reproduce, not anecdotes, because anecdotal evidence gets dismissed by non-techies. You know what management & C-suite are like
Im currently using Unsloth Qwen 3.8 Q6_K_XL GGUF for coding in c# and c++, in a very large .NET solution (~110 projects)
Its been extremely reliable on medium and xhigh with llama.cpp and the Pi harness but I would like a little more context if possible. My 32GB R9700 is pretty maxed out at 128K ctx and Q8 KV.
I was wondering if any smaller quants were just as reliable for complex c# and c++ tasks.
Also, I use Q8 for KV cache, does dropping to Q4 KV cause issues like it did for Qwen 3.6?
When I run the latest build of lemonade-sdk llama.cpp, I always get back gibberish after I input a prompt.
EDIT - ISSUE FOUND
Supplying more than one GPU via --device causes problems.
Passing just one works.
I don't get this issue with the Vulkan build so will revert to that until they fix it.
FIX (WORKING COMMAND LINE):
llama-server.exe --model C:\LLMs\unsloth\Qwen3.8-27B-GGUF\Qwen3.8-27B-UD-Q6_K_XL.gguf --mmproj C:/LLMs/unsloth/Qwen3.8-27B-GGUF/mmproj-F16.gguf --no-mmap --host 127.0.0.1 --port 1234 --verbosity 4 --log-verbosity 4 --no-webui --jinja --ctx-size 131072 --device ROCm0 --main-gpu 0 --batch-size 2048 --ubatch-size 1024 --threads 9 --parallel 1 --cache-type-k q8_0 --cache-type-v q8_0 --flash-attn on --kv-offload --kv-unified --spec-type draft-mtp --spec-draft-n-max 3 --spec-draft-n-min 0 --spec-draft-p-min 0.75 --reasoning-preserve --reasoning-format deepseek --temp 1.0 --top-p 0.95 --top-k 20 --min-p 0.0 --log-file C:\LLMs\logs\llama-server.log
ORIGINAL POST (WITH BROKEN COMMAND LINE):
This does not happen if I use the "normal" llama.cpp Vulkan build with the same GGUF.
Has anyone managed to get the latest lemonade-sdk RocM build working on Windows?
If so, how did you set it up?
My llama-server args are:
llama-server.exe --model C:\LLMs\unsloth\Qwen3.8-27B-GGUF\Qwen3.8-27B-UD-Q6_K_XL.gguf --mmproj C:/LLMs/unsloth/Qwen3.8-27B-GGUF/mmproj-F16.gguf --no-mmap --host 127.0.0.1 --port 1234 --verbosity 4 --log-verbosity 4 --no-webui --jinja --ctx-size 131072 --device ROCm0,ROCm1 --main-gpu 0 --batch-size 2048 --ubatch-size 1024 --threads 9 --parallel 1 --cache-type-k q8_0 --cache-type-v q8_0 --flash-attn on --kv-offload --kv-unified --spec-type draft-mtp --spec-draft-n-max 3 --spec-draft-n-min 0 --spec-draft-p-min 0.75 --reasoning-preserve --reasoning-format deepseek --chat-template-kwargs "{\"reasoning_effort\":\"xhigh\"}" --temp 1.0 --top-p 0.95 --top-k 20 --min-p 0.0 --log-file C:\LLMs\logs\llama-server.log
Device RocM0: R9700 32GB
Device RocM1: 9070XT 16GB.
I'm 54 next month, I love jogging, are there any mens runners clubs near Paisley town centre? Nothing too serious though.
TIA.
I'm running llama.cpp at the moment (or plan to, when I get the HIP build working)
Before, when I had a single GPU, I would do the following to speed up token generation on Qwen 3.6 27B Q6_K or Q8_K:
Now I have two GPUs: a 9070XT 16GB and an R9700 AI Pro 32GB. Both at PCI5 x8.
I can only run LM Studio with this setup as my Vulkan and RocM builds of llama.cpp aren't working correctly (driver issues the former, and lemonade-sdk issues the latter) and LMs token generation speed isn't great. I am using "split evenly" for layers.
Is there any other ways of speeding up token generation I'm missing?
I know DFlash exists, I've not tried that yet. Any other plugins I could use? I asked Claude and it's giving me old info, I know how fast tech moves.
Rig:
AMD Ryzen 9900X
Asus ProArt X870E Wifi
64GB DDR5 RAM
9070XT 16GB - PCI slot one
9700 AI Pro 32GB - PCI slot 2
M2 Gen5 SSDs.
Windows 11
I'm running llama.cpp at the moment (or plan to, when I get the HIP build working)
Before, when I had a single GPU, I would do the following to speed up token generation on Qwen 3.6 27B Q6_K or Q8_K:
Now I have two GPUs: a 9070XT 16GB and an R9700 AI Pro 32GB. Both at PCI5 x8.
I can only run LM Studio with this setup as my Vulkan and RocM builds aren't working correctly (driver issues the former, and lemonade-sdk issues the latter) and its token generation isn't great. I am using "split evenly" for layers.
Is there any other ways of speeding up token generation I'm missing?
I know DFlash exists, I've not tried that yet. Any other plugins I could use? I asked Claude and it's giving me old info, I know how fast tech moves.
Rig:
AMD Ryzen 9900X
Asus ProArt X870E Wifi
64GB DDR5 RAM
9070XT 16GB - PCI slot one
9700 AI Pro 32GB - PCI slot 2
M2 Gen5 SSDs.
Windows 11
Over the past month I've been watching Nurse Julie's (American hospice worker) Youtube videos.
I've found them fascinating, and a little disturbing if I'm honest.
What she said about "The Rally" (a strange burst of energy in the dying before death such as verbal communication returning to Alzheimers patients) and "Visioning" (seeing or dreaming of deceased relatives shortly before death) and how common they are, has shocked me a bit.
For a UK based hospice nurse, are these things you've personally seen, and do they happen often? What do Doctors make of this? Especially non-religious types?
My deepest apologies for posting this question here, if its the wrong place to ask. I do not want to upset anyone here who has lost a loved one or whose loved one is dying. If it's the wrong place can you advise me what one I should use.
Hi,
Can anyone help here?
lemonade-sdk make nightly builds of their llama cpp fork for RocM Windows x64 but their gfx1201 variant for RDNA4 cards dont work on my system - I get the following error message:
rocBLAS error from hip error code: 'hipErrorInvalidKernelFile':218
0.14.606.864 E ggml_cuda_compute_forward: SOLVE_TRI failed
D:/a/llamacpp-rocm/llamacpp-rocm/llama.cpp/ggml/src/ggml-cuda/ggml-cuda.cu:106: ROCm error
Anyone know of a fix? It needs to be Windows.
I cannot use Vulkan, the drivers are borked and crash.
On Windows 11, when I run unsloth/Qwen 3.6 27.b mtp q8_0 on a llama.cpp build targeting RocM ( obtained from lemonade-sdk/llamacpp-rocm: Fresh builds of llama.cpp with AMD ROCm™ 7 acceleration) and go to the chat window (http://localhost:1234/chat) I get absolute gibberish back.
Yet when I run the same model using LM Studio, using the app's bespoke llama.cpp targeting RocM it works fine.
TL:DR
I would like to use the exact same CLI version of llama.cpp that LM Studio uses. I found the llama.cpp build here: .lmstudio\extensions\backends\llama.cpp-win-x86_64-amd-rocm-avx2-2.27.1 but it won't run standalone. I get: this message:
Anyone know where I can get an up to date llama.cpp build with RocM for Windows 11?
Here's the Qwen batch file I'm trying to run.
llama-server ^
-m "unsloth\Qwen3.6-27B-MTP-GGUF\Qwen3.6-27B-Q8_0.gguf" ^
--alias qwen3627b ^
--host 127.0.0.1 ^
--port 1234 ^
--verbosity 3 ^
--jinja ^
--ctx-size 262144 ^
--n-gpu-layers 999999 ^
--n-cpu-moe 0 ^
--main-gpu 0 ^
--tensor-split 0 ^
--split-mode layer ^
--ctx-checkpoints 32 ^
--batch-size 2048 ^
--ubatch-size 512 ^
--threads 12 ^
--parallel 1 ^
--cache-type-k q8_0 ^
--cache-type-v q8_0 ^
--flash-attn on ^
--kv-offload ^
--kv-unified ^
--no-direct-io ^
--spec-type draft-mtp ^
--spec-draft-n-max 2 ^
--spec-draft-n-min 0 ^
--spec-draft-p-min 0.75 ^
--temp 0.6 ^
--top-p 0.95 ^
--top-k 20 ^
--min-p 0.0 ^
--presence-penalty 0.0 ^
--repeat-penalty 1.0
Any extreme gore zombie horror books set in the Romero universe?
Not bothered about sex scenes or anything just serious Romero/Fulci type gore albeit in word form.
Thanks in advance.
I've seen people talking about EGPUs to add graphics cards for LLM VRAM.
That's an intriguing idea.
I have an X870E ProArt Wifi mobo, it supports USB4, and GPT 5.6 advises me that a USB4/Thunderbolt EGPU enclosure is viable and will let me extend my VRAM (48GB - 9070xt 16GB and 9700 AI Pro 32GB)
However, how good is it in practice? Anyone using such enclosures?
Lets say I put a cheap 9060 into the EGPU for extra VRAM. Am I crippling what I already have?
I understand there's extensions like Continue which let VS Code CoPilot "speak" to llama.cpp but I believe Bring Your Own Model (BYOM) has made that obsolete?
I have really tried to get BYOM working, I even asked Claude for instructions, but I can't get it working. I have to use a proxy which exposes an ollama endpoint and forwards to llama.
I can connect to LM Studio no problem with BYOM, but would rather use llama.cpp because it uses much less RAM (and VRAM).
My setup:
Would anyone have any recommendations, apart from obsolete extensions?
Even better, any working JSON I could copy? This is really annoying.
Thanks in advance.
I have the following rig:
When I start LM Studio and open unsloth/Qwen 27B, I can "chat" to the model. As soon as I ALT-TAB to another window, the task manager, I get an AMD crash defender message and my R9700 is disabled in the device manager.
I get the same problem if I run llama.cpp.
I did a little digging and someone else on reddit said they fixed the issue by disabling ASPM Mode Control in the BIOS, which I did. It's not fixing the issue.
I have the latest AMD chipset drivers, the latest AMD Adrenalin, the latest Windows 11 patches.
Anyone had this issue before? Is there a fix?
EDIT: SWITCHING FROM VULKAN TO ROCM ON LM STUDIO SEEMS TO HAVE DONE THE TRICK (and increased token generation speed a LOT!).
Yes, from Vulkan to RocM - not the other way around.
I have the following rig:
When I start LM Studio and open unsloth/Qwen 27B, I can "chat" to the model. As soon as I ALT-TAB to another window, the task manager, I get an AMD crash defender message and my R9700 is disabled in the device manager.
if I disable the 9070XT so that there's no split between GPUs, I get the same issue.
Same problem if I run llama.cpp from command line.
I did a little digging and someone else on reddit said they fixed the issue by disabling ASPM Mode Control in the BIOS, which I did. It's not fixing the issue.
Anyone had this issue before? Is there a fix?
Here's my LM Studio settings:
Here's my llama.cpp command line - note there's no layer split in this:
llama-server ^
-m "unsloth\Qwen3.6-27B-MTP-GGUF\Qwen3.6-27B-Q8_0.gguf" ^
--alias qwen3627b ^
--host 127.0.0.1 ^
--port 1234 ^
--jinja ^
--chat-template-kwargs "{\"preserve_thinking\":true}" ^
-c 131072 ^
--n-gpu-layers 65 ^
--flash-attn on ^
--parallel 1 ^
--cache-type-k q8_0 ^
--cache-type-v q8_0 ^
--threads 12 ^
--threads-batch 12 ^
--spec-type draft-mtp ^
--spec-draft-n-max 3 ^
--spec-draft-p-min 0.75 ^
--temp 0.6 ^
--top-p 0.95 ^
--top-k 20 ^
--min-p 0.0 ^
--presence-penalty 0.0 ^
--repeat-penalty 1.0 ^
--reasoning-preserve
I have a Corsair RM-1000e 1000w PSU, a Ryzen 9900X, and a dual GPU (R9700 AI Pro, Asus 9070XT OC Prime) setup on an Asus ProArt X870E.
I would like to connect two 8 pin connectors to my motherboard so I can overclock my 9900X but I don't have the sockets available on the PSU.
I was thinking of using a 1 into 2 PCIe cable splitter (I believe they are called pigtails?) for my 9070XT but AI recommends against it.
This is what AI recommends I do:
Native 12V-2x6 cable
└── Radeon AI Pro R9700
One EPS/CPU cable
└── Motherboard CPU_12V_1
Three separate Corsair Type 4 PCIe cables
├── RX 9070 XT 8-pin socket 1
├── RX 9070 XT 8-pin socket 2
└── RX 9070 XT 8-pin socket 3
Motherboard CPU_12V_2
└── Leave disconnected
TL:DR Has anyone ever used PCIe splitters for the 9070XT? Are they safe? What brand would you recommend? Do I even need to worry about this as I'm using a 9900X not a 9950x?
Or do I need a 1200w with more sockets?
I've seen posts talking about creating browser games in one shot, but I'm wondering if anyone has one-shotted (or even 1000-shotted) a 3D game that uses OpenGL or DirectX with locally hosted LLMs?
Typically I'd think of Unity for serious game dev, but I figure why rule anything out?
If I could import a fully rigged 3D model into my solution and ask the LLM "create animation logic for this FBX" that would be amazing.
I am still trying to gauge the limits of these things, and want to know if anyone's done this before.
Hi,
I have a Ryzen 9900x, a MSI Tomahawk 870 Wifi and 64GB of RAM.
I bought an R9700 AI pro 32GB last week, to replace my NVDIA 4070.
After 2 solid weeks of tinkering with LLMs and getting advice from you, I am thinking 32GB might not be enough for running competent AI models with large context for refactoring (I may be wrong though). I would like to install my 4070 as a second GPU just for its VRAM.
Has anyone done this on AM5? Did your GPU bottleneck due to PCIE speed halving? Did llama have any issues splitting layers? Is it even worth doing?
Or do you know of an affordable motherboard that can accommodate two GPUs that run at full speed?
I'm interested to know what affordable motherboard, if any, can handle both cards at their full speeds.
I want to occasionally game with the R9700 as well (it is a slightly underclocked 9070XT after all) so any setup that won't drop FPS is ideal.
there's someone always screaming at her man on Bruce Road in the small hours. They both sound heavy junkies. This goes on night after night.
I can hear her screaming from streets away. I know its Bruce Road, neighbours told me.
What is going on? Anyone know what score is with the street? I am seriously considering moving out.
Edit:
Let me rephrase the question: is Bruce road notorious? Because if it is where they dump the jailbirds, junkies and homeless I'll move out as quick as I moved in.
Edit:
Moving out from Gallowhill road back to where I was.