u/Commercial-Catch-680

TL;DR: Found a fix for 100% CPU usage of llama-server on AMD RX 6600 GPUs. Fixes in github link at the end!

Disclaimer: Claude Opus 5.5 was used to run traces and find the fixes! Posting here so that others in the same boat might find it helpful.

---

I've been using gemma4:e4b-qat locally on my server for a couple months now.

Here's my setup:

CPU: i3-10100F

RAM: 32GB (4x 8GB)

Board: MSI Z490 Gaming Plus

GPU: 7x AMD RX 6600

I had this rig from ETH mining a couple years ago when that was a thing! It was sitting there collecting dust for a long time, so I had an idea to use it to run some local LLM experiments and started tinkering with it. After testing different models, settled on gemma4:e4b-qat at 128k context which runs comfortably at 5.5/8GB VRAM (tell me if you found a better model for the 8GBs).

I am running an app that takes a list of stock tickers and runs the TradingAgents (https://github.com/tauricresearch/tradingagents) pipeline on them twice a day (market open and market close) as a fun side project to see how good that is.

So, for running TradingAgents which makes about 30-40 LLM calls per ticker, I am running a custom llama-pool which runs a llama-server per GPU with gemma4:e4b-qat model at 128k context. The thing I noticed was that my CPU is at 100% while it's running the pipeline so I started digging into it using Claude Opus 5.5. After a couple hours of testing, traces, LLM calls, it found a fix and applied them which reduced the CPU usage down to ~20% for 7 parallel llama-servers.

The fixes and instructions are all published here: https://github.com/nandyalu/rocm-llamacpp-cpu-fix

For those of you that doesn't know about Trailarr, here's a short description.

Trailarr is an open-source app for downloading/managing [local] trailers for your media library. It's a docker app that comes with everything pre-installed (yt-dlp, ffmpeg with hardware acceleration, etc) and has a UI, downloads trailers from Youtube and places them alongside your movies and series in a format that Plex agrees with. Configurable options for resolution, codecs, file naming, etc., for trailer downloads.

Plex Integration: Trailarr has been around for a while now (first release Aug-2024), and needed Radarr/Sonarr to work until now. I have worked on integrating Plex into the app (with Plex OAuth - get's it's own token, no need to hunt for the admin Plex token) so it's no longer depends on Radarr/Sonarr and can be used with only Plex without Radarr/Sonarr.

Can detect if movie/show has a Plex Pass trailer and skip downloading local trailer. Minimum resolution of Plex Pass trailer required to skip can be configured.
Can send Library Scan notification to Plex to scan the movie/show folder on trailer download.

Released in v0.9.0 yesterday, looking for feedback and suggestions for improvements.

Links:

Note: I wrote this entire post and majority of the app's codebase! As mentioned above Trailarr has been around for about 20 months now, not vibe coded. I do use AI sometimes to get ideas and brainstorm some ideas, however, the app is mostly me. And yes, I can write markdown - I wrote all of the Trailarr docs in markdown.

Found a fix for AMD RX 6600 100% CPU usage