u/wingers999

Okay, so firstly thank you for all your comments on my other questions about what OS is best to use etc, very useful

I have now tried Windows 11 LTSC IOT Enterprise - I ran the same benchmark tests using llama-bench, and asked same questions using ollama in both and yes LTSC is a much slimmed down version, but in reality the results from my basic testing of local llm were no quicker in LTSC than they were in my full bloated Windows 11 - so don't think that is the route for me.

So think Linux may well be my best route and also a good excuse to learn new areas, so

• I had a look at CachyOS and just didn't really like the interface etc, if I am going to Linux I at least need to like it
• I tried Fedora - seems okay
• Finally for now went with Ubuntu 26.04 as most guides on internet I see seem to refer / relate to using Ubuntu, so for now for a Linux newbie it makes sense I think to go this route.

So onto some questions please:

aI am getting confused with ROCm and what exactly I need to do on Ubuntu (or any other distro) to install it. Seems to be so many mixed comments, suggestion and guides around. For example on the AMD site I see two different pages relating to it

https://rocm.docs.amd.com/projects/radeon-ryzen/en/latest/docs/install/installryz/native_linux/install-ryzen.html

and

https://rocm.docs.amd.com/projects/install-on-linux/en/latest/install/quick-start.html

but not sure which ones I should follow and which steps are actually relevant for me

To refresh my device is a Corsair AI Workstation 300 with AMD Ryzen AI Max+ 395 (Strix Halo) and 128GB RAM, I have set 1GB in the BIOS, so rest is available to be in Linux as I understand it as shared VRAM.

So perhaps someone who uses similar CPU and Ubuntu could guide me through steps I should be doing to utilize the most I can out of this system.

So from a base install of Ubuntu 26.04 what should or shouldn't I be installing.

Much appreciated.

Anyone using Windows 11 for Local LLM - what do you do to "debloat" Windows to get rid of unnecessary RAM hogging background services etc? to make it quicker and free up more resources for LLM?

Or do you instead use a version such as Windows 11 IOT LTSC

Or do you just not use Windows and use Linux instead?

Interested to hear everyone's thoughts on this

Hi

Lots of previous discussions have suggested that instead of Windows 11 I try Linux to get better Local LLM speeds on my Corsair AI Workstation 300 with AMD Ryzen AI Max+ 395 and 128GB RAM

I have some questions if you don't mind so I can make sure I do all of this correctly, as some of my initial tests didn't go so well (see bottom of post):

1) Choice of Distro?

Ubuntu or Fedora

2) Shared VRAM settings in grub and BIOS

A lot of sites say about setting ttm.pages_limit and amdgpu.gttsize

Options seem to be:

a) editing grub and adding:

amd_iommu=off amdgpu.gttsize=131072 ttm.pages_limit=33554432

or

b) install AMD Tools and using amd-ttm to set the shared vram

sudo apt install pipx

pipx install amd-debug-tools

amd-ttm

amd-ttm --set 100

A lot of sites I found the articles are older, so what is the current best way to do this and what should I set both via these settings and in BIOS?

3) ROCM or Vulkan?

Do I use ROCM or Vulkan with Ollama / LM Studio / Lemonade etc?

And if so best way to install / configure e.g. if Ollama what envionment variables need setting

Previous tests and issues

I initially installed Ubuntu 26.04 but had issues with ROCM drivers and found lots of posts about 24.04 being better choice, so installed that

Running models in Ollama seemed to work with Vulkan after adding below:

Environment="OLLAMA_VULKAN=1"
Environment="OLLAMA_FLASH_ATTENTION=1"
Environment="ROCR_VISIBLE_DEVICES="

But without the Environment="ROCR_VISIBLE_DEVICES=" entry I got errors trying to use models with ROCM:

ollama run llama3.3:70b
Error: 500 Internal Server Error: llama runner process has terminated: cudaMalloc failed: out of memory
error loading model: unable to allocate ROCm0 buffer
panic: unable to load model: /usr/share/ollama/.ollama/models/blobs/sha256-4824460d29f2058aaf6e1118a63a7a197a09bed509f0e7d4e2efb1ee273b447d

I then tried LM Studio and it worked fine with Vulkan set as Runtime, but with ROCM I just keep getting "Failed to load model" with no further error info

I also tried Lemonade-Server and that again works with Vulkan but not ROCM

Summary

So what was I doing wrong in initial tests and based on answers to my questions what is my best option to get the best model performance on this system.

Thanks for reading - sorry it is a long post, but wanted to give all detail possible

Anything else you need to know to help then just asks

Lots of advice in my previous question about best tools / models for coding seems to indicate I would be better off running Linux on my AMD Ryzen™ AI Max+ 395 / 128GB RAM device.

I work with Windows and have always used it, so that is my area of expertise, but more than happy to give Linux a try, just want some advice on the best distro to use on this device where I can still then run ollama or lm studio or whatever is best to run in linux and get the most out of my hardware.

just ordered a separate NVME SSD for my device so I can install Linux on separate drive for ease.

so please recommend any linux distros, and software to use once installed.

Also any step by step guides that would help me initially configure it would be great

thanks

Hi I am looking for some advice on the best local models I can use for coding via VS Code or similar, running on a Corsair AI Workstation 300 with AMD Ryzen AI Max+ 395 / 128Gb (96Gb shared for VRAM).

Please suggest best models I can use along with tips on best extensions for VS Code or similar to work with.

So far I have managed to get some models running okay on this device, notes below, but would appreciate comments / help from others using similar spec hardware.

qwen3.6:35b-a3b - works quickly but created code often has issues, regularly gets "stuck" and goes in circles trying to fix obvious errors

qwen3.6:27b - slow than above as expected, but produces better output, although still a lot of coding issues to often fix before projects will build / run

Have tested with VS Code (using Roo Code and Cline extensions), also tried OpenCode but kept getting errors

Found that VS Code extensions worked better when accessing models via Ollama rather than LM Studio, although in testing LM Studio does seem quicker on this system, but definitely not so reliable via Roo Code

In all cases have increased context size as much as VRAM allowed.

Any other suggestions on

a) Models I could try which would give better coding results than qwen3.6:27b ?

b) different tweaks / settings to try to improve performance and output

c) different apps / extensions / IDE's etc to try