Running modern local AI natively on Monterey on a Mac Pro 5,1
The interesting part of this exercise is not really that a Mac Pro 5,1 can run an 8B language model. With enough RAM and a vaguely modern GPU, that was always likely to be possible somehow.
What I wanted to know was whether it could be done natively under macOS Monterey, without installing Linux, without moving the machine onto an unsupported newer version of macOS, and without retreating to CPU-only inference.
The answer is yes. This solves a particular problem of mine: local AI being the only reason I considered moving to OS X 14. However, that comes with a distinct downgrade in GPU.
I now have llama.cpp running natively on Monterey 12.7.5 on a dual-X5680 Mac Pro 5,1, with an RX 6600 XT doing full GPU offload through Vulkan/MoltenVK. Qwen3-8B runs at about 33–34 tokens per second and Qwen2.5-Coder-7B at about 39.
That is, I think, the more useful result here. Monterey is old enough that quite a lot of current local-AI tooling either no longer targets it, assumes a newer Apple toolchain, assumes Apple Silicon, or assumes an x86 processor with AVX. The Mac Pro 5,1 manages to offend all four assumptions simultaneously.
The machine itself is:
Mac Pro 5,1
macOS Monterey 12.7.5
2 × Xeon X5680
12 physical / 24 logical cores
64 GB RAM
RX 6600 XT 8 GB
NVMe SSD
The principal nuisance is Westmere. The X5680 has SSE4.2, but no AVX, AVX2, FMA, F16C or BMI2, so simply downloading a contemporary binary is rather optimistic. The solution was to compile llama.cpp myself, explicitly disable the unsupported instruction sets, and use the Radeon through Vulkan.
The relevant build was:
/opt/local/bin/cmake -S . -B build-vulkan \
-DCMAKE_BUILD_TYPE=Release \
-DGGML_METAL=OFF \
-DGGML_VULKAN=ON \
-DVulkan_INCLUDE_DIR=/opt/local/include \
-DVulkan_LIBRARY=/opt/local/lib/libMoltenVK.dylib \
-DVulkan_GLSLC_EXECUTABLE=/opt/local/bin/glslc \
-DGGML_NATIVE=OFF \
-DGGML_SSE42=ON \
-DGGML_AVX=OFF \
-DGGML_AVX2=OFF \
-DGGML_FMA=OFF \
-DGGML_F16C=OFF \
-DGGML_BMI2=OFF \
-DGGML_OPENMP=OFF
Then:
/opt/local/bin/cmake --build build-vulkan --target llama-cli llama-server -j 6
I linked directly against MacPorts' MoltenVK library, which avoided some rather tedious Vulkan-loader trouble under Monterey and spared me an evening of studying loader paths with the concentration normally reserved for disputed Balkan frontiers.
The resulting performance has been:
| Model | Backend | Threads | Prompt t/s | Generation t/s |
|---|---|---|---|---|
| Qwen3-1.7B Q4_K_M | CPU | 6 | 23.4 | 5.7 |
| Qwen3-1.7B Q4_K_M | CPU | 12 | 19.5 | 7.6 |
| Qwen3-1.7B Q4_K_M | CPU | 24 | 16.7 | 6.0 |
| Qwen3-1.7B Q4_K_M | Metal, 10 layers | 12 | 18.3 | 6.3 |
| Qwen3-1.7B Q4_K_M | Metal, full | 12 | 18.4 | 4.1 |
| Qwen3-1.7B Q4_K_M | Vulkan, 1 layer | 12 | 19.3 | 11.3 |
| Qwen3-1.7B Q4_K_M | Vulkan, full | 12 | 31.7 | 65.6 |
| Qwen3-8B Q4_K_M | Vulkan, full | 12 | 157.0 | 33.7 |
| Qwen3-8B Q4_K_M | Vulkan, full | 24 | 192.9 | 33.0 |
| Qwen2.5-Coder-7B Q4_K_M | Vulkan, full | 12 | 153.2 | 39.0 |
| Qwen3-14B Q4_K_M | CPU/GPU hybrid | 12 | 3.5 | 1.9 |
The prompt figures should not be treated as laboratory measurements, since the prompts differed. The generation figures are rather more useful.
A few things emerged. Twelve physical cores were faster than twenty-four logical ones on the 1.7B model. Hyper-Threading remains a useful invention, but it is not an ecclesiastical miracle: twelve Westmere cores do not become twenty-four merely because the operating system has been persuaded to count them twice.
The 8B model behaves differently under full Vulkan offload. At 24 threads, prompt processing rose from 157 to 193 tokens per second while generation remained essentially unchanged: 33.7 versus 33.0.
Metal was dreadful. I did eventually persuade the RX 6600 XT to run through Metal, but only with a shared-buffer workaround. Even then, full Metal offload managed 4.1 tokens per second on Qwen3-1.7B. The same model through Vulkan produced 65.6.
The natural size for this machine seems to be 7–8B Q4 models. Qwen3-8B runs at roughly 33–34 tokens per second and Qwen2.5-Coder-7B at 39. Both feel entirely normal in use.
Qwen3-14B is another matter. It runs, because 64 GB of RAM leaves ample room for it, but at roughly two tokens per second once much of the work falls back onto the Xeons.
My normal Qwen3-8B invocation is:
cd ~/llama.cpp
VK_ICD_FILENAMES=/opt/local/share/vulkan/icd.d/MoltenVK_icd.json \
./build-vulkan/bin/llama-cli \
-m ~/llama-models/Qwen3-8B-Q4_K_M.gguf \
-ngl 99 \
-c 4096 \
--threads 24 \
--jinja \
--temp 0.6 \
--top-k 20 \
--top-p 0.95 \
--min-p 0
I now have several models on the machine. Qwen3-8B is the ordinary general-purpose model. Qwen2.5-Coder-7B handles R and programming. Qwen3-VL-8B is there for images and screenshots. SDXL runs separately through stable-diffusion.cpp for image generation.
The next step is to stop treating these as amusing Terminal demonstrations and expose them through llama-server, so that RStudio can call them locally. The useful arrangement is not to ask an LLM to pretend to perform statistical analysis. It is to let the model write the R code, let R execute it, and then return the real output to the model for draft interpretation. That begins to resemble an actual local agent rather than a chatbot.
There is also something pleasingly appropriate about the whole exercise. The Mac Pro was sold as a large, expandable workstation with too many drive bays, too much memory capacity and processors intended to sit under sustained load. Fifteen or sixteen years later, one can put in a modern Radeon, 64 GB of RAM and an NVMe drive and set it to work running local language models. In other words, the thing is still doing precisely what it was built to do; only the computation being brought upon the hardware has changed.
It is not the fastest AI computer one can buy.
It has, however, already been bought, remains almost absurdly repairable by modern standards, and runs 7–8B local models quickly enough that I have no immediate reason to replace it. Not bad for 2010, hey?!