r/macpro

Image 1 — Two months later: ToshLLM has turned Intel Mac Pros + AMD GPUs into seriously capable local AI machines
Image 2 — Two months later: ToshLLM has turned Intel Mac Pros + AMD GPUs into seriously capable local AI machines
Image 3 — Two months later: ToshLLM has turned Intel Mac Pros + AMD GPUs into seriously capable local AI machines
Image 4 — Two months later: ToshLLM has turned Intel Mac Pros + AMD GPUs into seriously capable local AI machines
Image 5 — Two months later: ToshLLM has turned Intel Mac Pros + AMD GPUs into seriously capable local AI machines
▲ 41 r/macpro

Two months later: ToshLLM has turned Intel Mac Pros + AMD GPUs into seriously capable local AI machines

I posted here about ToshLLM roughly two months ago, when I had just gotten local LLM inference working properly on Intel Macs with AMD GPUs by patching llama.cpp's Metal backend and writing an AMD-specific Flash Attention path.

A lot has happened since then.

What started as a way to make my RX 6700 XT useful for local inference has grown into a much broader AMD Metal optimization project, and one of the most interesting things has been seeing other people test it on hardware I don't own: W6900X setups, W6800X Duo, Vega II Duo, W5700X, Vega, Polaris, eGPUs, and multi-GPU Mac Pros.

The AMD backend itself has changed substantially.

Vega, Radeon VII, RX 400/500 and Radeon Pro Vega/WX

These cards now have dedicated Wave64 tuning instead of inheriting settings measured on newer RDNA cards.

On a Vega 64, depending on the model and quantization:

  • generation improved by roughly 4-24%
  • multi-conversation serving improved by 9-12%
  • short cached follow-ups can be up to 39% faster
  • MTP gained about 8%
  • long-prompt processing also improved
  • MoE models can experimentally use the wider matrix path previously used only on newer Radeon cards

Some measured examples:

Workload Before Current
9B Q5_K_M generation 31.2 t/s 38.8 t/s
8B Q6_K generation 35.5 t/s 41.4 t/s
7B / 64-expert MoE 136.0 t/s 155.3 t/s

The goal here isn't just "make it run" anymore. I'm increasingly tuning kernels separately for the different AMD generations.

RDNA has kept getting faster too

A number of quantization and model-specific paths have been optimized.

For example:

  • IQ2_M generation on a 1.5B model: 92.1 -> 140.8 t/s
  • Qwen3.6 14B-A3B: 62.2 -> 70.8 t/s
  • Qwen3.5 4B + MTP: 70.0 -> 81.6 t/s
  • Qwen3.5 4B + DFlash: 68.9 -> 82.6 t/s
  • gpt-oss-20B prompt processing: 1093 -> 1292 t/s

Q4_K, Q5_K, Q6_K, q5_0/q5_1 and several other paths have received AMD-specific kernel work as well.

ToshLLM is becoming useful as a server, not just a chat app

Concurrent inference has improved quite a bit.

On a 9B Q5_K_M model:

  • 2 simultaneous conversations: 56.4 -> 70.8 t/s
  • 3 simultaneous conversations: 60.5 -> 81.0 t/s

There is now a browser UI based on llama.cpp's web interface, router mode can switch between models without restarting the server, multiple servers can run simultaneously, and there is a dedicated embeddings server for local RAG.

The API remains OpenAI-compatible, and I've started seeing people use ToshLLM's bundled llama-server as a backend for other software rather than only using the SwiftUI application. That was not something I expected when I started this.

Image generation has become much lighter on VRAM

This has been another area I've been optimizing.

For example, on my RX 6700 XT:

  • Z-Image 1600x900: 8.2 GB -> 966 MB VRAM
  • Z-Image 1024x1024: 2.4 GB -> 690 MB
  • image generation is roughly 12-14% faster
  • SD 1.5 768x768: 2690 MB -> 281 MB
  • SD 1.5 768x768: 149 sec -> 73 sec

SD 1.5 is now particularly interesting for older 4-6 GB Radeon cards.

There is also experimental local video generation with Wan models now. It has improved substantially and Wan 2.2 5B can run in 12 GB VRAM, but I want to be clear that I still consider the video side experimental and it needs more work before I put it in the same category as LLM inference.

Community testing has become a big part of the project

This is probably the part I've enjoyed most.

People are now submitting benchmarks from configurations such as dual W6900X, W6800X Duo, W5700X, Vega and other machines I could never test myself.

There is a public benchmark database at:

https://toshllm.com/benchmarks

ToshLLM itself is still completely free and GPL-3.0.

No account, no telemetry, no cloud inference. The OpenAI-compatible server, chat, image generation, benchmarks, RAG/embeddings, etc. all run locally.

It's still pre-1.0 and the current DMGs are not notarized yet, so macOS requires Open Anyway on first launch. Notarization is something I'm working toward.

If anyone here still has a Mac Pro full of Radeon hardware, I'm especially interested in results from:

  • W5700X
  • W6800X / W6800X Duo
  • W6900X
  • Vega II / Vega II Duo
  • Radeon VII
  • multi-MPX configurations

At this point some of these old Intel Mac Pros have absurd amounts of usable VRAM for local AI, and I'm curious how far we can push them.

ToshLLM: https://toshllm.com

Source / releases: https://github.com/engeldlgado/toshllm

u/engeldlgado — 1 day ago
▲ 19 r/macpro

My experience using a 2013 Mac Pro 6,1 in 2026

There are already plenty of posts like this, but I figured I'd make another one :,)

Choosing the trash can and upgrading it

First of all, I live in France, so I don't get the same kinds of deals you can find in the US or UK. Despite that, buying one from the UK still made more sense financially than buying one locally in the land of croissants.

Buying the absolute base model didn't seem worthwhile once I factored in the cost of upgrading it, so I went for a compromise: a base machine with 64 GB of RAM already installed. Out of all the upgrades, RAM was probably the most expensive and annoying one to figure out, mainly because there are so many different DDR3 options.

The machine itself cost me £130, plus around €70 for shipping from the UK.

I looked at all the CPU options and eventually decided that the 12-core isn't particularly useful for my purposes. Its single-core performance is too low for general everyday use, and I don't need this machine to act as a server. I went with the middle ground and bought a Xeon E5-2667 v2 from AliExpress for €35.

The storage is still the original 256 GB SSD for now, but I plan to replace it with a 1 TB NVMe drive using an adapter later.

I specifically avoided chasing the D500/D700 models. The D300s are slower, obviously, but I preferred reliability over theoretical performance.

Why did I even want a trash can?

My original idea was to move more demanding tasks (which I don't actually have that many of) from my Apple Silicon MacBook to a separate Intel-based Mac with easy x86 support and lots of RAM. This also means my MacBook can basically become a typewriter. I'm a writer, so most of what I actually need from it is writing and web browsing. The biggest advantage is that the laptop doesn't have to heat up and waste battery doing random background stuff, which should hopefully also help with battery longevity.

I didn't need another MacBook, so that was immediately out.

An iMac wasn't really appealing either. I don't want to deal with removing the display, and considering how unreliable shipping can be here, I also didn't want to depend on finding a seller willing to ship an iMac safely.

A 2017 Mac mini would have been a decent option, but the soldered SSD was a downside for me. I also simply like having external components around a computer, so this particular choice was partly aesthetic.

The Mac Pro 7,1 would be a fantastic experiment, and I'd love to own one eventually, but I'm not ready to spend the money on that experiment yet.

So I ended up with the trash can.

I won't talk about the design; you all know perfectly well that it's a beautiful device. Without the lid, it reminds me personally of the R2-Q5 droid from Star Wars.

What OSes did I install?

macOS Sonoma + OCLP

I installed macOS Sonoma using OpenCore Legacy Patcher.

Why not Sequoia? A lot of the newer features aren't supported anyway, so I don't really see a practical reason to upgrade. Choosing Sonoma is mostly just a personal preference.

The system itself is stable, but there are a few annoying issues caused by running a modern OS on unsupported hardware.

First, I had to give up on Safari because it feels rather slow, especially on YouTube.

I switched to the latest version of Firefox with a Safari-like theme because I prefer that look. Unfortunately, YouTube still has some problems. For example, the video can occasionally get stuck visually while continuing to play in small blocks, and similar glitches randomly appear for a few seconds. I've tried different extensions and various Firefox flags to reduce or disable AV1/VP9-related video support, but I never managed to completely solve it. At this point I'm just too lazy to spend even more time debugging YouTube.

The second issue is the infamous red screen in Chromium-based apps, such as Discord.

The patch available on GitHub didn't work for me, so I launch affected applications through a shell script using Automator. That works perfectly fine. The important thing is to disable the app's automatic launch at login, otherwise you can run into the problem again.

If I eventually get tired of dealing with these little issues, I can always go back to Monterey or install Fedora Linux. I'm already comfortable using the terminal, so we'll see.

Windows 10

Windows 10 installed without any major problems.

I didn't use Boot Camp because I already had my own lightweight Windows build that I preferred.

I don't know whether Windows 11 can be installed, but honestly I wouldn't use it anyway. I personally like it even less than Windows 10.

Everything works surprisingly well (For Windows). The custom drivers installed without any problems.

I don't really need it on this machine, except for the occasional homebrew application that is simply easier to run there without having to perform some ridiculous ritual in macOS.

What pleasantly surprised me?

64 GB of RAM

This was probably the biggest reason I like this machine.

With current Apple Silicon pricing, buying a Mac with 64 GB of unified memory is still way too expensive for me personally. Getting that much RAM in an old Intel Mac for relatively little money was a very nice compromise.

It's not nearly as slow as I expected

Honestly, I expected this thing to be a potato compared to Apple Silicon. It isn't.

Obviously, it is nowhere near an M-series Mac in terms of efficiency or raw modern performance, but for everyday tasks the E5-2667 v2 is surprisingly snappy.

Web browsing, writing, file management and general desktop use don't feel like I'm sitting in front of a 2013 computer.

Universal Control

Surprisingly, Universal Control works between my two Macs with OCLP.

That's a pretty nice quality-of-life feature that I didn't expect to work.

Legacy iOS devices

If you have older iOS devices, especially jailbroken ones, having an Intel Mac around can be extremely convenient.

Some tools and workflows either don't work properly on Apple Silicon or aren't supported at all, so having an Intel machine available can save a lot of trouble.

Native OS X Mavericks

The fact that I can natively install OS X Mavericks on this thing is honestly pretty damn cool.

More than anything, it shows how much support this machine has received from the community over the years. A computer from 2013 still being this flexible in 2026 is kind of ridiculous.

The speaker

I expected the built-in speaker to be absolutely terrible.

It's obviously nothing special, but it has a surprisingly usable volume range. I assumed it was basically only good for the startup sound, but if you just have some music or a video playing quietly in the background, it's perfectly usable.

Given everything I'd heard about it, I expected much worse.

Light gaming

I also have a decent collection of consoles, so I don't really use the Mac Pro for serious gaming.

But occasionally I like playing things such as Terraria, Don't Starve Together or Minecraft.

I actually tried running several of them with mods just as an experiment, and everything worked surprisingly well. Even multiplayer was stable and I didn't have any weird ping issues.

What I don't like

Using both GPUs

Honestly, just forget about trying to make both GPUs useful.

Almost nothing properly supports them, and when something does, getting it to work is usually more trouble than it's worth.

The dual-GPU setup is probably one of the most disappointing parts of this machine from a modern perspective. It's more of a curiosity than something useful

Local LLMs / KoboldCpp

You can run local LLMs through KoboldCpp, but the experience isn't particularly great.

The DDR3 memory isn't exactly fast, and getting proper GPU acceleration working isn't really an option in my current setup, so the CPU ends up doing most of the work.

If anyone has recommendations for getting better performance out of local LLMs on this machine, I'd be happy to hear them.

I might also try doing it properly under Windows, since I basically forgot it existed and ended up doing everything through terminal hacks on macOS as usual.

Final thoughts

Overall, I'm honestly pretty happy with the machine.

I bought it because I wanted a cheap Intel Mac with lots of RAM, x86 compatibility, decent everyday performance and a completely different role from my Silicon MacBook.

And for that purpose, the trash can actually works surprisingly well in 2026.

It's definitely an old computer, and you have to be willing to deal with OCLP, old drivers, weird software issues and the occasional hacky workaround. (4.1, 5.1 owners will throw a slipper at me)

But there's something genuinely impressive about a 2013 workstation still being this usable thirteen years later.

Thanks for reading.

I'm still fairly new to this whole Mac Pro/OCLP stuff, so I'd be happy to hear any advice, corrections or recommendations.

reddit.com
u/Just_Mellow — 1 day ago
▲ 28 r/macpro

Mac Pro 2019 128GB??

As the title suggests but then again it could be mislabelled

u/8Bitoaster — 2 days ago
▲ 3 r/macpro

2013 mac pro

Hey, so I’m from Sweden and there are barely any 2013 Mac Pros being sold here, so I’ve been checking listings 24/7 There are a few on eBay, but I’m not really sure what a good price is.

I’ve seen configs like 16GB RAM, 256GB or 512GB storage, a D300 GPU, and a 4-core or 6-core CPU for around $273 USD. I’m not sure what the absolute maximum I should pay is. I don’t need a super powerful machine it’s mostly just for looks and some light tasks, definitely not a main work computer so I’m guessing somewhere in the $200s USD is reasonable?

Also, what are some good tips, tricks, or settings I should tweak once I get one? :d

reddit.com
u/AttemptSpecialist434 — 2 days ago
▲ 2 r/macpro+3 crossposts

Which Mac?

Hello, I’m looking on getting a new Mac but not sure which one. I currently have the MacBook M4 Air which is a amazing laptop. Performance is good but I am running out of storage and there is a lack of ports. For context I do graphic design, photography, and videography. I think additional ports and storage would be a great advantage. I use a adapter to to transfer my files which slows down speeds. I’m looking at a Mac pro for the additional ports, most definitely looking for a used one. I’m just wondering how much of a difference a M1 Pro is compared to the M4 Air. Would I notice it? Any recommendations would be helpful looking for M1-M4 Pro, seen a few good deals on fb. But I genuinely have no idea if a old M chip would be that much of a difference performance wise. I found a M1 16 gb 1TB for $750 which seems good to me. Has the same amount of ram as my M4 air and 4x the storage. Let me know!

reddit.com
u/Murky-Effort2207 — 3 days ago
▲ 17 r/macpro+1 crossposts

Qwen3.8-27B for the RAM Poor Mac user:

For those of you that want a functional 24GB Mac laptop while having this overthinking creature boosting your ideas.

With and without MTP (for the desperate)

huggingface.co
u/JLeonsarmiento — 3 days ago
▲ 7 r/macpro+1 crossposts

Mac Pro 5,1 Giving Up The Ghost (Multiple Issues Stacking)

I have a Mac Pro 2010 that I have been using quite regularly for 13 years (and it was used when I got it). It is, unfortunately, now failing, and it's hard to diagnose the problem because multiple issues are stacking at the same time.

About a year ago, it would occasionally fail to start; if I pressed the power button, it would whir with some volume (sounding like the fans in high gear) and I'd need to hold the power button to shut it down and try again. It would usually start eventually.

A few weeks ago, one of the hard drives I had installed failed. This did not impede the function of the tower overall, since I had four drives (boot for Snow Leopard, boot for Mavericks, and two storage drives as a mirrored RAID; one of the two RAID slices died, so nothing was lost). I shut down the machine, ordered a replacement drive, installed it, and the machine booted fine. When I tried to rebuild the RAID, though, the entire machine froze (no kernel panic; it just froze). I had to hold the power button to shut it down, and now, it rarely starts. Sometimes, it does the loud whir; sometimes it just doesn't start (the power light turns on and I hear a few soft clicks, but nothing else). On a few occasions, it'll start but freeze again after a few minutes.

These problems started at the same time I tried to rebuild my RAID with a new drive, but that's just correlation not causation. I suspect the failure to rebuild the RAID was caused by the computer dying, rather than the other way around, but I can't be certain since everything is basically happening at the same time.

Any assistance would be appreciated, and I will happily provide clarification and additional information as need be.

Edit: None of the capacitors are blown.

Edit Again: Seems the main problem was that a dust clump got in and lodged onto the heatsink.

reddit.com
u/WoonMySocket — 3 days ago
▲ 3 r/macpro+3 crossposts

RIP Mac book pro

I bought mac book pro m5 chip.After 1 week, my mac fallen down and a dent happened and it’s working with no issues.After using it for 6 months the whole display become black so we went to the near by apple service centre,They have said that they will change the whole screen and because of dent the warranty is not applicable and we have to pay 73,899/- here the point is the black screen is not happened because of the dent. I didn’t have apple care🥺…
#macbookpro

reddit.com
u/Due_Bobcat_4039 — 4 days ago
▲ 6 r/macpro

I'm not sure what I did to my 5,1

Good Morning

I am having difficulty with my cheese-greater. Below is what I am experiencing. I tried to be as detailed as possible, so please excuse the length of my description. Any assistance would be greatly appreciated.

I have a moderately upgraded Mac Pro, 4,1 flashed to a 5,1. Dual Xeon 5690s, dual cd drives,Radeon 580x (8gb) and my HDDs have been replaced with SSDs via adapters in the cradles. In case of emergency, I keep an extra HDD with OS Mojave on it. Up until yesterday, I was running OS Sequoia on OpenCore.

I recently upgraded to a 7,1 and put the old unit in my closet, unused for about a month. I wasn't sure what to do with it.

As it was too good to leave alone, I decided to take it to my office as a replacement for my ancient Mac mini.

No matter what I tried, it would not boot up. I only plugged in the essentials, monitor, keyboard and mouse using a hub for the keyboard and mouse. I could hear a faint chime when I turned it on, but nothing. I then tired my Mojave drive and again, nothing.

Once I replaced the button battery, I was able to get Mojave to boot up, albeit slowly because of the hdd. After a few reboots, it began to work normally under Mojave.

I then did an NPRAM reset, the chime was much louder and Mojave began to run like normal.

When I swap the SSD, nothing happens. I booted in Mojave and added the SSD to see if the computer would read it. No luck, even in all three remaining drives.

I took the SSD home and ran a disk check on it and there were no errors.

I have OpenCore as an app on the Mojave drive and may consider placing a blank drive (if the computer will recognize it) in the computer and attempt to create a fresh Sequoia drive.

I have a full current (as of last month) Time Machine backup on an external drive and if I could get the unit to run, would then try a restore. But that seems to be a great deal of effort.

I am only looking to get it running and use the apps. All my data was transferred to the 7,1.

If anyone has some suggestions, they would be greatly appreciated.

Thank you in advance.

reddit.com
u/AlanTochterman — 4 days ago
▲ 1 r/macpro+2 crossposts

Please help me choose my mac

I am considering a question about purchasing a refurbished MacBook M2 Pro with an Apple M2 Pro chip, 19 GPU cores, 12 CPU cores, 16 GB of RAM, and 1 TB of storage, or a new MacBook Air with an M5 CPU, 10 cores, 8 GPU cores, a 16-core Neural Engine, 16 GB of RAM, and a 512 GB SSD.

Please help me I am a graphic design student seeking a laptop capable of opening numerous tabs and running Adobe applications such as Photoshop and Illustrator simultaneously without lag. Both options cost €1300; the M2 Pro is refurbished, while the M5 Air is new from Apple.

reddit.com
u/No-Revolution4448 — 5 days ago
▲ 113 r/macpro

Finally upgrading - Pro 5,1 to M4 Mini

I've been rocking a MacPro 5,1 for more than 10 years using OCLP as a daily driver and I work in the IT Support field. It works great - no issues other than Bluetooth isn't all that great. But it consumes so much energy and literally heats my office. Been thinking about an upgrade for a while and wish I had before the M4 Mini sold out a few months ago.

Today I bit the bullet and paid $1200 for the 24/512 - BestBuy just got them in stock. Very curious to see how things change / stay the same with the new machine and going to have to find a new name. I don't think naming it, "The Beast" will be quite right.

No one in my IT circle uses Mac and none of my normal friends would have any idea what this change means. So had to share.

u/Zoomer5475 — 7 days ago
▲ 6 r/macpro

2019 Mac Pro PCIE Windows

Has anyone managed to install windows on a PCIE NVME? and do I need to use boot camp?

reddit.com
u/8Bitoaster — 5 days ago
▲ 6 r/macpro

What is the official name of the Mac Pro Power Cable?

Hi,

What is the official name of the Power Socket (Female) and the Plug (Male) on the Mac Pro 5,1?

I'm building something that needs its own power and want to copy the Mac Pro.

Cheers

reddit.com
u/PhilbinFogg — 6 days ago
▲ 30 r/macpro

Decided to replace the front intake fan on my 3,1 as the old one was showing signs of wear

Got around to replacing the PSU intake fan on my 2008 Mac Pro as the old one would make a rattle when the system started up the new fan works very good of course it runs at full speed due to the proprietary pinout but I plan to solve that by getting a PWM fan controller later on as I don’t wanna mangle the fan connecter anymore than I had to

Machine is assembled an both psu temp sensors report 55 and 46c respectively

Photo showing macsfancontrol is just for reference SMC can’t actually control the RPM of the fan but it does detect the speed it is at fan is a Thermaltake TT-1225

u/Mojave0 — 7 days ago
▲ 221 r/macpro+3 crossposts

New setup

-hand built red oak herringbone desk with ikea alex cabinet and pine legs
-dowinx chair in brown
-lg ultrafine 31.5” main
-lg ultrafine 27” vertical
-aula f75 keyboard in white and brown color
-asus smartO mouse in oatmilk color
-ugreen usbc hub for macbook pro
-ugreen usb switch
-ugreen hdmi switch
-“southwestern tribal pattern” desk pad

Vertical monitor is ran off of my 14” macbook pro M4 pro 24gb

Main monitor switches between my 2013 macpro 6,1 with the 8 core chip and 64gb and xbox series x.

Ikea Alex cabinet houses my egpu that i’m currently working on and has a set of dual noctua 120mm fans over a deshrouded rx6800xt feeding through the inlet you can see by my leg space and out of the open back.

Any questions or links needed, please dont hesitate to ask. Also this is my first ever setup and desk build so any constructive criticism or ideas are welcomed.

u/Hefty-Difference5068 — 9 days ago
▲ 7 r/macpro

Running modern local AI natively on Monterey on a Mac Pro 5,1

The interesting part of this exercise is not really that a Mac Pro 5,1 can run an 8B language model. With enough RAM and a vaguely modern GPU, that was always likely to be possible somehow.

What I wanted to know was whether it could be done natively under macOS Monterey, without installing Linux, without moving the machine onto an unsupported newer version of macOS, and without retreating to CPU-only inference.

The answer is yes. This solves a particular problem of mine: local AI being the only reason I considered moving to OS X 14. However, that comes with a distinct downgrade in GPU.

I now have llama.cpp running natively on Monterey 12.7.5 on a dual-X5680 Mac Pro 5,1, with an RX 6600 XT doing full GPU offload through Vulkan/MoltenVK. Qwen3-8B runs at about 33–34 tokens per second and Qwen2.5-Coder-7B at about 39.

That is, I think, the more useful result here. Monterey is old enough that quite a lot of current local-AI tooling either no longer targets it, assumes a newer Apple toolchain, assumes Apple Silicon, or assumes an x86 processor with AVX. The Mac Pro 5,1 manages to offend all four assumptions simultaneously.

The machine itself is:

Mac Pro 5,1
macOS Monterey 12.7.5
2 × Xeon X5680
12 physical / 24 logical cores
64 GB RAM
RX 6600 XT 8 GB
NVMe SSD

The principal nuisance is Westmere. The X5680 has SSE4.2, but no AVX, AVX2, FMA, F16C or BMI2, so simply downloading a contemporary binary is rather optimistic. The solution was to compile llama.cpp myself, explicitly disable the unsupported instruction sets, and use the Radeon through Vulkan.

The relevant build was:

/opt/local/bin/cmake -S . -B build-vulkan \
-DCMAKE_BUILD_TYPE=Release \
-DGGML_METAL=OFF \
-DGGML_VULKAN=ON \
-DVulkan_INCLUDE_DIR=/opt/local/include \
-DVulkan_LIBRARY=/opt/local/lib/libMoltenVK.dylib \
-DVulkan_GLSLC_EXECUTABLE=/opt/local/bin/glslc \
-DGGML_NATIVE=OFF \
-DGGML_SSE42=ON \
-DGGML_AVX=OFF \
-DGGML_AVX2=OFF \
-DGGML_FMA=OFF \
-DGGML_F16C=OFF \
-DGGML_BMI2=OFF \
-DGGML_OPENMP=OFF

Then:

/opt/local/bin/cmake --build build-vulkan --target llama-cli llama-server -j 6

I linked directly against MacPorts' MoltenVK library, which avoided some rather tedious Vulkan-loader trouble under Monterey and spared me an evening of studying loader paths with the concentration normally reserved for disputed Balkan frontiers.

The resulting performance has been:

Model Backend Threads Prompt t/s Generation t/s
Qwen3-1.7B Q4_K_M CPU 6 23.4 5.7
Qwen3-1.7B Q4_K_M CPU 12 19.5 7.6
Qwen3-1.7B Q4_K_M CPU 24 16.7 6.0
Qwen3-1.7B Q4_K_M Metal, 10 layers 12 18.3 6.3
Qwen3-1.7B Q4_K_M Metal, full 12 18.4 4.1
Qwen3-1.7B Q4_K_M Vulkan, 1 layer 12 19.3 11.3
Qwen3-1.7B Q4_K_M Vulkan, full 12 31.7 65.6
Qwen3-8B Q4_K_M Vulkan, full 12 157.0 33.7
Qwen3-8B Q4_K_M Vulkan, full 24 192.9 33.0
Qwen2.5-Coder-7B Q4_K_M Vulkan, full 12 153.2 39.0
Qwen3-14B Q4_K_M CPU/GPU hybrid 12 3.5 1.9

The prompt figures should not be treated as laboratory measurements, since the prompts differed. The generation figures are rather more useful.

A few things emerged. Twelve physical cores were faster than twenty-four logical ones on the 1.7B model. Hyper-Threading remains a useful invention, but it is not an ecclesiastical miracle: twelve Westmere cores do not become twenty-four merely because the operating system has been persuaded to count them twice.

The 8B model behaves differently under full Vulkan offload. At 24 threads, prompt processing rose from 157 to 193 tokens per second while generation remained essentially unchanged: 33.7 versus 33.0.

Metal was dreadful. I did eventually persuade the RX 6600 XT to run through Metal, but only with a shared-buffer workaround. Even then, full Metal offload managed 4.1 tokens per second on Qwen3-1.7B. The same model through Vulkan produced 65.6.

The natural size for this machine seems to be 7–8B Q4 models. Qwen3-8B runs at roughly 33–34 tokens per second and Qwen2.5-Coder-7B at 39. Both feel entirely normal in use.

Qwen3-14B is another matter. It runs, because 64 GB of RAM leaves ample room for it, but at roughly two tokens per second once much of the work falls back onto the Xeons.

My normal Qwen3-8B invocation is:

cd ~/llama.cpp

VK_ICD_FILENAMES=/opt/local/share/vulkan/icd.d/MoltenVK_icd.json \
./build-vulkan/bin/llama-cli \
-m ~/llama-models/Qwen3-8B-Q4_K_M.gguf \
-ngl 99 \
-c 4096 \
--threads 24 \
--jinja \
--temp 0.6 \
--top-k 20 \
--top-p 0.95 \
--min-p 0

I now have several models on the machine. Qwen3-8B is the ordinary general-purpose model. Qwen2.5-Coder-7B handles R and programming. Qwen3-VL-8B is there for images and screenshots. SDXL runs separately through stable-diffusion.cpp for image generation.

The next step is to stop treating these as amusing Terminal demonstrations and expose them through llama-server, so that RStudio can call them locally. The useful arrangement is not to ask an LLM to pretend to perform statistical analysis. It is to let the model write the R code, let R execute it, and then return the real output to the model for draft interpretation. That begins to resemble an actual local agent rather than a chatbot.

There is also something pleasingly appropriate about the whole exercise. The Mac Pro was sold as a large, expandable workstation with too many drive bays, too much memory capacity and processors intended to sit under sustained load. Fifteen or sixteen years later, one can put in a modern Radeon, 64 GB of RAM and an NVMe drive and set it to work running local language models. In other words, the thing is still doing precisely what it was built to do; only the computation being brought upon the hardware has changed.

It is not the fastest AI computer one can buy.

It has, however, already been bought, remains almost absurdly repairable by modern standards, and runs 7–8B local models quickly enough that I have no immediate reason to replace it. Not bad for 2010, hey?!

reddit.com
u/MyCreoleWay — 7 days ago
▲ 13 r/macpro

Mac Pro 1.1 running Debian 13 : iso to share !

So I looked at my old MacPro 1.1 and to be honest I said "you going to live again little buddy".

He has a calmer life now that I use two MacPros for my homelab : Trashcan with 64gb and a MacPro 7.1 with several GPU for AI , the poor 1.1 was not turned on for ... years ?

So I decided to install Debian ... nada. Ubuntu. Fedora. Arch.. Omarchy... some of them booted the usb , installed and then nothing.

So I searched and google and remembered : Mac Pro 1.1 ( and up to 5.1 I think ) have the most stupid arrangement on earth : EFI32 bits and AMD64 Xeon cpus... so booting a linux AMD64 image will result in the distorted wanting to write to EFI64 which the MacPro doesn't have so... no go.

We live in a world of AI so I asked Claude Code to open up the latest Debian 13 distro ISO, inject the EFI32 boot loader ( grub ) and the install process use it.

It worked and got my MacPro 1.1 running latest Debian 13 without a problem. Apart from the idea of having only 6gb ram and slow SATA drives and consume gazzilion of watts, hey, its running.

Anyone want the iso ? Happy to share if requested.

reddit.com
u/gproenca — 8 days ago
▲ 170 r/macpro

Downgrading to Mavericks on trashcan sold by Apple in 2017

Apple made a change in the base config in 2017 and has High Sierra shipped instead of original Mavericks, it's still possible to revert back to Mavericks just like the original 2013 batch?

Reason : I plan to repurpose as an airgapped Mavericks machine to exclusively run my old DTP software suites.

The current boot is Monterey 12.7.6

I have a spare 512GB SSUBX ssd and plan to use it for Mavericks boot volume.

I prefer separate physical boot volume instead of partition and as a boot failsafe just in case.

Cheers!

u/sabtrueday — 10 days ago
▲ 89 r/macpro

embarking on a new mac pro journey

hey everyone, long time no see.

i’ve finally gotten my hands on an eGPU enclosure for my 2013 mac pro in anticipation of my upcoming video about eGPUs and the 2013 mac pro. i previously have done a lot of experimentation with the 2010 mac pro, so this is my first foray into modding such a “modern” mac. i’ve got down tests i plan on doing…. including an RTX card under windows! stay tuned.

u/wyattaj25 — 11 days ago