u/ikeo1

I ran the same Qwen Image 2.1 ComfyUI workflow on two Mac Studios and a DGX Spark. The gallery shows the timing overview, matched poster outputs and matched café outputs.

Warm poster medians, three runs per variant, 1024 × 1024:

System	Base, 40 steps	Turbo, 8 steps
M5 Ultra, 256 GiB	34.826 s	7.551 s
DGX Spark GB10, nominal 128 GB	38.627 s	8.373 s
M3 Ultra, 96 GiB	155.503 s	32.483 s

Turbo was about 4.6–4.8× as fast in this workload. M5 finished about 10% sooner than Spark. All 24 poster renders contained the three requested text lines in our manual review. The café examples showed more fine texture with base and a smoother look with Turbo; two prompts are not a general quality ranking.

Both Macs used PyTorch/MPS, Spark used CUDA, and all three used the same verified BF16 diffusion/VAE checkpoints with effective FP16 text encoding on GPU. This is not an MLX comparison. Same prompts, seeds, sampler and CFG; Turbo kept its published eight-step schedule. Mac OS/driver versions differ, so these are configurations as tested. Warm conditioning/prefix cache was reused, but each repeat generated a new image with a new matched seed.

My new Mac sits atop my first Mac. The thirty year tyranny of the tower is over at last!

Those are out of the box, some1 with a preview access got the machine early and runs benches on oMLX which collects them and publishes.

Qwen27b 4b performance looks to be on par with rtx4090 (unsurprising considering specs).
>Qwen3.8-27B-MLX-4bit 4bit 8k 1,800 50.6

GLM-5.3 flash 4b numbers look quite bad tbh. I'm expecting twice that for ds4 flash (not 4.1) and qwen3.8-flash-next fp8.
>GLM-5.3-Flash-oQ4e 4bit 16k 955.4 33.0

edit:
>Qwen3.8-Flash-Next-oQ4e-mtp 4bit 64k 2,574 79.9
just appeared, looks not bad

edit2:
results are gone from the page for me now, there's a screen of them in the comments.

https://omlx.ai/benchmarks/performance?chip=M5&device=%5B%22M5%22%2C%22Ultra%22%5D&hide_specprefill=1

[deleted]

New to LLMs. Been running LM studio at first and then later tried oMLX.

Prompt processing stays around ~100tps and generation is at average of ~6.5 fps as if I was running entirely on the CPU. I don't know where to even start with this. I know 27b Qwen is a dense model, but surely it cannot be this bad? I see other people get multitudes faster token generation on their M3 Max machines, what am I doing wrong here?

using omlx installed through brew , pi agent.

Here's the benchmark result:

oMLX - LLM inference, optimized for your Mac

https://github.com/jundot/omlx

Benchmark Model: Qwen3.6-27B-MLX-8bit

Engine: Auto

================================================================================

Single Request Results

--------------------------------------------------------------------------------

Test TTFT(ms) TPOT(ms) pp TPS tg TPS E2E(s) Throughput Peak Mem

pp1024/tg128 8444.6 136.35 121.3 tok/s 7.4 tok/s 25.761 44.7 tok/s 28.34 GB

pp4096/tg128 30611.1 114.53 133.8 tok/s 8.8 tok/s 45.157 93.5 tok/s 29.80 GB

Continuous Batching

pp1024 / tg128

--------------------------------------------------------------------------------

Batch tg TPS Speedup pp TPS pp TPS/req TTFT(ms) E2E(s)

1x 7.4 tok/s 1.00x 121.3 tok/s 121.3 tok/s 8444.6 25.761

2x 15.3 tok/s 2.07x 138.8 tok/s 69.4 tok/s 14755.6 31.533

4x 19.9 tok/s 2.69x 138.8 tok/s 34.7 tok/s 29156.1 55.226

I managed to run the following quant of Qwen3.8 Flash: https://huggingface.co/AtomicChat/Qwen3.8-Flash-Next-GGUF/tree/main/Qwen3.8-Flash-Next-AD-4.27bpw-Q4_K_M-M64

The MacBook

14 Inch

M3 MAX 40c GPU

512GB SSD

64GB RAM

First, downloading the model

I went with Qwen3.8-Flash-Next-AD-4.27bpw-Q4_K_M-M64, it has a larger memory footprint, but it should provide the best results. I used the following script to download all files:

hf download AtomicChat/Qwen3.8-Flash-Next-GGUF \ --include "Qwen3.8-Flash-Next-AD-4.27bpw-Q4_K_M-M64/*"

This puts all the files in the ~/.cache/huggingface/hub/models--AtomicChat--Qwen3.8-Flash-Next-GGUF directory.

Compiling latest llama.cpp

The latest master branch of llama.cpp contains the fix for the qwen4next architecture, so I cloned that repo and built from source:

# 1st time clone
git clone https://github.com/ggml-org/llama.cpp.git ~/code/llama.cpp

After that I built the llama-cli and llama-server with the flags for Apple Silicon:

# Pull and recompile for Apple Silicon
cd ~/code/llama.cpp &&
git pull &&
rm -rf ~/code/llama.cpp/build &&
cmake -B build -DBUILD_SHARED_LIBS=OFF -DGGML_METAL=ON &&
cmake --build build --config Release -j --target llama-cli llama-server

This will give you a llama-server binary compiled for your system, which you can use via the command:

~/code/llama.cpp/build/bin/llama-server --model ~/.cache/huggingface/hub/models--AtomicChat--Qwen3.8-Flash-Next-GGUF/snapshots/142262902a46f7daed19c79d0771534c8106ad59/Qwen3.8-Flash-Next-AD-4.27bpw-Q4_K_M-M64/Qwen3.8-Flash-Next-AD-4.27bpw-Q4_K_M-M64-00001-of-00033.gguf \
      -ngl 99 \
      -fa on \
      --jinja \
      -c 128192 \
      --port 8080

A few notes here:

Context is 128k

FA is ON

Port is 8080 (or whatever you prefer)

This will not run unless you assign more that the default RAM for the GPU on the MacBook (amount_of_gpu_ram * 1024).

You can do this via the command for 56 GB:

sudo sysctl iogpu.wired_limit_mb=57344

or 59 GB:

sudo sysctl iogpu.wired_limit_mb=60416

Or 60 GB (60 * 1024)

sudo sysctl iogpu.wired_limit_mb=61440

After this, you can see the llama-server start up and fill the memory.

The latest llama.cpp patch allows the necessary parts to be streamed from disk, so the model fits into the assigned RAM.

The prompt: create an animated SVG of a pelican riding a motorcycle in a leather jacket. Add a transition into the night.

I used this prompt over various models to test their capability, it requires more than just static SVG generation, and tends to take longer to complete. I can say that Qwen3.8 Flash has provided the best result so far in terms of detail and animations.

The performance

It started off with above 20 TPS generation speeds, which was impressive. At the end of the context window (128K) it reached around 12 TPS. PP speed was around 300.

GPU bandwidth was around 15-20GB/s, which I assume is due to the SSD streaming.

It loaded the model only once from SSD (60 GB hit) and gradually loaded more from disk as it continued. I could see this in the Disk tab from Activity Monitor.

The result

When it comes to detail, I have not seen any result like this from a local model.

Qwen3.8 27B does a good job and is faster, but doest not compare on terms of quality to Qwne3.8 Flash.

The animation is also smooth, well designed and glitch-free.

It also created a python script to take snapshots of the transition to verify that the timings are correct and that the Z-index for each element is done correctly, really impressive.

The verdict

This was a long-horizon task, it took 117 minutes and did not complete because it kept on adding details to the SVG (stars, grass, glow, etc)

An earlier attempt did succeed within the provided context.

I've seen this with other MoE models, too. Qwen3.6 35B A3B does the same thing, it keeps on trying to improve the design.

Recommendations

Personally, I would keep this model around for specific coding tasks that do not require a large context. I would also move this model to an USB4 NVME enclosure to make sure the internal SSD does not wear out.

I pinned the fans to 100% using TG PRO, it kept things managable (around 90C) but it sounds like a jet engine doing so.

20 TPS is fine, it als varies from 20 to 12 back to 17 so I would say stable (as far as the machine is concerned).

Memory-wise, its really tight, the memory pressure is mostly green, goes into amber often. You cannot run anything else on the machine whilst the model is loaded.

Congratulations to the LLAMA.CPP team for the effort in making this possible!

EDIT: a second attempt gave me this: https://files.catbox.moe/pl0bff.svg

Second EDIT: I tried running the whole GGUF from an external SSD, llama-server did not accept an APFS partition as the source, but it does accept the exFAT partition.

I suppose this is due to encryption and llama being unable to read from that...

EDIT 3: just make sure Terminal.app has full disk access and quit terminal before running llama-server to give it access to the external disk (being APFS or exFAT)

APFS is going to be faster.

EDIT 4: another attempt at the same prompt: https://files.catbox.moe/z5pfdv.svg

We’re glad to share that we’re official sponsors of Oh Shoot at Prado Olympic Shooting Park. We’re looking forward to working with the Oh Shoot team and expanding the availability of shooting sports and training for people in SoCal.

Thanks to everyone for the support while we continue to bring information and training to people in California!

We’re proud to support the largest club in Southern California! The Oh Shoot team does an amazing job and we’re happy to join them as a sponsor.

Hey there,

It's been awhile since I've checked in on the texas side but seems like you have some good options with https://texasarmsexchange.com .. I'm still updating texanarmory.com with all fixes and patches but just curious what do you think of texasguntrader so far? It seems like they've largely rebounded from the security hack or people just continue to deal with the issues there.

I'm just dropping in as we have some modern platforms and looking to see how texas can get the word out. Also, would you guys be interested in more classes or training of that sort with different instructors?

The other community I've been building up is up to 42k members so the software is good and battle tested. So I'm curious what you guys would need to make life better.