I developed an LSP, VS-Code extension and NPM package, please try it out and give me your thoughts!

Local AI hardware is basically a religious war with better benchmarks.

NVIDIA RTX GPUs are the sports cars: fast VRAM, CUDA, absurd token throughput if the model fits.

Apple Silicon is the weirdly elegant camper van: unified memory means you can often fit much larger models locally, especially on something like an M4 Max with up to 128GB RAM.

So the tradeoff is simple:

RTX = faster kitchen

Mac = bigger fridge

I run Qwen 3.6 27B locally on an RTX 5090 inside Thoth because 32GB VRAM is the sweet spot for my daily driver setup: fast, private, and no API round trips.

But Thoth is designed local-first, not NVIDIA-first.

Ollama, llama.cpp, OpenAI-compatible local endpoints, the point is that your AI should run where you want it to run.

Your machine. Your models. Your memory. Your data. Cloud optional. Local by default.

I need some advice. I’m a family doctor and I’d like to use a local model to help me reconstruct the medical history of my new patients the day before their appointment.

Here’s the idea: for each patient, I paste the text content of their available medical reports (without personal information) into the chat and ask the model to generate a short summary of the patient’s medical history and the tests performed, along with their results. Being able to get a sense of the patient before even seeing them would be a huge help, but I don’t want the data to leave my computer.

My computer is a laptop with an Intel 155H processor and 32GB of DDR5 RAM. Which model could I use? Or would the models suitable for my computer not be able to do a decent job?

I am very excited about the future of local AI. With the spread of AI agents, the amount of VRAM now achievable locally, the quality of small and medium LLMs, and the community growing around all of this, the future is looking very good.

I am writing this to document my successes with the following:

• Mac Pro 2019
• Ubuntu 24.04.4 LTS (Ubuntu Server specifically, in my case)
• Dual AMD Radeon PRO W6900X with Infinity Fabric Link Bridge
• Dual AMD Radeon PRO W6800X Duo with Infinity Fabric Link Bridge
• ROCm 7.2.3
• PyTorch 2.10
• Triton 3.6
• vLLM (Write up pending)
• Hermes Agent (Research Pending)

I wrote a couple of old guides. Check them out for reference, as needed:

• Linux & Local LLM/AI (Useful for moving from macOS the first time)
• Proxmox, Ubuntu, ROCm, & Local LLM/AI (Useful for Proxmox setup)

I'm going to focus on setting up Ubuntu and all the packages needed for the infrastructure of local AI.

Important: This is an experimental community guide. Some parts involve patched kernels, unsupported GPU configurations, and boot-level PCIe changes. This worked for my Mac Pro 2019 systems, but you should expect troubleshooting, and you should be comfortable recovering from a failed boot. I am not responsible for any outcome of using this guide, whether it be positive, negative, or anything in between.

1. Choices & Decisions

• Mac Pro 2019: It's what I had available to me.
• W6900X: It's what I had available to me.
• W6800X Duo: It's what I had available to me.
• Ubuntu LTS: The ROCm-supported OS family I am most comfortable with. Alternative: RHEL
• Ubuntu 24.04 LTS: The latest Ubuntu LTS version supported by ROCm at the time of writing. Alternative: Ubuntu 22.04 LTS
• Ubuntu Server: To avoid desktop overhead and keep the system headless. Alternative: Ubuntu Desktop LTS
• Data Room: I placed the Macs in a Data Room, so I don't hear the loud fans. Alternative: Place it at your desk, or anywhere else.
• DRM/AMDGPU: I opted to use the GPU driver in the kernel, to patch it to support the Infinity Fabric Link Bridge. Alternative: Install DKMS and AMDGPU.
• Kernel: Patched Ubuntu 6.17 HWE kernel, based on Ubuntu’s linux-hwe-6.17 source package, to support the Infinity Fabric Link Bridge. Alternative: Standard Ubuntu kernel.
• ROCm: AMD’s CUDA alternative for AMD GPUs. Alternative: Vulkan
• ROCm 7.2.3: Latest ROCm that supports my GPUs at the time of writing. Alternative: Outdated ROCm.
• vLLM: Concurrent utilization of loaded LLMs. Alternative: Ollama & Llama.cpp
• Hermes Agent: More tool-savvy and self-learning. Alternative: OpenClaw
• GitHub: All my files and commands have been uploaded to GitHub, to make this guide shorter than 40,000 characters. Alternative: Multiple Guides...

Please let me know if the GitHub links do not work.

These are the choices I made, and I am still refining them. They work for me. Keep in mind that this is all held together with the digital equivalent of duct tape. If you change anything, it may or may not work. If you do, I would genuinely appreciate hearing what you tried, what worked, what failed, and why you changed it.

2. Setting up Ubuntu after Installation

Step 00: Infinity Fabric Link (Jumper & Bridge)

Please remove the Infinity Fabric Link Jumper(s) or Bridge from the GPU. Ubuntu 24 kernels do not currently support it, as of 6.17.

Specifically, with kernel 6.8, none of the GPUs will work. When upgrading to 6.17, only one GPU will work.

If you have an Infinity Fabric Link Jumper or Bridge, follow the patch section later in the guide to make it work with your GPUs.

Step 01: Update, Upgrade, and Tweak the System

What we will do:

• Change ubuntu.sources from http to https
• Attach to Ubuntu Pro (This is optional, and requires interaction)
• Update & Full-Upgrade
• Upgrade to the latest HWE kernel
• Remove cloud-init
• Make all Ethernet ports accept DHCPv4 automatically
• Modify Grub to include "loglevel=7 log_buf_len=16M iommu=pt" kernel flags
• Reboot

Copy the following command into your command line interface of choice:

curl -fsSL "https://raw.githubusercontent.com/FaisalBiyari/MacPro2019LocalAI/refs/heads/main/Reddit/Mac%20Pro%202019%20Local%20AI%20Guide%3A%20Ubuntu%2024.04%2C%20ROCm%207.2.3%2C%20PyTorch%202.10%2C%20and%20Infinity%20Fabric%20Link/2.%20Setting%20up%20Ubuntu%20after%20Installation/Step%2001%3A%20Update%2C%20Upgrade%2C%20and%20Tweak%20the%20System" | bash

Step 02: Install T2 Linux Repository

Since we are using a Mac Pro 2019, which is a Mac with a T2 chip, some additional packages are required to be able to properly communicate with the hardware.

What we will do:

• Set up the T2 Ubuntu 24 (Noble) Repository
• Install 3 Packages: applesmc-t2 apple-bce t2fanrd
• Reboot

Copy the following command into your command line interface of choice:

curl -fsSL "https://raw.githubusercontent.com/FaisalBiyari/MacPro2019LocalAI/refs/heads/main/Reddit/Mac%20Pro%202019%20Local%20AI%20Guide%3A%20Ubuntu%2024.04%2C%20ROCm%207.2.3%2C%20PyTorch%202.10%2C%20and%20Infinity%20Fabric%20Link/2.%20Setting%20up%20Ubuntu%20after%20Installation/Step%2002%3A%20Install%20T2%20Linux%20Repository" | bash

Step 03: Enable T2 Fan Daemon

After installing the T2 packages, the command below is used to activate the fan service.

What we will do:

• Enable the t2fanrd systemd service

Copy the following command into your command line interface of choice:

curl -fsSL "https://raw.githubusercontent.com/FaisalBiyari/MacPro2019LocalAI/refs/heads/main/Reddit/Mac%20Pro%202019%20Local%20AI%20Guide%3A%20Ubuntu%2024.04%2C%20ROCm%207.2.3%2C%20PyTorch%202.10%2C%20and%20Infinity%20Fabric%20Link/2.%20Setting%20up%20Ubuntu%20after%20Installation/Step%2003%3A%20Enable%20T2%20Fan%20Daemon" | bash

Step 03-Optional: Set Fans to Maximum

I do not trust Apple Cooling. I would rather the fans wear out and replace them for a few dollars, versus the GPUs (especially the Duo models) being damaged due to overheating.

What we will do:

• Set all 4 fans to maximum speed

Copy the following command into your command line interface of choice:

curl -fsSL "https://raw.githubusercontent.com/FaisalBiyari/MacPro2019LocalAI/refs/heads/main/Reddit/Mac%20Pro%202019%20Local%20AI%20Guide%3A%20Ubuntu%2024.04%2C%20ROCm%207.2.3%2C%20PyTorch%202.10%2C%20and%20Infinity%20Fabric%20Link/2.%20Setting%20up%20Ubuntu%20after%20Installation/Step%2003-Optional%3A%20Set%20Fans%20to%20Maximum" | bash

Step 04: Download and Install ROCm 7.2.3

This section will install ROCm 7.2.3, but it will NOT install dkms or amdgpu drivers. I opted to use the kernel driver, drm/amdgpu, so I can later patch it to support the Infinity Fabric Link Bridge.

What we will do:

• Make a new directory to save all downloaded files
• Download ROCm installer
• Install ROCm Dependencies
• Install ROCm
• Give all users access to ROCm
• Add ROCm to path
• Show you a bunch of output displaying your GPUs, which are working with ROCm or the driver, etc.
• Reboot

Copy the following command into your command line interface of choice:

curl -fsSL "https://raw.githubusercontent.com/FaisalBiyari/MacPro2019LocalAI/refs/heads/main/Reddit/Mac%20Pro%202019%20Local%20AI%20Guide%3A%20Ubuntu%2024.04%2C%20ROCm%207.2.3%2C%20PyTorch%202.10%2C%20and%20Infinity%20Fabric%20Link/2.%20Setting%20up%20Ubuntu%20after%20Installation/Step%2004%3A%20Download%20and%20Install%20ROCm%207.2.3" | bash

Step 05: Install Python Tools

We will be using Python and pip to install several packages for local AI. The following commands are to set up the correct versions, as well as some quality of life choices.

What we will do:

• Install these packages: 2to3 python-is-python3 python3-pip python3-venv python3-dev python3-setuptools
• Install or upgrade these packages, system wide: pip wheel setuptools
• Install numpy 1.26.4 specifically, system wide

Copy the following command into your command line interface of choice:

curl -fsSL "https://raw.githubusercontent.com/FaisalBiyari/MacPro2019LocalAI/refs/heads/main/Reddit/Mac%20Pro%202019%20Local%20AI%20Guide%3A%20Ubuntu%2024.04%2C%20ROCm%207.2.3%2C%20PyTorch%202.10%2C%20and%20Infinity%20Fabric%20Link/2.%20Setting%20up%20Ubuntu%20after%20Installation/Step%2005%3A%20Install%20Python%20Tools" | bash

Step 06: Install PyTorch & Other ROCm Related Wheels

Not everything here is needed for everyone. I included what I could, what worked, and what had some value to some local AI use case.

What we will do:

• Install PyTorch Wheels
• Add AMD ROCm APT Repository
• Set AMD ROCm Apt Repository at priority 700 (Higher than Ubuntu)
• Fix some ROCm Symlinks conflicting with MIGraphX
• Install MIGraphX & Half packages
• Install ONNX Runtime package
• Install TensorFlow ROCm package
• Install Apex Wheel
• Clean up packages
• Reboot

Copy the following command into your command line interface of choice:

curl -fsSL "https://raw.githubusercontent.com/FaisalBiyari/MacPro2019LocalAI/refs/heads/main/Reddit/Mac%20Pro%202019%20Local%20AI%20Guide%3A%20Ubuntu%2024.04%2C%20ROCm%207.2.3%2C%20PyTorch%202.10%2C%20and%20Infinity%20Fabric%20Link/2.%20Setting%20up%20Ubuntu%20after%20Installation/Step%2006%3A%20Install%20PyTorch%20%26%20Other%20ROCm%20Related%20Wheels" | bash

Step 07: Verifying Everything

We just completed installing everything in the standard way. We just need to verify that everything is now set up correctly.

What we will do:

• Give you several boxes showing the status of everything we just set up

Copy the following command into your command line interface of choice:

curl -fsSL "https://raw.githubusercontent.com/FaisalBiyari/MacPro2019LocalAI/refs/heads/main/Reddit/Mac%20Pro%202019%20Local%20AI%20Guide%3A%20Ubuntu%2024.04%2C%20ROCm%207.2.3%2C%20PyTorch%202.10%2C%20and%20Infinity%20Fabric%20Link/2.%20Setting%20up%20Ubuntu%20after%20Installation/Step%2007%3A%20Verifying%20Everything" | bash

3. Infinity Fabric Link Jumper / Bridge

AMD released several GPUs specifically for the Mac Pro 2019 that support their Infinity Fabric.

These GPUs and the Infinity Fabric Links are discussed in these posts:

• AMD Radeon Pro Vega II Duo and Infinity Fabric Link
• Linux on Mac Pro 2019: Infinity Fabric Link, Multi-GPU, and the Current State of AMD XGMI Support

The first set of GPUs that support it were the AMD Radeon PRO Vega II & Vega II Duo. The PC equivalent is an AMD Radeon PRO VII, which also supports an Infinity Fabric Link.

The second set of GPUs are the AMD Radeon PRO W6800X, W6800X Duo, and W6900X. These GPUs are in the Sienna Cichlid family of GPUs. Also referred to as RDNA2.

At the announcement of the Sienna Cichlid family, these GPUs were marketed as supporting xGMI. The Infinity Fabric Link is the physical bridge / jumper. xGMI is the software path that allows the GPUs to communicate over that link. However, on release, only the Apple MPX GPUs actually supported the Infinity Fabric Links, while the standard versions did not.

This might explain why support for xGMI on Sienna Cichlid was added between 2019 and 2020 to the Linux kernel drm/amdgpu, but later removed in 2022.

Many of us here in the subreddit tried to figure out the problem with the Infinity Fabric Link, and tried to find a solution to it. One such redditor actually cracked it; creating a patch to the current kernel drm/amdgpu driver, which through my testing seems to have completely solved the Infinity Fabric Link regression that happened in 2022.

You'll need to keep in mind that this is just the first step. While we are moving forward, there is still the question of ROCm support, HIP support, and everything else.

Step 01: Download, Build, & Install the Patched Kernel Files

Let's start. We will do the following:

• Make a directory to download kernel source
• Install packages required to patch the kernel
• Activate the source to download kernel source
• Patch drm/amdgpu
• Build a full patched kernel
• Install the patched kernel

Copy the following command into your command line interface of choice:

curl -fsSL "https://raw.githubusercontent.com/FaisalBiyari/MacPro2019LocalAI/refs/heads/main/Reddit/Mac%20Pro%202019%20Local%20AI%20Guide%3A%20Ubuntu%2024.04%2C%20ROCm%207.2.3%2C%20PyTorch%202.10%2C%20and%20Infinity%20Fabric%20Link/3.%20Infinity%20Fabric%20Link%20Jumper-Bridge/Step%2001%3A%20Download%2C%20Build%2C%20%26%20Install%20the%20Patched%20Kernel%20Files" | bash

With this, you are now the proud user of a patched kernel that supports the Infinity Fabric Links on the Sienna Cichlid MPX GPUs.

At this point, shut the system down, reinstall the Infinity Fabric Link Jumper or Bridge, then boot back into the patched kernel.

Step 02: Verify Patched Kernel & GPU Initialization

We should probably run a verification one last time. Copy the following command into your command line interface of choice:

curl -fsSL "https://raw.githubusercontent.com/FaisalBiyari/MacPro2019LocalAI/refs/heads/main/Reddit/Mac%20Pro%202019%20Local%20AI%20Guide%3A%20Ubuntu%2024.04%2C%20ROCm%207.2.3%2C%20PyTorch%202.10%2C%20and%20Infinity%20Fabric%20Link/3.%20Infinity%20Fabric%20Link%20Jumper-Bridge/Step%2002%3A%20Verify%20Patched%20Kernel%20%26%20GPU%20Initialization" | bash

While more testing is still needed, this is quite the achievement for the community. Thank you again, anonymous redditor.

4. AMD Duo MPX GPUs and Setting BAR Correctly

I have been using my Mac Pro 2019 with Dual AMD Radeon PRO W6800X Duo for local AI inference for some time now, and I have not had any BAR-related problems. However, since I moved from using Proxmox to having Ubuntu 24 on bare-metal, I have started noticing some BAR warnings and errors.

It seems that this problem may come from the way the Mac Pro firmware allocates PCIe resources before Linux takes over, specifically when using Duo MPX GPUs.

One redditor, whose account is now deleted, shared a GitHub link to what I can only describe as someone's documentation of how he fixed the BAR issue on Vega II Duo GPUs. I have dubbed this the nbritton's method.

Our goal now is to use nbritton's method, adapted for the W6800X Duo. I tried to make it also work as a copy and paste solution for the Vega II Duo as well, but I have not tested it.

Warning: This changes GPU driver load order and PCIe BAR allocation behavior. If something goes wrong, you may need to boot from a recovery kernel, remove the service, or undo the GRUB changes. Also, note that SGLang's AMD GPU documentation recommends pci=realloc=off iommu=pt, which conflicts with nbritton's method because nbritton's method depends on PCIe BAR reallocation behavior. In other words, pci=realloc must not be disabled for this method.

Let's start.

We will do the following:

• Blacklist amdgpu
• Add pci=realloc to grub
• Configure resize-gpu-bars.service
• Set up nbritton's method files

Copy the following command into your command line interface of choice:

curl -fsSL "https://raw.githubusercontent.com/FaisalBiyari/MacPro2019LocalAI/refs/heads/main/Reddit/Mac%20Pro%202019%20Local%20AI%20Guide%3A%20Ubuntu%2024.04%2C%20ROCm%207.2.3%2C%20PyTorch%202.10%2C%20and%20Infinity%20Fabric%20Link/4.%20AMD%20Duo%20MPX%20GPUs%20and%20Setting%20BAR%20Correctly" | bash

5. Finalize the Infrastructure

After completing the linked sections above, we should have:

• Install Ubuntu (You did this on your own or using a previous guide)
• Prepare Ubuntu's environment
• Set up T2 related environment
• Installed ROCm
• Installed PyTorch and several other local AI optimizing software
• Patched the kernel (linux-hwe-6.17, source 6.17.0-29.29~24.04.1) to support xGMI and the Infinity Fabric Link Bridge and Jumper.
• Set up nbritton's method for Duo MPX GPUs BAR correction

Once you're done, please reboot to make sure everything sticks. Then repeat step 07: Verify Everything, above to verify everything is correct and as it should be.

6. Local AI

Now that the infrastructure is ready, it's time to move to our frameworks of choice.

While I definitely plan to expand, I have focused mainly on text generation. When I first started, consideration was Ollama, Llama.cpp, and vLLM. I see new options now, such as SGLang as well.

I am excited to share that vLLM supports this setup and works well. I hope to release a separate guide for it soon.

For the purpose of this guide, I will continue with Ollama, for the simplicity of it, and a Hello World type scenario.

Step 01: Install and Configure Ollama

We will do the following:

• Set up Ollama
• Fix ollama.service vs. ollama serve separate model libraries

Copy the following command into your command line interface of choice:

curl -fsSL "https://raw.githubusercontent.com/FaisalBiyari/MacPro2019LocalAI/refs/heads/main/Reddit/Mac%20Pro%202019%20Local%20AI%20Guide%3A%20Ubuntu%2024.04%2C%20ROCm%207.2.3%2C%20PyTorch%202.10%2C%20and%20Infinity%20Fabric%20Link/6.%20Local%20AI/Step%2001%3A%20Install%20and%20Configure%20Ollama" | bash

Step 02: Verify Ollama Setup

We will do the following:

• Verify Ollama services and data folders permissions

Copy the following command into your command line interface of choice:

curl -fsSL "https://raw.githubusercontent.com/FaisalBiyari/MacPro2019LocalAI/refs/heads/main/Reddit/Mac%20Pro%202019%20Local%20AI%20Guide%3A%20Ubuntu%2024.04%2C%20ROCm%207.2.3%2C%20PyTorch%202.10%2C%20and%20Infinity%20Fabric%20Link/6.%20Local%20AI/Step%2002%3A%20Verify%20Ollama%20Setup" | bash

Step 03: Download and Run Models

We will do the following:

• Download and run our first model

Copy the following command into your command line interface of choice:

ollama run qwen3.5:0.8b --verbose

You can find more models on Ollama's website. Below are some other models I am considering:

ollama pull qwen3.6:27b
ollama pull gemma4:31b-it-q4_K_M
ollama pull granite4.1:30b
ollama pull medgemma:27b
ollama pull mistral-medium-3.5:128b
ollama pull gpt-oss:120b
ollama pull qwen3.5:122b
ollama pull nemotron-3-super:120b

7. Done

With this, we are done with this guide.

It has been a long journey setting up this infrastructure, and preparing for the actual goal.

My testing was done on Mac Pro 2019 systems with dual W6900X MPX modules and dual W6800X Duo MPX modules. I have not tested this with Vega II or Vega II Duo MPX GPU modules.

Next, I plan to focus on vLLM for a while. Optimization, quantization, and automation of operations.

After that, I hope to dive into Hermes Agent by Nous, with the hope of building multiple agents around a few local models run on vLLM, communicating and working together.

Expanding to images or vision, as well as to voice, is also down the pipeline.

The possibilities are endless. I hope to hear what everyone else experiences with this guide and with local AI in general: what worked, what failed, what workloads you are running, what use cases you care about, what problems you hit, and what solutions you found.

Looking forward to seeing how everyone takes advantage of this guide, and local AI.

8. Credit

Credit where credit is due. A lot of the information here was gathered from the community in bits and pieces.

I do want to take the opportunity to thank the anonymous redditor for his/her contribution (creating the whole kernel patch). THANK YOU!

• Nikolas Britton for the nbritton method, fixing the BAR issue on the AMD Duo MPX GPUs.

• u/AdityaGarg8 for always being supportive, no questions asked.

• My AI of choice, for the support through all of this.

• r/MacPro2019LocalAI redditors, for keeping in touch, and motivating me to continue going. You guys are the real MVPs.

---

Disclaimer: I wrote this post myself. I also used AI as a tool to help clean up the wording and formatting.

Resources:

• GitHub Repository hosting these guides.

I want to try to use cause it’s apparently free access to Claude and api tokens I don’t know if it’s some sort of elaborate scam or something?

Hey Reddit, need some help optimizing a workflow setup for my wife without completely overpaying.

Our current home setup uses the $10/mo Google One family plan (2TB). The web version of Gemini is great and rarely gives us limit issues, but she needs to work locally with files and folders for content creation (blogs, social copy, deep content planning—no video or image work).

I tried putting her on the new Antigravity Desktop app to let her work out of her local directories. Huge mistake—30 minutes of multi-file agent work and she completely exhausted a weekly limit. The rate limits on these local desktop apps feel way tighter than standard web chats.

(For context: I run Ollama and OpenCode Go with open-source models for my own programming work, not content writing.)

I have a $200 Codex plan for my business, but sharing it on two devices sounds like a recipe for a messy, overlapping history. I’m debating whether to buy her a separate $20 Gemini Advanced sub to keep it simple, or pivot her over to OpenAI / GPT-5.5.

1. Between Gemini Advanced and OpenAI ($20 tiers), which model actually writes better content? We need something that excels at long-form blogs and strategic planning without sounding robotic.
2. How do I bypass these local app limits without buying another flat subscription? Is there a smarter way to let her work with local folders without hitting an immediate wall?

Thanks for any advice!

As a solo founder of a streetwear brand, I was bleeding cash renting cognition from cloud providers to automate my ops. I recently moved my backend to what I call a Sovereign Stack running Ollama locally on my Windows machine.

It took a bit of configuration, but it gets me 80% of the capability of frontier models for exactly zero dollars. I actually used it to help deploy my latest storefront architecture completely autonomously. If you are a bootstrapped founder and your API bills are scaling faster than your revenue, I highly recommend looking into local inference to handle your routine AI tasks. Happy to answer any questions about the setup!

We need a bot that automatically rejects any post that begins with "I built a..."

https://preview.redd.it/9hyo0sf8x92h1.png?width=1216&format=png&auto=webp&s=e4879686703d6c9eec7469d44776b5b88e436dfd

INFO:
this is the screenshot of cc-switch.
i am using ds-v4-pro with ollama local proxy to cloud.

per session, you gain around 6.5M token ( including cached) (6.3M input, 0.1M output)
* ollama doesn't provide cached token count.

per full session = 16.6% weekly usage

roughtly you can have ard 140M token (included cached).

at 93% cache hit rate (what i see in opencode-go) , it worth to $1 per session only....

i am not gonna to renew my ollama anymore...

Im looking to have a coding agent that can be used in vscode like copilot but with ollama. What can I do to use qwen in vscode? As well as what specs are recommended for someone trying to vibe code projects with a decent quality.

UPDATE: Seems like if I want to get any good alms to run evidently I need to at least do 3k. We'll see how it goes.

Currently using OpenClaw with Claude Opus 4.7 for browser automation workflows — pulling listings, researching properties, drafting documents, running multi-step agent tasks. Paying $280/month between Claude and Codex subscriptions.

Seriously considering a Mac Studio M4 Ultra 192GB to run local AI and cut that bill down. From everything I've read, the best local setup gets you to roughly 85% of cloud quality.

My main questions for anyone who's actually run both side by side:

• For routine browser automation (multi-step tasks, form filling, research workflows) — is the gap noticeable day to day?
• Where does local actually fall short vs Opus in your experience?
• Is the 192GB worth the $7k or does the $3,999 128GB Studio cover most of the same ground?

Not a developer, more of a power user running automated real estate workflows. Privacy is a plus but mainly trying to figure out if the quality drop is something I'd feel constantly or just on edge cases.

I use Claude Code via Ollama to manipulate files and folders on my MacBook.

I’ve tried smaller models like Gemma 4 and Qwen 2.5 Coder in 7B, but they don’t work well (or maybe I just don’t know how to use them properly).

I’ve also tried larger 14B models, such as Qwen2.5‑Code‑14B, but when I run a prompt, my MacBook slows down a lot, sometimes freezes for a few seconds, and I have to wait several minutes. I was wondering if this is normal.