u/Hephaestite

I want to share a little project I've been working on for a few weeks now, an entirely C# .NET 10 AOT compiled terminal multiplexer, currently only Linux and MacOS but I do intent to add Windows as a fully supported platformat eventually.

Hypa v1.0

https://github.com/Hypabolic/Hypa

I actually started building this just as proof it could be done and be high performance in .NET as a sort of way of kicking back against basically every tool these days being built in Rust (including Microsoft recently porting Copilot itself to Rust).

My inital aim was to build a simple herdr type mux clone but as I built it I've ended up going beyond that scope and building things not available in herdr.

One feature I really wanted for a start were 'background panes' essentially a normal terminal pane doing whatever you (or an agent) wants it to do but it never appears in the TUI until you ask for it (or the agent running inside it requests it)

The next feature I wanted was remote machines built in natively, which herdr does do, but I wanted to totally avoid things like SSH and managing keys, in Hypa we call these remote machines cubes (hypa cubes...). Hypa uses the QUIC protocol to connect to remote machines and certificate management is handled for you behind the scenes. All you need is the join code and some type of network access to the remote machine.

I've really enjoyed building this as a hobby project, and I hope some of you will give it a go and offer me some feedback.

I look forward to seeing what you all think and getting some feedback, but most of all I'm just happy I can show that these sort of highly performant terminal experiences CAN and SHOULD be built in C# .NET and native AOT is mature enough to support them.

Just for fun this weekend I ended up doing a pure csharp dotnet implementation of the Python tool 'Terminal Text Effects' https://github.com/ChrisBuilds/terminaltexteffects for absolutely no other reason than A) I don't like Python and B) Why not?

Its output is byte for byte identical to what the Python tool would output given the same input and seed values... with the one exception that my dotnet version starts in under 12ms and the Python version is somewhere around 86ms.

All built as native AOT and no dependencies.

https://github.com/Hypabolic/Hypa-TTFX

If anybody wants it I might publish it as a library (or you can just fork it and do it yourself)

-- Edited to include the link to my own code that I totally forgot to include because I'm an idiot lol

I've been looking at whats possible with older hardware for a while now, previously getting some decent performance out of an ancient 'Trashcan' 2013 Mac Pro -- A truly stunning 18 tok/s Gen on Qwopus3.5-9B-Coder-MTP lol.

Today I've been testing my 2019 Mac Pro now that llama.cpp can use the Metal based cards in these machines (thanks to ToshLLM) and its a very different story.

Absolutely blown away by the performance of what is now 7 year old hardware. Small models like Qwen 4B are exceptionally fast, at 78 tok/s in benchmarking (and still very useful for things like classification, sentiment analysis, etc) but it's the 35B range of models that are really something else.

Ornith 1.0 35B is by far the fastest of this size model, coming out at 50 tok/s gen and 680+ PP, but benchmarks really only tell part of the story and it's only when in active use with 70% full context that you ever really see the truth... in this case it's still good news, because at 131k context and 66% of that used we are still getting 30+ tok/s Gen and 250-ish tok/s PP.

Qwen3.6 35B A3B is a bit slower even with MTP on, at 46 tok/s Gen and 660+ PP in the benchmarks and a similar slowdown under real load at 25 tok/s Gen and 200-ish PP.

Some recent updates to ToshLLM have resulted in significant performance -- Ornith 1.0 has almost doubled in PP speed -- and stability improvements (I would previously get frequent crashes or inference collapsing to NaN). It's now at the point I don't feel like I need to run Linux to do local LLMs on the Mac Pro.

I'd love to know what sort of performance people are getting out of the bigger / higher performance GPUs that were available for the 2019 Mac Pro.

This machine cost me $2000 AUD ($1400 USD / £1000 GBP) which I think is a pretty amazing deal for the level of performance you get from what is technically now obsolete hardware. It's also interesting to compare the pricing from when it was new ($9,399 USD / $13,480 AUD / £7100-ish) and compare it to what you can buy now for that price, the DGX Spark, GX10 or any Strix Halo machine would out perform it for about 2/3rds of the brand new price... so maybe Rammaggedon isn't that bad after all?

All testing done in Pi with my crossbar local llm plugin against the grok build codebase at 131k context.

OS: macOS Tahoe 26.5.2 (25F84) x86_64
CPU: Intel(R) Xeon(R) W-3223 (16) @ 3.50 GHz                                
GPU 1: AMD Radeon Pro W5700X 16G VRAM
GPU 2: AMD Radeon Pro W5700X 16G VRAM                   
Memory: 37.60 GiB / 96.00 GiB

I've been looking at whats possible with older hardware for a while now, previously getting some decent performance out of an ancient 'Trashcan' 2013 Mac Pro. Today I've been testing my 2019 Mac Pro now that llama.cpp can use the Metal based cards in these machines (thanks to ToshLLM).

Absolutely blown away by the performance of what is now 7 year old hardware. Small models like Qwen 4B are exceptionally fast, at 78 tok/s in benchmarking (and still very useful for things like classification, sentiment analysis, etc) but it's the 35B range of models that are really something else.

Ornith 1.0 35B is by far the fastest of this size model, coming out at 50 tok/s gen and 680+ PP, but benchmarks really only tell part of the story and it's only when in active use with 70% full context that you ever really see the truth... in this case it's still good news, because at 131k context and 66% of that used we are still getting 30+ tok/s Gen and 250-ish tok/s PP.

Qwen3.6 35B A3B is a bit slower even with MTP on, at 46 tok/s Gen and 660+ PP in the benchmarks and a similar slowdown under real load at 25 tok/s Gen and 200-ish PP.

Some recent updates to ToshLLM have resulted in significant performance -- Ornith 1.0 has almost doubled in PP speed -- and stability improvements (I would previously get frequent crashes or inference collapsing to NaN). It's now at the point I don't feel like I need to run Linux to do local LLMs on the Mac Pro.

I'd love to know what sort of performance people are getting out of the bigger / higher performance GPUs that were available for the 2019 Mac Pro.

This machine cost me $2000 AUD ($1400 USD / £1000 GBP) which I think is a pretty amazing deal for the level of performance you get from what is technically now obsolete hardware. It's also interesting to compare the pricing from when it was new ($9,399 USD / $13,480 AUD / £7100-ish) and compare it to what you can buy now for that price, the DGX Spark, GX10 or any Strix Halo machine would out perform it for about 2/3rds of the brand new price... so maybe Rammaggedon isn't that bad after all?

OS: macOS Tahoe 26.5.2 (25F84) x86_64
CPU: Intel(R) Xeon(R) W-3223 (16) @ 3.50 GHz                                
GPU 1: AMD Radeon Pro W5700X 16G VRAM
GPU 2: AMD Radeon Pro W5700X 16G VRAM                   
Memory: 37.60 GiB / 96.00 GiB

Absolutely blown away by the performance of what is now 7 year old hardware. Small models like Qwen 4B are exceptionally fast, at 78 tok/s in benchmarking (and still very useful for things like classification, sentiment analysis, etc) but it's the 35B range of models that are really something else.

Ornith 1.0 35B is by far the fastest of this size model, coming out at 50 tok/s gen and 680+ PP, but benchmarks really only tell part of the story and it's only when in active use with 70% full context that you ever really see the truth... in this case it's still good news, because at 131k context and 66% of that used we are still getting 30+ tok/s Gen and 250-ish tok/s PP.

Qwen3.6 35B A3B is a bit slower even with MTP on, at 46 tok/s Gen and 660+ PP in the benchmarks and a similar slowdown under real load at 25 tok/s Gen and 200-ish PP.

Some recent updates to ToshLLM have resulted in significant performance -- Ornith 1.0 has almost doubled in PP speed -- and stability improvements (I would previously get frequent crashes or inference collapsing to NaN). It's now at the point I don't feel like I need to run Linux to do local LLMs on the Mac Pro.

I'd love to know what sort of performance people are getting out of the bigger / higher performance GPUs that were available for the 2019 Mac Pro.

OS: macOS Tahoe 26.5.2 (25F84) x86_64
CPU: Intel(R) Xeon(R) W-3223 (16) @ 3.50 GHz                                
GPU 1: AMD Radeon Pro W5700X 16G VRAM
GPU 2: AMD Radeon Pro W5700X 16G VRAM                   
Memory: 37.60 GiB / 96.00 GiB

Can’t believe how many people are sleeping on this one simple trick to reducing your token consumption. Simply write all your instructions and context onto an atomically flat crystalline surface with a 2kv scanning electron microscope.

My first time running a model on the 2019 Mac Pro and have to say it's performed amazingly. Zero tuning done at getting 55tk/s on the Q4 K XL quant and 70tk/s on the Q4 K XS quant. Thats about twice the speed of my RTX4080 with CPU offloading, and roughly 5-7x faster that the 2013 Mac Pro with dual D700's that is sat next to it. To say I'm impressed would be an understatement!

Finally acquired a 2019 Mac Pro to join my 2013 trash can. Unbelievably this still had the factory stickers on it and honestly looks like it’s never been used.

A month or so ago I started building a CLI tool essentially with the goal of replicating some of the functionality of a popular Rust based tool called RTK. The premise is fairly simple, the CLI tool sits between your AI coding agent and whatever bash tool its trying to use and intercepts the response and then minimises it to avoid unnecessary token consumption.

While RTK already existed, I wanted to do some extra stuff and figured it would be a fun thing to try and implement in C# with nativeAOT compilation, and additionally I didn't really want to learn rust (just vibecode it and just trust that Claude / Codex was doing it right).

Hypabolic/Hypa

I'm not at v0.1.1 and I've ended up going way beyond what I initially intended in terms of feature set. I've implemented a full MCP Server and Client for a start, so coding harnesses can talk to Hypa via MCP but additionally Hypa can act as the actual MCP client for anything you'd want to connect to.

The biggest advantage of this MCP proxy is that it keeps the MCP connections and their entire tool lists out of the agents context window. Shockingly just a few connections can eat up 15-20% of context (more if using local LLMs with smaller context windows).

I've also built a dedicated markdown mode for it, where it will index and parse repo markdown docs, giving the calling agent access to a table of contents and direct access to markdown sections, code blocks, etc.

My next job is to try and extend it's harness support. Currently its limited to just Claude and Codex, but I want to add VS Code, Pi, OpenCode, and maybe a few more.

As an example of what the calls and token savings look like:

● Bash(hypa dotnet build 2>&1 | tail -5)
  ⎿  ok (build succeeded)
     [hypa: 308→5 tok, -98%, reducer=dotnet-build]

● Bash(hypa dotnet test --filter "CodeIndex|MarkdownStructureProvider|CodeStructureProviderRegistry|CodeLanguageRegistry" 2>&1 | tail -20)
  ⎿  Test run for /home/matthew/development/atomic/Hypa/tests/Hypa.GoldenTests/bin/Debug/net10.0/Hypa.GoldenTests.dll (.NETCoreApp,Version=v10.0)
     Test run for /home/matthew/development/atomic/Hypa/tests/Hypa.IntegrationTests/bin/Debug/net10.0/Hypa.IntegrationTests.dll (.NETCoreApp,Version=v10.0)
     Test run for /home/matthew/development/atomic/Hypa/tests/Hypa.UnitTests/bin/Debug/net10.0/Hypa.UnitTests.dll (.NETCoreApp,Version=v10.0)
     Passed!  - Failed:     0, Passed:    10, Skipped:     0, Total:    10, Duration: 4 s - Hypa.UnitTests.dll (net10.0)
     [hypa: 683→185 tok, -73%, reducer=dotnet-test]

● Bash(hypa git diff HEAD~1 --name-only 2>/dev/null || git diff HEAD~1 --name-only)
  ⎿  src/Hypa.Cli/Commands/CodeCommand.cs
     src/Hypa.Runtime/Application/Services/CodeIndexService.cs
     src/Hypa.Sdk/CodeIntelligence/CodeIntelligenceModels.cs
     [hypa: 88→43 tok, -51%, reducer=git-diff]

  Searched for 3 patterns, read 1 file, listed 1 directory (ctrl+o to expand)

● Bash(hypa dotnet test tests/Hypa.UnitTests --no-build 2>&1 | tail -8)
  ⎿  Test run for /home/matthew/development/atomic/Hypa/tests/Hypa.UnitTests/bin/Debug/net10.0/Hypa.UnitTests.dll (.NETCoreApp,Version=v10.0)
     Passed!  - Failed:     0, Passed:  1299, Skipped:     0, Total:  1299, Duration: 29 s - Hypa.UnitTests.dll (net10.0)
     [hypa: 126→93 tok, -26%, reducer=dotnet-test]

Small gains individually but these really add up over the course of a session.

I'd love to get some other people using it and feedback on it. Hopefully some bug reports and suggestions on the tool parsers that people would like to see.

Full docs are available here Hypa — Open Source CLI Tool | Hypabolic

Hope to get some feedback from you all (even if you don't like it)

The “Trash Can” Mac Pro, once the most expensive machine you could buy from Apple, mine was just shy of £10,000 in 2016 — that’s £14k in today’s money.

Until recently mine was just running as a kubernetes single node development platform, it’s 64gb of ram and 24 logical cores made it perfect for that.

Its most powerful asset, a pair of D700 GPUs, essentially sat idle for years… that is until yesterday when I discovered that while its old southern islands based GPUs weren’t supported in ROCm, they were now supported under Vulkan — thanks to new drivers and a new Linux kernel.

That means it can run basically any model that llama cpp can throw at its 12gb of VRAM. Time to do some benchmarks, right?

Qwen 3.5 9B Q4 MTP — 11 t/s output at 70k context
Qwen 2.5 coder q4 — 22 t/s output at 70k context

Not exactly lightening fast but totally usable, especially for planning tasks where you can just set it and forget it.

The thing that’s really blown my mind though is that the planning output from qwen 3.5 is significantly, and it’s not even close, better than Claude Sonnet 4.6. It absolutely smashed planning on a complex csharp .net 10 app with nuget packages that sonnet struggled with, qwen just googled the docs.

Mind blown 🤯

What other ancient hardware are people running that’s still capable of doing real LLM work?