u/JLeonsarmiento

Jailbroken AI glasses:

3.8-27B has ruined 3.5/3.6-35B’s for me. It’s just absurdly superior.

Applied science work, from workflow design, data pipeline, results analysis, article/reports writing and data publishing online. 5 projects I did in the past replicated from start to finish.

3x to 4x more total wall time. Yes, HUGE toll on how much you can do in a day if this was the only model you could use in your laptop.

But oh my…the quality of that thing. The stupid level of attention to detail. I have the Z.ai api, so I can compare it with 5.3 and 5.3-flash:

The gap between 5.3 (flash and regular) and 3.8-27B is much less, smaller when not plain tiny, than the gap between 3.8-27B and any of the 3.5/3.6-35B-A3B (vanilla, kat, Ornith/tiel, nex-2).

Only Ornith came close, but it never matched it.

But it’s also spending 22 to 33% less tokens (effort =medium) and less ram footprint, so you get more done without hitting limits,compaction, etc.

So yeah, guess I’ll sip more tea, play the piano, whatever. Let that fat bottom Qwen work.

u/JLeonsarmiento — 1 month ago

▲ 7 r/LowEndLocalAI+1 crossposts

A collection of 3-bit_XL MoE models for the Ram Poor Mac user: 24 to 32 GB Ram MacBookAir and base MacBookPro

I’ll keep adding the latest releases to this.

huggingface.co

u/JLeonsarmiento — 1 month ago

▲ 33 r/LowEndLocalAI

GPT-OSS-20B in a 10 years old ThinkPad Laptop - [Intel i7 7600U], [iGPU: practically inexistent ], [UMA | 16GB]

Everybody's playing with their Qwens and their GPUs, but I am going abandonware here:

Lenovo X1 Yoga - 30 GB/s Memory bandwith (!!!) - GPU or iGPU: yes, but CPU is just faster - 16 GB RAM.

Tried all the new tiny Dense and MoE (Ling-Tiny, LFM2.58B, MiniCPM5, NanBeige4.2, Gemma-4 e2b and e4b) but, despite what benchmarks say, those are still dominated by GPT-OSS-20B for general use ( a combination of chatbot + agentic using Pi). Yeah, 20B vs and those little guys... but if it can fit in RAM you should go for the biggest one.

It will be slow, but it will nail the task.

Llama.cpp command:

llama-server --no-models-autoload -dev none -t 4 -fa off -c 16384 --jinja --host 127.0.0.1 --port 8080

Then just call it from ¨Pi of go LocalHost:8080 for chat... you know the drill.

u/JLeonsarmiento — 1 month ago

▲ 7 r/oMLX

Reduce SSD tear.

Probably over thinking about this…but here is my approach. Open to comments and suggestions:

Turn on hot cache (in ram memory cache) in general settings and assign a comfortable chunk ~3GB to ~8GB: somehow it’s was disabled on my oMLX, still don’t know if it was me or the defaults.
Enable the use of TurboQuant K/V cache on each model settings: it has a toll on decode t/s generation speed, but the reduced memory footprint, and due to this amount of data written to memory and

read back from

D is reduced,

that is just huge. Totally worth.
edit oMLX settings.json file to use a folder on an external SSD for cache storage: move the tearing write/erase/write/erase routine away from your irreplaceable internal SSD into something that you can replace or upgrade at any point.
Keep cached K/V relevant: to get the most from cached K/V files make them last longer by updating your coding agents 1 time per month, and, keep your sessions start consistent to be able to reuse cached files. For example, on Pi or Hermes, I always start new sessions with a simple “Hi” so the cached system prompt (3K to 24K respective) is actually used and application startup is immediate.

reddit.com

u/JLeonsarmiento — 1 month ago