u/CrowKing63

https://preview.redd.it/d3pd24iath1h1.jpg?width=3024&format=pjpg&auto=webp&s=c7255015b2c48ddeea387abc48c7db7e772e4404

I don’t think I’m the only one who dislikes battery-powered products. Power wheelchairs, smartphones, wireless earbuds, and most of all, Bluetooth adaptive switch interfaces! Whenever I have a nightmare, it almost always involves a battery problem. Is it just me? Wireless devices that rely on batteries promise freedom, but it’s really only temporary freedom.

The Bluetooth adaptive interface I’ve been using for years has recently started to make me a bit anxious about its battery. As you know, these things are pretty expensive. The cheapest ones are around $300, right? I debated whether I should buy a new one, then thought, why not just make a wired device myself? I have a strong interest in gaming devices, so I knew about very cheap products with similar capabilities, and I outsourced the programming to AI.

For this attempt, I bought an EPS32 C3 board and two 3.5mm mono sockets (one as a backup). My parents helped with the wiring. Programming? That’s hardly even a challenge these days! I’m writing this with that very setup right now.

Hello. I'm trying various approaches to do a very simple coding task with a local model.

I looked at the official Pi documentation and a YouTube video, and tried to connect my llama.cpp model in model.json, but when I run Pi from the terminal it keeps starting with an “unknown”.

I asked Gemini about it, but the problem isn’t resolved. What should I look into?

Thank you.

I'm testing running local LLMs on a gaming mini PC (AMD 7840HS, 32 GB RAM) paired with an eGPU (Radeon 9060XT with 16 GB VRAM). Since I'm not very familiar with using llama.cpp, I kept getting unsatisfactory results, but with the recent Gemma4 24B A4B IQ4 NL model I finally reached 25.9 t/s. I even connected it to OpenCode and tried asking questions from my codebase, and it seems usable at this level.

llama-server -hf unsloth/gemma-4-26B-A4B-it-GGUF:UD-IQ4_NL
 --fit on
 --fit-ctx 128000
 --fit-target 256
 -np 1
 -fa on
 --no-mmap
 --mlock
 --threads 8
 -b 512 -ub 256
 -ctk q8_0 -ctv q8_0
 --temp 0.6 --top-p 0.95 --top-k 20 --min-p 0.0
 --presence-penalty 0.0 --repeat-penalty 1.0 --reasoning-budget -1

This is the result of using it this way.

Increase -b and -ub any further, it won't even load. Are there any unnecessary arguments or arguments that could be optimized?

Thanks.