Intel ARC B70 Is earning a spot on the best card for the price.
▲ 64 r/LocalLLM+1 crossposts

Intel ARC B70 Is earning a spot on the best card for the price.

Intel Arc B70 was already a great option for the price/vram state, but its now having immense performance gains as vLLM gets further optimized for the XPU cores.

After a lot of trial and error, I got these numbers:

https://preview.redd.it/8tr4tl2btzhh1.png?width=2366&format=png&auto=webp&s=b6a58b92af31e397122d68650b7c37a7bec9b2e5

Full recipe is here:

https://github.com/SergiioB/intel-arc-pro-b70-inference-cookbook

For the latest updates on ARC B70 Serving, follow me on X im very active:

https://x.com/SergiiioBS

Im now seeing that most of the fixes have been implemented in upstream, I will be trying and see if I get some gains.

I'm

reddit.com
u/Barrysoft8 — 13 days ago
▲ 1 r/coches

Mi hermano es mecánico y yo informático. Hartos de las apps de mantenimiento, nos hemos hecho la nuestra

​

Buenas gente!

Mi hermano lleva unos cuantos años años en el taller y yo programo. Entre los dos hemos probado Drivvo, Fuelio, un par de hojas de cálculo y tropecientas apps mas Todas tienen lo mismo: o son un Excel con esteroides, o te meten anuncios hasta en la sopa, o la "IA" es un chatbot que te dice "consulta a tu mecánico" para todo..

Así que nos hemos montado IntelliAuto para nosotros. La uso en mi Fiesta y él en su Focus y furgo del taller. Llevamos unos meses con ella y sabemos que tenemos un producto muy valido para todos vosotros...

Lo que la hace diferente de lo que hay:

-Mecánico con IA de verdad: Le preguntas "cada cuánto cambio la distribución en un 1.0 EcoBoost?" y te responde con intervalos reales, no un "consulta el manual". Sabe de coches y de motos (cadena, válvulas, kit de arrastre...).

- Escáner de tickets. Haces una foto al ticket de la gasolinera o del taller y te saca litros, €/L, fecha e importe solo. Sin picar nada a mano.

- Privacidad. Las matrículas, DNI y números de tarjeta se borran en el móvil antes de que nada salga a internet. No guardamos tus datos personales en ningún servidor.

- Motos incluidas. Mantenimiento específico (engrase de cadena, presiones, reglaje de válvulas). No es un "coche pero con dos ruedas"!

- Informe PDF. Exportas todo el historial ordenado y se lo enseñas al comprador cuando vendas el coche. Un historial completo sube el valor real.

- Recordatorios. ITV, seguro, impuesto de circulación, aceite, filtros... por km o por fecha, tú eliges.

Sin anuncios. Funciona offline (todo se guarda en el móvil). La nube es opcional y solo si quieres backup.

Se llama IntelliAuto

Muchas gracias!!!

u/Barrysoft8 — 26 days ago

IntelliAuto - Vehicle Maintenance Tracker with AI

🎁Giveaway: Leave a comment and I'll DM you a promo code for the Pro version!

YouTube Video Demohttps://www.youtube.com/watch?v=uh2Q5GysE_E

IntelliAuto is an ad-free app to track your car and motorcycle expenses, featuring a receipt scanner and an integrated AI mechanic.

What are It's Features?

AI Mechanic: Ask questions about your specific vehicle and get tailored maintenance advice or compatible parts.

Receipt Scanner: Auto-crops and extracts liters, prices, and dates from fuel/mechanic receipts instantly.

Privacy First: License plates, IDs, and card numbers are scrubbed locally before AI processing.

Motorcycle Support: Includes specific tracking for bikes (e.g., chain maintenance).

PDF Reports: Export a professional service history to boost resale value.

Smart Reminders: Easy recurring alerts for Insurance, Inspections, and Road Tax.

Pricing: Free to use (offline-first). Premium/Pro tiers available for cloud sync, advanced AI, and unlimited scans.

No Ads.

Download on Google Play: https://play.google.com/store/apps/details?id=com.barrysoft.IntelliAuto

u/Barrysoft8 — 26 days ago

IntelliAuto - Vehicle Maintenance Tracker with AI

https://preview.redd.it/v9jxdt89kefh1.png?width=1024&format=png&auto=webp&s=ce49e965cec1397e028f2719c86ae567638c8403

🎁Giveaway: Leave a comment and I'll DM you a promo code for the Pro version!

YouTube Video Demo: https://www.youtube.com/watch?v=uh2Q5GysE_E

IntelliAuto is an ad-free app to track your car and motorcycle expenses, featuring a receipt scanner and an integrated AI mechanic.

What are It's Features?

AI Mechanic: Ask questions about your specific vehicle and get tailored maintenance advice or compatible parts.

Receipt Scanner: Auto-crops and extracts liters, prices, and dates from fuel/mechanic receipts instantly.

Privacy First: License plates, IDs, and card numbers are scrubbed locally before AI processing.

Motorcycle Support: Includes specific tracking for bikes (e.g., chain maintenance).

PDF Reports: Export a professional service history to boost resale value.

Smart Reminders: Easy recurring alerts for Insurance, Inspections, and Road Tax.

Pricing: Free to use (offline-first). Premium/Pro tiers available for cloud sync, advanced AI, and unlimited scans.

No Ads.

Download on Google Play: https://play.google.com/store/apps/details?id=com.barrysoft.IntelliAuto

reddit.com
u/Barrysoft8 — 26 days ago
▲ 4 r/LocalLLM+1 crossposts

Update: B70 SYCL build b10053 + PR #25690 numbers, and 118B Laguna S 2.1 with partial expert offload

Updated production numbers (single B70, SYCL, build b10053 + PR #25690, LocalMaxxing 2026-07-22):

Qwen3.6-35B-A3B

• Quant: UD-Q4\_K\_XL

• Config: 256K, 150W

• Prefill t/s: 1,603.5

• Gen t/s: 69.7

Qwen3.6-35B-A3B

• Quant: UD-Q5\_K\_M

• Config: 256K, 150W

• Prefill t/s: 1,601.3

• Gen t/s: 67.0

Ornith-1.0-35B

• Quant: Q5\_K\_M

• Config: 256K, 150W

• Prefill t/s: 1,589.8

• Gen t/s: 78.7

ThinkingCap-Qwen3.6-27B

• Quant: Q4\_K\_M

• Config: 200K, MTP-4, 165W

• Prefill t/s: 621.3

• Gen t/s: 27.5

Flash attention on, KV cache q8\_0 K / q4\_1 V. 100/100 quality gate passed.

Laguna S 2.1 (118B MoE, 8B active) on a single B70

Poolside released Laguna S 2.1 last week: 118B params, 256 routed experts plus 1 shared, top-10 routing, 8B active per token. Support landed in llama.cpp via PR #25165, and the model is 34.6 GB at IQ2\_XXS (Unsloth Dynamic), which doesn't fit in 32 GB VRAM with all experts on GPU.

The standard approach is -ot ".*ffn.*exps.*=CPU", which puts all expert weights on CPU and keeps attention and dense layers on GPU. It gave me 4.8 t/s, and when I checked the verbose log I found only 2.4 GB VRAM in use with 24 GB sitting completely idle while every token triggered CPU expert lookups across 256 experts per layer.

Partial expert offload fixes this. Instead of sending all experts to CPU, keep experts for layers 0-39 on GPU and send only layers 40-47 to CPU:

-ot "blk\.(4[0-9])\.ffn_.*_exps\.=CPU"

All experts CPU

• GPU layers: 0

• CPU layers: 48

• Gen t/s: 4.8

• vs baseline: 1.0x

0-23 GPU

• GPU layers: 24

• CPU layers: 24

• Gen t/s: 8.1

• vs baseline: 1.7x

0-33 GPU

• GPU layers: 34

• CPU layers: 14

• Gen t/s: 10.2

• vs baseline: 2.1x

0-35 GPU

• GPU layers: 36

• CPU layers: 12

• Gen t/s: 12.7

• vs baseline: 2.6x

**0-39 GPU**

• GPU layers: 40

• CPU layers: 8

• Gen t/s: 15.3

• vs baseline: 3.2x

0-43 GPU

• GPU layers: 44

• CPU layers: 4

• Gen t/s: OOM

• vs baseline: crash

Each additional GPU layer set adds roughly 2 t/s until you hit the VRAM wall. Pushing to 44 GPU layers OOM'd the system, so 40 is the safe ceiling on a 32 GB card at this quantization. One regex change in the -ot flag, no code changes or dependencies.

Quality at IQ2\_XXS is coherent across identity, code generation, and math prompts. The model identifies as Poolside, generates correct Python with docstrings and type hints, and reasons through arithmetic step by step.

DFlash speculative decoding: tested, skipped

u/lukepm tested DFlash on 2× RTX 5090 and found that default flags made it 2.5x slower, while tuning brought it to parity. I got the same result on the B70 with his tuned flags (--spec-draft-n-max 7 --spec-draft-p-min 0.75): 5.0 t/s vs 4.8 baseline, within noise.

Laguna routes each token to 10 of 256 experts, so a 16-token verification batch can touch up to 160 experts per layer. When experts are CPU-resident, verification cost scales with draft batch size. Speculative decoding helps when GPU compute is the bottleneck, but here the bottleneck is expert memory access, so adding more verify tokens just adds more CPU expert lookups.

For B70 / limited-VRAM MoE owners:

  1. Run with -v and check VRAM usage — if it's far below capacity, you're wasting it

  2. Use partial expert offload: -ot "blk\.(N[0-9])\.ffn_.*_exps\.=CPU" where N is the first CPU layer

  3. Fill VRAM to about 1 GB from the limit, then back off one layer if it crashes

  4. Skip spec decode for fine-grained MoE with partial offload

Hardware: Arc Pro B70 32GB (150W), Ryzen 7 5700X3D, 32 GB DDR4-3200, NVMe

Software: llama.cpp Poolside fork 04b2b72, SYCL/Level Zero, oneAPI 2026.0.0

reddit.com
u/Barrysoft8 — 28 days ago

Built an app that remembers and keeps track of maintenance for me..

I really believe this is something that this subreddit might find useful..

​

I'm an IT kid and I was always terrible about keeping service records organized. Receipts in the glovebox, texts from the shop, maybe a note on my phone.. and when something went wrong I could never remember when the last oil change was or what parts I used, which then took hours to investigate again what the right part was.

​

So I built an Android app that handles all of it. You scan a repair invoice or fuel receipt with your camera and it pulls out the details automatically — the work done, the parts, the cost, the date ,and logs it against your vehicle profile.

​

There's also an assistant you can ask stuff like "when was my last brake job?" or "what's that clicking noise?" and it answers based on your car's actual history, not generic advice. It includes safety disclaimers so it's not pretending to replace a real mechanic, but it's been surprisingly useful for quick questions.

​

A few things that have been handy:

- It tracks costs over time so you can see what you're actually spending on maintenance

- It suggests compatible parts based on your car model and links to buy them

- You can export a full PDF of your service record — useful if you're selling or just want to show your mechanic what's been done

- All data stays on your phone in an encrypted database, no account needed

​

I built this with my brother who's a mechanic, so a lot of the features came from what he said people always ask about at the shop.

​

Full disclosure: I'm the developer. It's free to try on the Play Store. Would appreciate any feedback from people who actually work on cars.

​

https://play.google.com/store/apps/details?id=com.barrysoft.IntelliAuto

​

u/Barrysoft8 — 2 months ago
▲ 4 r/AndroidAppTesters+2 crossposts

An AI car assistant that knows your car

An AI car assistant that actually knows your car's maintenance history

​

Most car maintenance apps are just glorified notepads. I wanted something smarter, so I built IntelliAuto with my brother (a mechanic).

​

Here's what it does:

​

📷 Scan any repair invoice or fuel receipt with your camera — the app reads it and logs everything automatically. No manual data entry.

​

🤖 Ask the AI assistant about your car. It knows your vehicle's full history, so you can ask things like "why is my check engine light on?" or "when was my last oil change?" and get answers based on your actual records.

​

🔧 Get smart maintenance reminders. The app learns from your service history and tells you what's coming up before it becomes a problem.

​

🛒 Find compatible parts. Based on your car model and recorded services, the app suggests parts with direct links to buy them.

​

📄 Export a full PDF maintenance report anytime — great for resale value or mechanic visits.

​

All your vehicle data stays on your device, encrypted. No accounts, no cloud required.

​

Whether you're a car enthusiast or someone who just wants to stay on top of maintenance, IntelliAuto takes the headache out of vehicle care.

​

https://play.google.com/store/apps/details?id=com.barrysoft.IntelliAuto

​

u/Barrysoft8 — 2 months ago
▲ 37 r/IntelArcPro+3 crossposts

Avoid CUDA monopoly at all costs. AMD is an alternative.

Hey everyone,

There’s a massive misconception that if you aren't dropping $2,000 on an NVIDIA GPU, you can't run serious Local AI workflows. I wanted to see how far I could push a consumer AMD card, and therefore bought a rx7800xt 16b VRAM.

Right now, my workstation node is running llama-server hosting a DENSE 27B model -> Qwopus3.6-27B-v2-Q3_K_S.gguf (12 GB) and Qwen3.6-35B-A3B-UD-IQ3_XXS.gguf (13 GB Mixture of Experts, 3B active parameters per token) continuously. I am regularly feeding it contexts that reach 91k to 128k tokens in my daily workflows.

Here is the exact setup, compiler parameters and optimization flags.

THE COMPILER BUILD
To get flash attention and RDNA3 optimizations working correctly on ROCm 6.4.4, I built llama.cpp from source using these specific cmake flags:

cmake -B build -DGGML_HIP=ON -DGPU_TARGETS=gfx1101 -DrocWMMA_FATTN=ON
cmake --build build --config Release

This targets the gfx1101 architecture of the RX 7800 XT directly and compiles support for hardware-accelerated Flash Attention kernels.

THE EXACT RUNTIME FLAGS
My systemd service runs the server with this exact command line:

llama-server --host localhost --port 8080 --api-key xxxx --parallel 1 --n-gpu-layers 99 --batch-size 512 --ubatch-size 128 --flash-attn on --cache-type-k q8_0 --cache-type-v q4_0 --ctx-size 131072 --reasoning off --sleep-idle-seconds 300 --cache-prompt --temp 0.7 --top-p 0.8 --top-k 20 --min-p 0 --presence-penalty 1.5 --repeat-penalty 1

HOW I CRUSHED THE VRAM LIMIT: KV CACHE QUANTIZATION
A model like Qwopus 27B or Qwen 35B MoE fits in 16GB VRAM at a small context size. But at 128K context, the raw FP16 Key-Value (KV) cache alone would consume upwards of 32 GB of VRAM, making it impossible to run on consumer hardware.

To solve this, we split and quantize the cache:
- Key cache is quantized to 8-bit (q8_0) using --cache-type-k q8_0
- Value cache is quantized to 4-bit (q4_0) using --cache-type-v q4_0

This compresses the memory footprint of the KV cache by roughly 5.6x. Thanks to this optimization, the entire active model weights plus the 128K token KV cache sit comfortably in VRAM, utilizing exactly 96% of the 7800 XT's memory. No layers spill into slow system RAM, avoiding the PCIe transfer bottleneck entirely.

THE MATH BEHIND 128K CONTEXT: YaRN ROPE SCALING
Qwopus and Qwen architectures use Rotary Position Embeddings (RoPE). Because these models have a base context window smaller than 128K, running at 131,072 tokens requires positional frequency scaling.

Instead of basic linear scaling (which stretches all frequencies equally and destroys the model's short-range spelling and grammatical coherence), llama.cpp utilizes YaRN (Yet another RoPE extensioN).

YaRN divides the embedding dimensions into three frequency bands:

  1. High-frequency dimensions: These represent immediate, local token relationships. YaRN leaves these completely un-stretched so the model does not lose its spelling accuracy or close-context grammar.
  2. Low-frequency dimensions: These represent long-range structure. YaRN scales these linearly by a factor of 4.0 to cover the 128K space.
  3. Mid-frequency dimensions: These are smoothly interpolated to avoid abrupt attention transitions.

This uneven scaling prevents the attention entropy and perplexity from exploding. In practice, the model remains highly coherent and retains logical consistency even at 91k+ tokens.

REAL-WORLD TELEMETRY AND SPEED
During heavy prompt processing, the card maintains solid throughput:
- Prefill speed: ~210 tokens/second (utilizing flash attention)
- Decode speed: 11-17 tokens/second
- GPU Power: Draws ~188W (with a systemd power cap set at 190W via rocm-smi)
- GPU Temps: Stable between 52 C and 70 C across edge, junction, and memory sensors.

If you are running consumer AMD hardware, do not settle for small context sizes. Build with ROCm, turn on Flash Attention, quantize your Key/Value cache separately, and let YaRN handle the frequency scaling.

I wrote up a detailed guide comparing these measurements, native Windows vs Linux ROCm paths, and power sweeps on my blog here: https://sergiiob.dev/posts/rx7800-xt-llama-cpp-benchmarks-moe-context

I share my daily telemetry runs, local model benchmarks, and hardware configurations on X. If you want to see live updates and benchmarks, follow along here: https://x.com/SergiiioBS

u/Barrysoft8 — 14 days ago