u/d4mations

10 issues. Highlights: TurboQuant engine crashes, distributed timeouts, dashboard polling.

🐛 **Bugs**

**Engine & Quantization**
• #2780 **TurboQuantMSEState missing `ndim` crash** — Engine crashes when `turboquant_kv` enabled (Qwen3.5 SDPA split).
• #2778 **Engine crash `ndim` missing** — Same crash during MTP verification with TurboQuant active.

**Distributed & Cluster**
• #2720 **Cluster tab auto-polls `/plan`** — 10s polling loop triggers 400s and disruptive page redirects.
• #2712 **Timeout hard-coded to 300s** — No prefill heartbeats kills long distributed requests.
• #2727 **Coordinator locked to workstation role** — Can't switch to headless; loses reserved 32 GiB.

**Sampling & Profiles**
• #2801 **`presence_penalty` + VLM-MTP = 500** — Applying profile with penalty to VLM-MTP model fails.

**UI & Networking**
• #2800 **Wired-limit rounding loop** — UI rounds threshold to impossible value (124519) on 128 GiB Mac.
• #2520 **Remote MCP connection fails** — MCP server unreachable on v0.5.5+.

✨ **Features & RFEs**

• #2825 **Benchmarks omit model settings** — Published notes lack settings, hurting reproducibility.
• #2711 **`thinking_budget` rejected in distributed** — Clusters get unbounded or no reasoning; bypassed anyway.

Total: 10 issues.

**Summary:** 19 Issues. Focus on Cluster stability, TurboQuant bugs, and UI improvements.

---

### 🐛 Critical Bugs
* **#2780** [Bug] Engine loop crashes with `TurboQuantMSEState` missing `ndim` (Qwen35 SDPA split).
* **#2778** [Bug] Engine crash during MTP verification with TurboQuant enabled (`TurboQuantMSEState` has no attribute `ndim`).
* **#2483** [Bug] `preserve_mid_system_cache` fails for Claude Code (system nudges break prefix).
* **#2680** [Bug] Peer reported as 'worker runtime is not installed'; node-budgets 503s.
* **#2679** [Bug] Cluster activation rejects default macOS computer name (invalid node ID).
* **#2694** [Bug] Brew process fails during installation.
* **#2354** [Bug] oQ sensitivity probe crashes with `group size 16` (NVFP4 checkpoint).
* **#2476** [Bug] Hardcoded English labels in Model Settings bypass i18n.
* **#2726** [Bug] Cluster gate compares `mlx`/`mlx-lm` dist-info against module constants.
* **#2695** [Bug] Cluster runtime gate does not check Python interpreter version parity.
* **#2707** [Bug] Cluster gate treats Python minor-version difference as hard block.
* **#1856** [Bug] Fail to quantize DeepSeek V4 by v0.4.4rc1.

### ✨ Features & UI
* **#2768** [Feature] Cluster: Cache-aware scheduler scoring function (prefix affinity).
* **#2766** [Feature] Cluster: Model placement view with replica drill-down.
* **#2393** [Feature] Menu bar dashboard: compact token-rate + clearer resource indicators.
* **#2692** [Feature] Add global toggle to stop exposing backend MCP tools to clients.
* **#1808** [Feature] Community Intelligence Benchmark proposal.

### 📊 Research & Reports
* **#2558** [Research] DeepSeek V4 prefill optimization roadmap (M3 Ultra ablation).
* **#1374** [Report] Anubis + OMLX Inference Speed Anomaly Investigation Report.

---
Generated for Jundot/omlx community.

**Summary:** 18 issues resolved/discussed. 🐛 Bugs: 14 | ✨ Features: 3 | 🔧 Improvements: 1

---

### 🐛 Bugs (14)
* #2696 [Bug] Web-search always returns 3 hits regardless of results.
* #2695 [Bug] Cluster runtime gate missing Python interpreter version check.
* #2680 [Bug] Peer reported as worker runtime missing despite correct build.
* #2476 [Bug] Hardcoded English labels bypass i18n in Model Settings.
* #2694 [Bug] Brew process fails during installation.
* #2354 [Bug] oQ sensitivity probe crashes with NVFP4 checkpoint group size error.
* #2679 [Bug] Cluster activation rejects default macOS computer name (invalid node ID).
* #2572 [Bug] Failed to run MTPLX model even after importing sidecar.
* #2677 [Bug] Admin panel reasoning_effort preset silently rewrites values.
* #2545 [Bug] RecursionError escapes parse_tool_calls on deeply nested payloads.
* #2584 [Bug] DeepSeek-V4-Flash streaming emits reasoning as output_text.
* #2452 [Bug] quantize_oq_streaming crashes on MLX-affine quantized sources.
* #2581 [Bug] Memory enforcer counts reclaimable buffer pool as live usage.
* #2646 [Bug] VLM chat path drops tools before scheduler Request.

### 🔧 Improvements (1)
* #2707 [Improvement] Cluster gate downgrades Python minor-version difference to warning.

### ✨ Features (3)
* #2692 [Feature] Add global toggle to stop exposing backend MCP tools to clients.
* #1808 [Feature] Request for Community Intelligence Benchmark.
* #2425 [Feature] Add native web search capability to oMLX.

📅 **Summary**
Total Issues: 9 | 🐛 Bugs: 8 | ✨ Features: 1

**🐛 Bugs**
• #2572 [0.5.8.dev2] Failed to run MTPLX model even after importing sidecar
• #2677 Admin panel `reasoning_effort` preset silently rewrites stored values
• #2545 `RecursionError` escapes parse_tool_calls on deeply nested tool-call payloads
• #2584 DeepSeek-V4-Flash-0731: streaming emits reasoning as `output_text`
• #2452 `quantize_oq_streaming` crashes on MLX-affine quantized sources (uint32)
• #2581 Memory enforcer counts reclaimable MLX buffer pool as live usage
• #2646 VLM chat path drops `tools` before scheduler Request
• #2536 Prefix cache corrupts ArraysCache layers (KDA / Kimi-Linear hybrids crash)

**✨ Features**
• #2425 Add native web search capability to oMLX

**🐛 Bugs (4)**
* #2546 • store_cache boundary snapshots grow MLX allocator pool quadratically
Long-context sessions (70–80k tokens) cause RAM explosion and watchdog self-kill.
* #2551 • Mixed CacheList block storage is quadratic on SSD
Block storage writes scale quadratically; 282.7 GB usage for one session.
* #2507 • Tool call dropped when an argument contains a literal </tool_call>
Non-greedy match in `tool_calling.py` drops arguments with markers.
* #2521 • Deepseek v4 flash overestimate KV+SDPA to 81.25 GB for 200K prompt
Memory calculation error on M3 Ultra causes benchmark failure.

**💡 Features (2)**
* #2566 • Request Ling-3.0-flash-oQ4e-fp16-mtp for M1 and M2 macs
Generating quantized model variants for Apple Silicon hardware.
* #2517 • Use process-scoped overrides for Codex CLI launches
Prevent rewriting `~/.codex/config.toml` with local model settings.

Total: 6 issues

📊 **Overview**
Date Range: 2026-08-06 → 2026-08-08
Total Issues: 1

🐞 **Issue #1932 • 0.8.1 version**
Labels: [None]
Summary: App fails on iMac M3 8GB with v0.8.1 (Z-image Turbo 4bit)

Date range: 2026-08-05 to 2026-08-07. Total issues: 1.

### 🐛 Bugs (1)

**#1966** [bug] candle krea_2_turbo load failed: backend op failed: DriverError(CUDA_ERROR_SYSTEM_DRIVER_MISMATCH, "system has unsupported display driver / cuda driver combination")

*Summary:* CUDA driver mismatch error when loading candle krea_2_turbo model.

**🐛 Bugs**
• **#2537** Pi agent stream timeout despite SSE Keepalive Mode 'Comment'
• **#2422** jina-reranker-v3.5-mlx fails to load correctly

**✨ Features**
• **#2455** Support enforcing `thinking_budget` inside MTP speculative decoding

**📊 Summary**
Total Issues: 3

📅 Range: 2026-07-31 → 2026-08-02 | 📊 Total: 7

🐛 **BUGS**
• #1826 Claude Code with local LLMs via oMLX occasionally discard prefix cache | `BUG` - Prefix caching is intermittently lost during Claude Code sessions.
• #2448 Embedding throughput degrades steadily with server uptime, eventually stops completing requests | `BUG` - Sustained `/v1/embeddings` workload causes throughput to wedge from 331 down to ~0 tok/s.
• #2447 Embedding compile smoke-test passes but real requests fall back to eager | `BUG` - Smoke test omits `attention_mask`, causing Qwen3 embedding model to skip compiled path in production.
• #2285 The second request from the agent does not catch the answer | `BUG` - Agent workflows fail to capture responses on subsequent requests.
• #2428 Segfault with xgrammar postinstall on 0.5.4rc1 | `BUG` - Custom kernel + xgrammar installation triggers a segfault during postinstall.

💡 **FEATURES & ENHANCEMENTS**
• #2198 Feature request: Add option to skip cache writes on per-request basis | `FEATURE` - API parameter needed to disable cache writing for one-off requests.

📚 **DOCS**
• #2181 Bug: documentation for Native MTP excludes newly-supported GLM | `DOCS` - Model settings UI and docs still reference older versions, missing GLM support.

**FEATURES & IMPROVEMENTS**
* **#2198**: Add optional parameter to skip cache writes per request.
* Request for API flag to disable caching on one-time operations.

**BUGS & STABILITY**
* **#2428**: Segfault with xgrammar postinstall on 0.5.4rc1.
* Postinstall crashes with custom kernel flags; previous commits worked fine.
* **#2405**: Mistral Small 3.2 MLX fails to emit tool calls (v0.5.3).
* Reported on macOS M5 Max with 8bit model; tools not triggered.

**MODEL SUPPORT & DFLASH ISSUES**
* **#2181**: Docs for Native MTP exclude newly-supported GLM models.
* Model settings dialog in 0.5.0 missing GLM entries.
* **#2153**: DFlash fails on Gemma 4 MLX (`gemma4_unified` unsupported).
* Speculative decoding disabled for all current Gemma 4 builds.
* **#1737**: DFlash option disabled for Gemma4 12b in web admin (v0.4.2).
* Feature unavailable despite model support claims.

**AGENT & TEAM CAPABILITIES**
* **#2285**: Second agent request fails to capture answer.
* "Research project" tasks succeed initially but fail on follow-up calls.
* **#99**: Improve understanding of subagents/teams structure.
* Request for better configuration handling for agent teams.

(Total Issues: 8)

**🐛 Bugs**
* **#1937 New Version not starting**: App fails to launch with `api_initializing` error; SceneWorks API readiness-critical startup is still running.

**🐛 Bugs**
* **#99** Support better understanding of subagents/teams – Proposal to improve configuration handling for subagent structures.
* **#2153** DFlash fails on Gemma 4 MLX models: model_type `gemma4_unified` not supported by Gemma4TargetOps – Speculative decoding disabled for all current Gemma 4 builds due to unsupported model type.
* **#1737** No DFlash for Gemma4 12b – UI option disabled in oMLX v0.4.2; error message displayed at settings bottom.
* **#2405** Mistral Small 3.2 MLX does not emit tool calls through oMLX 0.5.3 – Tool calling fails on M5 Max with specific 8-bit community model.
* **#2254** 0.5.1 Qwen3.6-27B-0Q4-MTP suddenly slow down – Performance degradation (PP/TPS drops) reported after upgrade to v0.5.1.
* **#2219** External VLM MTP is silently bypassed after scheduler chunked prefill – Text prompts ignored when external VLM MTP enabled with chunked scheduling.
* **#2291** Mistral/Devstral tool calling broken: generated [TOOL_CALLS] token is treated as stop → empty message – Tool calls fail immediately on models using Mistral format (Devstral 2, Small).
* **#2317** DFlash fails with Qwen3.6-35B-A3B-4bit – Speculative decoding cannot be enabled despite using correct DFlash model variant.

🐛 **BUGS**
#1937: New Version not starting — App fails to launch with `api_initializing` error due to pending API readiness check.

**🐛 BUGS & PERFORMANCE ISSUES**
* **#2317**: DFlash fails with Qwen3.6-35B-A3B-4bit
* DFlash activation fails specifically when loading the Qwen3.6 35B A3B 4-bit model, despite using the correct DFlash variant.
* **#2254**: Qwen3.6-27B-0Q4-MTP speed regression after upgrade
* Performance drops significantly in lighting-mtp mode for Qwen3.6-27B-0Q4-MTP following the upgrade to version 0.5.1, reversing previous speed gains.
* **#2326**: GLM-5.2 Alis dynamic 3-bit MTP load failure
* Dynamic 3-bit MTP loading fails due to a type mismatch where the remapped module is incorrectly constructed as 4-bit.

**⚠️ FEATURE & INTEGRATION BREAKAGES**
* **#2219**: External VLM MTP bypassed after chunked prefill
* When using external VLM MTP with scheduler chunked prefill, the MTP mechanism is silently bypassed for text prompts.
* **#2291**: Mistral/Devstral tool calling returns empty messages
* Tool calling breaks for Mistral/Devstral models (e.g., Devstral 2) as `[TOOL_CALLS]` tokens are treated as stop sequences, resulting in zero completion tokens.

I’m trying to send an image to gemma4-31b that has mmproj enabled in llama.cpp but it fails to read it. The model has no problem reading images in other harnesses or uis but it just won’t read it in opencode cli through the web ui. Any suggestions?

**🐛 Bug Fixes**
* **#2326**: GLM-5.2 Alis dynamic 3-bit MTP fails to load because remapped MTP module is constructed as 4-bit.
* *Summary*: oMLX 0.5.2 incorrectly constructs the remapped MTP module as 4-bit instead of the detected 3-bit, causing load failures for GLM-5.2 Alis models.

**📊 Stats**
* Total Issues: 1
* Labels: Bug

🐞 **BUGS**
* **#1858**: FLUX.2 models remain "Incomplete" after repeated downloads due to missing scheduler config
* **#1808**: Possible update path error on Windows desktop (uninstalling 0.8.0 did not resolve)

📊 **Summary**
* **Total Issues**: 2
* **Date Range**: 2026-07-25 to 2026-07-27

**🐛 Bugs (2)**
* **#2221**: Tool call output as raw XML instead of internal tool invocation with qwen3.6-27b
* Model occasionally outputs raw XML for tool calls after ~20 conversation rounds.
* **#2359**: Server running the model causes laptop overheating when the AI model is in IDLE state
* System overheats even when the server is idle and not actively processing requests.

**🔗 Repository**: Jundot/omlx
**📅 Period**: July 24–26, 2026