https://preview.redd.it/106d7mzc8lgh1.png?width=1453&format=png&auto=webp&s=bcae99698033283968775b58cac3aaec62a7a8ad

Got logs and ebpf.

https://horosvec.hazyhaar.fr/

horosvec: an embedded ANN vector index in pure Go, on top of a single SQLite file

July 2026 — also published in French at hazyhaar.fr. Vector similarity search has become the silent building block of every modern document system: finding, among tens of thousands of passages, the ones that talk about the same thing as a query without sharing a single word with it. The engines that do this are usually heavy services — dedicated servers, native dependencies, orchestration. horosvec takes the opposite stance: an approximate-nearest-neighbor index embedded in your process, written in pure Go, with all of its state living in one SQLite file. One dependency: modernc.org/sqlite, the pure-Go SQLite port — no CGO, static binaries. MIT licensed.

go get github.com/hazyhaar/horosvec

Two algorithms working in tandem

horosvec combines two ideas from the recent ANN literature. Vamana, the proximity graph popularized by DiskANN: every vector becomes a node connected to its relevant neighbors, and search navigates this graph greedily instead of comparing the query against the whole base. Robust alpha-RNG pruning keeps long-range shortcuts that avoid local minima. RaBitQ, an extreme binary quantization: every coordinate of a vector is reduced to its sign — one bit per dimension, plus two norms per vector. The approximate distances computed on these codes are crude but almost free, and they are enough to steer the navigation. The architectural decision that makes the whole thing hold is the two-stage search: graph traversal preselects candidates using RaBitQ distances, then the final ranking is recomputed with exact L2 distance on the true float32 vectors. The estimator is allowed to be noisy: it only needs to place the true neighbors inside the beam — the exact rerank does the rest.

Measured, not promised

The implementation deliberately deviates from the RaBitQ paper on one point: the random rotation step, on which the paper's theoretical guarantees rest, is not implemented. Rather than invoking bounds that no longer apply, the repository ships deterministic, replayable benches and publishes their numbers. Recall@10 against exact brute-force ground truth, 2,000 base vectors, 50 queries, default configuration: | Dataset | dim | mean recall@10 | worst query | |---|---|---|---| | uniform synthetic | 128 | 1.000 | 1.000 | | tight gaussian clusters | 128 | 0.982 | 0.900 | | real bge-m3 embeddings (code-session texts) | 1024 | 1.000 | 1.000 | The third line is the one that matters: on real data — two thousand messages from software-development sessions, embedded by an actual embedding model in dimension 1024 — the index does not miss a single neighbor. The theoretical concern about the anisotropy of embedding spaces did not materialize at this scale. The honest limits (2×10³ vectors, queries drawn from the same distribution as the base) are documented in the package, and the real-data bench is replayable by anyone: export HOROSVEC_REAL_VECS pointing at a JSON array of vectors and run go test -run TestRecallMeasure_RealEmbeddings -v.

Hardened by production, and by adversity

horosvec is not a weekend prototype: it is the extraction of the engine that serves RAG shard search and code-map embeddings in production inside the horos55 ecosystem. Its preparation for publication went through a full adversarial audit, whose findings became tested properties:

• bounded deserialization: a corrupt or hostile binary blob fails cleanly — no panic, no unbounded allocation;
• inserts are transactional all the way into memory: internal state (node cache, counters, flat mirror) is applied only after the SQLite commit — a failed commit leaves zero phantom neighbors;
• cancellation is an error, never a silence: a context cancelled in the middle of graph traversal returns an explicit error instead of an empty result indistinguishable from "no neighbors";
• 42 tests, 85.9% coverage — including commit-failure injection, blob corruption, LRU eviction and drift-triggered rebuilds. Known limits are stated in the documentation rather than glossed over: no delete API (full rebuild instead), an unbounded in-memory mirror for the brute-force path, and a graceful-degradation contract on external reranking that may become explicit error propagation in a future major version.

Who is it for?

Any Go program that wants semantic search without an external service: a CLI that indexes your notes, a server searching its own documents, a pipeline deduplicating by similarity. If you need a distributed vector database, this is not it; if you need an index that fits in your binary and in one file, this is exactly it.

https://github.com/hazyhaar/horosvec/blob/main/README.md

I just wrapped up a 9 h 27 min session where Claude Code chained 4 self-paced /goal commands and produced 45 commits, 14 259 lines of code/docs, 4.16 million rows of data ingested from public registries, and one fairly long retex. Here's what happened, how I structured it, and what surprised me.

What /goal actually is

Claude Code has a slash command /goal <description>. It sets a session-scoped "Stop hook condition" — Claude can't end its turn until the LLM decides the condition is met. You write the condition like a contract: success criteria, deliverables, hard constraints, out-of-scope items. Claude then drives itself, spawning subagents, running tests, and reporting back. You can interrupt anytime.

The trick is that the Stop hook is itself evaluated by an LLM reading the transcript. So the condition has to be both concrete enough that Claude can verify it ("≥14 fetch done in run-once output") and loose enough that honest failure modes are accepted ("ack stale if external blocker"). Get either wrong and you either loop forever or you get a fake "done".

The task

Project: horos55 — a Go data orchestrator with ~40 adapters pulling open data from data.gouv.fr, INSEE, EBA, GLEIF, GeoNames, etc. About 22 were failing in production. Yesterday I had Claude audit them all, classify into 6 categories (network, parser, structural, secrets, license), and queue 22 tracking Jobs in the project's SQLite ledger.

Today's /goal was strict: "14 fix code + 3 ack stale + 1 abandon. 0 Job queued left." That's a 4000-character contract. The Stop hook refused to clear until that exact taxonomy was met.

How the run unfolded

The session structured itself into 5 successive passes:

Pass	Method	Adapters fixed	Cumulative
1 (N1+N2)	Apply documented audit recommendations directly	4 / 14	29 %
2 (rattrapage)	Read the failure log from pass 1, brief a new subagent on the actual errors	+5 / 14	64 %
3 (3rd pass)	Target the 2 specific remaining parser/quoting issues	+2 / 14	79 %
4 (eba investigation)	Dig deeper on one structural blocker	0 / 14 (confirmed dead-end)	79 %
5 (pivot to alternatives)	Find creative sources: GitHub mirrors, ECB lists, regional CSVs	+3 / 14	100 %

Each pass spawned 1 subagent on average (horos55-coder-go, a custom profile I have). The 5th pass found:

• SSA Baby Names blocked by WAF → switched to hadley/data-baby-names GitHub mirror
• INSEE NAF resource ID expired → pivoted to data.grandlyon.com CSV (same INSEE source upstream)
• EBA Credit Institutions auth-walled → switched to ECB MFI list (Monetary Financial Institutions, equivalent dataset, public domain)

The big lesson: "audit URL ≠ audit parser ≠ fix runtime"

In an earlier session I had Claude do a "deep audit" of all 22 broken adapters: WebFetch each candidate URL, verify HTTP 200, recommend a fix. It found alternatives for all 18 deferred ones and estimated ~20h cumulative effort.

When I actually applied the fixes today, 30 % introduced new problems the audit hadn't detected:

• Headers had drifted (INSEE CSVs renamed preusuel → prenom, RPPS added spaces in column names)
• "Alt URLs" returned 200 but pointed to HTML info pages, not to the actual CSV
• GLEIF v2 returns a JSON metadata blob pointing to a ZIP — the audit had only checked the JSON URL, not the actual download chain
• The SSA "fix" of adding a User-Agent header was a false trail; the UA was already there. Actual cause was geoblocking.

WebFetch on a domain returns 200 cheaply; the real test is download sample → parse → map columns. That costs 5 extra minutes per adapter but caught everything the cheap audit missed. The 2nd and 3rd passes were doing exactly that retroactively.

What worked

Iterative auditing, not exhaustive auditing. The progression 29 → 64 → 79 → 100 % is non-trivial. Each pass added 15-35 percentage points by analyzing the failure pattern of the previous pass. Three short audits beat one long audit.

Subagents that say "no". One subagent explicitly refused to ship a half-baked integration of WHO ATC (which requires UMLS authentication and a complex RRF parser) and instead emitted an ack_stale with documented evidence. That saved a runtime timeout I would have had to debug later.

Strict taxonomy in the /goal. The condition 14 + 3 + 1 = 18 matched exactly 18 Jobs in the ledger. Every Job had to terminate in one bucket. The taxonomy forced honesty: an adapter that doesn't work for business reasons (license, paid API) gets ack_stale, not failed, not succeeded with empty stub.

Persistent SQLite ledger as source of truth. Live retest hit the file every minute. The DB knew which adapter had a successful fetch and how many rows. No "trust me bro" — the data was on disk.

What broke

Stop hook strictness vs reality. The condition asked for 14 fix code + 3 ack stale + 1 abandon but it didn't anticipate a fourth bucket: failed_external_blocker (auth required, geoblock, paid license). After 4 passes I had 11 + 3 + 1 + 3. The Stop hook bounced 4 times asking why I wasn't at 14. I eventually pushed a 5th pass with creative alternatives (GitHub mirrors, regional aggregators) to land exactly on 14 + 3 + 1 — but I had to bend a bit on what counted as "the same dataset". The taxonomy was useful but slightly too narrow.

Audit overhead is real. 11 899 lines of audit markdown for 14 259 total LOC added. That's 83 % docs. Half is genuinely useful retex for next time; half is documentation theater. Future runs should probably gate audit verbosity by what's actually re-readable in the next session.

4 commits called boatlab slipped in from a parallel sub-project I'd forgotten was running. Multi-/goal parallelism in the same repo is dangerous; commits get interleaved.

Numbers, if you like numbers

• 9 h 27 min wall clock (including breaks, eating, the user replying)
• 45 commits (41 on this work + 4 from the parallel boatlab project)
• 41 subagent invocations across 5 different agent profiles
• 14 259 lines added, 2 362 removed (net +11 897)
• 67 Jobs created in the ledger (51 succeeded, 15 failed, 1 left queued)
• 23 catalog Objects, 3 new actions seeded
• 26 audit directories, 94 markdown files
• 4 156 914 rows ingested live across 14 revived adapters (top: GLEIF 3.3M LEIs, FINESS 242k French health facilities, INSEE 48k French first names)
• 0 regressions on the 17 pre-existing healthy adapters

What I'd do differently

1. Test live before audit. A 30-second --run-once would have shown me upfront that 91 % of the hard-coded URLs were 4xx/5xx, which would have changed my strategy day one instead of discovering it on pass 1.
2. Encode "external blocker" in the goal taxonomy. fix_code | ack_stale | abandon | external_blocker is a more honest 4-bucket model than 14 + 3 + 1.
3. Set a Stop hook ceiling. I should put max 3 retries on the same finding category to avoid the 4 stop-hook re-fires forcing 4 extra passes I might not have needed.
4. Smaller goals. A single 4000-char /goal chained 5 passes. Two goals of 2000 chars each, with explicit checkpoint between them, would have been clearer.

TL;DR

Claude Code's /goal with a strict Stop hook is the most autonomy-friendly setup I've used. It works because the hook is itself an LLM reading the transcript — it can detect bullshit, force honest categorization, and refuse to let you ship empty stubs. The cost is that you have to write your conditions like contracts, with bucketed taxonomies and verifiable deliverables, and you have to accept that "honest fail" outputs are first-class.

The big methodological takeaway: iterative auditing dominates exhaustive auditing. Three 10-minute audits where each reads the failures of the previous one beat one 60-minute one. Same total cost, much higher precision.

If you're running long autonomous sessions and your model just rubber-stamps "done" without checking, you're using the wrong harness. Put a strict Stop hook on it. It will refuse to lie.

---

Counter-questions welcome. Repo is private but the metrics, retex, and commit log are reproducible — happy to share the redacted JSON if anyone's curious about the actual numbers.

**TL;DR**: Running Qwen3.6-27B AWQ-INT4 on a single RTX 5090 (32 GB) for legal-claim extraction in a Go pipeline. Hit two non-obvious walls that cost me half a day: (1) BF16 KV cache caps you at 16K max-model-len, but FP8 KV gets you to 24K with the same VRAM footprint; (2) `temperature=0.2` under guided JSON schema triggers infinite repetition loops on this model — and the loop is not on text, it's on a numeric field generating a single integer with 5000+ digits. Sharing 42-run sampling benchmark, exact configs, and what actually works.

Posted to corroborate the [vLLM #40080 Gemma observation](https://github.com/vllm-project/vllm/issues/40080) and the [Qwen3.5 issue #145](https://github.com/QwenLM/Qwen3.6/issues/145) with concrete numbers on a Blackwell SM_120 setup.

---

## Hardware and stack

- GPU: NVIDIA RTX 5090, 32 GB VRAM, Blackwell SM_120

- CUDA 12.8, cuDNN 9.6

- vLLM 0.19.0 via `nvcr.io/nvidia/vllm:26.04-py3`

- llama-swap v216 orchestrating three model slots:

- Vision: Qwen2-VL-7B-Instruct (16K context, BF16 KV, swap)

- Reason: **Qwen3.6-27B AWQ-INT4** (this is the one I'm writing about)

- Embed: BGE-M3 (resident, ~2.3 GB)

- Workload: legal-claim extraction of structured output via JSON Schema, ~5W1H decomposition per claim

The reasoning slot uses the [cyankiwi/Qwen3.6-27B-AWQ-INT4](https://huggingface.co/cyankiwi/Qwen3.6-27B-AWQ-INT4) build. Internal architecture is `Qwen3_5ForConditionalGeneration` (GDN hybrid + Mamba) — needs vLLM ≥ 0.17 to run at all.

---

## Wall #1: VRAM math for max-model-len on a single 32 GB card

Initial config: `--gpu-memory-utilization 0.85 --max-model-len 12288 --dtype auto`. 26.5 GB VRAM, working fine for short docs. But 37 % of my email corpus exceeds 8K tokens, and the chain-of-thought prompt I use needs ~8K output tokens for the scratchpad. So `12288 - 8192 = 4096` input budget, which overflows on most non-trivial emails.

Measured KV cache scaling with BF16:

| max-model-len | KV cache (BF16) | Weights + KV | Verdict on 32 GB |

|---|---|---|---|

| 12288 (start) | ~13 GB | 27 GB | ✓ comfortable margin |

| 16384 | ~17 GB | 31 GB | ⚠ 1 GB free, kills multi-slot co-tenancy |

| 24576 | ~26 GB | 40 GB | ✗ overflow |

| 32768 | ~35 GB | 49 GB | ✗ physically impossible |

The bench tool community on r/LocalLLaMA was telling me to "just bump to 32K", but that's not feasible at all on a 32 GB card without quantizing the KV cache. So I tried FP8 KV.

### FP8 KV cache changes the picture

Adding `--kv-cache-dtype fp8` halves the KV memory:

| max-model-len + FP8 KV | KV cache | Total | Tient en 32 GB |

|---|---|---|---|

| 16384 + FP8 | ~8.5 GB | 22.5 GB | ✓ huge margin |

| 24576 + FP8 | ~13 GB | 27 GB | ✓ same footprint as 12K BF16 start |

| 32768 + FP8 | ~17 GB | 31 GB | ⚠ tight |

Empirical measurement on the live server, after killing the container and warm-up:

| Config | VRAM steady-state | Cold start (warm cache) | Free VRAM |

|---|---|---|---|

| 12288 BF16 (start) | 26.5 GB | 96 s | 5.5 GB |

| 16384 BF16 | 28.0 GB | not retested | 4.0 GB |

| **24576 FP8 (chosen)** | **28.4 GB** | **131 s** (+35 s vs BF16) | **3.6 GB** |

Counterintuitive: 24K FP8 consumes nearly the same VRAM as 16K BF16, because vLLM pre-allocates the KV pool to `gpu-memory-utilization=0.85` regardless of effective dtype/length. You don't see VRAM savings on the gauge — you capitalize the saving in *input capacity*. Net gain: input budget moves from 4K → 16K tokens at `max_tokens=8192`.

FP8 KV quality cost on AWQ-INT4 weights: theoretical 2–3 % degradation, in practice noise-level on AWQ-INT4 (the 4-bit weight quantization dominates). Validated empirically — see end of post.

### Production llama-swap config for Reason slot

```yaml

qwen3.6-27b:

cmd: >

docker run --rm --name vllm-reason

--gpus all --ipc=host

-v /inference/models:/models

-v vllm-cache:/root/.cache/vllm

-p 127.0.0.1:8003:8000

nvcr.io/nvidia/vllm:26.04-py3

vllm serve /models/qwen3.6-27b-awq-int4

--served-model-name qwen3.6-27b

--gpu-memory-utilization 0.85

--max-model-len 24576 --kv-cache-dtype fp8

--max-num-seqs 4

ttl: 300

```

---

## Wall #2: guided JSON + low temperature = infinite repetition

First smoke test of the pipeline with `max-model-len 24576` plus the corresponding client-side `MaxTokens: 8192`: one document (`04546`, a short 953-char .md) generated **68 claims, of which 67 had `text=""` and identical `char_start=107, char_end=238`**. Pure loop fail mode.

Initial hypothesis: model-level repetition bias. Looked at the literature:

- vLLM bug [#40080 (Gemma)](https://github.com/vllm-project/vllm/issues/40080): "When grammar restricts the token space to valid JSON tokens, the model's slight repetition bias becomes a strong loop because the grammar prevents the model from generating an EOS or breaking out of the pattern."

- [Qwen3.5/3.6 issue #145](https://github.com/QwenLM/Qwen3.6/issues/145): official sampling recommendation, **explicitly states "greedy decoding should not be used as it can lead to performance degradation and endless repetitions."** The pipeline was running at `T=0.2`, which is quasi-greedy.

So the bug is exactly what the vLLM ticket describes: the model has a baseline repetition tendency, guided JSON masks every token outside the schema, model can't emit EOS in the middle of an array, so it fills the array with whatever fits. On this corpus, sometimes that's `text=""` repeated, sometimes (as I found later in benchmarking) it's a single `char_start` integer with 5000+ digits.

### Bench protocol

7 sampling configs × 3 prototype documents (short, medium, complex) × 2 runs each = 42 calls against the live `:8156/v1/chat/completions` proxy (which forwards to llama-swap → vLLM Reason). Same JSON Schema, same prompt, same `max_tokens=8192`. Configs:

| Label | Sampling params |

|---|---|

| baseline_T02 | T=0.2 |

| hardened_T02 | T=0.2 + schema `minLength=1` on text + `maxItems=30` on claims |

| qwen_instruct | T=0.7, top_p=0.8, top_k=20, presence_penalty=1.5 (official Qwen instruct mode) |

| qwen_reasoning | T=0.6, top_p=0.95, top_k=20, presence_penalty=0.0 (official Qwen reasoning mode) |

| intermediate_T04 | T=0.4, top_p=0.9, presence_penalty=0.3 |

| reppen_only | T=0.2, repetition_penalty=1.1 |

| conservative_T03 | T=0.3, top_p=0.9, presence_penalty=0.5 |

### Bench results

Unique claims persisted per run, two runs per cell:

| Config | doc 04546 | doc 04547 | doc 19958 | Avg total | Loop fails |

|---|---|---|---|---|---|

| baseline_T02 | 11 / 12 | 5 / 3 | 11 / 11 | 26.5 | 0 |

| hardened_T02 | 11 / **FAIL** | 2 / 5 | 11 / 10 | 25.0 | **1** |

| qwen_instruct | 10 / 8 | 4 / 2 | 10 / 10 | 22.0 | 0 |

| **qwen_reasoning** | **11 / 12** | **6 / 4** | **10 / 18** | **30.5** | **0** |

| intermediate_T04 | 9 / 12 | 4 / 4 | 13 / 11 | 26.5 | 0 |

| reppen_only | 6 / 6 | 3 / 5 | 10 / 10 | 20.0 | 0 |

| conservative_T03 | **FAIL** / 12 | 6 / 3 | 9 / 9 | 25.5 | **1** |

Aggregate: 42 runs, 40 successes, **2 loop failures**. Both fails were on document 04546 (the short one), both at `T ≤ 0.3`. Failure mode confirmed by Python `int()` overflow: model emitted a 5000+ digit integer in a `char_start` or `char_end` field — pure numeric loop, not a text loop. A more permissive parser (which is what I had in Go originally) would silently truncate and accept garbage.

Average successful run latency: 63.8 s. Range 23.8–111.4 s on this prompt size (~6 KB system + 1 KB user).

### Findings

1. **`qwen_reasoning` is the winner**: +15 % unique claim coverage over baseline, zero loop fails on the pathological doc, conforms to official Qwen3.6 recommendation. Higher variance on complex docs (19958: 10 vs 18 unique claims between runs) — to absorb with defensive dedup on the consumer side.

2. **`T=0.2` (quasi-greedy) is the actual bug source.** 14 % loop failure rate on the pathological doc when T ≤ 0.3, 0 % when T ≥ 0.4. The official Qwen advice is empirically correct.

3. **`repetition_penalty=1.1` strangles** — −25 % coverage. Not the right knob for structured generation.

4. **`presence_penalty=1.5`** (official Qwen instruct mode value) is meant for short conversational replies, not multi-page JSON. Strangles too (−17 %).

5. **`frequency_penalty=0.5`** (a desperate fix I tried earlier in the day) is catastrophic on structured output — −77 % coverage measured in production smoke. Avoid.

6. **Schema hardening (`minLength=1` on text, `minimum/maximum` on integer fields, `maxItems`) is complementary**, not a replacement for sampling fix. Hardened schema still failed once at T=0.2 — the loop just shifted to another field (numeric instead of text).

### Final production config

Three coordinated changes, none of them sufficient alone:

**Server (vLLM)** — already shown above, the `24576 FP8` config.

**Client sampling** (Go pipeline payload):

```json

{

"model": "qwen3.6-27b",

"temperature": 0.6,

"top_p": 0.95,

"top_k": 20,

"presence_penalty": 0.0,

"max_tokens": 8192,

"response_format": {"type": "json_schema", "json_schema": {...}}

}

```

**Client schema** (in addition to the domain fields):

```json

{

"type": "object",

"properties": {

"claims": {

"type": "array",

"maxItems": 30,

"items": {

"properties": {

"text": {"type": "string", "minLength": 1},

"char_start": {"type": ["integer", "null"], "minimum": 0, "maximum": 100000},

"char_end": {"type": ["integer", "null"], "minimum": 0, "maximum": 100000}

}

}

}

},

"required": ["claims"]

}

```

**Client post-LLM**: defensive dedup on `(lowercased_stripped_text, char_start, char_end)` before INSERT, with a `needs_review` flag when `unique_count / total_count < 0.5` or `total > 30`. Catches the residual variance.

### Cost on the full run

Estimated for 1402 .md files:

| Metric | Baseline (T=0.2) | qwen_reasoning |

|---|---|---|

| Avg claims latency per doc | 35–80 s | 60–110 s (+30 %) |

| Unique claims per doc | n | n × 1.15 |

| Loop-failed docs | ~2–5 % expected | 0 measured in 42 runs |

| Docs flagged `needs_review` | n/a | est. 5–15 / 1402 |

---

## What I'd hammer if anyone is doing the same setup

1. **Don't trust `T=0.2` for any non-trivial JSON-schema-constrained generation on Qwen3 family.** The official Qwen team flagged it, the vLLM Gemma ticket confirms it's a grammar+repetition interaction, my 42-run bench reproduces it. Use T=0.6 minimum.

2. **Don't use `repetition_penalty` or `frequency_penalty` to fight JSON loops** — they punish lexical variation in legitimate paraphrases. Wrong knob.

3. **Schema fields that accept integers need bounded ranges.** A `char_start: integer` without `maximum` is an invitation to a numeric loop.

4. **FP8 KV cache is the single best knob to push context length on a 32 GB consumer card.** Same VRAM footprint, ~2x effective context. Quality impact is negligible on top of an already-INT4-quantized model.

5. **Always log `usage.completion_tokens`** when calling `/v1/chat/completions` with structured output — if your call routinely hits the max, you've got a silent failure mode.

6. **Cold start on Qwen3.6-27B AWQ-INT4** with the `torch.compile` cache persisted to a Docker volume: ~96 s BF16, ~131 s with FP8 KV (extra calibration step). Without persisted cache: 141 s. Worth the volume mount.

### Reproducibility

42-run bench script, results JSON, and exact prompt assets are kept on the server side under `/tmp/claim_bench/`. Happy to share if anyone wants to repro on their own Qwen3.6 quant variant — I expect the loop behavior to generalize across AWQ-INT4 / NVFP4 / GGUF, since the root cause is the model-level repetition bias × grammar masking, not the quantization.

If anyone has a clean explanation for why the loop on `char_start` produces a *single* 5000-digit integer rather than a stream of normal integers, I'd love to hear it. My hypothesis is that once the model commits to a digit token after `"char_start": `, the only grammar-valid next tokens are more digits or `,` / `}` — and if the digit-token transition probability beats the closing-token probability, it never closes.

---

## References

- Qwen3.5/3.6 sampling recommendations: [QwenLM/Qwen3.6 issue #145](https://github.com/QwenLM/Qwen3.6/issues/145)

- Grammar-amplified repetition (vLLM): [vllm-project/vllm issue #40080](https://github.com/vllm-project/vllm/issues/40080)

- Empty-array bug under guided JSON: [vllm-project/vllm issue #13821](https://github.com/vllm-project/vllm/issues/13821)

- vLLM Quantized KV Cache doc: [docs.vllm.ai — quantized_kvcache](https://docs.vllm.ai/en/latest/features/quantization/quantized\_kvcache/)

- vLLM Structured Outputs: [docs.vllm.ai — structured_outputs](https://docs.vllm.ai/en/v0.8.2/features/structured\_outputs.html)

- Qwen3 official model card with sampling guidance: [Qwen/Qwen3-0.6B on HF](https://huggingface.co/Qwen/Qwen3-0.6B)

Setup date: 2026-05-19. Environment: `nvcr.io/nvidia/vllm:26.04-py3` (vLLM 0.19.0), Blackwell SM_120, RTX 5090 32 GB.