Recent news from the leakers claiming that at this very moment deepseek v4 pro ga is rolling out. Hurray!
https://x.com/i/status/2087558842271813860
Also grok 4.6 coming. Hurra? https://x.com/i/status/2087561166864146813
What the hell happening? Seems like everyone waited for the qwen goes open weights lmfo
I see comments everywhere praising the grokbot but as far I see there is no value added different than a hermes on vps. What is the value I am missing here ? Actually I remember that lot of chinese Aı company also build the similar products under the name of bla bla claw etc. I am not mentioning the pricing and the not able choose the model you used on it
I see comments everywhere praising the grokbot but as far I see there is no value added different than a hermes on vps. What is the value I am missing here ? Actually I remember that lot of chinese Aı company also build the similar products under the name of bla bla claw etc. I am not mentioning the pricing and the not able choose the model you used on it
EDIT: TL;DR up top, fair complaint about the length. 2 minute read.
Loaded 3 years of chat history (3,629 conversation chunks, 23M chars, 36k extracted memories) into self hosted Hindsight. Ran it as a shadow import for 10 hours while my old memory provider kept serving, then benchmarked both on the same 45 case eval set and switched.
Same 45 questions, same eval harness, both arms measured identically:
| old provider | Hindsight |
|---|---|
| recall | 0.514 |
| context tokens per query | 1,724 |
| questions fully answered | 23/40 |
| stale facts leaked | 0 |
| p95 latency | 9ms |
Per category, where it actually moved:
| category | old | Hindsight |
|---|---|---|
| episodic recall | 0.80 | 1.00 |
| recency | 0.60 | 1.00 |
| cross session | 0.50 | 0.80 |
| temporal | 0.21 | 0.26 |
| procedure | 0.40 | 0.40 |
| supersession | 0.53 | 0.53 |
| profile lookup | 1.00 | 0.83 |
Short version: it is much better at "what happened in that conversation months ago" and level or slightly worse at everything else. Cross session recall was the whole reason I did this, and it went from coin flip to 0.8.
Memory injects ~3,600 extra tokens into every turn, so I assumed the bill goes up. Ran 8 complex questions through the real agent, once while memory was silently broken and once after fixing it:
| memory broken | memory working |
|---|---|
| wall time | 739s |
| input tokens | 1,513k |
| tool calls | 36 |
It adds 3,600 tokens per turn and still cuts half a million tokens off the total. When the agent already knows the answer it stops going on a file reading expedition. If you run an agent with tools and think memory is a cost center, measure it.
Compare at equal token budget, not equal result count. At top_k=8 both providers looked tied. But the old one stores long paragraphs and Hindsight stores short facts, so 8 vs 8 handed one side 4x the context. That single mistake almost made me abandon the project.
Retrieval depth is not linear. k=48 scored worse than k=32 while costing 1,000 more tokens. Saturates completely at k=96, and k=128 buys 399 extra tokens for exactly zero recall.
Good answers do not prove memory works. 8 real sessions, all 8 answers correct. Then I checked what was actually injected: 7 of 8 got zero memory. The agent had just read files with tools instead. Cold start was eating the entire retrieval budget. I nearly shipped a dead memory system with a clean test report.
Four config values turn cutover into a silent no-op. Wrong mode (spawns its own instance and never touches your bank), wrong bank id, recall_types pointing at a type I had 300 of instead of the 36,000 I had, and auto recall off. Everything reports healthy and memory does nothing.
A broken service and an idle one log the same line. My catch-up scan had a flag meaning "only messages with id at or below zero". It ran on schedule, reported success, inserted nothing, forever. Identical to "no new conversations". Found it by removing the flag and watching 2 episodes appear instantly.
Score thresholds cannot make it say "I don't know." Made up questions score 1.039, real ones 1.077. No separation, so no threshold works. It abstains fine at the model layer anyway, just not at retrieval.
Consolidation is real but expensive. 100 memories per 50 minutes, so two weeks for my bank, and while running it pushed live query latency from 1.1s to 10.7s. Turned it off and kept the 4 stale facts.
Current state: 36,000 memories, live as main provider, ~1.2s recall, 8/8 on my hard question set, token bill down.
Full detail and the rest of the bugs below.
---------------------------------------------------------------------------------------------------
That was the part I did not expect, so I am putting it first.
Adding a memory layer means you inject extra context into every single message. Mine adds about 3,600 tokens per turn. So I assumed the bill goes up and I would be trading money for quality.
I measured it with the same 8 complex questions, once with memory actually reaching the model and once without:
memory broken memory working
sessions with memory 1/8 8/8
total wall time 739s 416s 44% faster
total input tokens 1,513k 1,011k 33% fewer
tool calls 36 26 28% fewer
It adds 3,600 tokens per turn and still saves half a million tokens overall. Reason is dumb in hindsight: when the agent already knows the answer it stops going on a file reading expedition. Ten fewer tool round trips paid for the memory many times over.
If you are running an agent with tools and you think memory is a cost center, measure it. Mine is a savings.
Three years of conversation history spread across two different agent stacks, two SQLite databases, hundreds of session JSON files, agent JSONL logs, and a pile of markdown notes. Nothing was searchable in any useful way. Ask the agent something from four months ago and it had no idea.
Goal was to get all of it into Hindsight (open source agent memory, MIT, self hosted with Docker) and then decide, with actual numbers, whether it beats the simple memory provider I was already using.
Total: 3,629 episodes, 23.2 million characters submitted, 35,800 extracted memories. Took about 10 hours of processing.
I ran the whole thing as a shadow import first. Real provider untouched, agent kept serving normally, nothing in the live prompt path. Only flipped the switch after the numbers came in.
First A/B I ran said Hindsight was barely better than what I already had. 0.514 recall versus 0.528. Basically noise. I almost stopped there.
Then I noticed the token column. Same top_k of 8 for both arms:
old provider: 0.514 recall, 1,724 context tokens
Hindsight: 0.528 recall, 462 context tokens
The old provider stores long paragraphs. Hindsight stores short precise facts. Eight of each is not a fair fight. I was giving one side four times the context budget and then concluding it knew more.
So I swept retrieval depth:
k=8 0.500 recall, 448 tokens
k=16 0.500 recall, 883 tokens
k=24 0.583 recall, 1,308 tokens <- beats baseline on BOTH axes
k=32 0.597 recall, 1,590 tokens
k=48 0.583 recall, 2,613 tokens <- goes DOWN
k=64 0.639 recall, 3,562 tokens
k=96 0.694 recall, 5,089 tokens
k=128 0.694 recall, 5,488 tokens <- zero gain, pure cost
Three things fall out of that table.
At k=24 it wins on both axes at once. More recall for fewer tokens than the thing I was replacing.
The curve is not monotonic. k=48 scores worse than k=32 while costing a thousand more tokens. More context does not reliably mean better ranking.
It saturates hard at k=96. Going to 128 buys 399 extra tokens and exactly zero recall. The ceiling is what is in the bank, not how deep you dig.
Final numbers at full data, k=24:
old provider Hindsight
recall 0.514 0.597 +16%
context tokens 1,724 1,450 -16%
fully covered cases 23/40 27/40
stale facts returned 0 4
p95 latency 9ms 2,099ms
Per category, biggest wins were episodic recall (0.8 to 1.0), recency (0.6 to 1.0) and cross session (0.5 to 0.8). Procedure and supersession came out level. Profile lookups got slightly worse.
Worth saying: my metric checks whether the correct fact is present in the returned context. It does not check whether the model then used it. At k=96 the right answer sits inside 5,000 tokens where attention dilution is real and my metric is blind to it. That asymmetry is why I picked the smaller operating point.
Old provider answers in 9ms. Hindsight takes about 1.2 seconds. That is a 130x regression on paper.
My first instinct was to tune it. Threads, budget, batch size, the usual. Then I read the integration code and found the actual problem was not speed at all.
The plugin warms a recall at the end of each turn and serves it on the next one. But it was ignoring the query argument entirely. So the memory injected into your current question was retrieved using your previous question. Ask about your project list, get memories about whatever you said before that.
Latency was never the bug. Relevance was.
Fixed it by tracking which question a warm result belongs to. If it matches, serve instantly at zero cost. If it does not, recall for the current question inside a bounded budget and fall back to the warm result if the budget expires.
Then I broke it in a new way, which was educational.
I set the budget to 2.5 seconds based on my own measurement of 1.1 second recalls. Ran the scenario suite. 8 out of 10 returned nothing, and every single one took exactly 2,501 ms.
Turns out the client funnels everything through one event loop. My timed out threads were abandoned but still holding that loop, so every following call queued behind a corpse and hit the ceiling too. A budget that is too tight does not make things fast, it creates a pileup.
Raised it past the real p95 and it went from 2/10 to 8/10 with p50 at 1.2 seconds. Same code, one number.
At the very end I ran 8 real sessions through the actual agent, fresh session each time, and asked hard questions. All 8 answers were correct and detailed. Looked like a clean win.
Then I checked how much memory was actually injected into each one.
session 1: 12,765 chars
session 2: 1,192 chars
session 3: 1,192 chars
...
session 8: 1,192 chars
That 1,192 is a fixed header. Seven out of eight sessions got zero memory. The answers were good because the agent went and read files with tools instead. It worked for it.
If I had judged by answer quality alone I would have shipped a memory system that was not being used and never known.
Cause was cold start. First recall in a fresh process spent the whole budget building the HTTP client and returned nothing. I added a warmup at session init, which then raced against the real query on that same single event loop and made it worse. Fixed it by making the query wait for the warmup instead of competing with it.
After that, 8/8 sessions with 12,000 to 14,000 characters of memory each, and the numbers at the top of this post.
Lesson I keep relearning: a good output is not proof the thing you built is what produced it.
Score thresholds cannot make it say "I don't know." The API takes a min_scores parameter. I swept it from 0.2 to 0.65 and got byte identical results every time. Looked at the raw scores:
made up questions: final score max 1.039 to 1.067
real questions: final score max 1.077 to 1.100
There is no separation. The final score saturates near 1.0 for everything, semantic overlaps, keyword has a bit of signal but still overlaps. Multilingual embeddings put every well formed sentence in roughly the same neighborhood. Abstention is not solvable at the retrieval layer, full stop.
The funny part: end to end it works fine anyway. I asked it what coffee I drank in a city in 1987 and it said it had no record of that, and added that I would not have been born yet based on my age in memory. The model handles it even though retrieval hands it 40 irrelevant facts. I was pessimistic about the wrong layer.
Consolidation is real but expensive. It merges duplicate and contradictory memories into synthesized observations, and it is the only mechanism that fixes stale facts. I measured it at 100 memories per 50 minutes. For 36,000 memories that is roughly two weeks of background processing. Worse, while it runs it competes with live recall for the LLM and the DB, and pushed my query latency from 1.1 to 10.7 seconds. Turned it off. Left the four stale facts. Not worth it right now.
prefer_observations and type filters did nothing measurable.
Four config values were wrong in a way that would have made the cutover a silent no-op:
observation, and observations only exist after consolidation. I had 36,000 world and experience memories and roughly 300 observations. It would have returned almost nothingAll four look harmless in a config file. Together they mean you flip the switch, everything reports healthy, and memory silently does nothing.
Other things worth knowing:
operation_id must be a UUID. I was using a truncated sha256 for deterministic idempotency and got a 422. Switched to UUID5 over a fixed namespace, which keeps determinism and satisfies the validator.
Watch container memory. Mine was sitting at 980MB against a 1GB limit before any load, and the cgroup had already hit its ceiling 1,513 times. There is a closed upstream issue about API memory growth on older versions. I raised the limit and cut the DB pool. Anonymous RSS turned out to be a stable 855MB baseline, not a leak, but a multi day import would have OOM looped on the original setting.
Check your worker slot math. Mine had 2 slots with 1 reserved for consolidation, which was disabled. So the import ran at exactly one concurrent extraction and I wondered why it was slow. Freeing that slot and raising the count took throughput from 157 to 630 episodes an hour.
Rate limits are real on the heavy tail. The last third of my queue was the long multi turn conversations, 30 to 40 extracted facts each, 3 to 6 minutes apiece. Provider started returning 429s. Circuit breaker plus durable retry handled it with zero lost work, but my ETA went from 3 hours to 9.
Writing this part because the pipeline bugs were nastier than the integration ones and every single one was found by measuring, not by reading code.
336 real notes were being silently excluded. My coverage rules classified anything outside a memory/ directory as a workspace working file. That swept up identity documents, an ideas folder, reports and findings. All genuine user authored content. Caught it by auditing the exclusion list instead of trusting the residual count, which was happily reporting zero.
11,138 false redactions from file paths. I built the secret scanner to register configured secret values from env and config files. It also registered anything long and high entropy, which includes filesystem paths. Paths appear constantly in developer conversations. Every occurrence got replaced with a redaction marker. My memory would have gone in full of holes.
Then config identifiers did the same thing. After the path fix it was still firing 14,565 times. The model id, the bank name, ordinary lowercase-with-dashes strings sitting under keys named token or auth. Added a shape test asking whether a real credential could plausibly look like this. False hits went from 111 per 364k characters down to 2.
Split secret detection was masking entire episodes. If a credential appeared in fragments, the code masked from the first fragment to the last. In a long conversation that is the whole thing. Changed it to mask only the fragments.
A circuit breaker that never used its configured cooldown. Off by one on the exponent, so the first trip always waited double. Found by a test that asserted the documented contract.
The catch-up service could never have worked. I passed --hermes-max-message-id 0 to the periodic rescan, meaning "only messages with id at or below zero". No new conversation would ever have been captured. The scan reported success every time it ran, inserting nothing, which looks identical to "nothing new to do". Only caught it because I removed the flag and immediately saw 2 new episodes appear.
That last one is my favorite failure mode. A broken thing and a correctly idle thing produce the same log line.
Run it as a shadow first. Mine ran 10 hours against a live agent that never noticed, on a separate bank and volume, with the old provider still serving. Cutover was one config line after the numbers were in.
Compare at equal token budget, not equal result count. This flipped my entire conclusion.
Verify the plumbing separately from the output. Answer quality told me everything was fine while seven of eight sessions were getting no memory at all.
Measure the thing you actually run. My clean 1.1 second recalls came from hitting the API directly. The real integration path had a cold start that cost the first message of every session its entire memory, and I would not have seen it from the outside.
Idempotency is worth building on day one. A full rescan of 3,626 items created zero new identities, so I could rerun the scanner whenever I wanted without thinking about it. Content addressed IDs, deterministic operation IDs, insert or ignore.
Do not trust a zero. Residual count zero, dead letter count zero, inserted zero. Each of those meant something was working right in one place and something was silently broken in another. Zero is a claim, go check what produced it.
Current state: 36,000 memories, live as the main provider, roughly 1.2 second recall, 8 out of 8 on my hard question set, and a token bill that went down. Consolidation off, four known stale facts, abstention working at the model layer despite retrieval offering no help. Rollback is one config value and I have verified snapshots of everything.
Happy to answer questions about any of it.
Spent most of today putting Muse Glimmer through a proper harness on a single 3090 (24GB, Q4_K_XL + DFlash, no mmproj). Posting because two of the things I hit cost me hours and I'd rather you skip them.
The gotcha that made me almost write the model off
I ran my usual eval suite and it scored 6/13. Half the failures had completely empty responses. I was about to conclude the quant was broken.
It wasn't. Muse thinks before every single answer, and my suite had per-case max_tokens between 60 and 500. At xhigh reasoning it burns the whole budget thinking and never emits the answer — you get finish_reason: length and an empty content. Bumped the budget and the same suite went to 11/13.
If your harness caps output tokens low, this model will look like it's failing when it's actually just been cut off mid-thought. Measured on a trivial "which city" question: low = 120 tokens, medium = 318, high = 1245, xhigh = 1760. Give it at least 16k of headroom.
Also worth knowing: --reasoning-budget 0 does not disable thinking on this template. It thinks at every level.
Speed at filled context, not empty slots
Most numbers I see are decode measured with a nearly empty KV. That flatters everything. Here's decode with the context actually filled, greedy so the speculative acceptance is reproducible:
| filled prompt | DFlash off | DFlash on |
|---|---|---|
| ~2K | 34.6 tok/s | 62.6 tok/s |
| 105,671 | 21.6 tok/s | 37.8 tok/s |
| 191,015 | — | 40.5 tok/s |
So DFlash is ~1.75-1.8x on Ampere and the gain does not collapse as the context fills. Prefill goes 916 → 500 → 435 tok/s over the same range.
One thing nobody seems to mention: DFlash acceptance depends on your sampling temperature. Same config, same prompt — greedy gave 0.131 acceptance / 37.8 tok/s, temp 1.0 gave 0.093 / 30.5 tok/s. If you're doing agentic work at low temp you get more out of the drafter than the prose benchmarks suggest.
--spec-draft-n-max 15 is the actual ceiling, by the way. The drafter's block_size is 16 and llama.cpp clamps to block_size - 1.
Context past 131k
Config says max_position_embeddings: 131072 with no rope scaling, but the layer layout is why it stretches: 39 sliding-window layers (2048) with rope_theta 500000, and 13 full-attention layers with rope_theta = 0. The global layers are NoPE. So there's no rope extrapolation to break — the sliding layers never see more than 2048 positions, and the global ones have no positional encoding at all.
Ran a needle test at three depths (8% / 49% / 91%) with q8_0 KV and xhigh reasoning:
llama-server hard-caps the slot to n_ctx_train in server-context.cpp, so -c 200000 alone gets you a 400 with "exceeds the available context size". You need:
--override-kv muse-glimmer.context_length=int:262144,dflash.context_length=int:262144
Note the dflash key. I missed it the first time and only overrode the main model.
KV is genuinely cheap because only 13 layers hold long-range state: 7,072 bytes/token at q8_0, 13,312 at f16. At 262k that's 1.7GB q8 / 3.3GB f16. On a 3090 the VRAM ceiling works out to roughly 650k tokens with q8 — VRAM stops being the constraint, prefill time becomes it.
I went with q8_0 KV. It scored 3/3 at 200k, so f16 can only tie it, and it saves 1.35GB.
Non-English
This is the part that surprised me most. I'm Turkish and every local model I've tried in this size class is either stiff or subtly wrong in Turkish. Muse handled all of it: proofreading, a multilingual status task, conversational writing, a short creative piece, and a critical-thinking prompt where it had to name a logical fallacy and lay out how to test the claim — all in Turkish, all clean. It correctly called out a benchmark-to-real-users inference as a proxy/external-validity problem, in Turkish, unprompted about the terminology.
Model card says 100+ languages. For Turkish specifically I'd say it's the first local 30B I'd actually let write something a customer reads.
Where it actually fails
Not going to pretend this thing is flawless. It consistently failed one interval-merging task in one-shot mode, even with a 12k token budget. It wrote start <= last_end where integer intervals need start <= last_end + 1, so [(10,10),(11,13)] came back unmerged instead of [(10,13)]. Failed twice, deterministically.
But — and this is the interesting part — the same task through an agent loop passed. It ran the tests, saw the failure, and fixed it. So the one-shot weakness closes when you let it iterate. Which tracks with it being trained for agentic use rather than one-shot Q&A. Use it as an agent, not as an answer box.
Harness token audit
Since I had the server logs, I counted actual tokens per harness across 6 tasks (coding, tool use, and four language/reasoning tasks). Numbers are from prompt eval time / eval time in llama-server, not from what the CLIs report:
| harness | passed | turns | input tok | output tok | total | system prompt |
|---|---|---|---|---|---|---|
| pi | 6/6 | 20 | 77,101 | 10,319 | 87,420 | 11,744 |
| prime-agent | 5/6 | 34 | 78,220 | 12,924 | 91,144 | 12,381 |
| opencode | 6/6 | 21 | 239,228 | 17,782 | 257,010 | 39,024 |
opencode burned 2.9x the tokens for the same work, entirely because of a ~39k token system prompt that gets resent every turn. On a 128k context that's a third of your window gone before you type anything. Nothing wrong with opencode as a tool, but on a local model where you're paying for every prefill token in wall-clock time, it's a real cost.
prime-agent took 34 turns and still landed near pi's token count, because its per-turn context stays lean. Its one "failure" was asking me a clarifying question instead of producing the list — arguably correct behavior, just bad for a non-interactive -p run.
My config
llama-server \
--model Muse-Glimmer-30B-UD-Q4_K_XL.gguf \
--spec-type draft-dflash \
--spec-draft-model dflash-kquant.gguf \
--spec-draft-ngl all --spec-draft-n-max 15 \
-c 262144 \
--override-kv muse-glimmer.context_length=int:262144,dflash.context_length=int:262144 \
-ngl 999 -fa on -fit off --parallel 1 \
--cache-type-k q8_0 --cache-type-v q8_0 \
-b 2048 -ub 512 \
--cache-reuse 1024 --reasoning-preserve --jinja \
--temp 1.0 --top-p 0.95 --top-k 64
Sits around 21GB with 262k allocated.
One more: the reasoning level is a template variable called reasoning_strength, not reasoning_effort. Your CLI's --thinking high flag probably sends reasoning_effort and does nothing. Set it server-side instead:
--chat-template-kwargs '{"reasoning_strength":"xhigh"}'
Levels are low / medium / high / xhigh, default high. The template does no validation, so if you pass "max" it'll happily render "Reasoning strength: max." into the system prompt — an untrained value. Stick to the four.
You can verify what's actually being rendered without burning a generation:
curl -s -X POST localhost:8080/apply-template -H 'Content-Type: application/json' \
-d '{"messages":[{"role":"user","content":"hi"}]}'
Caveats: single seed (7391) on the eval suite, one run per harness in the token audit, and I didn't test past 200k filled. My prior model on this box was Qwen3.6-27B at 128k doing 10-13 tok/s at filled context, so take the comparison as "same box, same day," not a controlled study.
Hey guys,
So i wanna test the capabilities of this new model thanks to the opencode go subscription, however my experience with opencode and glm 5.2 was not like i imagined or people advertised on reddit. I couldnt get performance on the critical thinking with opencode and i thought maybe harness was not suitable for that model.
​
My question is which harness you guys using with this model and how is the experience for you?
​
Btw brainstorming and critical thinking is much more important for my use cases.
i dont think my issue with Anthropic is just limits or pricing or one bad Claude Code week
the bigger problem is trust
Anthropic built its whole public image around being the responsible ai company. safer more careful more honest more user aligned. and honestly that branding worked on me for a while
but the last few months made that harder to believe
Claude Code quality dropped and a lot of users noticed it. people kept saying it felt worse at coding more forgetful and less reliable. then Anthropic later posted their own postmortem and admitted there were real issues. reasoning defaults changed. a cache bug caused context problems. a system prompt change hurt coding quality
so users were not just imagining it
then the Pro plan confusion happened. for a short time it looked like Claude Code was being moved away from the regular Pro plan and pushed toward more expensive plans. Anthropic said it was only a small test and reverted it but that still damaged trust. it looked like the company was testing how much users would tolerate
then there are the usage limits. i understand compute is expensive. i understand demand is high. but from the user side it often feels like you are paying for access and still constantly rationing messages. that is not a great user experience
and the data retention change also feels important. even if it is opt in Anthropic is still asking consumer users to let their data train future models and be retained much longer. again maybe that is normal for an ai company but that is exactly the point. Anthropic keeps acting more normal while still branding itself as morally different
same with the copyright settlement around books. people can argue the legal details but it still weakens the clean ethical image
i am not saying OpenAI is better. OpenAI has plenty of problems
my point is that Anthropic feels more disappointing because they sold themselves as the trustworthy alternative
when a company builds its identity around trust the standard should be higher
so my question is simple
what would Anthropic actually need to do to regain user trust
clearer limits
no confusing pricing tests
better communication when model behavior changes
public changelogs for Claude Code quality changes
stronger guarantees around user data
because right now it feels less like a special responsible ai company and more like a normal ai company with better branding