Hey folks 👋

We just open sourced TrueForge, our vendor-neutral agent harness for building general-purpose agents.

It handles the runtime pieces that get painful quickly : context management, tool/MCP execution, subagents, sandboxing, approvals, persistent state, and more.

We also benchmarked the harness itself. With the same Opus 4.8 model, TrueForge delivered a similar solve rate at ~30% lower cost than Claude Managed Agents. Switching to an open model pushed that to ~75% lower cost on the same benchmark.

Would love feedback from people building agents.

⭐ Star the repo: https://github.com/truefoundry/trueforge

📖 Read the launch article: https://x.com/truefoundry/status/2090081376330715176

I've been building TokenMizer, a local proxy that sits between your editor/CLI and whatever model you're using (Claude, GPT, Ollama) and handles two things I kept re-solving by hand: session checkpoint/resume, and a graph-based memory instead of a flat transcript.

The problem: once a long agent session hits the context limit, the usual fix is summarization, and summaries lose the reasoning behind a decision, not just the decision itself. I'd see a summary saying "switched to Argon2" with no trace of why bcrypt was rejected, so the agent would re-litigate the same tradeoff two sessions later. Flat transcripts have the opposite problem: everything is kept, but nothing is prioritized, so retrieval is just recency-biased keyword luck.

What TokenMizer does differently: instead of one growing text blob, decisions, constraints, and open questions are stored as nodes with edges (this decision depends on that constraint, this question was resolved by that decision). Checkpointing snapshots that graph plus a resumable session state, so you can kill a session and pick it back up without replaying the whole history through the model again.

Where it's rough: there's no eval harness yet comparing retrieval quality against a naive flat-transcript baseline, so right now my evidence is anecdotal (my own sessions), not benchmarked. I also learned the hard way that benchmarking your own memory system by asking it questions only it can answer is circular, so I'm holding off on publishing numbers until I have an honest comparison.

Repo: github.com/Shweta-Mishra-ai/tokenmizer (I'm the author). It's a Python project, MIT licensed. If you've hit the same summarization-loses-reasoning problem, I'd be interested in how you're handling it, and PRs/issues on the eval-harness gap would genuinely help.

pagedMark removes AI provenance from content you generated yourself. Two different things, and it is worth separating them. The first is metadata: C2PA Content Credentials, EXIF, XMP, IPTC, the generator parameters. That part is easy and verifiable, and a screenshot does it too. The second is the invisible pixel watermark that a screenshot does not touch, the SynthID class of marks, which has to be disrupted by regenerating the image itself.

Coverage on the image side is ChatGPT, gpt-image, DALL·E, Sora, Gemini and Nano Banana for the invisible marks, plus a registry of visible vendor labels (Doubao, Jimeng, Qwen, Kling, Yuanbao, Baidu, LibLibAI, Samsung Galaxy AI). On the video side it handles the visible marks from Sora, Veo, Seedance, Dola, Hailuo and Kling, and the metadata that travels with them.

The reason this is worth a post rather than a link is that it is built for Apple Silicon instead of ported to it. I spent several days getting the pipeline to run correctly on an M5 with 16 GB, meaning predictable and measured rather than merely launching. Most of what I assumed turned out to be wrong, so the measurements are below.

The four-step distillation LoRA invents texture, and more steps make it worse

A low strength edit runs the tail of a long schedule: strength 0.15 executes the last four steps of twenty seven. A LoRA distilled for four timesteps spanning the entire noise range is off its distribution there. Wherever nothing conditions the model, and flat dark fabric gives a Canny ControlNet no edges at all, it fills the gap from its prior. On a night photograph that arrives as coloured camouflage across black clothing.

Asking the distilled model for more steps made the artifact worse, which is what identified the distillation rather than the step count as the cause. Dropping the LoRA costs roughly three times the wall time and buys back both fidelity and correctness.

Three wrong theories I paid for first, in case they save someone else the time. Not the fp16 VAE: a bare encode and decode round trip of the same crop is clean in fp16 and in fp32, tiled or whole, at 34.6 dB. Not Metal's fp16 in general: bf16 measured marginally worse. Not Canny picking up sensor noise: the Canny map of that region is completely empty, which was the actual clue.

Metal pages instead of failing, so memory has to be measured

torch.mps.recommended_max_memory() reports 11.84 GiB on a 16 GB machine. Exceed it and nothing raises an error. The process starts swapping, and a run that should take 23 seconds takes an hour instead.

With VAE tiling disabled, a 1.57 MP frame peaks at 18.74 GiB and takes 59 seconds. With tiling it peaks at 10.92 GiB and takes 23 seconds. So tiling carries real weight on a small machine, but its boundaries leave a faint texture, which is why it is now decided per frame from the device budget rather than switched on globally.

Diffusion untiled at 2.5 MP went into swap and did not finish within twelve minutes. Tiled at 1024 px, a 5.07 MP frame holds 10.93 GiB, finishes in 88 seconds, and keeps its native geometry.

Sequential CPU offload works on MPS, and it is what makes 8 GB usable

The stack is 7.7 GiB of weights. An 8 GB Mac reports a working set of roughly 5.3 GiB, so it does not fit however the activations are handled. Streaming the weights one module at a time:

Twenty seven times less peak memory for 3.4 times the wall time. The plan is chosen from the measured budget and then printed, because a run three times slower than the fast path looks broken unless it says why.

Two Metal gaps worth knowing if you are porting anything

torch.float8_e4m3fn does not exist on MPS at all. The error is RuntimeError: Undefined type Float8_e4m3fn. Any pipeline that streams float8 weights, which several VRAM managed stacks do, cannot be loaded there under any configuration.

SAM's processor emits its box and point prompts as float64, which Metal also has no type for, so moving the batch to the device raises rather than degrading. A single cast fixes it, but nothing tells you that is the problem.

The expensive one: fp16 sampling on MPS returns zeros silently

I added a memory optimisation that encodes the two fixed prompts once and drops the text encoders, saving a measured 1.52 GiB of the 8.79 GiB the loaded stack holds. Two of four face crops then came back as all zero black rectangles. Deterministically, at the same seed, with nothing raised anywhere.

The embeddings were innocent. CPU fp16, MPS fp16 and fp32 encodings of that prompt agree to 0.0009 on tensors with a standard deviation of 3.06, and the same crop generated in isolation is correct either way. Freeing unrelated memory changed the allocation pattern the crops met after the global pass, and that alone was enough. I withdrew the optimisation and added a guard that drops any empty crop instead of compositing it.

If you run fp16 diffusion on Metal, check your output for degeneracy. It will not tell you.

What it does not claim

Regeneration is not payload deletion. The image changes: faces, text and fine detail move, and the numbers above are the measured size of that change rather than a reassurance.

No public local decoder exists for SynthID class marks, so identify reports unknown and never clean. Verification is the provider's verifier or nothing. The 0.15 operating point comes from the upstream project's record against openai.com/verify on CUDA. I have not re-run that check on Metal, and Metal is not bit identical to CUDA, so I am claiming the same operating point and not the same verdict.

It is for content you generated or own. The visible mark registry accepts AI generation labels only. Stock agency previews, marketplace and classifieds watermarks are deliberately out of scope, and that boundary is in the repository rather than only in this comment.

Because "how much did that cost my picture" is the whole question

pagedmark measure before.png after.png

PSNR over the frame, PSNR per detected face, and how much mid band structure appeared where the source was flat and dark. The third metric is the one that caught the camouflage, and it took two attempts. Per pixel chroma statistics rank the artifact below the source, because the source's own sensor grain carries more per pixel variance than the invented blotches do. A plain band ratio fails too, since any linear filter reports doubled grain and doubled blotches identically. Normalising mid band energy by fine detail energy, against the same ratio in the source, measures the shape of the spectrum instead of its size.

uv tool install "pagedmark[diffusion]"
pagedmark invisible photo.png -o clean.png
pagedmark invisible photo.png --preview      # 46.6 s instead of 112.6 s

Code: https://github.com/doofzoff/pagedMark
PyPI: https://pypi.org/project/pagedmark/

Happy to answer anything about the Metal specifics. That is the part I would have wanted written down before I started.

I’ve been testing how far I can push long-context serving on a single A100 before the KV cache becomes the thing that kills concurrency.

Here’s the latest result:

9 concurrent users
~128K context per user
1× A100 80GB
Needle checks run independently for every user
FP8 holds the first 5 users

One technical detail I also want to correct from some of the earlier shorthand:

The V tail is tiered 4/3/2-bit bit-plane. It is not plain INT4.

Some of the recipe labels are stale. The actual pool sizes line up with the tiered 4/3/2-bit representation, so calling the tail “INT4” would describe the old label rather than what is actually being stored.

Method

I don’t keep the entire KV cache at one fixed precision.

The cache is divided into regions. Newer or more sensitive KV stays at higher precision, while older regions progressively move into cheaper representations.

As the context grows, the cache footprint can keep falling without forcing the same quantization level across the entire cache.

The 9-user run is currently the capacity end of the curve.

I’m also rerunning the 2-user and 4-user points using the same V2 tiered recipe. The July numbers came from the earlier V1 recipe family, and I want the next throughput curve to be completely apples-to-apples.

So far, a single A100 is keeping 9 users at ~128K context resident, with the per-user retrieval checks still passing.

There are a lot of knobs here: concurrency, context length, per-user throughput, precision, and memory.

I want to make those knobs much more flexible so long-context serving isn’t immediately dictated by the KV-memory wall. At scale, that can have a very real impact on how much useful inference you can get out of the same hardware.

Happy to share more of the pool math, the 4/3/2-bit layout, or the vLLM implementation if anyone wants to dig into it.

I've always gotten frustrated and wasted time explaining the same thing to an AI every time I start a new chat from an existing one or when I start another convo with a whole new AI model. That's why I built a tool that fixes that, it condenses everything in a chat into one simple .md file you can carry across different AI tools.

PS: Please contribute or give your feedback so that we can grow and make this community tool better.

https://github.com/legoambarish/portable-handoff

I went to bed with a fine-tune running. Woke up eight hours later to a loss that had been NaN since step 300 — the whole night of GPU time gone, and nothing had told me. If you fine-tune, you've lived some version of this: the run that trained at learning-rate-zero the entire time, or quietly ate a dataset with 40 broken rows, and you only find out at the end.

So I built a deterministic linter for training runs — trainproof (pip install trainproof, MIT) — and then I spent days trying to prove it wrong.

One Qwen2.5-3B QLoRA, run five times: once clean, four times with exactly one thing sabotaged. Four it caught instantly. One fooled it completely — and that one taught me the most.

It wasn't the NaN run. It wasn't the fp16 overflow. It wasn't the learning rate cranked 100× too high (that spiked the gradient norm to 2650× the median — caught in seconds).

It was shuffled labels. Pure garbage — a dataset that literally cannot be learned.

That run reduced its loss by 62%. On the curve it looked like textbook-healthy training. It was learning absolutely nothing — just memorizing the statistics of noise, which any network will happily do. From its own loss curve it is indistinguishable from a real run. No single-run, loss-only rule can catch it. So instead of pretending my tool is magic, I wrote that limitation straight into the README — and added a compare mode, because the failure is visible the moment you put it next to a known-good run.

That's the whole philosophy: no ML judging ML, no "87%-confidence" scores. Every check is a deterministic rule that either fires or doesn't, and every finding cites the exact numbers behind it. When it can't be sure, it says so instead of guessing.

Across a run's life:

• Before a single GPU-second — lints the dataset + tokenizer (malformed JSONL with the line number, empty rows, duplicates, missing eos_token, pad==eos, over-length samples). Exits non-zero → straight into CI.
• During training — one-line HuggingFace callback. Warns by default; flip on stop_on_fail and it aborts a doomed run itself. In testing it killed a diverging run at step 20 of 300 — 93% of the scheduled steps never ran.
• After — reads the log: diverged / flatlined / NaN'd / spiked.
• vs a baseline — the ratio rules that catch the shuffled-labels case.

All five sabotaged runs' real logs ship in the repo (examples/gallery/) with a 15-run / 3-seed evidence matrix — reproduce every verdict yourself. Reads plain logs (HF trainer_state.json, Coqui, JSONL/CSV), doesn't import torch.

Repo: https://github.com/Mormolykos/trainproof · pip install trainproof

Honest question: what's burned your GPU hours? If a deterministic check would've saved you, tell me — it goes in, with credit.

Ten days ago I posted a theory preprint here and got told, correctly, that it had no evidence behind it. So I built a method out of it and ran it on Terminal-Bench 3.0.

On a binary patching task where the public record shows 59 runs from 11 different model and agent setups and zero passes, one run using the method scored 19 of 19 on the official verifier, inside the original 90 minute limit.

Two ways to poke at this, and I'd genuinely like both.

The easy one: just run that task with whatever setup you already use. It's called ico-path-patch, it's public, 90 minute limit, 19 checks, all or nothing. 59 public runs from 11 different configurations, none passed. If your stack gets through it with none of my stuff involved, that's a much more interesting data point than anything I posted, and it kills my claim. Fine by me.

The harder one: take the method and go after the leaderboard with it. The idea is one line — before solving the task, have the agent build itself a small service for that task, then solve the task through the service. The method is the set of rules for what that service has to pin down. Everything else is your own agent, your own model, your own runs. If it works for you, the score is yours.

My runs took forty to ninety minutes each and cost a few dollars. Nothing in the setup is mine except the method text. Everything I ran is on the repo, including what failed and what I changed in between.

The task: https://hub.harborframework.com/tasks/terminal-bench/ico-path-patch/latest

The 60 trial rows behind that zero-pass baseline, with the query: https://github.com/amingclawdev/charting-loop/blob/main/public/results/ico-path-patch/job-009/PUBLIC-TRIALS.json

How to try the method:
https://github.com/amingclawdev/charting-loop/blob/main/docs/REPLICATION-INVITATION.md

The original preprint post : https://www.reddit.com/r/ResearchML/comments/1vjeznd/the_charting_loop_a_probabilistic_theory_of/

Disclosure up front: I built this. It's free, MIT, and shipped (npm: rungraph).

Claude Code and Codex CLI write full session transcripts to disk, and rungraph reconstructs them into interactive run graphs. The MCP server is the part this sub might find interesting: npx rungraph mcp --install gives your agent tools over its own history. list_runs, get_graph, find_nodes, get_detail, focus_nodes, get_current_view, open_visualization.

The design problem was context size. A real 176-node run is about 20k tokens as a full graph, 13.5k in the compact projection, and 1.1k through find_nodes. Narrowing beats projecting, so the tool descriptions steer agents to find_nodes first, then get_detail for one node's actual error text.

The fun tool is focus_nodes. You ask Claude in your own terminal "why did the Edit on token.js keep failing", it answers there, and the dashboard you have open lights up the exact nodes the answer is about, then returns a deep link that restores the same highlight against that dashboard (or a bundle the recipient has open). Honest limitation: with no dashboard watching, the call still succeeds and just reports that the highlight was skipped. The read tools parse straight from disk, so they work with no server running at all.

Implementation note for the protocol nerds: the JSON-RPC transport is hand-rolled over stdio because the package has zero runtime dependencies, which keeps the npx install tiny. If more than one dashboard is live (yours, plus a bundle someone sent you), list_runs merges them and every other tool routes by run id.

Live Demo: https://fayzan123.github.io/rungraph/

Repo: https://github.com/fayzan123/rungraph

If you wire it into a client other than Claude Code, I'd like to hear whether the tool descriptions hold up