seeking scrubbed telemetry to improve open source local agentic performance.
Hi all. A couple days ago a new research paper came out on δ-mem adaptors. I have been deep in it applying it to my local agent’s memory system.
I converted the research project to MLX (I run on apple silicon) and made it work with qwen3.5:9b GatedDeltaNet was NOT easy but I think I have it figured out. I haven’t pushed my adaptor trainer yet, but I will once it’s perfectly tuned. I have been benchmarking it for agentic use and results have been very promising, based on the research papers adaptor and passing good attention data, it improves similar questions / answers and summarizations quality by 30%.
Basically, it allows a local model to perform way better.. but there’s a hole that needs to be filled before this is useful.
To make this usable for local agentic use, an adaptor needs to be trained on high quality curated sessions with tool use.
So I’m looking for volunteers that have telemetry data on, and are willing to contribute scrubbed sessions for training. I only have about 100 sessions since I turned on telemetry, for v1 of a trained adaptor I want about 500 more. For a final version, 2400 more sessions at least would be ideal.
I added my telemetry-to-useful-training-data repo to GitHub, here’s a prompt you can give an agent to scrub your own data:
https://github.com/elimaine/agent-memory-forge/blob/main/prompts/telemetry_scrubber_handoff.md
or you can use my toolbelt in the same repo. Scrubbing can be done by any llm that can handle the context length, so if you have a local model you can use that. I used gpt5.4. Though I’d prefer full scrubbed telemetry in cases i find more optimal ways to do formatting/goal awareness.
Send me a link to the scrubbed data and I’ll add it to training.
What you get: I’ll upload an agentic tool use specific delta men adaptor to huggingface as well as the sidecar mlx delta mem repo so anyone can run it. As well as benchmarks so you know what improvement to expect.
Let me know in your message if it’s okay to share your sessions as the training data. If you don’t specify, I will treat it as private. Also let me know if you are using PyTorch/CuDA, if nobody contributing requests it I’ll probably just upload the MLX version with a diy conversion tool.
This will allow you to use the local models and get 8% better answers just for free, with up to 30% if you pass in more relevant context. You can just pass in past relevant sessions or compacted away session history without increasing your context window!
I am first targeting qwen3.5:9b, with my sights set on larger quantized models (27-32b) once everything is stable.
Thanks for reading yall.