u/Ok_Row9465

Quick disclosure up front: I work on Synap, an AI memory system, so factor that into the self-promotion angle. Posting because the argument itself seemed useful to this community, not just to point at the paper.

https://preview.redd.it/er568xctu6fh1.png?width=1310&format=png&auto=webp&s=9413b7de0caa9b00c60b74c4c38e103e9ea69920

https://preview.redd.it/ecgt8vctu6fh1.png?width=1308&format=png&auto=webp&s=a021eb431faf1a8c31190c5e300140c341a2f324

https://preview.redd.it/u9umpsctu6fh1.png?width=1308&format=png&auto=webp&s=6a24b56d2f24d91b7f67519dea7f5bcaa166929c

The core argument from my end is that while most teams treat agent memory and inference cost as something a bigger context window or a cheaper model or a better retriever will eventually fix; I argue that in practice; most of this cost and most of the failure modes are architectural viz. what an agent remembers, when it forgets, and how much it costs to reason are decisions made before any model is involved and not afterwards.

The paper works through this with an evaluation harness and the underlying study data included, rather than just asserting it. 23 pages, 6 figures, 4 tables.

Link: arxiv.org/abs/2607.21503

Curious to hear your views, points you agree with, disagreements, etc.

P.S.: We have upfront stated that we will only share indicative architecture and details; since we are sharing the principle of what we believe works and why; and go into the reference implementation to some extent; while keeping it IP Safe for our organization maximem.ai

Recently, I had the chance to speak at length with the CIO of a large enterprise (obviously can't share the identity), around their thoughts on semantic layers, ontologies and agentic systems. They are fairly active in the CIO circles and have been engaging with their peers on the topic. Notes below are a mix from both our observations.

Some obvious observations first:

1. Large enterprises are disproportionately focussed on building internal agents (rather than customer-facing ones), with the focus on reducing talent costs and they are already realizing that the infra for it is far from ready
2. Enterprises are understanding the pain and the need for context management but they don't have the right terminology for it yet
3. Most enterprises are pointing agents at fragmented internal systems and hoping the model infers business meaning across them which obviously breaks quickly in production.

A few interesting aspects that emerged:

1. Static ontologies are dead on arrival. The real world environment changes daily but the semantic model updates once a quarter and hence the system is stale before it ships. Even human organizations get redesigned every few years because reality moves. An intelligent system should be able to reorganize its internal understanding far more often than that. The better analogy is cognition, not schema design: continuous consolidation, continuous re-linking, continuous updating of what matters.

2. The bottleneck is not data access, it is context selection. The real question is rarely "how do I retrieve more information." It is what context is right for this decision, what should be ignored and how fast that can be assembled at the speed the task demands. A person making a judgment call is not querying a giant flat database. They are drawing on a compressed, evolving, relevance-weighted internal model and that is much closer to the actual design problem.

3. Enterprise semantics gets misread in two opposite directions. Some people flatten it to metadata and catalog descriptions. Others make it so abstract it cannot be operationalized. The real need sits in between: technical enough to run in production, dynamic enough to evolve with the business and grounded enough to encode institutional meaning without collapsing under latency, security and ownership constraints.

4. Vendor semantics is not organizational semantics. Every major platform is now shipping its own semantic layer, but a company's core institutional knowledge cannot be fully outsourced to whichever vendor has the best UI this quarter. Meaning scattered across product surfaces owned by different vendors gets you local optimizations but never a coherent institutional model. This might be one of the more unresolved problems in enterprise AI right now.

5. The hard part is representing judgment, not just knowledge. Most valuable work inside a company is not a deterministic logic tree. People get hired for how they interpret incomplete information and make calls under ambiguity, not just for what they know. So the real question is not how to build a company knowledge base. It is how to build systems that inherit evolving decision context, not just stored facts.

One more thing, the same need gets called an ontology, a knowledge graph, a semantic layer, a context graph, a company brain, agent memory or institutional memory, sometimes all in one conversation. That pattern usually means the need is ahead of the label.

My rough takeaway: we may be underrating how much "intelligence at work" depends on continuously evolving context, not model quality or data availability alone. The next real layer probably is not another copilot or orchestration framework. It is whatever can unify fragmented meaning, keep it current, and make it queryable at decision speed without collapsing under latency, trust, or governance constraints.

Genuinely curious how people here see it: are semantic layers and context graphs the actual missing layer for enterprise agents or is this still too early, too abstract, or too category-confused to matter yet?

Recently, I had the chance to speak at length with the CIO of a large enterprise (obviously can't share the identity), around their thoughts on semantic layers, ontologies and agentic systems. They are fairly active in the CIO circles and have been engaging with their peers on the topic. Notes below are a mix from both our observations.

Some obvious observations first:

1. Large enterprises are disproportionately focussed on building internal agents (rather than customer-facing ones), with the focus on reducing talent costs and they are already realizing that the infra for it is far from ready
2. Enterprises are understanding the pain and the need for context management but they don't have the right terminology for it yet
3. Most enterprises are pointing agents at fragmented internal systems and hoping the model infers business meaning across them which obviously breaks quickly in production.

A few interesting aspects that emerged:

1. Static ontologies are dead on arrival. The real world environment changes daily but the semantic model updates once a quarter and hence the system is stale before it ships. Even human organizations get redesigned every few years because reality moves. An intelligent system should be able to reorganize its internal understanding far more often than that. The better analogy is cognition, not schema design: continuous consolidation, continuous re-linking, continuous updating of what matters.

2. The bottleneck is not data access, it is context selection. The real question is rarely "how do I retrieve more information." It is what context is right for this decision, what should be ignored and how fast that can be assembled at the speed the task demands. A person making a judgment call is not querying a giant flat database. They are drawing on a compressed, evolving, relevance-weighted internal model and that is much closer to the actual design problem.

3. Enterprise semantics gets misread in two opposite directions. Some people flatten it to metadata and catalog descriptions. Others make it so abstract it cannot be operationalized. The real need sits in between: technical enough to run in production, dynamic enough to evolve with the business and grounded enough to encode institutional meaning without collapsing under latency, security and ownership constraints.

4. Vendor semantics is not organizational semantics. Every major platform is now shipping its own semantic layer, but a company's core institutional knowledge cannot be fully outsourced to whichever vendor has the best UI this quarter. Meaning scattered across product surfaces owned by different vendors gets you local optimizations but never a coherent institutional model. This might be one of the more unresolved problems in enterprise AI right now.

5. The hard part is representing judgment, not just knowledge. Most valuable work inside a company is not a deterministic logic tree. People get hired for how they interpret incomplete information and make calls under ambiguity, not just for what they know. So the real question is not how to build a company knowledge base. It is how to build systems that inherit evolving decision context, not just stored facts.

One more thing, the same need gets called an ontology, a knowledge graph, a semantic layer, a context graph, a company brain, agent memory or institutional memory, sometimes all in one conversation. That pattern usually means the need is ahead of the label.

My rough takeaway: we may be underrating how much "intelligence at work" depends on continuously evolving context, not model quality or data availability alone. The next real layer probably is not another copilot or orchestration framework. It is whatever can unify fragmented meaning, keep it current, and make it queryable at decision speed without collapsing under latency, trust, or governance constraints.

Genuinely curious how people here see it: are semantic layers and context graphs the actual missing layer for enterprise agents or is this still too early, too abstract, or too category-confused to matter yet?

Mem0 publishes 93.4% on LongMemEval as their state-of-the-art overall score. When we ran their hosted product through a clean evaluation harness (gpt-5 answerer, binary judge with no lean-toward-yes instruction, 5-seed mean), the best we could get was 73.8%. A 19.6-point gap on the same memory system and the same data.

We dug further, the gap is in their public benchmark harness. Reading their prompts.py file at the commit they shipped right before their April 14 announcement (commit bd063eea, April 3, 2026):

1. Dataset-specific equivalence rules in the answer prompt.

https://preview.redd.it/va27d4jzvw8h1.png?width=3024&format=png&auto=webp&s=2d835fafc5a1583cef7fed3c6343b405d4b37dad

Lines 138 to 148 contain 14 rules that map 1-to-1 to specific public LongMemEval question_ids. A sample, verbatim:

>

The point of LongMemEval is that the system has to figure out when "scratch grains" should count as "layer feed." Hardcoding the equivalence into the answer prompt skips the reasoning step.

2. The dataset hints get applied inside a hidden chain-of-thought block.

https://preview.redd.it/szk9ka57ww8h1.png?width=2940&format=png&auto=webp&s=56e04fb8e44dd8b4707a7062f8d07d116a86c58a

Line 53: Before answering, reason step-by-step inside <mem_thinking> tags:
Line 65: The user will only see text outside the <mem_thinking> tags.

The judge only sees the final cleaned answer. The dataset-keyed reasoning is invisible to anyone sampling outputs.

3. The judge is explicitly told to default to "yes."

https://preview.redd.it/xroeatxaww8h1.png?width=3006&format=png&auto=webp&s=126fad0f35c618e523a1eef3a864a76870c85fbc

Line 269 of the same file: IMPORTANT BIAS CHECK: You have a tendency to say "no" too quickly. Before concluding "no", you MUST verify the answer is truly wrong, not just differently worded. When in doubt, lean toward "yes".

Lines 328 to 334 add a 5-step gauntlet to clear before marking anything WRONG. No comparable gauntlet exists before marking anything CORRECT.

4. Bonus finding in their LoCoMo judge.

https://preview.redd.it/67yu69beww8h1.png?width=3024&format=png&auto=webp&s=93159462881bb10e168473dea546895099c25dfb

Different file, same repo, commit edcd6f1d (April 9, 2026). Line 212 of benchmarks/locomo/prompts.py:

>

Read the last clause carefully. Evidence can promote a WRONG prediction to CORRECT. The same evidence cannot demote a CORRECT prediction to WRONG. A one-directional score lift, written into the judge by hand.

Mem0 named this mechanism in their own commit messages. The April 3 commit message reads: "Sync prompts from evals: CONTEXT CHECK, Rule 14 (contradictions), conflicting numbers, personalization scan, BIAS CHECK in judge, chain-of-thought <judge_thinking> tags, 5-step FINAL CHECK." Their engineer typed the words "BIAS CHECK in judge" and "5-step FINAL CHECK" into git, on April 3, eleven days before the announcement of new SOTA numbers.

Verify in 2 minutes (direct GitHub permalinks at the pinned commits):

• L145 chandelier: github.com/mem0ai/memory-benchmarks/blob/bd063eea04de4f8a19927beea155afa094a01905/benchmarks/longmemeval/prompts.py#L145
• L269 BIAS CHECK: same file, #L269
• L212 LoCoMo override: github.com/mem0ai/memory-benchmarks/blob/edcd6f1d42400837b1fcb6997716f1769dc51a37/benchmarks/locomo/prompts.py#L212
• April 3 commit message: github.com/mem0ai/memory-benchmarks/commit/bd063eea04de4f8a19927beea155afa094a01905

I tried meeting with their founder and communicating the issue; since the past 2-3 weeks, but we couldn't and I thought that it might be time for the community to learn about it.

Full-disclosure: I am the founder of Maximem.ai - another Agentic Memory and Context Management company. This is not an attempt to malign, but to put their latest numbers into perspective.

Agents doing serious work with serious volume of conversations (either human-agent or agent-agent) need serious memory and context-management with high accuracy and I know most developers are still using crude summarization as well as RAG for short-term and long-term memory management. These easily break.

I have been working on a memory and agentic context stack that takes care of the harder parts: automatic entity resolution, temporal sensitivity (knowing which fact is current when two conflict), memory scoping across users and agents and conscious forgetting. It scores very highly on benchmarks and is pretty fast as well.

I want to sit down with developers and walk through the product have them wire it into something they are building and try to break it. Online works too if you are not local (I am in SF). No pitch, no pressure. I want the feedback, including the parts where it falls over.

If you build agents and have run into the memory problems, comment or DM me and I will send a time. Happy to answer technical questions here too.

I used to post once a while and time it well. Each post used to get 5K+ views. Then I bought into popular advice and started posting 4-5x a week. Now combined weekly views are around 5K. Is this because I am still getting seen mostly by my former colleagues and not the ICP? Does this genuinely change with time (its just been 1 week).

I can DM you the link. Would be great to get feedback and questions before judges (in next 60 mins)

I was at Vector Space Day on Thursday in SF organized by Qdrant. Sharing some notes.

No one wants to be Vector DB. Qdrant team mentioned on stage that they want to get rid of the database positioning and be known as a search engine. Turbopuffer already describes itself as a search engine built on object storage. I believe storing vectors is a commodity now and the differentiation is in query-time behavior (hybrid retrieval, scoring control, execution configurability, etc.).

HubSpot stores 20B+ vectors on self-hosted Qdrant. They built an internal "Vectors as a Service" platform with Kafka indexers in front of the clusters. They even wrote their own Kubernetes operator because Helm is purely a templating tool: it cannot make API calls, react to metrics or rebalance shards based on cluster state. Their operator runs a reconcile loop every 60 seconds. If you are weighing managed vs self-hosted, this is the real cost of self-hosting at scale.

Quantization numbers were shown per embedding model. float32 vs scalar vs more aggressive schemes and the recall degradation is not uniform across embedding families. Some degrade gracefully, some do not. Benchmark on your own data before committing. (couldn't grab snaps of the slides).

An oncology research company (Oncotelic) presented "manifold folding". They re-shape the embedding space with metric learning so strong and weak biomedical evidence separate into different regions before indexing because nearest-neighbor on the raw space was mixing evidence quality for them.

Salesforce search still runs on Solr. Met someone at the event and learnt this. Apparently Solr's indexing is really powerful and a migration at their scale is not realistic or at least not high-ROI right now.

My takeaway, with a disclosure that I work on agent memory so I am biased: vector retrieval gives you similarity and nothing else. No provenance, no document creation time, no organization, no protection against poisoned context. For agents, that gap is where most of the unsolved work sits.

Curious what others running large vector deployments think, especially anyone who has hit the Helm wall or has real quantization recall numbers to compare.

1. **Too good is bad**

Indian callers have spent years talking to contact centre agents who cough, lose their place, ask you to repeat yourself, background noise of other agents. That experience has calibrated what a real voice sounds like. ElevenLabs Flash, ElevenLabs v3 are too smooth, too clean. Callers register it as wrong before they can explain why. There is a cottage industry of builders curating voice libraries specifically to introduce roughness.

2. **Latency benchmarks vs production latency**

Sub-500ms is claimed constantly but achieved rarely and are often measured on a bare prompt with no system context, no retrieval, no tool calls in the loop. Indian telephony infrastructure adds 200-300ms as a fixed floor. benchmark below that needs to be read carefully.

3. **Multi-agent handoff is messier than it looks.**

Two things fail usually: a) monolithic single-agent prompt with everything stuffed in which are fragile and break at scale. b) clean sequential handoff except the tool call has to render the full prior message before the next agent starts and that gap is noticeable on a phone call. The workaround that is often employed is streaming a pre-recorded filler phrase the moment a handoff triggers, buys 2-3 seconds.

4. **A problem that has no name yet.**

Context and memory management features frequently in these conversations but is not spoken as-is. Manifests as either LLM cost issues (for example: Gemini Live has no prompt caching so a session starting at 10k tokens hits 100k+ by turn 10 and model behaviour gets unpredictable past ~120-174k), latency threads (RAG adding 200-400ms while also being the only path to accurate answers) and prompt engineering threads (10k-50k token prompts as a maintenance liability).

Every team is solving one piece from scratch. Nobody has the layer underneath.

5. **Indic language situation.**

Hinglish i.e. Hindi-English code-switching is solved, is true for the metro-city, educated, urban India. Regional dialects, Tamil-English patterns, Marathi-Hindi, rural India; these are not well-defined problems and the real problems that production deployments keep running into.

Happy to share the full piece here, if folks are interested.