u/gimalay

I've been giving my coding agent access to a folder of markdown files as its long-term memory. It works surprisingly well for open-ended questions — "why did we choose Postgres over DynamoDB?" or "what's the context behind the auth rewrite?" The agent finds the right document, reads it, gives a solid answer.

Then my teammate asked: "Which of our API decisions are still in draft status?"

The agent read through every decision document. It took 40 seconds. It missed two because the word "draft" didn't appear in the body — I'd just never gotten around to finishing them. It hallucinated one as "draft" because the text said "this approach is still a draft idea" in a different context.

The failure mode was obvious once I saw it: I was asking a structured question against unstructured data. The agent had to parse natural language to extract what was essentially a database query. Of course it got it wrong.

The fix was adding YAML frontmatter to every document:

---
title: "Use Postgres for the event store"
type: decision
status: accepted
domain: infrastructure
created: 2026-01-15
---

Now every document carries its own metadata as machine-readable fields — not buried in prose where the agent has to guess. Status, type, domain, dates, relationships — all queryable.

The query that previously took 40 seconds and got it wrong:

iwe find --filter 'status: draft' --project title,domain,created -f json

Instant. Correct. No token cost.

Once I started modeling metadata this way, a whole class of questions that used to require the agent to "think" became trivial lookups:

iwe find --filter '{type: decision, domain: infrastructure}' --project title,status -f json

iwe count --filter 'status: draft'

iwe find --filter '{status: published, created: { $gte: "2026-04-01" }}' \
  --sort created:-1 --project title,domain -f json

The pattern that emerged: there are two kinds of questions you ask a knowledge base.

Navigational questions — "tell me about X" — where you want the agent to read documents and synthesize an answer. Full-text retrieval works fine for these. The content matters.

Structured questions — "how many X are in state Y" — where the answer is a filter, a count, or a sort. These should never touch the LLM at all. They're database queries. If your knowledge base can't answer them without reading every document, you're missing a layer.

Frontmatter is that layer. It turns each document into a row with typed columns, while keeping the body as freeform prose for the navigational questions. The agent uses CLI queries for structured questions and document retrieval for everything else.

The tradeoffs:

• You have to define a schema and maintain it. If you're sloppy about filling in frontmatter, the queries return garbage. Garbage in, garbage out.
• There's upfront work to retrofit existing documents. But here's where fast, cheap models shine — I pointed a fast, cheap model at each document with a simple prompt: "read this document and extract these fields: type, status, domain, created date. Return YAML." It costs almost nothing per document and it's surprisingly accurate for structured extraction. I ran it over my whole KB in under a minute for a few cents. The fast models aren't great at reasoning over your whole knowledge base, but they're perfect at reading one document and pulling out metadata. I spot-checked maybe 10% and fixed a handful of errors. Way faster than tagging everything by hand.
• You need a tool that can query frontmatter. I use IWE which has a CLI with filter, projection, and sort — but you could build something similar with any YAML parser and a bit of scripting.

Here's the workflow that actually made this practical:

Design the schema with a smart model. I sat down with a capable model and described my knowledge base — what kinds of documents I have, what questions I want to ask, what dimensions matter. In about ten minutes of back and forth, we landed on a schema: type, status, domain, priority, created date. The smart model is good at this — it asks "do you ever need to filter by X?" and you realize yes, you do. You wouldn't think of half the fields on your own.

Deploy a swarm of fast agents to populate it. Once the schema is locked, you don't need a smart model to fill it in. I pointed a fast model at every document — one doc per call, same prompt: "read this and extract these fields as YAML frontmatter." Under a minute, a few cents total. Fast models are perfect for structured extraction from a single document. They don't need to reason across your whole knowledge base — they just need to read one file and pull out values. I spot-checked maybe 10% and fixed a handful of errors.

Start querying. Now the questions that used to require the agent to read everything and guess become precise, instant lookups:

iwe count --filter 'status: draft'

iwe find --filter '{status: accepted, domain: infrastructure}' \
  --project title,priority,created --sort priority:-1 -f json

iwe find --filter '{priority: { $gte: 3 }, status: draft}' \
  --project title,domain --sort created:-1 -f json

Counts, filters, sorts, projections — all against frontmatter fields, no tokens burned reading document bodies.

The thing I didn't expect: the agent started maintaining the schema better than I did. I give it a system prompt instruction — when you create a new document, always include frontmatter with these fields. It's more consistent about it than I am. And auditing for gaps is just another query:

iwe find --filter '{type: decision, domain: null}'
iwe find --filter '{type: decision, priority: null}'

No reading. No guessing. Just: which documents am I forgetting to tag?

The meta-realization: the expensive model designs the schema, the cheap models populate it, and after that most structured questions don't need an LLM at all — they're just queries. You're paying for intelligence exactly where it matters and using deterministic lookups everywhere else.

Curious if others have landed on a similar split, or if you're handling structured questions differently.