I was reading the new PageIndex blog today and they just announced something called the PageIndex File System. If you haven't heard of PageIndex, it's the vectorless RAG framework that doesn't use embeddings at all. Instead of chunking docs and doing semantic similarity search, it represents each doc as a tree (sections → subsections → pages → content) and has an LLM navigate the tree to find answers. Repo is at like 26k stars, hit #1 on GitHub Trending earlier this year.

The criticism that always made sense to me was: ok but that only works on one document at a time, how does this scale to a real enterprise corpus with millions of docs? And the cost concern that came with it — if an LLM is navigating a tree on every query, doesn't that blow up?

Their answer starts with an observation I think is genuinely elegant: a file system is already a tree. Folders → subfolders → files. So they just made the folder hierarchy another layer of the same tree the LLM already knows how to navigate. One continuous tree from the top of your drive down into the internal structure of a specific document.

But the post is honest about why that alone doesn't actually work, which is the part I found interesting. Three problems with just inheriting your folder structure:

1. Tons of corpora have no real hierarchy — flat S3 buckets, SharePoint dumps, document management systems where everything is in one pool
2. A folder tree is one-dimensional — a contract belongs to a vendor AND a region AND a fiscal year AND a product line, but a folder forces you to pick one
3. Folder labels are often garbage (misc/, final_v3_USE_THIS_ONE/, 2019_legacy/) so the LLM ends up navigating noise

So they solve it with three things, and this is where the query-time strategy comes in:

Virtual nodes — when no usable hierarchy exists, they synthesize one. Topic clustering groups documents into nodes, and LLM-inferred metadata (category, summary, key entities) becomes additional internal nodes. The same document can sit under multiple virtual ancestors at once, which a real folder tree fundamentally can't express.

Query-dependent tree construction — this is the part that genuinely changes how I think about retrieval. The tree isn't fixed at ingestion. It's built on demand, per query. The example they use: "What did vendor X charge us in 2024?" wants a tree organized by vendor → year. "Show me all contracts up for renewal next quarter" wants a tree organized by status → renewal date. Same corpus, completely different tree depending on what you're asking. No re-ingestion, no re-embedding — the structure gets composed at query time from the metadata axes that are actually relevant. They also mention the system improves over time because traversal patterns from past queries refine the virtual nodes.

Adaptive tree search (this is where the cost concern dies) — the LLM doesn't blindly walk every level. At each node, it picks a strategy. If the children have informative labels, it goes layer-by-layer and prunes early. If the labels are uninformative, it does what they call dynamic flattening — collapses the entire subtree down to the leaves and just defers to the actual content. Useless intermediate levels get skipped entirely, so the LLM only burns calls where the structure is actually carrying signal. The depth of the search shrinks to the depth that's actually informative for that specific question.

That last piece is what makes the cost story actually work at million-doc scale. You're not paying for an LLM to navigate every node of a giant tree — you're paying for it to navigate exactly the parts that are useful for this query.

What do you think of their approach?

I was reading the new PageIndex blog today and they just announced something called the PageIndex File System. If you haven't heard of PageIndex, it's the vectorless RAG framework that doesn't use embeddings at all. Instead of chunking docs and doing semantic similarity search, it represents each doc as a tree (sections → subsections → pages → content) and has an LLM navigate the tree to find answers. Repo is at like 26k stars, hit #1 on GitHub Trending earlier this year.

The criticism that always made sense to me was: ok but that only works on one document at a time, how does this scale to a real enterprise corpus with millions of docs? And the cost concern that came with it — if an LLM is navigating a tree on every query, doesn't that blow up?

Their answer starts with an observation I think is genuinely elegant: a file system is already a tree. Folders → subfolders → files. So they just made the folder hierarchy another layer of the same tree the LLM already knows how to navigate. One continuous tree from the top of your drive down into the internal structure of a specific document.

But the post is honest about why that alone doesn't actually work, which is the part I found interesting. Three problems with just inheriting your folder structure:

1. Tons of corpora have no real hierarchy — flat S3 buckets, SharePoint dumps, document management systems where everything is in one pool
2. A folder tree is one-dimensional — a contract belongs to a vendor AND a region AND a fiscal year AND a product line, but a folder forces you to pick one
3. Folder labels are often garbage (misc/, final_v3_USE_THIS_ONE/, 2019_legacy/) so the LLM ends up navigating noise

So they solve it with three things, and this is where the query-time strategy comes in:

Virtual nodes — when no usable hierarchy exists, they synthesize one. Topic clustering groups documents into nodes, and LLM-inferred metadata (category, summary, key entities) becomes additional internal nodes. The same document can sit under multiple virtual ancestors at once, which a real folder tree fundamentally can't express.

Query-dependent tree construction — this is the part that genuinely changes how I think about retrieval. The tree isn't fixed at ingestion. It's built on demand, per query. The example they use: "What did vendor X charge us in 2024?" wants a tree organized by vendor → year. "Show me all contracts up for renewal next quarter" wants a tree organized by status → renewal date. Same corpus, completely different tree depending on what you're asking. No re-ingestion, no re-embedding — the structure gets composed at query time from the metadata axes that are actually relevant. They also mention the system improves over time because traversal patterns from past queries refine the virtual nodes.

Adaptive tree search (this is where the cost concern dies) — the LLM doesn't blindly walk every level. At each node, it picks a strategy. If the children have informative labels, it goes layer-by-layer and prunes early. If the labels are uninformative, it does what they call dynamic flattening — collapses the entire subtree down to the leaves and just defers to the actual content. Useless intermediate levels get skipped entirely, so the LLM only burns calls where the structure is actually carrying signal. The depth of the search shrinks to the depth that's actually informative for that specific question.

That last piece is what makes the cost story actually work at million-doc scale. You're not paying for an LLM to navigate every node of a giant tree — you're paying for it to navigate exactly the parts that are useful for this query.

What do you think of their approach?