In one cross-app task, MCP retrieval took 21 calls. The equivalent filesystem stage took ~0.3 seconds.
▲ 4 r/mcp

In one cross-app task, MCP retrieval took 21 calls. The equivalent filesystem stage took ~0.3 seconds.

MCP gets several important things right, particularly standardized integrations, authentication and transactional actions.

But should agents also depend on runtime MCP calls to gather substantial context across applications?

We tested this across 20 scenarios using the same agent harness, model, prompts and machines:

  • Official Slack, Notion and Linear MCP integrations
  • The same permitted data synchronized and mounted as files

The filesystem implementation was Locality, which I work on.

The most revealing trace involved identifying product-launch risks across Slack, Linear, Notion and a Git repository.

The MCP agent gathered the evidence iteratively:

  • 21 MCP calls
  • Roughly 30 seconds inside tool calls
  • About one minute for the retrieval stage

The filesystem agent used parallel rg and file operations across the same sources. The equivalent stage took roughly 0.3 seconds.

Across 60 paired runs, the filesystem setup reduced LLM costs by 27% and end-to-end latency by 32%. Its answers were preferred in 70% of the blind comparisons.

Our takeaway is a separation of responsibilities:

  • MCP for actions
  • Filesystems for data and context

Locality keeps permitted application data synchronized and exposes it as files. The agent can then search, filter and combine context through one interface instead of traversing multiple application-specific tools during execution.

This isn’t necessarily an argument against MCP as a protocol. It is an argument against using runtime tool calls as the primary context-retrieval layer for broad, read-heavy work. Interestingly, MCP already supports file:// resources, but most integrations still expose context through tool calls rather than a filesystem-like resource layer.

The benchmark focused on cross-application research and synthesis rather than transactional actions.

Full methodology and traces

For people building MCP servers and agent infrastructure: does this separation match what you’re seeing - MCP for actions and another layer for context?

u/ml_guy1 — 8 hours ago
▲ 2 r/mlops

We benchmarked runtime MCP retrieval vs a mounted context data plane across 60 paired agent runs

Agent context retrieval is often treated as model behavior. In production, we found that it behaves more like a data-plane architecture decision.

We compared two approaches:

  • Retrieve Slack, Notion and Linear data through official MCP integrations during each agent run.
  • Pre-sync the permitted data and mount it into the agent sandbox as files.

The mounted implementation was Locality Cloud, which I work on.

The evaluation used 20 cross-application scenarios with three paired trials each. We ran six AWS t3.large instances, kept the agent harness, model, prompts and machines consistent, and performed 180 blind comparisons of the outputs.

Compared with runtime MCP retrieval, the mounted setup:

  • Produced the preferred answer in 70% of scenarios.
  • Reduced LLM costs by 27%.
  • Reduced end-to-end latency by 32%.
  • Required 61% fewer tool calls.
  • Used roughly 40% fewer tokens.

The traces suggest that the agents weren’t reasoning substantially faster. They were spending less time traversing application data.

In one scenario, the agent had to reconcile product launch risks across Slack, Linear, Notion and a Git repository. One evidence-gathering stage took roughly 0.3 seconds using parallel filesystem operations. The MCP setup spent about a minute on the same stage, making 21 calls with approximately 30 seconds of tool-call time.

The broader MLOps lesson for us is that mounted file system context isn’t simply a cache. It becomes a production data plane with its own requirements:

  • Freshness: changes need to arrive through webhooks, polling or a pre-run synchronization boundary. Staleness must be observable.
  • Permissions: each sandbox should receive only the sources and subtrees required for that run, without broad application credentials.
  • State: remote state, mounted state and the last synchronized state must be tracked separately so pulls, writes and conflicts are unambiguous.
  • Write review: agent edits should produce an inspectable operation plan before they are synchronized back to the source.
  • Recovery: interrupted writes need journaling, idempotency and explicit conflict handling rather than silent retries.

This architecture also creates new operational costs: connector maintenance, synchronization lag, storage, conflict resolution and recovery testing.

We still expect live APIs or MCP to be preferable for transactional actions, narrow lookups and data that cannot tolerate synchronization delay. The emerging pattern looks less like “files instead of MCP” and more like two planes:

  • A mounted context plane for broad, read-heavy discovery and synthesis.
  • A live action plane for transactional operations.

Locality Cloud is our managed implementation of the mounted context plane, with an on-premises option for organizations that need to keep the synchronization layer inside their environment.

Full details with analysis, traces and scenario-level results:

https://www.locality.dev/blog/locality-why-filesystems-perform-better-than-mcps-for-production-agents

How are teams operating production agents separating their context plane from their action plane? If you materialize application data before execution, how do you handle freshness, permissions and failed synchronization?

u/ml_guy1 — 1 day ago

We benchmarked MCP vs filesystem access across 20 production-agent scenarios. The filesystem setup cut LLM costs by 27% and latency by 32%

We gave the same agent 20 cross-application tasks using two different setups:

  • Official Slack, Notion and Linear MCP integrations.
  • The same application data synchronized and mounted as files using Locality, which I work on.

We kept the agent harness, model, prompts and machines the same. We ran each scenario three times and conducted 180 blind, randomized comparisons of the resulting answers.

Compared with MCP, the filesystem setup:

  • Produced higher quality answer in 70% of the blind evaluations.
  • Reduced LLM costs by 27%.
  • Reduced latency by 32%.
  • Required 61% fewer tool calls.
  • Used roughly 40% fewer tokens.

The traces suggest that most of these gains came from gathering context, not from differences in reasoning.

In one scenario, the agent had to identify product-launch risks by comparing evidence across Slack, Linear, Notion and a Git repository.

The filesystem agent searched across those sources using a small set of parallel rg and file operations. During one evidence-gathering stage, those operations took roughly 0.3 seconds.

The MCP agent spent about a minute on the same stage, making 21 calls with roughly 30 seconds of tool-call time as it iteratively gathered context.

The agents spent similar amounts of time reasoning. The main difference was how they found and inspected the evidence needed to reason.

Our takeaway is that a filesystem gives agents one composable interface for searching, filtering and reading across sources. This enables ready discovery of context which can be retrieved at scale. MCP gives them multiple application-specific interfaces, which can create longer retrieval chains for context-heavy work.

This benchmark focused on cross-application research and synthesis, so it doesn’t cover every MCP use case. MCP may still be the better interface for individual actions and lightweight integrations.

Full analysis, traces and scenario-level results linked below.

If you’re running agents in production, do you fetch application context through tools at runtime or synchronize it into the environment beforehand? Where have you seen either approach break down?

reddit.com
u/ml_guy1 — 1 day ago
▲ 20 r/Notion

I am building Locality that mounts Notion as local files and folders for easy use by agents.

I've always wished I could work on my Notion pages like local files.

Edit notion pages with my editor, search for pages locally, let Claude/Codex help across multiple documents, review changes locally, then sync them back.

So I ended up building Locality.dev.

It mounts your Notion workspace as local folders and Markdown files while keeping Notion as the source of truth. That means your editor, scripts, and AI coding agents can read and work with your entire workspace using normal file operations instead of custom integrations.

I'd genuinely love feedback from people who use Notion every day. What would you use something like this for?

u/ml_guy1 — 1 month ago