u/Forminator

SF 24.0 came out yesterday (May 19) with a built-in MCP server. No third-party setup, no Python wrapper — it's baked directly into the application. I spent today running it through real audit workflows. Here's what I found.

Setup (~10 minutes, Windows)

Configuration > Settings > MCP Server

Set a port (default 11435), point it at a working directory, enable the Node.js runtime. The warning is legitimate — this lets Claude write and execute JS on your machine. Only enable it on a machine you fully control.

Hit "Start MCP Server" until you see "Valid Configuration".

Then add this to your claude_desktop_config.json (C:\Users\YourName\AppData\Roaming\Claude\claude_desktop_config.json):

{
  "mcpServers": {
    "screaming-frog": {
      "command": "node",
      "args": ["C:\\Users\\YourName\\seo_spider_mcp_server\\index.js"]
    }
  }
}

Restart Claude Desktop from the system tray (not just the window). Done.

The 8 tools you get

sf_check — verify installation and licence
crawl_site — start a headless crawl, no GUI needed
crawl_status — poll crawl progress
list_crawls — all saved crawls with DB IDs
export_crawl — any tab, bulk export, or report
read_crawl_data — read exports with filtering + pagination
delete_crawl — cleanup
storage_summary — disk usage

export_crawl covers everything: Internal:All, Response Codes:All, H1:All, Hreflang:All, Canonicals:All, Structured Data:All, Security:All, plus all bulk exports (All Inlinks, All Redirect Chains etc.) and summary reports.

Two ways to work

GUI crawl + MCP analysis — configure in SF GUI (JS rendering, custom extraction, GA4/GSC), run the crawl, close the GUI, hand all analysis to Claude. This is what I use for real client audits.

Full headless — crawl_site → poll crawl_status → export → analyse entirely through Claude. Works well for simple crawls and automated monitoring.

Note: JS rendering, custom extraction, and API integrations still require GUI setup first. The MCP can't configure those programmatically yet.

What's actually useful in practice

The workflow that used to be crawl → export CSV → filter in Excel → write up findings now runs in a single prompt. "Export Response Codes:All and All Inlinks, show me every 404 with the pages linking to it sorted by inlink count" just works.

Same for hreflang audits, redirect chain analysis, on-page checks. The analysis side is where this saves real time.

Limitations worth knowing

Full paid licence required
Default max response size is 100,000 bytes — on large crawls you'll hit this, increase it in the MCP settings or work in filtered batches
Crawl Analysis (enables Link Score, orphan detection, near-duplicate checking) needs to have run before those data points are available
Not every SF feature is supported yet — they've said they'll extend functionality over time

One thing worth doing if you use this regularly

Build a skill file — a Markdown reference document Claude reads at the start of relevant conversations. Eliminates re-explaining what SF can export and what filters exist every session. Mine covers all export syntax, pre-written prompts for common audit types, and issue definitions. Makes a real difference over time.

One thing I noticed during testing that I haven't seen documented anywhere: after a crawl finishes there seems to be a state where exports fail with an IOException until you run sf_clear_crawl → sf_list_crawls → sf_load_crawl. Not 100% sure if this is expected behaviour or something specific to my setup — mentioning it in case anyone else hits it.

Tested on Windows, SF 24.0, Claude Desktop. Happy to answer questions.

Screaming Frog 24.0 dropped yesterday with a native MCP — tested it today, here's what actually works