Should every web page expose an AI-friendly JSON representation?
My website already includes AI-related files such as llms.txt.
I'm considering creating a separate JSON file for every page and article so AI systems can understand the content more easily and accurately. I would reference this JSON file from the page's <head> using a <link> tag.
The JSON file could contain information such as:
- Page URL
- Canonical URL
- Title
- Summary / Description
- Main Content (clean article content)
- Author
- Published Date
- Last Updated
- Entities (people, companies, places, products, etc.)
- Keywords / Topics
- FAQ
- ...
My idea is that AI crawlers could read this structured JSON instead of having to extract the main content from noisy HTML that contains navigation menus, sidebars, ads, comments, JavaScript, tables, and other non-essential elements.
I have two questions:
- Could this approach reduce the chances of AI crawlers misunderstanding a page or extracting incorrect information from HTML, advertisements, tables, comments, or other noisy content?
- Do you think a page-level JSON file like this could help AI systems better understand a page and potentially improve AI recommendations, citations, or other AI-generated responses in the future? Why or why not?
u/taylor-morgan2066 — 5 days ago