
How far can you automate building a web test suite from a URL?
I wanted to see how far you can get toward generating a useful web test suite if you start with nothing but a URL.
So I built an experiment around it.
The crawler uses a real browser to observe the site, then tries to infer things like page families, navigation patterns, menus, controls and URL patterns. From that it builds an intermediate model and generates executable tests.
One thing I specifically wanted to avoid was the usual “crawl every URL and generate a test for every page” approach. A site with 500 URLs doesn't necessarily have 500 different kinds of pages.
I also wanted the generated tests to be verifiable. The generated locators are checked against the pages they came from, and the generator has tests that deliberately corrupt generated output to see whether the semantic checks catch it.
For a real-world experiment, I ran it against atos.net.
The resulting project is here if anyone wants to poke around:
https://github.com/ghost-in-the-kernel/atos-net
It ended up with 34 inferred page types, along with smoke/navigation/menu/broken-link tests, page objects, crawl data, saved pages and the other artifacts produced during the run.
The actual generator is here:
https://github.com/ghost-in-the-kernel/suite-builder
I'm curious where people think the useful boundary is.
How much of a web application's structural/regression testing should be discoverable automatically, and where does human-written business intent need to take over?
I'm particularly interested in hearing from people who've maintained large Selenium/Playwright suites. What parts of the suite do you think are mostly mechanical enough to generate, and what parts really aren't?