Mapping the emerging landscape of UX research for AI-built software
Update, June 30: Thanks for the comments here. I’ve tightened the map to make the calibration boundary more explicit. I’m not arguing that synthetic users replace UX research. The question I’m tracking is: what, if anything, can automated/browser/model-based signals reliably detect before a human researcher or designer gets involved?
The latest version separates browser-agent/task-completion infrastructure from UX diagnosis, includes skeptical evidence like first-click misalignment and synthetic-participant critiques, and keeps human review/handoff as a first-class part of the landscape.
https://mphaxise.github.io/ai-agent-ux-research-platform/
Original Post:
AI agents are starting to build software faster than our ability to judge whether that software is actually worth shipping.
There is already a lot of infrastructure forming around browser QA, task completion, screenshots, test runs, and evals. My guess is that much of this becomes standard infrastructure.
But shipping is not the same as completing a task.
A product can pass a browser test and still confuse users, create mistrust, miss the real need, or solve the wrong problem.
I’ve been mapping the emerging AI agent UX research landscape: products, workflows, papers, benchmarks, and experimental approaches focused on how agents validate what they build.
The current map tracks:
- synthetic UX research and AI user testing
- browser-agent evaluation infrastructure
- human review and handoff patterns
- launch readiness and trust signals
- agent UX observability
- emerging papers and benchmarks
My working thesis:
Browser QA is becoming infrastructure. The next gap is calibrated UX diagnosis.
Agents need to know what failed, why it matters, how confident the system is, what should be fixed, and when human validation is required.
This is still early and messy, but I’m planning to keep updating the map monthly as the space evolves.
Curious what people here think:
- Are there tools, papers, or workflows I should add?
- Are “synthetic users” actually useful in your experience, or mostly noise?
- Where do you see the biggest gap between task completion and real usability?