Comparing how OpenAI models recommend brands across 270 category questions (the biggest change wasn’t the brand list)
We wanted to understand what happens to brand recommendations when the model changes but the questions stay the same.
We gave GPT-5.4, GPT-5.5, and GPT-5.6 Sol the same panel of 270 category questions across six industries.
A few findings from GPT-5.5 to GPT-5.6 Sol:
- 70% of matched answers became shorter.
- Median answer length fell from 224.5 to 141 words.
- Median named brands only moved from 21 to 20.
- Explicit caveats fell from 40.7% to 20.4%.
- Decision-framework language fell from 33.3% to 13%.
- Retail shortlists narrowed from 20 to 13 brands, while Travel widened from 24 to 27.
The interesting part is that model updates don't create one universal change in brand visibility. They can compress explanations, remove caveats, ask for more context, or handle individual markets differently.
The report includes the methodology, industry breakdowns, exact model values, and links to the underlying model answers:
https://app.nyman.media/insights/ai-visibility
I’d be interested in feedback on the findings and also the methodology. What categories, models, or question types would you test next?