What do you do with prompts where the AI changes its mind constantly?
Not talking about normal variation between runs. I mean prompts where the whole answer seems unstable for weeks.
One day there are 5 recommendations, next time there are 3 completely different ones, then a week later it gives a general answer and recommends nobody.
At some point does a prompt become too noisy to be useful as a benchmark?
I'm tempted to flag a few of ours separately rather than letting them mess with the overall trend.
u/randomredditer5743 — 6 days ago