Do we overestimate the value of LLM intermediate outputs for humans? Looking for examples across fields
I've been noticing a pattern in how people use LLMs, and I'm curious if you've seen it elsewhere.
Here is the scenario:
An LLM workflow has an intermediate artifact (e.g. debating between different personas, a chain-of-thought trace, a planning document, etc.
That artifact is useful within the LLM pipeline and it improves the final output (better code, more persuasive essay, more accurate diagnosis).
A human looks at that artifact and thinks: "Well, if it helps the LLM, it should help me too."
So the artifact may get repurposed as a human-facing tool something to read, review, or edit (e.g. LLM reasoning traces shown to clinicians as "interpretable" decision support.).
The assumption that "what works for the model will work for me" seems to be at play.
Curious to hear your thoughts and experiences, or more in-depth research related to this scenario.