What if agent observability started with the expected outcome, not the trace?
I've been digging into silent failures in production workflows, and I keep noticing the same gap:
"The run succeeded" and "the task succeeded" are not the same signal.
A trace can tell me:
the model responded
tool calls returned successfully
latency was normal
the final output was valid JSON
But none of those necessarily tells me whether the actual task happened.
For example:
New order received → agent runs
→ API returns 200
→ JSON is valid
→ workflow is green
→ invoice never gets created
I'm thinking about an outcome-first model where each workflow/agent has an explicit expected outcome, for example:
Order received → invoice exists within 15 min
Lead received → CRM record exists
Document processed → required fields populated
Then observability becomes:
Expected → Actual → Evidence → Alert
rather than just:
Run → Trace → Error
I'm curious how people here think about this.
Do you define expected business outcomes explicitly today, or are you mostly inferring health from traces, evaluations, and model/tool metrics?
And when an agent says it completed something but the external side effect didn't happen, where does that verification currently live?