We test whether an A2A agent actually does what its Agent Card claims, and sign the result so anyone can verify it
A2A agents publish an Agent Card that lists their skills, inputs and outputs. Other agents and teams decide whether to trust an agent based on that card. Nobody checks whether the agent actually behaves the way the card says, so we built something that does.
How it works:
- The card is fingerprinted (RFC 8785 canonical JSON + SHA-256). Every result is tied to the exact card version that was tested.
- Tests are drafted only from the examples the card itself declares. A human approves the criteria before anything runs.
- A separate runner process is the only part that holds the agent's credentials. It only talks to the approved host and redacts secrets before anything is stored.
- Layer 1 is deterministic: status, schema, media type, required content. Layer 2 is a pinned model (Claude Haiku 4.5) used only for checks that need meaning, like "is this a correct summary". The model, endpoint and rubric version are recorded with every result.
- Results go into an append-only store and the report is signed with Ed25519. Change one byte and verification fails.
Design choices: no single trust score (pass / fail / inconclusive per skill), inconclusive never counts as pass, every report lists what was NOT tested, and the final decision is always "flag for human review".
We ran it against 8 of the official A2A sample agents. Several declare "text" as a media type where the spec expects a MIME type like "text/plain". Small, but it's exactly the kind of card-vs-behavior gap we want to surface.
Caveats:
- A2A 1.0 only, no 0.3 yet.
- Coverage is only as good as the card's declared examples. Thin examples mean thin tests, and the report says so.
- Layer 2 is a model's judgment, not proof. That's why it's pinned, versioned and only runs after Layer 1.
Next up: adversarial probes (undeclared requests, prompt injection, failure handling).
If you're building an A2A agent, we'll test it for free and send you the signed report: https://www.suncly.com
What would you want to see in a report like this before trusting someone else's agent?