what runs
green-check CI
Unit tests against mocks — the real agent is never exercised end to end.
layered testing
An explicit smoke invocation calls the real skill with a real payload.
readiness + invocation proof
AI agent testing you can prove ran.
You can't trust an agent you haven't tested, but a deployment health check is not a behavior test. a2a keeps the layers explicit: deployment verification checks the expected image, runtime, live Agent Card, and advertised skills; source review adds advisory findings; and an explicit smoke invocation calls the real skill and evaluates its output. That invocation can emit a signed ExecutionReceipt for what ran, while the expected-result assertion remains in the test record.
deployment readiness · advisory source review · real skill invocation
0
testing layers
0
automatic skill executions in deploy verify
receipt per real invocation
the problem
A green checkmark isn't proof the agent works.
Agent testing becomes unreliable when one green status is asked to prove everything. Unit tests can validate code without exercising the hosted runtime. Deployment verification can prove the expected image is ready and discoverable without calling a skill. Source review can flag risk without determining whether an output is correct. And an invocation receipt can record a real run without knowing your business assertion. Collapse those layers and a card check gets mistaken for behavior coverage, or a CI status gets mistaken for signed execution evidence.
Unit tests against mocks never exercise the running agent end to end.
A valid Agent Card proves discovery, not that each advertised skill returns the right result.
Advisory source review can find risk, but it does not gate the build or execute the code.
A signed invocation and a test assertion are related evidence, not the same record.
the a2a way
Separate readiness, behavior, and evidence.
Use deployment verification to establish that the expected runtime, Agent Card, and advertised skill surface are live. Use the advisory reviewer to inspect committed source. Then invoke critical skills with bounded inputs and explicit output assertions. The real invocation emits its own signed ExecutionReceipt; the test harness records whether the returned result passed your acceptance criteria.
Test behavior with a real invocationDeployment readiness stays explicitSource review is a separate signalOutput assertions live in the testRe-run after every material changeSigned evidence for the invocation
Test behavior with a real invocation
A behavior smoke test calls an actual agent skill with a real payload and asserts on the returned result. That is the end-to-end test: the running handler, its granted tools, and its real output path — not merely a mocked function or a healthy process.
Deployment readiness stays explicit
Before a deployment is marked live, the control plane checks the expected image, Knative runtime readiness, live Agent Card, and advertised skill surface. That catches build, reconcile, packaging, and discovery failures. It does not execute every skill or validate a sample output.
Source review is a separate signal
The agent-reviewer inspects committed source asynchronously and stores status, findings, and severity counts in a review record linked to the deployment timeline. It is advisory today: useful evidence for an owner, not an automatic build or rollout gate.
Output assertions live in the test
The smoke harness owns the expected result, schema checks, and domain assertions for the skill it invokes. Deployment verification can prove that a skill is advertised; only a real invocation plus explicit assertions can show that the skill behaved as expected for that input.
Re-run after every material change
A code edit, dependency update, configuration change, or repair invalidates the previous behavior result. Invoke the changed skill again against the intended revision and keep the new result alongside that deployment instead of treating an earlier green check as transferable.
Signed evidence for the invocation
A real hosted invocation through the API, MCP, or /invoke entry point emits an Ed25519-signed ExecutionReceipt covering agent, caller, skill, input hash and result preview, verified grant IDs, outcome/result preview, and timing. Deployment verification and source review remain separate records.
side-by-side
One green check vs. layered agent testing.
coverage
green-check CI
You test the functions you remembered to; the wiring goes untested.
layered testing
Deploy checks image/runtime/card/skills; the smoke test owns behavior and output assertions.
review
green-check CI
A human eyeballs the diff if they have time, or nobody does.
layered testing
An advisory agent-review record adds findings without pretending to be a hard gate.
after a fix
green-check CI
A green check from an earlier run gets trusted for code that changed.
layered testing
The changed revision is invoked again and evaluated against explicit assertions.
the evidence
green-check CI
A green check that scrolls off the CI log and can't be reproduced later.
layered testing
Deployment readiness stays on the deploy; the real invocation has its own signed receipt.
dimension
green-check CI
layered testing
what runs
Unit tests against mocks — the real agent is never exercised end to end.
An explicit smoke invocation calls the real skill with a real payload.
coverage
You test the functions you remembered to; the wiring goes untested.
Deploy checks image/runtime/card/skills; the smoke test owns behavior and output assertions.
review
A human eyeballs the diff if they have time, or nobody does.
An advisory agent-review record adds findings without pretending to be a hard gate.
after a fix
A green check from an earlier run gets trusted for code that changed.
The changed revision is invoked again and evaluated against explicit assertions.
the evidence
A green check that scrolls off the CI log and can't be reproduced later.
Deployment readiness stays on the deploy; the real invocation has its own signed receipt.
questions
Frequently asked.
How does a2a cloud test AI agents?
Use three layers. First, run local or CI tests for your code and contracts. Second, let deployment verification check that the expected image is reconciled, the Knative runtime is ready, the live Agent Card loads, and skills are advertised. Third, explicitly invoke the important skills with bounded smoke inputs and assert on their outputs. The deployment and advisory source review have their own records; each real hosted invocation can emit its own signed ExecutionReceipt.
What's the difference between testing an agent and observing it?
Testing asks a specific question with an expected result: invoke this skill with this input and check these assertions. Deployment verification asks whether the expected runtime is ready and discoverable. Observability helps inspect what happened during a run. A signed ExecutionReceipt is run-level evidence for the actual invocation; it does not turn a deployment readiness check or advisory source review into a signed test result.
Does deployment verification execute every advertised skill?
No. It checks the expected image and runtime, fetches the live Agent Card, and verifies that the card exposes a valid advertised skill surface. It does not call every skill or decide whether a returned business result is correct. Add explicit invocation tests for critical skills and define the expected output or invariant in the smoke harness.
Does the automated reviewer block a broken build or deploy?
Not today. The agent-reviewer runs asynchronously against committed source and persists its findings in an AgentReviewRun linked to the deployment timeline. Its verdict is advisory and the deployment does not wait for it. Build failures are reported by the build stage; runtime, Agent Card, and advertised-skill failures are reported by deployment verification; behavior failures belong to the explicit invocation test.
Why isn't a passing CI check enough evidence a test ran?
A CI status is useful orchestration metadata, but it is not the signed execution record of an agent call. When the smoke test invokes a hosted skill, that real run can emit an Ed25519-signed ExecutionReceipt with its execution fields and result preview. Keep the test's expected-output assertion in the test report, and correlate it with the invocation and deployment records rather than claiming one record proves all three.
keep reading
Related guides.
All guides live in the guides index.
don't trust the agent
Check readiness. Invoke the skill. Assert the result.
a2a cloud keeps agent testing evidence honest. The deployment timeline records source, image, runtime, Agent Card, advertised skills, logs, verification, and advisory review findings. Your smoke harness invokes the real skill with bounded input and checks the returned output. That invocation emits a signed ExecutionReceipt for the run; your test record carries the acceptance assertion. Separate records, one traceable testing story.