Agent-built mock
The agent controls the fake, but the fake only contains behavior the agent already anticipated. Its blind spots become the test suite’s blind spots.
Controlled coding-agent benchmark
100% bug coverage. 55% lower cost. 35% faster.
Same model. Same task. Same repository. The only variable was dependency access. Agents with Veris caught every planted integration bug in every run; no other test environment did.
Also: 38% fewer output tokens · zero fake dependency code
Issue
Prevent a duplicate charge after an ambiguous payment result.If the first charge succeeded but its response was lost, reconcile external state and do not charge again.
one acceptance testCoding agent + self-built fakeAgent authors its own dependency environment
agent workspacecode + integration tests
src/retry.tstests/retry.test.tsconfig/dependencies
What the agent can test against
Agent-authored fakecanned responseguessed state
Coverage varied by run
Green tests. Unreliable coverage.
The same agent caught 60%, 80%, or 100% depending on which vendor behavior its self-built fake happened to encode.
60–100%coverage by run
+123%avg. cost vs Veris
+54%avg. time vs Veris
Coding agent + VerisVeris Dependency Sandbox connected
agent workspacecode + integration tests
src/retry.tstests/retry.test.tsconfig/dependencies
What the agent can test against
Veris Dependency Sandboxpaymentstaxdatabase
Verified every run
Every bug caught. Every run.
The agent caught every planted integration bug in all three runs against stateful, controllable dependencies.
100%coverage every run
−55%avg. cost vs fake
−35%avg. time vs fake
Comparison basis: agent-built fake vs Veris Dependency Sandbox · three fresh-agent runs per condition · cost and time compare these two conditions
Why Veris catches what others miss
Integration testing needs truth and control.
A mock gives an agent control without certainty. A vendor test mode gives it real behavior without control over the hardest states. Veris gives it both.
Vendor test mode
The happy path is real, but critical outages, lost responses, and asynchronous states often cannot be produced on demand.
Veris Dependency Sandbox
Stateful, vendor-true dependencies with failure states the agent can create, inspect, reset, and reproduce as part of its own loop.
A directional experiment, controlled for the mechanism
The result came from the environment—not a different agent.
Model
Held constant across runs
Task
The same integration-testing brief
Repository
The same real billing codebase
Variable
Dependency access only
How the benchmark was graded
Fresh coding agents wrote integration tests using Veris, static mocks, live vendor test mode, or their own fakes. After each agent finished, pre-registered integration failures were installed and its suite was rerun. A failing test meant the bug was protected; a passing test meant the bug could ship.
Directional benchmark · 12 fresh agent runs · one production-derived codebase · July 2026. Cost, wall clock, and token comparisons use the agent-built-fake runs as the baseline. The benchmark demonstrates the mechanism; it is not a universal estimate for every repository or task.
The next PR
Let the agent prove the integration before a reviewer has to.
Bring Claude Code, Codex, Cursor, or your own agent. Veris supplies the isolated, stateful dependencies it needs to find the failures mocks cannot.