// solutions.training_and_loops

Train yoursupport agentbefore you deploy. Keep training it after.

Before launch, battle-test your agent until it passes QA and compliance. After launch, feed every production failure back into simulation, so the agent that shipped keeps getting better.

  • scenarios generated from your code, prompts, and production traces
  • root-cause analysis with concrete fixes to harness and instructions
  • RL fine-tuning on verified scenarios when your model is open-source
CodePromptsProduction tracesObservability systems
Works with every major agentic framework
LangGraphLangChainCrewAIPydantic AIOpenAIHugging FaceOllamaYour own harness

// the_problem

Agents plateau after launch. Failures are found by customers, fixes are guesswork, and evals report scores without driving improvement.

Before

An eval suite that says 74% and nothing about why. Incidents get patched one prompt tweak at a time, each fix silently breaking something else, and the agent six months after launch is the agent you launched.

With Veris

Every failure, caught before deploy or replayed from production, is root-caused, fixed, and pinned as a regression scenario. When your model is open-source, the same verified runs become RL rewards.

// with_veris_you_get

Improvement stops being guesswork: every run ends in a fix, a regression test, or a training signal.

11/15
scenarios failed for an agent that looked done: bugs that never surface in a unit test, caught in simulation
94.4%
accuracy after RL fine-tuning on an enterprise task, up from 80%: a 4B open model beating frontier baselines
+50%
task score from RL fine-tuning a 14B open model on detection engineering, closing the gap to frontier models
100%
tool coverage from auto-generated scenarios, on an agent that could previously only be smoke-tested by hand

// how_it_works

The Veris loop

The same four moves on every Veris page, voiced for this job.

01
Compose
Simulate the tools and systems your agent depends on: the same environment before deployment and after.
env: CRM + ticketing + knowledge base + customer persona
02
Generate
Create scenarios and verifiers from your code, prompts, and production traces, with direct integrations into major observability systems.
seed: production_traces.jsonl (30d) → 128 scenarios · refund flow stressed
03
Simulate
Battle-test the agent at scale: a QA gate before deploy, a replay ground for every incident after.
nightly · on every push · post-incident
04
Act
Turn verified failures into improvement: root-cause analysis with concrete fixes, and RL fine-tuning when your model is open-source.
RCA → 6 fixes · RL → reward/mean 0.787

// the_engine_underneath

One engine. Simulated worlds.

Every Veris solution runs on the same simulation engine. Compose an environment by listing the systems in your stack (databases, APIs, MCPs, and the humans in the loop) and describe a scenario in plain language. The engine simulates that world: actors stay on goal and behave like people, services stay consistent and high-fidelity, and every run is isolated and repeatable.

Runs in your VPC · SOC 2 Type II · Thousands of concurrent agents · Synthetic data or your own

Explore the platform →
▸ Your agent · your code
agent :8080test scenarios
↓ every outbound call
▦ Simulated services
postgresstripesalesforceinternal APIsMCPs
◉ Simulated humans
customeranalystapprover
⚙ Veris Simulation EngineOrchestration · Fidelity · Consistency · Rewards
Simulated services

…and dozens more, from banking cores to ERPs — or bring an API spec and Veris builds the simulator

// what_you_get

What you get

01RCA with proposed fixes

Every failure is traced to its cause in the agent's harness or instructions, with a concrete fix rather than a red X. Veris replays the run, isolates the decision that went wrong, and points at the line that made it.

veris.console — root cause
veris rca run_h82kqm/scn_0114
replaying failed run… 14 steps
✕ step 9: agent skipped balance check before refund
cause: system_prompt.md:42 — refund rule ambiguous
fix: add explicit precondition to refund instruction
fix: expose check_balance in tool schema description
re-run with fixes → 128/128 passed

02RL training loop

When your model is open-source, verified scenarios become rewards: reinforcement fine-tuning on the exact tasks your agent keeps failing. The verifier that graded the failure is the same signal that trains it away.

veris.console — training
reward/meansteps →
GRPO · Qwen3-8Breward 0.787kl_div 0.026

03Never waste a failure

Production incidents flow in through your observability stack and come back as regression scenarios the agent must pass forever after. The failure your customer found becomes a test no future version can fail quietly.

veris.console — incident loop
INC-2041 · refund posted twice (prod)
trace imported from observability stack
scnset_x9vq2w · 32 scenarios seeded
pinned to regression suite · runs every push

04A gate before every deploy

The full scenario suite runs on every change, so an improvement in one place can't quietly break another. The gate either opens or it doesn't.

veris.console — deploy gate
scenario suite · 128/128 passed
policy enforcement ≥ 92% · held
regression INC-2041 · passed
latency p95 < 2.5s · 1.9s

// before_you_ask

Questions teams ask before the first loop

$Do I need an open-source model to get value from this?

No. The core loop of scenario generation, simulation, root-cause analysis, and regression gating works with any model behind any API, closed or open. The open-source requirement only applies to the last step: RL fine-tuning, where the model weights have to be yours to update. On any model, every run ends in an improvement report with proposed prompt and harness fixes: concrete edits to instructions and tool definitions you can ship directly. On open models, the same verified runs additionally become RL rewards.

$Where do the scenarios come from, and how do I know they're realistic?

Three sources: your code and prompts (read directly), your documents (PRDs, policies, regulations), and your production traces (pulled in through integrations with major observability platforms). Scenarios seeded from production traces are replays of things that actually happened to your agent, so realism is inherited rather than assumed. Every scenario ships with a verifier that defines what passing means, so you can audit the criteria before you trust the score.

$How reliable are the verifiers?

Reliable enough to gate deploys on. Wherever an outcome can be checked objectively (a tool call made, a record written, a number matching), verifiers check environment state, not vibes. Where judgment is required, graders are framed around specific failure modes rather than open-ended quality, which in our published comparison reached 94% consistency versus 66% for conventional grading, and we run consistency tests on every grader before its verdicts count. The criteria are yours to read and edit, so you can audit what passing means before you trust the score.

$How is this different from the eval suite we already run?

An eval reports a score; the loop produces changes. When a run fails, Veris replays it, isolates the decision that went wrong, and points at the instruction or tool definition that caused it, with a proposed fix. The failure is then pinned as a permanent regression scenario, so the fix can't quietly rot. Evals tell you the agent is at 74%; the loop tells you why, what to change, and proves next week's version didn't reintroduce it.

$What does the incident-to-scenario flow look like in practice?

A production incident lands in your observability stack as a trace. Veris imports the trace, reconstructs the environment state that produced it, and generates a scenario set around it: the original failure plus variations that stress the same weakness. Those scenarios join the regression suite that runs on every push. Median time from incident to pinned regression coverage is measured in minutes, not sprint cycles.

$How does this work with our observability and telemetry stack?

It plugs into it rather than replacing it. Connect the platform you already use (Datadog, an OpenTelemetry pipeline, or an LLM-observability tool) and production traces flow into Veris as raw material: incidents become replayable scenarios, and real usage patterns steer scenario generation. Your instrumentation stays exactly as it is; Veris consumes what it already emits.

$What happens to our production data when traces flow in?

It stays under your controls. Traces can be redacted before Veris sees them, scenario generation preserves the shape of an incident rather than the customer inside it, and the whole pipeline runs in your VPC or on-prem if that's where you deploy. Retention is configurable, and your traces are used for your scenarios and your training, nothing else.

$How does RL fine-tuning work, and where does it run?

Every simulated run is already a labeled training sample: the scenario's verifier graded it. When your model is open-source, Veris uses those verified outcomes as rewards for reinforcement fine-tuning (GRPO and similar), targeting exactly the tasks your agent keeps failing. Training runs in your infrastructure or ours, with the same VPC deployment options as the simulation engine, so weights and traces never leave your boundary unless you want them to.

$Does fine-tuning on our failures make the model worse at everything else?

No. That's what the regression suite is for. Improvement targets the failure modes, and every candidate, whether a tuned prompt or tuned weights, is evaluated on held-out scenarios plus the full regression suite before it ships. In our published incident case study, the improved agent fixed the failure while holding performance on every prior scenario, without overfitting to a single example. If a candidate regresses anywhere, the gate stays shut.

$When should I use Veris?

At any point in the agent's development. Before a line of agent code exists, scenarios generated from your docs define what it must handle. During development, the simulation loop is your QA gate. At launch, it's the evidence for review. In production, incidents feed back as regression scenarios and training signal. Teams enter the loop wherever they are, and the rest of it attaches naturally.

$What do you need from us to get started?

Repo access (or just the prompts and tool definitions), and optionally a connection to your observability stack for production traces. The first scenario suite is typically running within a day: connect the repo, generate scenarios, run the loop. No framework migration: Veris sits around your agent, not inside it.

// book_a_demo

See it on your stack.

Book Demo →