Veris joins the Fireworks Frontier Ecosystem
Today at Forge, its first conference, Fireworks introduced the Frontier Ecosystem: partners across the AI stack that help organizations build, evaluate, train and deploy specialized intelligence. Veris is a launch partner.
Fireworks trains and serves open models. Veris builds the world the model is tested and trained in: a full, stateful twin of your production environment, the people your agent deals with, and tasks whose pass or fail is decided by code. From today, any agent on Veris Bench runs on any Fireworks model with your own key (docs). Here is the whole loop in the product.
Trains the model (SFT, DPO, RL) and serves it.
Builds a stateful twin of your production environment, and the tasks that decide whether the job got done.
1. Set up the environment: a stateful twin of production
A bench runs in a world, and the world is a full, stateful simulation of the production environment your agent works in. Every system it touches is a twin: a copy of the real service that speaks the same API your code already calls, holds the records, and remembers what happened between calls. Create a payment and it shows up in the next list. Issue a refund and the balance moves. Send a request the vendor would refuse and you get the vendor's own refusal, error body included. Nothing is a mock with a canned answer.
Twins are built by measuring the live vendor, not by reading its docs. In our fidelity study, which runs Nango's open-source integration templates unmodified against 15 real services and their twins, the two agreed on 703 of 721 operations, with zero cases where the twin succeeded and the vendor refused. Where the vendor ships a runnable engine, such as Apache Fineract for core banking, the real thing runs behind the twin's front door.
The twin catalog covers the systems an enterprise agent lives in: Stripe, Marqeta and Fineract for money, Salesforce, HubSpot and Attio for CRM, Gmail, Outlook, Slack and Teams for communication, Jira, Linear and Notion for work, Google Drive and Dropbox for files, GitHub, Postgres over the wire protocol, OpenSanctions for screening, and the identity providers in front of all of them. An internal system gets a twin from its API spec in about two days.
Every twin in a world is rendered from one truth graph, so the CRM, the ledger and the mailbox agree about the same customer, the clock reads the same everywhere, and the duplicates, late postings and misprints of real operations are present on purpose and on record. Your data shapes the world as priors and is never copied into it. The world is frozen at a date, and every attempt boots a sealed copy, so the same requests return byte-identical responses on every run. Simulated customers and colleagues, cast from your users, sit on the other side of the conversation.
2. Create tasks, for regression or for training
Tasks come from what you already hold. Policies compile into clauses, production traces and incidents become scenarios, process documents and subject-matter experts supply the edge cases, and you can write one by hand. Every task carries its checks before any agent runs: what the agent must say, which calls it must and must not make, and how the world must look afterwards. A reference run has to pass, and for a policy task a run that breaks the clause has to fail, or the task does not join the bench.
The same task serves two jobs. Marked for regression, it runs on every pull request from CI and fails the job when the pass rate drops below your floor. Marked for training, its verifier is the reward.
3. Run a trial, read the results page
Add your Fireworks key under Provider keys and register your agent as a candidate: your own container, a hosted endpoint, or an open-source harness such as Pi or OpenCode on the Fireworks model you choose. Register one image under several models and they compete under the same agent. Then run a trial: every candidate meets every task and every persona, three repeats, fifty at a time.
The results page is a leaderboard: pass rate with a 95% confidence interval, cost per task, turns, tool calls and duration for each candidate. Open any attempt to see each check's verdict, the transcript, and every tool call and service request the agent made. Add a new version to the same trial and it faces the same tasks and the same worlds, so the comparison is fair.
4. Hill-climb with managed RL on Fireworks
With an open model on the leaderboard, the bench becomes the training signal. Pick the candidate and Veris runs reinforcement fine-tuning (GRPO) on Fireworks, with the bench's verifiers as the reward and the tasks the agent keeps failing as the target. Each checkpoint lands back on the same trial as a new candidate, so the leaderboard is the hill-climb: the base model, each training step, and the frontier baseline you started from, side by side. Held-out tasks and the regression suite gate every checkpoint before it counts. The one that wins is served on Fireworks.
Prefer your own trainer? The bench exports to Harbor, so the tasks, worlds and graders run anywhere that reads it, with Fireworks' Training API doing the updates while the environment stays on Veris.
The verifier that graded the failure is the signal that trains it away. Nothing in the loop depends on a judge that can be talked into a pass.