Post-train agents on a company that does not exist
Era generates a whole enterprise and serves its systems to an agent the way the vendors do: Salesforce, HubSpot, Zendesk, Jira, Gong, Slack and a document store, each as MCP tools and its own HTTP API. Because the company is generated rather than collected, every question about it has one deterministic key derived from the served bytes. That turns a rollout into a verifiable reward, a failing rollout into the negative half of a preference pair, and a seed into a fresh, contamination-free environment.
Generator, environment, rollout, grader
Beside the corpus the generator keeps a counterparty: the knowledge people hold but no system records. It shapes what the systems contain and it gives the grader the key. The policy gets the connectors and nothing else; no key, no rule, no judge model ever enters the environment.
What the trainer gets back per rollout: exact match against the key (plus the recorded partial score), every tool call and result, model turns, prompt and completion tokens, wall time, provider cost, the stop reason (answered, budget exhausted, refused, harness error), and which surface the policy used (MCP tools, vendor HTTP, or a code sandbox over the same REST).
Where frontier policies fail, measured
The current hard board: eight questions about one fintech where the convenient reading of the data is wrong. Every lane below is a model under a harness, three runs per question. Numbers are recomputed from the runs' own results and are live at benchmarks.html.
The same model under different scaffolds lands in different places. The gap between a bare loop and a loop with a code sandbox is the behaviour a post-trained policy should carry inside itself: paging, joining and recomputing without being handed a tool that does it.
Each cell is one question under one lane, 0 to 3 of 3 runs exact. A row that is dark across every lane is pure penalty; a row that is bright is solved. The mixed cells in between are where a gradient lives.
Rollout price is set by the policy and the harness, not by the environment. Era itself is a container per connector; the bill is the model's tokens over 20 to 120 tool calls per question.
Three trap shapes, weighted by what they teach
Every hard question sets one of three traps. The weights are part of the board and can be reused as reward weights directly: a selection trap held at 0 of 3 costs three times what a rule execution at 3 of 3 earns.
| Trap | What the literal reading gets wrong | Weight |
|---|---|---|
| hidden rule from a clue | A rule is planted indirectly in something served (a transcript line, a footnote, a handbook clause) with a decoy nearby. The solver has to recognise the clue as a rule, decide it applies, and execute it over the connectors. | 1 to 2 |
| selection trap | Several candidates are all well-formed: two systems disagree, a board was moved and the frozen copies are still referenced, several proposal rounds exist. Only one is in force. | 3 |
| stale report, live recompute | A spreadsheet carries the method and a number that was right on its date. The key is the same method run over the live system today. | 1 |
A mixed question is a preference pair for free: same task, same tenant, one trajectory that hit the key and one that did not. Solved and failed columns are SFT positives and hard negatives respectively.
From the easier set to the hard board to fresh companies
The 29 easier v2 questions (multi-hop joins, dossiers, root-cause reads, handbook conventions) show a policy can do the ordinary work. The eight hard questions carry 90% of the board's total and are where current frontier models sit between 0 and 0.75. Past the board, every seed, industry and estate size is a new company with new keys, so the curriculum does not run out and nothing in it can be memorised.
Six properties, in the trainer's terms
- 1Verifiable rewards for tool use.
Keys are scalars, ids or small JSON objects derived from the served bytes and compared exactly. Reward is deterministic, cheap and needs no judge model, so it cannot be gamed by writing a persuasive answer.
- 2Unlimited, contamination-free trajectories.
Every company is seeded and regenerable: new industries, sizes and system mixes on demand, no real records or people, nothing that can already sit in a pretraining corpus.
- 3Reproducible states.
The data plane is read-only and every lane runs against a committed replay corpus whose sha256 is recorded per run. Two rollouts of the same task see byte-identical systems, so reward variance is the policy's alone.
- 4Difficulty and reward weights by construction.
Trap shape, weight, systems touched and the counterparty rule behind each key are generator output, not hand-written prompts. Turning them into curriculum buckets or reward weights is a config change.
- 5Trace-rich SFT and preference data.
Every run records tool calls, model turns, tokens, cost, wall time and stop reason. Three repeats per question yield same-task pass and fail pairs; the oracle's own REST derivations give golden demonstrations.
- 6Harness-independent ground truth.
The same key grades a bare tool loop, a ReAct agent with a code sandbox, ADK and pi. Train against whichever surface the deployment will use and grade all of them on identical truth.
One tenant per rollout worker
Each connector runs as a container serving REST and MCP on one port. The control plane provisions isolated tenants and mints one token per tenant, so a training job holds many companies at once and every worker's environment is independent.
What Era is not yet
Nemotron 3.5 Super on Era
The loop above, run end to end on NVIDIA's Nemotron 3.5 Super: benchmarked on the Query and Compute tasks, LoRA-SFT on graded Era rollouts for abstention, re-measured on a held-out company. Read the case study.