The execution hierarchy
An Experiment groups comparable Runs, but a direct Run does not have to belong to one.
Definitions
| Object | Meaning | Identity |
|---|---|---|
| TaskSet | An immutable bundle of Tasks and their complete referenced content. | sha256:… |
| Task | One immutable unit of work: steps or instructions, Environments, seed Artifacts, tools, and optional Verifiers. | task_key within its TaskSet |
| Experiment | A pinned Task selection evaluated across one or more agent configurations. | exp_… |
| Run | One agent configuration expanded over every selected Task and repetition. | run-… |
| Rollout | One retryable Task-and-index slot inside a Run. | task_key#0 |
| Attempt | One immutable, sealed execution that Verifiers grade. | attempt-… |
| Session | One ephemeral Environment-preparation lifetime. It may serve an Attempt, env serve, or a fresh Verifier Environment. | Runtime-assigned session or lease ID |
Session versus Attempt
A Session is ephemeral machinery that prepares and opens Environments. It can drive an Attempt, support env serve, or host a fresh Verifier Environment, and it can exist without an Attempt. An Attempt is the durable execution episode whose submission, evidence, and verification provenance remain after its Session closes.
Verification and evidence
Verifiers grade frozen Attempt evidence, not live Environment state after execution. A Verifier records facts, may project them to a score, and applies its declared decision rule to produce a verdict. Registry UI and Runtime reports then interpret those verdicts through separate projections. The UI labels a succeeded, sealed rollout with no usable verdict Unscorable. Runtime report snapshots carry their own success policy; under the current default, a succeeded rollout passes only when at least one verdict was recorded and every recorded verdict is pass.