Skip to documentation
DocsEvaluations

Credentials and required bindings

Satisfy every Task and agent requirement without persisting secret values in content.

On this page

Content names requirements, not values

A Task, Environment, Verifier, or agent harness declares a logical binding. The Run plan selects a source for that binding. Secret values never enter TaskSet bytes, digests, agent options, or ordinary result records.

Resolve requirements in the Experiment composer

  1. 1
    Automatic match

    When exactly one compatible project credential exists, Fleet can resolve it without a chooser.

  2. 2
    Choose a credential

    When several candidates exist, explicitly select the one intended for this agent.

  3. 3
    Use once

    Supply a value for this Run only. It is not saved as a reusable credential.

  4. 4
    Save a credential

    If authorized, create a project credential and name the requirement it satisfies.

Create a reusable Registry credential

Terminal
printf %s "$ANTHROPIC_API_KEY" | flt credentials set acme/anthropic-default \
  --satisfies anthropic-api-key --value-stdin
flt credentials list --namespace acme

Hosted workers can resolve pasted values, aws-secrets-manager-json, and HTTPS fleet-keyring sources. Environment variables, files, keychains, and 1Password are local Runtime sources and cannot be dereferenced by a hosted worker.

Saving a pasted value requires the Registry deployment to have a secret store. Direct --source references name material outside the project and are restricted to platform operators; other users save credentials by pasting values.

Keep flt contexts and credential files aligned

Terminal
flt context list
flt context current
flt context show
flt context create staging
flt context use staging
flt context use default
flt context remove staging

An flt context selects an isolated local Store, settings file, credential file, and state directory together. The fleet administration CLI has no context switch. Runtime also does not inherit the selected flt context: when a hosted target uses credential_host, set FLEET_CREDENTIALS to the credential path shown by flt context show if it is not the default file.

Access required for hosted evaluations

OperationRequired access
Select and execute a TaskSetRead access to its source Repository.
Create an Experiment or RunWrite access to the results project.
Read prior RunsRead access to their results project.
Use a saved credentialRead access to that project credential.
Create, update, or remove a credentialCredential write access in its project.
Save a direct external source referencePlatform-operator access.

Supply a one-run value from the CLI

Terminal
export ANTHROPIC_API_KEY="..."
fleet-runtime runs submit benchmark -n 1 \
  --agent claude --harness-var ANTHROPIC_API_KEY=@ --follow

The @ form reads the value from the current environment and carries it through the Run binding plan. The value is not written into command arguments or immutable Task content.

Why Create is blocked

Submission is blocked while any required binding has no source, two agent cards disagree about the same credential being created, the result project is not writable, or an agent configuration is incomplete. Resolve the inline requirement row rather than retrying an unchanged submission.