Running

Configuration

Everything lives in saffron.config.json at the project root. Every key is optional: the defaults below are what runs when the file is absent. saffron status --json prints the effective configuration, so you can see what a run will actually use.

{
  "baseURL": "https://stage.your-app.com",
  "features": "features",
  "actionTimeoutMs": 5000,
  "pollIntervalMs": 100,
  "retries": 1
}

Timing

Option Default Description
actionTimeoutMs 5000 How long a single action may take before it fails. This is the budget for Playwright's own waiting (visible, stable, enabled, receives events) and the deadline for a polled assertion. Raise it for a slow app; a step that needs much longer is usually better written as an explicit wait.
pollIntervalMs 100 How often a polled assertion re-checks the page inside that budget. Lower reacts sooner and costs more CPU; raise it on a busy CI box or for an app that repaints slowly. Values below 10ms are clamped.
retries 1 Extra attempts per action, with a 250ms × attempt backoff, before the step counts as failed and the agent is asked to heal. This is not a scenario re-run. Absorbed retries are visible: a retried ×N chip in the report, a line in the terminal.

Project

Option Default Description
baseURL none The app under test, so steps can say "the login page" instead of a full URL. Overridden by --base-url.
features "features" Directory holding .feature and .saffron files, searched recursively.
storageState none Playwright storage-state JSON (cookies and localStorage) applied to every browser context, so replays and the agent start authenticated. See §8.

Runtime

Option Default Description
browser "chromium" Replay engine: chromium, firefox or webkit. Recording and healing always use Chromium, because the agent attaches over CDP.
workers 1 Parallel replay workers. Agent work stays sequential, so this speeds up cached runs, not recordings.
snapshotMode "none" none has the agent take page snapshots deliberately rather than after every action, measured at about 60% fewer AI calls and 44% lower cost per recording. Use "full" when the agent struggles to locate elements on a highly dynamic page.

The agent

Option Default Description
model the agent's own default Model used for recording and authoring.
healModel same as model A cheaper model for heal sessions and their refine passes. Recordings keep model.
maxTurns 100 Turn limit for one agent session. Raise it for a very long scenario.
reuseSteps true Seed a new recording from steps already recorded elsewhere in the project, so the agent only fills the gaps. --no-reuse disables it for one run.
verifyProposals true Proof-replay every recording at zero AI before filing it, so a proposal you review is one that already replays. --no-verify disables it for one run.

Gates

Option Default Description
strict false Treat yellow (passed, pending review) as failure, so CI is green only on reviewed caches. Also --strict.
assertionPolicy "strict" strict never lets the agent adapt an assertion. adaptable-mid allows mid-scenario checkpoint assertions to adapt with a recorded note; the final assertion block of a scenario is always strict.

How timing actually works

A cached replay does not sleep. Three mechanisms stack, and each one is visible in the report:

  1. Playwright's actionability wait. Every interaction waits for the element to be visible, stable, enabled and able to receive the event, up to actionTimeoutMs. Nothing is recorded for this: it is how Playwright behaves.
  2. Polled assertions. An assertion re-checks every pollIntervalMs until actionTimeoutMs runs out, sweeping every recorded locator on each tick, so a match inside an iframe does not have to wait out the main page's full budget first.
  3. Action retries. After that, the action is retried retries times with a growing backoff. Retries that were absorbed are reported rather than hidden, because a step that only passes on the second attempt is something you want to know about.

A failure that survives all three is a real failure, and that is what the agent heals. There is no whole-scenario retry: rerunning a flaky test until it passes is what Saffron replaces with a recorded, reviewable fix.

Timing

Option Default Description
actionTimeoutMs 5000 How long a single action may take before it fails. This is the budget for Playwright's own waiting (visible, stable, enabled, receives events) and the deadline for a polled assertion. Raise it for a slow app; a step that needs much longer is usually better written as an explicit wait.
pollIntervalMs 100 How often a polled assertion re-checks the page inside that budget. Lower reacts sooner and costs more CPU; raise it on a busy CI box or for an app that repaints slowly. Values below 10ms are clamped.
retries 1 Extra attempts per action, with a 250ms × attempt backoff, before the step counts as failed and the agent is asked to heal. This is not a scenario re-run. Absorbed retries are visible: a retried ×N chip in the report, a line in the terminal.

Project

Option Default Description
baseURL none The app under test, so steps can say "the login page" instead of a full URL. Overridden by --base-url.
features "features" Directory holding .feature and .saffron files, searched recursively.
storageState none Playwright storage-state JSON (cookies and localStorage) applied to every browser context, so replays and the agent start authenticated. See §8.

Runtime

Option Default Description
browser "chromium" Replay engine: chromium, firefox or webkit. Recording and healing always use Chromium, because the agent attaches over CDP.
workers 1 Parallel replay workers. Agent work stays sequential, so this speeds up cached runs, not recordings.
snapshotMode "none" none has the agent take page snapshots deliberately rather than after every action, measured at about 60% fewer AI calls and 44% lower cost per recording. Use "full" when the agent struggles to locate elements on a highly dynamic page.

The agent

Option Default Description
model the agent's own default Model used for recording and authoring.
healModel same as model A cheaper model for heal sessions and their refine passes. Recordings keep model.
maxTurns 100 Turn limit for one agent session. Raise it for a very long scenario.
reuseSteps true Seed a new recording from steps already recorded elsewhere in the project, so the agent only fills the gaps. --no-reuse disables it for one run.
verifyProposals true Proof-replay every recording at zero AI before filing it, so a proposal you review is one that already replays. --no-verify disables it for one run.

Gates

Option Default Description
strict false Treat yellow (passed, pending review) as failure, so CI is green only on reviewed caches. Also --strict.
assertionPolicy "strict" strict never lets the agent adapt an assertion. adaptable-mid allows mid-scenario checkpoint assertions to adapt with a recorded note; the final assertion block of a scenario is always strict.

How timing actually works

A cached replay does not sleep. Three mechanisms stack, and each one is visible in the report:

  1. Playwright's actionability wait. Every interaction waits for the element to be visible, stable, enabled and able to receive the event, up to actionTimeoutMs. Nothing is recorded for this: it is how Playwright behaves.
  2. Polled assertions. An assertion re-checks every pollIntervalMs until actionTimeoutMs runs out, sweeping every recorded locator on each tick, so a match inside an iframe does not have to wait out the main page's full budget first.
  3. Action retries. After that, the action is retried retries times with a growing backoff. Retries that were absorbed are reported rather than hidden, because a step that only passes on the second attempt is something you want to know about.

A failure that survives all three is a real failure, and that is what the agent heals. There is no whole-scenario retry: rerunning a flaky test until it passes is what Saffron replaces with a recorded, reviewable fix.

Authenticated apps

Generate a Playwright storage state once, point Saffron at it:

npx playwright open --save-storage=.auth/state.json https://stage.your-app.com
{ "storageState": ".auth/state.json" }

Every scenario's browser context (replay and agent) starts with those cookies/localStorage. Refresh the file when the session expires; keep it out of git.

Parallel and cross-browser runs have their own section: §15.

More on authentication

storageState (config or --storage-state) applies a Playwright storage-state JSON (cookies + localStorage) to every scenario's browser context: cached replays and the agent start authenticated. All knobs live in saffron.config.json: baseURL, features, actionTimeoutMs, pollIntervalMs, retries, model, maxTurns, storageState, strict. Every one of them, with its default and what it changes, is in the configuration reference.

Runtime: browsers, workers, models

  • Cross-browser replay: --browser firefox|webkit replays the committed suite on Gecko/WebKit engines: replay runs on Playwright's browser engine, which drives all three. Recording and healing always use Chromium (the agent attaches over CDP); under another engine those scenarios fail with explicit record-on-chromium guidance instead of a confusing error.
  • Parallel workers: --workers N runs zero-AI replays concurrently (one browser context each). Scenarios that turn out to need the agent are re-run sequentially with full agent semantics, mid-run healing still continues from the exact failed page state.
  • healModel: point heal sessions (and their refine passes) at a cheaper model than recordings: most heals are simple locator fixes.