Writing tests

Setup and teardown

A scenario assumes the application is in a known state: "cancel an order" needs an order to exist. Most scenarios also leave something behind. Without preparing state before a run and cleaning up after it, a suite passes on a fresh database and fails on the second run.

Saffron has no code layer, so it hands this to whatever you already have: an npm script, a SQL file, a curl.

{
  "baseURL": "http://localhost:3000",
  "setup": "npm run db:seed",
  "teardown": "npm run db:clean"
}

Each key takes one command or a list, run in order:

{
  "setup": ["npm run db:reset", "npm run db:seed", "node scripts/create-test-users.mjs"],
  "teardown": ["node scripts/delete-test-users.mjs"]
}

What runs when

setup Once per saffron run, before any browser opens, in the project root.
teardown Once per saffron run, after the last scenario and after the browser has closed.
reset Before each fresh execution a recording or a proof opens: the agent's recording of a scenario not recorded yet (or stale), each proof replay and stability proof, and the sensitivity check. Never before an ordinary replay, and never in the middle of a repair, which continues the replay's own execution. The commands get SAFFRON_SCENARIO, SAFFRON_FEATURE and SAFFRON_RESET_FOR (recording or proof), beside SAFFRON_BASE_URL and SAFFRON_ENV. Refused with more than one agent worker (§15).
A reset command fails Before a recording: that scenario is red, counted as blocked by the environment, and no agent session starts. Before a proof: that proof fails with the reason, so the proposal is filed unverified. The rest of the run goes on.
A setup command fails The run stops with exit code 2 and no scenario runs. Scenarios against a half-prepared application only produce misleading reds. The rest of the setup list is skipped, and teardown still runs, because the failed setup may have created half of its data.
Scenarios fail Teardown still runs.
You press Ctrl-C Saffron stops whatever is running, a setup command included, waits for it to be gone, closes the browser, runs teardown, writes the report of the scenarios that finished, marked interrupted (§4), then exits with code 130. Teardown itself is never interrupted: a second Ctrl-C while it runs is answered with a note and Saffron still waits for it.
A teardown command fails The scenario verdicts stand and the report is written. The remaining teardown commands still run. If every scenario passed, the run exits with code 2 instead of 0, because what was left behind is the next run's false red.
A command hangs It is stopped after hookTimeoutMs (five minutes by default) and counts as failed.

Setup may generate or refresh JSON files used by {data:...} steps, such as an expected date range calculated for today. Saffron validates these references and any environment variables they name after setup succeeds, using fresh file contents. Missing or invalid data stops the run before a browser opens; teardown still runs. With --no-hooks, the required files must already exist and contain current values. File-backed Examples are read when scenarios are discovered, so those files must exist before the run.

What the commands receive

They run through the shell with your environment, plus these variables, so one script can serve every target:

Variable Value
SAFFRON_BASE_URL The base URL of this run: --base-url, or baseURL from the config.
SAFFRON_ENV The data environment of this run (--env, SAFFRON_ENV or env). Empty when none is selected.
SAFFRON_SHARD The shard of a sharded run, such as 2/4 (§15). Empty otherwise. Every shard runs setup and teardown, so use it to keep the machines' seed data apart.
// scripts/create-test-users.mjs
const api = process.env.SAFFRON_BASE_URL + "/api/test-users";
await fetch(api, { method: "POST", body: JSON.stringify({ email: "admin@test.com" }) });

Their output is shown as they run, on stderr. The report lists each command, how it ended and how long it took.

Skipping them

npx saffron run features/orders.saffron --no-hooks

Use --no-hooks when the state is already prepared, for example when you re-run one scenario against an application you seeded a minute ago.

Setup, teardown and {unique:name} together

They solve different halves of the same problem:

Data Use
Data scenarios only read (a catalogue, an admin account) Seed it in setup.
Data a scenario creates (a new customer, an order) Name it with {unique:name} (§16), so two runs and two parallel scenarios never collide.
Whatever the run left behind Sweep it up in teardown, for example everything matching the pattern your unique values use.

Setup and teardown run once per run, not once per scenario. A scenario that needs its own private state should create it in its own steps, a Background or a StepSet.

A supporting script, when Saffron cannot express a value

Most values that change from day to day need no script: {date...} tokens write days, ranges and names in any locale, captures compare what the page showed before, and {unique:name} gives every run its own values. When a requirement still cannot be expressed at replay, a small script in setup can write the expected values into a git-ignored file under data/, which the scenarios read through {data:...} tokens. Saffron reads data/ after setup has run, so every replay, the proof replay included, checks fresh values.

Such a script stays trustworthy when it is small (one file in the project's scripts folder, no new dependencies), derives its values from the test's inputs or an independent source and never from what the app displays, states its locale, time zone and day-boundary assumptions, exits non-zero with a message when it fails (which stops the run before any scenario), and keeps secrets out of what it writes. Commit the script, not its output. --no-hooks skips it: the scenarios then read what its last run left, or stop naming the data file when there is none. The agent skill follows the same rules, asks before adding a script, and reports which Saffron capability could replace it, so a script that many projects write becomes a candidate for the runner itself.