Writing tests

API tests (beta)

Beta. API test automation is new in 1.2.0, the headline of that release. It is built, reviewed and covered by tests, and still being shaped by use: the recording format and what the agent records may change between releases. Report what you find in the community repo.

A scenario can test an HTTP API with no browser at all, or call the API from inside a browser scenario. The workflow is the one you know: the agent records each scenario once, every later run replays it at zero tokens, and a Then step is never healed.

Write an API feature

Tag the feature (or a scenario) @api. Its steps describe requests and what their responses must say, in plain language, as the browser steps do:

@api
Feature: Customers API

  Scenario: Subscribe a new customer to a plan
    When I create a customer named "cust-{unique:name}" through the API
    And I put the customer on the "Basic" plan for 3 months
    Then the customer should be on the "Basic" plan for 3 months
    And the customer should match the contract

  Scenario: A missing customer answers 404
    When I read customer 999999 through the API
    Then the response status should be 404

A request that sends more than a value or two (an API key, a correlation id, query parameters, body fields) reads better with its values in a 2-column data table under the step, one row each, than on one long line. Each cell records as its <table:key> wherever the request sends it (its path, a header, the body, a form field), so editing a cell replays at zero AI, and renaming or adding a key re-records the step:

When I validate the no-show reservation with these request details
  | reservation    | {data:no-show.chargeable} |
  | API key        | {env:NOSHOW_API_KEY}      |
  | correlation ID | {unique:correlation}      |
Then the response status should be 200

An @api scenario runs in the API medium: no browser opens (a run of @api scenarios alone needs no browser installed, so an API-only CI job skips npx playwright install), each execution gets a request client of its own (its own cookie jar), and a browser session named with As "buyer" is a client of its own too. A browser action recorded under an @api scenario fails the step and says so. Relative paths go to api.baseURL in saffron.config.json, or to the run's base URL when that is not set:

{
  "baseURL": "http://localhost:4173",
  "api": { "baseURL": "http://localhost:4173", "contract": "api/openapi.json" }
}

Most APIs want a token first. Sign in once in a fixture, and every scenario of the feature sends what the login returned:

@api
Feature: Orders API

  Fixture: Signed in
    Given I sign in through the API as "{env:API_USER}" with password "{env:API_PASSWORD}"

  Scenario: My orders are listed
    When I list my orders through the API
    Then the response status should be 200

The agent records the login as a request that keeps the token the response holds, with extract:

{
  "action": "request",
  "method": "POST",
  "path": "/api/login",
  "body": { "user": "{env:API_USER}", "password": "{env:API_PASSWORD}" },
  "status": "200",
  "extract": { "token": "/token" }
}

and the scenario's request sends it back as a header:

{
  "action": "request",
  "method": "GET",
  "path": "/api/orders",
  "headers": { "Authorization": "Bearer {captured:token}" },
  "status": "200"
}

The password stays an {env:...} value, resolved only when the request is sent. token names a credential, so the value the API handed out is masked as {captured:token} in the transcript, the report, the trace and the agent's own view, and is sent only to an API under test. A token a later capture replaces (a refresh, a poll that rotates it) stays masked, so a response that quotes the old one shows {captured:token} too. A login used by a single scenario is the same two steps inside it, with no fixture.

Checks written as exact phrases

In an API scenario, a Then written in one of a few phrases that each mean exactly one check is recorded as that check, with no agent: the agent only records the steps around it, and the proof replays it like any other.

Phrase	Check
`Then the response should answer 404` (also: answers, returns, responds with; `status 2xx`)	the latest response's status
`Then the /status field is active` (a word, `"quoted text"`, a number, `true`, `false` or `null`)	that JSON field of the latest response
`Then the response matches the Customer schema`	the latest response against that schema of the pinned contract
`Then every response in "registrations" should have status 200 or 202` (`in "..."` optional: the latest group)	each response of a request group answers one of the statuses
`Then at least one response should have status 200`	at least that many responses of the group answer it
`Then every 202 response should have a numeric "Retry-After" header` (or `a "Retry-After" header`)	that header of each response of that status
`Then every 200 response should have JSON field "/success" equal to true`	that field of each response of that status
`Then every 200 response should have a non-empty "/paymentReference"`	the field holds a value in each
`Then all 200 responses should have the same "/paymentReference"`	one non-empty value, shared by all
`Then every response should match the contract`	each response against its operation's schema for its status

Nothing else is guessed: any other wording, or one of these with more said ("is not active", "the response should answer 404 or 410": a single response takes one status, two fields in one step), goes to the agent as before. A group's "status 200 or 202" is the one phrase that lists several. This is an option, not a syntax to learn: every step can stay plain language. Like every check, a compiled one is never healed. It applies while step reuse is on (the default; --no-reuse turns both off).

Several requests at once

An idempotency test sends one operation several times at the same moment, each request with something of its own (a correlation ID), and checks every response, and the responses against each other. Write the requests as a table under one step: a header row, then a row per request, its first cell naming it:

@api
Feature: Concurrent no-show registration

  Background:
    Given no-show registrations use these shared request details:
      | reservation     | {data:no-show.chargeable} |
      | amount          | 25.5                      |
      | API key         | {env:NOSHOW_API_KEY}      |
      | idempotency key | {unique:registration}     |

  Scenario: Concurrent duplicates refer to the same payment
    When I send these no-show registrations concurrently as "registrations":
      | request | correlation ID  |
      | first   | {unique:first}  |
      | second  | {unique:second} |
      | third   | {unique:third}  |
    Then every response in "registrations" should have status 200 or 202
    And every response should echo its own request's correlation ID
    And at least one response should have status 200
    And every 200 response should have JSON field "/success" equal to true
    And all 200 responses should have the same "/paymentReference"

One recording, one request per row. The agent records the request once, as the group the step names; replay sends it once per row, every row before any answers. Each row's cell is recorded as <row:column> where the request sends it, so editing a cell sends the new value with no recording again. Adding, removing or renaming a row, or changing the header, records the step again: those are other requests. A <row:column> naming no column of the table fails the step instead of sending the token as text. The group's name may be an Examples column (as "<batch>"), and its checks then read that row's group.
Shared values. A value every row sends is written in the step, or in another step's table, such as the Background's above, which sends nothing itself. {unique:registration} resolves once per scenario run, so every row sends the same key, and {unique:first} and {unique:second} differ. {env:...} values stay tokens in the recording and masked in the transcript.
Every response is kept with its row, whatever order they answer in. A request that gets no response (a timeout, a refused connection) is kept too: the step fails once every request has settled, listing each row, and the transcript keeps the ones that answered. A group is never sent again on failure, and polls nothing: a hidden retry would change the count an idempotency test checks. The step that sends a group never fails on a status, as a single request does on one that is no 2xx: the group's Then steps check the statuses, so write one.
Checks say their scope: every response, the responses of one status ("every 202 response"), a named one ("the duplicate should echo ..."), at least some, or all of them sharing a field's value. A response's own cell is its own row's (every response should echo its own request's correlation ID). A check of "the response" after a group fails: it names none of them. A check filtered by a status holds when no response has that status, so pair "every 200 response ..." with "at least one response should have status 200"; the transcript says how many responses each check checked. A "same" check needs a value: an empty one in every response fails.

A failure names the row:

the group "registrations" fails every response of the group to have the header X-Correlation-Id "<row:correlation ID>": 1 of 3 failed
  first   200  passed
  second  202  failed: expected the X-Correlation-Id header to be "...", got "someone-else"
  third   202  passed

A group sends at most 20 requests: it is a few requests racing for one outcome, never a load test. Requests in flight together do not guarantee the server processes them at the same moment on every run; a test that needs "the duplicate arrives while the first is processing" needs the API under test to hold the first (a test delay or gate). saffron eject does not export a group yet.

Several APIs

A scenario can call more than one API: a customers API and an orders API, say. The default one is api.baseURL and api.contract; name the others under api.services, each with its own base URL and, optionally, its own pinned contract:

{
  "api": {
    "baseURL": "https://api.example.test",
    "contract": "api/openapi.json",
    "services": {
      "orders": { "baseURL": "https://orders.example.test", "contract": "api/orders.json" }
    }
  }
}

A step names the API it calls ("When I place an order through the orders API"), and the recording keeps the name, never the host: the request says "service": "orders" and a path relative to that API's base URL. The same recording then replays against whichever hosts the config names, so a run against staging points the APIs there for that run alone:

npx saffron run --filter @api --api-url https://staging-api.test --api-url orders=https://staging-orders.test

Each API keeps its own rules: an {env:...} value is sent only to the base URL of the API its request names, a schema check uses the contract of the API that answered, and a recording checked against a contract goes stale when that contract changes. Pin a named API's contract with npx saffron api pin <url or file> --service orders. A service name is letters, digits, - and _, starting with a letter, and every service needs a baseURL; a config that breaks either, or an --api-url naming a service the config does not have, stops the run (exit 2).

A recording that holds a request is at schema version 6: saffron-ai 1.1.0 and earlier ask to be updated rather than replay it. One that polls, sends a form or a file, keeps a response header, or checks a list's size, every entry, a bound, a pattern or a difference is at version 7, which the first API release asks to be updated for in the same way.

Pin the contract

Schema checks validate responses against an OpenAPI 3 document committed to the project, never one fetched at run time:

npx saffron api pin https://api.example.test/docs

The source can be a Swagger UI page (the document it loads is pinned), the document's own URL, or a file, as JSON or as YAML. A redirect (a docs page moved, http to https) is followed, up to five, and each address on the way is checked as the first is: none may be link-local. --service <name> pins a named API's contract (see "Several APIs" above). It is written as JSON to api/openapi.json (--out names another file); add it to saffron.config.json as shown above. A download that takes longer than 30 seconds, or a document over 20 MB, is refused. Saffron reads OpenAPI 3.0 and 3.1, as JSON or YAML (api.contract may name a YAML file too); convert a Swagger 2.0 document first. A 3.0 document's exclusiveMinimum: true beside a minimum is read as 3.0 means it. The agent is handed a summary of the contract when it records (each operation, its parameters, body fields and response codes), so it sends what the API accepts.

A change to the pinned document is a change to review. A recording whose checks were made against the old document is checked again against the new one on the next run: a zero-AI replay of the same recording, filed as a proposal (yellow) when it passes, and red, with the run exiting 1 and no proposal filed, when the API breaks the new contract. The proposal keeps the Examples row, bindings and data the recording was made with, whichever row ran the check, and the time it was made; the contract's fingerprint changes, and checkedAt records when it was proved again (step reuse ranks it by that time). A check that would change any action is red instead. No agent session runs either way, and saffron status lists such a recording as checked again on the next run. An unchanged scenario never passes green over a contract it was not checked against. Recording it again is saffron run --rerecord. The same holds when api.contract names another file, or none, even with the old file still there unchanged: a recording names the contract it was checked against, and is stale over any other.

When a contract change breaks a request many scenarios send (a body field the API now requires), write the step the same way everywhere: each scenario then holds a copy of one recorded request. The first heal's proposal, accepted with npx saffron accept <file> --propagate, fixes every copy, and the other scenarios replay it with no agent session of their own (§5). A copy checked against the old contract is still checked again against the new one, as above.

What the agent records

For each step the agent sends requests with Saffron's own tool, so the recording holds exactly what was sent:

request: the method, the path, the headers and the JSON body, with the step's values as the step writes them. An Examples column, a {data:...}, {env:...}, {date...} or {unique:...} token resolves at replay as it does in a browser step. A value written into the path is percent-encoded where it would change the URL: #, ?, a space, and in the query &, = and + too, so A#1 is sent as /items/A%231; a / and an existing %20 are sent as they are, and a token that opens the path (a captured link) is sent whole. A number or boolean that follows a row or data value stays a number or a boolean when it is sent: the recording keeps it as { "$value": "<seats>", "$type": "number" }. A form ("I sign in with the form") is sent as form, field to text, as application/x-www-form-urlencoded; an upload ("I upload the invoice") as multipart, field to text or to a file of the project by its path, { "file": { "$file": "data/invoices/sample.pdf" } }. A file outside the project is refused, and the transcript keeps its path and size, never its bytes.
until: a step that waits for a state ("until the export is done", "once the order is paid, within 10 seconds") records a GET that is sent again until its response has the expected status and satisfies the field checks in until.json, every intervalMs (default 1 s, 100 ms at the fastest) for at most timeoutMs (default 30 s, the step's own when it names one, 10 minutes at the longest). A poll that runs out is red; the run's retries never send it again. The step's checks then read the response that satisfied it, and every attempt stands in the transcript. A request may carry its own timeoutMs when a step allows more or less than the run's action timeout (a long export, a strict limit). Every duration (timeoutMs, maxMs, until.intervalMs, until.timeoutMs) is milliseconds, or a text with a unit that may carry the step's tokens: "2 s", "500 ms", "<seconds> s". A bound an Outline row sets (within <seconds> seconds) is recorded as its column with the unit, so each row keeps its own; a bare number recorded on one row would hold for every row, which is why the agent writes the text. Only a request that may run again polls: GET, HEAD, OPTIONS, or one marked idempotent; a POST with until is refused before it is sent. A Then is never sent again instead: the wait belongs to the When that reads.
concurrent and statuses: a request sent several times at once, every copy before any answers, for a step about a double submission or a race (When I submit the booking twice at once, Then one is created and the other refused): "concurrent": 2, "statuses": ["201", "409"]. The statuses are checked in any order; without them every copy must answer a 2xx. The step's checks read the copy that answered the first status. Such a request never polls, and saffron eject does not export it yet. When a copy gets no response, the step fails once every copy has settled, and the transcript keeps the ones that answered.
group: a request sent once per row of its step's data table, as the group of that name, every row before any answers (see "Several requests at once" above). The first column names each request; <row:column> is its own cell. It takes no status, until, extract or concurrent: its responses are checked by expectGroup. It takes no idempotent either: a group is never sent again.
expectGroup: the responses of a group (group, absent the latest), each one, those answering status, or the members named, holding one check: statuses (one of them), json, header with value or pattern, matchesContract (with schema naming a component), or same, a JSON Pointer at which all hold one non-empty value. atLeast asks only that many to hold it. A recording with either is schema version 19.
extract: response fields kept for later steps, name to JSON Pointer ({ "customer": "/id" }), or a response header by name ({ "created": "header:Location" }), so a Location an API answers a creation with is the path of the next request. A pointer segment that spells a column the step names (/plans/Basic/id in I keep the <plan> plan's id) is recorded as that column, /plans/<plan>/id, so each Examples row keeps its own field. From then on the value is written as {captured:customer} wherever a later request repeats it, in a path, a header or a body, so replay uses the id its own request got. A browser step can use it too: Then the page should show "Customer {captured:customer}". A browser visit to a captured link (a sign-in URL a request returned) is recorded as {captured:name} too, even when the link is on the base URL, so replay follows the link its own request got.
A whole JSON payload in an Examples column. When a step names a column whose cells are JSON (When I place the order <payload>, with {"sku":"A-1","qty":2} on one row and {"sku":"B-7","qty":5} on the next) and the request's body, or a part of it, is that JSON, the body is recorded as the column, so each row sends its own payload as JSON. Before, the payload was kept as fixed fields and every other row was refused. A body that only resembles the cell is recorded as sent, and ties the recording to its row. The proposal names the body each row sends (row requests: step 1 ("I place the order <payload>") sends POST /orders with each row's own body: row 1 {...}; row 2 {...}), and a cell that is not JSON fails its row, naming the column. The cell is read as JSON before any token in it is filled in: a {env:...} inside a string stays that string's text, and a token outside any string ({"qty": {captured:count}}) is one JSON value of its own, so a secret can never add a field. The same holds in exported tests.
The expected status, when the request's step names it, or the checks right after it do ("When I read customer 7" then "Then the customer is not found"): its code, its reason or its class, in words that speak of a response ("should answer 404", "is refused with Conflict", "fails with a 4xx"), after "a", "with", "get" or "receive" ("I get a 404"), or as a reason of several words ("is not found"). A number in "I order 450 units" or "a crate with 500 units" is no status: a number followed by a noun of its own is a quantity, while one followed by its reason, a word for a status, the end of the sentence or a word that carries it on ("returns 201 with the new customer") is a status. A response with another status than the one expected is not recorded either. Without one, any 2xx replays, and a request that got another status is not recorded: the agent sends it again expecting that status only when the step names it, and fails the step otherwise. A status the step does not name is refused as an expectation before anything is sent, so a failing API never becomes a recording that expects the failure.

A Then step records what it checks on the latest response of its session (a Then that needs a request of its own sends it first):

expectStatus: the status, exact or a class.
expectJson: JSON field checks by pointer, with a type for numbers, booleans and null, exists for presence and item for an entry of a list wherever it stands, as the browser's response checks are written. For the whole of a list, count (exactly that many items) and every (checks that hold on each item; an empty list fails, so "every order is paid" never passes over no orders). For a value's shape, differs (not this value, read as its type; with no type, the number 5 and the text "5" are the same, as a captured id is kept as text), pattern (text matching a regular expression) and min and max (a number within bounds, either or both; the value as the API wrote it and the bound as the check does are compared exactly, so a number JavaScript cannot hold as written (9007199254740993.0, 9007199254740991.1, 9.007199254740993e15, 1e9999999999) is never rounded onto the bound). Each is text with the step's tokens, resolved at replay like equals, and the browser's response checks take them too. A row value in a pattern is escaped and kept whole: ^Order <id> for 1 never matches "Order 12". An exported test refuses one there. A row value that leaves a pattern or a differs empty is refused, as it would match any value, and so is an expected number with more digits than a number holds (0.10000000000000001): check it as a string. A group's same compares numbers exactly, however large. In a request body, a whole number beyond 2^53 sent as a JSON number (a captured int64 id, a row's <id>) is refused, as it would be sent rounded to another one: send it as a string.
expectHeader: a response header by name, equal to a value, matching a regular expression, or present ("the response is JSON", "it sets a Location"). Header names are matched whatever their case, and a row value in the pattern is escaped and kept whole, as in a JSON pattern.
expectDuration: the response arrived within so many milliseconds ("within 2 seconds"), measured from sending the request to its headers and body being read.
expectSchema: the body against the pinned contract, the schema the contract gives the operation's response, or a named component schema. A response the contract defines once under components.responses and references with $ref is checked against that definition's schema. A response the contract gives no body (a 204) passes when it has none. The operation is looked up with or without the path of api.baseURL (/api/v1) and of the document's servers. The formats JSON Schema names (date-time, email, uuid, uri and the others) are checked; a format the document invents is not.

Saffron verifies each check against the response before it records it and refuses one that does not hold, so a recorded check is one that held.

While recording, a write (POST, PUT, PATCH, DELETE) is sent only to a path the pinned contract lists or the step names ("I start the job at /api/job/start"); one to a path the agent guessed is refused before it is sent, since a guessed DELETE that exists deletes. The agent looks around with GET, HEAD and OPTIONS only, and a response field that is a credential is hidden from it unless the step keeps it with extract. When an endpoint the step needs is outside the contract and the step does not name it, the step fails and says so: name the path in the step, or pin a contract that lists the endpoint. A project with no contract constrains nothing: pin one to constrain the agent's writes.

A request's JSON body is read against the contract too: when the contract gives the operation a request schema and the body does not match it, the agent's reply says so and what is wrong. A body the API refused is a note, since a step may send a bad body on purpose and expect the 4xx (named with the status it expects). A body it accepted all the same is not recorded: an attempt the API ignored part of (a guessed field, "false" as text) would be sent on every replay. When the contract is out of date, pin it again with saffron api pin.

The body is judged as sent, its tokens resolved: {date+1} as a date, and {captured:id} as the text it sends, so an integer field takes the number itself ("customerId": 42), which the recording keeps as its token.

A step keeps one request per operation: a request sent again to the same method and path right after the step's last one takes its place, so the agent's earlier tries are not replayed. A step that asks for several ("twice", "two customers", "2 customers", "for each plan", "once more") keeps them all, and so does a request that kept a value (extract) the next one does not, one sent again expecting another answer (a second DELETE that must answer 404), one that sent a value the step names and the next does not ("set the currency to SEK and turn guest names on" is two writes), one followed by the same fields with other values the step does not name ("add a mug and a hat": product 17, then 23), and one sent again with the same idempotency key.

A credential goes once, where the step puts it: a request sending its Authorization (or API key) header's credential in its body or URL as well is refused while recording, unless the step names both the body and the header.

Replay rules

A request that may not run twice is never retried. GET, HEAD and OPTIONS get the configured retries; every other method gets one attempt, so a POST that times out fails the step instead of creating a second record. A recording can say otherwise with "idempotent": true on a request.
A proof replay is a second execution, with {unique:...} values of its own: a scenario that creates records names them with a unique token, as the example does, and each execution creates its own. Clean up with the project's teardown command.
A value the API hands out that is a credential, read from the words of its capture's name or its field (token, /access_token, sessionId, api_key, accessKey, privateKey, authorization, csrf, signature, nonce, refresh, a Set-Cookie header), or a captured object holding one (a key whose last word names a secret, such as accessToken, authorization, pwd, otp or api_key; never one that only mentions it, such as sessionTitle, pinColor or total_tokens), each such token masked on its own (a value of 8 characters or more, or a code of 4 to 8 digits under otp or pin; a short word such as "auth": "yes" is no secret), is kept like an {env:...} value: it shows as {captured:token} in the transcript, the report, the agent's own view of the response and every error, and a request that carries it is sent only to an API under test, the default one or a named one (api.services), whichever answered the login. One aimed elsewhere is refused before it is sent. A browser step that carries one (a goto, a field it fills) acts only on the base URL's origin, a frame inside the page included: an embedded frame of another origin never gets it, judged by the origin each document reported when it was made, before its own scripts ran, nor a sandboxed frame or page whose origin is opaque (a sandbox attribute or Content-Security-Policy), nor any frame inside one, nor a page loaded before the run began watching, and a key press with no target or a dialog's answer never carries one. A browser step's error and the execution trace show it as its token, as they show an {env:...} value. It is masked in every spelling, a JSON-escaped one too. An id or a name captured the same way travels freely.
Secrets travel as tokens: Authorization: "Bearer {env:API_TOKEN}" is resolved when the request is sent and masked back in the recording, the proposal, the report and the transcript. A request that carries an {env:...} value, whether the step, a row's cell or a data value wrote it, is sent only to the API's base URL (api.baseURL, else the run's): one aimed elsewhere, by the agent or by a committed recording, is refused before it is sent. A {captured:...} value is used as plain text, so a token inside what a response said never resolves.
No request goes to a link-local address (169.254.x.x, fe80::), where a cloud keeps its metadata service and credentials, written out or behind a host name that resolves to one, looked up for every request.
A multipart file is a file of the project: a path outside it, a link inside it that leads out, a link that leads to a hidden file, a hard link, or a hidden file or folder (.env, .git) is refused, and a request with a file goes only to an API under test.
A credential the agent kept is written as {captured:name} wherever the agent sends it again: a header, a JSON body, a form field or a multipart text part.
While recording, the agent sends requests only to the APIs under test (api.baseURL and api.services): a URL a response or a page puts in front of it is refused before anything is sent.
A redirect is a response, never followed: a 3xx is what the request got, so a step that expects one names it ("is redirected", a 3xx status), checks Location with a header check, or keeps it with extract header:Location and reads it in the next step. A test that follows redirects silently would pass over a redirect it should have noticed.
A request that gets no response (a timeout, a refused connection) fails with what went wrong, never with Playwright's call log, which lists the request's headers and cookies.
When the app or API under test does not answer at all (nothing listens on its port), no agent session starts to record or heal: the scenario is red with "no agent session was started" and why, so a server left down costs no tokens. This holds for browser scenarios too. Only the APIs a scenario names are checked (its recording's, or one its steps call an API, "the orders API" or "the orders service"; a word alone such as "orders" names none), so another API being down blocks nothing, and behind a proxy (HTTPS_PROXY and the like) no check is made.
A Then that fails after every request before it answered as recorded is the API's answer: no agent session starts for it, since a Then is never healed. The scenario is red, saying so; fix the app, or change the step and record it again.
A scenario that runs again sends its requests again: a proof replay, a heal's replay, and in a parallel run (--workers above 1) a red scenario that runs once more with the agent. In a browser matrix an @api scenario runs in the first browser's pass only.
An Outline row never passes on another row's data: a body, a path, a header, a schema's name or an extract pointer that holds the recorded row's value is checked the way a typed value is. A body's key or a header's name follows the row only in a step that names the column (I clear the <field> field); elsewhere a key such as name is the API's field, whatever a row holds. An array index in a pointer (/items/1/sku) is never written as a column; in a step that names a column with that value, it binds the recording to its row. A pointer still holding a row value the step does not name binds the recording to its row, and another row fails with a message instead of reading the wrong field.
A number or boolean in a body or a JSON check follows a row's value only in a field named like the column (gift for <gift>), or as the one such value in a step that names the column. {gift: false, express: false} keeps express as recorded, and the recording is bound to its row: another row is refused with a message, never sent with express set to its own value. Where the step writes the column itself ({admin: <admin>, active: true}), a value under a name unrelated to the column is a constant: kept as recorded, without binding the row.

Requests and responses, after the run

Saffron keeps a transcript of every request the HTTP family sent in a scenario, with the step each was sent under: the method, the path, the status, the time it took, and the headers and bodies, secrets masked. A credential header (Authorization, Cookie, Set-Cookie, an API key or a CSRF token) is kept only when all it says is an {env:...} token, as in Bearer {env:API_TOKEN}; a login's cookie or a captured token shows as [credential, not kept]. The report lists them per step, each line opening onto the request and the response with JSON pretty-printed, and saffron trace replays them (§18): an API scenario opens from its transcript, with the selected request and response where a browser replay shows the page, and a browser scenario that called the API lists each step's requests in its Details pane, each opening onto its headers and bodies.

Nothing is drawn while a scenario runs. A request viewer that renders inside the runner during execution, as some Cypress plugins do, lags and freezes runs with large responses; Saffron renders the transcript afterwards, from the file, so the only cost during a run is keeping the text, and api.transcript in saffron.config.json sets how much of it:

Value	What is kept
`"on-failure"` (default)	the transcript of a scenario that is not green, as `requests.json` in its evidence folder; each proof replay's beside it as `proof-1-requests.json`, `proof-2-...`, which the report and `saffron trace` show under the proof's name and outcome, the run's own first
`"on"`	every scenario's, green included, so the report shows what a passing API test did
`"off"`	for a scenario that is not green, the lines alone: method, path, status and timing, no headers or bodies, and each check without the error it quoted

Any other value stops the run (exit 2) and names the three.

Beside its requests, a step's checks are kept as it made them (✓ expect the response status 201, ✗ with its error for one that did not hold), so the report and the trace viewer show what was checked under each step, not only what was sent.

A transcript travels with the report in a CI bundle like a screenshot does.

Calling the API from a browser scenario

A browser scenario can call the API directly in a step ("I create a customer through the API"). The request goes through the browser session's own client, so it carries the cookies the UI signed in with, and its extracted values reach the browser steps that follow. Set api.contract to check such responses against the contract too.

The agent gets the API tools (saffron_request and the response checks) and the contract's summary in a browser scenario only when the project sets api in saffron.config.json ("api": {} is enough) or the scenario's replay already called the API, so a project that never calls it records as before. A missing or broken api.contract fails only the schema checks that need it there; the browser steps run as before.

Export to Playwright Test

saffron eject (§22) writes API steps as calls on the exported helper's request client: the request, its status, JSON field, header and duration checks, extractions and polling. A schema check (expectSchema) is exported too: eject reads the pinned contract of the API the request went to and writes the schema it checks against into the spec, self-contained (every $ref it reaches copied beside it), as a SCHEMAS list at the top of the file. For a check that names no schema, that is every response the operation gives, by status, so the exported test picks the one for the status it got, as replay does, and holds a response the contract gives no body to having none. The helper validates with ajv and ajv-formats, with the options replay uses, and loads them only when a schema check runs: a project with schema checks installs them, as eject says when it writes one.

npm install --save-dev ajv@^8 ajv-formats@^3

The schemas are the contract as it was when you ejected: eject again after the contract changes.

Fixtures: steps a feature runs once

A feature whose every scenario needs the same thing first (a signed-in client, a seeded customer) declares it once, in a Fixture: block at the top of a .saffron file tagged @api:

@api
Feature: Customers API

  Fixture: A seeded customer
    Given a customer named "seed-{unique:name}" exists through the API

  Scenario: The seeded customer can be read
    When I read the seeded customer through the API
    Then the response status should be 200

The runner performs the fixture once per run, before the first scenario of its feature, and the values its requests keep with extract reach every scenario as {captured:name}, read-only: a scenario's own extract under the same name shadows the copy in that scenario alone. A fixture is recorded, proved, healed and accepted like a scenario, its recording beside the feature's others under .saffron/cache, its Then steps never healed. A credential it captured (a login's token) is masked in every scenario's transcript, report and agent session like one the scenario captured.

A fixture's verdict is every scenario's floor: a red fixture (a failed step, a stale recording with no agent, a recording that did not prove) makes every scenario of the feature red with the fixture's error and its steps skipped; a fixture whose proposal is pending makes a green scenario yellow and names the proposal. The fixture has no row of its own in the report. The workers of one run share one fixture run (the parallel phase and the agent phase each run it once, since the first runs without an agent); a shard is a run of its own and runs the fixture itself; a proof replay of a scenario never re-runs the fixture.

The rules: .saffron files only (Gherkin gives Background no tags, so a .feature file that uses Fixture: gets the rename advice a StepSet gives); the feature must be @api; one fixture per file, at the feature level, named, with no Examples, and not sharing a scenario's name. A fixture may invoke StepSets. Fixtures shared between features are a StepSet invoked from each feature's fixture. A browser fixture whose cookies the scenarios inherit is not supported.

Reusable named operations are StepSets. A StepSet step that sends a request and keeps a value with extract (a customer named <name> exists through the API) is a named operation: invoked by name from any scenario or fixture, recorded once, seeded from the step index everywhere else, and its result read as {captured:name}. See STEP-SETS.md.

Beyond the medium

What the API medium leaves to other tools, on purpose:

GraphQL works as a POST with a JSON body and field checks under /data and /errors; there is no query-aware tooling.
Generating tests from the contract, negative cases and fuzzing: that is what Schemathesis and Specmatic do, deterministically, from the same OpenAPI document you pin. Run one of them beside Saffron; Saffron's scenarios say what the API must do for your users, a fuzzer says what it must not crash on.
Mock servers and service virtualization: WireMock, Prism and Microcks serve a contract; Saffron tests the real API.
gRPC, WebSocket, server-sent events and message queues are other transports, each a medium of its own; none is planned.
Fuzzy whole-object matching in the style of Karate's #ignore: Saffron checks a response's shape with the pinned contract and its values with verified field checks instead, so a relaxed match never records a check that did not hold.

Not yet

saffron eject writes a scenario that reads a fixture's values as test.fixme. A browser fixture with inherited cookies is a design of its own.