CI and review

CI

- run: npx playwright install chromium --with-deps
- run: npx saffron run --no-agent --strict   # replay only: fast, free, deterministic
# or, in a job that may heal, gating on review:
- run: npx saffron run --strict              # a heal waiting for review exits 1
  env:
    ANTHROPIC_API_KEY: ${{ secrets.ANTHROPIC_API_KEY }}

A browser or an API in another place

When the browser runs in a container, or the API sits behind a proxy, tell Saffron how to reach them under transport in saffron.config.json:

{
  "transport": {
    "browser": { "cdpEndpoint": "http://browser:9222" },
    "api": { "proxy": "http://proxy.internal:8080", "ignoreHTTPSErrors": true, "headers": { "X-Tenant": "{env:TENANT}" } }
  }
}

browser.cdpEndpoint: a Chromium already running with remote debugging. Saffron connects to it instead of launching one, and it records and replays, as the agent's browser tools attach to it too.
browser.wsEndpoint: a Playwright server (npx playwright run-server) of any engine. It replays only: recording needs a CDP endpoint.
api: every API request goes through proxy, accepts a self-signed certificate with ignoreHTTPSErrors, and carries headers. The headers go only to an API under test, never to another address a request names; each {env:NAME} in them is read when the request is sent, one that is not set is refused instead of sent empty, and a value that is not text is refused. A value read from {env:...}, or written whole for a header whose name ends in a credential's word, a version aside (Authorization, X-Api-Key, X-Session-Id, X-Hub-Signature-256, Cookie, X-API), is a secret, masked wherever it would show; Accept: application/json, X-Session-Name and X-Api-Version are not. A browser scenario's API requests carry these headers as an @api scenario's do.
module: a module of the project that decides these at run time, for a setup that starts a container or looks its address up. Its default export has either hook, or both, and each returns what the settings above would hold (or nothing, to keep them):

// saffron-transport.mjs, with "transport": { "module": "saffron-transport.mjs" }
export default {
  async browser({ browser, headed }) {
    return { cdpEndpoint: await startBrowserContainer() };
  },
  async api() {
    return { proxy: process.env.API_PROXY };
  },
};

Coverage per pull request

A passing suite says nothing about what a change removed. Tag each scenario with the requirement it covers (@REQ-12, @WA-TC1: any tag matching requirementTag), and saffron coverage maps each requirement to its scenarios, the checks their recordings make and the last run's verdict. saffron coverage --write saves that map as .saffron/coverage.json; commit it beside the recordings. In a pull request build, compare it with the target branch's:

- run: npx saffron coverage --base origin/main --markdown --fail-on-loss

It lists the scenarios added and removed, the requirement IDs no scenario covers any more, and each check a recording no longer makes (a weakened check), read from the recordings themselves. --fail-on-loss exits 1 when anything was lost, --markdown gives text to post on the pull request and --json gives it as data. Nothing is inferred: a requirement is a tag you wrote, never a guess.

CI posture

--strict turns yellow into a failing exit code, so builds stay red until a human reviews each adaptation. The repo ships a GitHub Actions workflow that runs the unit suite plus a no-AI E2E proof: zero-token replay, broken assertion → red, stale cache → red, heal propagation → green.

CI heals travel to the reviewer. A heal in CI used to be a dead end: its proposal was filed in a workspace thrown away with the job. saffron export packs the pending proposals, the run's report and the screenshots and traces it names into one zip, with a manifest of where the run happened (repository, branch, commit, CI run and attempt, shard), and CI uploads it as an artifact. saffron import on the reviewer's checkout refuses a bundle from another repository, of a newer format, or with a file that does not match its digest, then checks every proposal's binding against the files there: a proposal made against a different scenario text or committed recording is refused by name, the rest are filed byte for byte, so the revision the reviewer accepts is the one CI filed. The imported run becomes the latest report, linked to its CI run, and saffron trace replays it; shard bundles merge into one report. Accepting leaves an ordinary working-tree change to commit and push; no second agent session is paid for.

From either CI, in either editor. saffron runs lists the branch's runs from GitHub Actions (through gh) or Azure Pipelines (through az), in one shape, with how many of each run's proposals are pending here, and saffron import --run <id> downloads a run's bundles and imports them. The VS Code CI runs view and the JetBrains CI Runs tab call these two commands and nothing else, so neither editor knows which service ran the tests; the CI tool keeps the credentials. Imported proposals carry the run they came from (importedFrom in saffron status --json), shown in both editors' Proposals.

Look before you import. saffron report --run <id> shows a CI run's results without changing your checkout: the run's bundles are downloaded once into .saffron/ci/ (kept for the newest 10 runs, up to 1 GB), checked as an import would check them, and each proposal is judged against your files, importable or not and why, with nothing filed. saffron trace --run <id> replays its traces, view only. A run without results says why (still running, expired, or no bundle uploaded), and importing afterwards reuses the download. In both IDEs, opening a run's row shows its totals, its red and yellow scenarios and its proposals, with Open Report and Open Trace.

What a run left, and a run as one page (1.0.0). On GitHub, saffron runs marks each run from a single listing of the repository's artifacts, joined to the runs by workflow_run.id (five pages at most, a courtesy: when gh cannot say, no marker): results and their size, a summary only, expired, none, or still running. saffron export --report-only writes a saffron-summary (report and screenshots, no proposals or traces, summary: true in its manifest) that viewing falls back to when a run has no full bundle and import refuses. Viewing a run writes it as one page beside its report (panel.html, named as panelHtml in the --json, and printed by saffron report --run <id> --panel), in the report's design, screenshots embedded; the editors read that file; buttons call window.saffronAct(action, feature, scenario, browser), which each editor defines (a webview message in VS Code, a JCEF query in JetBrains), the way the dashboard's saffronOpen works.

Or CI opens a draft pull request. saffron export --branch <name> commits the proposal files, not the recordings, onto the commit the run tested (a pull request's head, not the merge commit CI checked out) as a new branch, without touching the CI checkout, pushes it, and on GitHub opens a draft pull request for the tested branch through gh (or az for an Azure Repos repository). The description holds each proposal's verification, the agent's account and the diff, all in code blocks so nothing a page or an agent wrote can mention anyone or render as markup, and the commands to accept. The reviewer switches to the branch, accepts, commits the recordings and marks it ready. An existing branch is never pushed onto.