Saffronv1.3.4
Get started
Core concepts

The execution model

Your .saffron and .feature files Gherkin, parsed into scenarios, with step sets expanded. Is there a committed cache for it? YES NO, OR STALE EVERY RUN · 0 TOKENS Zero-AI replay Saffron replays the cached actions in a real browser. GREEN all steps pass RED a Then fails: never healed ACTION FAILS AFTER RETRIES UI MOVED · AI HEALS Agent heals in place Same browser session, from the failed step on. FIRST RUN · AI RECORDS Known steps replay Wordings recorded before are reused, zero AI. Agent records the gaps Drives the real browser, only for the new steps. PROOF BEFORE REVIEW Candidate cache What the agent did, as replayable JSON. Zero-AI proof replay The recording must replay on its own to count. One refine pass Then re-verified. FAILS RE-VERIFY Proposal Verified or unverified, with a narrative and a diff. YELLOW YOU REVIEW saffron accept Nothing is applied without you. Committed cache JSON in git, reviewed like a snapshot. NEXT RUN REPLAYS FOR FREE Every run writes an HTML and JSON report status · AI calls · tokens · cache traffic · cost or plan usage · adaptations · feature diffs
Saffron execution model

Every scenario ends in one of three states:

State Meaning Exit code
● green Replayed from cache deterministically 0
● yellow Passed, but the AI had to record or adapt. A cache proposal awaits review 0 (1 with --strict)
● red Failed: assertion false, goal unreachable, or no cache with AI disabled 1

Zero-token replay

Saffron replays a cached scenario itself, in a real browser. The cache is a JSON file of recorded actions, not a generated script: Saffron's replayer reads each action and performs it on Playwright's browser engine, which is where the auto-waiting comes from. On top of that: per-action retry with backoff, and locator candidates tried in order (semantic first, CSS fallback last). No agent is loaded and no MCP server runs. How long a scenario takes depends on how many steps it has and how fast your app answers; Saffron adds no waiting of its own. If the feature file's text no longer matches the cache, the cache is stale and the scenario re-records rather than replaying the wrong thing.

AI mode: recording and healing

This is the only place Playwright MCP is involved. The agent (Claude Agent SDK by default, or another provider, see below) operates the browser through Playwright MCP's tools, attached over CDP to the same browser the replayer uses (with trace on, the MCP runs inside Saffron on the scenario's own browser context, so its calls land in the trace), and Saffron turns what it did into cache actions. That matters for healing: when a cached step fails mid-run, the agent inherits the exact page state at the point of failure and continues from there: runtime healing, not a post-run repair loop.

  • The recorder normalizes the agent's MCP tool calls into cache actions and groups them per Gherkin step; bookkeeping tools (saffron_step, saffron_capture, saffron_assert, saffron_done) give the agent a structured way to announce steps, record captures/assertions it has verified against the live page, and finalize with a narrative.
  • In heal mode the agent must diagnose before adapting: legitimate UI drift → adapt minimally and report the deviation; application defect → fail with an explanation. Bounded retries run before any AI is invoked, so transient flakes don't burn tokens.
  • Heals say what kind of change they adapted to. The agent is told the page replay stopped on and must name the change in its adaptation notes and narrative: a locator or label change on the same page, a page removed or merged into another, a page added or the flow reordered, or timing only. Saffron does not rely on that account alone: it compares the healed recording with the committed one and adds its own flow change: notes for every click, key press, goto or tab move a step no longer performs (or newly performs), preceded by the page replay stopped on. A button rewritten to a new name is paired with the old click and reads as a locator change in saffron diff, not as a flow change.
  • The agent takes over the live page, and only the steps it owns. Before a heal, a refinement or a seeded recording segment, Saffron closes the pages of its other browser contexts (the agent attaches over CDP and would otherwise see them as extra tabs) and tells the agent which tab replay left off on, and at which URL. Steps that already ran before the session keep their recording. A heal or refinement records nothing until the agent announces the failed step: what it does before that is diagnosis and is discarded (it is told to look with snapshots, not clicks). A seeded recording segment files early actions under the first step it may record and drops an early tab selection. Anything filed under an earlier step is discarded. Each case leaves a recording: note in the adaptations.
  • Session failures (turn limits, process errors) fail only their scenario, never the suite.
  • Deliberate snapshots (snapshotMode: "none", the default): tool responses carry no automatic page snapshot; the agent snapshots only when it needs to see the page, and batches tool calls per turn. A/B on a live site: 42→17 AI calls, 3.5M→1.3M cache-read tokens, $2.07→$1.16 per recording, with the proof replay passing first try.
  • A pass cannot leave an assertion unchecked. saffron_done with success is refused once while a Then step the agent announced has recorded no check, and every refusal from Saffron's tools is an MCP error, so an agent that fires several calls at once (Codex runs them as one script) stops at the refusal and fixes it inside the session instead of filing a proposal the proof replay would mark unverified. A second pass stands, so an agent that cannot express a check does not burn its turns; the proof replay still marks that recording unverified.