Saffronv0.7.1 · free to use
GitHub
Core concepts

The execution model

Your .saffron and .feature files Gherkin, parsed into scenarios, with step sets expanded. Is there a committed cache for it? YES NO, OR STALE EVERY RUN · 0 TOKENS Zero-AI replay Saffron replays the cached actions in a real browser. GREEN all steps pass RED a Then fails: never healed ACTION FAILS AFTER RETRIES UI MOVED · AI HEALS Agent heals in place Same browser session, from the failed step on. FIRST RUN · AI RECORDS Known steps replay Wordings recorded before are reused, zero AI. Agent records the gaps Drives the real browser, only for the new steps. PROOF BEFORE REVIEW Candidate cache What the agent did, as replayable JSON. Zero-AI proof replay The recording must replay on its own to count. One refine pass Then re-verified. FAILS RE-VERIFY Proposal Verified or unverified, with a narrative and a diff. YELLOW YOU REVIEW saffron accept Nothing is applied without you. Committed cache JSON in git, reviewed like a snapshot. NEXT RUN REPLAYS FOR FREE Every run writes an HTML and JSON report status · AI calls · tokens · cache traffic · cost or plan usage · adaptations · feature diffs
Saffron execution model

Every scenario ends in one of three states:

State Meaning Exit code
● green Replayed from cache deterministically 0
● yellow Passed, but the AI had to record or adapt. A cache proposal awaits review 0 (1 with --strict)
● red Failed: assertion false, goal unreachable, or no cache with AI disabled 1

Zero-token replay

Saffron replays a cached scenario itself, in a real browser. The cache is a JSON file of recorded actions, not a generated script: Saffron's replayer reads each action and performs it on Playwright's browser engine, which is where the auto-waiting comes from. On top of that: per-action retry with backoff, and locator candidates tried in order (semantic first, CSS fallback last). No agent is loaded and no MCP server runs. How long a scenario takes depends on how many steps it has and how fast your app answers; Saffron adds no waiting of its own. If the .feature text no longer matches the cache, the cache is stale and the scenario re-records rather than replaying the wrong thing.

AI mode: recording and healing

This is the only place Playwright MCP is involved. The agent (Claude Agent SDK) operates the browser through Playwright MCP's tools, attached over CDP to the same browser the replayer uses, and Saffron turns what it did into cache actions. That matters for healing: when a cached step fails mid-run, the agent inherits the exact page state at the point of failure and continues from there: runtime healing, not a post-run repair loop.

  • The recorder normalizes the agent's MCP tool calls into cache actions and groups them per Gherkin step; bookkeeping tools (saffron_step, saffron_capture, saffron_assert, saffron_done) give the agent a structured way to announce steps, record captures/assertions it has verified against the live page, and finalize with a narrative.
  • In heal mode the agent must diagnose before adapting: legitimate UI drift → adapt minimally and report the deviation; application defect → fail with an explanation. Bounded retries run before any AI is invoked, so transient flakes don't burn tokens.
  • Session failures (turn limits, process errors) fail only their scenario, never the suite.
  • Deliberate snapshots (snapshotMode: "none", the default): tool responses carry no automatic page snapshot; the agent snapshots only when it needs to see the page, and batches tool calls per turn. A/B on a live site: 42→17 AI calls, 3.5M→1.3M cache-read tokens, $2.07→$1.16 per recording, with the proof replay passing first try.