Why Saffron

End-to-end tests written in plain language, with zero-token cached replay and runtime AI healing.

Free to use · npm: saffron-ai · site

Write tests in plain language, as .saffron files or standard .feature files. .saffron is Saffron's own test language and goes further than Gherkin: step sets for reusable flows, {env:VAR} tokens that keep secrets out of recordings and reports, and {data:...} references to data files. There are no step definitions and no glue code. On the first run, an AI agent (Claude by default, or Codex, Antigravity or Cursor on the subscription you already have, driving Playwright MCP) executes each scenario in a real browser and records what it did into a cache: a human-reviewable JSON action list committed to git. Every run after that, Saffron replays the cache in a real browser: zero AI calls, zero tokens, at the speed of the browser and your app. The cache is Saffron's own JSON, not a generated script; replay runs on Playwright's browser engine, and Playwright MCP is only how the agent operates the browser while it records or heals.

When the UI changes and a cached step fails at runtime, the agent takes over mid-execution with full scenario context, adapts, and finishes the run, then files a cache proposal you accept or reject, like a snapshot test update. Reports state exactly what was adapted and what it cost.

The rules that keep it honest

• Assertions are sacred. By default no Then is ever healed: the AI may help reach an assertion, never make it pass; enforced mechanically, not by prompt. Projects may opt into adaptable-mid for mid-scenario checkpoints, but the final assertion block of a scenario is strict under every policy, forever.
• Three result states. Green = cached pass. Yellow = passed with AI adaptation (pending your review). Red = failed.
• Diagnose before adapting. On a step failure the agent first decides: UI drift (heal) or application defect (fail with a diagnosis).
• Caches are git artifacts. Proposals show diffs, including a suggested feature-file edit when the written steps no longer match reality, and nothing is committed without you.
• Verified proposals. Every recording/heal is proof-replayed zero-AI before it's filed (with one bounded refinement pass on failure), so proposals arrive stamped verified ✓ or honestly UNVERIFIED ✗.
• Honest cost reporting. Reports break out prompt-cache reads/writes (the real bill of agent sessions) next to the in+out token count, per scenario and in totals, and name the models used. On an API key the dollars are what was billed; on a Claude subscription the headline is the 5-hour plan window before and after the run, with the dollars as the API-equivalent.
• Your subscription, your choice of agent. Recording and healing run on the plan already logged in on the machine: Claude (default), Codex on a ChatGPT plan, Google AI Pro/Ultra through the Antigravity CLI (experimental), or Cursor. Set "provider" in the config; each one runs through its vendor's official client, and every provider records through the same recorder, so the caches are identical in kind.
• Trend memory. Every run appends to .saffron/history.jsonl; reports show deltas vs the previous run and 20-run sparklines, and Saffron flags chronic scenarios (healing repeatedly: re-record instead of paying again) plus recurring failure themes.

Step reuse: new scenarios get cheaper as your suite grows

Like a maturing Cucumber suite, most of a new feature file is steps you already have. Saffron derives a step index from committed caches and, when recording a new scenario, replays every seeded island of known steps zero-AI. The agent records only the genuinely novel steps, wherever they sit. Exact step text is the identity (Cucumber-style); quoted values may differ ("admin" seeds from "bob"); data-table steps seed on matching keys. Measured on a live site: a 6-step scenario with one novel step recorded for $0.15 / 6 AI calls, vs $2.07 for a full unseeded recording of comparable length.

Step sets (shipped): .saffron files add the StepSet: keyword for named, reusable step sequences invoked with StepSet <name> inside any scenario. Sets expand at parse time, so their steps cache and seed like ordinary steps; editing a set makes every invoking scenario honestly stale (re-recorded mostly seeded), and heal edits route to the set definition, one fix, every caller follows. Sets are project-wide: keep application-wide flows in a sets-only library file (convention: features/shared.steps.saffron) and invoke them from any feature.

IDE integration & authoring (shipped): saffron steps lists the project vocabulary with recorded/divergent/unrecorded badges (--snippets for native VS Code completion), saffron mcp serves it to AI assistants, saffron lsp brings completion/navigation/diagnostics to JetBrains (incl. Community editions via LSP4IJ) and Neovim, saffron author drafts feature files from prose in your own vocabulary, and duplicate wordings that record identical actions get behavior-proven rename proposals. The Saffron VS Code extension is on the Marketplace (code --install-extension ChathurangaJayasinghe.saffron-vscode). JetBrains users get the same in one click from the Saffron JetBrains plugin (Marketplace, source): .saffron file type, bundled grammar, LSP4IJ wiring. Both editors also run from the IDE: right-click Run on .saffron files and folders, Saffron run configurations (JetBrains), and a Saffron panel with feature files, tags to run, proposals to accept or reject, vocabulary health, config, and the run report as an in-editor dashboard, all fed by saffron status --json.

Data tables and doc strings are first-class: 2-column key/value tables parameterize the recording (<table:username>), multi-row record tables parameterize per cell (<table:1:firstName>), and """ doc strings record as <docstring>, so editing values or content replays at zero tokens, while structural changes (keys, headers, row counts) honestly re-record. Unambiguous params are scenario-wide, so a later assertion on a note's text follows content edits too. Secrets stay out of everything: {env:VAR} resolves from the environment (or a git-ignored .env) at replay, recordings and reports are masked back to the token, and missing variables fail fast by name.

API tests (beta)

The same plain-language scenarios test a REST/JSON API with no browser: tag a feature @api and the agent records each step as requests with typed bodies and kept values, and checks of status, fields, headers, timing and the schema of a pinned OpenAPI contract (saffron api pin). Replay sends the requests again at zero tokens, a write never twice, and a changed contract makes the recordings that checked it stale. Fixture: blocks set up once per feature, browser scenarios can call the API too, and secrets and the credentials an API hands out stay out of recordings, reports and traces. See "API tests" in the guide. API test automation is in beta: it works, and it is still being shaped by use.