Why Saffron

Gherkin-native test runner with zero-token cached replay and runtime AI healing.

Free to use · npm: saffron-ai · site

Write tests as plain .feature files. On the first run, an AI agent (Claude Agent SDK + Playwright MCP) executes each scenario in a real browser and records what it did into a cache. A human-reviewable JSON action list committed to git. Every run after that, Saffron replays the cache in a real browser: zero AI calls, zero tokens, at the speed of the browser and your app. The cache is Saffron's own JSON, not a generated script; replay runs on Playwright's browser engine, and Playwright MCP is only how the agent operates the browser while it records or heals.

When the UI changes and a cached step fails at runtime, the agent takes over mid-execution with full scenario context, adapts, and finishes the run, then files a cache proposal you accept or reject, like a snapshot test update. Reports state exactly what was adapted and what it cost.

The rules that keep it honest

• Assertions are sacred. By default no Then is ever healed: the AI may help reach an assertion, never make it pass; enforced mechanically, not by prompt. Projects may opt into adaptable-mid for mid-scenario checkpoints, but the final assertion block of a scenario is strict under every policy, forever.
• Three result states. Green = cached pass. Yellow = passed with AI adaptation (pending your review). Red = failed.
• Diagnose before adapting. On a step failure the agent first decides: UI drift (heal) or application defect (fail with a diagnosis).
• Caches are git artifacts. Proposals show diffs, including a suggested .feature edit when the written steps no longer match reality, and nothing is committed without you.
• Verified proposals. Every recording/heal is proof-replayed zero-AI before it's filed (with one bounded refinement pass on failure), so proposals arrive stamped verified ✓ or honestly UNVERIFIED ✗.
• Honest cost reporting. Reports break out prompt-cache reads/writes (the real bill of agent sessions) next to the in+out token count, per scenario and in totals, and name the models used. On an API key the dollars are what was billed; on a Claude subscription the headline is the 5-hour plan window before and after the run, with the dollars as the API-equivalent.
• Trend memory. Every run appends to .saffron/history.jsonl; reports show deltas vs the previous run and 20-run sparklines, and Saffron flags chronic scenarios (healing repeatedly. Re-record instead of paying again) plus recurring failure themes.

Step reuse: new scenarios get cheaper as your suite grows

Like a maturing Cucumber suite, most of a new feature file is steps you already have. Saffron derives a step index from committed caches and, when recording a new scenario, replays every seeded island of known steps zero-AI. The agent records only the genuinely novel steps, wherever they sit. Exact step text is the identity (Cucumber-style); quoted values may differ ("admin" seeds from "bob"); data-table steps seed on matching keys. Measured on a live site: a 6-step scenario with one novel step recorded for $0.15 / 6 AI calls, vs $2.07 for a full unseeded recording of comparable length.

Step sets (shipped): .saffron files, a superset dialect of Gherkin, add the StepSet: keyword for named, reusable step sequences invoked with StepSet <name> inside any scenario. Sets expand at parse time, so their steps cache and seed like ordinary steps; editing a set makes every invoking scenario honestly stale (re-recorded mostly seeded), and heal edits route to the set definition, one fix, every caller follows. Sets are project-wide: keep application-wide flows in a sets-only library file (convention: features/shared.steps.saffron) and invoke them from any feature.

IDE integration & authoring (shipped): saffron steps lists the project vocabulary with recorded/divergent/unrecorded badges (--snippets for native VS Code completion), saffron mcp serves it to AI assistants, saffron lsp brings completion/navigation/diagnostics to JetBrains (incl. Community editions via LSP4IJ) and Neovim, saffron author drafts feature files from prose in your own vocabulary, and duplicate wordings that record identical actions get behavior-proven rename proposals. The Saffron VS Code extension is on the Marketplace (code --install-extension ChathurangaJayasinghe.saffron-vscode). JetBrains users get the same in one click from the Saffron JetBrains plugin (Marketplace, source): .saffron file type, bundled grammar, LSP4IJ wiring. Both editors also run from the IDE: right-click Run on .saffron files and folders, Saffron run configurations (JetBrains), and a Saffron panel with feature files, tags to run, proposals to accept or reject, vocabulary health, config, and the run report as an in-editor dashboard, all fed by saffron status --json.

Data tables and doc strings are first-class: 2-column key/value tables parameterize the recording (<table:username>), multi-row record tables parameterize per cell (<table:1:firstName>), and """ doc strings record as <docstring>, so editing values or content replays at zero tokens, while structural changes (keys, headers, row counts) honestly re-record. Unambiguous params are scenario-wide, so a later assertion on a note's text follows content edits too. Secrets stay out of everything: {env:VAR} resolves from the environment (or a git-ignored .env) at replay, recordings and reports are masked back to the token, and missing variables fail fast by name.