Saffronv1.3.4
Get started
Editors and agents

Editor setup

Everything below is optional: Saffron runs fine from the CLI alone, but completion from your own step vocabulary is what keeps wording (and token spend) under control.

VS Code

Full experience: the Saffron extension, on the VS Code Marketplace: search "Saffron" in the Extensions view, or

code --install-extension ChathurangaJayasinghe.saffron-vscode

You get .saffron syntax highlighting (first-class StepSet), Ctrl/Cmd+click on {env:VAR} tokens to open the variable in .env (or .env.example), step completion with ●/●/○ status badges, StepSet completion, go-to-definition on invocations, hover cards, and diagnostics (unknown/duplicate set names, near-duplicate wording warnings).

Run from the editor. Right-click a .saffron file or a folder of them (Explorer or editor) and pick Saffron: Run, Run (replay only, no AI), Run (headed browser) or Re-record; the play button in the editor title runs the open file. Commands run as a VS Code task in the workspace folder that owns the file, using that folder's own saffron-ai install (npx otherwise). Arguments reach the process directly, so a file name is never read as shell syntax. The Command Palette adds Run all feature files, Open latest report and Accept all pending proposals.

Run one scenario. Above each Scenario: line, in .saffron files and in the .feature files of a project with a saffron.config.json, three lenses: ▶ Run scenario, Replay only and Headed. A click runs that scenario alone, by name (saffron run <file>:<name>), as a task in the project that owns the file; on a Scenario Outline it runs every row. The file is saved first, since the runner reads it from disk, and the panel's replay-only and headed toggles apply as they do to every run. Scenarios are found in the file's own Gherkin language (# language:). Needs saffron-ai 0.9.8 or later. VS Code's editor.codeLens setting hides all lenses.

The Saffron side panel. The Saffron icon in the activity bar opens two views. Feature files lists every .saffron file with its scenarios as children (click one to jump to it), inline Run and Replay-only buttons per file, checkboxes with Run selected, and Run all. Run options and status holds the replay-only and headed toggles, which apply to every run started from the extension, the last run's totals with the report one click away, and the proposals pending review with Accept all. It refreshes as files, reports and proposals change. Five more sections read saffron status --json: Last run (the scenarios that failed or healed, with the step that failed; click one to open the screenshot of the page at that moment, use the inline action to jump to the scenario, or Open Replay to step through its execution trace, §18; the screenshot opens as a copy checked against its run, named after the scenario and kept in the extension's own storage until the window closes, and the row's tooltip gives the file in the project), Tags (tick and Run tagged), Proposals (tick, hover for the narrative, proof-replay verdict and cost, then Accept selected, Reject selected or Accept all; one a later run replaced since the panel listed it is refused, with a prompt to refresh; click one to open its scenario; its ... menu accepts one ticked proposal's StepSet steps for reuse (accept --step-set), opens a draft pull request with the pending proposals after asking for a branch (export --branch), exports the review bundle (export), and writes the recordings as Playwright specs after asking, for the open file or every one (eject)), Health and config (vocabulary counts, divergent steps, duplicate wordings proven by identical recordings, the effective config with one click to the file, recent pass rates) and Orphaned recordings (caches and proposals whose scenario no longer exists, with the reason; click one to open it, or use the title action to run saffron prune --yes after a confirmation). CI runs lists the current branch's GitHub Actions or Azure Pipelines runs and imports one run's heals into the checkout with a click (§20). The Last run title bar has Open Replay (every scenario the run traced, handy when everything passed and no row is listed) and Open Report (the run's HTML report in the browser). Saffron: Open dashboard shows the suite's quality dashboard (the page saffron dashboard renders, §6) in an editor tab, refreshed after each run; click a scenario's name to open it.

No-extension fallback (two commands):

// settings.json: Gherkin highlighting for .saffron
{ "files.associations": { "*.saffron": "feature" } }

npx saffron steps --snippets   # native completion from your vocabulary

Re-run --snippets after recording sessions to keep completions fresh.

JetBrains (IntelliJ IDEA, WebStorm, PyCharm, Rider, incl. Community)

The Saffron plugin (JetBrains Marketplace; source at https://github.com/s-chathuranga-j/saffron-jetbrains-plugin) makes this a one-click install, like the Cucumber plugin: it registers the .saffron file type with highlighting from the bundled grammar, and wires saffron lsp through the free LSP4IJ plugin, when it is installed, for *.saffron and *.feature: completion badges, StepSet go-to-definition, hover, diagnostics. The only prerequisite is saffron-ai in the project (npm i -D saffron-ai); the plugin runs the project-local binary and shows a hint if the package is missing.

Running from the IDE. Four ways, all landing in the Run tool window:

  • Run configurations: Run/Debug Configurations → + → Saffron. run takes files or folders, tags, and replay-only / headed / re-record switches plus extra arguments; report opens the latest report; accept and reject take proposal files (blank means --all); prune lists the recordings no scenario owns any more; export writes the review bundle (--branch <name> in extra arguments opens a draft pull request); eject writes Playwright specs, for the files named or every one; trace opens the execution replay of the scenario named in Scenario: a scenario name or feature:scenario, as it is or wrapped whole in double quotes; blank opens every traced scenario, starting on a failed one. Tick Store as project file to share a configuration through .run/.
  • Right-click → Run on a .saffron file (project view or editor) or a folder holding them. The configuration it creates stays in the run widget for re-runs. In the editor with the caret inside a scenario, it runs that scenario alone (Run 'Successful login'), LSP4IJ or not.
  • Run lenses above each Scenario: line, with LSP4IJ installed: ▶ Run scenario, Replay only and Headed run that scenario alone, by name (saffron run <file>:<name>; every row of an outline). A click saves the files and runs a temporary configuration made for the scenario, named after it (Run 'Successful login'), which stays in the run widget. A saved configuration of that name is never changed: the lens makes its own beside it. Scenario runs need saffron-ai 0.9.8 or later.
  • The Saffron tool window (right side, in projects with a saffron.config.json), eight tabs. Files: every feature file with its scenario count, search, tick files and Run Selected, Run All, Open Report, Accept All Proposals, the last run's totals and the number of proposals pending review; double-click opens a file. Last Run: what failed or healed in the last run, with the failed step; double-click opens the screenshot of the page at that moment (a copy checked against its run, named after the scenario and kept in the IDE's system folder for a day), and Open Replay steps through the scenario's execution trace (§18); with no row selected (a run where everything passed lists none) it opens every scenario the run traced. Open Report opens the run's HTML report in the browser. Proposals: tick proposals, read the narrative, proof-replay verdict and cost, then Accept Selected, Reject Selected or Accept All; one a later run replaced since the tab listed it is refused, never decided unseen, and comes back unticked on refresh. An imported proposal names the CI run it came from. Accept a StepSet's Steps files one ticked proposal's StepSet steps for reuse; Open Draft Pull Request asks for a branch, then pushes the pending proposals to it; Export Bundle writes the review bundle; Eject to Playwright asks, then writes specs for the open file or every one. CI Runs: the current branch's GitHub Actions or Azure Pipelines runs; Import Run brings one run's heals into the checkout (§20). Tags: tick tags and Run Tagged. Health: vocabulary counts, divergent steps, duplicate wordings proven by identical recordings, the effective config with one click to the file. Orphans: caches and pending proposals whose scenario no longer exists, grouped and explained; double-click opens one, Remove All runs saffron prune --yes after a confirmation. Dashboard: the suite's quality page that saffron dashboard renders (§6, the same one VS Code and the browser show); click a scenario's name to open it. The tabs read saffron status --json; the Dashboard needs saffron-ai 0.9.4 or later.

Runs use the login shell's environment, so Node from nvm or Homebrew is found even when the IDE was started from the Dock.

Install it from Settings → Plugins → Marketplace (search "Saffron"), or from the listing; install the free LSP4IJ plugin too for completion, go-to-definition and diagnostics. It is optional, so Saffron also installs on IDE builds LSP4IJ has not reached yet.

Manual setup (any JetBrains IDE, no plugin): two separate pieces, the LSP gives you intelligence, a TextMate bundle gives you colors:

  1. Syntax highlighting: Settings → Editor → TextMate Bundles → + and pick the bundle shipped inside the package: node_modules/saffron-ai/textmate/saffron.
  2. Completion, diagnostics, hover, go-to-definition, install the free LSP4IJ plugin (Red Hat), then Settings → Languages & Frameworks → Language Servers → +: command npx saffron lsp, working directory = your project root, file patterns *.saffron and *.feature. (On IntelliJ Ultimate 2023.2+ the native LSP API also works.)

npx saffron lsp is not something you run by hand, it is a stdio language server the IDE launches. Run it in a terminal and it prints exactly these instructions (with the bundle's absolute path).

You get the same completion badges, StepSet go-to-definition, hover, and diagnostics as VS Code: served by saffron lsp, which is backed by Saffron's real parser.

A project nested inside a bigger folder. If the IDE is opened above the Saffron project (a solution folder with the tests in Tests/E2E, say), each .saffron file is resolved against the nearest saffron.config.json above it, so StepSets, {data:...} references and completion still come from the right project. Running from the tool window still expects the project itself to be the IDE's root.

Neovim

vim.api.nvim_create_autocmd("FileType", {
  pattern = { "cucumber" },
  callback = function()
    vim.lsp.start({ name = "saffron", cmd = { "npx", "saffron", "lsp" } })
  end,
})
vim.filetype.add({ extension = { saffron = "cucumber" } })

The server also offers Run lenses above each scenario, to a client that asks for them with init_options = { runLenses = true } in vim.lsp.start. Their command, saffron.runScenario, is the client's to run: its argument names the project root, the target (features/login.saffron:Successful login, or a line for a scenario without a usable name), the scenario and the mode (run, replay for --no-agent, headed for --headed); run npx saffron run <target> with those flags in root.

AI assistants (any editor)

Assistants can't read completion popups: serve them the vocabulary over MCP so generated feature files reuse your exact wordings:

claude mcp add saffron -- npx saffron mcp     # Claude Code

Cursor/Copilot/JetBrains AI Assistant: add an MCP server with command npx saffron mcp in their respective MCP settings. Six tools are exposed: search_steps, list_step_sets, project_status (the same overview as saffron status --json: what is cached, pending review, tagged, and the vocabulary health), run_plan (what a run of the suite would replay and record, as saffron run --plan --json says it), api_contract (the operations of the pinned OpenAPI contract, for writing @api steps) and verify. Every tool reads saffron.config.json on each call; while the file cannot be read (saved half way, a stray comma), the tools answer on the config they last read and add configProblem, saying what is wrong with it.

verify is for a coding agent that changed the app or the tests: it replays the scenarios it names, or the whole suite, with saffron run --no-agent, so nothing is recorded, repaired or accepted, and answers in one versioned shape (saffron-verify/1): the commit and whether the working tree differs from it, the environment, each scenario's verdict with the checks it executed and its evidence, what it could not verify (not recorded, stale, refused or blocked), the proposals waiting for a person, and specChanges against the reviewed baseline: the coverage map (§9) committed at baseRef (HEAD by default). A check a recording no longer makes, a removed scenario or a requirement left uncovered shows there even when every replay is green, so an agent cannot weaken a test quietly. And for whole files from prose: npx saffron author requirements.txt.

The authoring skill applies when creating, translating, or editing Saffron scenarios in .saffron or .feature files, browser and @api alike (its references/api.md covers API tests). It does not start an authoring workflow for application changes or requests only to run tests, inspect reports, review proposals, or change configuration. Inspect the relevant browser flow, reuse recorded wording, use explicit capture names, and write waits against observed UI or API signals. Each scenario should test one behavior with its own baseline and final assertions.

saffron author produces a draft from requirements and vocabulary; it does not inspect the browser or validate execution. Supply observed labels, loading signals, and prerequisites in the requirements. A working test requires recording the affected scenarios, reviewing their verified proposals for faithful assertions, accepting those proposals, and passing the affected scenarios with --no-agent. Parsing alone is insufficient. For draft-only requests or unavailable execution, state which validation remains. After upgrading the package, rerun saffron init to refresh the installed skill and managed project instructions. A skill you edited is replaced by the shipped one too, but kept first under .saffron/backups/skills/, and init says where.

Claude Code: the Saffron plugin

Claude Code users get the language server, the MCP server and the skill in one install, from the public marketplace repo:

/plugin marketplace add s-chathuranga-j/saffron-claude-plugin
/plugin install saffron@saffron

With the plugin, Claude sees diagnostics for .saffron and .feature files after every edit (unknown or duplicate StepSet names, a missing StepSet keyword, near-duplicate wordings) and can jump to StepSet definitions and {env:VAR} variables, the way an IDE does. The MCP tools and the skill are the same ones saffron init installs per project, so run npx saffron init --no-mcp there to avoid registering the server twice. The plugin pins the saffron-ai version it was built from and starts it through npx.

The plugin metadata and README link to the documentation, data handling statement, license terms and support.

Codex: the same plugin

The plugin repo is a Codex marketplace too:

codex plugin marketplace add s-chathuranga-j/saffron-claude-plugin
codex plugin add saffron@saffron

Codex gets the MCP server and the skill. Codex plugins have no language server slot, so diagnostics and StepSet navigation stay Claude Code only. As with Claude Code, run npx saffron init --no-mcp in a project that uses the plugin so the MCP server is not registered twice.

The plugin runs locally and requires Node.js 22.12 or later. Its MCP tools read the current project's vocabulary and status. Recording and healing also require Playwright browsers and a configured AI provider, with any AI charges on your own account. The public repository above is the installation source; official directory approval is a separate step.

AI agents: the bundled skill

The npm package ships an Agent Skill, skills/saffron/SKILL.md plus reference sheets, that teaches any skill-aware agent (Claude Code, Codex, Cursor, Copilot, Gemini CLI, …) how to write Saffron tests well: reuse the recorded vocabulary before inventing wordings, keep Then steps as the sacred verdict, {env:VAR} for secrets, StepSet syntax, network waits, page furniture, and the run → review → accept workflow. Install it into your project with:

npx saffron init                       # .claude/skills + .agents/skills (default)
npx saffron init --agents claude,agents,copilot,cursor

init also registers the saffron mcp server in each host's project-scoped config: .mcp.json (Claude Code), .cursor/mcp.json, .vscode/mcp.json (Copilot/VS Code): merged into whatever servers are already there, so assistants get search_steps, list_step_sets, project_status and api_contract with no manual setup (Codex has no project-level file; init prints the one-liner). And it appends a short managed block to AGENTS.md (created if absent) and to CLAUDE.md when one exists, pointers that tell agents to load the skill and never hand-edit caches. Everything is idempotent; re-run after npm update saffron-ai to refresh the copy. --no-instructions / --no-scaffold / --no-mcp narrow it. Projects using npx skills-npm discover the skill from node_modules automatically.