CI and review

Heal pull requests

For teams that review in pull requests rather than in an editor, CI can open the review itself. saffron export --branch <name> writes the bundle as usual, then commits the pending proposal files (the proposals, not the recordings) onto the commit the run tested as a new branch, pushes it, and opens a draft pull request for the branch the run tested. The reviewer reads each heal in the pull request and accepts it on a checkout of the branch: no agent session on their machine, and no second heal.

What the pull request holds

Its title says how many proposals it carries and for which branch: Saffron: 2 proposals to review for main.
Its branch is one commit on top of the commit the run tested, adding the proposal files under .saffron/proposals/. The recordings in .saffron/cache/ are untouched: nothing is accepted until someone runs saffron accept.
Its description names the run it came from (linked to the CI run), the branch and commit, and the commands to review and accept. Then, for each proposal: its feature and scenario, whether it is a heal or a first recording, its verification (verified ✓: a zero-AI proof replay passed; UNVERIFIED ✗: the proof replay failed, and why; or not verified, when proof replays were off), the agent's account of what it did, and the diff of the recording a reviewer is approving. Everything a page or an agent wrote sits in a code block, so it cannot mention anyone, link anywhere or render as markup. A description that would pass the service's limit (65,536 characters on GitHub, 4,000 on Azure DevOps) stops before it, and says how many proposals to see with saffron diff on the branch.

On GitHub Actions

permissions:
  contents: write        # push the heal branch
  pull-requests: write   # open the draft pull request
steps:
  - uses: actions/checkout@v4
  # ... install, start the app ...
  - run: npx saffron run --strict
    env:
      ANTHROPIC_API_KEY: ${{ secrets.ANTHROPIC_API_KEY }}   # the agent's key
  - name: Open a draft pull request with the heals
    if: failure()
    env:
      GH_TOKEN: ${{ github.token }}
    run: npx saffron export --out saffron-bundle.zip --branch "saffron/heal-${{ github.run_id }}-${{ github.run_attempt }}"
  # The same run, for review in an IDE or the terminal too (§19, §20).
  - uses: actions/upload-artifact@v4
    if: failure()
    with:
      name: saffron-bundle
      path: saffron-bundle.zip
      overwrite: true      # a re-run replaces the bundle instead of failing the upload

The job heals only with the agent's key or login (ANTHROPIC_API_KEY for Claude, CURSOR_API_KEY for Cursor; Codex and Antigravity need a login on the machine, §19).

Let the workflow open pull requests. Settings, Actions, General, "Allow GitHub Actions to create and approve pull requests", on the repository or its organization. Without it the branch is pushed, the pull request is refused, and the job says why.
Give each run's branch its own name. A branch that already exists on origin is never pushed onto, so the name carries the run id and the attempt. In a sharded run add the shard (-${{ matrix.shard }}): one pull request per shard that healed.
Forks. A pull request from a fork gets a read-only token, so its build cannot push a branch. The uploaded bundle still carries the heals.
Checks on the heal branch. A pull request opened with the workflow's own token starts no workflows (a GitHub rule against loops). The reviewer's push of the accepted recordings does. A build of the heal branch before that finds proposals its machine did not file (§5): it reports their scenarios yellow, pending review, until the proposals are accepted. It does not replay them, and it does not heal again.

On Azure Pipelines

For a repository on Azure Repos the pull request opens with az repos pr create, as the pipeline's build service. Give it what that needs:

Check out with persistCredentials: true, so the job can push.
Grant the project's build service Contribute, Create branch and Contribute to pull requests on the repository (Project settings, Repositories, the repository, Security).
Give az its azure-devops extension and the job's token:

trigger:
  branches:
    exclude:
      - saffron/heal-*   # the heal branch must not start this pipeline again
pr: none               # pull request builds stay replay-only (§9)

steps:
- checkout: self
  persistCredentials: true
# ... install, start the app ...
- script: npx saffron run --strict
  displayName: Run the suite
  env:
    ANTHROPIC_API_KEY: $(ANTHROPIC_API_KEY)   # the agent's key: a secret variable, mapped
- script: |
    az extension add --name azure-devops
    npx saffron export --out $(Build.ArtifactStagingDirectory)/saffron-bundle.zip --branch "saffron/heal-$(Build.BuildId)-$(System.JobAttempt)"
  displayName: Open a draft pull request with the heals
  condition: failed()
  env:
    AZURE_DEVOPS_EXT_PAT: $(System.AccessToken)
- task: PublishPipelineArtifact@1
  condition: failed()
  inputs:
    targetPath: $(Build.ArtifactStagingDirectory)/saffron-bundle.zip
    artifact: saffron-bundle-$(System.JobAttempt)

The trigger leaves the heal branches out: pushing one would otherwise start this pipeline on it, where the same scenarios fail, the agent heals them again and opens another pull request. pr: none does the same for the heal pull request on a GitHub repository; validate pull requests with a replay-only pipeline (§9). The job heals only with the agent's key or login, mapped into the run step as above (§19), and the artifact name carries the attempt so a re-run can publish its own (§19, Re-runs).

Azure Pipelines support was tested against a stand-in az, not yet in a real Azure DevOps organization: please report problems.

A GitHub repository built by Azure Pipelines opens its pull request on GitHub, through gh: check out with persistCredentials: true so the job can push, and give gh a GH_TOKEN that may open pull requests.

Elsewhere

On another CI service, or for a repository on another host, the branch is pushed and no pull request is attempted. The description is written to saffron-pull-request.md beside the bundle, for a pull request step of your own.

Review and accept

The commands are in the description, with a cd into the project first when it lives in a subfolder of the repository:

git fetch origin saffron/heal-1234-1 && git switch saffron/heal-1234-1
npx -y -p saffron-ai saffron diff
npx -y -p saffron-ai saffron accept --all
git add -A .saffron/cache && git add -u .saffron/proposals && git commit -m "Accept Saffron proposals" && git push

Then mark the pull request ready for review, and merge it as usual.

npx -y -p saffron-ai saffron names the package: the project's own install when there is one. In a checkout without node_modules, a bare npx saffron would fetch an unrelated npm package named saffron.
accept --all leaves UNVERIFIED proposals out: read one, and accept it by file (npx -y -p saffron-ai saffron accept <file>) if it is right, or reject it.
saffron accept checks each proposal's binding again and refuses one whose scenario or recording changed since it was filed.
The last line stages exactly what accept changed: the new recordings, and the proposal files it removed. -u leaves out any other proposal pending on your machine, and works where .saffron/proposals/ is git-ignored. A replay you ran to check the heal adds a line to .saffron/history.jsonl, which is no part of the heal and stays out.
You can accept in your IDE instead: once you have switched to the branch, its proposals are pending in your checkout, and the Proposals view or tab lists them (§11).

What happens, exactly

Where the branch starts. On the commit the run tested. On a pull request build that is the head of the pull request's branch, not the merge commit CI checked out, and the draft pull request goes to that branch: a pull request into the pull request, so merging it updates the original. A shallow checkout fetches the head first. The branch only ever starts on the checkout's HEAD, a commit HEAD merged in, or one it descends from: anything else is refused before anything is fetched or pushed. --base <branch> picks another target, and is needed after a tag build, which tested no branch.
What does not fit is left out. Each proposal's binding is checked again against the commit the branch starts on: its scenario as written there, the recording committed there, and the StepSets and data folder as that commit has them, not as the CI checkout does. One that does not fit is left out of the branch, with the reason. When none fits, nothing is pushed.
Nothing else moves. The commit is made beside the checkout, which stays on the commit and branch it was on, and it holds the proposal files only, even where .saffron/proposals/ is git-ignored.
Who the commit is by. Git's configured name and email when set; otherwise github-actions[bot] on GitHub Actions, and Saffron elsewhere.
The bundle comes first. export writes the bundle before it makes the branch, so a branch that cannot be pushed still leaves a bundle to upload.

What it prints:

bundle: saffron-bundle.zip
  2 proposal(s) · 6 screenshot and trace file(s) · the run's report
  from run github:1234 · main @ 1a2b3c4 · github.com/acme/shop
Review it in a checkout of this repository with: saffron import <bundle>
branch: saffron/heal-1234-1 pushed to origin (9f8e7d6: 2 proposals on top of 1a2b3c4)
draft pull request: https://github.com/acme/shop/pull/57

Exit	Means
0	The branch is pushed and the draft pull request is open; or no pull request was attempted (another host) and the description file is written; or there were no pending proposals, so no branch
1	The branch is pushed, but the pull request could not be opened: the reason is printed, and the description is in `saffron-pull-request.md`
2	No branch was pushed, and the reason is printed in one line. The bundle is written, unless the export itself failed (an `--out` that is a folder, say)

When it does not work

Message	What to do
`branch ... already exists on origin`	Give each run its own branch: the run id and attempt, and the shard in a sharded run
`cannot tell which branch the pull request is for: pass --base <branch>`	The run tested no branch (a tag, or a detached checkout): add `--base main`
`no proposal fits ..., the commit the branch would start on`	Every proposal was made against a scenario or recording that commit does not have. Review the uploaded bundle with `saffron import` instead (§19)
`cannot push ... to origin`	The job may not push: `contents: write` on GitHub; `persistCredentials`, Contribute and Create branch on Azure DevOps
`the commit to build on, ..., is not in this clone and could not be fetched`	The job could not fetch the pull request's head from origin: check that it may read the repository
`gh pr create failed: ...`	Usually the repository setting above, or a missing `pull-requests: write`
`az repos pr create failed: ...`	The build service lacks Contribute to pull requests, or `AZURE_DEVOPS_EXT_PAT` is not set
`the gh CLI is not installed here`	Install `gh` on the agent (GitHub's hosted runners have it)
`... is not a branch name ...`	Use a name git accepts, not starting with `-` or `@`
`a branch name with a space, a quote or any of ...`	The reviewer pastes `--branch` into a shell: use letters, digits, `/`, `-`, `_` and `.`