Agent upgrade check

Agents update often, and an update can quietly change how well they handle your code. Keep a small set of fixed tasks, run them after each upgrade, and compare against the last run.

Build a task set

Save three to five representative tasks as prompt files under ~/.beswarm/prompts: a bug fix, a small feature, a refactor, a test-writing task. Each should end with a check the agent must run, such as npm test or cargo test.

beswarm prompt list   # note the ids

Run the set

Each task gets a fresh worktree, so every run starts from the same commit:

beswarm worktree add --name check-3 --workspace api
beswarm worktree add --name check-4 --workspace api
beswarm worktree add --name check-5 --workspace api
beswarm terminal list   # find each worktree's terminal id

Start one agent per task in its worktree's terminal:

beswarm agent start --name check-3 --agent claude <terminal-id> --group upgrade --prompt-id 3 --no-focus
beswarm agent start --name check-4 --agent claude <terminal-id> --group upgrade --prompt-id 4 --no-focus
beswarm agent start --name check-5 --agent claude <terminal-id> --group upgrade --prompt-id 5 --no-focus

Then wait for all of them:

for id in 3 4 5; do
  beswarm agent wait "check-$id" --until done,blocked --timeout 3600
done

Keep the results

Ask each agent to finish by writing its outcome — did the check pass, what changed, anything surprising — to ../../reports/upgrade/<version>/check-<id>.md (the repo root's reports/; the agents run inside .worktrees/). Put the agent's version in the folder name, so runs accumulate side by side.

Compare against last time

beswarm agent start --name compare --agent codex --group upgrade \
  --prompt "Compare reports/upgrade/<new-version>/ with reports/upgrade/<old-version>/. For each task, did it get better, worse or stay the same? Flag any task that used to pass and now fails. Write to reports/upgrade/compare.md." \
  --wait --timeout 1200

If a task regressed, you know before it hits real work. Pin the old version while you decide.

Clean up with beswarm agent close --group upgrade and worktree remove for each check-*.