Workspace runner: execute independent verification before review #5

Merged
arthur merged 1 commit from feat/workspace-verification into main 2026-09-14 20:51:16 +00:00
Owner

Workspace runner: execute independent verification (verification-v1)

The runner half of console PR z3k-tech/console#29. After project-rebase and project-review have prepared fresh detached checkouts, and before any reviewer agent starts, the runner now executes the project's member-approved verification commands and reports what it observed.

What changes

  • project-verify (new): asks the console for the commands bound to the current snapshot, refuses to run if the console's snapshot hash differs from delivery-snapshot.json in the review checkouts, starts a run, executes each command with sh -c from that repository's checkout root, and submits exit code, duration, output SHA-256, the last 4000 characters and the log name. The submit call uses the run's one-shot jsv_… token only; the agent token is never sent there and is stripped from the command environment along with CONSOLE_URL.
  • Outcome handling in card-runner: passed or skipped continues to the reviewer as before. failed moves the card back to ready with no reviewer started; the console has already commented the failing output on the card for the builder's next launch. blocked or a run that could not start moves the card to waiting for a person.
  • Deadlines: the run stops at the console's per-run minutes or the card's bounded deadline, whichever is earlier. A command still running at that point is killed by process group and reported as timeout; commands not yet started are reported timeout too. Exit 126/127 is reported as blocked, never as a failure of the change.
  • No behaviour change for projects without verification enabled: the console answers "not enabled" and the runner skips.

Verification

WORKSPACE_TEST_SCRIPTS=$PWD python3 -B workspace/test_delivery_loops.py: 29 tests pass, including three new ones with an in-memory console: observed exit codes and tails are posted with the run token only and the agent credential is absent from the command environment; a deadline kills a slow command and marks it and the rest timeout; not-enabled, already-passed, snapshot-mismatch and out-of-checkout repositories never start a run. The image build runs this suite before publishing.

Limits

Commands run inside the Studio pod with its filesystem and network; this is not a hostile-process sandbox, and workspace Git credentials remain reachable to a command that looks for them. Logs are kept under the review directory's verification/ folder and are not uploaded. The runner must not print secrets into command output, since tails are stored as sent.

## Workspace runner: execute independent verification (`verification-v1`) The runner half of console PR z3k-tech/console#29. After `project-rebase` and `project-review` have prepared fresh detached checkouts, and before any reviewer agent starts, the runner now executes the project's member-approved verification commands and reports what it observed. ### What changes - **`project-verify`** (new): asks the console for the commands bound to the current snapshot, refuses to run if the console's snapshot hash differs from `delivery-snapshot.json` in the review checkouts, starts a run, executes each command with `sh -c` from that repository's checkout root, and submits exit code, duration, output SHA-256, the last 4000 characters and the log name. The submit call uses the run's one-shot `jsv_…` token only; the agent token is never sent there and is stripped from the command environment along with `CONSOLE_URL`. - **Outcome handling in `card-runner`:** `passed` or `skipped` continues to the reviewer as before. `failed` moves the card back to `ready` with no reviewer started; the console has already commented the failing output on the card for the builder's next launch. `blocked` or a run that could not start moves the card to `waiting` for a person. - **Deadlines:** the run stops at the console's per-run minutes or the card's bounded deadline, whichever is earlier. A command still running at that point is killed by process group and reported as `timeout`; commands not yet started are reported `timeout` too. Exit 126/127 is reported as `blocked`, never as a failure of the change. - **No behaviour change** for projects without verification enabled: the console answers "not enabled" and the runner skips. ### Verification `WORKSPACE_TEST_SCRIPTS=$PWD python3 -B workspace/test_delivery_loops.py`: 29 tests pass, including three new ones with an in-memory console: observed exit codes and tails are posted with the run token only and the agent credential is absent from the command environment; a deadline kills a slow command and marks it and the rest `timeout`; not-enabled, already-passed, snapshot-mismatch and out-of-checkout repositories never start a run. The image build runs this suite before publishing. ### Limits Commands run inside the Studio pod with its filesystem and network; this is not a hostile-process sandbox, and workspace Git credentials remain reachable to a command that looks for them. Logs are kept under the review directory's `verification/` folder and are not uploaded. The runner must not print secrets into command output, since tails are stored as sent.
feat(workspace): run approved verification commands before the reviewer starts
All checks were successful
build-workspace / workspace (push) Successful in 29m31s
6dbdff909e
project-verify fetches the member-approved commands from the console, runs them in the fresh detached review checkouts with the agent credential stripped from the environment, and posts observed exit codes, durations, output digests and tails with the run's one-shot token. A failed run sends the card back to the builder without starting a reviewer; a blocked or unrunnable one waits for a person. Projects without verification are unaffected.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01DxRveKF16xrVKUGR5Wpd8x
arthur merged commit 6dbdff909e into main 2026-09-14 20:51:16 +00:00
Sign in to join this conversation.
No reviewers
No labels
No milestone
No project
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference
z3k-images/images!5
No description provided.