fix(review-suite): pass verification_commands as a declared variable, never LLM-extracted from prose

verification_commands — the shell commands the adversary's run_checks stage
executes — were extracted by the LLM parse node from {{initial_prompt}}, a
flat prompt that also carries untrusted plan/diff text pasted from the repo
under review. A planted '## VERIFICATION' block was indistinguishable from the
caller's declaration: prompt injection -> arbitrary command execution.

- adversary + review-gauntlet graphs declare verification_commands as a graph
  variable (default '[]'); the key leaves initial_state (or_insert seeding
  would otherwise shadow it) and leaves both parse nodes' instructions,
  output_schema.properties and required.
- review-gauntlet forwards it to run_adversary via a structured inputs:
  passthrough (lone {{verification_commands}} template), not lane-prompt prose.
- run_checks.py accepts a JSON list or a JSON-encoded string of one (variables
  arrive as strings); '[]'/[]/missing = none declared; any other shape fails
  closed with a PIPELINE-FAULT (nothing executed) that verdict.py turns into
  DIVERGES complaint #1.
- architect/sisyphus spawn templates and both READMEs document the
  --variables / --agent-variable form and the trust boundary. Known residual
  (llm-node auto-merge of undeclared output keys) is disclosed in the docs.
- Tests: list/JSON-string acceptance, fail-closed no-execution proof,
  prompt-prose injection proof, verdict wiring, asset pins for both graphs and
  orchestrator configs, dispatch-inputs > variable-override > default
  precedence.
This commit is contained in:
Dark-Alex-17 authored and Dark-Alex-17 committed 2026-09-14 10:50:34 -06:00
1 parent 00c3b60c2f
commit 300c7aeff0
9 files changed
+501 -47

No files matched your search

+9
View File
@@ -315,8 +315,13 @@ instructions: |
Rigor: <rigor> Security posture: <posture>
Plan / acceptance criteria: <pasted verbatim — enables the adversary lane; omit when none>
Local-run recipe / usage suites: <from the plan — enables the probe lane; omit when none>"
--variables {"verification_commands": "[\"cargo test --all\", \"cargo clippy -- -D warnings\"]"}
```
Verification commands go in `--variables` (a JSON-array string), never in the prompt: the prompt
also carries pasted plan text from the repo under review, and whatever is declared here is
executed with a shell — a trust boundary the prose must not cross.
It computes diff signals deterministically, selects lanes (code-review always; adversary when a
spec/plan is present; security on attack-surface signals; probe on consumer-facing surface with a
run recipe — judgment can only WIDEN the selection, never narrow it), runs the selected lanes in
@@ -392,8 +397,12 @@ instructions: |
PLAN — acceptance criteria to check against:
<paste the task/step spec + acceptance criteria VERBATIM — not a summary>"
--variables {"verification_commands": "[\"cargo test --all\", \"cargo clippy -- -D warnings\"]"}
```
Same trust boundary as the gauntlet spawn: verification commands are declared via `--variables`,
never in the prompt, because the prompt also carries untrusted pasted text.
### Handling adversary findings
- **`ADVERSARIAL_REVIEW: DIVERGES` blocks completion.** Do not mark the task done. Resume the SAME coder session (`agent__spawn --session_id <id> --prompt "Fix these plan-conformance failures: <complaints pasted verbatim>"`) — do not spawn a fresh coder. After the fix, re-run `adversary` ONCE to confirm it now CONFORMS; if it still DIVERGES on the same criteria after one fix cycle, STOP and escalate to the user (the plan or the approach may be wrong — consider `oracle`).