fix(review-suite): pass verification_commands as a declared variable, never LLM-extracted from prose
verification_commands — the shell commands the adversary's run_checks stage
executes — were extracted by the LLM parse node from {{initial_prompt}}, a
flat prompt that also carries untrusted plan/diff text pasted from the repo
under review. A planted '## VERIFICATION' block was indistinguishable from the
caller's declaration: prompt injection -> arbitrary command execution.
- adversary + review-gauntlet graphs declare verification_commands as a graph
variable (default '[]'); the key leaves initial_state (or_insert seeding
would otherwise shadow it) and leaves both parse nodes' instructions,
output_schema.properties and required.
- review-gauntlet forwards it to run_adversary via a structured inputs:
passthrough (lone {{verification_commands}} template), not lane-prompt prose.
- run_checks.py accepts a JSON list or a JSON-encoded string of one (variables
arrive as strings); '[]'/[]/missing = none declared; any other shape fails
closed with a PIPELINE-FAULT (nothing executed) that verdict.py turns into
DIVERGES complaint #1.
- architect/sisyphus spawn templates and both READMEs document the
--variables / --agent-variable form and the trust boundary. Known residual
(llm-node auto-merge of undeclared output keys) is disclosed in the docs.
- Tests: list/JSON-string acceptance, fail-closed no-execution proof,
prompt-prose injection proof, verdict wiring, asset pins for both graphs and
orchestrator configs, dispatch-inputs > variable-override > default
precedence.
This commit is contained in:
1 parent
00c3b60c2f
commit
300c7aeff0
9 files changed
+501
-47
No files matched your search
@@ -333,8 +333,13 @@ instructions: |
|
||||
Forced lanes: adversary<, probe — add probe when the task changed consumer-facing surface and the plan carries a local-run recipe>
|
||||
Plan / acceptance criteria: <the task's acceptance criteria, pasted verbatim>
|
||||
Local-run recipe / usage suites: <from the plan, when present>"
|
||||
--variables {"verification_commands": "[\"cargo test --all\", \"cargo clippy -- -D warnings\"]"}
|
||||
```
|
||||
|
||||
Verification commands go in `--variables` (a JSON-array string), never in the prompt: the
|
||||
prompt also carries pasted plan text from the repo under review, and whatever is declared
|
||||
here is executed with a shell — a trust boundary the prose must not cross.
|
||||
|
||||
It returns `GAUNTLET: PASS/BLOCKED`. On BLOCKED apply the same fix-and-rerun rules as below
|
||||
(resume the SAME Sisyphus session with the findings verbatim, then a FRESH gauntlet run).
|
||||
The direct `adversary`/probe spawns below are the FALLBACK when the gauntlet is not installed.
|
||||
@@ -351,8 +356,12 @@ instructions: |
|
||||
<paste the task index.md body + the relevant PLAN-<slug>.md section VERBATIM>
|
||||
<paste the plan's ## Quality bar section VERBATIM — recorded dropped practices are
|
||||
conformance facts, not divergences>"
|
||||
--variables {"verification_commands": "[\"cargo test --all\", \"cargo clippy -- -D warnings\"]"}
|
||||
```
|
||||
|
||||
Same trust boundary as the gauntlet spawn: verification commands are declared via
|
||||
`--variables`, never in the prompt, because the prompt also carries untrusted pasted text.
|
||||
|
||||
- **`ADVERSARIAL_REVIEW: DIVERGES`** → treat it as a blocker: resume the SAME Sisyphus session
|
||||
(`agent__spawn --session_id <id> --prompt "Fix these plan-conformance failures: <adversary
|
||||
complaints, verbatim>"`) — do not spawn a fresh one. Re-run `adversary` ONCE after the fix to
|
||||
|
||||
Reference in new issue
Block a user