fix(review-suite): pass verification_commands as a declared variable, never LLM-extracted from prose

verification_commands — the shell commands the adversary's run_checks stage
executes — were extracted by the LLM parse node from {{initial_prompt}}, a
flat prompt that also carries untrusted plan/diff text pasted from the repo
under review. A planted '## VERIFICATION' block was indistinguishable from the
caller's declaration: prompt injection -> arbitrary command execution.

- adversary + review-gauntlet graphs declare verification_commands as a graph
  variable (default '[]'); the key leaves initial_state (or_insert seeding
  would otherwise shadow it) and leaves both parse nodes' instructions,
  output_schema.properties and required.
- review-gauntlet forwards it to run_adversary via a structured inputs:
  passthrough (lone {{verification_commands}} template), not lane-prompt prose.
- run_checks.py accepts a JSON list or a JSON-encoded string of one (variables
  arrive as strings); '[]'/[]/missing = none declared; any other shape fails
  closed with a PIPELINE-FAULT (nothing executed) that verdict.py turns into
  DIVERGES complaint #1.
- architect/sisyphus spawn templates and both READMEs document the
  --variables / --agent-variable form and the trust boundary. Known residual
  (llm-node auto-merge of undeclared output keys) is disclosed in the docs.
- Tests: list/JSON-string acceptance, fail-closed no-execution proof,
  prompt-prose injection proof, verdict wiring, asset pins for both graphs and
  orchestrator configs, dispatch-inputs > variable-override > default
  precedence.
This commit is contained in:
Dark-Alex-17 authored and Dark-Alex-17 committed 2026-09-14 10:50:34 -06:00
1 parent 00c3b60c2f
commit 300c7aeff0
9 files changed
+501 -47

No files matched your search

+11 -11
View File
@@ -31,6 +31,12 @@ variables:
- name: project_dir
description: Repository the change lives in; signals and every review lane run against it.
default: '.'
- name: verification_commands
description: |
JSON array of shell commands the CALLER declares for the deterministic
run_checks stage — e.g. '["cargo test --all"]'. Never inferred from the
prompt; empty array = none declared.
default: '[]'
settings:
max_loop_iterations: 10
@@ -44,7 +50,6 @@ initial_state:
security_posture: standard
probe_context: ''
extra_context: ''
verification_commands: []
forced_lanes: []
changed_files: []
file_count: 0
@@ -109,10 +114,6 @@ nodes:
names). Empty array = automatic selection.
- `extra_context`: any remaining caller context reviewers should see,
condensed.
- `verification_commands`: the exact build/test/lint shell commands the
caller explicitly declares for verification (verbatim strings, in the
order given). Empty array when the caller declares none — NEVER invent,
guess, or auto-detect commands; undeclared stays undeclared.
prompt: |
Extract the review-request facts from this request:
@@ -139,11 +140,7 @@ nodes:
type: string
project_dir_in:
type: string
verification_commands:
type: array
items:
type: string
required: [diff_spec, plan_context, rigor, security_posture, probe_context, forced_lanes, extra_context, project_dir_in, verification_commands]
required: [diff_spec, plan_context, rigor, security_posture, probe_context, forced_lanes, extra_context, project_dir_in]
max_iterations: 3
max_attempts: 2
state_updates:
@@ -326,6 +323,8 @@ nodes:
description: Plan-conformance review lane.
agent: adversary
timeout: 7200
inputs:
verification_commands: '{{verification_commands}}'
prompt: |
Adversarial plan-conformance review in {{project_dir}}.
@@ -334,7 +333,8 @@ nodes:
The spec / plan / acceptance criteria it must satisfy:
{{plan_context}}
Verification commands (declared by the caller; empty = none declared): {{verification_commands}}
Verification commands arrive as a structured input (the
verification_commands variable), not in this prompt.
{{extra_context}}