fix(review-suite): pass verification_commands as a declared variable, never LLM-extracted from prose

verification_commands — the shell commands the adversary's run_checks stage
executes — were extracted by the LLM parse node from {{initial_prompt}}, a
flat prompt that also carries untrusted plan/diff text pasted from the repo
under review. A planted '## VERIFICATION' block was indistinguishable from the
caller's declaration: prompt injection -> arbitrary command execution.

- adversary + review-gauntlet graphs declare verification_commands as a graph
  variable (default '[]'); the key leaves initial_state (or_insert seeding
  would otherwise shadow it) and leaves both parse nodes' instructions,
  output_schema.properties and required.
- review-gauntlet forwards it to run_adversary via a structured inputs:
  passthrough (lone {{verification_commands}} template), not lane-prompt prose.
- run_checks.py accepts a JSON list or a JSON-encoded string of one (variables
  arrive as strings); '[]'/[]/missing = none declared; any other shape fails
  closed with a PIPELINE-FAULT (nothing executed) that verdict.py turns into
  DIVERGES complaint #1.
- architect/sisyphus spawn templates and both READMEs document the
  --variables / --agent-variable form and the trust boundary. Known residual
  (llm-node auto-merge of undeclared output keys) is disclosed in the docs.
- Tests: list/JSON-string acceptance, fail-closed no-execution proof,
  prompt-prose injection proof, verdict wiring, asset pins for both graphs and
  orchestrator configs, dispatch-inputs > variable-override > default
  precedence.
This commit is contained in:
Dark-Alex-17 authored and Dark-Alex-17 committed 2026-09-14 10:50:34 -06:00
1 parent 00c3b60c2f
commit 300c7aeff0
9 files changed
+501 -47

No files matched your search

+53
View File
@@ -824,6 +824,59 @@ mod tests {
assert!(!state.contains_key("foo"));
}
// The dispatcher seeds a child's `inputs:` before the executor seeds
// variables, so a structured input outranks both a spawn-time override
// and the declared default — the contract the review-gauntlet →
// adversary `verification_commands` passthrough relies on.
fn seeded_after_dispatch(
inputs: Option<HashMap<String, Value>>,
agent_vars: Option<&AgentVariables>,
) -> HashMap<String, Value> {
let mut state = HashMap::new();
crate::graph::dispatch::seed_initial_state(&mut state, "prompt", inputs);
seed_variables(&mut state, &[variable("foo", Some("default"))], agent_vars);
state
}
#[test]
fn dispatch_inputs_win_over_variable_override_and_default() {
let agent_vars: AgentVariables =
IndexMap::from([("foo".to_string(), "override".to_string())]);
let inputs = HashMap::from([("foo".to_string(), json!("from-inputs"))]);
let state = seeded_after_dispatch(Some(inputs), Some(&agent_vars));
assert_eq!(state.get("foo"), Some(&json!("from-inputs")));
}
#[test]
fn dispatch_without_inputs_lets_variable_override_beat_default() {
let agent_vars: AgentVariables =
IndexMap::from([("foo".to_string(), "override".to_string())]);
let state = seeded_after_dispatch(None, Some(&agent_vars));
assert_eq!(state.get("foo"), Some(&json!("override")));
}
#[test]
fn dispatch_without_inputs_or_override_lands_declared_default() {
let state = seeded_after_dispatch(None, None);
assert_eq!(state.get("foo"), Some(&json!("default")));
}
#[test]
fn dispatch_lone_array_input_survives_variable_seeding_as_array() {
let agent_vars: AgentVariables =
IndexMap::from([("foo".to_string(), "override".to_string())]);
let inputs = HashMap::from([("foo".to_string(), json!(["echo a", "echo b"]))]);
let state = seeded_after_dispatch(Some(inputs), Some(&agent_vars));
assert_eq!(state.get("foo"), Some(&json!(["echo a", "echo b"])));
}
fn agent_node(id: &str, teammates: bool) -> Node {
Node {
id: id.into(),