Commit Graph
9 Commits
Author SHA1 Message Date
Dark-Alex-17 e240be8bda feat: improved the adversary agent to look for facts and not criteria. 2026-09-15 12:02:16 -06:00
Dark-Alex-17 68e893d684 fix(adversary): red verification runs block CONFORMS; re-derive timeout envelope; robust runner
verdict.py: a recorded verification run with exit≠0 (including the -1
TIMEOUT sentinel) or a skipped record now forces DIVERGES with a numbered
complaint naming the command — previously a [FAIL] line sat in the report
appendix while the sentinel read CONFORMS, and callers route on the
sentinel alone. Fail-closed by the same rule: an ENVIRONMENT runner-error
marker when commands WERE declared is one red complaint (the declared
commands never ran, so the run is unproven); the "none declared" marker
and an empty declaration stay non-red. Headers name the red-run count.

run_checks.py: decode command output with encoding=utf-8/errors=replace
(a single non-UTF-8 byte used to raise inside communicate() and collapse
the whole evidence record to an ENVIRONMENT marker); bound the post-kill
pipe drain by DRAIN_TIMEOUT_SECS so a session-escaped grandchild holding
the pipe cannot turn a TIMEOUT record into a node kill.

graph.yaml: settings.timeout 5400 → 9000 = 3600 (run_checks node timeout)
+ 5400 (the previous whole-graph budget, kept for every other stage); the
old value predated run_checks and a graph-level timeout kills from
outside, losing the DIVERGES sentinel. Descriptions and README doctrine
updated (CONFORMS = all MET + zero holistic complaints + zero pipeline
faults + zero red runs).

Tests: envelope pin (settings >= run_checks + 5400), gauntlet lane
timeouts > child graph timeouts and gauntlet >= max_attempts × lane +
margin over every agent lane, non-UTF-8 output, bounded drain (unix),
red/skipped/timeout/none-declared/ENVIRONMENT×declaration matrix,
numbering continuity, header punctuation, wording pins on graph.yaml,
run_checks.py and both READMEs. The deadline test's hung command is now
python3-based instead of GNU sleep (cmd.exe on the Windows CI leg).
2026-09-14 10:50:34 -06:00
Dark-Alex-17 300c7aeff0 fix(review-suite): pass verification_commands as a declared variable, never LLM-extracted from prose
verification_commands — the shell commands the adversary's run_checks stage
executes — were extracted by the LLM parse node from {{initial_prompt}}, a
flat prompt that also carries untrusted plan/diff text pasted from the repo
under review. A planted '## VERIFICATION' block was indistinguishable from the
caller's declaration: prompt injection -> arbitrary command execution.

- adversary + review-gauntlet graphs declare verification_commands as a graph
  variable (default '[]'); the key leaves initial_state (or_insert seeding
  would otherwise shadow it) and leaves both parse nodes' instructions,
  output_schema.properties and required.
- review-gauntlet forwards it to run_adversary via a structured inputs:
  passthrough (lone {{verification_commands}} template), not lane-prompt prose.
- run_checks.py accepts a JSON list or a JSON-encoded string of one (variables
  arrive as strings); '[]'/[]/missing = none declared; any other shape fails
  closed with a PIPELINE-FAULT (nothing executed) that verdict.py turns into
  DIVERGES complaint #1.
- architect/sisyphus spawn templates and both READMEs document the
  --variables / --agent-variable form and the trust boundary. Known residual
  (llm-node auto-merge of undeclared output keys) is disclosed in the docs.
- Tests: list/JSON-string acceptance, fail-closed no-execution proof,
  prompt-prose injection proof, verdict wiring, asset pins for both graphs and
  orchestrator configs, dispatch-inputs > variable-override > default
  precedence.
2026-09-14 10:50:34 -06:00
Dark-Alex-17 7ada3fb47c fix(adversary): narrow died-marker predicate, degraded banner for died criteria, hermetic script tests 2026-09-14 10:50:34 -06:00
Dark-Alex-17 3f49db86b2 fix(adversary): fail closed per criterion when check_criterion dies
A live run died at map_criteria: one heavy criterion exhausted
max_iterations (10) with no fallback and sank the whole 17-criterion
review. check_criterion now has a 30-iteration budget, a tool-budget
instruction, and falls back to criterion_fault, which records a
schema-shaped UNMET verdict with a PIPELINE-FAULT evidence marker for
that one criterion; verdict.py renders it as DIED rather than judged.
2026-09-14 10:50:34 -06:00
Dark-Alex-17 1e46743420 fix(TASK-017): bound run_checks runtime and fail-close its node-level death
Adversary conformance-review fixes for the TASK-017 exec-evidence stage:

- run_checks.py: total-runtime deadline (3300s = node timeout minus margin);
  each command gets min(900s, remaining budget), commands past the deadline
  are recorded as {cmd, skipped: deadline} entries so in-script handling
  stays authoritative instead of the node timeout killing the script from
  outside. POSIX hardening: per-command process group (start_new_session)
  + killpg on timeout so hung grandchildren cannot linger; non-POSIX
  limitation documented in the docstring.
- graph.yaml: run_checks gains fallback: pipeline_fault; check_criterion
  treats SKIPPED records as at-best PARTIAL (unproven, never failed).
- pipeline_fault.py: run_checks-stage attribution grounded in real engine
  behavior (script-node fallback writes nothing into state, executor.rs
  script arm) — nonempty diff_text proves facts completed, so the fault is
  the verification runner; emits 'PIPELINE-FAULT: verification runner
  killed'. Empty diff_text still attributes to diff resolution.
- verdict.py: renders skipped records as [SKIPPED] instead of a bogus
  [FAIL] exit-? line; non-degraded verdict semantics untouched.
- tests: outer-except ENVIRONMENT degradation, deadline timeout+skip (env
  seam ADVERSARY_RUN_CHECKS_DEADLINE_SECS), pipeline_fault stage
  attribution (parse / facts / run_checks), check_criterion prompt cites
  {{exec_results}}, verdict SKIPPED rendering; run_adversary_script_env
  helper for env-carrying invocations.
- cargo fmt violations in 3d7dea9 test code fixed (fmt --check exit 0).
2026-09-14 10:50:33 -06:00
Dark-Alex-17 d50059a56f feat(TASK-017): adversary execution evidence + fail-closed routing
- parse extracts caller-declared verification_commands (schema + initial_state);
  never invents commands. parse gains fallback + state_updates capturing the
  failure string (TASK-016 fallback-capture warning stays silent).
- new run_checks script node (facts -> run_checks -> map_criteria, timeout
  3600): runs each declared command in the project dir (900s per-command
  timeout), records exec_results {cmd, exit, duration_s, tail~50}; 'none
  declared' / ENVIRONMENT markers instead of guessing or failing the node.
- check_criterion judges execution criteria against {{exec_results}} only:
  recorded green run = MET citing it, recorded failure = UNMET, markers =
  PARTIAL with an ENVIRONMENT note.
- fail-closed routing (R1/R2): parse/facts fallback -> pipeline_fault marker
  script -> verdict; holistic fallback -> verdict with holistic_failure
  captured. verdict.py: any PIPELINE-FAULT forces DIVERGES with the fault as
  complaint #1, per-criterion results still reported; report gains a
  'Verification runs:' section. Non-degraded semantics unchanged.
- R5 rust tests (python3-guarded, local cmd_available): verdict fault path,
  holistic-failure fault, CONFORMS regression, run_checks markers + recorded
  runs, crit_gate contract regression.
2026-09-14 10:50:33 -06:00
Dark-Alex-17 3254fe9f3c feat: Fixed graph agents with new agent variable passing functionality 2026-09-12 01:37:20 -06:00
Dark-Alex-17 99e592565a feat: Significant code review suite rewrite with improved graph agent features 2026-09-11 15:33:21 -06:00