Rust test comments/assert messages and deep-research script docstrings
cited numbered requirement labels (R3/R5) and task labels (TASK-016,
TASK-021) that are defined only in an external plan document. Reword
each to say what it means on its own: fail-safe crash guards, the
validator's fallback-capture warning, task-queue anchors.
FIX A: run_checks.py passed no env= to Popen, so verification commands
inherited the live GRAPH_STATE/GRAPH_STATE_FILE. Every suite script's
load_state() prefers GRAPH_STATE_FILE, so when the adversary reviews a repo
whose own tests exec these scripts (this repo's R5 tests) the leaked file
silently overrode the test state -> false failures. Popen now gets an env
with all GRAPH_STATE* keys removed. The five sibling script-test helpers
(review-gauntlet, code-reviewer, finding-verifier, step-runner,
deep-research) gain .env_remove("GRAPH_STATE_FILE") for hermeticity. New
R5 test loads state via GRAPH_STATE_FILE and asserts a shell-neutral
python3 -c probe sees both vars UNSET.
FIX B: the depth check compares the child's absolute depth against the
SPAWNER's own max_agent_depth (root = 0), so under
architect(0)->sisyphus(1)->review-gauntlet(2)->code-reviewer(3) the
code-review lane faulted "Max agent depth exceeded (4/3)". Limits are now
child depth + 1 headroom along the documented chains, with the chain in a
comment next to each key: code-reviewer 3->5, domain-reviewer 2->6,
architecture-reviewer 2->4, step-runner default->4 (explicit),
review-gauntlet stays 4. domain-reviewer's file-reviewer hop is documented
but inert (can_spawn_agents unset) - left unwired by decision; noted in
config and pinned via a DOCUMENTED_BUT_UNWIRED exemption in the new
bundled-config test, which pins every value and asserts per-hop headroom +
reachability along four chains.
The crash-path assertion checked only "PIPELINE-FAULT:"; verdict.py's died() keys on "PIPELINE-FAULT: criterion ", so a regression there would have passed the test yet stopped the crashed marker from rendering as DIED.
A live code-reviewer run degraded through synthesize's fallback; the
graph already retries once (max_attempts: 2) and falls back to verdict
with synth_failure captured, but nothing pinned that wiring. Add the
wiring test so a future edit cannot silently drop the retry or route.
A live run died at map_criteria: one heavy criterion exhausted
max_iterations (10) with no fallback and sank the whole 17-criterion
review. check_criterion now has a 30-iteration budget, a tool-budget
instruction, and falls back to criterion_fault, which records a
schema-shaped UNMET verdict with a PIPELINE-FAULT evidence marker for
that one criterion; verdict.py renders it as DIED rather than judged.
A live code-reviewer run reported "LLM node failed: Failed to call
chat-completions api" with the root cause invisible, because the
state_updates fault text used `{e}` (top context only). Format with
`{e:#}` so fault scripts and warn! logs see the whole anyhow chain.
R3 guards: parse_request (crash → ask_topic), bootstrap_research,
combine_findings (crash → fault-text findings; also lifts branch-local
PIPELINE-FAULT findings into pipeline_faults, deduped), reflexion_gate
(crash fails forward to synthesize), incorporate_feedback (crash still
loops with a generic feedback note), verify_sources (top-level guard +
pipeline_notes fold).
Fallbacks per the plan's naming pins: plan → plan_fault → end_fault
(fail-closed; gains state_updates plan_failure per the TASK-016 rule);
research_one_question → branch-local question_fault (map collects the
fault finding); vet_sources/critique → degrade-visibly markers that
continue to the stage each node's own next pointed to; synthesize →
synth_fault → end_fault. end_fault renders
'DEEP_RESEARCH FAILED — PIPELINE-FAULT: <fault text>'.
Accepted-path visibility: verify_sources folds pipeline_faults into
pipeline_notes ('' when fault-free), appended to end_accepted's output
and the approval question — happy-path output stays byte-identical.
initial_state seeds questions: [] so a dying plan inside the static
fan-out leaves the sibling knowledge_lookup lane mapping over zero
items while end_fault renders.
R5: 19 python3-guarded regression tests + GraphParser wiring test;
bundled warning baseline unchanged.
- prompts.rs DEFAULT_SPAWN_INSTRUCTIONS: example IDs task_1/task_2 -> "1"/"2"
(real numeric-string IDs; old format silently produced dep_error + unblocked
task), plus task-queue rules: root tasks with agent are never auto-dispatched,
task_complete is what dispatches dependents, auto-dispatched agents must be
collected, failed tasks permanently block dependents.
- architect config: optional task-queue mirroring guidance for parallel mode
(disk task files stay the durable store; queue is in-memory).
- sisyphus config: short note on encoding dependency-shaped bursts as
agent__task_create chains; todos remain the tracking source of truth.
The three fallback-capture keys (orient_failure, sweep_failure,
handoff_failure) were write-only dead state: captured on LLM-node failure
but interpolated nowhere, so end_failure rendered an empty STEP_FAILED
shell and write_handoff carried no trace a dead sweep.
- new crash-proof script node note_llm_fault distills orient/handoff
'LLM node...' failure text into fault_note (prefix-anchored; empty when
neither key holds a fault); orient and write_handoff fallbacks now route
through it before end_failure
- end_failure renders 'Pipeline fault (if any): {{fault_note}}' (sentinel
and all existing fields preserved; fault_note defaulted in initial_state)
- write_handoff prompt interpolates {{sweep_failure}} with a
PIPELINE-FAULT VISIBILITY instruction mirroring the review_report
treatment (fault -> Deviations entry + step_summary; success/empty
ignored)
- tests: note_llm_fault fault/clean/crash/truncation paths via the
GRAPH_STATE harness; wiring test pins the retargeted fallbacks,
note_llm_fault routing, {{fault_note}} in end_failure, and the
sweep-fault prompt/instruction anchors; warning baseline unchanged
finding-verifier: parse gains max_attempts 2 + parse_failure capture +
fallback to a new parse_fault marker that rewrites verdicts to a single
UNVERIFIABLE PIPELINE-FAULT entry and continues to done (sentinel
preserved — the caller's verify lane degrades instead of dying);
verify_one gains a branch-local verify_fault marker normalizing the
engine failure text into a schema-conformant UNVERIFIABLE verdict.
step-runner: implement gains fallback end_failure (no retry — reruns
against a mutated tree are unsafe; STEP_FAILED renders the failure via
coder_result); independent_review gains fallback write_handoff with the
handoff instructions and route_review treating an "Agent node failed:"
review_report as review-did-not-run (fail-visible, guard ahead of the
🔴 grep); orient/edge_case_sweep/write_handoff capture their failure
text via state_updates, shrinking the bundled fallback-capture warning
baseline back to zero.
R5 tests: fault-marker normalization + crash paths (raw non-JSON
GRAPH_STATE), route_review.sh fault/critical/clean routing via bash+jq,
and structural wiring tests pinning fallbacks, sentinels, and
end_failure's initial_state-backed template keys.
render.py's empty-diff stub bypass is anchored on the 'PIPELINE-FAULT:'
prefix in reason/attention, but verdict.py's crash guard emitted
'verdict computation error: …' / 'verdict script error: …' without it —
on an empty diff a crashed verdict gate collapsed into the
'No changes to review.' stub and the fail-closed NEEDS-HUMAN verdict
never reached the final report.
- prefix the crash-guard reason and attention entry with PIPELINE-FAULT:
- strengthen the existing crash test to assert the prefix on both fields
- add R5 chained test: empty diff + crash-inducing state piped
verdict.py -> render.py must render the fault, never the stub
- add crash-guard symmetry tests for parse_fault.py / aux_fault.py
(raw non-JSON GRAPH_STATE keeps the fault marker) and a test that a
fault survives the attention[:5] cap alongside 6 user flags
Every llm/agent stage that can die persistently now falls back to a
machine-readable PIPELINE-FAULT sentinel instead of killing the graph:
- review_domain -> branch-local domain_fault marker (banner-prefixed
domain_report the map collects; completeness_gate precedent)
- verify -> verdict (verifier fault renders findings unverified + a
PIPELINE-FAULT attention line)
- parse -> parse_fault -> render (complete NEEDS-HUMAN verdict_out;
render's no-changes stub is bypassed for fault verdicts)
- refine_groups -> cover_gate (deterministic-proposal fallback absorbs
the failure; degradation noted in groups_note)
- aux_lanes -> aux_fault -> synthesize (aux_note fault marker; visible,
never blocking - aux is enrichment)
- synthesize -> verdict (synth_failure marker; keyed off the failure
prefix + non-empty diff, never findings==[], so clean zero-finding
reviews stay MERGE-READY)
verdict.py gains unified fault accounting (R2): any PIPELINE-FAULT in
domain_reports / verifier_output / state fault keys forces NEEDS-HUMAN,
prefix-anchored so a report merely quoting the marker mid-text never
trips it; old 'PIPELINE FAULT' spelling migrated. Faults are prepended
to attention so the [:5] cap can never hide one. Dedicated *_failure
state keys capture engine failure text on every fallback+output_schema
node, keeping the bundled warning baseline at zero.
15 python3-guarded R5 tests: fault paths, quoted-marker precedence,
happy-path byte-compat regression, marker normalization, crash-path
fail-closed (verdict/cover_gate/domain_fault), verifier-fault-on-empty-
diff render, and the no-changes stub regression.
Adversary finding 1: verdict_gate treated any lane report CONTAINING
'PIPELINE-FAULT:' as a lane fault, so a real review quoting the marker
silently bypassed red-finding counting and sentinel parsing. Lane-fault
detection is now precise — only a report that IS the lane_fault.py
marker (starts with it) routes to the fault rule. Adds the missing
regression tests: reds block despite MERGE-READY, NEEDS-HUMAN without
reds passes with attention, probe INCONCLUSIVE blocks with the
environment note, a gate crash fails closed, and a quoting report is
routed to the red rule, not the fault rule.
Adversary finding 2: default_lanes overwrote caller-forced lanes and
dropped the security lane the deterministic signals guaranteed. The
fallback now unions its defaults with caller-forced lanes (canonicalized
via the same aliases build_items uses) and forces security on
auth/deps/exec signals or hardened posture — selection degrades wider,
never narrower, and stays deterministic via a fixed lane order. Tests
cover forced-lane survival, alias canonicalization, unknown-name
tolerance, each security signal, and determinism.
- lane_fault.py + fallback on all four run_* lane nodes: an agent-node
failure is normalized into a PIPELINE-FAULT lane result instead of
killing the graph
- verdict_gate.py: PIPELINE-FAULT lane results block distinctly from
SKIPPED and missing-sentinel; a PIPELINE-FAULT in signals_error blocks
at the gauntlet level
- build_items.py: restructured into main() + fail-closed guard (crash
emits empty lanes + PIPELINE-FAULT) and folds lanes_degraded into
lanes_summary
- parse: state_updates/fallback -> parse_fault.py, which records the
fault in signals_error and jumps to the verdict gate
- select_lanes: state_updates/fallback -> default_lanes.py, a
deterministic wider-never-narrower lane fallback
- verification_commands threaded through parse into run_adversary's
prompt (declared-only, never invented)
- run_probe prompt gains the aborted-attempt reconcile line
- 8 gauntlet_* R5 tests in src/config/request_context.rs (script harness
+ graph interpolation)
Adversary conformance-review fixes for the TASK-017 exec-evidence stage:
- run_checks.py: total-runtime deadline (3300s = node timeout minus margin);
each command gets min(900s, remaining budget), commands past the deadline
are recorded as {cmd, skipped: deadline} entries so in-script handling
stays authoritative instead of the node timeout killing the script from
outside. POSIX hardening: per-command process group (start_new_session)
+ killpg on timeout so hung grandchildren cannot linger; non-POSIX
limitation documented in the docstring.
- graph.yaml: run_checks gains fallback: pipeline_fault; check_criterion
treats SKIPPED records as at-best PARTIAL (unproven, never failed).
- pipeline_fault.py: run_checks-stage attribution grounded in real engine
behavior (script-node fallback writes nothing into state, executor.rs
script arm) — nonempty diff_text proves facts completed, so the fault is
the verification runner; emits 'PIPELINE-FAULT: verification runner
killed'. Empty diff_text still attributes to diff resolution.
- verdict.py: renders skipped records as [SKIPPED] instead of a bogus
[FAIL] exit-? line; non-degraded verdict semantics untouched.
- tests: outer-except ENVIRONMENT degradation, deadline timeout+skip (env
seam ADVERSARY_RUN_CHECKS_DEADLINE_SECS), pipeline_fault stage
attribution (parse / facts / run_checks), check_criterion prompt cites
{{exec_results}}, verdict SKIPPED rendering; run_adversary_script_env
helper for env-carrying invocations.
- cargo fmt violations in 3d7dea9 test code fixed (fmt --check exit 0).
The adversary conformance review flagged that the fallback warn! at
outcome_from (src/graph/agent.rs:71) was never asserted — only the retry
warn had captured-text coverage. Adds one test installing the shared warn
collector, filtered by the unique node id 'fallback_warn_capture' for
parallel safety, asserting the warn names both the node and the fallback
target and carries the error chain.
Agent nodes gain an optional fallback target with llm-node parity: on any
failure (retries exhausted, hard error, or structured-extraction failure)
the failure chain is written into state as 'Agent node failed: <chain>'
and the chain routes to the fallback node. Without a fallback the error
propagates unchanged. Validator collects agent fallback edges for
missing-target checks, reachability, and map-branch locality.
Hoist the llm node's transient-error matcher to graph::is_transient_error,
extend it with broken-pipe / connection-error / error-sending-request
signatures, and give AgentNode an opt-in max_attempts retry loop with
llm-parity semantics: contexts applied before the transient check, fresh
wall-clock budget per attempt, peer identity (assignment + registry)
re-armed before each retry, retirement exactly once after the final
attempt, and structured extraction outside the loop.
- agent__spawn accepts an optional variables object (string->string);
values overlay the parent-inherited agent variables per key, with
inheritance preserved as the fallback layer
- unknown variable names, non-string values, and non-object shapes
return self-correcting status:error JSON listing the agent's
declared variables
- spawned children can no longer trigger the interactive variable
prompt: a new interactive gate in Agent::init_agent_variables routes
missing required variables to the existing bail path while leaving
CLI/REPL prompting and --info skip semantics unchanged
- agent__task_create persists variables on the task, agent__task_list
echoes them, and task auto-dispatch threads them into the spawn
- tool schemas and built-in spawn docs updated; unit tests extended
in agents.rs and taskqueue.rs
Dark-Alex-17
github-actions[bot]