fix(TASK-017): bound run_checks runtime and fail-close its node-level death
Adversary conformance-review fixes for the TASK-017 exec-evidence stage:
- run_checks.py: total-runtime deadline (3300s = node timeout minus margin);
each command gets min(900s, remaining budget), commands past the deadline
are recorded as {cmd, skipped: deadline} entries so in-script handling
stays authoritative instead of the node timeout killing the script from
outside. POSIX hardening: per-command process group (start_new_session)
+ killpg on timeout so hung grandchildren cannot linger; non-POSIX
limitation documented in the docstring.
- graph.yaml: run_checks gains fallback: pipeline_fault; check_criterion
treats SKIPPED records as at-best PARTIAL (unproven, never failed).
- pipeline_fault.py: run_checks-stage attribution grounded in real engine
behavior (script-node fallback writes nothing into state, executor.rs
script arm) — nonempty diff_text proves facts completed, so the fault is
the verification runner; emits 'PIPELINE-FAULT: verification runner
killed'. Empty diff_text still attributes to diff resolution.
- verdict.py: renders skipped records as [SKIPPED] instead of a bogus
[FAIL] exit-? line; non-degraded verdict semantics untouched.
- tests: outer-except ENVIRONMENT degradation, deadline timeout+skip (env
seam ADVERSARY_RUN_CHECKS_DEADLINE_SECS), pipeline_fault stage
attribution (parse / facts / run_checks), check_criterion prompt cites
{{exec_results}}, verdict SKIPPED rendering; run_adversary_script_env
helper for env-carrying invocations.
- cargo fmt violations in 3d7dea9 test code fixed (fmt --check exit 0).
This commit is contained in:
5 files changed
+303
-51
No files matched your search
@@ -1,15 +1,23 @@
|
||||
#!/usr/bin/env python3
|
||||
"""Fail-closed fault marker for the adversary graph.
|
||||
|
||||
Reached ONLY via `fallback:` routes (parse, facts). Appends a machine-readable
|
||||
"PIPELINE-FAULT:" marker to `pipeline_faults` and continues to `verdict`,
|
||||
which turns any recorded fault into DIVERGES — the graph never dies without
|
||||
its sentinel, and a degraded run can never read as CONFORMS.
|
||||
Reached ONLY via `fallback:` routes (parse, facts, run_checks). Appends a
|
||||
machine-readable "PIPELINE-FAULT:" marker to `pipeline_faults` and continues
|
||||
to `verdict`, which turns any recorded fault into DIVERGES — the graph never
|
||||
dies without its sentinel, and a degraded run can never read as CONFORMS.
|
||||
|
||||
Stage attribution: an llm-node failure writes an "LLM node …failed: <chain>"
|
||||
string into `parse_failure` via parse's `state_updates`; anything else means
|
||||
the deterministic facts (diff resolution) script failed, which leaves no
|
||||
failure text in state by design.
|
||||
Stage attribution — grounded in engine behavior: a SCRIPT node's fallback
|
||||
route writes NOTHING into state (src/graph/executor.rs script arm routes to
|
||||
the fallback without recording the error), so the two script stages are
|
||||
discriminated by which stage outputs are present:
|
||||
- parse (llm): its `state_updates` wrote an "LLM node …failed: <chain>"
|
||||
string into `parse_failure` — attribute to parse.
|
||||
- run_checks vs facts (scripts): diff_facts.py unconditionally writes a
|
||||
NONEMPTY `diff_text` (even its internal-failure path writes
|
||||
"(diff unavailable)"), while the graph's initial value is ''. A nonempty
|
||||
`diff_text` proves facts completed, so the fault is the verification
|
||||
runner (run_checks killed at the node level, e.g. by the node timeout);
|
||||
an empty `diff_text` means facts itself died.
|
||||
"""
|
||||
|
||||
import json
|
||||
@@ -30,11 +38,19 @@ def main():
|
||||
faults = [f for f in (state.get("pipeline_faults") or []) if isinstance(f, str)]
|
||||
parse_failure = state.get("parse_failure")
|
||||
detail = parse_failure.strip() if isinstance(parse_failure, str) else ""
|
||||
diff_text = state.get("diff_text")
|
||||
facts_completed = isinstance(diff_text, str) and bool(diff_text.strip())
|
||||
if detail.startswith("LLM node"):
|
||||
detail = detail.replace("\n", " ")
|
||||
if len(detail) > MAX_DETAIL_CHARS:
|
||||
detail = detail[: MAX_DETAIL_CHARS - 1] + "…"
|
||||
faults.append(f"PIPELINE-FAULT: parse failed — cannot review: {detail}")
|
||||
elif facts_completed:
|
||||
faults.append(
|
||||
"PIPELINE-FAULT: verification runner killed — the run_checks stage "
|
||||
"died at the node level before recording exec_results; execution "
|
||||
"criteria are unproven"
|
||||
)
|
||||
else:
|
||||
faults.append(
|
||||
"PIPELINE-FAULT: diff resolution failed — changed files and diff text "
|
||||
|
||||
Reference in new issue
Block a user