feat(TASK-017): adversary execution evidence + fail-closed routing
- parse extracts caller-declared verification_commands (schema + initial_state);
never invents commands. parse gains fallback + state_updates capturing the
failure string (TASK-016 fallback-capture warning stays silent).
- new run_checks script node (facts -> run_checks -> map_criteria, timeout
3600): runs each declared command in the project dir (900s per-command
timeout), records exec_results {cmd, exit, duration_s, tail~50}; 'none
declared' / ENVIRONMENT markers instead of guessing or failing the node.
- check_criterion judges execution criteria against {{exec_results}} only:
recorded green run = MET citing it, recorded failure = UNMET, markers =
PARTIAL with an ENVIRONMENT note.
- fail-closed routing (R1/R2): parse/facts fallback -> pipeline_fault marker
script -> verdict; holistic fallback -> verdict with holistic_failure
captured. verdict.py: any PIPELINE-FAULT forces DIVERGES with the fault as
complaint #1, per-criterion results still reported; report gains a
'Verification runs:' section. Non-degraded semantics unchanged.
- R5 rust tests (python3-guarded, local cmd_available): verdict fault path,
holistic-failure fault, CONFORMS regression, run_checks markers + recorded
runs, crit_gate contract regression.
This commit is contained in:
5 files changed
+564
-26
No files matched your search
@@ -0,0 +1,58 @@
|
||||
#!/usr/bin/env python3
|
||||
"""Fail-closed fault marker for the adversary graph.
|
||||
|
||||
Reached ONLY via `fallback:` routes (parse, facts). Appends a machine-readable
|
||||
"PIPELINE-FAULT:" marker to `pipeline_faults` and continues to `verdict`,
|
||||
which turns any recorded fault into DIVERGES — the graph never dies without
|
||||
its sentinel, and a degraded run can never read as CONFORMS.
|
||||
|
||||
Stage attribution: an llm-node failure writes an "LLM node …failed: <chain>"
|
||||
string into `parse_failure` via parse's `state_updates`; anything else means
|
||||
the deterministic facts (diff resolution) script failed, which leaves no
|
||||
failure text in state by design.
|
||||
"""
|
||||
|
||||
import json
|
||||
import os
|
||||
|
||||
MAX_DETAIL_CHARS = 500
|
||||
|
||||
|
||||
def load_state():
|
||||
if path := os.environ.get("GRAPH_STATE_FILE"):
|
||||
with open(path) as f:
|
||||
return json.load(f)
|
||||
return json.loads(os.environ.get("GRAPH_STATE", "{}"))
|
||||
|
||||
|
||||
def main():
|
||||
state = load_state()
|
||||
faults = [f for f in (state.get("pipeline_faults") or []) if isinstance(f, str)]
|
||||
parse_failure = state.get("parse_failure")
|
||||
detail = parse_failure.strip() if isinstance(parse_failure, str) else ""
|
||||
if detail.startswith("LLM node"):
|
||||
detail = detail.replace("\n", " ")
|
||||
if len(detail) > MAX_DETAIL_CHARS:
|
||||
detail = detail[: MAX_DETAIL_CHARS - 1] + "…"
|
||||
faults.append(f"PIPELINE-FAULT: parse failed — cannot review: {detail}")
|
||||
else:
|
||||
faults.append(
|
||||
"PIPELINE-FAULT: diff resolution failed — changed files and diff text "
|
||||
"could not be resolved; criterion verification never ran"
|
||||
)
|
||||
print(json.dumps({"pipeline_faults": faults}))
|
||||
|
||||
|
||||
try:
|
||||
main()
|
||||
except Exception as e: # noqa: BLE001 — the fault marker itself must never crash
|
||||
print(
|
||||
json.dumps(
|
||||
{
|
||||
"pipeline_faults": [
|
||||
f"PIPELINE-FAULT: fault-marker script error: {e} — treat the "
|
||||
"run as degraded"
|
||||
]
|
||||
}
|
||||
)
|
||||
)
|
||||
Reference in new issue
Block a user