Files
Dark-Alex-17 156f3193a9 fix(review-suite): detect nested finding-verifier faults; harden render/fault markers
code-reviewer verdict.py: finding-verifier now completes "successfully"
with fault markers INSIDE its FINDING_VERIFIER_RESULTS payload
(parse_fault's `pipeline-fault` sentinel id, verify_fault / verdict_gate
crash-guard notes prefixed PIPELINE-FAULT:), which the old
"Agent node failed:" prefix check never saw — a dead parse degraded
silently to "unverified" with no fault recorded. Detect them, anchored on
the sentinel id / note prefix only (never a substring over the report),
counted over the raw entry list so duplicate crash-guard ids are not
collapsed. Faults lead the attention list uncapped; the display cap of 5
applies to non-fault entries; the "always-human trigger(s) fired" reason
fires only when non-fault triggers exist.

code-reviewer render.py: CONTRACT comments at the verdict and summary
lines review-gauntlet's gate parses; every LLM-influenced value rendered
after the summary line (dropped_titles, rigor, bar_provenance,
resolved_surfaces, reason) is newline-sanitised so none can forge a
second summary line.

finding-verifier verdict_gate.py: the outer crash-guard note carries the
PIPELINE-FAULT: prefix (the after-retry UNVERIFIABLE path stays
unprefixed — model misbehaviour, not infrastructure); README names all
three emitters.

deep-research: vet_fault's UNVETTED note tells the critic not to request
revision solely because the assessment is unavailable (it matched
critique's PREPRINT/UNVERIFIED REVISE trigger verbatim, burning the
revision budget on an infra fault); vet_fault and critique_fault dedupe
repeated faults and preserve accumulated faults on crash via
safe_faults(); question_fault keeps its plan-pinned prefix and appends the
lane's question identity so distinct dead lanes no longer collapse in
combine_findings' dedupe.
2026-09-14 10:50:34 -06:00

72 lines
2.5 KiB
Python

#!/usr/bin/env python3
"""Degrade-visibly marker for a dead critique stage of deep-research.
Reached ONLY via critique's `fallback:` route. Grounded in engine behavior:
on llm-node failure the engine writes "LLM node …failed: <chain>" into
`critique` via the node's state_updates BEFORE routing to the fallback.
Critique is a quality lane, not the product, so this path degrades instead
of failing closed: it records a PIPELINE-FAULT (surfaced later in the
report's Pipeline notes) and rewrites `critique` to a neutral note, then
continues to `reflexion_gate` — the stage critique's own `next` pointed to.
The rewritten note deliberately contains NO "VERDICT:" line:
reflexion_gate's parse_verdict defaults to PASS when no verdict line is
found (its documented malformed-critique behavior), so a dead critique
proceeds to synthesis instead of burning the revision budget on a fault.
"""
import json
import os
MAX_DETAIL_CHARS = 500
CRITIQUE_NOTE = (
"Critique unavailable — the critique stage failed (pipeline fault); "
"proceeding to synthesis without an automated review pass."
)
def load_state():
if path := os.environ.get("GRAPH_STATE_FILE"):
with open(path) as f:
return json.load(f)
return json.loads(os.environ.get("GRAPH_STATE", "{}"))
def safe_faults():
try:
state = load_state()
return [f for f in (state.get("pipeline_faults") or []) if isinstance(f, str)]
except Exception: # noqa: BLE001 — best-effort preservation only
return []
def main():
state = load_state()
faults = [f for f in (state.get("pipeline_faults") or []) if isinstance(f, str)]
raw = state.get("critique")
detail = raw.strip().replace("\n", " ") if isinstance(raw, str) else ""
if not detail.startswith("LLM node"):
detail = "the critique stage died without recording a failure detail"
if len(detail) > MAX_DETAIL_CHARS:
detail = detail[: MAX_DETAIL_CHARS - 1] + "…"
fault = f"PIPELINE-FAULT: critique failed — {detail}"
if fault not in faults:
faults.append(fault)
print(json.dumps({"pipeline_faults": faults, "critique": CRITIQUE_NOTE}))
try:
main()
except Exception as e: # noqa: BLE001 — the fault marker itself must never crash
faults = safe_faults()
faults.append(f"PIPELINE-FAULT: critique failed — fault-marker script error: {e}")
print(
json.dumps(
{
"pipeline_faults": faults,
"critique": CRITIQUE_NOTE,
}
)
)