Dark-Alex-17
156f3193a9
code-reviewer verdict.py: finding-verifier now completes "successfully" with fault markers INSIDE its FINDING_VERIFIER_RESULTS payload (parse_fault's `pipeline-fault` sentinel id, verify_fault / verdict_gate crash-guard notes prefixed PIPELINE-FAULT:), which the old "Agent node failed:" prefix check never saw — a dead parse degraded silently to "unverified" with no fault recorded. Detect them, anchored on the sentinel id / note prefix only (never a substring over the report), counted over the raw entry list so duplicate crash-guard ids are not collapsed. Faults lead the attention list uncapped; the display cap of 5 applies to non-fault entries; the "always-human trigger(s) fired" reason fires only when non-fault triggers exist. code-reviewer render.py: CONTRACT comments at the verdict and summary lines review-gauntlet's gate parses; every LLM-influenced value rendered after the summary line (dropped_titles, rigor, bar_provenance, resolved_surfaces, reason) is newline-sanitised so none can forge a second summary line. finding-verifier verdict_gate.py: the outer crash-guard note carries the PIPELINE-FAULT: prefix (the after-retry UNVERIFIABLE path stays unprefixed — model misbehaviour, not infrastructure); README names all three emitters. deep-research: vet_fault's UNVETTED note tells the critic not to request revision solely because the assessment is unavailable (it matched critique's PREPRINT/UNVERIFIED REVISE trigger verbatim, burning the revision budget on an infra fault); vet_fault and critique_fault dedupe repeated faults and preserve accumulated faults on crash via safe_faults(); question_fault keeps its plan-pinned prefix and appends the lane's question identity so distinct dead lanes no longer collapse in combine_findings' dedupe.
74 lines
4.1 KiB
Markdown
74 lines
4.1 KiB
Markdown
# finding-verifier
|
||
|
||
A **graph** agent that independently verifies code-review findings against the working
|
||
tree. It exists because false positives kill hands-off trust faster than missed bugs do —
|
||
and because an orchestrator verifying its own workers' findings inline can skip, fudge,
|
||
or confirmation-bias the check. A graph can't: the structure IS the discipline.
|
||
|
||
## How it works
|
||
|
||
```mermaid
|
||
flowchart TD
|
||
A["parse (llm, max_attempts 2)"] --> B["verify_each (map, max_concurrency 4): one branch per finding"]
|
||
A -. "fallback" .-> PF["parse_fault (script): verdicts = one UNVERIFIABLE PIPELINE-FAULT entry"]
|
||
PF --> D
|
||
B -. "spawns × N" .-> C["verify_one (llm + fs tools, max_attempts 2)"]
|
||
C --> G["verdict_gate (script): machine-validates the JSON contract"]
|
||
G -. "reject-retry once" .-> C
|
||
C -. "fallback" .-> VF["verify_fault (script): branch-local UNVERIFIABLE marker"]
|
||
B --> D(["done (end): FINDING_VERIFIER_RESULTS"])
|
||
```
|
||
|
||
1. **parse** — extracts every 🔴/🟡 finding from the spawn prompt into a structured list
|
||
(id, severity verbatim, path, lines, self-contained claim). Extraction only — no judging.
|
||
Gets two attempts (`max_attempts: 2`); if both fail, its `fallback` routes to
|
||
`parse_fault`, which rewrites `verdicts` to a single UNVERIFIABLE entry naming the fault
|
||
and jumps straight to `done`.
|
||
2. **verify_each** — a `map` node fans out one `verify_one` branch per finding. Skipping a
|
||
finding is structurally impossible; results collect in input order.
|
||
3. **verify_one** — reads the cited code (greps for moved code before concluding anything)
|
||
and answers ONE narrow question: does the cited code exist, and does it behave as the
|
||
claim says? Verdicts:
|
||
- `VERIFIED` — with the decisive line(s) quoted verbatim (`path:line`).
|
||
- `FALSE` — the code doesn't exist anywhere or provably doesn't do what the claim says.
|
||
Requires PROOF, not disagreement — doubt never lands here.
|
||
- `UNVERIFIABLE` — couldn't establish either way (runtime behavior, external systems).
|
||
|
||
Each branch is a chain: `verify_one → verdict_gate`. The gate machine-validates the JSON
|
||
contract (verdict enum; evidence required for VERIFIED/FALSE; note for UNVERIFIABLE),
|
||
stamps the authoritative id FROM the finding item (the model's echo is never trusted),
|
||
rejects back to `verify_one` exactly once with the reason, and on a second failure records
|
||
UNVERIFIABLE — a malformed branch never sinks the map. If `verify_one` itself dies (both
|
||
`max_attempts: 2` attempts fail), its `fallback` routes to `verify_fault`, which
|
||
normalizes the engine's failure text into a branch-local UNVERIFIABLE verdict the map
|
||
still collects.
|
||
4. **done** — a deterministic `end` node emits the raw verdict list plus the caller
|
||
contract (VERIFIED = keep + paste evidence; FALSE = drop + tally; UNVERIFIABLE = keep,
|
||
marked). No final LLM step that could editorialize.
|
||
|
||
The dashed edges are fault paths. All three fault emitters (`parse_fault`, `verify_fault`,
|
||
and `verdict_gate`'s own crash guard) surface as `PIPELINE-FAULT:` text in a verdict's
|
||
`note`, and a fault can only ever produce UNVERIFIABLE — degraded output is never a
|
||
passing verdict. Every path ends at `done`, so the `FINDING_VERIFIER_RESULTS`
|
||
sentinel is always emitted and the caller's verify lane degrades instead of dying.
|
||
|
||
## What it is NOT
|
||
|
||
- **Not a second opinion.** It never re-judges severity, style, or whether a finding
|
||
"matters" — that authority stays with the reviewers and the orchestrator's rigor rules.
|
||
- **Not a reviewer.** It reads only what findings cite (plus greps to locate moved code).
|
||
|
||
## Caller
|
||
|
||
Spawned by [`code-reviewer`](../code-reviewer/README.md) during its Finding Verifier
|
||
pass, ONCE per review, with the full 🔴/🟡 list and `project_dir`. If this agent isn't
|
||
installed, code-reviewer runs the same contract inline as a fallback.
|
||
|
||
## Files
|
||
|
||
- `graph.yaml` — the agent (no tools.sh; the graph grants `fs_read`/`fs_cat`/`fs_grep`
|
||
per node).
|
||
- `scripts/verdict_gate.py` — the deterministic contract gate.
|
||
- `scripts/parse_fault.py`, `scripts/verify_fault.py` — the fault markers behind the
|
||
dashed fallback edges.
|