docs: sync agent READMEs + diagrams with resilience hardening

This commit is contained in:
Dark-Alex-17 committed 2026-09-14 10:50:34 -06:00
1 parent f230c28bf4
commit fdae98adff
8 files changed
+216 -30

No files matched your search

+22 -4
View File
@@ -49,6 +49,12 @@ Complaints:
2. ... 2. ...
``` ```
Every verdict — CONFORMS or DIVERGES — ends with a `Verification runs:` section: the recorded
PASS/FAIL results (command, exit code, duration, failure tail) of the caller-declared verification
commands, or an explicit marker when none were declared or the run never reached that stage. The
one exception is the verdict script's own crash guard, which emits a bare DIVERGES with a
"verdict computation error" line — and no section — when verdict computation itself dies.
A `DIVERGES` verdict **blocks** completion. The caller (sisyphus/architect) must reconcile it — A `DIVERGES` verdict **blocks** completion. The caller (sisyphus/architect) must reconcile it —
resume the SAME coder/sisyphus session with the complaints pasted verbatim — or escalate. It mirrors resume the SAME coder/sisyphus session with the complaints pasted verbatim — or escalate. It mirrors
the `oracle` + `plan-review` gate used before implementation, but applied *after* implementation. the `oracle` + `plan-review` gate used before implementation, but applied *after* implementation.
@@ -60,21 +66,29 @@ violation) and cites `file:line`. Vague complaints are not emitted.
```mermaid ```mermaid
flowchart TD flowchart TD
A["parse (llm): extract acceptance criteria"] --> B["diff_facts (script): resolve + bound the diff"] A["parse (llm): extract acceptance criteria + declared verification commands"] --> B["diff_facts (script): resolve + bound the diff"]
B --> C["map over criteria: one fresh-context branch each (parallel)"] B --> R["run_checks (script): run the declared verification commands ONCE, record results"]
R --> C["map over criteria: one fresh-context branch each (parallel)"]
C --> D["check_criterion (llm + fs/ast tools)"] C --> D["check_criterion (llm + fs/ast tools)"]
D --> E["crit_gate (script): MET needs change+test evidence"] D --> E["crit_gate (script): MET needs change+test evidence"]
E -. "reject-retry once; exhausted = PARTIAL" .-> D E -. "reject-retry once; exhausted = PARTIAL" .-> D
E --> F["holistic (llm): ADDITIVE-only hunt - absence, scope drift, substitution, gamed tests"] E --> F["holistic (llm): ADDITIVE-only hunt - absence, scope drift, substitution, gamed tests"]
F --> G["verdict (script): CONFORMS iff ALL MET + zero extras; no criteria = DIVERGES"] F --> G["verdict (script): CONFORMS iff ALL MET + zero extras + zero faults; no criteria = DIVERGES"]
G --> H(["ADVERSARIAL_REVIEW: CONFORMS | DIVERGES"]) G --> H(["ADVERSARIAL_REVIEW: CONFORMS | DIVERGES"])
A -. "fallback" .-> P["pipeline_fault (script): record PIPELINE-FAULT marker"]
B -. "fallback" .-> P
R -. "fallback" .-> P
F -. "fallback" .-> G
P --> G
``` ```
The conformance doctrine is baked into the graph's own nodes: The conformance doctrine is baked into the graph's own nodes:
1. **Every** acceptance criterion gets its own fresh-context verification branch — MET / PARTIAL / UNMET / DIVERGED, machine-gated (MET requires the satisfying change AND the proving test, cited). No test ⇒ at best PARTIAL. Partial coverage of the criteria list is structurally impossible. 1. **Every** acceptance criterion gets its own fresh-context verification branch — MET / PARTIAL / UNMET / DIVERGED, machine-gated (MET requires the satisfying change AND the proving test, cited). No test ⇒ at best PARTIAL. Partial coverage of the criteria list is structurally impossible.
2. Ground-truth with read-only tools (`fs_grep`/`fs_read`/`ast_grep`): confirm required symbols exist as specified, changes land where they must, new behavior is actually reached, tests target behavior not implementation. 2. Ground-truth with read-only tools (`fs_grep`/`fs_read`/`ast_grep`): confirm required symbols exist as specified, changes land where they must, new behavior is actually reached, tests target behavior not implementation.
3. Hunt adversarially for the **absent**: skipped criteria, scope creep, interface/approach substitution, out-of-scope touches, downstream contract breakage. 3. Execution evidence is recorded **once, deterministically**: `run_checks` runs ONLY the caller-declared verification commands (never invented or auto-detected), sequentially under a 3300s total deadline — commands the deadline leaves no budget for are recorded as SKIPPED. Criterion branches have no execution tools by design: "tests pass"-style criteria are judged against the record — a recorded green run is MET citing it; no declared command ⇒ at best PARTIAL.
4. Hunt adversarially for the **absent**: skipped criteria, scope creep, interface/approach substitution, out-of-scope touches, downstream contract breakage.
5. **Fail closed on pipeline faults**: if parse, diff resolution, or the verification runner dies, its fallback routes to `pipeline_fault`, which records a `PIPELINE-FAULT:` marker and continues to the verdict (a failed holistic pass falls back straight to the verdict, which synthesizes the same marker). Any recorded fault forces DIVERGES with the fault as complaint #1 — degraded is never passing, and the sentinel is always emitted: the graph never dies without one.
It is **read-only** — it produces a verdict, never a fix. It is **read-only** — it produces a verdict, never a fix.
@@ -93,6 +107,9 @@ Run get_diff (or --base main), or: <paste diff>
## PLAN — acceptance criteria to check against ## PLAN — acceptance criteria to check against
<paste the task index.md body + the relevant PLAN-*.md section, verbatim> <paste the task index.md body + the relevant PLAN-*.md section, verbatim>
## VERIFICATION (optional)
<exact build/test/lint commands to run, one per line — undeclared commands are never guessed>
" "
``` ```
@@ -106,6 +123,7 @@ coyote -a adversary --agent-variable project_dir /path/to/repo \
### Tools ### Tools
- The graph's `diff_facts` script resolves the diff itself (staged → unstaged → `HEAD~1`, or an explicit ref/range named in the prompt) — there is no `get_diff` tool anymore. - The graph's `diff_facts` script resolves the diff itself (staged → unstaged → `HEAD~1`, or an explicit ref/range named in the prompt) — there is no `get_diff` tool anymore.
- The `run_checks` script executes the caller-declared verification commands and records the results — criterion branches cite the record and never run anything themselves.
- Criterion branches carry read-only `fs_read`/`fs_cat`/`fs_grep`/`ast_grep` for ground-truth checks. - Criterion branches carry read-only `fs_read`/`fs_cat`/`fs_grep`/`ast_grep` for ground-truth checks.
## Related ## Related
+3
View File
@@ -160,6 +160,9 @@ By default (`parallel_tasks: 0`) tasks run **sequentially** on the single run br
- Architect integrates: completed task branches merge into the run branch **one at a time**, with a - Architect integrates: completed task branches merge into the run branch **one at a time**, with a
full build + test run after every merge. Conflicts go back to that task's Sisyphus session to full build + test run after every merge. Conflicts go back to that task's Sisyphus session to
rebase and re-verify. rebase and re-verify.
- Optionally, the `agent__task_*` queue may mirror the on-disk `blocked_by` graph so completing a
task auto-dispatches its unblocked dependents. The queue is in-memory convenience only — disk task
files remain the durable source of truth.
- Worktrees and task branches are cleaned up after each clean merge. Phase F (single draft PR + - Worktrees and task branches are cleaned up after each clean merge. Phase F (single draft PR +
CI-check watch) is unchanged in both modes. CI-check watch) is unchanged in both modes.
+29 -6
View File
@@ -14,21 +14,44 @@
```mermaid ```mermaid
flowchart TD flowchart TD
A["parse (llm)"] --> B["facts (script): diff, quality bar, ledger, history, trigger signals"] A["parse (llm)"] --> B["facts (script): diff, quality bar, ledger, history, trigger signals"]
A -. "fallback" .-> PF["parse_fault (script): PIPELINE-FAULT verdict"]
PF --> L
B --> C["linters (script): tflint / hadolint / actionlint / buf breaking"] B --> C["linters (script): tflint / hadolint / actionlint / buf breaking"]
C --> D["group_domains (script) --> refine_groups (llm) --> cover_gate (script: exact cover)"] C --> D1["group_propose (script): deterministic domain proposal"]
D --> E["map over domain slices (parallel): domain-reviewer leaders, teammates on"] D1 --> D2["refine_groups (llm)"]
D --> F["aux_lanes (llm): org-context + prior-art spawns"] D2 --> D3["cover_gate (script: exact cover)"]
D --> G["downstream_sweep (script)"] D2 -. "fallback" .-> D3
D3 --> E["map over domain slices (parallel): domain-reviewer leaders, teammates on, max_attempts: 2"]
D3 --> F["aux_lanes (llm): org-context + prior-art spawns"]
D3 --> G["downstream_sweep (script)"]
E --> H["completeness_gate (script): required sections even when clean"] E --> H["completeness_gate (script): required sections even when clean"]
E -. "fallback" .-> DF["domain_fault (script): ⚠️ PIPELINE-FAULT slice banner"]
H -. "reject-retry once" .-> E H -. "reject-retry once" .-> E
H --> I["synthesize (llm: dedup + prose, NO verdict authority)"] H --> I["synthesize (llm: dedup + prose, NO verdict authority)"]
F --> I F --> I
F -. "fallback" .-> AF["aux_fault (script): PIPELINE-FAULT lane-skip note"]
AF --> I
G --> I G --> I
I --> J["verify: finding-verifier (agent node, unskippable)"] I --> J["verify: finding-verifier (agent node, unskippable, max_attempts: 2)"]
J --> K["verdict (script): FALSE-drops, rigor folding, MERGE-READY / NEEDS-HUMAN arithmetic"] I -. "fallback" .-> K
J --> K["verdict (script): FALSE-drops, rigor folding, fault accounting, MERGE-READY / NEEDS-HUMAN arithmetic"]
J -. "fallback" .-> K
K --> L["render (script): the standard report"] K --> L["render (script): the standard report"]
``` ```
**Failure fails closed.** Every LLM/agent node retries once (`max_attempts: 2` — the
domain-reviewer lanes and the verifier included), then routes to a `fallback:` instead of killing
the graph, and every fallback that loses review coverage leaves a `PIPELINE-FAULT` marker: a dead
`parse` emits a complete fault verdict and jumps straight to render; a dead domain lane becomes a
`> ⚠️ PIPELINE-FAULT` banner where its slice report would be; a dead `refine_groups` falls
through to the cover gate, which keeps the deterministic proposal; a dead synthesis or verifier
falls through to the verdict script. The verdict script counts every fault and forces
`NEEDS-HUMAN` — a degraded run never reads `MERGE-READY`. The aux context lanes are the one
exception: they are enrichment, so their fault (`aux_fault`) is surfaced as a lane-skip note but
never blocks. Fault detection is prefix-anchored (a report that merely *quotes* a marker doesn't
trip it), and the report is always emitted: the verdict script's own crash guard emits a
`PIPELINE-FAULT` NEEDS-HUMAN, and render bypasses its "No changes to review." stub for fault
verdicts — a fault renders as a full report, never silence.
A CodeRabbit-style code review orchestrator that coordinates per-file reviews and synthesizes findings into a unified A CodeRabbit-style code review orchestrator that coordinates per-file reviews and synthesizes findings into a unified
report. report.
+67 -6
View File
@@ -22,11 +22,13 @@ agent, this is the file to read alongside the
## Workflow ## Workflow
17 nodes. Solid arrows are static `next` / `routes` edges declared in 23 nodes. Solid arrows are static `next` / `routes` edges declared in
`graph.yaml`; script nodes can also route dynamically via `_next` (shown as `graph.yaml`; script nodes can also route dynamically via `_next` (shown as
labeled branches out of the diamond). Dotted arrows show `map` fan-out — the labeled branches out of the diamond). Dotted arrows show `map` fan-out — the
`research_each_question` node spawns one `research_one_question` branch per `research_each_question` node spawns one `research_one_question` branch per
sub-question and joins them before continuing. sub-question and joins them before continuing — and `fallback` edges
(labeled "fallback"), taken when a node fails outright (see *Fault paths*
below).
```mermaid ```mermaid
flowchart TD flowchart TD
@@ -38,18 +40,27 @@ flowchart TD
bootstrap_research --> plan bootstrap_research --> plan
bootstrap_research --> knowledge_lookup bootstrap_research --> knowledge_lookup
plan["plan<br/>llm + output_schema"] --> research_each_question plan["plan<br/>llm + output_schema"] --> research_each_question
plan -. "fallback" .-> plan_fault{"plan_fault<br/>script"}
plan_fault --> end_fault
knowledge_lookup[("knowledge_lookup<br/>rag")] --> research_each_question knowledge_lookup[("knowledge_lookup<br/>rag")] --> research_each_question
research_each_question[\research_each_question<br/>map/] research_each_question[\research_each_question<br/>map/]
research_each_question -. "spawns × N" .-> research_one_question["research_one_question<br/>llm + web tools"] research_each_question -. "spawns × N" .-> research_one_question["research_one_question<br/>llm + web tools"]
research_one_question -. "fallback" .-> question_fault{"question_fault<br/>script"}
research_each_question --> combine_findings research_each_question --> combine_findings
combine_findings{"combine_findings<br/>script"} --> vet_sources combine_findings{"combine_findings<br/>script"} --> vet_sources
vet_sources["vet_sources<br/>llm + classify_source"] --> critique vet_sources["vet_sources<br/>llm + classify_source"] --> critique
vet_sources -. "fallback" .-> vet_fault{"vet_fault<br/>script"}
vet_fault --> critique
critique["critique<br/>llm"] --> reflexion_gate critique["critique<br/>llm"] --> reflexion_gate
critique -. "fallback" .-> critique_fault{"critique_fault<br/>script"}
critique_fault --> reflexion_gate
reflexion_gate{"reflexion_gate<br/>script"} reflexion_gate{"reflexion_gate<br/>script"}
reflexion_gate -->|"PASS"| synthesize reflexion_gate -->|"PASS"| synthesize
reflexion_gate -->|"REVISE (budget left)"| research_each_question reflexion_gate -->|"REVISE (budget left)"| research_each_question
reflexion_gate -->|"REVISE (budget spent)"| synthesize reflexion_gate -->|"REVISE (budget spent)"| synthesize
synthesize[["synthesize<br/>agent → report-writer"]] --> verify_sources synthesize[["synthesize<br/>agent → report-writer"]] --> verify_sources
synthesize -. "fallback" .-> synth_fault{"synth_fault<br/>script"}
synth_fault --> end_fault
verify_sources{"verify_sources<br/>script"} --> approve verify_sources{"verify_sources<br/>script"} --> approve
approve{{"approve<br/>approval"}} approve{{"approve<br/>approval"}}
approve -->|"accept"| end_accepted approve -->|"accept"| end_accepted
@@ -59,13 +70,14 @@ flowchart TD
end_accepted(["end_accepted<br/>report"]) end_accepted(["end_accepted<br/>report"])
end_rejected(["end_rejected"]) end_rejected(["end_rejected"])
end_fault(["end_fault<br/>DEEP_RESEARCH FAILED"])
``` ```
### Node-type breakdown ### Node-type breakdown
| Type | Nodes | | Type | Nodes |
|-----------------------------|-----------------------------------------------------------------------------------------------------------------------| |-----------------------------|-----------------------------------------------------------------------------------------------------------------------|
| `script` (Python) | `parse_request`, `bootstrap_research`, `combine_findings`, `reflexion_gate`, `verify_sources`, `incorporate_feedback` | | `script` (Python) | `parse_request`, `bootstrap_research`, `combine_findings`, `reflexion_gate`, `verify_sources`, `incorporate_feedback`, plus the fault markers `plan_fault`, `question_fault`, `vet_fault`, `critique_fault`, `synth_fault` |
| `llm` (tools: `[]`) | `plan`, `critique` | | `llm` (tools: `[]`) | `plan`, `critique` |
| `llm` (with tool whitelist) | `research_one_question`, `vet_sources` | | `llm` (with tool whitelist) | `research_one_question`, `vet_sources` |
| `rag` | `knowledge_lookup` — local corpus retrieval | | `rag` | `knowledge_lookup` — local corpus retrieval |
@@ -73,7 +85,7 @@ flowchart TD
| `agent` | `synthesize` — spawns the `report-writer` sub-agent | | `agent` | `synthesize` — spawns the `report-writer` sub-agent |
| `input` | `ask_topic` | | `input` | `ask_topic` |
| `approval` | `approve` | | `approval` | `approve` |
| `end` | `end_accepted`, `end_rejected` | | `end` | `end_accepted`, `end_rejected`, `end_fault` |
## Parallel execution ## Parallel execution
@@ -217,6 +229,46 @@ folds that text into `research_feedback` and loops back to
`settings.max_loop_iterations` (40) is the engine's infinite-loop `settings.max_loop_iterations` (40) is the engine's infinite-loop
backstop: it caps the total visits to any single node. backstop: it caps the total visits to any single node.
## Fault paths
Every stage that can die outright has a `fallback:` edge to a small
fault-marker script — the dashed edges in the diagram. A fallback fires
after a node's retries are exhausted (`research_one_question` and
`synthesize` each get two attempts via `max_attempts: 2`); the engine
writes its failure text into the node's state key before routing, and
the marker normalizes that text into a `PIPELINE-FAULT: ...` entry in
`pipeline_faults`. The five routes:
- `plan -> plan_fault -> end_fault` — fail-closed: no plan means
nothing to research.
- `research_one_question -> question_fault` — branch-local, inside the
map branch: one dead research lane becomes a fault finding instead of
sinking the whole map.
- `vet_sources -> vet_fault -> critique` — degrade-visibly: vetting is
a quality lane, not the product; `source_assessment` becomes a
neutral "unvetted" note.
- `critique -> critique_fault -> reflexion_gate` — degrade-visibly: the
rewritten critique deliberately has no `VERDICT:` line, so the gate's
malformed-critique PASS default proceeds to synthesis.
- `synthesize -> synth_fault -> end_fault` — fail-closed: no report
means no product.
On the fail-closed routes, `end_fault` renders the
`DEEP_RESEARCH FAILED — PIPELINE-FAULT: ...` sentinel. On the
degrade-visibly routes, `verify_sources` folds the accumulated faults
into a `## Pipeline notes` section shown at the approval step and
appended to the accepted report (`{{report}}{{pipeline_notes}}`;
empty when there were no faults). Degraded output is never a silent
pass — every fault is visible to the reviewer or the caller — and some
end node always renders its sentinel, so the graph degrades instead of
dying.
All 11 scripts also carry a top-level crash guard (R3): a script that
throws records a fault and fails forward where the flow allows it (a
crashed `reflexion_gate` proceeds to `synthesize`; a crashed
`verify_sources` reports that sources were NOT checked) instead of
killing the run.
## Running ## Running
```sh ```sh
@@ -241,6 +293,9 @@ coyote -a deep-research # no prompt -> triggers ask_topic
- `verify_sources` probes every cited URL / DOI with an HTTP HEAD - `verify_sources` probes every cited URL / DOI with an HTTP HEAD
request and reports which are unreachable, so the human reviewer request and reports which are unreachable, so the human reviewer
sees broken citations before approving. sees broken citations before approving.
- Fault paths degrade visibly, never silently: a `PIPELINE-FAULT`
can only ever surface as a failure sentinel or a `## Pipeline
notes` section, never as a clean report.
## Customizing ## Customizing
@@ -270,7 +325,7 @@ coyote -a deep-research # no prompt -> triggers ask_topic
``` ```
assets/agents/deep-research/ assets/agents/deep-research/
graph.yaml - agent config + 17-node workflow graph.yaml - agent config + 23-node workflow
tools.sh - classify_source custom tool tools.sh - classify_source custom tool
README.md - this file README.md - this file
knowledge/ knowledge/
@@ -279,9 +334,15 @@ assets/agents/deep-research/
scripts/ scripts/
parse_request.py - _next: bootstrap_research, or ask_topic if no topic parse_request.py - _next: bootstrap_research, or ask_topic if no topic
bootstrap_research.py - fan-out source: next [plan, knowledge_lookup] bootstrap_research.py - fan-out source: next [plan, knowledge_lookup]
plan_fault.py - fail-closed fault marker: dead plan -> end_fault
question_fault.py - branch-local fault marker for a dead research lane
combine_findings.py - joins map output (question_findings) into findings combine_findings.py - joins map output (question_findings) into findings
vet_fault.py - degrade-visibly fault marker: unvetted sources
critique_fault.py - degrade-visibly fault marker: skipped critique
reflexion_gate.py - _next: research_each_question (revise) or synthesize reflexion_gate.py - _next: research_each_question (revise) or synthesize
verify_sources.py - HTTP HEAD on cited URLs / DOIs synth_fault.py - fail-closed fault marker: no report -> end_fault
verify_sources.py - HTTP HEAD on cited URLs / DOIs; folds pipeline_faults
into the report's "## Pipeline notes" section
incorporate_feedback.py - _next: research_each_question, with user feedback incorporate_feedback.py - _next: research_each_question, with user feedback
``` ```
+25 -5
View File
@@ -7,13 +7,23 @@ or confirmation-bias the check. A graph can't: the structure IS the discipline.
## How it works ## How it works
``` ```mermaid
parse (llm) ──▶ verify_each (map, max_concurrency 4) ──▶ done (end) flowchart TD
└─▶ verify_one (llm + fs tools) ──▶ verdict_gate (script), one CHAIN per finding A["parse (llm, max_attempts 2)"] --> B["verify_each (map, max_concurrency 4): one branch per finding"]
A -. "fallback" .-> PF["parse_fault (script): verdicts = one UNVERIFIABLE PIPELINE-FAULT entry"]
PF --> D
B -. "spawns × N" .-> C["verify_one (llm + fs tools, max_attempts 2)"]
C --> G["verdict_gate (script): machine-validates the JSON contract"]
G -. "reject-retry once" .-> C
C -. "fallback" .-> VF["verify_fault (script): branch-local UNVERIFIABLE marker"]
B --> D(["done (end): FINDING_VERIFIER_RESULTS"])
``` ```
1. **parse** — extracts every 🔴/🟡 finding from the spawn prompt into a structured list 1. **parse** — extracts every 🔴/🟡 finding from the spawn prompt into a structured list
(id, severity verbatim, path, lines, self-contained claim). Extraction only — no judging. (id, severity verbatim, path, lines, self-contained claim). Extraction only — no judging.
Gets two attempts (`max_attempts: 2`); if both fail, its `fallback` routes to
`parse_fault`, which rewrites `verdicts` to a single UNVERIFIABLE entry naming the fault
and jumps straight to `done`.
2. **verify_each** — a `map` node fans out one `verify_one` branch per finding. Skipping a 2. **verify_each** — a `map` node fans out one `verify_one` branch per finding. Skipping a
finding is structurally impossible; results collect in input order. finding is structurally impossible; results collect in input order.
3. **verify_one** — reads the cited code (greps for moved code before concluding anything) 3. **verify_one** — reads the cited code (greps for moved code before concluding anything)
@@ -28,12 +38,19 @@ parse (llm) ──▶ verify_each (map, max_concurrency 4) ──▶ done (end)
contract (verdict enum; evidence required for VERIFIED/FALSE; note for UNVERIFIABLE), contract (verdict enum; evidence required for VERIFIED/FALSE; note for UNVERIFIABLE),
stamps the authoritative id FROM the finding item (the model's echo is never trusted), stamps the authoritative id FROM the finding item (the model's echo is never trusted),
rejects back to `verify_one` exactly once with the reason, and on a second failure records rejects back to `verify_one` exactly once with the reason, and on a second failure records
UNVERIFIABLE — a malformed branch never sinks the map. UNVERIFIABLE — a malformed branch never sinks the map. If `verify_one` itself dies (both
Kept, marked unverified. `max_attempts: 2` attempts fail), its `fallback` routes to `verify_fault`, which
normalizes the engine's failure text into a branch-local UNVERIFIABLE verdict the map
still collects.
4. **done** — a deterministic `end` node emits the raw verdict list plus the caller 4. **done** — a deterministic `end` node emits the raw verdict list plus the caller
contract (VERIFIED = keep + paste evidence; FALSE = drop + tally; UNVERIFIABLE = keep, contract (VERIFIED = keep + paste evidence; FALSE = drop + tally; UNVERIFIABLE = keep,
marked). No final LLM step that could editorialize. marked). No final LLM step that could editorialize.
The dashed edges are fault paths. Both fault markers surface as `PIPELINE-FAULT:` text in
a verdict's `note`, and a fault can only ever produce UNVERIFIABLE — degraded output is
never a passing verdict. Every path ends at `done`, so the `FINDING_VERIFIER_RESULTS`
sentinel is always emitted and the caller's verify lane degrades instead of dying.
## What it is NOT ## What it is NOT
- **Not a second opinion.** It never re-judges severity, style, or whether a finding - **Not a second opinion.** It never re-judges severity, style, or whether a finding
@@ -50,3 +67,6 @@ installed, code-reviewer runs the same contract inline as a fallback.
- `graph.yaml` — the agent (no tools.sh; the graph grants `fs_read`/`fs_cat`/`fs_grep` - `graph.yaml` — the agent (no tools.sh; the graph grants `fs_read`/`fs_cat`/`fs_grep`
per node). per node).
- `scripts/verdict_gate.py` — the deterministic contract gate.
- `scripts/parse_fault.py`, `scripts/verify_fault.py` — the fault markers behind the
dashed fallback edges.
+37 -7
View File
@@ -11,13 +11,21 @@ gets the chance to rationalize past it.
```mermaid ```mermaid
flowchart TD flowchart TD
parse["parse<br/>(llm: extract request facts)"] --> signals["signals<br/>(script: deterministic diff facts)"] parse["parse<br/>(llm: extract request facts)"] --> signals["signals<br/>(script: deterministic diff facts)"]
parse -. "fallback" .-> pfault["parse_fault<br/>(script: PIPELINE-FAULT marker)"]
pfault --> gate
signals --> select["select_lanes<br/>(llm: ADDITIVE-only judgment)"] signals --> select["select_lanes<br/>(llm: ADDITIVE-only judgment)"]
select --> build["build_items<br/>(script: rules floor ∪ additions;<br/>forced lanes override exactly)"] select --> build["build_items<br/>(script: rules floor ∪ additions;<br/>forced lanes override exactly)"]
select -. "fallback" .-> dlanes["default_lanes<br/>(script: deterministic degraded<br/>selection — wider, never narrower)"]
dlanes --> build
build --> mcr["map: code-review"] & madv["map: adversary"] & msec["map: security"] & mpb["map: probe"] build --> mcr["map: code-review"] & madv["map: adversary"] & msec["map: security"] & mpb["map: probe"]
mcr -->|"over: 0-or-1 items"| rcr[["code-reviewer"]] mcr -->|"over: 0-or-1 items"| rcr[["code-reviewer<br/>(max_attempts: 2)"]]
madv --> radv[["adversary"]] madv --> radv[["adversary<br/>(max_attempts: 2)"]]
msec --> rsec[["security-reviewer"]] msec --> rsec[["security-reviewer<br/>(max_attempts: 2)"]]
mpb --> rpb[["probe"]] mpb --> rpb[["probe<br/>(max_attempts: 2)"]]
rcr -. "fallback" .-> lfault["lane_fault<br/>(script: PIPELINE-FAULT lane result,<br/>collected by the map)"]
radv -. "fallback" .-> lfault
rsec -. "fallback" .-> lfault
rpb -. "fallback" .-> lfault
mcr & madv & msec & mpb --> gate["verdict_gate<br/>(script: sentinel regex — deterministic)"] mcr & madv & msec & mpb --> gate["verdict_gate<br/>(script: sentinel regex — deterministic)"]
gate --> done(["GAUNTLET: PASS | BLOCKED"]) gate --> done(["GAUNTLET: PASS | BLOCKED"])
``` ```
@@ -49,6 +57,21 @@ those rules are *structure*:
with an environment note — it is never treated as PASS or FAIL. with an environment note — it is never treated as PASS or FAIL.
- **Verdict independence is structural**: lanes run as isolated sub-agents - **Verdict independence is structural**: lanes run as isolated sub-agents
with no `teammates:` flag — no cross-lane messaging, by construction. with no `teammates:` flag — no cross-lane messaging, by construction.
- **Failure fails closed, structurally** (`fallback:` routes + `PIPELINE-FAULT`
markers): every LLM node and lane retries once (`max_attempts: 2`), then a
fallback fires instead of killing the graph — and every fallback degrades
toward BLOCKED, never toward a pass. A dead lane becomes a `PIPELINE-FAULT:`
lane result (`lane_fault.py`) that the gate reports as BLOCKED *naming the
lane* — distinct from SKIPPED (not selected). A dead `parse` records a fault
and jumps straight to the gate (`parse_fault.py`). A dead `select_lanes`
degrades to a deterministic selection (`default_lanes.py`): code-review +
adversary, plus probe on consumer surface, plus security on auth/deps/exec
signals or hardened posture, unioned with caller-forced lanes — wider, never
narrower. A crashed `build_items` records a fault so an all-SKIPPED gate
still blocks. Fault detection is prefix-anchored — a real review that merely
*quotes* "PIPELINE-FAULT:" flows to the normal sentinel rules. And the
`GAUNTLET:` sentinel is always emitted: even a crashed gate prints
`GAUNTLET: BLOCKED`, never silence.
## Sentinels parsed ## Sentinels parsed
@@ -74,6 +97,9 @@ Plan / acceptance criteria:
Local-run recipe / usage suites: Local-run recipe / usage suites:
<how to boot the service locally + where existing suites live — enables the probe lane> <how to boot the service locally + where existing suites live — enables the probe lane>
Verification commands: (optional — exact build/test/lint commands, passed
verbatim to the adversary lane; omitted = none declared, never invented)
Forced lanes: (optional — e.g. 'adversary, probe' to run exactly those)" Forced lanes: (optional — e.g. 'adversary, probe' to run exactly those)"
``` ```
@@ -89,9 +115,13 @@ final report verbatim.
- The caller decides *whether* review is warranted at all (the gauntlet is for - The caller decides *whether* review is warranted at all (the gauntlet is for
non-trivial work — that's why code-review is unconditionally on); the non-trivial work — that's why code-review is unconditionally on); the
gauntlet decides *which* of the other lanes apply and what the verdicts mean. gauntlet decides *which* of the other lanes apply and what the verdicts mean.
- Lane sub-agents get generous finite timeouts (2h each; 3h whole-gauntlet). - Lane sub-agents get generous finite timeouts (2h each; 3h whole-gauntlet)
A lane that dies or times out produces no sentinel — which is a BLOCKED, and one retry (`max_attempts: 2`). A lane that still dies or times out falls
never a silent pass. back to the `lane_fault` marker — BLOCKED naming the lane, never a silent
pass.
- Caller-declared verification commands are extracted verbatim by `parse` and
passed through to the adversary lane's prompt — never invented or
auto-detected; undeclared stays undeclared.
- Signals failures degrade gracefully: if git can't compute the diff, lane - Signals failures degrade gracefully: if git can't compute the diff, lane
selection falls back to caller context + forced lanes, and the report notes selection falls back to caller context + forced lanes, and the report notes
it. The auth-path regex deliberately over-fires (`auth(?!or)` matches it. The auth-path regex deliberately over-fires (`auth(?!or)` matches
+1
View File
@@ -57,6 +57,7 @@ Spawnable sub-agents (from `config.yaml`):
- 💻 **CLI Coding**: Provides a natural language interface for writing and editing code. - 💻 **CLI Coding**: Provides a natural language interface for writing and editing code.
- 🔄 **Task Management**: Tracks progress and context across complex operations. - 🔄 **Task Management**: Tracks progress and context across complex operations.
- 🛠️ **Tool Integration**: Seamlessly uses system tools for building, testing, and file manipulation. - 🛠️ **Tool Integration**: Seamlessly uses system tools for building, testing, and file manipulation.
- 🧵 **Task-queue bursts**: Dependency-shaped fan-outs (explore → code → verify) may be encoded as `agent__task_create` chains — dependents auto-dispatch when `agent__task_complete` unblocks them; todos remain the tracking source of truth.
- 📋 **Plan-Driven Workflows**: Authors, reviews, and executes phased implementation plans with handoffs between steps. - 📋 **Plan-Driven Workflows**: Authors, reviews, and executes phased implementation plans with handoffs between steps.
- 🚦 **Escalation policy**: `escalation_policy=autonomous` pre-delegates decision-making — standing rulings (`~/.config/coyote/share/standing-rulings.md`, recorded via `coyote --macro record-ruling`) and evidence-backed recommended options are taken without pausing and batched into a “Decisions taken autonomously” digest; secrets, money movement, and destructive actions always escalate. - 🚦 **Escalation policy**: `escalation_policy=autonomous` pre-delegates decision-making — standing rulings (`~/.config/coyote/share/standing-rulings.md`, recorded via `coyote --macro record-ruling`) and evidence-backed recommended options are taken without pausing and batched into a “Decisions taken autonomously” digest; secrets, money movement, and destructive actions always escalate.
+32 -2
View File
@@ -27,6 +27,7 @@ flowchart TD
gate_blocked -->|"yes"| orient gate_blocked -->|"yes"| orient
gate_blocked -->|"no"| end_blocked gate_blocked -->|"no"| end_blocked
orient["orient<br/>llm, read-only"] --> route_staleness orient["orient<br/>llm, read-only"] --> route_staleness
orient -. "LLM fault" .-> note_llm_fault
route_staleness{"route_staleness<br/>script"} route_staleness{"route_staleness<br/>script"}
route_staleness -->|"major deviation"| gate_deviation route_staleness -->|"major deviation"| gate_deviation
route_staleness -->|"else"| implement route_staleness -->|"else"| implement
@@ -35,6 +36,7 @@ flowchart TD
gate_deviation -->|"abort"| end_rejected gate_deviation -->|"abort"| end_rejected
gate_deviation -->|"other (user guidance)"| implement gate_deviation -->|"other (user guidance)"| implement
implement[["implement<br/>agent → coder"]] --> route_coder_result implement[["implement<br/>agent → coder"]] --> route_coder_result
implement -. "coder crashed<br/>(no retries)" .-> end_failure
route_coder_result{"route_coder_result<br/>script"} route_coder_result{"route_coder_result<br/>script"}
route_coder_result -->|"CODER_COMPLETE"| verify_format_lint route_coder_result -->|"CODER_COMPLETE"| verify_format_lint
route_coder_result -->|"REJECTED / FAILED"| end_failure route_coder_result -->|"REJECTED / FAILED"| end_failure
@@ -51,14 +53,18 @@ flowchart TD
fix_loop_gate -->|"budget left"| implement fix_loop_gate -->|"budget left"| implement
fix_loop_gate -->|"budget spent"| end_failure fix_loop_gate -->|"budget spent"| end_failure
edge_case_sweep["edge_case_sweep<br/>llm"] --> route_sweep edge_case_sweep["edge_case_sweep<br/>llm"] --> route_sweep
edge_case_sweep -. "LLM fault" .-> write_handoff
route_sweep{"route_sweep<br/>script"} route_sweep{"route_sweep<br/>script"}
route_sweep -->|"5+ files or boundary"| independent_review route_sweep -->|"5+ files or boundary"| independent_review
route_sweep -->|"else"| write_handoff route_sweep -->|"else"| write_handoff
independent_review[["independent_review<br/>agent → code-reviewer"]] --> route_review independent_review[["independent_review<br/>agent → code-reviewer"]] --> route_review
independent_review -. "2 attempts failed" .-> write_handoff
route_review{"route_review<br/>script"} route_review{"route_review<br/>script"}
route_review -->|"🔴 critical findings"| implement route_review -->|"🔴 critical findings"| implement
route_review -->|"else"| write_handoff route_review -->|"else"| write_handoff
write_handoff["write_handoff<br/>llm"] --> check_handoff write_handoff["write_handoff<br/>llm"] --> check_handoff
write_handoff -. "LLM fault" .-> note_llm_fault
note_llm_fault{"note_llm_fault<br/>script"} --> end_failure
check_handoff{"check_handoff<br/>script"} check_handoff{"check_handoff<br/>script"}
check_handoff -->|"schema valid"| gate_user_review check_handoff -->|"schema valid"| gate_user_review
check_handoff -->|"one retry"| write_handoff check_handoff -->|"one retry"| write_handoff
@@ -81,8 +87,32 @@ End nodes emit sentinel outcomes for the caller:
- `STEP_BLOCKED` — `depends_on` unsatisfied and the user declined to proceed. - `STEP_BLOCKED` — `depends_on` unsatisfied and the user declined to proceed.
- `STEP_REJECTED` — user aborted at the deviation gate, or the coder's plan - `STEP_REJECTED` — user aborted at the deviation gate, or the coder's plan
was rejected at its approval gate. was rejected at its approval gate.
- `STEP_FAILED` — coder failed, the step-level fix budget was exhausted, or - `STEP_FAILED` — the coder failed or crashed, the step-level fix budget was
the handoff failed validation twice. exhausted, the handoff failed validation twice, or an orient/handoff LLM
fault was recorded (rendered as `Pipeline fault:` in the output).
## Fault handling
Node crashes degrade the run visibly instead of masquerading as success:
- `implement` has **no retries** — rerunning a coder that died mid-edit
against an already-mutated tree is unsafe. Its failure text lands in
`coder_result` and the run falls straight to `end_failure`.
- `independent_review` retries once (`max_attempts: 2`); if both attempts
fail it falls back to `write_handoff`, which flags "PIPELINE-FAULT:
independent review did not run" in the handoff rather than presenting the
failure text as findings. `route_review` carries the same guard before its
🔴 grep, so a reviewer fault never spends a fix-loop attempt.
- `edge_case_sweep` faults likewise fall through to `write_handoff` and are
flagged as "PIPELINE-FAULT: edge-case sweep did not run".
- `orient` and `write_handoff` faults route through `note_llm_fault`, which
distills the engine's failure text into a `fault_note` rendered in the
`STEP_FAILED` output — never an empty shell.
Degraded is never passing (a skipped review or sweep is recorded as NOT RUN,
never paraphrased as covered), and the sentinel is always emitted — every
path terminates at one of the four end nodes, so the caller always gets a
parseable outcome.
## Usage ## Usage