docs: sync agent READMEs + diagrams with resilience hardening
This commit is contained in:
8 files changed
+216
-30
No files matched your search
@@ -49,6 +49,12 @@ Complaints:
|
||||
2. ...
|
||||
```
|
||||
|
||||
Every verdict — CONFORMS or DIVERGES — ends with a `Verification runs:` section: the recorded
|
||||
PASS/FAIL results (command, exit code, duration, failure tail) of the caller-declared verification
|
||||
commands, or an explicit marker when none were declared or the run never reached that stage. The
|
||||
one exception is the verdict script's own crash guard, which emits a bare DIVERGES with a
|
||||
"verdict computation error" line — and no section — when verdict computation itself dies.
|
||||
|
||||
A `DIVERGES` verdict **blocks** completion. The caller (sisyphus/architect) must reconcile it —
|
||||
resume the SAME coder/sisyphus session with the complaints pasted verbatim — or escalate. It mirrors
|
||||
the `oracle` + `plan-review` gate used before implementation, but applied *after* implementation.
|
||||
@@ -60,21 +66,29 @@ violation) and cites `file:line`. Vague complaints are not emitted.
|
||||
|
||||
```mermaid
|
||||
flowchart TD
|
||||
A["parse (llm): extract acceptance criteria"] --> B["diff_facts (script): resolve + bound the diff"]
|
||||
B --> C["map over criteria: one fresh-context branch each (parallel)"]
|
||||
A["parse (llm): extract acceptance criteria + declared verification commands"] --> B["diff_facts (script): resolve + bound the diff"]
|
||||
B --> R["run_checks (script): run the declared verification commands ONCE, record results"]
|
||||
R --> C["map over criteria: one fresh-context branch each (parallel)"]
|
||||
C --> D["check_criterion (llm + fs/ast tools)"]
|
||||
D --> E["crit_gate (script): MET needs change+test evidence"]
|
||||
E -. "reject-retry once; exhausted = PARTIAL" .-> D
|
||||
E --> F["holistic (llm): ADDITIVE-only hunt - absence, scope drift, substitution, gamed tests"]
|
||||
F --> G["verdict (script): CONFORMS iff ALL MET + zero extras; no criteria = DIVERGES"]
|
||||
F --> G["verdict (script): CONFORMS iff ALL MET + zero extras + zero faults; no criteria = DIVERGES"]
|
||||
G --> H(["ADVERSARIAL_REVIEW: CONFORMS | DIVERGES"])
|
||||
A -. "fallback" .-> P["pipeline_fault (script): record PIPELINE-FAULT marker"]
|
||||
B -. "fallback" .-> P
|
||||
R -. "fallback" .-> P
|
||||
F -. "fallback" .-> G
|
||||
P --> G
|
||||
```
|
||||
|
||||
The conformance doctrine is baked into the graph's own nodes:
|
||||
|
||||
1. **Every** acceptance criterion gets its own fresh-context verification branch — MET / PARTIAL / UNMET / DIVERGED, machine-gated (MET requires the satisfying change AND the proving test, cited). No test ⇒ at best PARTIAL. Partial coverage of the criteria list is structurally impossible.
|
||||
2. Ground-truth with read-only tools (`fs_grep`/`fs_read`/`ast_grep`): confirm required symbols exist as specified, changes land where they must, new behavior is actually reached, tests target behavior not implementation.
|
||||
3. Hunt adversarially for the **absent**: skipped criteria, scope creep, interface/approach substitution, out-of-scope touches, downstream contract breakage.
|
||||
3. Execution evidence is recorded **once, deterministically**: `run_checks` runs ONLY the caller-declared verification commands (never invented or auto-detected), sequentially under a 3300s total deadline — commands the deadline leaves no budget for are recorded as SKIPPED. Criterion branches have no execution tools by design: "tests pass"-style criteria are judged against the record — a recorded green run is MET citing it; no declared command ⇒ at best PARTIAL.
|
||||
4. Hunt adversarially for the **absent**: skipped criteria, scope creep, interface/approach substitution, out-of-scope touches, downstream contract breakage.
|
||||
5. **Fail closed on pipeline faults**: if parse, diff resolution, or the verification runner dies, its fallback routes to `pipeline_fault`, which records a `PIPELINE-FAULT:` marker and continues to the verdict (a failed holistic pass falls back straight to the verdict, which synthesizes the same marker). Any recorded fault forces DIVERGES with the fault as complaint #1 — degraded is never passing, and the sentinel is always emitted: the graph never dies without one.
|
||||
|
||||
It is **read-only** — it produces a verdict, never a fix.
|
||||
|
||||
@@ -93,6 +107,9 @@ Run get_diff (or --base main), or: <paste diff>
|
||||
|
||||
## PLAN — acceptance criteria to check against
|
||||
<paste the task index.md body + the relevant PLAN-*.md section, verbatim>
|
||||
|
||||
## VERIFICATION (optional)
|
||||
<exact build/test/lint commands to run, one per line — undeclared commands are never guessed>
|
||||
"
|
||||
```
|
||||
|
||||
@@ -106,6 +123,7 @@ coyote -a adversary --agent-variable project_dir /path/to/repo \
|
||||
### Tools
|
||||
|
||||
- The graph's `diff_facts` script resolves the diff itself (staged → unstaged → `HEAD~1`, or an explicit ref/range named in the prompt) — there is no `get_diff` tool anymore.
|
||||
- The `run_checks` script executes the caller-declared verification commands and records the results — criterion branches cite the record and never run anything themselves.
|
||||
- Criterion branches carry read-only `fs_read`/`fs_cat`/`fs_grep`/`ast_grep` for ground-truth checks.
|
||||
|
||||
## Related
|
||||
|
||||
@@ -160,6 +160,9 @@ By default (`parallel_tasks: 0`) tasks run **sequentially** on the single run br
|
||||
- Architect integrates: completed task branches merge into the run branch **one at a time**, with a
|
||||
full build + test run after every merge. Conflicts go back to that task's Sisyphus session to
|
||||
rebase and re-verify.
|
||||
- Optionally, the `agent__task_*` queue may mirror the on-disk `blocked_by` graph so completing a
|
||||
task auto-dispatches its unblocked dependents. The queue is in-memory convenience only — disk task
|
||||
files remain the durable source of truth.
|
||||
- Worktrees and task branches are cleaned up after each clean merge. Phase F (single draft PR +
|
||||
CI-check watch) is unchanged in both modes.
|
||||
|
||||
|
||||
@@ -14,21 +14,44 @@
|
||||
```mermaid
|
||||
flowchart TD
|
||||
A["parse (llm)"] --> B["facts (script): diff, quality bar, ledger, history, trigger signals"]
|
||||
A -. "fallback" .-> PF["parse_fault (script): PIPELINE-FAULT verdict"]
|
||||
PF --> L
|
||||
B --> C["linters (script): tflint / hadolint / actionlint / buf breaking"]
|
||||
C --> D["group_domains (script) --> refine_groups (llm) --> cover_gate (script: exact cover)"]
|
||||
D --> E["map over domain slices (parallel): domain-reviewer leaders, teammates on"]
|
||||
D --> F["aux_lanes (llm): org-context + prior-art spawns"]
|
||||
D --> G["downstream_sweep (script)"]
|
||||
C --> D1["group_propose (script): deterministic domain proposal"]
|
||||
D1 --> D2["refine_groups (llm)"]
|
||||
D2 --> D3["cover_gate (script: exact cover)"]
|
||||
D2 -. "fallback" .-> D3
|
||||
D3 --> E["map over domain slices (parallel): domain-reviewer leaders, teammates on, max_attempts: 2"]
|
||||
D3 --> F["aux_lanes (llm): org-context + prior-art spawns"]
|
||||
D3 --> G["downstream_sweep (script)"]
|
||||
E --> H["completeness_gate (script): required sections even when clean"]
|
||||
E -. "fallback" .-> DF["domain_fault (script): ⚠️ PIPELINE-FAULT slice banner"]
|
||||
H -. "reject-retry once" .-> E
|
||||
H --> I["synthesize (llm: dedup + prose, NO verdict authority)"]
|
||||
F --> I
|
||||
F -. "fallback" .-> AF["aux_fault (script): PIPELINE-FAULT lane-skip note"]
|
||||
AF --> I
|
||||
G --> I
|
||||
I --> J["verify: finding-verifier (agent node, unskippable)"]
|
||||
J --> K["verdict (script): FALSE-drops, rigor folding, MERGE-READY / NEEDS-HUMAN arithmetic"]
|
||||
I --> J["verify: finding-verifier (agent node, unskippable, max_attempts: 2)"]
|
||||
I -. "fallback" .-> K
|
||||
J --> K["verdict (script): FALSE-drops, rigor folding, fault accounting, MERGE-READY / NEEDS-HUMAN arithmetic"]
|
||||
J -. "fallback" .-> K
|
||||
K --> L["render (script): the standard report"]
|
||||
```
|
||||
|
||||
**Failure fails closed.** Every LLM/agent node retries once (`max_attempts: 2` — the
|
||||
domain-reviewer lanes and the verifier included), then routes to a `fallback:` instead of killing
|
||||
the graph, and every fallback that loses review coverage leaves a `PIPELINE-FAULT` marker: a dead
|
||||
`parse` emits a complete fault verdict and jumps straight to render; a dead domain lane becomes a
|
||||
`> ⚠️ PIPELINE-FAULT` banner where its slice report would be; a dead `refine_groups` falls
|
||||
through to the cover gate, which keeps the deterministic proposal; a dead synthesis or verifier
|
||||
falls through to the verdict script. The verdict script counts every fault and forces
|
||||
`NEEDS-HUMAN` — a degraded run never reads `MERGE-READY`. The aux context lanes are the one
|
||||
exception: they are enrichment, so their fault (`aux_fault`) is surfaced as a lane-skip note but
|
||||
never blocks. Fault detection is prefix-anchored (a report that merely *quotes* a marker doesn't
|
||||
trip it), and the report is always emitted: the verdict script's own crash guard emits a
|
||||
`PIPELINE-FAULT` NEEDS-HUMAN, and render bypasses its "No changes to review." stub for fault
|
||||
verdicts — a fault renders as a full report, never silence.
|
||||
|
||||
A CodeRabbit-style code review orchestrator that coordinates per-file reviews and synthesizes findings into a unified
|
||||
report.
|
||||
|
||||
@@ -22,11 +22,13 @@ agent, this is the file to read alongside the
|
||||
|
||||
## Workflow
|
||||
|
||||
17 nodes. Solid arrows are static `next` / `routes` edges declared in
|
||||
23 nodes. Solid arrows are static `next` / `routes` edges declared in
|
||||
`graph.yaml`; script nodes can also route dynamically via `_next` (shown as
|
||||
labeled branches out of the diamond). Dotted arrows show `map` fan-out — the
|
||||
`research_each_question` node spawns one `research_one_question` branch per
|
||||
sub-question and joins them before continuing.
|
||||
sub-question and joins them before continuing — and `fallback` edges
|
||||
(labeled "fallback"), taken when a node fails outright (see *Fault paths*
|
||||
below).
|
||||
|
||||
```mermaid
|
||||
flowchart TD
|
||||
@@ -38,18 +40,27 @@ flowchart TD
|
||||
bootstrap_research --> plan
|
||||
bootstrap_research --> knowledge_lookup
|
||||
plan["plan<br/>llm + output_schema"] --> research_each_question
|
||||
plan -. "fallback" .-> plan_fault{"plan_fault<br/>script"}
|
||||
plan_fault --> end_fault
|
||||
knowledge_lookup[("knowledge_lookup<br/>rag")] --> research_each_question
|
||||
research_each_question[\research_each_question<br/>map/]
|
||||
research_each_question -. "spawns × N" .-> research_one_question["research_one_question<br/>llm + web tools"]
|
||||
research_one_question -. "fallback" .-> question_fault{"question_fault<br/>script"}
|
||||
research_each_question --> combine_findings
|
||||
combine_findings{"combine_findings<br/>script"} --> vet_sources
|
||||
vet_sources["vet_sources<br/>llm + classify_source"] --> critique
|
||||
vet_sources -. "fallback" .-> vet_fault{"vet_fault<br/>script"}
|
||||
vet_fault --> critique
|
||||
critique["critique<br/>llm"] --> reflexion_gate
|
||||
critique -. "fallback" .-> critique_fault{"critique_fault<br/>script"}
|
||||
critique_fault --> reflexion_gate
|
||||
reflexion_gate{"reflexion_gate<br/>script"}
|
||||
reflexion_gate -->|"PASS"| synthesize
|
||||
reflexion_gate -->|"REVISE (budget left)"| research_each_question
|
||||
reflexion_gate -->|"REVISE (budget spent)"| synthesize
|
||||
synthesize[["synthesize<br/>agent → report-writer"]] --> verify_sources
|
||||
synthesize -. "fallback" .-> synth_fault{"synth_fault<br/>script"}
|
||||
synth_fault --> end_fault
|
||||
verify_sources{"verify_sources<br/>script"} --> approve
|
||||
approve{{"approve<br/>approval"}}
|
||||
approve -->|"accept"| end_accepted
|
||||
@@ -59,13 +70,14 @@ flowchart TD
|
||||
|
||||
end_accepted(["end_accepted<br/>report"])
|
||||
end_rejected(["end_rejected"])
|
||||
end_fault(["end_fault<br/>DEEP_RESEARCH FAILED"])
|
||||
```
|
||||
|
||||
### Node-type breakdown
|
||||
|
||||
| Type | Nodes |
|
||||
|-----------------------------|-----------------------------------------------------------------------------------------------------------------------|
|
||||
| `script` (Python) | `parse_request`, `bootstrap_research`, `combine_findings`, `reflexion_gate`, `verify_sources`, `incorporate_feedback` |
|
||||
| `script` (Python) | `parse_request`, `bootstrap_research`, `combine_findings`, `reflexion_gate`, `verify_sources`, `incorporate_feedback`, plus the fault markers `plan_fault`, `question_fault`, `vet_fault`, `critique_fault`, `synth_fault` |
|
||||
| `llm` (tools: `[]`) | `plan`, `critique` |
|
||||
| `llm` (with tool whitelist) | `research_one_question`, `vet_sources` |
|
||||
| `rag` | `knowledge_lookup` — local corpus retrieval |
|
||||
@@ -73,7 +85,7 @@ flowchart TD
|
||||
| `agent` | `synthesize` — spawns the `report-writer` sub-agent |
|
||||
| `input` | `ask_topic` |
|
||||
| `approval` | `approve` |
|
||||
| `end` | `end_accepted`, `end_rejected` |
|
||||
| `end` | `end_accepted`, `end_rejected`, `end_fault` |
|
||||
|
||||
## Parallel execution
|
||||
|
||||
@@ -217,6 +229,46 @@ folds that text into `research_feedback` and loops back to
|
||||
`settings.max_loop_iterations` (40) is the engine's infinite-loop
|
||||
backstop: it caps the total visits to any single node.
|
||||
|
||||
## Fault paths
|
||||
|
||||
Every stage that can die outright has a `fallback:` edge to a small
|
||||
fault-marker script — the dashed edges in the diagram. A fallback fires
|
||||
after a node's retries are exhausted (`research_one_question` and
|
||||
`synthesize` each get two attempts via `max_attempts: 2`); the engine
|
||||
writes its failure text into the node's state key before routing, and
|
||||
the marker normalizes that text into a `PIPELINE-FAULT: ...` entry in
|
||||
`pipeline_faults`. The five routes:
|
||||
|
||||
- `plan -> plan_fault -> end_fault` — fail-closed: no plan means
|
||||
nothing to research.
|
||||
- `research_one_question -> question_fault` — branch-local, inside the
|
||||
map branch: one dead research lane becomes a fault finding instead of
|
||||
sinking the whole map.
|
||||
- `vet_sources -> vet_fault -> critique` — degrade-visibly: vetting is
|
||||
a quality lane, not the product; `source_assessment` becomes a
|
||||
neutral "unvetted" note.
|
||||
- `critique -> critique_fault -> reflexion_gate` — degrade-visibly: the
|
||||
rewritten critique deliberately has no `VERDICT:` line, so the gate's
|
||||
malformed-critique PASS default proceeds to synthesis.
|
||||
- `synthesize -> synth_fault -> end_fault` — fail-closed: no report
|
||||
means no product.
|
||||
|
||||
On the fail-closed routes, `end_fault` renders the
|
||||
`DEEP_RESEARCH FAILED — PIPELINE-FAULT: ...` sentinel. On the
|
||||
degrade-visibly routes, `verify_sources` folds the accumulated faults
|
||||
into a `## Pipeline notes` section shown at the approval step and
|
||||
appended to the accepted report (`{{report}}{{pipeline_notes}}`;
|
||||
empty when there were no faults). Degraded output is never a silent
|
||||
pass — every fault is visible to the reviewer or the caller — and some
|
||||
end node always renders its sentinel, so the graph degrades instead of
|
||||
dying.
|
||||
|
||||
All 11 scripts also carry a top-level crash guard (R3): a script that
|
||||
throws records a fault and fails forward where the flow allows it (a
|
||||
crashed `reflexion_gate` proceeds to `synthesize`; a crashed
|
||||
`verify_sources` reports that sources were NOT checked) instead of
|
||||
killing the run.
|
||||
|
||||
## Running
|
||||
|
||||
```sh
|
||||
@@ -241,6 +293,9 @@ coyote -a deep-research # no prompt -> triggers ask_topic
|
||||
- `verify_sources` probes every cited URL / DOI with an HTTP HEAD
|
||||
request and reports which are unreachable, so the human reviewer
|
||||
sees broken citations before approving.
|
||||
- Fault paths degrade visibly, never silently: a `PIPELINE-FAULT`
|
||||
can only ever surface as a failure sentinel or a `## Pipeline
|
||||
notes` section, never as a clean report.
|
||||
|
||||
## Customizing
|
||||
|
||||
@@ -270,7 +325,7 @@ coyote -a deep-research # no prompt -> triggers ask_topic
|
||||
|
||||
```
|
||||
assets/agents/deep-research/
|
||||
graph.yaml - agent config + 17-node workflow
|
||||
graph.yaml - agent config + 23-node workflow
|
||||
tools.sh - classify_source custom tool
|
||||
README.md - this file
|
||||
knowledge/
|
||||
@@ -279,9 +334,15 @@ assets/agents/deep-research/
|
||||
scripts/
|
||||
parse_request.py - _next: bootstrap_research, or ask_topic if no topic
|
||||
bootstrap_research.py - fan-out source: next [plan, knowledge_lookup]
|
||||
plan_fault.py - fail-closed fault marker: dead plan -> end_fault
|
||||
question_fault.py - branch-local fault marker for a dead research lane
|
||||
combine_findings.py - joins map output (question_findings) into findings
|
||||
vet_fault.py - degrade-visibly fault marker: unvetted sources
|
||||
critique_fault.py - degrade-visibly fault marker: skipped critique
|
||||
reflexion_gate.py - _next: research_each_question (revise) or synthesize
|
||||
verify_sources.py - HTTP HEAD on cited URLs / DOIs
|
||||
synth_fault.py - fail-closed fault marker: no report -> end_fault
|
||||
verify_sources.py - HTTP HEAD on cited URLs / DOIs; folds pipeline_faults
|
||||
into the report's "## Pipeline notes" section
|
||||
incorporate_feedback.py - _next: research_each_question, with user feedback
|
||||
```
|
||||
|
||||
|
||||
@@ -7,13 +7,23 @@ or confirmation-bias the check. A graph can't: the structure IS the discipline.
|
||||
|
||||
## How it works
|
||||
|
||||
```
|
||||
parse (llm) ──▶ verify_each (map, max_concurrency 4) ──▶ done (end)
|
||||
└─▶ verify_one (llm + fs tools) ──▶ verdict_gate (script), one CHAIN per finding
|
||||
```mermaid
|
||||
flowchart TD
|
||||
A["parse (llm, max_attempts 2)"] --> B["verify_each (map, max_concurrency 4): one branch per finding"]
|
||||
A -. "fallback" .-> PF["parse_fault (script): verdicts = one UNVERIFIABLE PIPELINE-FAULT entry"]
|
||||
PF --> D
|
||||
B -. "spawns × N" .-> C["verify_one (llm + fs tools, max_attempts 2)"]
|
||||
C --> G["verdict_gate (script): machine-validates the JSON contract"]
|
||||
G -. "reject-retry once" .-> C
|
||||
C -. "fallback" .-> VF["verify_fault (script): branch-local UNVERIFIABLE marker"]
|
||||
B --> D(["done (end): FINDING_VERIFIER_RESULTS"])
|
||||
```
|
||||
|
||||
1. **parse** — extracts every 🔴/🟡 finding from the spawn prompt into a structured list
|
||||
(id, severity verbatim, path, lines, self-contained claim). Extraction only — no judging.
|
||||
Gets two attempts (`max_attempts: 2`); if both fail, its `fallback` routes to
|
||||
`parse_fault`, which rewrites `verdicts` to a single UNVERIFIABLE entry naming the fault
|
||||
and jumps straight to `done`.
|
||||
2. **verify_each** — a `map` node fans out one `verify_one` branch per finding. Skipping a
|
||||
finding is structurally impossible; results collect in input order.
|
||||
3. **verify_one** — reads the cited code (greps for moved code before concluding anything)
|
||||
@@ -28,12 +38,19 @@ parse (llm) ──▶ verify_each (map, max_concurrency 4) ──▶ done (end)
|
||||
contract (verdict enum; evidence required for VERIFIED/FALSE; note for UNVERIFIABLE),
|
||||
stamps the authoritative id FROM the finding item (the model's echo is never trusted),
|
||||
rejects back to `verify_one` exactly once with the reason, and on a second failure records
|
||||
UNVERIFIABLE — a malformed branch never sinks the map.
|
||||
Kept, marked unverified.
|
||||
UNVERIFIABLE — a malformed branch never sinks the map. If `verify_one` itself dies (both
|
||||
`max_attempts: 2` attempts fail), its `fallback` routes to `verify_fault`, which
|
||||
normalizes the engine's failure text into a branch-local UNVERIFIABLE verdict the map
|
||||
still collects.
|
||||
4. **done** — a deterministic `end` node emits the raw verdict list plus the caller
|
||||
contract (VERIFIED = keep + paste evidence; FALSE = drop + tally; UNVERIFIABLE = keep,
|
||||
marked). No final LLM step that could editorialize.
|
||||
|
||||
The dashed edges are fault paths. Both fault markers surface as `PIPELINE-FAULT:` text in
|
||||
a verdict's `note`, and a fault can only ever produce UNVERIFIABLE — degraded output is
|
||||
never a passing verdict. Every path ends at `done`, so the `FINDING_VERIFIER_RESULTS`
|
||||
sentinel is always emitted and the caller's verify lane degrades instead of dying.
|
||||
|
||||
## What it is NOT
|
||||
|
||||
- **Not a second opinion.** It never re-judges severity, style, or whether a finding
|
||||
@@ -50,3 +67,6 @@ installed, code-reviewer runs the same contract inline as a fallback.
|
||||
|
||||
- `graph.yaml` — the agent (no tools.sh; the graph grants `fs_read`/`fs_cat`/`fs_grep`
|
||||
per node).
|
||||
- `scripts/verdict_gate.py` — the deterministic contract gate.
|
||||
- `scripts/parse_fault.py`, `scripts/verify_fault.py` — the fault markers behind the
|
||||
dashed fallback edges.
|
||||
@@ -11,13 +11,21 @@ gets the chance to rationalize past it.
|
||||
```mermaid
|
||||
flowchart TD
|
||||
parse["parse<br/>(llm: extract request facts)"] --> signals["signals<br/>(script: deterministic diff facts)"]
|
||||
parse -. "fallback" .-> pfault["parse_fault<br/>(script: PIPELINE-FAULT marker)"]
|
||||
pfault --> gate
|
||||
signals --> select["select_lanes<br/>(llm: ADDITIVE-only judgment)"]
|
||||
select --> build["build_items<br/>(script: rules floor ∪ additions;<br/>forced lanes override exactly)"]
|
||||
select -. "fallback" .-> dlanes["default_lanes<br/>(script: deterministic degraded<br/>selection — wider, never narrower)"]
|
||||
dlanes --> build
|
||||
build --> mcr["map: code-review"] & madv["map: adversary"] & msec["map: security"] & mpb["map: probe"]
|
||||
mcr -->|"over: 0-or-1 items"| rcr[["code-reviewer"]]
|
||||
madv --> radv[["adversary"]]
|
||||
msec --> rsec[["security-reviewer"]]
|
||||
mpb --> rpb[["probe"]]
|
||||
mcr -->|"over: 0-or-1 items"| rcr[["code-reviewer<br/>(max_attempts: 2)"]]
|
||||
madv --> radv[["adversary<br/>(max_attempts: 2)"]]
|
||||
msec --> rsec[["security-reviewer<br/>(max_attempts: 2)"]]
|
||||
mpb --> rpb[["probe<br/>(max_attempts: 2)"]]
|
||||
rcr -. "fallback" .-> lfault["lane_fault<br/>(script: PIPELINE-FAULT lane result,<br/>collected by the map)"]
|
||||
radv -. "fallback" .-> lfault
|
||||
rsec -. "fallback" .-> lfault
|
||||
rpb -. "fallback" .-> lfault
|
||||
mcr & madv & msec & mpb --> gate["verdict_gate<br/>(script: sentinel regex — deterministic)"]
|
||||
gate --> done(["GAUNTLET: PASS | BLOCKED"])
|
||||
```
|
||||
@@ -49,6 +57,21 @@ those rules are *structure*:
|
||||
with an environment note — it is never treated as PASS or FAIL.
|
||||
- **Verdict independence is structural**: lanes run as isolated sub-agents
|
||||
with no `teammates:` flag — no cross-lane messaging, by construction.
|
||||
- **Failure fails closed, structurally** (`fallback:` routes + `PIPELINE-FAULT`
|
||||
markers): every LLM node and lane retries once (`max_attempts: 2`), then a
|
||||
fallback fires instead of killing the graph — and every fallback degrades
|
||||
toward BLOCKED, never toward a pass. A dead lane becomes a `PIPELINE-FAULT:`
|
||||
lane result (`lane_fault.py`) that the gate reports as BLOCKED *naming the
|
||||
lane* — distinct from SKIPPED (not selected). A dead `parse` records a fault
|
||||
and jumps straight to the gate (`parse_fault.py`). A dead `select_lanes`
|
||||
degrades to a deterministic selection (`default_lanes.py`): code-review +
|
||||
adversary, plus probe on consumer surface, plus security on auth/deps/exec
|
||||
signals or hardened posture, unioned with caller-forced lanes — wider, never
|
||||
narrower. A crashed `build_items` records a fault so an all-SKIPPED gate
|
||||
still blocks. Fault detection is prefix-anchored — a real review that merely
|
||||
*quotes* "PIPELINE-FAULT:" flows to the normal sentinel rules. And the
|
||||
`GAUNTLET:` sentinel is always emitted: even a crashed gate prints
|
||||
`GAUNTLET: BLOCKED`, never silence.
|
||||
|
||||
## Sentinels parsed
|
||||
|
||||
@@ -74,6 +97,9 @@ Plan / acceptance criteria:
|
||||
Local-run recipe / usage suites:
|
||||
<how to boot the service locally + where existing suites live — enables the probe lane>
|
||||
|
||||
Verification commands: (optional — exact build/test/lint commands, passed
|
||||
verbatim to the adversary lane; omitted = none declared, never invented)
|
||||
|
||||
Forced lanes: (optional — e.g. 'adversary, probe' to run exactly those)"
|
||||
```
|
||||
|
||||
@@ -89,9 +115,13 @@ final report verbatim.
|
||||
- The caller decides *whether* review is warranted at all (the gauntlet is for
|
||||
non-trivial work — that's why code-review is unconditionally on); the
|
||||
gauntlet decides *which* of the other lanes apply and what the verdicts mean.
|
||||
- Lane sub-agents get generous finite timeouts (2h each; 3h whole-gauntlet).
|
||||
A lane that dies or times out produces no sentinel — which is a BLOCKED,
|
||||
never a silent pass.
|
||||
- Lane sub-agents get generous finite timeouts (2h each; 3h whole-gauntlet)
|
||||
and one retry (`max_attempts: 2`). A lane that still dies or times out falls
|
||||
back to the `lane_fault` marker — BLOCKED naming the lane, never a silent
|
||||
pass.
|
||||
- Caller-declared verification commands are extracted verbatim by `parse` and
|
||||
passed through to the adversary lane's prompt — never invented or
|
||||
auto-detected; undeclared stays undeclared.
|
||||
- Signals failures degrade gracefully: if git can't compute the diff, lane
|
||||
selection falls back to caller context + forced lanes, and the report notes
|
||||
it. The auth-path regex deliberately over-fires (`auth(?!or)` matches
|
||||
|
||||
@@ -57,6 +57,7 @@ Spawnable sub-agents (from `config.yaml`):
|
||||
- 💻 **CLI Coding**: Provides a natural language interface for writing and editing code.
|
||||
- 🔄 **Task Management**: Tracks progress and context across complex operations.
|
||||
- 🛠️ **Tool Integration**: Seamlessly uses system tools for building, testing, and file manipulation.
|
||||
- 🧵 **Task-queue bursts**: Dependency-shaped fan-outs (explore → code → verify) may be encoded as `agent__task_create` chains — dependents auto-dispatch when `agent__task_complete` unblocks them; todos remain the tracking source of truth.
|
||||
- 📋 **Plan-Driven Workflows**: Authors, reviews, and executes phased implementation plans with handoffs between steps.
|
||||
- 🚦 **Escalation policy**: `escalation_policy=autonomous` pre-delegates decision-making — standing rulings (`~/.config/coyote/share/standing-rulings.md`, recorded via `coyote --macro record-ruling`) and evidence-backed recommended options are taken without pausing and batched into a “Decisions taken autonomously” digest; secrets, money movement, and destructive actions always escalate.
|
||||
|
||||
|
||||
@@ -27,6 +27,7 @@ flowchart TD
|
||||
gate_blocked -->|"yes"| orient
|
||||
gate_blocked -->|"no"| end_blocked
|
||||
orient["orient<br/>llm, read-only"] --> route_staleness
|
||||
orient -. "LLM fault" .-> note_llm_fault
|
||||
route_staleness{"route_staleness<br/>script"}
|
||||
route_staleness -->|"major deviation"| gate_deviation
|
||||
route_staleness -->|"else"| implement
|
||||
@@ -35,6 +36,7 @@ flowchart TD
|
||||
gate_deviation -->|"abort"| end_rejected
|
||||
gate_deviation -->|"other (user guidance)"| implement
|
||||
implement[["implement<br/>agent → coder"]] --> route_coder_result
|
||||
implement -. "coder crashed<br/>(no retries)" .-> end_failure
|
||||
route_coder_result{"route_coder_result<br/>script"}
|
||||
route_coder_result -->|"CODER_COMPLETE"| verify_format_lint
|
||||
route_coder_result -->|"REJECTED / FAILED"| end_failure
|
||||
@@ -51,14 +53,18 @@ flowchart TD
|
||||
fix_loop_gate -->|"budget left"| implement
|
||||
fix_loop_gate -->|"budget spent"| end_failure
|
||||
edge_case_sweep["edge_case_sweep<br/>llm"] --> route_sweep
|
||||
edge_case_sweep -. "LLM fault" .-> write_handoff
|
||||
route_sweep{"route_sweep<br/>script"}
|
||||
route_sweep -->|"5+ files or boundary"| independent_review
|
||||
route_sweep -->|"else"| write_handoff
|
||||
independent_review[["independent_review<br/>agent → code-reviewer"]] --> route_review
|
||||
independent_review -. "2 attempts failed" .-> write_handoff
|
||||
route_review{"route_review<br/>script"}
|
||||
route_review -->|"🔴 critical findings"| implement
|
||||
route_review -->|"else"| write_handoff
|
||||
write_handoff["write_handoff<br/>llm"] --> check_handoff
|
||||
write_handoff -. "LLM fault" .-> note_llm_fault
|
||||
note_llm_fault{"note_llm_fault<br/>script"} --> end_failure
|
||||
check_handoff{"check_handoff<br/>script"}
|
||||
check_handoff -->|"schema valid"| gate_user_review
|
||||
check_handoff -->|"one retry"| write_handoff
|
||||
@@ -81,8 +87,32 @@ End nodes emit sentinel outcomes for the caller:
|
||||
- `STEP_BLOCKED` — `depends_on` unsatisfied and the user declined to proceed.
|
||||
- `STEP_REJECTED` — user aborted at the deviation gate, or the coder's plan
|
||||
was rejected at its approval gate.
|
||||
- `STEP_FAILED` — coder failed, the step-level fix budget was exhausted, or
|
||||
the handoff failed validation twice.
|
||||
- `STEP_FAILED` — the coder failed or crashed, the step-level fix budget was
|
||||
exhausted, the handoff failed validation twice, or an orient/handoff LLM
|
||||
fault was recorded (rendered as `Pipeline fault:` in the output).
|
||||
|
||||
## Fault handling
|
||||
|
||||
Node crashes degrade the run visibly instead of masquerading as success:
|
||||
|
||||
- `implement` has **no retries** — rerunning a coder that died mid-edit
|
||||
against an already-mutated tree is unsafe. Its failure text lands in
|
||||
`coder_result` and the run falls straight to `end_failure`.
|
||||
- `independent_review` retries once (`max_attempts: 2`); if both attempts
|
||||
fail it falls back to `write_handoff`, which flags "PIPELINE-FAULT:
|
||||
independent review did not run" in the handoff rather than presenting the
|
||||
failure text as findings. `route_review` carries the same guard before its
|
||||
🔴 grep, so a reviewer fault never spends a fix-loop attempt.
|
||||
- `edge_case_sweep` faults likewise fall through to `write_handoff` and are
|
||||
flagged as "PIPELINE-FAULT: edge-case sweep did not run".
|
||||
- `orient` and `write_handoff` faults route through `note_llm_fault`, which
|
||||
distills the engine's failure text into a `fault_note` rendered in the
|
||||
`STEP_FAILED` output — never an empty shell.
|
||||
|
||||
Degraded is never passing (a skipped review or sweep is recorded as NOT RUN,
|
||||
never paraphrased as covered), and the sentinel is always emitted — every
|
||||
path terminates at one of the four end nodes, so the caller always gets a
|
||||
parseable outcome.
|
||||
|
||||
## Usage
|
||||
|
||||
|
||||
Reference in new issue
Block a user