Merge pull request #28 from Dark-Alex-17/feat/PLAN-review-retry

PLAN-review-retry: self-healing review gate (fault-only lane retries) + untimed approvals
This commit is contained in:
Alex Clarke authored and GitHub committed 2026-09-14 11:46:15 -06:00
commit 9a6ffcb8ca
20 files changed
+2814 -99

No files matched your search

+1 -1
View File
@@ -3,7 +3,7 @@ name: CI
on:
pull_request:
branches:
- '*'
- '**'
push:
branches:
- main
+1 -1
View File
@@ -177,7 +177,7 @@ By default (`parallel_tasks: 0`) tasks run **sequentially** on the single run br
| [`explore`](../explore/README.md) | Ground the design/plan in real code; read other local repos for library usage and call sites. |
| [`librarian`](../librarian/README.md) | External docs / OSS examples for unfamiliar libraries. |
| [`whetstone`](../whetstone/README.md) | Continuous improvement — spawned once before the final report when the run addressed PR review feedback from outside its own lanes; converts review misses into user-approved suite edits / ledger entries. Never blocks the run. |
| [`review-gauntlet`](../review-gauntlet/README.md) | Preferred per-task conformance gate — one spawn running adversary (+ probe for consumer-facing tasks) with a machine-parsed GAUNTLET: PASS/BLOCKED verdict; direct adversary/probe spawns remain the fallback. |
| [`review-gauntlet`](../review-gauntlet/README.md) | Preferred per-task conformance gate — one spawn running adversary (+ probe for consumer-facing tasks) with a machine-parsed GAUNTLET: PASS/BLOCKED verdict; a `GAUNTLET_REVIEW_INCOMPLETE:` line is an infrastructure fault escalated to the user (retry / accept NEEDS-HUMAN / abort), never a divergence; direct adversary/probe spawns remain the fallback. |
## Related skills
+22 -3
View File
@@ -341,9 +341,28 @@ instructions: |
prompt also carries pasted plan text from the repo under review, and whatever is declared
here is executed with a shell — a trust boundary the prose must not cross.
It returns `GAUNTLET: PASS/BLOCKED`. On BLOCKED apply the same fix-and-rerun rules as below
(resume the SAME Sisyphus session with the findings verbatim, then a FRESH gauntlet run).
The direct `adversary`/probe spawns below are the FALLBACK when the gauntlet is not installed.
It returns `GAUNTLET: PASS/BLOCKED`. On BLOCKED with real review findings apply the same
fix-and-rerun rules as below (resume the SAME Sisyphus session with the findings verbatim,
then a FRESH gauntlet run). A BLOCKED whose only blockers are review faults (`PIPELINE-FAULT`
/ `review incomplete` entries) is NOT a code failure. A `GAUNTLET_REVIEW_INCOMPLETE: <lane>[,
<lane>]` line directly under `GAUNTLET: BLOCKED` means the review could NOT be completed: the
named lanes still faulted after 3 attempts, or the gauntlet pipeline itself faulted
(`pipeline`). This is an infrastructure fault, not a code finding.
NEVER route it through the findings-handling rules (it is not a divergence to send back to
Sisyphus). NEVER claim done while it stands.
Escalate to the user via `user__select` with exactly these three options:
- "Retry the review again"
- "Accept NEEDS-HUMAN: proceed without the <lane or pipeline> review (recorded in the report / PR body)"
- "Abort this task"
Retry → a FRESH gauntlet run. Accept → close the task and record the skipped review (lane
names or `pipeline`) in the task's `log.md`, the Phase F digest, and the PR body. Abort →
leave the task `in-progress`, log the abort in its `log.md`, and stop to report. Findings AND
incomplete on the same output → fix the findings first, then re-run the gauntlet (the re-run
retries the lane) — escalate only if the line persists. Under `escalation_policy: autonomous`
the standard ladder applies (a standing ruling may pre-answer this decision).
The direct `adversary`/probe spawns below are the FALLBACK when the gauntlet is not installed;
on that path an adversary/probe result that is ONLY a `PIPELINE-FAULT` (no verdict on the
code) is likewise an infrastructure fault to retry, never a divergence.
- **Spawn `adversary`** with the diff base and the criteria pasted in:
```
+79 -14
View File
@@ -26,8 +26,10 @@ flowchart TD
radv -. "fallback" .-> lfault
rsec -. "fallback" .-> lfault
rpb -. "fallback" .-> lfault
mcr & madv & msec & mpb --> gate["verdict_gate<br/>(script: sentinel regex — deterministic)"]
gate --> done(["GAUNTLET: PASS | BLOCKED"])
mcr & madv & msec & mpb --> rgate["retry_gate<br/>(script: fault-only re-run gate;<br/>stash + restore, ≤3 attempts,<br/>wall-clock budget)"]
rgate -. "_next: retry faulted lanes" .-> build
rgate --> gate["verdict_gate<br/>(script: sentinel regex — deterministic)"]
gate --> done(["GAUNTLET: PASS | BLOCKED<br/>(+ GAUNTLET_REVIEW_INCOMPLETE: … when any lane or the pipeline was incomplete)"])
```
## Why a graph
@@ -49,12 +51,17 @@ those rules are *structure*:
- **Conditional lanes without dynamic routing**: each lane is a `map` node
over a 0-or-1-item list — an empty list means the map runs zero branches
(map-over-nothing), which *is* the skip mechanism. All four maps join at
the verdict gate on equal-length paths.
the retry gate on equal-length paths; it falls through to the verdict
gate, or routes back to `build_items` for faulted lanes only.
- **The verdict is a regex, not an opinion** (`verdict_gate.py`): a missing
sentinel is a lane FAILURE, never a pass; the critical count from the
code-review report's summary line (raw 🔴 count if absent) blocks
regardless of its verdict line; a code-review lane whose own verdict line
records a pipeline fault blocks (a degraded review is never a pass);
records a pipeline fault blocks (a degraded review is never a pass); the
adversary's own degraded-run header (the `Criteria: …` line directly under
its sentinel recording a pipeline fault, died criterion checks, or its
verdict script's crash stub) blocks as a pipeline fault,
not as a DIVERGES finding;
`NEEDS-HUMAN` without 🔴 passes but
is surfaced under *Human attention required*; probe `INCONCLUSIVE` blocks
with an environment note — it is never treated as PASS or FAIL.
@@ -77,12 +84,59 @@ those rules are *structure*:
`GAUNTLET:` sentinel is always emitted: even a crashed gate prints
`GAUNTLET: BLOCKED`, never silence.
## Fault-only re-runs
`retry_gate.py` sits between the lane maps and the verdict gate and re-runs
only lanes that **faulted** — a missing report (stood in for by a synthetic
`PIPELINE-FAULT: <lane> lane produced no report` marker so it never renders as
SKIPPED), a non-string report, the `lane_fault` marker, a sentinel-less
report, or a nested graph's own degraded-run wording (the code-review verdict
line / the adversary's header line directly under its sentinel). A real verdict
(DIVERGES, FAIL, INCONCLUSIVE, NEEDS-HUMAN, …) is never re-run: the review's
judgment stands. Each lane gets up to 3 attempts, all inside a wall-clock
budget (`MAX_RETRY_ELAPSED_SECS`) sized to fit the graph timeout.
- **Stash + restore**: every lane report that ran this pass is stashed in
`kept_results`; a re-run pass overwrites the untouched lanes' collected
results with `[]`, so their stashed reports are restored before the gate.
The verdict gate always sees each lane's *last* report.
- **Retry never narrows selection**: lane selection is not re-run. A re-run
pass rebuilds items only for the retried lanes, and the stashed results
stand in for the rest.
- **Exhaustion is recorded, not hidden**: a lane still faulted after its
attempts or the budget lands in `review_incomplete`, and its fault flows to
the verdict gate as-is.
- **Crash guards append**: the `build_items.py` and `retry_gate.py` crash
guards append to an existing `signals_error` rather than overwrite it, and
`build_items.py` preserves the prior `lanes_summary`.
A non-empty `GAUNTLET_REVIEW_INCOMPLETE:` line structurally implies
`GAUNTLET: BLOCKED`: the verdict gate forces every lane named in
`review_incomplete` to a BLOCKED row (`review incomplete after N attempt(s)`)
and a matching blocker, so a restored earlier report can never turn an
incomplete review into a pass — and malformed `review_incomplete` bookkeeping
is itself recorded as a `pipeline` fault, never dropped.
Whenever at least one lane (or the pipeline) could not complete, the first
line reads `GAUNTLET: BLOCKED` followed immediately by
`GAUNTLET_REVIEW_INCOMPLETE: <lane>[, <lane>]`. The names are the exhausted
lanes from `review_incomplete`, plus the `pipeline` pseudo-lane whenever a
pipeline-level `PIPELINE-FAULT:` fired — parse death, or a crash in the
builder, the retry gate, the default-lanes fallback, or the verdict gate
itself (the default-lanes fallback's crash still runs the widened lanes, so
`pipeline` appears alongside real lane rows). The report gains a
`## Review incomplete` section naming each exhausted lane with its
attempt count; `PIPELINE-FAULT` lane rows carry ` ×N` when the lane was
re-run (N > 1). The *named* entries are faults to re-run, never code
findings; the line can accompany real findings from the other lanes, which
still appear under `## Blockers` and must be fixed as usual.
## Sentinels parsed
| Lane | Sentinel | Blocks on |
|------|----------|-----------|
| code-reviewer | `**Verdict: MERGE-READY \| NEEDS-HUMAN**` | critical count from the report's summary line (raw 🔴 count if absent); missing sentinel; own verdict line records a pipeline fault |
| adversary | `ADVERSARIAL_REVIEW: CONFORMS \| DIVERGES` | DIVERGES; missing sentinel |
| adversary | `ADVERSARIAL_REVIEW: CONFORMS \| DIVERGES` | DIVERGES; own degraded-run header (pipeline fault, died criterion checks, or its verdict script's crash stub); missing sentinel |
| security-reviewer | `SECURITY_REVIEW: PASS \| FAIL` | FAIL; missing sentinel |
| probe | `USAGE_PROBE: PASS \| FAIL \| INCONCLUSIVE` | FAIL; INCONCLUSIVE (environment note); missing sentinel |
@@ -115,22 +169,33 @@ coyote -a review-gauntlet --agent-variable project_dir /abs/path/to/repo \
--agent-variable verification_commands '["cargo test --all"]' "<prompt>"
```
The reply ends with `GAUNTLET: PASS` or `GAUNTLET: BLOCKED`, a lane status
The reply opens with `GAUNTLET: PASS` or `GAUNTLET: BLOCKED` (plus the
`GAUNTLET_REVIEW_INCOMPLETE:` line when applicable), followed by a lane status
table, the lane-selection reasons, blockers, attention items, and each lane's
full report in a collapsible section. On BLOCKED: fix the findings per your
own findings-handling rules, then spawn a **fresh** gauntlet run. On PASS with
*Human attention required*: complete the work but carry those items into your
final report verbatim.
full report in a collapsible section. On BLOCKED with a
`GAUNTLET_REVIEW_INCOMPLETE:` line: the named lanes never completed — this is an
infrastructure fault, never a code finding. Fix any non-fault blockers listed
under `## Blockers` first (the re-run retries the lane); if the line persists,
escalate to the user with three options — retry the review again (a **fresh**
gauntlet run), accept NEEDS-HUMAN and proceed without that lane (recorded in the
report / PR body), or abort — never route it through your findings-handling
rules. On BLOCKED without it: fix the findings per your own findings-handling
rules, then spawn a **fresh** gauntlet run. On PASS with *Human attention
required*: complete the work but carry those items into your final report
verbatim.
## Notes
- The caller decides *whether* review is warranted at all (the gauntlet is for
non-trivial work — that's why code-review is unconditionally on); the
gauntlet decides *which* of the other lanes apply and what the verdicts mean.
- Lane sub-agents get generous finite timeouts (2h each; 3h whole-gauntlet)
and one retry (`max_attempts: 2`). A lane that still dies or times out falls
back to the `lane_fault` marker — BLOCKED naming the lane, never a silent
pass.
- Lane sub-agents get generous finite per-attempt timeouts (see the
`timeout:` on each `run_*` node) and one retry (`max_attempts: 2`) inside
the lane, then up to 3 attempts of the whole lane from `retry_gate.py` while
the wall-clock budget (`MAX_RETRY_ELAPSED_SECS`) allows; `settings.timeout`
is derived from that envelope in graph.yaml's comment. A lane that still
dies or times out falls back to the `lane_fault` marker — BLOCKED naming the
lane and `GAUNTLET_REVIEW_INCOMPLETE`, never a silent pass.
- The `verification_commands` variable is forwarded to the adversary lane as a
structured `inputs:` passthrough on `run_adversary` (a lone `{{…}}` template
lands the raw value in the child's state) — never through the lane prompt
+29 -14
View File
@@ -41,13 +41,14 @@ variables:
settings:
max_loop_iterations: 10
log_state_snapshots: true
# max over agent lanes of (max_attempts × lane timeout) = 2 × 11400 = 22800
# (agent-node retries each get a fresh per-attempt budget,
# src/graph/agent.rs retry_transient/bounded_attempt) + 1200 margin for
# the script stages (signals/select/build/gate) = 24000; the parse and
# select_lanes llm nodes carry no `timeout:` and are bounded by
# max_iterations.
timeout: 24000
# A faulted lane can be re-run by retry_gate for up to 3 passes, and each
# pass's agent node retries up to max_attempts = 2 with a fresh per-attempt
# budget (src/graph/agent.rs retry_transient/bounded_attempt):
# 3 passes × 2 attempts × 11400 (max lane timeout) = 68400, + 1200 margin
# for the script stages (signals/select/build/retry gate/verdict gate)
# = 69600; the parse and select_lanes llm nodes carry no `timeout:` and
# are bounded by max_iterations.
timeout: 69600
initial_state:
diff_spec: worktree
@@ -84,9 +85,15 @@ initial_state:
adversary_results: []
security_results: []
probe_results: []
lane_attempts: {}
kept_results: {}
retry_lanes: []
review_incomplete: []
gauntlet_started_at: 0
lanes_summary: ''
gauntlet_verdict: ''
gauntlet_report: ''
gauntlet_incomplete_line: ''
start: parse
@@ -251,7 +258,7 @@ nodes:
build_items:
id: build_items
type: script
description: Deterministic lane builder — hard rules are the floor, LLM flags may only add, forced lanes override exactly. Emits one 0-or-1-item list per lane; an empty list skips that lane's map.
description: Deterministic lane builder — hard rules are the floor, LLM flags may only add, forced lanes override exactly. Emits one 0-or-1-item list per lane; an empty list skips that lane's map. On a re-run pass (retry_lanes non-empty) it skips selection and emits items only for those lanes, each carrying its attempt number.
script: scripts/build_items.py
timeout: 30
next: [map_code_review, map_adversary, map_security, map_probe]
@@ -265,7 +272,7 @@ nodes:
output_key: lane_out
collect_into: code_review_results
max_concurrency: 1
next: verdict_gate
next: retry_gate
map_adversary:
id: map_adversary
@@ -276,7 +283,7 @@ nodes:
output_key: lane_out
collect_into: adversary_results
max_concurrency: 1
next: verdict_gate
next: retry_gate
map_security:
id: map_security
@@ -287,7 +294,7 @@ nodes:
output_key: lane_out
collect_into: security_results
max_concurrency: 1
next: verdict_gate
next: retry_gate
map_probe:
id: map_probe
@@ -298,7 +305,7 @@ nodes:
output_key: lane_out
collect_into: probe_results
max_concurrency: 1
next: verdict_gate
next: retry_gate
run_code_review:
id: run_code_review
@@ -412,10 +419,18 @@ nodes:
script: scripts/lane_fault.py
timeout: 30
retry_gate:
id: retry_gate
type: script
description: Deterministic fault-only re-run gate. Stashes every lane report that ran this pass, restores reports a re-run pass overwrote with [] (a lane not being re-run maps over nothing), classifies each lane that ran as completed / faulted / skipped, counts attempts, and routes back into build_items for faulted lanes that still have attempts left inside the wall-clock budget; otherwise it records them in review_incomplete and falls through to verdict_gate. A real verdict is never re-run, and a gate crash never retries — it records a PIPELINE-FAULT and falls through.
script: scripts/retry_gate.py
timeout: 30
next: verdict_gate
verdict_gate:
id: verdict_gate
type: script
description: Deterministic sentinel parser and gate. A missing sentinel is a lane failure, never a pass; probe INCONCLUSIVE blocks with an environment note; the code-review critical count is read from render.py's summary line (raw 🔴 count if absent) and blocks regardless of the lane's own verdict line; a code-review verdict line that records the nested graph's own pipeline fault blocks as a degraded lane.
description: Deterministic sentinel parser and gate. A missing sentinel is a lane failure, never a pass; probe INCONCLUSIVE blocks with an environment note; the code-review critical count is read from render.py's summary line (raw 🔴 count if absent) and blocks regardless of the lane's own verdict line; a code-review verdict line that records the nested graph's own pipeline fault blocks as a degraded lane. Lanes retry_gate gave up on, and any pipeline-level fault, are named on a GAUNTLET_REVIEW_INCOMPLETE machine line so a caller can tell an incomplete review from a failed one.
script: scripts/verdict_gate.py
timeout: 60
next: done
@@ -424,6 +439,6 @@ nodes:
id: done
type: end
output: |
GAUNTLET: {{gauntlet_verdict}}
GAUNTLET: {{gauntlet_verdict}}{{gauntlet_incomplete_line}}
{{gauntlet_report}}
@@ -9,6 +9,10 @@ skip mechanism — no dynamic routing anywhere.
The LLM's add_* flags can only ADD lanes. A flag that is false never
removes a lane the hard rules selected.
A re-run pass (retry_gate emitted `retry_lanes`) skips selection: only the
named lanes get an item, stamped with their attempt number, and every other
lane maps over nothing — retry_gate restores their earlier reports.
Fail-closed: a builder crash emits all four lists empty plus a
PIPELINE-FAULT in signals_error — verdict_gate blocks on it and the crash
never kills the graph.
@@ -16,6 +20,7 @@ never kills the graph.
import json
import os
import time
# Keep in sync with default_lanes.ALIASES — a caller-forced lane must
# canonicalize the same way on the degraded path as it does here.
@@ -35,6 +40,18 @@ ALIASES = {
"usage-pattern-testing": "probe",
}
# Keep in sync with retry_gate.LANES
LANE_KEYS = {
"code-review": "code_review_items",
"adversary": "adversary_items",
"security": "security_items",
"probe": "probe_items",
}
# Filled by main() so the crash guard can append to an upstream fault
# instead of overwriting it, and keep the first pass's lanes_summary.
crash_context = {"signals_error": "", "lanes_summary": ""}
def load_state():
if path := os.environ.get("GRAPH_STATE_FILE"):
@@ -45,11 +62,38 @@ def load_state():
def main():
state = load_state()
prior = state.get("signals_error")
crash_context["signals_error"] = prior if isinstance(prior, str) else ""
# retry_gate's wall-clock budget is measured from this stamp; signals.py
# is the normal stamper, this is the defensive fallback.
stamp = {} if state.get("gauntlet_started_at") else {"gauntlet_started_at": time.time()}
def text(key):
v = state.get(key)
return v.strip() if isinstance(v, str) else ""
crash_context["lanes_summary"] = text("lanes_summary")
retry_lanes = [lane for lane in (state.get("retry_lanes") or []) if lane in LANE_KEYS]
if retry_lanes:
attempts = state.get("lane_attempts") or {}
retried = [(lane, int(attempts.get(lane) or 0) + 1) for lane in retry_lanes]
note = "- retried: [" + ", ".join(f"{lane} (attempt {n})" for lane, n in retried) + "]"
summary = text("lanes_summary")
print(
json.dumps(
{
**{key: [] for key in LANE_KEYS.values()},
**{
LANE_KEYS[lane]: [{"lane": lane, "attempt": n}] for lane, n in retried
},
"lanes_summary": f"{summary}\n{note}" if summary else note,
"retry_lanes": [],
**stamp,
}
)
)
return
forced_raw = state.get("forced_lanes") or []
forced, unknown = [], []
for lane in forced_raw:
@@ -126,6 +170,7 @@ def main():
"security_items": [{"lane": "security"}] if "security" in lanes else [],
"probe_items": [{"lane": "probe"}] if "probe" in lanes else [],
"lanes_summary": "\n".join(f"- {r}" for r in reasons),
**stamp,
}
)
)
@@ -134,6 +179,9 @@ def main():
try:
main()
except Exception as e: # noqa: BLE001 — a builder crash must never kill the graph
fault = f"PIPELINE-FAULT: lane builder crashed — {e}; no lanes were run"
if crash_context["signals_error"]:
fault = f"{crash_context['signals_error']}; {fault}"
print(
json.dumps(
{
@@ -141,8 +189,8 @@ except Exception as e: # noqa: BLE001 — a builder crash must never kill the g
"adversary_items": [],
"security_items": [],
"probe_items": [],
"lanes_summary": "",
"signals_error": f"PIPELINE-FAULT: lane builder crashed — {e}; no lanes were run",
"lanes_summary": crash_context["lanes_summary"],
"signals_error": fault,
}
)
)
@@ -0,0 +1,206 @@
#!/usr/bin/env python3
"""Deterministic fault-only re-run gate for the review gauntlet.
Sits between the four lane maps and the verdict gate. Every lane map runs
on every pass — a lane that is not being re-run maps over [] — so a re-run
pass overwrites the untouched lanes' collected results with []. The gate's
hard rules:
- every lane that ran this pass is stashed in kept_results, completed
and faulted alike, and a lane whose map ran nothing this pass gets
its stashed report restored — the verdict gate always sees each
lane's LAST report;
- a lane is FAULTED only when its report is missing (the map ran but
collected nothing — replaced by a synthetic PIPELINE-FAULT marker so
the verdict gate never reads it as SKIPPED), non-string, the
lane_fault.py marker, sentinel-less, or carries the nested graph's own
degraded-run wording; a real verdict (DIVERGES, FAIL, INCONCLUSIVE,
NEEDS-HUMAN, …) is never re-run — the review's judgment stands;
- a faulted lane is re-run only while it has attempts left AND the
gauntlet is inside its wall-clock budget; otherwise it is recorded in
review_incomplete and its fault flows to the verdict gate as-is;
- malformed review_incomplete bookkeeping (not a list, or non-str
entries) keeps its str entries and records a PIPELINE-FAULT in
signals_error — never silently dropped, never split into lane names;
- the gate itself must never retry by accident: any internal error
records a PIPELINE-FAULT in signals_error and falls through to the
verdict gate, which blocks on it.
"""
import json
import os
import re
import time
# 69600 (graph timeout) − 2 × 11400 (largest lane envelope) − 1200 (script stages)
# = 45600; 600 s of slack below that.
MAX_RETRY_ELAPSED_SECS = 45000
MAX_ATTEMPTS = 3
# lane name -> (items key, results key)
LANES = {
"code-review": ("code_review_items", "code_review_results"),
"adversary": ("adversary_items", "adversary_results"),
"security": ("security_items", "security_results"),
"probe": ("probe_items", "probe_results"),
}
# Must stay identical to verdict_gate.py's sentinel regexes: a lane this
# gate calls completed must be one the verdict gate can parse, or a drift
# would re-run lanes the verdict gate accepts (or accept lanes it cannot
# read). A cross-pin test compares the literals in both files.
SENTINELS = {
"code-review": r"Verdict:\**\s*\**\s*(MERGE-READY|NEEDS-HUMAN)",
"adversary": r"ADVERSARIAL_REVIEW:\s*(CONFORMS|DIVERGES)",
"security": r"SECURITY_REVIEW:\s*(PASS|FAIL)",
"probe": r"USAGE_PROBE:\s*(PASS|FAIL|INCONCLUSIVE)",
}
# Filled by main() so the crash guard can append to an upstream fault
# instead of overwriting it, and report the lanes it had already declined.
crash_context = {"signals_error": "", "review_incomplete": []}
# Duplicated verbatim in the sibling gate script — the two copies must stay identical.
ADVERSARY_DEGRADED_HEADER = re.compile(
r"Criteria: .*("
r"degraded run: pipeline fault recorded"
r"|degraded run: \d+ criterion check\(s\) died"
r"|verdict computation error:"
r")"
)
def load_state():
if path := os.environ.get("GRAPH_STATE_FILE"):
with open(path) as f:
return json.load(f)
return json.loads(os.environ.get("GRAPH_STATE", "{}"))
def adversary_degraded(report):
# The adversary's verdict.py emits its degraded-run header on the line
# directly under the FIRST sentinel; anchoring there keeps a report that
# merely quotes the wording later (observations, complaints) a real verdict.
lines = report.splitlines()
for i, ln in enumerate(lines):
if re.search(r"ADVERSARIAL_REVIEW:\s*(CONFORMS|DIVERGES)", ln):
nxt = lines[i + 1] if i + 1 < len(lines) else ""
return bool(ADVERSARY_DEGRADED_HEADER.match(nxt))
return False
def nested_degraded(lane, report):
# The nested graphs complete with a sentinel even when a stage inside
# them died; their own degraded-run wording marks a partial review,
# which is not a completed one. Same anchoring as verdict_gate.py.
if lane == "code-review":
verdict_line = re.search(r"^\*\*Verdict: .*$", report, re.MULTILINE)
return bool(
verdict_line
and re.search(
r"^\*\*Verdict: NEEDS-HUMAN\*\* — .*"
r"(pipeline fault\(s\) recorded|PIPELINE-FAULT:|report rendering error)",
verdict_line.group(0),
)
)
if lane == "adversary":
return adversary_degraded(report)
return False
def is_faulted(lane, report):
if not isinstance(report, str):
return True
if report.strip().startswith("PIPELINE-FAULT:"):
return True
if not re.search(SENTINELS[lane], report):
return True
return nested_degraded(lane, report)
def attempt_count(attempts, lane):
try:
return int(attempts.get(lane) or 0)
except (TypeError, ValueError):
return 0
def main():
state = load_state()
prior = state.get("signals_error")
signals_error = prior if isinstance(prior, str) else ""
crash_context["signals_error"] = signals_error
out = {}
declared = state.get("review_incomplete")
if isinstance(declared, list):
incomplete = [name for name in declared if isinstance(name, str)]
malformed = len(incomplete) != len(declared)
else:
incomplete = []
malformed = declared is not None
if malformed:
fault = (
"PIPELINE-FAULT: malformed review_incomplete bookkeeping "
f"({repr(declared)[:80]})"
)
signals_error = f"{signals_error}; {fault}" if signals_error else fault
crash_context["signals_error"] = signals_error
out["signals_error"] = signals_error
crash_context["review_incomplete"] = incomplete
raw_attempts = state.get("lane_attempts")
attempts = dict(raw_attempts) if isinstance(raw_attempts, dict) else {}
kept = dict(state.get("kept_results") or {})
faulted = []
for lane, (items_key, results_key) in LANES.items():
results = state.get(results_key) or []
if not state.get(items_key):
if not results and lane in kept:
out[results_key] = [kept[lane]]
continue
report = results[0] if results else None
if report is None:
report = f"PIPELINE-FAULT: {lane} lane produced no report"
out[results_key] = [report]
kept[lane] = report
if is_faulted(lane, report):
faulted.append(lane)
attempts[lane] = attempt_count(attempts, lane) + 1
# A missing/zero stamp reads as an ancient start and declines every
# retry — the safe direction when the clock's origin is unknown.
started = state.get("gauntlet_started_at") or 0
within_budget = (time.time() - float(started)) <= MAX_RETRY_ELAPSED_SECS
retry = [lane for lane in faulted if within_budget and attempts[lane] < MAX_ATTEMPTS]
declined = [lane for lane in faulted if lane not in retry]
incomplete = sorted(set(incomplete) | set(declined))
crash_context["review_incomplete"] = incomplete
out.update(
{
"lane_attempts": attempts,
"kept_results": kept,
"retry_lanes": retry,
"review_incomplete": incomplete,
}
)
if retry:
out["_next"] = "build_items"
print(json.dumps(out))
try:
main()
except Exception as e: # noqa: BLE001 — a gate crash must fall through, never retry
fault = f"PIPELINE-FAULT: retry gate crashed — {e}"
if crash_context["signals_error"]:
fault = f"{crash_context['signals_error']}; {fault}"
print(
json.dumps(
{
"retry_lanes": [],
"review_incomplete": crash_context["review_incomplete"],
"signals_error": fault,
}
)
)
@@ -12,6 +12,7 @@ import json
import os
import re
import subprocess
import time
def load_state():
if path := os.environ.get("GRAPH_STATE_FILE"):
@@ -37,6 +38,10 @@ def git(*args):
out = {"project_dir": proj}
# retry_gate measures its wall-clock budget from this stamp; it is set
# before the git work so a failed diff still starts the clock.
if not state.get("gauntlet_started_at"):
out["gauntlet_started_at"] = time.time()
try:
if spec in ("", "worktree"):
names = git("diff", "--name-only", "HEAD")
@@ -8,14 +8,20 @@ re-reads, no judgment. The gate's hard rules:
- a non-string lane result (the map collected an object instead of the
lane's rendered text) blocks — it cannot carry a sentinel;
- a PIPELINE-FAULT lane result (the lane's agent died after retries;
detected only when the report IS the lane_fault.py marker, i.e. it
STARTS with it — a real review merely quoting the marker flows to the
normal rules) or a PIPELINE-FAULT in signals_error (builder/parse
fault) blocks — a degraded pipeline is never a pass;
detected only when the report IS the lane_fault.py marker or
retry_gate.py's no-report marker, i.e. it STARTS with it — a real
review merely quoting the marker flows to the normal rules) or a
PIPELINE-FAULT in signals_error (builder/parse fault) blocks — a
degraded pipeline is never a pass;
- a code-review lane whose OWN verdict line records a pipeline fault
(code-reviewer's verdict.py / render.py degraded-run wording) blocks —
the nested graph completed, but with a dead domain lane, verifier, or
synthesis behind it, so its critical count is not trustworthy;
- an adversary lane whose OWN header line (the line directly under its
sentinel) records a degraded run — a pipeline fault, died criterion
checks, or its verdict script's crash stub — blocks as a PIPELINE-FAULT,
not as a DIVERGES finding: the nested graph completed with a dead stage
behind it, so its verdict is not a judgment on the code;
- a non-zero critical count in the code-review report blocks regardless
of the lane's own verdict line — derived from render.py's deterministic
summary line (`*Reviewed N files, found X critical, …*`); the raw 🔴
@@ -24,14 +30,37 @@ re-reads, no judgment. The gate's hard rules:
- probe INCONCLUSIVE blocks with an ENVIRONMENT note — it is never
treated as PASS or FAIL (fix the environment per the plan's local-run
recipe, then re-run);
- a PIPELINE-FAULT lane row carries its attempt count (` ×N`) when retry_gate
re-ran it, so the reader can tell a first-pass death from an exhausted
retry budget;
- an incomplete review — lanes retry_gate gave up on (review_incomplete)
or a pipeline-level fault (reported under the `pipeline` pseudo-lane) —
emits the GAUNTLET_REVIEW_INCOMPLETE: machine line and a "Review
incomplete" section: that is a fault to re-run, never a code finding.
Every lane named in review_incomplete is forced to a BLOCKED row, so a
restored earlier report can never turn an incomplete review into a
pass: a non-empty machine line structurally implies GAUNTLET: BLOCKED.
Malformed review_incomplete bookkeeping (not a list, or non-str
entries) is itself recorded as a `pipeline` fault, never dropped;
- the gate itself must never crash into a silent pass: any internal
error emits GAUNTLET: BLOCKED naming the error.
error emits GAUNTLET: BLOCKED naming the error, with the pipeline
marked incomplete.
"""
import json
import os
import re
# Duplicated verbatim in the sibling gate script — the two copies must stay identical.
ADVERSARY_DEGRADED_HEADER = re.compile(
r"Criteria: .*("
r"degraded run: pipeline fault recorded"
r"|degraded run: \d+ criterion check\(s\) died"
r"|verdict computation error:"
r")"
)
def load_state():
if path := os.environ.get("GRAPH_STATE_FILE"):
with open(path) as f:
@@ -39,10 +68,26 @@ def load_state():
return json.loads(os.environ.get("GRAPH_STATE", "{}"))
def adversary_degraded(report):
# The adversary's verdict.py emits its degraded-run header on the line
# directly under the FIRST sentinel; anchoring there keeps a report that
# merely quotes the wording later (observations, complaints) a real verdict.
lines = report.splitlines()
for i, ln in enumerate(lines):
if re.search(r"ADVERSARIAL_REVIEW:\s*(CONFORMS|DIVERGES)", ln):
nxt = lines[i + 1] if i + 1 < len(lines) else ""
return bool(ADVERSARY_DEGRADED_HEADER.match(nxt))
return False
def main():
state = load_state()
blockers, attention, lane_lines, reports = [], [], [], []
blockers, attention, reports = [], [], []
rows = {}
malformed = set()
lane_attempts = state.get("lane_attempts")
if not isinstance(lane_attempts, dict):
lane_attempts = {}
def lane_report(key):
arr = state.get(key) or []
@@ -55,7 +100,17 @@ def main():
return json.dumps(item)
def record(lane, status, detail=""):
lane_lines.append(f"| {lane} | {status} | {detail} |")
rows[lane] = (status, detail)
def attempt_count(lane):
try:
return int(lane_attempts.get(lane) or 0)
except (TypeError, ValueError):
return 0
def attempts_suffix(lane):
n = attempt_count(lane)
return f" ×{n}" if n > 1 else ""
def fault_excerpt(report):
line = next(
@@ -64,10 +119,11 @@ def main():
return line.strip()[:200]
def is_lane_fault(report):
# A genuine lane fault is the ENTIRE report emitted by lane_fault.py,
# which always begins with the marker. Matching the marker anywhere
# would let a real review that merely QUOTES "PIPELINE-FAULT:" bypass
# 🔴 counting and sentinel parsing.
# A genuine lane fault is the ENTIRE report — the lane_fault.py marker
# or retry_gate.py's no-report marker — which always begins with
# "PIPELINE-FAULT:". Matching the marker anywhere would let a real
# review that merely QUOTES it bypass 🔴 counting and sentinel
# parsing.
return report.strip().startswith("PIPELINE-FAULT:")
def lane_fault_blocker(lane, report):
@@ -75,7 +131,7 @@ def main():
f"{lane}: PIPELINE-FAULT — the lane failed after retries and produced "
f"no verdict; a degraded lane is never a pass ({fault_excerpt(report)})"
)
record(lane, "BLOCKED", "PIPELINE-FAULT (lane failed)")
record(lane, "BLOCKED", f"PIPELINE-FAULT (lane failed{attempts_suffix(lane)})")
def malformed_blocker(lane, report):
blockers.append(f"{lane}: malformed lane result (non-string item) — never a pass")
@@ -129,10 +185,11 @@ def main():
blockers.append(
f"code-review: {reds} 🔴 CRITICAL finding(s) — fix before claiming done"
)
suffix = attempts_suffix("code-review")
if m_fault and reds:
record("code-review", "BLOCKED", f"PIPELINE-FAULT (degraded lane); {reds} 🔴")
record("code-review", "BLOCKED", f"PIPELINE-FAULT (degraded lane{suffix}); {reds} 🔴")
elif m_fault:
record("code-review", "BLOCKED", "PIPELINE-FAULT (degraded lane)")
record("code-review", "BLOCKED", f"PIPELINE-FAULT (degraded lane{suffix})")
elif reds:
record("code-review", "BLOCKED", f"{reds} 🔴 finding(s)")
elif m and m.group(1) == "MERGE-READY":
@@ -162,7 +219,17 @@ def main():
else:
reports.append(("adversary", adv))
m = re.search(r"ADVERSARIAL_REVIEW:\s*(CONFORMS|DIVERGES)", adv)
if m and m.group(1) == "CONFORMS":
if adversary_degraded(adv):
blockers.append(
"adversary: PIPELINE-FAULT — degraded run, pipeline fault recorded "
"inside the adversary; a degraded lane is never a pass"
)
record(
"adversary",
"BLOCKED",
f"PIPELINE-FAULT (degraded lane{attempts_suffix('adversary')})",
)
elif m and m.group(1) == "CONFORMS":
record("adversary", "GREEN", "CONFORMS")
elif m:
blockers.append(
@@ -228,16 +295,69 @@ def main():
)
record("probe", "BLOCKED", "missing verdict sentinel")
# retry_gate restores an exhausted lane's LAST report, which may be an
# earlier clean verdict — an incomplete lane must block regardless of
# what its row parsed to.
declared = state.get("review_incomplete")
incomplete = set()
malformed_bookkeeping = None
if isinstance(declared, list):
incomplete = {name for name in declared if isinstance(name, str)}
if any(not isinstance(name, str) for name in declared):
malformed_bookkeeping = repr(declared)[:80]
elif declared is not None:
malformed_bookkeeping = repr(declared)[:80]
for lane in sorted(incomplete - {"pipeline"}):
if rows.get(lane, ("", ""))[0] == "BLOCKED":
continue
after = f" after {n} attempt(s)" if (n := attempt_count(lane)) else ""
blockers.append(
f"{lane}: review incomplete{after} — the lane never produced a usable verdict"
)
record(lane, "BLOCKED", f"review incomplete{after}")
# A gauntlet-level fault (build_items crash, dead parse stage) means the
# lanes above were never built — SKIPPED rows alone must not read as PASS.
signals_error = state.get("signals_error")
if isinstance(signals_error, str) and "PIPELINE-FAULT:" in signals_error:
pipeline_fault = isinstance(signals_error, str) and "PIPELINE-FAULT:" in signals_error
if pipeline_fault:
blockers.append(f"pipeline: {signals_error}")
incomplete.add("pipeline")
if malformed_bookkeeping is not None:
blockers.append(
"pipeline: PIPELINE-FAULT — malformed review_incomplete bookkeeping "
f"({malformed_bookkeeping}); never a pass"
)
incomplete.add("pipeline")
if "pipeline" in incomplete and not any(b.startswith("pipeline:") for b in blockers):
blockers.append(
"pipeline: review incomplete — a pipeline stage never completed "
"(no fault text recorded)"
)
verdict = "BLOCKED" if blockers else "PASS"
verdict = "BLOCKED" if blockers or incomplete else "PASS"
incomplete_lines = []
for name in sorted(incomplete):
if name == "pipeline":
incomplete_lines.append(
f"- pipeline: {signals_error}"
if pipeline_fault
else "- pipeline: review incomplete (no fault text recorded)"
)
elif n := attempt_count(name):
incomplete_lines.append(f"- {name}: no completed review after {n} attempt(s)")
else:
incomplete_lines.append(f"- {name}: no completed review")
incomplete_line = (
"\nGAUNTLET_REVIEW_INCOMPLETE: " + ", ".join(sorted(incomplete))
if incomplete
else ""
)
md = "\n".join(
["| lane | status | detail |", "|------|--------|--------|"] + lane_lines
["| lane | status | detail |", "|------|--------|--------|"]
+ [f"| {lane} | {status} | {detail} |" for lane, (status, detail) in rows.items()]
)
md += "\n\n## Lane selection\n" + (state.get("lanes_summary") or "(none recorded)")
if blockers:
@@ -246,11 +366,21 @@ def main():
md += "\n\n## Human attention required\n" + "\n".join(f"- {a}" for a in attention)
if state.get("signals_error"):
md += f"\n\n> {state['signals_error']}"
if incomplete_lines:
md += "\n\n## Review incomplete\n" + "\n".join(incomplete_lines)
md += "\n\n## Full lane reports\n"
for lane, rep in reports:
md += f"\n<details><summary>{lane}</summary>\n\n{rep}\n\n</details>\n"
print(json.dumps({"gauntlet_verdict": verdict, "gauntlet_report": md}))
print(
json.dumps(
{
"gauntlet_verdict": verdict,
"gauntlet_report": md,
"gauntlet_incomplete_line": incomplete_line,
}
)
)
try:
@@ -261,6 +391,7 @@ except Exception as e: # noqa: BLE001 — the gate must never crash into a sile
{
"gauntlet_verdict": "BLOCKED",
"gauntlet_report": f"verdict gate error: {e} — treat as a failed gate, not a pass.",
"gauntlet_incomplete_line": "\nGAUNTLET_REVIEW_INCOMPLETE: pipeline",
}
)
)
+1 -1
View File
@@ -47,7 +47,7 @@ Spawnable sub-agents (from `config.yaml`):
- **[code-reviewer](../code-reviewer/README.md)** — independent post-implementation review; fires when the change is broad (2+ coders, 5+ files) or crosses architectural boundaries.
- **[adversary](../adversary/README.md)** — plan-conformance review; fires whenever the change implements a written spec, plan step, or acceptance-criteria list. Orthogonal to `code-reviewer` — both can run.
- **[security-reviewer](../security-reviewer/README.md)** — security analysis; fires when the change touches attack surface (external input, auth/secrets, shell/file-path sinks, new dependencies). Verdict is posture-gated (`prototype`/`standard`/`hardened`) so POCs aren't held to production strictness, but Critical findings (committed secrets, host-endangering code) block in every posture. Orthogonal to both other reviewers — all three can run.
- **[review-gauntlet](../review-gauntlet/README.md)** — preferred single entry point for the post-implementation review passes: a graph that computes diff signals deterministically, runs the applicable lanes (code-reviewer / adversary / security-reviewer / probe) in parallel, and machine-parses their verdict sentinels into GAUNTLET: PASS/BLOCKED — skipping a lane or misreading a verdict becomes structurally impossible. The reviewer bullets above describe the lanes it runs and remain the spawn fallback when it is not installed.
- **[review-gauntlet](../review-gauntlet/README.md)** — preferred single entry point for the post-implementation review passes: a graph that computes diff signals deterministically, runs the applicable lanes (code-reviewer / adversary / security-reviewer / probe) in parallel, and machine-parses their verdict sentinels into GAUNTLET: PASS/BLOCKED — skipping a lane or misreading a verdict becomes structurally impossible. A `GAUNTLET_REVIEW_INCOMPLETE:` line under BLOCKED is an infrastructure fault (a lane exhausted its retries, or the pipeline faulted) that Sisyphus escalates to the user — retry / accept NEEDS-HUMAN / abort — never a finding to fix. The reviewer bullets above describe the lanes it runs and remain the spawn fallback when it is not installed.
- **[step-runner](../step-runner/README.md)** — graph agent that executes one step of a phased plan repo. Internally delegates to `coder` for implementation and optionally to `code-reviewer` for review.
- **[whetstone](../whetstone/README.md)** — continuous improvement; spawned ONCE at the end of a run that addressed review feedback from outside Sisyphus's own lanes (humans, or a teammate's independent review agent — never its own reviewers' output). Classifies each piece of feedback CATCH/PARTIAL/MISS against what the run's own reviewers flagged and converts the misses into user-approved suite improvements (skill edits, ledger entries). Never blocks run completion.
+21 -5
View File
@@ -329,11 +329,27 @@ instructions: |
parallel verdict-independent, and returns a machine-parsed `GAUNTLET: PASS` or `GAUNTLET: BLOCKED`
with a lane table, blockers, attention items, and every lane's full report.
- `GAUNTLET: BLOCKED` blocks completion. Route each blocker through the findings-handling rules in
the reviewer sections below (🔴 → coder fix in the SAME session; DIVERGES → the adversary rules;
security FAIL → the security rules; probe INCONCLUSIVE → fix the ENVIRONMENT per the plan's
local-run recipe first — it is never a code verdict). Then spawn a FRESH gauntlet run; lane
selection re-runs from the new diff.
- `GAUNTLET: BLOCKED` blocks completion. Route each blocker that is a real review finding through
the findings-handling rules in the reviewer sections below (🔴 → coder fix in the SAME session;
DIVERGES → the adversary rules; security FAIL → the security rules; probe INCONCLUSIVE → fix the
ENVIRONMENT per the plan's local-run recipe first — it is never a code verdict). Then spawn a
FRESH gauntlet run; lane selection re-runs from the new diff. A BLOCKED whose only blockers are
review faults (`PIPELINE-FAULT` / `review incomplete` entries) is NOT a code failure — the next
rule owns it.
- `GAUNTLET_REVIEW_INCOMPLETE: <lane>[, <lane>]` (the line directly under `GAUNTLET: BLOCKED`)
means the review could NOT be completed: the named lanes still faulted after 3 attempts, or the
gauntlet pipeline itself faulted (`pipeline`). This is an infrastructure fault, not a code
finding. NEVER route it through the findings-handling rules. NEVER claim done while it stands.
Escalate to the user via `user__select` with exactly these three options:
- "Retry the review again"
- "Accept NEEDS-HUMAN: proceed without the <lane or pipeline> review (recorded in the report / PR body)"
- "Abort this task"
Retry → spawn a FRESH gauntlet run. Accept → finish the work and record the skipped review
(lane names or `pipeline`) in your final report and in the PR body when you author one. Abort →
stop; leave the code in a working state and report. Findings AND incomplete on the same output →
fix the findings first, then re-run the gauntlet (the re-run retries the lane) — escalate only if
the line persists. Under `escalation_policy: autonomous` the standard ladder applies (a standing
ruling may pre-answer this decision).
- `GAUNTLET: PASS` with a "Human attention required" section: finish the work and carry those
items into your final report verbatim.
- The reviewer sections below remain authoritative for HANDLING each lane's findings, and are the
+13 -17
View File
@@ -14,13 +14,13 @@
model: openai:gpt-4o # Specify the LLM to use
temperature: null # Set default temperature parameter, range (0, 1)
top_p: null # Set default top-p parameter, with a range of (0, 1) or (0, 2) depending on the model
reasoning_effort:
null # Reasoning effort level for models that support it (e.g. low, medium, high).
reasoning_effort: null # Reasoning effort level for models that support it (e.g. low, medium, high).
# Only valid when the agent's model declares reasoning_levels.
agent_session: null # Set a session to use when starting the agent. (e.g. temp, default); defaults to globally set agent_session
name: <agent-name> # Name of the agent, used in the UI and logs
description: <description> # Description of the agent, used in the UI
version: 1 # Version of the agent
# Auto-Continue (Todo System)
# The auto-continue system provides built-in task tracking for improved reliability.
# When enabled, the model can create todo lists and the system will automatically
@@ -30,6 +30,7 @@ auto_continue: false # Enable automatic continuation when incomplete todos remai
max_auto_continues: 10 # Maximum number of automatic continuations before stopping
inject_todo_instructions: true # Inject the default todo tool usage instructions into the agent's system prompt
continuation_prompt: null # Custom prompt used when auto-continuing (optional; uses default if null)
# Sub-Agent Spawning System
# Enable this agent to spawn and manage child agents in parallel.
# See https://github.com/Dark-Alex-17/coyote/wiki/Agents for detailed documentation.
@@ -38,17 +39,16 @@ can_spawn_agents: false # Enable the agent to spawn child agents
# - explore # If omitted (the default), ALL installed agents are spawnable. This is the unrestricted default.
# - coder # Provide a list to restrict. Match is exact and case-sensitive (use directory names).
# - oracle # An empty list ([]) means literally nothing spawnable.
# Also filters `agent__list_available` output so the LLM only sees what it can spawn.
# Graph agents (graph.yaml) ignore this; they declare spawn targets in agent nodes.
# Also filters `agent__list_available` output so the LLM only sees what it can spawn.
# Graph agents (graph.yaml) ignore this; they declare spawn targets in agent nodes.
max_concurrent_agents: 4 # Maximum number of agents that can run simultaneously
max_agent_depth: 3 # Maximum nesting depth for sub-agents (prevents runaway spawning)
max_concurrent_jobs:
5 # Max background jobs (`job__*` tools) running at once for this agent
max_concurrent_jobs: 5 # Max background jobs (`job__*` tools) running at once for this agent
# (overrides the global setting; 0 disables background jobs for this agent)
inject_spawn_instructions: true # Inject the default agent spawning instructions into the agent's system prompt
summarization_model: null # Model to use for summarizing sub-agent output (e.g. 'openai:gpt-4o-mini'); defaults to current model
summarization_threshold: 4000 # Character threshold above which sub-agent output is summarized before returning to parent
escalation_timeout: 300 # Seconds a sub-agent waits for a user interaction response before timing out (default: 5 minutes)
escalation_timeout: 0 # 0 = wait indefinitely for the reply (default); N = give up after N seconds
mcp_servers: # Optional list of MCP servers that the agent utilizes
- github # Corresponds to the name of an MCP server in the `<coyote-config-dir>/mcp.json` file
mcp_tools: # Optional per-server tool allowlist for the agent's MCP servers
@@ -59,26 +59,22 @@ global_tools: # Optional list of additional global tools to enable for the agent
- web_search
- fs
- python
skills_enabled:
true # Master switch for skills in this agent (default: inherit from global).
skills_enabled: true # Master switch for skills in this agent (default: inherit from global).
# Skills also require `function_calling_support: true` in the global config.
enabled_skills: # Optional list of skills available when this agent runs.
# Must be a subset of global `visible_skills`. Omit to inherit the global default.
- git-master
- git-master # Must be a subset of global `visible_skills`. Omit to inherit the global default.
- ai-slop-remover
inject_skill_instructions:
true # Inject a short hint pointing the model at `skill__list` when skills are enabled
inject_skill_instructions: true # Inject a short hint pointing the model at `skill__list` when skills are enabled
# (default: true). Suppressed automatically when no skills are available.
skill_instructions: null # Custom text for the skill hint (optional; uses built-in default if null)
enabled_macros: # Optional list of macros invocable when this agent is active in the REPL.
- generate-commit-message # An empty list disables all macros. Omit to inherit the role/global default.
memory:
null # Per-agent memory override (default: inherit). Set to `false` to disable memory
memory: null # Per-agent memory override (default: inherit). Set to `false` to disable memory
# for this agent regardless of workspace/global presence. See the Memory wiki page.
dynamic_instructions: false # Whether to use dynamic instructions for the agent; if false, static instructions are used
instructions:
| # Static instructions for the agent; ignored if dynamic instructions are used
instructions: # Static instructions for the agent; ignored if dynamic instructions are used
|
You are a AI agent designed to demonstrate agent capabilities.
<tools>
+2 -1
View File
@@ -889,7 +889,7 @@ fn default_summarization_threshold() -> usize {
}
fn default_escalation_timeout() -> u64 {
300
0
}
impl AgentConfig {
@@ -1341,6 +1341,7 @@ variables:
assert_eq!(config.max_concurrent_agents, 4);
assert_eq!(config.max_agent_depth, 3);
assert_eq!(config.max_auto_continues, 10);
assert_eq!(config.escalation_timeout, 0);
assert!(config.mcp_servers.is_empty());
assert!(config.global_tools.is_empty());
assert!(config.conversation_starters.is_empty());
+6
View File
@@ -1383,6 +1383,12 @@ clients:
assert_eq!(cfg.clients.len(), 1);
}
#[test]
fn config_template_does_not_carry_the_per_agent_escalation_timeout_key() {
// `escalation_timeout` is a per-agent setting; the global template must never grow it.
assert!(!CONFIG_TEMPLATE.contains("escalation_timeout"));
}
#[test]
fn config_enabled_macros_empty_string_is_some_empty() {
let cfg: Config = serde_yaml::from_str("enabled_macros: \"\"").unwrap();
+1 -1
View File
@@ -200,7 +200,7 @@ pub(in crate::config) const DEFAULT_SPAWN_INSTRUCTIONS: &str = indoc! {"
1. Read the child's question and options.
2. If you can answer from context, call `agent__reply_escalation` with your answer.
3. If you need the user's input, call the appropriate `user__*` tool yourself, then relay the answer via `agent__reply_escalation`.
4. **Respond promptly**; the child agent is blocked and waiting (5-minute timeout).
4. **Respond promptly**; the child blocks until answered — no timeout by default (`escalation_timeout` per agent).
"};
pub(in crate::config) const DEFAULT_JOB_INSTRUCTIONS: &str = indoc! {"
File diff suppressed because it is too large. Load diff
+5 -1
View File
@@ -1711,7 +1711,7 @@ fn handle_reply_escalation(ctx: &mut RequestContext, args: &Value) -> Result<Val
}
None => Ok(json!({
"status": "error",
"message": format!("No pending escalation found with id '{escalation_id}'. It may have already been replied to or timed out."),
"message": format!("No pending escalation found with id '{escalation_id}'. It may have already been replied to."),
})),
}
}
@@ -3093,6 +3093,10 @@ mod tests {
)
.unwrap();
assert_eq!(result["status"], "error");
assert_eq!(
result["message"],
"No pending escalation found with id 'missing'. It may have already been replied to."
);
}
#[test]
+138 -9
View File
@@ -13,7 +13,7 @@ use tokio::sync::oneshot;
pub const USER_FUNCTION_PREFIX: &str = "user__";
const DEFAULT_ESCALATION_TIMEOUT_SECS: u64 = 300;
const DEFAULT_ESCALATION_TIMEOUT_SECS: u64 = 0;
const CUSTOM_MULTI_CHOICE_ANSWER_OPTION: &str = "Other (custom)";
pub fn user_interaction_function_declarations() -> Vec<FunctionDeclaration> {
@@ -289,18 +289,33 @@ async fn handle_escalated(ctx: &RequestContext, action: &str, args: &Value) -> R
root_queue.submit(request);
let timeout = Duration::from_secs(timeout_secs);
match tokio::time::timeout(timeout, rx).await {
Ok(Ok(reply)) => Ok(json!({ "answer": reply })),
Ok(Err(_)) => Ok(json!({
"error": "Escalation was cancelled. The parent agent dropped the request",
"fallback": "Make your best judgment and proceed",
})),
Err(_) => Ok(json!({
await_escalation_reply(rx, timeout_secs).await
}
/// Waits for the parent's reply. `timeout_secs == 0` waits indefinitely; a
/// dropped sender (the parent cancelled the request) resolves either way.
async fn await_escalation_reply(rx: oneshot::Receiver<String>, timeout_secs: u64) -> Result<Value> {
let reply = if timeout_secs == 0 {
rx.await
} else {
match tokio::time::timeout(Duration::from_secs(timeout_secs), rx).await {
Ok(reply) => reply,
Err(_) => {
return Ok(json!({
"error": format!(
"Escalation timed out after {timeout_secs} seconds waiting for user response"
),
"fallback": "Make your best judgment and proceed",
}));
}
}
};
match reply {
Ok(reply) => Ok(json!({ "answer": reply })),
Err(_) => Ok(json!({
"error": "Escalation was cancelled. The parent agent dropped the request",
"fallback": "Make your best judgment and proceed",
})),
}
}
@@ -331,6 +346,8 @@ fn parse_options(args: &Value) -> Result<Vec<String>> {
#[cfg(test)]
mod tests {
use super::*;
use std::fs;
use std::path::{Path, PathBuf};
#[test]
fn headless_select_returns_structured_json() {
@@ -351,4 +368,116 @@ mod tests {
assert_eq!(v["action"], "confirm");
assert_eq!(v["options"], json!([]));
}
#[test]
fn escalation_timeout_defaults_to_unlimited() {
assert_eq!(DEFAULT_ESCALATION_TIMEOUT_SECS, 0);
}
#[tokio::test(start_paused = true)]
async fn zero_timeout_waits_indefinitely_for_the_reply() {
let (tx, rx) = oneshot::channel();
let wait = tokio::spawn(await_escalation_reply(rx, 0));
tokio::time::advance(Duration::from_secs(3600)).await;
assert!(!wait.is_finished());
tx.send("approved".to_string()).unwrap();
let v = wait.await.unwrap().unwrap();
assert_eq!(v, json!({ "answer": "approved" }));
}
#[tokio::test(start_paused = true)]
async fn nonzero_timeout_still_expires_with_the_existing_message() {
let (tx, rx) = oneshot::channel::<String>();
let wait = tokio::spawn(await_escalation_reply(rx, 5));
tokio::task::yield_now().await;
tokio::time::advance(Duration::from_secs(4)).await;
assert!(!wait.is_finished());
tokio::time::advance(Duration::from_secs(2)).await;
let v = wait.await.unwrap().unwrap();
assert_eq!(
v["error"],
"Escalation timed out after 5 seconds waiting for user response"
);
assert_eq!(v["fallback"], "Make your best judgment and proceed");
assert!(tx.send("too late".to_string()).is_err());
}
#[tokio::test(start_paused = true)]
async fn zero_timeout_resolves_when_the_sender_is_dropped() {
let (tx, rx) = oneshot::channel::<String>();
drop(tx);
let v = tokio::time::timeout(Duration::from_secs(1), await_escalation_reply(rx, 0))
.await
.expect("the wait must complete once the sender is gone")
.unwrap();
assert!(
v["error"]
.as_str()
.unwrap()
.starts_with("Escalation was cancelled"),
"{v}"
);
assert_eq!(v["fallback"], "Make your best judgment and proceed");
}
#[test]
fn stale_timeout_wording_is_gone_from_shipped_text() {
// Split so this file does not match its own needles.
let needles = [
concat!("(5-minute", " timeout)"),
concat!("escalation_timeout:", " 300"),
concat!("default:", " 5 minutes"),
];
let root = Path::new(env!("CARGO_MANIFEST_DIR"));
let example = root.join("config.agent.example.yaml");
let example_text = fs::read_to_string(&example).unwrap();
for needle in ["escalation_timeout: 0", "0 = wait indefinitely"] {
assert!(
example_text.contains(needle),
"{} lost the shipped wording {needle:?}",
example.display()
);
}
let mut files = vec![example];
collect_files(&root.join("src"), &mut files);
collect_files(&root.join("assets"), &mut files);
let this_file = root.join(file!());
assert!(
files.contains(&this_file),
"walk did not reach {}",
this_file.display()
);
let mut hits = Vec::new();
for path in &files {
let text = String::from_utf8_lossy(&fs::read(path).unwrap()).into_owned();
for needle in needles {
if text.contains(needle) {
hits.push(format!("{}: {needle:?}", path.display()));
}
}
}
assert!(
hits.is_empty(),
"stale timeout wording found:\n{}",
hits.join("\n")
);
}
fn collect_files(dir: &Path, out: &mut Vec<PathBuf>) {
for entry in fs::read_dir(dir).unwrap().flatten() {
let path = entry.path();
if path.is_dir() {
collect_files(&path, out);
} else if path.is_file() {
out.push(path);
}
}
}
}
+111
View File
@@ -1325,6 +1325,117 @@ nodes:
);
}
/// A script `_next` back-edge into a node with a static fan-out must
/// re-fire every fan-out target on the next pass, and a map that
/// re-runs over zero items must overwrite its `collect_into` with `[]`
/// rather than leave the previous pass's results in place.
#[tokio::test]
async fn script_next_back_edge_refires_static_fan_out_and_empty_map_rerun_clears_collect_into()
{
if !cmd_available("python3") {
eprintln!("skipping: python3 not available");
return;
}
let ws = TestWorkspace::new();
ws.write_script(
"build.py",
r#"#!/usr/bin/env python3
import os, json
state = json.loads(os.environ.get("GRAPH_STATE", "{}"))
if state["pass"] == 0:
print(json.dumps({"a_items": [1], "b_items": [1]}))
else:
print(json.dumps({"a_items": [], "b_items": [2]}))
"#,
);
ws.write_script(
"worker.py",
r#"#!/usr/bin/env python3
import os, json
state = json.loads(os.environ.get("GRAPH_STATE", "{}"))
print(json.dumps({"output": f"report-{state['item']}"}))
"#,
);
ws.write_script(
"gate.py",
r#"#!/usr/bin/env python3
import os, json
state = json.loads(os.environ.get("GRAPH_STATE", "{}"))
if state["pass"] == 0:
print(json.dumps({"pass": 1, "_next": "build"}))
else:
print(json.dumps({}))
"#,
);
let yaml = r#"
name: fan_out_rerun_test
start: build
initial_state:
pass: 0
a_items: []
b_items: []
a_results: []
b_results: []
nodes:
build:
type: script
script: build.py
next: [map_a, map_b]
map_a:
type: map
over: "{{a_items}}"
as: item
branch: worker
collect_into: a_results
next: gate
map_b:
type: map
over: "{{b_items}}"
as: item
branch: worker
collect_into: b_results
next: gate
worker:
type: script
script: worker.py
state_updates: {}
timeout: 60
gate:
type: script
script: gate.py
next: done
done:
type: end
output: "{{a_results}}|{{b_results}}"
"#;
let graph: Graph = serde_yaml::from_str(yaml).unwrap();
let mut ctx = make_ctx();
let abort = create_abort_signal();
let result = GraphExecutor::new(graph, &ws.dir)
.execute(&mut ctx, abort)
.await
.unwrap_or_else(|e| panic!("executor failed: {e:#}"));
let (a, b) = result
.split_once('|')
.unwrap_or_else(|| panic!("expected `a|b`, got: {result}"));
let a: Value =
serde_json::from_str(a).unwrap_or_else(|_| panic!("a_results not JSON: {result}"));
let b: Value =
serde_json::from_str(b).unwrap_or_else(|_| panic!("b_results not JSON: {result}"));
assert_eq!(
b,
serde_json::json!(["report-2"]),
"the back-edge into build must re-fire both fan-out targets: {result}"
);
assert_eq!(
a,
serde_json::json!([]),
"map_a must re-fire on the back-edge pass and clear its collect_into over zero items: {result}"
);
}
#[tokio::test]
async fn parallel_branch_error_aborts_super_step() {
if !cmd_available("bash") {
+9 -1
View File
@@ -2281,6 +2281,7 @@ where
mod tests {
use super::*;
use std::path::PathBuf;
use std::sync::atomic::{AtomicU64, Ordering};
use std::time::{SystemTime, UNIX_EPOCH};
struct TempDir {
@@ -2289,10 +2290,17 @@ mod tests {
impl TempDir {
fn new(tag: &str) -> Self {
let unique = SystemTime::now()
// Wall-clock nanos alone are not unique across parallel tests
// (coarse clocks hand the same tick to concurrent callers, so two
// tests would share a directory and overwrite each other's files);
// a process-wide counter makes every name distinct.
static SEQ: AtomicU64 = AtomicU64::new(0);
let nanos = SystemTime::now()
.duration_since(UNIX_EPOCH)
.unwrap()
.as_nanos();
let seq = SEQ.fetch_add(1, Ordering::Relaxed);
let unique = format!("{nanos}-{seq}");
let path = env::temp_dir().join(format!("coyote-rag-{tag}-{unique}"));
fs::create_dir_all(&path).unwrap();
Self { path }