docs: Added documentation for the new whetstone and finding-verifier agents
2 files changed
+24
-3
No files matched your search
+23
-3
@@ -871,7 +871,9 @@ Coyote comes packaged with some useful built-in agents:
|
|||||||
tracked on disk in a `plans/` directory), spawns one `sisyphus` per task on a single run branch with a
|
tracked on disk in a `plans/` directory), spawns one `sisyphus` per task on a single run branch with a
|
||||||
per-task `adversary` conformance check (plus a `probe` usage-pattern check when the task changes
|
per-task `adversary` conformance check (plus a `probe` usage-pattern check when the task changes
|
||||||
consumer-facing surface), and finishes with ONE draft PR, CI checks watched to green,
|
consumer-facing surface), and finishes with ONE draft PR, CI checks watched to green,
|
||||||
plus follow-up task files for every manual action the code can't do for itself. Supports opt-in
|
plus follow-up task files for every manual action the code can't do for itself. Ends with a
|
||||||
|
`whetstone` continuous-improvement pass when the run addressed review feedback from outside its
|
||||||
|
own lanes. Supports opt-in
|
||||||
worktree-based parallel task execution. Plans declare a quality bar: `rigor` (poc/prototype/production)
|
worktree-based parallel task execution. Plans declare a quality bar: `rigor` (poc/prototype/production)
|
||||||
plus the `surfaces` being built (rest-api, cli, library, worker, iac, db-migration, frontend, ci-cd, or
|
plus the `surfaces` being built (rest-api, cli, library, worker, iac, db-migration, frontend, ci-cd, or
|
||||||
`other:<label>` via a design-time `librarian` lookup), which flows into every downstream reviewer, with
|
`other:<label>` via a design-time `librarian` lookup), which flows into every downstream reviewer, with
|
||||||
@@ -898,6 +900,12 @@ Coyote comes packaged with some useful built-in agents:
|
|||||||
review skills (`rest-api-review`, `cli-review`, `library-review`, `worker-review`, `iac-review`,
|
review skills (`rest-api-review`, `cli-review`, `library-review`, `worker-review`, `iac-review`,
|
||||||
`migration-review`, `cicd-review`), whose findings carry `[convention]`/`[correctness]` markers used for
|
`migration-review`, `cicd-review`), whose findings carry `[convention]`/`[correctness]` markers used for
|
||||||
rigor folding and the evidence-cited rejection protocol downstream.
|
rigor folding and the evidence-cited rejection protocol downstream.
|
||||||
|
* `finding-verifier`: A graph-based fact-checker for code-review findings (used by the `code-reviewer` agent).
|
||||||
|
Maps over a review's 🔴/🟡 findings in parallel, fresh context per finding, so the verifier inherits none of
|
||||||
|
the reviewer's assumptions, re-reads the cited code, and returns `VERIFIED` (with quoted evidence), `FALSE`
|
||||||
|
(only with proof, e.g. the "missing" guard exists at another path:line), or `UNVERIFIABLE` (kept, marked).
|
||||||
|
It never re-judges severity. It only tests whether a claim is true. `code-reviewer` uses the verdicts to
|
||||||
|
drop provably-false findings before the final report; dropped findings are tallied, never silently vanished.
|
||||||
* `gatekeeper`: A plan self-containedness gate (via the `plan-gatekeeping` skill). Audits a plan against
|
* `gatekeeper`: A plan self-containedness gate (via the `plan-gatekeeping` skill). Audits a plan against
|
||||||
the "sealed container" standard, meaning every question a context-free implementer will hit is either answered
|
the "sealed container" standard, meaning every question a context-free implementer will hit is either answered
|
||||||
inline or delegated via a VERIFIED pointer to the exact code/docs that answer it (where infra code goes,
|
inline or delegated via a VERIFIED pointer to the exact code/docs that answer it (where infra code goes,
|
||||||
@@ -929,11 +937,13 @@ Coyote comes packaged with some useful built-in agents:
|
|||||||
Designed to be delegated to by `sisyphus` alongside `code-reviewer` and `adversary`.
|
Designed to be delegated to by `sisyphus` alongside `code-reviewer` and `adversary`.
|
||||||
* `sisyphus`: A powerhouse orchestrator agent for writing complex code and acting as a natural language interface for
|
* `sisyphus`: A powerhouse orchestrator agent for writing complex code and acting as a natural language interface for
|
||||||
your codebase (similar to ClaudeCode, Gemini CLI, Codex, or OpenCode). Uses sub-agent spawning to delegate to
|
your codebase (similar to ClaudeCode, Gemini CLI, Codex, or OpenCode). Uses sub-agent spawning to delegate to
|
||||||
`explore`, `librarian`, `coder`, `oracle`, `code-reviewer`, `adversary`, `security-reviewer`, `probe`, and `step-runner`. On
|
`explore`, `librarian`, `coder`, `oracle`, `code-reviewer`, `adversary`, `security-reviewer`, `probe`, `step-runner`, and `whetstone`. On
|
||||||
non-trivial work it runs up to four complementary independent review passes: `code-reviewer` for code quality,
|
non-trivial work it runs up to four complementary independent review passes: `code-reviewer` for code quality,
|
||||||
`adversary` for plan conformance, and `security-reviewer` for exploitability when the change touches attack
|
`adversary` for plan conformance, and `security-reviewer` for exploitability when the change touches attack
|
||||||
surface, and `probe` for black-box usage-pattern verification when the change touches
|
surface, and `probe` for black-box usage-pattern verification when the change touches
|
||||||
consumer-facing surface. Also supports plan-driven workflows: authoring phased implementation plans (via the `plan-authoring`
|
consumer-facing surface. After a run that addressed review feedback from outside its own lanes (humans,
|
||||||
|
or a teammate's independent review agent), it spawns `whetstone` once to fold the review misses back into
|
||||||
|
the suite as user-approved improvements. Also supports plan-driven workflows: authoring phased implementation plans (via the `plan-authoring`
|
||||||
skill), having `oracle` review them, and executing them one reviewed step at a time. Handles review findings
|
skill), having `oracle` review them, and executing them one reviewed step at a time. Handles review findings
|
||||||
quality-bar-aware: the plan's `rigor` calibrates which severities block (🔴 always does) and maps to a default
|
quality-bar-aware: the plan's `rigor` calibrates which severities block (🔴 always does) and maps to a default
|
||||||
`security_posture`; below-threshold `[convention]` findings defer into follow-ups instead of blocking; and a
|
`security_posture`; below-threshold `[convention]` findings defer into follow-ups instead of blocking; and a
|
||||||
@@ -946,6 +956,16 @@ Coyote comes packaged with some useful built-in agents:
|
|||||||
sweep -> optional independent review (`code-reviewer`) -> evidence-backed handoff -> hard-stop user approval gate.
|
sweep -> optional independent review (`code-reviewer`) -> evidence-backed handoff -> hard-stop user approval gate.
|
||||||
Returns `STEP_COMPLETE`/`STEP_BLOCKED`/`STEP_REJECTED`/`STEP_FAILED` to the caller. Designed to be delegated to by
|
Returns `STEP_COMPLETE`/`STEP_BLOCKED`/`STEP_REJECTED`/`STEP_FAILED` to the caller. Designed to be delegated to by
|
||||||
`sisyphus`.
|
`sisyphus`.
|
||||||
|
* `whetstone`: A continuous-improvement agent for the review suite; the review suite reviews code; whetstone
|
||||||
|
reviews the review. Spawned by `sisyphus` and `architect` at the end of a run that addressed review feedback
|
||||||
|
from outside the run's own lanes (humans, or a teammate's independent review agent, never its own suite's
|
||||||
|
output; third-party agent findings count only when the author actually addressed them). Classifies each piece
|
||||||
|
of feedback as CATCH/PARTIAL/MISS against what the suite's own reviewers flagged, then generalizes the misses:
|
||||||
|
taste is dropped, one-offs go to a candidates inbox, org-specific classes become review-miss ledger entries,
|
||||||
|
and general classes are drafted as suite edits in house style. Everything is applied only with user approval,
|
||||||
|
and edits go where the suite lives: a configured `suite_repo` (then optionally mirrored to the live config,
|
||||||
|
drift-guarded), or directly into the live config with per-file backups when no repo is set. Additive only;
|
||||||
|
it never weakens an existing rule.
|
||||||
|
|
||||||
Coyote writes these built-in agents to your agents directory on first run and never overwrites them afterward, so any
|
Coyote writes these built-in agents to your agents directory on first run and never overwrites them afterward, so any
|
||||||
edits you make to them are preserved across Coyote updates. To discard your local changes and reinstall the built-in
|
edits you make to them are preserved across Coyote updates. To discard your local changes and reinstall the built-in
|
||||||
|
|||||||
+1
@@ -414,6 +414,7 @@ Coyote ships with thirty-five built-in skills, installed automatically on first
|
|||||||
| `adversarial-review` | `fs_read, fs_grep, fs_glob, fs_cat, fs_ls` | Adversarial plan-conformance review of an *implementation* against the plan it should satisfy (skipped criteria, scope drift, interface substitution); verdict `CONFORMS`/`DIVERGES`. Complements `code-review` (quality) and is the post-implementation counterpart to `plan-review`. |
|
| `adversarial-review` | `fs_read, fs_grep, fs_glob, fs_cat, fs_ls` | Adversarial plan-conformance review of an *implementation* against the plan it should satisfy (skipped criteria, scope drift, interface substitution); verdict `CONFORMS`/`DIVERGES`. Complements `code-review` (quality) and is the post-implementation counterpart to `plan-review`. |
|
||||||
| `plan-gatekeeping` | `fs_read, fs_grep, fs_glob, fs_cat, fs_ls` | Gatekeep a plan for *self-containedness* before finalization: the "sealed container" test — every implementer question answered inline or via a verified pointer; verdict `PLAN_GATE: SEALED`/`LEAKY` with the missing questions. |
|
| `plan-gatekeeping` | `fs_read, fs_grep, fs_glob, fs_cat, fs_ls` | Gatekeep a plan for *self-containedness* before finalization: the "sealed container" test — every implementer question answered inline or via a verified pointer; verdict `PLAN_GATE: SEALED`/`LEAKY` with the missing questions. |
|
||||||
| `usage-pattern-testing` | `fs_read, fs_cat, fs_grep, fs_glob, fs_ls, fs_write, fs_patch, execute_command` | Black-box verification of a change's consumer-facing surface (HTTP APIs, RPCs, CLIs) against a locally running instance with clean state: run the repo's existing usage suites first (regression check, whatever their format, e.g. Hurl files, curl scripts, collections), then author spec-first tests for the uncovered delta in the repo's suite conventions (tools like Hurl/grpcurl are examples, not requirements); every failure classified BUG / EXPECTED-CHANGE / ENV; verdict `USAGE_PROBE: PASS`/`FAIL`/`INCONCLUSIVE`. The only review gate that tests *behavior* by using the thing, not reading it. |
|
| `usage-pattern-testing` | `fs_read, fs_cat, fs_grep, fs_glob, fs_ls, fs_write, fs_patch, execute_command` | Black-box verification of a change's consumer-facing surface (HTTP APIs, RPCs, CLIs) against a locally running instance with clean state: run the repo's existing usage suites first (regression check, whatever their format, e.g. Hurl files, curl scripts, collections), then author spec-first tests for the uncovered delta in the repo's suite conventions (tools like Hurl/grpcurl are examples, not requirements); every failure classified BUG / EXPECTED-CHANGE / ENV; verdict `USAGE_PROBE: PASS`/`FAIL`/`INCONCLUSIVE`. The only review gate that tests *behavior* by using the thing, not reading it. |
|
||||||
|
| `review-misses` | none (knowledge-only) | The review-miss ledger: escaped defects (bugs that survived review and were caught later) recorded as reusable detection patterns in `~/.config/coyote/share/review-misses.md`. Review orchestrators load the skill, read the ledger, and route matching entries to reviewers as first-class checks. The feedback loop that makes the review stack converge instead of plateau. Companion macros: `record-miss` (record one escape) and `mine-review-misses` (batch-mine PR history); `whetstone` also writes entries. |
|
||||||
| `design-session` | `fs_read, fs_grep, fs_glob, fs_ls, fs_cat, fs_write` | AI-first design decomposition: ground a design doc in the actual codebase, produce a PLAN file with approach, alternatives, and a ~1-engineer-day task breakdown with measurable acceptance criteria. Includes a quality-bar round declaring `rigor` (poc/prototype/production) and `surfaces`, which calibrate every downstream review gate. |
|
| `design-session` | `fs_read, fs_grep, fs_glob, fs_ls, fs_cat, fs_write` | AI-first design decomposition: ground a design doc in the actual codebase, produce a PLAN file with approach, alternatives, and a ~1-engineer-day task breakdown with measurable acceptance criteria. Includes a quality-bar round declaring `rigor` (poc/prototype/production) and `surfaces`, which calibrate every downstream review gate. |
|
||||||
| `task-tracking` | `fs_read, fs_grep, fs_glob, fs_ls, fs_cat, fs_write, fs_patch, fs_mkdir` | File-based task tracking for plan-driven runs: TASK-NNN directory schema (`index.md` + append-only `log.md`), frontmatter lifecycle, numbering, follow-up tasks, and end-of-run consistency checks. |
|
| `task-tracking` | `fs_read, fs_grep, fs_glob, fs_ls, fs_cat, fs_write, fs_patch, fs_mkdir` | File-based task tracking for plan-driven runs: TASK-NNN directory schema (`index.md` + append-only `log.md`), frontmatter lifecycle, numbering, follow-up tasks, and end-of-run consistency checks. |
|
||||||
| `step-implementation` | `execute_command` | End-to-end protocol for executing one step of a phased implementation plan. |
|
| `step-implementation` | `execute_command` | End-to-end protocol for executing one step of a phased implementation plan. |
|
||||||
|
|||||||
Reference in new issue
Block a user