From 0fdd9018ef3e6bc4021303bf0c15530f04ad201e Mon Sep 17 00:00:00 2001 From: Alex Clarke Date: Thu, 27 Aug 2026 14:09:51 -0600 Subject: [PATCH] docs: Added documentation on the new probe agent and skill --- Agents.md | 22 +++++++++++++++++----- Skills.md | 3 ++- 2 files changed, 19 insertions(+), 6 deletions(-) diff --git a/Agents.md b/Agents.md index c117a78..355b71e 100644 --- a/Agents.md +++ b/Agents.md @@ -856,7 +856,8 @@ Coyote comes packaged with some useful built-in agents: doc, decomposes it into a quality-gated plan (`gatekeeper` self-containedness gate + `oracle` plan-review) and ~1-engineer-day task files (via the `design-session` and `task-tracking` skills, tracked on disk in a `plans/` directory), spawns one `sisyphus` per task on a single run branch with a - per-task `adversary` conformance check, and finishes with ONE draft PR — CI checks watched to green — + per-task `adversary` conformance check (plus a `probe` usage-pattern check when the task changes + consumer-facing surface), and finishes with ONE draft PR, CI checks watched to green, plus follow-up task files for every manual action the code can't do for itself. Supports opt-in worktree-based parallel task execution. * `coder`: An agent to assist you with all your coding tasks @@ -876,7 +877,8 @@ Coyote comes packaged with some useful built-in agents: * `gatekeeper`: A plan self-containedness gate (via the `plan-gatekeeping` skill). Audits a plan against the "sealed container" standard, meaning every question a context-free implementer will hit is either answered inline or delegated via a VERIFIED pointer to the exact code/docs that answer it (where infra code goes, - what DB tech, which layout to mirror, exact test commands), and returns a blocking + what DB tech, which layout to mirror, exact test commands, how to boot the service locally from a clean + state), and returns a blocking `PLAN_GATE: SEALED`/`LEAKY` verdict with the missing questions. The pre-implementation context counterpart to `plan-review` (executability) and `adversary` (post-implementation conformance). Designed to be delegated to by `architect`. @@ -885,6 +887,15 @@ Coyote comes packaged with some useful built-in agents: be delegated to by `sisyphus` whenever an unfamiliar library, API, or framework is involved. * `oracle`: An agent for high-level architecture, design decisions, complex debugging, and reviewing implementation plans before execution (via the `plan-review` skill, returning a `PLAN_REVIEW: OKAY`/`REJECT` verdict) +* `probe`: A black-box usage-pattern verifier (via the `usage-pattern-testing` skill). Where every other + review gate reads text, `probe` verifies *behavior*: it boots the changed code locally from a clean, + isolated state and exercises the consumer-facing surface (HTTP APIs, RPCs, CLIs) as a real cold-start + consumer. Runs the repo's existing usage suites first (regression check), derives expected behaviors + from the SPEC, never the implementation, so the implementer's misreadings can't become its assertions, + and authors tests for uncovered patterns in the repo's suite conventions (tools like Hurl/curl/grpcurl + are examples, not requirements). Returns a blocking `USAGE_PROBE: PASS`/`FAIL`/`INCONCLUSIVE` verdict; + its tests are deliverables adopted as permanent regression coverage. Designed to be delegated to by + `sisyphus` and `architect` when a change touches consumer-visible surface. * `report-writer`: An agent to polish research findings into clear, citation-preserving final reports * `security-reviewer`: A security analyst for code changes (via the `security-review` skill). Traces untrusted data from sources to dangerous sinks (injection, committed secrets, authn/authz gaps, path traversal, SSRF, unsafe @@ -894,10 +905,11 @@ Coyote comes packaged with some useful built-in agents: Designed to be delegated to by `sisyphus` alongside `code-reviewer` and `adversary`. * `sisyphus`: A powerhouse orchestrator agent for writing complex code and acting as a natural language interface for your codebase (similar to ClaudeCode, Gemini CLI, Codex, or OpenCode). Uses sub-agent spawning to delegate to - `explore`, `librarian`, `coder`, `oracle`, `code-reviewer`, `adversary`, `security-reviewer`, and `step-runner`. On - non-trivial work it runs three complementary independent review passes: `code-reviewer` for code quality, + `explore`, `librarian`, `coder`, `oracle`, `code-reviewer`, `adversary`, `security-reviewer`, `probe`, and `step-runner`. On + non-trivial work it runs up to four complementary independent review passes: `code-reviewer` for code quality, `adversary` for plan conformance, and `security-reviewer` for exploitability when the change touches attack - surface. Also supports plan-driven workflows: authoring phased implementation plans (via the `plan-authoring` + surface, and `probe` for black-box usage-pattern verification when the change touches + consumer-facing surface. Also supports plan-driven workflows: authoring phased implementation plans (via the `plan-authoring` skill), having `oracle` review them, and executing them one reviewed step at a time. * `sql`: A universal SQL agent that enables you to talk to any relational database in natural language * `step-runner`: A graph-based agent that executes ONE step of a phased implementation plan (a `plans/` repo authored diff --git a/Skills.md b/Skills.md index 11d1da2..7363b1b 100644 --- a/Skills.md +++ b/Skills.md @@ -379,7 +379,7 @@ whitelist — but their processes stay cached for fast re-load. # Built-in Skills -Coyote ships with eighteen built-in skills, installed automatically on first run: +Coyote ships with nineteen built-in skills, installed automatically on first run: | Skill | Granted tools / MCP servers | Purpose | |-----------------------|--------------------------------------------------------------------------|------------------------------------------------------------------------------| @@ -396,6 +396,7 @@ Coyote ships with eighteen built-in skills, installed automatically on first run | `plan-review` | `fs_read, fs_grep, fs_glob, fs_ls, fs_cat` | Adversarial review of implementation plans against executability, verifiability, and completeness standards. | | `adversarial-review` | `fs_read, fs_grep, fs_glob, fs_cat, fs_ls` | Adversarial plan-conformance review of an *implementation* against the plan it should satisfy (skipped criteria, scope drift, interface substitution); verdict `CONFORMS`/`DIVERGES`. Complements `code-review` (quality) and is the post-implementation counterpart to `plan-review`. | | `plan-gatekeeping` | `fs_read, fs_grep, fs_glob, fs_cat, fs_ls` | Gatekeep a plan for *self-containedness* before finalization: the "sealed container" test — every implementer question answered inline or via a verified pointer; verdict `PLAN_GATE: SEALED`/`LEAKY` with the missing questions. | +| `usage-pattern-testing` | `fs_read, fs_cat, fs_grep, fs_glob, fs_ls, fs_write, fs_patch, execute_command` | Black-box verification of a change's consumer-facing surface (HTTP APIs, RPCs, CLIs) against a locally running instance with clean state: run the repo's existing usage suites first (regression check, whatever their format, e.g. Hurl files, curl scripts, collections), then author spec-first tests for the uncovered delta in the repo's suite conventions (tools like Hurl/grpcurl are examples, not requirements); every failure classified BUG / EXPECTED-CHANGE / ENV; verdict `USAGE_PROBE: PASS`/`FAIL`/`INCONCLUSIVE`. The only review gate that tests *behavior* by using the thing, not reading it. | | `design-session` | `fs_read, fs_grep, fs_glob, fs_ls, fs_cat, fs_write` | AI-first design decomposition: ground a design doc in the actual codebase, produce a PLAN file with approach, alternatives, and a ~1-engineer-day task breakdown with measurable acceptance criteria. | | `task-tracking` | `fs_read, fs_grep, fs_glob, fs_ls, fs_cat, fs_write, fs_patch, fs_mkdir` | File-based task tracking for plan-driven runs: TASK-NNN directory schema (`index.md` + append-only `log.md`), frontmatter lifecycle, numbering, follow-up tasks, and end-of-run consistency checks. | | `step-implementation` | `execute_command` | End-to-end protocol for executing one step of a phased implementation plan. |