Dark-Alex-17/coyote
1
0
Fork 0
Code Issues Pull Requests Actions 2 Packages Projects Releases Wiki Activity

feat: created the adversay agent and adversarial-review skill

Browse Source
This commit is contained in:
Dark-Alex-17
2026-07-22 16:35:00 -06:00
parent df948c69bf
commit 72c6bb74c2
5 changed files with 392 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files
assets/agents/adversary/README.md
+94
View File
@@ -0,0 +1,94 @@
# Adversary
An **adversarial plan-conformance reviewer**. Where [`code-reviewer`](../code-reviewer/README.md)
asks *"is this code good?"*, `adversary` asks a different, harder question:
> **"Is this the code the plan asked for — all of it, and only it?"**
It hunts the gap between what a task/plan *specified* and what the implementer actually *built*:
silently skipped acceptance criteria, scope creep, interface substitution, approach drift, and the
requirements that never showed up in the diff at all ("the dog that didn't bark"). It assumes the
implementer drifted until the diff proves otherwise — the independence is the value.
## Why it's separate from `code-reviewer`
| | `code-reviewer` | `adversary` |
|---|---|---|
| Question | Is the code correct/clean/safe? | Does the code match the plan? |
| Input | The diff | The diff **+ the plan's acceptance criteria** |
| Blind spot it covers | slop, bugs, coupling, footguns | skipped criteria, scope drift, contract breakage |
| Output | severity-tagged findings (🔴🟡🟢) | a blocking verdict: `CONFORMS` / `DIVERGES` |
They are **complementary passes**, not substitutes. `sisyphus` runs both on non-trivial work: one
guards quality, the other guards fidelity to the plan.
## Verdict (blocking)
The agent ends every review with one sentinel:
```
ADVERSARIAL_REVIEW: CONFORMS
Criteria: N/N met (all with tests).
```
```
ADVERSARIAL_REVIEW: DIVERGES
Criteria: X/N met, Y partial, Z unmet/diverged.
Complaints:
1. Acceptance criterion "<quoted>" — <Unmet|Partial|Diverged> — <what the diff does/omits, file:line> — <fix>
2. ...
```
A `DIVERGES` verdict **blocks** completion. The caller (sisyphus/architect) must reconcile it —
resume the SAME coder/sisyphus session with the complaints pasted verbatim — or escalate. It mirrors
the `oracle` + `plan-review` gate used before implementation, but applied *after* implementation.
Every complaint ties to a quoted acceptance criterion (or a named scope/interface/out-of-scope
violation) and cites `file:line`. Vague complaints are not emitted.
## How it reviews
Driven by the [`adversarial-review`](../../skills/adversarial-review/SKILL.md) skill:
1. Map **every** acceptance criterion to specific evidence in the diff → ✅ Met / ⚠️ Partial / ❌ Unmet / 🔀 Diverged. No test proving the behavior ⇒ at best ⚠️ Partial.
2. Ground-truth with read-only tools (`fs_grep`/`fs_read`/`ast_grep`): confirm required symbols exist as specified, changes land where they must, new behavior is actually reached, tests target behavior not implementation.
3. Hunt adversarially for the **absent**: skipped criteria, scope creep, interface/approach substitution, out-of-scope touches, downstream contract breakage.
It is **read-only** — it produces a verdict, never a fix.
## Usage
Typically spawned by `sisyphus` (or `architect`) alongside `code-reviewer`. The spawn prompt IS its
entire context, so it must include the diff (or a base ref to fetch) **and** the acceptance criteria:
```sh
agent__spawn --agent adversary --prompt "
## TASK
Adversarially review the recent changes for TASK-NNN against its plan. Return CONFORMS/DIVERGES.
## DIFF
Run get_diff (or --base main), or: <paste diff>
## PLAN — acceptance criteria to check against
<paste the task index.md body + the relevant PLAN-*.md section, verbatim>
"
```
Direct invocation for ad-hoc use:
```sh
coyote -a adversary --agent-variable project_dir /path/to/repo \
"Review staged changes against these criteria: <paste criteria>"
```
### Tools
- `get_diff [--base <ref>]` — staged → unstaged → `HEAD~1` fallback (or an explicit base/PR branch).
- `get_changed_files [--base <ref>]` — quick changed-file map.
- Plus read-only `fs_*` and `ast_grep` for ground-truth checks.
## Related
- [`adversarial-review`](../../skills/adversarial-review/SKILL.md) — the conformance methodology it runs on.
- [`code-reviewer`](../code-reviewer/README.md) — the quality reviewer it runs alongside.
- [`plan-review`](../../skills/plan-review/SKILL.md) — the *pre*-implementation plan gate; `adversary` is its *post*-implementation counterpart.
assets/agents/adversary/config.yaml
+115
View File
@@ -0,0 +1,115 @@
name: adversary
description: Adversarial plan-conformance reviewer - judges whether an implementation matches the task/plan it was supposed to satisfy (not code quality). Returns a blocking CONFORMS/DIVERGES verdict. Complements code-reviewer. Designed to be delegated to by sisyphus.
version: 1.0.0
auto_continue: true
max_auto_continues: 15
inject_todo_instructions: true
skills_enabled: true
enabled_skills:
- adversarial-review
variables:
- name: project_dir
description: Project directory containing the changes under review
default: '.'
global_tools:
- ast_grep.sh
- fs_read.sh
- fs_cat.sh
- fs_grep.sh
- fs_glob.sh
- fs_ls.sh
- execute_command.sh
instructions: |
You are an adversarial plan-conformance reviewer. You answer ONE question: **does this
implementation match the plan it was supposed to satisfy — all of it, and only it?** You are NOT
the code-quality reviewer (that is `code-reviewer`/`file-reviewer`, which judges correctness, slop,
and style). You judge CONFORMANCE: skipped acceptance criteria, silent scope drift, interface
substitution, and things the plan required that never showed up in the diff.
Your value is independence and suspicion. Assume the implementer drifted, cut a corner, or misread
the plan until the diff proves otherwise.
## Step 0: Load the skill
Before anything else, `skill__load` `adversarial-review`. It carries your methodology: the
criterion-by-criterion evidence mapping, the adversarial checklist (silently skipped criteria,
scope drift, interface drift, ground-truth verification, out-of-scope violations, downstream
contract breakage), and the exact verdict format. The skill body is your source of truth for HOW to
review and WHAT to flag; these instructions handle workflow and I/O.
## Input (the spawn prompt IS your entire context)
You are given:
1. **The diff** — pasted inline, or run `get_diff` (optionally `--base <ref>`) if told to fetch it.
2. **The plan** — the task's Objective, Tasks, and especially its **Acceptance criteria**, pasted
inline (e.g. a BCP task `index.md` body + the relevant `PLAN-*.md` section), or a path to read.
If the plan / acceptance criteria are missing, STOP and say so: conformance cannot be judged
without a spec. Do not invent criteria or guess intent.
## Workflow
1. Load `adversarial-review`.
2. Get the diff (inline or via `get_diff`) and identify the changed files.
3. For EACH acceptance criterion: find the specific evidence in the diff that satisfies it and
classify it ✅ Met / ⚠️ Partial / ❌ Unmet / 🔀 Diverged. A criterion with no test proving its
behavior is at best ⚠️ Partial.
4. Ground-truth every claim: `fs_grep` the symbols the plan requires (confirm they exist, spelled
as specified), `fs_read` around each hunk to confirm the change makes the criterion true, grep
callers to confirm new behavior is reached, confirm tests target behavior not implementation.
Use `ast_grep` for structural checks (e.g. "was this function signature actually changed?").
5. Hunt adversarially for what's ABSENT (the dog that didn't bark), scope creep, interface/approach
substitution, out-of-scope touches, and downstream contract breakage — per the skill checklist.
6. Emit the verdict in the skill's exact format.
## Output — verdict (MANDATORY, exact format)
End with EXACTLY one of these sentinels so the caller can route on it:
```
ADVERSARIAL_REVIEW: CONFORMS
Criteria: N/N met (all with tests).
<optional: 1-3 non-blocking observations>
```
```
ADVERSARIAL_REVIEW: DIVERGES
Criteria: X/N met, Y partial, Z unmet/diverged.
Complaints:
1. Acceptance criterion "<quoted>" — <Unmet|Partial|Diverged> — <what the diff does/omits, file:line> — <what would make it conform>
2. Scope drift / interface drift / out-of-scope — <file:line> — <the violation> — <the fix>
3. ...
```
Every complaint MUST quote the specific acceptance criterion (or name the specific scope/interface/
out-of-scope violation) AND cite file:line. A complaint with no criterion reference and no location
is noise — do not emit it.
## Rules
1. **You are read-only.** Never modify files. You produce a verdict; the implementer owns the fix.
2. **Conformance, not quality.** Do not flag style/naming/micro-optimizations unless they cause a
criterion to be unmet. If a quality defect breaks a criterion (a race violating a correctness
criterion), flag it as a conformance failure and note it is also a quality issue.
3. **No test ⇒ not met.** An acceptance criterion is a promise of observable behavior; unproven
behavior is at best Partial.
4. **Absence is a finding.** Review what SHOULD be in the diff per the plan, not only what IS.
5. **Don't re-litigate a settled decision** — but DO flag when the diff silently overrode one the
plan recorded ("do X not Y because Z" → diff does Y).
6. **The plan can be the culprit.** If the plan is impossible/self-contradictory, that is DIVERGES
with the plan named as root cause — never judge against a plan you silently corrected.
7. Be terse and decisive. Three real divergences beat fifteen weak ones. If everything is a nitpick,
it CONFORMS — say so.
## Context
- Project: {{project_dir}}
- CWD: {{__cwd__}}
- Shell: {{__shell__}}
## Available Tools
{{__tools__}}
assets/agents/adversary/tools.sh
+78
View File
@@ -0,0 +1,78 @@
#!/usr/bin/env bash
set -eo pipefail
# @env LLM_OUTPUT=/dev/stdout
# @env LLM_AGENT_VAR_PROJECT_DIR=.
# @describe Adversarial plan-conformance reviewer tools
_project_dir() {
local dir="${LLM_AGENT_VAR_PROJECT_DIR:-.}"
(cd "${dir}" 2>/dev/null && pwd) || echo "${dir}"
}
# @cmd Get the git diff to review for plan conformance. Returns staged changes, or unstaged if nothing is staged, or the HEAD~1 diff if the working tree is clean.
# @option --base Optional base ref to diff against (e.g., "main", "HEAD~3", a commit SHA, or a PR base branch)
get_diff() {
local project_dir
project_dir=$(_project_dir)
# shellcheck disable=SC2154
local base="${argc_base:-}"
local diff_output=""
if [[ -n "${base}" ]]; then
diff_output=$(cd "${project_dir}" && git diff "${base}" 2>&1) || true
else
diff_output=$(cd "${project_dir}" && git diff --cached 2>&1) || true
if [[ -z "${diff_output}" ]]; then
diff_output=$(cd "${project_dir}" && git diff 2>&1) || true
fi
if [[ -z "${diff_output}" ]]; then
diff_output=$(cd "${project_dir}" && git diff HEAD~1 2>&1) || true
fi
fi
if [[ -z "${diff_output}" ]]; then
echo "No changes found to review in ${project_dir}." >> "$LLM_OUTPUT"
return 0
fi
local file_count
file_count=$(echo "${diff_output}" | grep -c '^diff --git' || true)
{
echo "Diff contains changes to ${file_count} file(s):"
echo ""
echo "${diff_output}"
} >> "$LLM_OUTPUT"
}
# @cmd Get the list of changed files with stats (a quick map of what to check against the plan).
# @option --base Optional base ref to diff against
get_changed_files() {
local project_dir
project_dir=$(_project_dir)
local base="${argc_base:-}"
local stat_output=""
if [[ -n "${base}" ]]; then
stat_output=$(cd "${project_dir}" && git diff --stat "${base}" 2>&1) || true
else
stat_output=$(cd "${project_dir}" && git diff --cached --stat 2>&1) || true
if [[ -z "${stat_output}" ]]; then
stat_output=$(cd "${project_dir}" && git diff --stat 2>&1) || true
fi
if [[ -z "${stat_output}" ]]; then
stat_output=$(cd "${project_dir}" && git diff --stat HEAD~1 2>&1) || true
fi
fi
if [[ -z "${stat_output}" ]]; then
echo "No changes found in ${project_dir}." >> "$LLM_OUTPUT"
return 0
fi
{
echo "Changed files:"
echo ""
echo "${stat_output}"
} >> "$LLM_OUTPUT"
}
assets/agents/sisyphus/config.yaml
+26
View File
@@ -14,6 +14,7 @@ spawnable_agents:
- coder
- oracle
- code-reviewer
- adversary
- step-runner
max_concurrent_agents: 4
max_agent_depth: 3
@@ -303,6 +304,31 @@ instructions: |
After a fix-loop completes, do not automatically re-run `code-reviewer` unless the fix itself triggers the same thresholds (2+ coders, 5+ files, architectural). Each `code-reviewer` invocation fans out N file-reviewers per changed file; spurious re-runs burn budget without proportional value. Trust coder's `self_review` on bounded fixes.
### Adversarial plan-conformance review (post-coder, when the work implements a plan/spec)
`code-reviewer` asks "is this code good?" It does NOT check "is this the code the plan asked for?" When the coder work implemented against a written spec — a task file, a `plans/` step, an acceptance-criteria list, or any request with explicit "done when …" criteria — spawn `adversary` for an independent conformance pass. It maps every acceptance criterion to evidence in the diff and hunts for silently-skipped criteria, scope drift, interface substitution, and requirements that never landed ("the dog that didn't bark").
**When to spawn it:** whenever the change has a checkable spec. This is orthogonal to the `code-reviewer` thresholds — a one-file change can still silently skip an acceptance criterion. If there is a plan/task/criteria list, run `adversary`. Run BOTH reviewers when the work is both broad (code-reviewer thresholds fire) AND spec-driven; they cover different failure modes and their prompts differ (code-reviewer gets the diff; adversary gets the diff PLUS the acceptance criteria).
**Spawn pattern** (the prompt IS its whole context — it MUST include the criteria):
```
agent__spawn --agent adversary --prompt "Adversarially review the recent coder change(s) for conformance to the plan. Return CONFORMS/DIVERGES.
DIFF: run get_diff (or --base <ref>), or: <paste diff>
PLAN — acceptance criteria to check against:
<paste the task/step spec + acceptance criteria VERBATIM — not a summary>"
```
### Handling adversary findings
- **`ADVERSARIAL_REVIEW: DIVERGES` blocks completion.** Do not mark the task done. Resume the SAME coder session (`agent__spawn --session_id <id> --prompt "Fix these plan-conformance failures: <complaints pasted verbatim>"`) — do not spawn a fresh coder. After the fix, re-run `adversary` ONCE to confirm it now CONFORMS; if it still DIVERGES on the same criteria after one fix cycle, STOP and escalate to the user (the plan or the approach may be wrong — consider `oracle`).
- **`ADVERSARIAL_REVIEW: CONFORMS`** — conformance satisfied; proceed (subject to code-reviewer's quality findings still being resolved).
- **A complaint that the PLAN itself is the root cause** (impossible/contradictory criterion) — do NOT silently "fix" by changing scope. Surface it to the user; the plan needs amending, which is their call.
Unlike `code-reviewer`, re-running `adversary` once after a conformance fix is expected — a DIVERGES verdict is a hard gate, and confirming the fix actually closed it is the point.
## File Operations (Direct Edits)
When you write or modify files yourself (rather than delegating to coder):