Compare commits

..
Author SHA1 Message Date
Dark-Alex-17 ae96a7e031 fmt: applied formatting 2026-08-12 19:35:27 -06:00
Dark-Alex-17 2658ca776e feat: Installed duckdb into the coyote image
CI / All (ubuntu-latest) (push) Failing after 29s
CI / All (macos-latest) (push) Has been cancelled
CI / All (windows-latest) (push) Has been cancelled
2026-08-12 19:29:26 -06:00
Dark-Alex-17 f8682102a0 docs: Added duckdb prerequisite
CI / All (ubuntu-latest) (push) Failing after 29s
CI / All (macos-latest) (push) Has been cancelled
CI / All (windows-latest) (push) Has been cancelled
2026-08-12 19:07:31 -06:00
Dark-Alex-17 3fa0f5c428 feat: Support managing MCP servers from the CLI directly
CI / All (ubuntu-latest) (push) Failing after 29s
CI / All (macos-latest) (push) Has been cancelled
CI / All (windows-latest) (push) Has been cancelled
2026-08-12 17:55:33 -06:00
Dark-Alex-17 68135b97d1 feat: append new built-in rag__query function to RAG contexts to allow further querying by LLMs
CI / All (ubuntu-latest) (push) Failing after 29s
CI / All (macos-latest) (push) Has been cancelled
CI / All (windows-latest) (push) Has been cancelled
2026-08-12 16:25:50 -06:00
Dark-Alex-17 b87a3460c4 fix: detect duplicate tool call IDs client-side before sending to Claude
CI / All (ubuntu-latest) (push) Failing after 29s
CI / All (macos-latest) (push) Has been cancelled
CI / All (windows-latest) (push) Has been cancelled
2026-08-12 13:33:24 -06:00
Dark-Alex-17 cb23da6490 feat: support static file bundling with sbx-mixins
CI / All (ubuntu-latest) (push) Failing after 29s
CI / All (macos-latest) (push) Has been cancelled
CI / All (windows-latest) (push) Has been cancelled
2026-08-12 13:18:38 -06:00
Alex Clarke 2a40a5a81d Merge pull request #14 from Dark-Alex-17/feat/rag-driver-abstraction-v3
CI / All (ubuntu-latest) (push) Failing after 31s
CI / All (macos-latest) (push) Has been cancelled
CI / All (windows-latest) (push) Has been cancelled
feat(rag): pluggable RagProvider abstraction with DuckDB and Qdrant drivers
2026-08-12 12:45:12 -06:00
Dark-Alex-17 ebba976a27 fmt: applied formatting 2026-08-12 12:07:38 -06:00
Dark-Alex-17 c84f9522e9 feat(rag): offer the storage driver when an agent initializes its RAG
Agent startup and graph rag nodes both run an interactive wizard when their
knowledge base has not been built, but neither offered the driver choice that
interactive named-RAG creation has, so both silently produced a yaml store.

A plain agent was the worse of the two: AgentConfig carries only documents, so
there was no way to get a duckdb RAG for one, interactively or declaratively. A
graph node could at least declare driver: in the workflow.

Agent startup now passes prompt_for_driver, and a rag node whose wizard runs is
asked too. The prompt is skipped when the node already declares a driver, and
sits inside the not-fully-specified branch after the non-interactive bail, so
declarative workflows and headless runs are unchanged. Temp RAGs still pass
false: they are deleted on the next run, so a persistent store would only leave
a sidecar behind.

The prompt moves to select_rag_driver rather than being duplicated.
2026-08-12 12:02:15 -06:00
Dark-Alex-17 81ed769f8a fix(rag): warn when a duckdb store is empty but files are indexed
A duckdb RAG is two files. The .yaml deliberately carries no vectors, and
open() runs CREATE TABLE IF NOT EXISTS, so a .yaml copied or synced without its
.duckdb sidecar produces a fresh empty store, hydrates to nothing, and answers
every query with nothing while .info rag still lists every indexed file.

Neither existing guard catches it: the anti-wipe check in rebuild_indexes needs
existing > 0, and the mandatory ? on hydration needs a genuine error, while an
absent store is the same Ok(empty) as a RAG with nothing indexed yet.

Warn rather than bail, so a store deleted on purpose still loads and can be
rebuilt.
2026-08-12 11:26:16 -06:00
Dark-Alex-17 6f7defe25f style: removed redundant comment 2026-08-12 10:56:34 -06:00
Dark-Alex-17 b837f82d7e fix(rag): keep a local Qdrant off an ambient proxy
Reverts the global proxy rework in 54685be and narrows it to the provider.

That commit took over proxy detection for every client in order to exempt
loopback and private ranges. Too broad: reqwest's detection also reads macOS
System Settings and the Windows registry behind its system-proxy feature, which
sits in its default set. Coyote disables default features today, so hand-rolling
the environment lookup happened to match — but re-enabling defaults later would
silently restore that support for main and not for the hand-rolled version. It
also made an explicitly configured proxy skip local hosts, which nobody asked
for: a proxy named for a LAN endpoint should be used.

build_client and utils are byte-identical to main again. The bypass now lives in
QdrantProvider::make_client, which is the only place that knows the target host,
and applies solely when that host is loopback, link-local, private or .local. A
public or cloud-hosted store keeps whatever the environment configures.

Also drops apply_proxy: with build_client reverted there was one caller left, and
set_proxy already covers it.

Both #[ignore]d live tests still pass against a Qdrant on loopback while an
ambient proxy that rejects it is in force.
2026-08-12 10:52:31 -06:00
Dark-Alex-17 54685be9a2 fix: keep loopback and LAN traffic off an ambient proxy
Pre-existing on main, not introduced by the driver work, but it makes a local
RAG backend unusable so it belongs with this change.

build_client only called set_proxy when a client had configured one of its own.
With nothing configured, reqwest's own detection applied, which sends every
request through a *_PROXY variable including ones bound for 127.0.0.1 or a LAN
address. A proxy cannot usefully forward those, and anything that intercepts
proxied traffic answers on behalf of a service that is running perfectly well,
so the error names the proxy rather than the store and reads as a Coyote fault.

Concretely, an installed Socket Firewall exports HTTP_PROXY to the processes it
wraps and rejects hosts outside its allow list. That turned a healthy Ollama on
the LAN into 'error decoding response body: expected value at line 2 column 1' —
its HTML refusal page parsed as JSON — and a loopback Qdrant into an HTTP 405.

Proxy handling is now always applied and always exempts loopback and private
ranges, with NO_PROXY merged in since replacing reqwest's detection also
replaces its handling of that variable. HTTP_PROXY and HTTPS_PROXY are kept
separate because they are allowed to differ. An explicitly configured proxy
still wins, and '-' still means none.

This also supersedes the unconditional no_proxy() added to the Qdrant client in
af9622d: that made it the only client to ignore a proxy outright, on a
justification I got wrong. It now shares this path, so a remote store behind a
real proxy keeps working.
2026-08-12 10:33:03 -06:00
Dark-Alex-17 4dd6e794b2 docs: removed redundant comment 2026-08-11 22:19:14 -06:00
Dark-Alex-17 af9622d31c fix(rag): stop routing Qdrant requests through an ambient proxy
make_client used a bare reqwest builder, which honours whatever proxy the
environment advertises. That made it the only HTTP client in Coyote to do so:
utils::set_proxy discards ambient settings and applies only Coyote's configured
proxy, and every other client goes through it.

The symptom is that a perfectly healthy Qdrant is unreachable and the error
belongs to the interposing proxy, not the store, so it reads as a Coyote or
Qdrant fault. Locally an installed Socket Firewall answered `.rag attach`
against 127.0.0.1:6333 with an HTML 'Connection Required' page and HTTP 405.

Both #[ignore]d live tests now pass against a real Qdrant; they failed with that
same 405 before this change, which is the first time either has run green.

A remote store that genuinely needs Coyote's configured proxy is a follow-up:
that means threading the proxy config into the provider.
2026-08-11 22:15:04 -06:00
Dark-Alex-17 6f586bd535 style: cleanup 2026-08-11 22:07:35 -06:00
Dark-Alex-17 78740db170 chore: ignore the .coyote workspace directory
It holds generated workspace state and should never be committed.
2026-08-11 22:03:45 -06:00
Dark-Alex-17 64d594f4ee refactor(rag): discover driver_config secrets by grammar, not field name
Sandbox provisioning only ever looked at driver_config["api_key"], so a driver
whose credential is called anything else would have been silently unprovisioned
inside a sandbox. It now scans every driver_config value and treats any that is
a secret placeholder as a credential, which is the same rule resolve_driver_config
already used at point of use.

The first one binds to the RAG's own service id, which is what the generated
mixin declares; any others register under their own names, as MCP secrets do.
The mixin still carries a single credential entry, so a driver needing two bound
secrets remains a follow-up.

Also drops the placeholder parser added in 74bc613. crate::vault::SECRET_RE is
already the canonical definition and was already imported here, so that was a
third implementation of the same grammar. Requiring the whole value to match is
what keeps a literal key from being read as a secret name and printed.

The api_key check is gone from RagData::validate: a generic config validator
should not know a provider's field names.
2026-08-11 22:03:45 -06:00
Dark-Alex-17 1322d73c7b style: further cleanup 2026-08-11 21:50:46 -06:00
Dark-Alex-17 de91ffa517 fix(rag): treat a zero min_score as no floor on Qdrant searches
parse_search_hits filtered on score > min_score, and the only caller passes
0.0. Qdrant Euclid collections score by negative distance, so every hit was
dropped and an attached Euclid collection returned nothing at all, silently.

This is the same trap the surrounding code already documents: score_threshold
is deliberately not sent because it is metric-aware and a 0.0 floor filters
everything out on Euclid. The local filter then reproduced it exactly. Only a
positive floor is now treated as a floor.
2026-08-11 21:07:20 -06:00
Dark-Alex-17 74bc613d94 fix(rag): address Copilot review findings on the driver abstraction
Five review comments, all real:

- hybrid_search ran its vector and keyword legs sequentially after the
  provider refactor; main ran them under tokio::join!. Restores the
  concurrency while keeping the degrade-on-error keyword behaviour, so a
  remote provider no longer pays two serial round trips per query.

- inject_rag_secrets derived a vault secret name by trimming braces, which
  leaves a literal key untouched. A RAG holding a plaintext api_key therefore
  looked the secret up by its own value and printed it to stderr on failure.
  Parsing is now strict and a non-placeholder is skipped with a warning that
  names no credential.

- validate() now refuses a driver_config.api_key that is not a {{NAME}}
  placeholder, so a plaintext key cannot reach the RAG YAML at all.

- Rag::create's catch-all arm treated any unrecognised driver as yaml. A typo
  built a yaml store, paid to embed the corpus, persisted the bad driver and
  only failed on the next run. Unknown drivers now fail immediately.

- The qdrant arm's error was written for a developer; it now tells the user
  that only attached collections are readable and points at .rag attach.
2026-08-11 21:04:21 -06:00
Dark-Alex-17 6d0a5550fe style: Removed some redundant comments 2026-08-11 20:56:51 -06:00
Dark-Alex-17 7b1c0342b4 fix(rag): delete the DuckDB write-ahead log alongside the store
Deleting a RAG removed its .duckdb file but left the sibling .duckdb.wal
behind. DuckDB only removes that log on a clean close, so any RAG whose
process was killed left one on disk, and creating a new RAG under the same
name let it inherit a write-ahead log describing someone else's data.

The test helper already cleaned the log up after itself, which is why no
test noticed the production path did not.
2026-08-11 16:55:19 -06:00
Dark-Alex-17 d6c114fe58 feat: simplified the duckdb selection prompt 2026-08-11 16:50:50 -06:00
Dark-Alex-17 912e00a627 docs(rag): correct the duckdb concurrency note in the driver prompt
The driver picker still told users a duckdb RAG can only be open in one Coyote
process at a time. That stopped being true once the store began opening
read-only for queries: any number of processes can now query it concurrently.

The restriction that remains is narrower and only bites while writing, so the
prompt now states that instead — several processes can query at once, but an
ingest or rebuild locks the others out until it finishes.
2026-08-11 16:38:55 -06:00
Dark-Alex-17 e006e29ff1 feat(rag): let several Coyote processes query one duckdb RAG at once
The DuckDB store was always opened read-write, which takes an exclusive file
lock, so a second Coyote process could not even read the RAG. Querying does not
write, and DuckDB permits many concurrent readers as long as no writer is
attached, so the store is now opened read-only whenever it already carries a
complete schema.

Creating or initializing the store still writes, as does rebuilding, so those
paths take the exclusive handle. The rebuild path upgrades a read-only
connection in place, which every clone of the handle observes because the mode
lives behind the shared mutex rather than beside it. Extension loads and the
HNSW persistence setting are per-connection and are re-established on the
upgraded connection.

An upgrade that loses the race for the write lock reports that another process
holds the RAG and that nothing was written, then reopens read-only so the
session can keep querying. A read-only handle also refuses writes outright, so a
missed upgrade cannot silently discard an ingest.
2026-08-11 14:58:18 -06:00
Dark-Alex-17 c0067d387c refactor(rag): drop the hardcoded embedding model hint from attach
The attach wizard mapped a collection's vector dimension to a hardcoded list of
model names and printed them as likely candidates. The list was never checked
against the models the user actually has configured, so it could recommend a
model they cannot select, and one entry was a parenthetical note rather than a
model id and so could never match anything. Any list like this rots as models
are released.

The dimension itself comes from the server and is worth stating, so it is still
printed, as is the warning that a mismatched embedding model returns bad
results. Deriving real candidates would need a dimension recorded against each
configured model, which the model config does not carry today.
2026-08-11 14:06:57 -06:00
Dark-Alex-17 118c346345 feat(rag): support string and UUID Qdrant point IDs
Point ids were read with as_u64() inside a filter_map, so a string id was
silently dropped and a UUID-keyed collection returned zero hits with no error.
The attach wizard therefore refused such collections and told the user to
rebuild with integer ids, which defeats the purpose of attaching to a
collection someone else already built. LangChain, a common way to populate
Qdrant, uses UUIDs by default.

The integer id was never load-bearing for this driver. DocumentId is a packed
(file, chunk) pair used positionally by the local drivers, but an attached RAG
holds no local files or vectors and every positional consumer already returns
early on it, so the id only has to survive the round trip from search back to
the content fetch. Ids that cannot make that trip as a u64 are interned behind
a synthetic handle and restored when the fetch is issued, leaving collections
that already use integer ids on exactly the path they used before.
2026-08-11 14:05:38 -06:00
Dark-Alex-17 dc677a2529 fix(sandbox): discover agent-scoped RAG mixin sidecars
An agent-scoped RAG writes its config to <data>/agents/<agent>/<rag>.yaml, so
its sbx mixin sidecar lands beside it as <rag>.sbx-mixin.yaml. Discovery scanned
the agents directory only for a file named exactly sbx-mixin.yaml, and scanned
for suffixed sidecars only in the top-level rags directory, so a RAG attached
while an agent was active contributed no network allow rule and no credential to
the sandbox. The failure was silent: the sandbox launched and the RAG was simply
unreachable from inside it.

The two collectors differed only in the filename shape they matched, so they are
now one scan that takes the set of layouts to look for. The agents directory
asks for both its own sbx-mixin.yaml and the suffixed sidecars one level in,
which is the shape that was missing. Discovery order is unchanged, and it is
load-bearing: each mixin becomes a --kit in list order and later ones layer over
earlier ones, so the workspace mixin must stay last.
2026-08-11 14:05:31 -06:00
Dark-Alex-17 5e2b9c98ad fix(rag): stop the attach wizard from silently accepting an empty collection
`sample_point_id` returns `None` for a collection with no points, so the
UUID guard's `if let Some(..)` fell straight through and the wizard attached
happily. The result is a RAG that answers every query with zero hits and
never says why.

Sample once, then check for emptiness explicitly. This warns and asks rather
than hard-failing: an empty collection is not necessarily a mistake, since
another tool may be about to populate it, and none of the wizard's remaining
probes can distinguish that from a misconfiguration. The confirmation
defaults to "no" so it cannot be walked past by accident, and `attach`
already refuses to run non-interactively, so no unattended path reaches it.
2026-08-11 13:55:59 -06:00
Dark-Alex-17 c458ca93a9 feat(rag): create the API key secret inline in the attach wizard
The wizard hard-errored with "Secret 'X' not found in vault. Run
`coyote --add-secret X` first.", throwing away every answer the user had
already given it. Offer to create the secret in place instead, deferring to
`Vault::add_secret` for the masked prompt, the provider write and the
confirmation line, then read it back.

Only a genuine `SecretError::NotFound` triggers the offer. An auth failure,
a provider outage, or the vault being disabled inside a sandbox all
propagate with their own message, because prompting for a value that cannot
be stored would fail one step later and bury the real cause. Declining the
offer fails with both ways out spelled: add the secret up front, or answer
"no" to the API-key question.
2026-08-11 13:50:33 -06:00
Dark-Alex-17 860566bf50 feat: let workflow rag nodes select a RAG driver
`RagNode` gains an optional `driver`, forwarded into `RagInitConfig` so a
graph node can build its knowledge base on duckdb instead of yaml. Nodes
that name no driver forward `None`, which still resolves to yaml, so
existing workflows are unaffected.

An unknown driver is rejected up front rather than at construction time.
`Rag::create` dispatches unknown drivers to its yaml catch-all, so a typo
would otherwise embed every document and persist the bogus string, after
which every subsequent load fails validation and the agent cannot start.
The check asks `RagData::validate()` through a probe value instead of
restating the list of valid drivers, so the two cannot drift.
2026-08-11 13:46:59 -06:00
Dark-Alex-17 7f90710427 refactor(rag): interpolate every driver_config value, not just api_key
Only `driver_config["api_key"]` was interpolated, so any credential-bearing
driver field added later would have shipped its raw `{{PLACEHOLDER}}` to the
server. Resolve every value instead, via `resolve_driver_config`.

Resolution still happens into a function-local copy and never touches
`RagData`: `save()` serializes `self.data` and is called by `.set rag_top_k`
and friends, so a resolved credential parked there would be written to the
RAG's YAML in plaintext. The literal `{{NAME}}` also has to survive on disk
because sandbox credential provisioning parses it back out to learn which
vault secret to bind. Scope stays `driver_config` deliberately: the rest of
a RAG file is ingested document text, where `{{...}}` is ordinary content.

Covered by a test that saves after a load and asserts the placeholder, not
the secret, is what reaches the file.
2026-08-11 13:46:32 -06:00
Dark-Alex-17 93a934439b fix(rag): fail loudly when a RAG's vault secret is missing
`interpolate_secrets` does not error on a secret the vault cannot resolve:
it substitutes the empty string and returns the name in its second tuple
element. `load_async` discarded that vec, so a typo'd or deleted vault
secret produced `api_key = ""` and an unexplained 401 from Qdrant.

Bail instead, naming the RAG and the missing secrets, matching what global
config loading already does.
2026-08-11 13:44:00 -06:00
Dark-Alex-17 ecda258d3a style: Cleaned up some minor styling issues 2026-08-11 13:04:27 -06:00
Dark-Alex-17 3e598065f8 fix(rag): serialize DuckDB extension installs to stop a Windows race
`ensure_extension` fell back to `INSTALL` whenever `LOAD` failed. With a cold
extension cache every thread's `LOAD` fails at once, so every thread ran
`INSTALL` concurrently for the same extension. DuckDB installs by downloading
to a temp file and then MOVING it into `~/.duckdb/extensions/...`; POSIX allows
replacing a file other handles hold open, so Linux and macOS survived, but
Windows rejects that move with "Access is denied" and the losing threads failed.

Guard the install step with a process-global mutex and re-check `LOAD` after
acquiring it. The re-check is what bounds the work to a single install: without
it every thread queued behind the winner would still run a redundant `INSTALL`
and repeat the same move over a file that is now open.

`LOAD` is per-connection, so it still runs on every connection; only `INSTALL`
is serialized. An already-installed extension takes the pre-lock fast path and
costs neither a lock nor network. The lock is never held across the connection
mutex, so it cannot invert lock order.

Traced with strace on a cold cache under default test parallelism: before, 17
threads moved files into the store (13 racing on vss alone); after, exactly one
rename per extension.
2026-08-10 16:13:36 -06:00
Dark-Alex-17 3abc30d633 fix(rag): emit an sbx kit v2 mixin and declare RAG credentials to the proxy
The RAG attach sidecar was written against the sbx kit v1 spec and still emitted schemaVersion "1" with network.allowedDomains, network.serviceDomains, network.serviceAuth, credentials.sources.<n>.env and environment.proxyManaged. Every one of those keys was removed in kit v2. Coyote does not validate mixins, it copies them byte-for-byte into spec.yaml, so the invalid document surfaced only as an opaque sbx failure with no indication of which mixin caused it.

generate_rag_sbx_mixin now builds the document from the shared serializer structs instead of a format! string, which is how the envelope drifted unnoticed in the first place. render_mixin_yaml and the RAG sidecar both go through a new render_mixin_document, giving one definition of the envelope and one enforcement point for the rule that every inject domain must also appear in permissions.network.allow.

Fix an auth bug the port exposed: inject_rag_secrets bound the API key with sbx secret set, but nothing ever emitted a matching credentials entry, so the proxy held a value with no inject rule and never rewrote the auth header. An attached RAG credential silently did not work inside the sandbox. The sidecar now declares that credential; a RAG with no API key declares none while still receiving egress.

Fix the service id: the bind passed the raw file stem instead of routing it through secret_service_id, so a RAG named My_Docs produced an illegal id. The bind and the generated credentials service now share that derivation and cannot disagree.

Retire sbx_domain_forms in favour of allow_entry_for_url, now pub(crate). It emitted both a bare host and host:port because v1 serviceDomains needed a bare key; v2 has no such need, so the extra entry is simply wrong. It also defaulted a schemeless host to port 6333 while normalize_base_url resolves it to http and port 80, meaning the allow entry named a port the client never dialled.
2026-08-10 15:58:40 -06:00
Dark-Alex-17 f68937611e fix(rag): install DuckDB vss and fts extensions when they are missing
The DuckDB schema init loaded the vss and fts extensions but nothing ever
installed them, so any machine without them already present failed with
'IO Error: Extension "vss.duckdb_extension" not found'. This surfaced as 13
failing tests in CI while passing locally, because local runs had the
extensions installed already.

Loading is attempted first so an extension that is already present costs
nothing and never touches the network; INSTALL is reached only once, on a
machine seeing the extension for the first time, and reports an actionable
message if it cannot download.

CI cached the extension directory but nothing populated it, so the cache
saved an empty directory forever. The cache key now derives from Cargo.lock
rather than a hardcoded DuckDB version, and a step on cache miss installs the
extensions so the post-job save has something to store.
2026-08-10 15:47:15 -06:00
Dark-Alex-17 a12cf84eb6 Merge remote-tracking branch 'refs/remotes/origin/main' 2026-08-10 15:41:50 -06:00
Dark-Alex-17 2b45e3a9b8 feat: improved wording and heuristic detection for sisyphus suite of agents
CI / All (ubuntu-latest) (push) Failing after 25s
CI / All (macos-latest) (push) Has been cancelled
CI / All (windows-latest) (push) Has been cancelled
2026-08-10 15:37:53 -06:00
Dark-Alex-17 91dbaf5533 feat: upgraded to sbx kit v2 spec for improved integration
CI / All (ubuntu-latest) (push) Failing after 25s
CI / All (macos-latest) (push) Has been cancelled
CI / All (windows-latest) (push) Has been cancelled
2026-08-10 15:26:16 -06:00
Dark-Alex-17 7a732436aa feat(rag): add attach-only Qdrant provider, attach wizard and sandbox wiring
Adds QdrantProvider as a read-only driver for pre-existing remote Qdrant
collections, an interactive '.rag attach' wizard, and the sandbox credential
and domain-whitelisting wiring that lets an attached RAG work inside a sandbox.

Attach-only by design: rebuild_indexes bails for both the attached and the
unattached case rather than silently succeeding. Coyote never writes to Qdrant
in this change.

Vectors are never hydrated back from Qdrant. Cosine collections L2-normalize
stored vectors on write, so reading them back returns unit-length copies of the
originals; the YAML vector copy is authoritative and the serialization guard
stays scoped to the duckdb driver alone.

Collections keyed by string or UUID point IDs are rejected at attach time. The
read path parses point ids as u64 inside a filter_map, so such a collection
would otherwise yield zero results with no error.

The three response-shape parsers are pure functions over an already-parsed JSON
body, unit-tested against captured fixtures, with the async wrappers delegating
to them rather than duplicating the logic.

'.rag' now splits its first argument, so '.rag attach <name>' no longer tries to
load a RAG literally named 'attach <name>'.
2026-08-10 13:40:22 -06:00
Dark-Alex-17 98d3ba4a83 feat(rag): add DuckDB provider behind the RAG driver abstraction
Phase 3 of the RAG driver abstraction. Adds a `DuckDbProvider` that keeps
vectors and document content in a `.duckdb` sidecar next to the existing
YAML metadata, selected by the `driver: duckdb` field.

- `src/rag/providers/duckdb.rs` (new): vector search via the vss extension
  and keyword search via fts, an all-or-nothing hydration path (a partial
  read is an error, never a shorter map), and an anti-wipe guard that
  refuses the destructive `CREATE OR REPLACE TABLE` when `data.vectors` is
  empty while `data.files` is not and the store still holds rows.
- `src/rag/mod.rs`: `sync_documents` now refreshes `bm25`/`node_to_docs`
  BEFORE the fallible `provider.rebuild_indexes`. `self.data` is already
  mutated by that point, so propagating a provider error afterwards would
  leave the derived in-memory state describing the previous corpus while
  `data` describes the new one. Both rebuilds are pure functions of
  `self.data` and cannot fail, so running them first is always safe.
- `src/config/paths.rs`: sidecar path helpers.
- `src/rag/providers/mod.rs`, `src/config/agent.rs`: driver dispatch and
  RAG cache keying.

Also keeps `RequestContext::rag_key` in lockstep with `rag` at the two
sites that were still missing it, so that a cache insert and its matching
invalidate are structurally incapable of disagreeing:

- `use_agent` assigned `self.rag` from the agent but never set `rag_key`.
  This one was live. Agent RAGs are inserted under `RagKey::Agent(<name>)`,
  so with `rag_key == None` the invalidation guards in `rebuild_rag` and
  `edit_rag_docs` matched nothing and `.rebuild rag` left the stale cache
  entry in place. Worse, a preceding `.rag <name>` left a stale
  `Named(<name>)` key attached to the agent's RAG, pointing the
  invalidation at an unrelated RAG's cache entry. Now mirrors the insert
  key exactly, yielding `None` when the agent has no RAG.
- `exit_agent` cleared `self.rag` but left `rag_key` behind. Latent rather
  than live, since `rebuild_rag`/`edit_rag_docs` both bail on
  `rag.is_none()` before reaching the invalidate guards, but the guards
  that make it unobservable are not the kind of thing to depend on.

Covered by `use_agent_does_not_carry_stale_rag_key`, and by a new
assertion in `exit_agent_clears_all_agent_state`.
2026-08-10 12:51:56 -06:00
Dark-Alex-17 d734276927 build(rag): add duckdb dependency and pin comfy-table to 7.1.4
Adds the duckdb crate with the bundled feature ahead of any provider
code, so the dependency and build surface can be proven on every CI
target on its own.

duckdb constrains comfy-table to ~7.1, so comfy-table moves from 7.2.2
to 7.1.4 while keeping custom_styling. That feature swaps
measure_text_width for an ANSI-stripping implementation that
render_table's pre-styled cells depend on; dropping it still compiles
and still passes every other test, and only corrupts column widths. A
regression test now renders a styled table at a fixed wrap width and
asserts every line has an equal ANSI-stripped display width.

comfy-table 7.1.4 pins crossterm 0.28 while coyote pins 0.29, so both
now build side by side. comfy_table::Color, Attribute and Cell are
consequently crossterm 0.28 types and must not be used; table styling
stays ANSI-string based.

Caches the DuckDB extension directory in CI, keyed on the runner OS and
the DuckDB version, so vss and fts survive an upstream outage.

Also switches merge_vector_results to f32::total_cmp. The previous
partial_cmp().unwrap_or(Equal) comparator is not total under NaN, which
sort_by is permitted to answer with a panic in release builds.
2026-08-10 11:59:57 -06:00
Dark-Alex-17 5049143fcc refactor(rag): extract RagProvider trait and add YamlProvider
Introduce a narrow `RagProvider` trait covering vector search and content
retrieval, and make `Rag` delegate to a boxed provider instead of owning an
HNSW index directly. `YamlProvider` is the sole implementation for now.

The trait deliberately stays narrow: embeddings, chunking, BM25 keyword
search, graph RAG, entity extraction, RRF merging and persistence all remain
on `Rag`/`RagData`, so a new storage backend does not have to reimplement
Coyote's indexing logic.

Notable points:

- `fetch_content`'s ordering contract is part of the trait, not an accident.
  Implementations must return results in input-`ids` order; `hybrid_search`
  passes an RRF-ranked list straight to the prompt builder, so a provider
  returning storage order would silently discard the ranking.
- The content store is keyed on `data.files`, never `data.vectors`. Both the
  content map and BM25 now route through the new `RagData::iter_documents()`
  so the two key spaces match by construction. `RagData::add` zips document
  ids with embeddings and truncates silently, so ids in `files \ vectors` are
  genuinely reachable.
- A provider keyword-search failure degrades to an empty ranker with a
  warning rather than failing the whole query; it is one of three RRF inputs.
  It deliberately does not fall back to the local BM25, which would be a
  silent ranking-algorithm swap once a provider with native FTS exists.
- The rerank path builds its text and id vectors from a single `fetch_content`
  result in one pass, so the reranker's positional indices cannot desync.

This is not a bit-for-bit no-op. `vector_search` now dedups by best score and
sorts globally instead of concatenating per-chunk hit lists. Single-chunk
queries (the common case) are unaffected. Multi-chunk queries get corrected
rank assignment and no longer let a document that matched several query
chunks accumulate multiple RRF contributions. There is no overall cap on the
merged pool — truncation remains `reciprocal_rank_fusion`'s job.

`RagData::get()` is removed: its only two callers were the content lookups
replaced here, and an unused private-module method fails the build under
`--deny warnings`. Its three tests were rewritten against `iter_documents()`,
one of which now guards the files-vs-vectors keying directly.

Implements Phase 2 of the RAG driver abstraction design (§6).
2026-08-10 11:41:37 -06:00
Dark-Alex-17 a968c3228d feat(rag): add driver/attached fields, validation floors and force-reingest
Phase 1 of the RAG driver abstraction (design doc sections 5.1-5.5a).

Data model:
- Add `driver: String` (serde default "yaml" via RagData::default_driver) and
  `attached: bool` as the first two fields of RagData, so driver metadata sits
  at the top of each RAG YAML. Old files without them load unchanged.
- Add `#[serde(default)]` to the non-Option fields so a minimal attached-RAG
  YAML deserializes, and add `skip_serializing_if` to `vectors` so an empty
  map renders no `vectors:` key.
- Add a hand-written `impl Default for RagData` delegating to `RagData::new()`.
  It is deliberately not derived: a derived impl yields `driver: ""`, which is
  not a valid driver string.

Validation (the price of the new serde defaults):
- Add `RagData::validate()`, called from `Rag::load()` after deserialization.
  It enforces the (driver, attached) matrix and, critically, numeric floors
  that the new defaults would otherwise mask: `top_k >= 1` unconditionally
  (a 0 makes every query return nothing, silently), and `chunk_size >= 1` plus
  `chunk_overlap < chunk_size` when not attached (a 0 chunk_size is a real
  divide-by-zero panic while sizing embedding batches).
- Reject `.set rag_top_k 0` at the setter, before the set/update fork. Without
  this, the new load-time floor turns one keystroke into an unloadable RAG:
  the setter saves immediately and no dot-command can reach the file again.

Rebuild actually re-embeds now:
- `.rebuild rag` and `--rebuild-rag` previously re-scanned paths and re-embedded
  nothing, because the content-hash skip fired regardless of the refresh flag.
  Extract that decision into a module-level `find_hash_skip()` free function and
  thread a `force_reingest` flag through `sync_documents()` and
  `refresh_document_paths()`, set true only from `rebuild_rag()`. `.edit rag-docs`
  stays incremental. Re-embedding costs time and API spend, so `rebuild_rag()`
  now prints a one-line file-count warning first (no prompt: the path is
  reachable from a non-interactive CLI flag).

Attached-RAG guards:
- Block `.rebuild rag` / `--rebuild-rag` and `.edit rag-docs` on attached RAGs,
  which Coyote did not index and whose source documents it does not own.
- Add `Rag::driver()`, `Rag::is_attached()` and `Rag::file_count()`, and surface
  driver/attached through `Rag::export()` so `.info rag` shows them.

Adds 12 unit tests (1299 -> 1311), including the two gate tests pinning that a
forced re-ingest does not hash-skip while an ordinary refresh still does.
2026-08-10 11:04:51 -06:00
Dark-Alex-17 f404acdbca Merge branch 'main' 2026-08-06 16:39:02 -06:00
Dark-Alex-17 efa570267d feat(mcp): send RFC 8707 resource indicator in OAuth flows
CI / All (ubuntu-latest) (push) Failing after 27s
CI / All (macos-latest) (push) Has been cancelled
CI / All (windows-latest) (push) Has been cancelled
2026-08-06 16:13:06 -06:00
Dark-Alex-17 9aeb9e6e2e Merge branch 'main' 2026-08-06 13:13:03 -06:00
Dark-Alex-17 3607a180d9 fix: don't output thinking blocks for claude-based models
CI / All (ubuntu-latest) (push) Failing after 23s
CI / All (macos-latest) (push) Has been cancelled
CI / All (windows-latest) (push) Has been cancelled
2026-08-06 13:12:36 -06:00
Dark-Alex-17 d429def0f6 fix: additional edge case fix for duplicate tool call IDs in anthropic API calls
CI / All (ubuntu-latest) (push) Failing after 27s
CI / All (macos-latest) (push) Has been cancelled
CI / All (windows-latest) (push) Has been cancelled
2026-08-06 10:30:33 -06:00
Dark-Alex-17 e606eb7c49 fix: removed temperature modifier in librarian agent to mitigate invisible errors
CI / All (ubuntu-latest) (push) Failing after 27s
CI / All (macos-latest) (push) Has been cancelled
CI / All (windows-latest) (push) Has been cancelled
2026-08-05 16:10:19 -06:00
Dark-Alex-17 a8fb32b6bd fix: strip reasoning blocks for structured LLM output in graph agents 2026-08-05 16:10:03 -06:00
Dark-Alex-17 1f7b8417fa fix: prevent rare duplicate tool call IDs in long running claude prompts 2026-08-05 16:01:40 -06:00
Dark-Alex-17 9540345ec7 feat: Added loaded indicators to .list tools/mcp-servers/skills
CI / All (ubuntu-latest) (push) Failing after 27s
CI / All (macos-latest) (push) Has been cancelled
CI / All (windows-latest) (push) Has been cancelled
2026-08-05 13:11:24 -06:00
Dark-Alex-17 70b6d51b55 test: Implemented unit tests to prevent regression on agent reasoning effort inheritance
CI / All (ubuntu-latest) (push) Failing after 25s
CI / All (macos-latest) (push) Has been cancelled
CI / All (windows-latest) (push) Has been cancelled
2026-08-04 13:12:37 -06:00
Dark-Alex-17 9e5e8a60f2 fix: agents inherit global reasoning effort if unset 2026-08-04 13:09:13 -06:00
Dark-Alex-17 d6447603bc feat: hide .recover from tab completions when no session is active
CI / All (ubuntu-latest) (push) Failing after 24s
CI / All (macos-latest) (push) Has been cancelled
CI / All (windows-latest) (push) Has been cancelled
2026-08-04 12:05:23 -06:00
Dark-Alex-17 e6dc24beb5 feat: add a new .recover command for sessions to recover from errors
CI / All (ubuntu-latest) (push) Failing after 24s
CI / All (macos-latest) (push) Has been cancelled
CI / All (windows-latest) (push) Has been cancelled
2026-08-04 12:02:31 -06:00
Dark-Alex-17 e76c3efe4e feat: integrated the git-ssh-sign kit into the coyote sandbox kit
CI / All (ubuntu-latest) (push) Failing after 26s
CI / All (macos-latest) (push) Has been cancelled
CI / All (windows-latest) (push) Has been cancelled
2026-08-04 10:56:50 -06:00
github-actions[bot] fac577589b chore: bump Cargo.toml and sandbox image to 0.8.3
CI / All (ubuntu-latest) (push) Failing after 26s
CI / All (macos-latest) (push) Has been cancelled
CI / All (windows-latest) (push) Has been cancelled
2026-08-03 21:28:37 +00:00
github-actions[bot] 737fc42ec1 bump: version 0.8.2 → 0.8.3 [skip ci] 2026-08-03 21:28:33 +00:00
Dark-Alex-17 888529f381 fix: infinite loop bug when attempting to interrupt a prompt exchange right before a session compression
CI / All (ubuntu-latest) (push) Failing after 25s
CI / All (macos-latest) (push) Has been cancelled
CI / All (windows-latest) (push) Has been cancelled
2026-07-31 16:00:26 -06:00
Dark-Alex-17 35e75e5b4f fix: ctrl-c inside of an auto-continue loop created an infinite loop
CI / All (ubuntu-latest) (push) Failing after 26s
CI / All (macos-latest) (push) Has been cancelled
CI / All (windows-latest) (push) Has been cancelled
2026-07-31 11:44:16 -06:00
github-actions[bot] dd7d75fd9f chore: bump Cargo.toml and sandbox image to 0.8.2
CI / All (ubuntu-latest) (push) Failing after 25s
CI / All (macos-latest) (push) Has been cancelled
CI / All (windows-latest) (push) Has been cancelled
2026-07-31 16:08:58 +00:00
github-actions[bot] 78f5fc8fb9 bump: version 0.8.1 → 0.8.2 [skip ci] 2026-07-31 16:08:53 +00:00
Dark-Alex-17 4f38214681 fix: sbx update doesn't allow undefined fields in sbx spec
CI / All (ubuntu-latest) (push) Failing after 25s
CI / All (macos-latest) (push) Has been cancelled
CI / All (windows-latest) (push) Has been cancelled
2026-07-31 10:07:29 -06:00
github-actions[bot] 019bd6f1c7 chore: bump Cargo.toml and sandbox image to 0.8.1
CI / All (ubuntu-latest) (push) Failing after 25s
CI / All (macos-latest) (push) Has been cancelled
CI / All (windows-latest) (push) Has been cancelled
2026-07-30 22:37:17 +00:00
github-actions[bot] 875b6749c2 bump: version 0.8.0 → 0.8.1 [skip ci] 2026-07-30 22:37:15 +00:00
Dark-Alex-17 69e1b98c44 fix: ctrl-c interruption doesn't discard session messages when throbber is showing
CI / All (ubuntu-latest) (push) Failing after 27s
CI / All (macos-latest) (push) Has been cancelled
CI / All (windows-latest) (push) Has been cancelled
2026-07-30 13:38:37 -06:00
Dark-Alex-17 0324436114 feat: ctrl-c interrupts ongoing prompt in a session, but lets the user inject more instructions mid-stream
CI / All (ubuntu-latest) (push) Failing after 24s
CI / All (macos-latest) (push) Has been cancelled
CI / All (windows-latest) (push) Has been cancelled
2026-07-30 10:21:19 -06:00
Dark-Alex-17 233c212d2a feat: improved function calling performance by allowing parallel tool calling
CI / All (ubuntu-latest) (push) Failing after 25s
CI / All (macos-latest) (push) Has been cancelled
CI / All (windows-latest) (push) Has been cancelled
2026-07-29 12:40:52 -06:00
Dark-Alex-17 e288b41365 fix: improper handling of fd-style globbing for directories in fs_glob 2026-07-29 12:40:33 -06:00
Dark-Alex-17 fbf6a6bdf4 fix: properly templated architect design doc path in starter commands
CI / All (ubuntu-latest) (push) Failing after 25s
CI / All (macos-latest) (push) Has been cancelled
CI / All (windows-latest) (push) Has been cancelled
2026-07-29 11:59:09 -06:00
Dark-Alex-17 57b72702b2 feat: created the architect and gatekeeper agents for dramatically improved coding performance
CI / All (ubuntu-latest) (push) Failing after 25s
CI / All (macos-latest) (push) Has been cancelled
CI / All (windows-latest) (push) Has been cancelled
2026-07-29 11:23:52 -06:00
Dark-Alex-17 38ba303c3c feat: Improved readability of session message exchange replays
CI / All (ubuntu-latest) (push) Failing after 26s
CI / All (macos-latest) (push) Has been cancelled
CI / All (windows-latest) (push) Has been cancelled
2026-07-29 10:15:35 -06:00
Dark-Alex-17 fc7bc0ff8f fix: .copy works when sessions are resumed 2026-07-29 09:50:40 -06:00
Dark-Alex-17 7f7ea758a7 fix: ACP session/prompt now drives the full tool-execution loop
CI / All (ubuntu-latest) (push) Failing after 25s
CI / All (macos-latest) (push) Has been cancelled
CI / All (windows-latest) (push) Has been cancelled
run_prompt_turn previously performed a single completion and returned,
leaving tool calls unexecuted. It now mirrors start_directive's loop:
call -> execute tools -> merge results -> continue until no tool results,
then check the pending-agents guardrail before returning.

The pending-agents guardrail injects a reminder prompt when sub-agents
are pending, matching the same loop termination semantics as --headless.
The session is NOT exited between turns (unlike start_directive) so
multi-turn ACP conversations retain history across session/prompt calls.

Manual gate (tool-probe agent, fs_write + fs_read tools):
  id 3 result: {"output":"DONE:probe.txt","stopReason":"end_turn"}
  probe.txt exists: YES, content: hello
2026-07-28 15:42:51 -06:00
Dark-Alex-17 06b2c384e3 fix: ACP spec conformance — ContentBlock prompt params and protocolVersion type
CI / All (ubuntu-latest) (push) Failing after 26s
CI / All (macos-latest) (push) Has been cancelled
CI / All (windows-latest) (push) Has been cancelled
DEFECT 1: session/prompt now accepts the spec-shaped params as primary:
  {"prompt": [{"type": "text", "text": "..."}]}
Text blocks are joined with newlines. Non-text block types are silently
ignored. The legacy params.text alias is preserved as a fallback. -32602
is returned only when neither a non-empty prompt array with text blocks
nor a non-empty text field is present.

DEFECT 2: initialize result now emits protocolVersion as integer 1
instead of the string "1", matching the ACP spec's InitializeResponse.

Tests: 4 new unit tests pin the spec-shaped prompt path, the non-text
block ignore behavior, the missing-both -32602 path, and the numeric
protocolVersion type.
2026-07-28 14:41:22 -06:00
Dark-Alex-17 d50de7c06a lint: fixed test ordering
CI / All (ubuntu-latest) (push) Failing after 25s
CI / All (macos-latest) (push) Has been cancelled
CI / All (windows-latest) (push) Has been cancelled
2026-07-27 19:52:51 -06:00
Dark-Alex-17 0956f08791 refactor: move ACP server dispatch into run() for shared flag setup
All CLI flag processing (--agent, --role, --rag, --model, --no-memory,
--no-stream, --no-workspace-instructions, etc.) now runs through run()
before the REPL/cmd split. The ACP server is dispatched right before
match is_repl, after apply_prelude and skills loading, so it benefits
from the complete context setup with no duplication or drift risk.
2026-07-27 19:44:11 -06:00
Dark-Alex-17 087d0c320c feat: apply --agent/--role/--rag/--model flags in --acp-server mode
These CLI flags were previously ignored because the ACP branch returned
before run() could apply them. Now the context is configured with the
requested agent, role, RAG index, and model before the server starts.
Session management remains protocol-driven via session/new and session/load.
2026-07-27 19:31:45 -06:00
Dark-Alex-17 2128390f99 fmt: applied formatting
CI / All (ubuntu-latest) (push) Failing after 25s
CI / All (macos-latest) (push) Has been cancelled
CI / All (windows-latest) (push) Has been cancelled
2026-07-27 19:26:29 -06:00
Dark-Alex-17 d008de1848 fix: restore stdout output for standalone --headless mode
RenderMode::Silent was incorrectly applied to --headless in addition to
--acp-server. Standalone headless should still display the LLM response;
only ACP mode requires stdout purity for JSON-RPC. The acp server's
run_prompt_turn already sets Silent before each prompt call.
2026-07-27 19:22:51 -06:00
Dark-Alex-17 d79787bf96 fix: suppress tool-call display in headless mode; initialize session on session/new 2026-07-27 19:14:28 -06:00
Dark-Alex-17 577c51b62f fix: skip stdin drain and set silent render mode when --acp-server is active 2026-07-27 18:59:09 -06:00
Dark-Alex-17 d462b09f80 feat: add headless profile to sbx-kit spec 2026-07-27 18:03:35 -06:00
Dark-Alex-17 b711e4983b feat: implement ACP user-interaction to request_permission bridge 2026-07-27 18:00:07 -06:00
Dark-Alex-17 6ae3efb06c feat: implement ACP session/load and session/cancel 2026-07-27 17:52:49 -06:00
Dark-Alex-17 02dd14394b feat: implement ACP session/prompt 2026-07-27 17:48:31 -06:00
Dark-Alex-17 f11d4ca760 feat: add ACP server skeleton with stdout-purity test 2026-07-27 17:31:47 -06:00
Dark-Alex-17 f1415067f2 feat: add --headless flag for unattended operation 2026-07-27 17:17:56 -06:00
Dark-Alex-17 af5c34fde5 Merge branch 'main' of github.com:Dark-Alex-17/coyote 2026-07-27 15:05:25 -06:00
Dark-Alex-17 2ffa278f2d test: testing potential nerdbox regression fix for coyote sandbox mode 2026-07-27 15:04:41 -06:00
Dark-Alex-17 f4cbee9611 docs: remove comment in spec.yaml about copying in coyote password file 2026-07-27 10:50:40 -06:00
Dark-Alex-17 e95fd1e06e ci: fix typo in coyote image tag; needs leading 'v'
CI / All (ubuntu-latest) (push) Failing after 23s
CI / All (macos-latest) (push) Has been cancelled
CI / All (windows-latest) (push) Has been cancelled
2026-07-27 09:51:23 -06:00
Dark-Alex-17 d79ea55e09 fix: include graph-agent descriptions in .agent <TAB> completions
CI / All (ubuntu-latest) (push) Failing after 25s
CI / All (macos-latest) (push) Has been cancelled
CI / All (windows-latest) (push) Has been cancelled
2026-07-27 09:34:36 -06:00
github-actions[bot] b898e4dd78 chore: bump Cargo.toml and sandbox image to 0.8.0
CI / All (ubuntu-latest) (push) Failing after 24s
CI / All (macos-latest) (push) Has been cancelled
CI / All (windows-latest) (push) Has been cancelled
2026-07-25 01:08:02 +00:00
github-actions[bot] 30e00f8332 bump: version 0.7.4 → 0.8.0 [skip ci] 2026-07-25 01:07:59 +00:00
Dark-Alex-17 58d9d4c64e docs: updated help message for --fresh flag
CI / All (ubuntu-latest) (push) Failing after 25s
CI / All (macos-latest) (push) Has been cancelled
CI / All (windows-latest) (push) Has been cancelled
2026-07-24 18:30:45 -06:00
Dark-Alex-17 63a768aa46 fix: fresh wizard openai-compatible support
CI / All (ubuntu-latest) (push) Failing after 25s
CI / All (macos-latest) (push) Has been cancelled
CI / All (windows-latest) (push) Has been cancelled
2026-07-24 18:29:24 -06:00
Dark-Alex-17 2c3f671efa fix: config existence check for --fresh sandboxes
CI / All (ubuntu-latest) (push) Failing after 24s
CI / All (macos-latest) (push) Has been cancelled
CI / All (windows-latest) (push) Has been cancelled
2026-07-24 18:09:06 -06:00
Dark-Alex-17 8b0a536f4e feat: Dynamically detect if a selected client in the sandbox first run wizard supports oauth
CI / All (ubuntu-latest) (push) Failing after 24s
CI / All (macos-latest) (push) Has been cancelled
CI / All (windows-latest) (push) Has been cancelled
2026-07-24 18:07:30 -06:00
Dark-Alex-17 5a1bb569b4 feat: Add support for the --fresh flag again with host environment configuration injection
CI / All (ubuntu-latest) (push) Failing after 24s
CI / All (macos-latest) (push) Has been cancelled
CI / All (windows-latest) (push) Has been cancelled
2026-07-24 18:00:41 -06:00
Dark-Alex-17 0c12580836 fix: Improve coyote sandbox startup time 2026-07-24 17:23:08 -06:00
Dark-Alex-17 df909325a7 fmt: applied formatting
CI / All (ubuntu-latest) (push) Failing after 25s
CI / All (macos-latest) (push) Has been cancelled
CI / All (windows-latest) (push) Has been cancelled
2026-07-24 16:50:27 -06:00
Dark-Alex-17 d379fcddf8 fix: bypass forgotten sandbox mode check for MCP secret interpolation
CI / All (ubuntu-latest) (push) Failing after 24s
CI / All (macos-latest) (push) Has been cancelled
CI / All (windows-latest) (push) Has been cancelled
2026-07-24 16:50:04 -06:00
Dark-Alex-17 b405acd8f0 feat: force overwrite global sbx secrets 2026-07-24 16:45:51 -06:00
Dark-Alex-17 805ae7112a feat: add sbx secrets globally
CI / All (ubuntu-latest) (push) Failing after 24s
CI / All (macos-latest) (push) Has been cancelled
CI / All (windows-latest) (push) Has been cancelled
2026-07-24 16:41:00 -06:00
Dark-Alex-17 6ddbf37523 feat: only create secrets local to a sandbox
CI / All (ubuntu-latest) (push) Failing after 25s
CI / All (macos-latest) (push) Has been cancelled
CI / All (windows-latest) (push) Has been cancelled
2026-07-24 16:36:16 -06:00
Dark-Alex-17 f510bb649b feat: Improved credentials management for docker sandboxes 2026-07-24 16:26:35 -06:00
Dark-Alex-17 c1b14bdfdf docs: Created a mermaid diagram for Sisyphus
CI / All (ubuntu-latest) (push) Failing after 24s
CI / All (macos-latest) (push) Has been cancelled
CI / All (windows-latest) (push) Has been cancelled
2026-07-24 13:55:31 -06:00
Dark-Alex-17 46f2a9eae2 docs: added in forgotten connection to the librarian agent diagram
CI / All (ubuntu-latest) (push) Failing after 25s
CI / All (macos-latest) (push) Has been cancelled
CI / All (windows-latest) (push) Has been cancelled
2026-07-24 13:44:07 -06:00
Dark-Alex-17 cf3a12141b docs: updated the graph agent diagrams to use mermaid diagrams for more easily readable diagramming in their READMEs
CI / All (ubuntu-latest) (push) Failing after 24s
CI / All (macos-latest) (push) Has been cancelled
CI / All (windows-latest) (push) Has been cancelled
2026-07-24 13:39:59 -06:00
Dark-Alex-17 8f13810f0f chore: updated models.yaml 2026-07-24 13:27:20 -06:00
Dark-Alex-17 eba8c86e21 fix: properly wrap sub-style changes in markdown rendering
CI / All (ubuntu-latest) (push) Failing after 25s
CI / All (macos-latest) (push) Has been cancelled
CI / All (windows-latest) (push) Has been cancelled
2026-07-24 12:13:33 -06:00
Dark-Alex-17 d75fb47de1 docs: Updated project license
CI / All (ubuntu-latest) (push) Failing after 24s
CI / All (macos-latest) (push) Has been cancelled
CI / All (windows-latest) (push) Has been cancelled
2026-07-23 21:07:51 -06:00
Dark-Alex-17 49a281bf93 feat: renamed --contents arg for fs_write/patch to --content since most models attempt that first and error otherwise
CI / All (ubuntu-latest) (push) Failing after 24s
CI / All (macos-latest) (push) Has been cancelled
CI / All (windows-latest) (push) Has been cancelled
2026-07-23 19:13:36 -06:00
Dark-Alex-17 b9474ce6ef feat: Used improved theme selections for tool call highlighting colors
CI / All (ubuntu-latest) (push) Failing after 25s
CI / All (macos-latest) (push) Has been cancelled
CI / All (windows-latest) (push) Has been cancelled
2026-07-23 18:51:57 -06:00
Dark-Alex-17 d328310880 feat: Improved theme-derived syntax highlighting for LLM tool call logging 2026-07-23 18:45:23 -06:00
Dark-Alex-17 5eb77ab467 feat: Improved coloring of LLM tool invocation outputs to make LLM output more readable and cohesive 2026-07-23 18:27:54 -06:00
Dark-Alex-17 a2c4f05c8c feat: renamed the user__ask to user__select and improved descriptions to improve model usage 2026-07-23 18:22:18 -06:00
Dark-Alex-17 89fadcca15 feat: Improved coloring/highlighting of tool calls to make LLM invocation logs easier to read 2026-07-23 18:12:18 -06:00
Dark-Alex-17 ab3a818507 docs: Added the new compression control fields to the config example file
CI / All (ubuntu-latest) (push) Failing after 24s
CI / All (macos-latest) (push) Has been cancelled
CI / All (windows-latest) (push) Has been cancelled
2026-07-23 12:40:38 -06:00
94 changed files with 14269 additions and 2018 deletions
+11
View File
@@ -36,6 +36,17 @@ jobs:
- uses: Swatinem/rust-cache@v2 - uses: Swatinem/rust-cache@v2
- name: Cache DuckDB Extensions
id: duckdb-extensions
uses: actions/cache@v4
with:
path: ~/.duckdb/extensions
key: duckdb-ext-${{ matrix.os }}-${{ hashFiles('Cargo.lock') }}
- name: Install DuckDB Extensions
if: steps.duckdb-extensions.outputs.cache-hit != 'true'
run: cargo test --all duckdb
- name: Test - name: Test
run: cargo test --all run: cargo test --all
+1 -1
View File
@@ -516,5 +516,5 @@ jobs:
file: Dockerfile file: Dockerfile
platforms: linux/amd64,linux/arm64 platforms: linux/amd64,linux/arm64
push: ${{ env.ACT != 'true' }} push: ${{ env.ACT != 'true' }}
tags: darkalex17/coyote:latest, darkalex17/coyote:${{ env.version }} tags: darkalex17/coyote:latest, darkalex17/coyote:v${{ env.version }}
build-args: COYOTE_VERSION=${{ env.version }} build-args: COYOTE_VERSION=${{ env.version }}
+1
View File
@@ -5,3 +5,4 @@
.idea/ .idea/
/coyote.iml /coyote.iml
/.idea/ /.idea/
.coyote
+194
View File
@@ -1,3 +1,197 @@
## v0.8.3 (2026-08-03)
### Fix
- infinite loop bug when attempting to interrupt a prompt exchange right before a session compression
- ctrl-c inside of an auto-continue loop created an infinite loop
## v0.8.2 (2026-07-31)
### Fix
- sbx update doesn't allow undefined fields in sbx spec
## v0.8.1 (2026-07-30)
### Feat
- ctrl-c interrupts ongoing prompt in a session, but lets the user inject more instructions mid-stream
- improved function calling performance by allowing parallel tool calling
- created the architect and gatekeeper agents for dramatically improved coding performance
- Improved readability of session message exchange replays
- apply --agent/--role/--rag/--model flags in --acp-server mode
- add headless profile to sbx-kit spec
- implement ACP user-interaction to request_permission bridge
- implement ACP session/load and session/cancel
- implement ACP session/prompt
- add ACP server skeleton with stdout-purity test
- add --headless flag for unattended operation
### Fix
- ctrl-c interruption doesn't discard session messages when throbber is showing
- improper handling of fd-style globbing for directories in fs_glob
- properly templated architect design doc path in starter commands
- .copy works when sessions are resumed
- ACP session/prompt now drives the full tool-execution loop
- ACP spec conformance — ContentBlock prompt params and protocolVersion type
- restore stdout output for standalone --headless mode
- suppress tool-call display in headless mode; initialize session on session/new
- skip stdin drain and set silent render mode when --acp-server is active
- include graph-agent descriptions in .agent <TAB> completions
### Refactor
- move ACP server dispatch into run() for shared flag setup
## v0.8.0 (2026-07-25)
### Feat
- Dynamically detect if a selected client in the sandbox first run wizard supports oauth
- Add support for the --fresh flag again with host environment configuration injection
- force overwrite global sbx secrets
- add sbx secrets globally
- only create secrets local to a sandbox
- Improved credentials management for docker sandboxes
- renamed --contents arg for fs_write/patch to --content since most models attempt that first and error otherwise
- Used improved theme selections for tool call highlighting colors
- Improved theme-derived syntax highlighting for LLM tool call logging
- Improved coloring of LLM tool invocation outputs to make LLM output more readable and cohesive
- renamed the user__ask to user__select and improved descriptions to improve model usage
- Improved coloring/highlighting of tool calls to make LLM invocation logs easier to read
- long-running session improvements
- Made sisyphus suite of agents all auto-approved for tool usage
- added ast_grep tool to Sisyphus suite agents
- created the adversay agent and adversarial-review skill
- new spawnable_agents field in agents to let users restrict what agents can be spawned by a parent agent
- replay pre-compressed messages as well when resuming sessions for users to see
- created a new builtin function for agents who can spawn other agents to list available agents via agent__list_available
- **render**: hanging-indent line wrapping for lists and blockquotes
- **render**: wire table state machine and finalize hook
- **render**: render markdown tables with comfy-table
- **render**: parse table cells and column alignments
- **render**: detect markdown table rows and separators
- **render**: add comfy-table dependency and table border style
- **render**: activate rich markdown renderer as default
- **render**: rich block-level markdown rendering (headings, quotes, lists, hr)
- **render**: rich inline markdown rendering (bold, italic, code, links)
- **render**: detect markdown block-level line types
- **render**: precompute markdown scope styles for rich rendering
- Created the raw_markdown configuration flag
- added a .fork command to fork a new session from a running conversation
- **oauth**: enable browser-paste PKCE flow for OpenAI-compatible providers
- copy host OAuth tokens into sandbox at launch
- implement OAuth 2.0 Device Authorization Grant (RFC 8628)
- hint that browser 'paste code' pages can be ignored during callback capture
- openai-compatible wizard offers OAuth when provider has bundled oauth defaults
- validate unique client names at config load
- bundle xAI OAuth defaults in models.yaml
- OAuth branch in openai_compatible prepare_* fns
- get_oauth_provider_for_client dispatcher + client_config_info update
- OpenAICompatibleOAuthProvider (config-driven OAuthProvider impl)
- add auth + oauth fields to OpenAICompatibleConfig
- add oauth field to ProviderModels
- add client_credentials support to prepare_oauth_access_token
- add OAuthConfig + OAuthFlow types to oauth.rs
- Added reasoning effort to the right prompt
- Improved support for Anthropic's extended thinking
- Also support GEMINI.md workspace instructions
- Improved workspace instructions support
- also detect .mcp.json configurations at workspace roots
- Created new .tool enable/disable and .mcp enable/disable aliases to make REPL usage more egonomic
- Created a new .list <kind> REPL command to make discoveribility easier in the REPL
- Support claude-style hidden workspace MCP configuration files via .mcp.json
- Allow users to customize the workspace-specific configuration directory name so they can use Coyote with other CLI clients like .claude
- Add reasoning_effort validation for the main configuration file
- Add validation for reasoning_effort settings to prevent users from specifying erroneous values
- Added support for modifying the reasoning effort of reasoning models
- Explicitly Prevent .undo usage in graph agents
- Added an .undo command to the REPL to let users have more control over the conversation
- Improve sandbox startup time by using the prebuilt Coyote image
- Make coyote available as a docker image
- Made fs_patch more flexible for different model preferences of patch formats
- Installed nano into the sandbox so that users can edit config files in the sandbox directly
- Support workspace-local skill definitions and MCP configurations
- fully functional graph-based RAG
- Implemented graph-based RAG
- Added a --dangerously-skip-permissions flag to skip permission prompts for tool invocations
- Remove the temperature hyperparameter from the diagnose role
- Added a new oauth.redirectHost field to make it possible to further extend MCP support
- Updated the REPL mcp auth path to use the prettified error messaging
- Improved error messaging for failed MCP starts because of auth issues
- Added support for specifying the oauth port and client ID in MCP server configs
- Implemented OAuth support for OpenAI models via Codex endpoints
- merge MCP config when installing bundled mcp config
- Implemented durable state for sisyphus
- Installed ast-grep for the explore agent to use for better code exploration
- Created the step-runner graph agent for more deterministic coding workflows to produce even more reliable and higher-quality results
- Improved oracle and sisyphus agents with skill integrations for the new skills
- Created new sisyphus family skills to improve performance
- Created new diagnostic role and skill for use in other contexts
- Added new memory functions for deleting and renaming memory files, as well as new lints for memory expiration dates and staleness of memories to improve the memory system
- Created a new iwe skill and installed the iwe MCP server for utilizing large knowledgebases
- Session-specific, file-backed history in the REPL
- Replay session output when a user re-enters a session so all output can be seen again
- Added confirmation message after MCP Oauth succeeds when invoked from --auth-mcp
- Created the --auth-mcp CLI flag to let users auth with remote MCP servers without needing to be in the REPL
- add OAuth authentication support for remote MCP servers
- Added mixin for sisyphus so the ddg MCP server can search arbitrary domains
- added improved error messaging on MCP server initialization
- prefer musl versions for linux when running --update/.update
### Fix
- fresh wizard openai-compatible support
- config existence check for --fresh sandboxes
- Improve coyote sandbox startup time
- bypass forgotten sandbox mode check for MCP secret interpolation
- properly wrap sub-style changes in markdown rendering
- removed accidental duplicate ast_grep tool in explore agent
- npm and npx need the /usr/local/share/npm-global/lib directory to exist to run properly so I've added it to the dockerfile
- added executable bit to adversary agent tools script
- fetch descriptions from graph agent configs as well when listing agents
- **render**: suppress blank lines above rendered table
- chown the full sandbox cache dir, not just the coyote subdir
- chown the whole coyote cache dir not just the oauth dir in the sandbox
- fix typo in Gemini's generation_config property to use camelCase exclusively
- **oauth**: treat missing expires_in as non-expiring device_code token
- **oauth**: send Accept: application/json in device flow requests
- OAuth callback listener skips speculative/malformed browser connections
- resolve reasoning effort for the prompt for global defaults as well
- Account for default model reasoning_effort when supplying that value for the REPL prompts
- model narration included in history and between tool calls to prevent repetition
- Don't terminate agent loops early for null tool output
- reduce code duplication by reusing the new concrete_tool_names function in .list tools
- Agent tools can only be modified via .tool enable/disable using tools in the allowed whitelist in the agent
- re-render agent sessions when entering agents with either pre-configured agent_session or when entering an agent directly into a session
- Per RFC 9728, enable dynamic discovery of OAuth endpoints in MCP using path-aware discovery
- hot-attach to MCP servers that require auth after running .mcp auth <name>
- Correctly inherit graph-global model for extractor model if none is defined
- no cursor timeout when user scrolls away from ongoing streaming output
- default to the nano or notepad when a configured editor is not found
- When EDITOR, VISUAL, or config.editor is defined, don't verify via which
- Added a loop exit condition for the diagnostics skill
- Added directness clause to the diganose role to improve prompt
- fs tools now output better error handling to guide the model more effectively
- Make fs_read more tolerant of various arg invocation formats.
- todo functions are injected properly to roles when roles have auto_continue: true and the REPL is started directly into the role
- updated the redirect URI for OAuth MCP to use localhost since that's what is whitelisted, not 127.0.0.1
- allow MCP OAuth refresh_token to be absent from initial token exchanges
- Overrode the default JSON content-type for MCP OAuth so its properly application/x-www-form-urlencoded
- typo in mcp file name
- Added uvx wrapper for macos-based sandboxes
### Refactor
- Standardized paths module function names to not use 'path' in the name and to just always be either 'dir' or 'file'
- main.rs resolve_oauth_client uses new dispatcher
- split run_oauth_flow into pkce + client_credentials dispatchers
### Perf
- updated the memory injection warning so it only logs once, rather than after each keystroke
## v0.7.4 (2026-07-02) ## v0.7.4 (2026-07-02)
### Feat ### Feat
+10 -1
View File
@@ -28,4 +28,13 @@ While Coyote has since diverged significantly and is now developed as an
independent project, its early foundation and inspiration came from the independent project, its early foundation and inspiration came from the
AIChat project. AIChat project.
AIChat is licensed under the MIT License. AIChat is licensed under the MIT License. The MIT license text and its
copyright notice are preserved in the [LICENSE-MIT](./LICENSE-MIT) file.
## Licensing
Coyote as a whole is licensed under the GNU Affero General Public License
v3.0 only (AGPL-3.0-only); see [LICENSE](./LICENSE). Substantial portions
derived from AIChat remain under the MIT License (Copyright (c) sigoden),
preserved in [LICENSE-MIT](./LICENSE-MIT). See [NOTICE](./NOTICE) for the
combined-licensing summary.
Generated
+951 -504
View File
File diff suppressed because it is too large Load Diff
+4 -3
View File
@@ -1,6 +1,6 @@
[package] [package]
name = "coyote-ai" name = "coyote-ai"
version = "0.7.4" version = "0.8.3"
edition = "2024" edition = "2024"
authors = ["Alex Clarke <alex.j.tusa@gmail.com>"] authors = ["Alex Clarke <alex.j.tusa@gmail.com>"]
description = "An all-in-one, batteries included LLM CLI Tool" description = "An all-in-one, batteries included LLM CLI Tool"
@@ -9,7 +9,7 @@ homepage = "https://github.com/Dark-Alex-17/coyote"
repository = "https://github.com/Dark-Alex-17/coyote" repository = "https://github.com/Dark-Alex-17/coyote"
categories = ["command-line-utilities"] categories = ["command-line-utilities"]
readme = "README.md" readme = "README.md"
license = "MIT" license = "AGPL-3.0-only"
rust-version = "1.95.0" rust-version = "1.95.0"
exclude = [".github", "CONTRIBUTING.md"] exclude = [".github", "CONTRIBUTING.md"]
@@ -17,8 +17,9 @@ exclude = [".github", "CONTRIBUTING.md"]
anyhow = "1.0.69" anyhow = "1.0.69"
bytes = "1.4.0" bytes = "1.4.0"
clap = { version = "4.5.40", features = ["cargo", "derive", "wrap_help"] } clap = { version = "4.5.40", features = ["cargo", "derive", "wrap_help"] }
comfy-table = { version = "7.2.2", features = ["custom_styling"] } comfy-table = { version = "7.1.4", features = ["custom_styling"] }
dirs = "6.0.0" dirs = "6.0.0"
duckdb = { version = "1.10505.0", features = ["bundled"] }
dunce = "1.0.5" dunce = "1.0.5"
futures-util = "0.3.29" futures-util = "0.3.29"
inquire = "0.9.4" inquire = "0.9.4"
+39 -1
View File
@@ -1,5 +1,5 @@
ARG COYOTE_VERSION ARG COYOTE_VERSION
FROM docker/sandbox-templates:shell-docker FROM docker/sandbox-templates:shell-docker AS build
ARG COYOTE_VERSION ARG COYOTE_VERSION
ARG TARGETARCH ARG TARGETARCH
@@ -36,6 +36,21 @@ RUN set -euo pipefail; \
install -m 0755 "$TMPDIR/usql_static" /usr/local/bin/usql; \ install -m 0755 "$TMPDIR/usql_static" /usr/local/bin/usql; \
rm -rf "$TMPDIR" rm -rf "$TMPDIR"
RUN set -euo pipefail; \
DUCKDB_VERSION=1.5.5; \
case "${TARGETARCH}" in \
amd64) DUCKDB_ARCH=amd64 ;; \
arm64) DUCKDB_ARCH=arm64 ;; \
*) echo "Unsupported TARGETARCH: ${TARGETARCH}" >&2; exit 1 ;; \
esac; \
TMPDIR=$(mktemp -d); \
curl -fsSL --retry 3 \
"https://github.com/duckdb/duckdb/releases/download/v${DUCKDB_VERSION}/duckdb_cli-linux-${DUCKDB_ARCH}.gz" \
-o "$TMPDIR/duckdb.gz"; \
gunzip "$TMPDIR/duckdb.gz"; \
install -m 0755 "$TMPDIR/duckdb" /usr/local/bin/duckdb; \
rm -rf "$TMPDIR"
USER 1000 USER 1000
RUN curl -LsSf https://astral.sh/uv/install.sh | sh && \ RUN curl -LsSf https://astral.sh/uv/install.sh | sh && \
@@ -67,6 +82,29 @@ RUN set -euo pipefail; \
chown 1000:1000 /home/agent/.cargo/bin/coyote; \ chown 1000:1000 /home/agent/.cargo/bin/coyote; \
rm -rf "$TMPDIR" rm -rf "$TMPDIR"
FROM scratch
ARG COYOTE_VERSION
COPY --from=build / /
ENV PATH="/home/agent/.cargo/bin:/home/agent/.local/bin:/usr/local/share/npm-global/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin" \
NPM_CONFIG_PREFIX="/usr/local/share/npm-global" \
NO_PROXY="localhost,127.0.0.1,::1,172.17.0.0/16" \
no_proxy="localhost,127.0.0.1,::1,172.17.0.0/16" \
BASH_ENV="/etc/sandbox-persistent.sh"
LABEL com.docker.sandboxes="templates" \
com.docker.sandboxes.base="ubuntu:questing" \
com.docker.sandboxes.flavor="shell-docker" \
com.docker.sandboxes.start-docker="true" \
org.opencontainers.image.title="coyote" \
org.opencontainers.image.description="An all-in-one, batteries-included LLM CLI tool: Shell Assistant, CLI & REPL mode, RAG, AI tools & agents, MCP servers, skills, and macros." \
org.opencontainers.image.source="https://github.com/Dark-Alex-17/coyote" \
org.opencontainers.image.version="${COYOTE_VERSION}"
WORKDIR /home/agent/workspace
USER 1000 USER 1000
ENTRYPOINT ["coyote"] ENTRYPOINT ["coyote"]
+657 -18
View File
@@ -1,22 +1,661 @@
The MIT License (MIT) GNU AFFERO GENERAL PUBLIC LICENSE
Version 3, 19 November 2007
Copyright (c) 2025 sigoden Copyright (C) 2007 Free Software Foundation, Inc. <https://fsf.org/>
Copyright (c) 2025 Alexander J. Clarke Everyone is permitted to copy and distribute verbatim copies
of this license document, but changing it is not allowed.
Permission is hereby granted, free of charge, to any person obtaining a copy Preamble
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:
The above copyright notice and this permission notice shall be included in all The GNU Affero General Public License is a free, copyleft license for
copies or substantial portions of the Software. software and other kinds of works, specifically designed to ensure
cooperation with the community in the case of network server software.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR The licenses for most software and other practical works are designed
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, to take away your freedom to share and change the works. By contrast,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE our General Public Licenses are intended to guarantee your freedom to
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER share and change all versions of a program--to make sure it remains free
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, software for all its users.
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
SOFTWARE. When we speak of free software, we are referring to freedom, not
price. Our General Public Licenses are designed to make sure that you
have the freedom to distribute copies of free software (and charge for
them if you wish), that you receive source code or can get it if you
want it, that you can change the software or use pieces of it in new
free programs, and that you know you can do these things.
Developers that use our General Public Licenses protect your rights
with two steps: (1) assert copyright on the software, and (2) offer
you this License which gives you legal permission to copy, distribute
and/or modify the software.
A secondary benefit of defending all users' freedom is that
improvements made in alternate versions of the program, if they
receive widespread use, become available for other developers to
incorporate. Many developers of free software are heartened and
encouraged by the resulting cooperation. However, in the case of
software used on network servers, this result may fail to come about.
The GNU General Public License permits making a modified version and
letting the public access it on a server without ever releasing its
source code to the public.
The GNU Affero General Public License is designed specifically to
ensure that, in such cases, the modified source code becomes available
to the community. It requires the operator of a network server to
provide the source code of the modified version running there to the
users of that server. Therefore, public use of a modified version, on
a publicly accessible server, gives the public access to the source
code of the modified version.
An older license, called the Affero General Public License and
published by Affero, was designed to accomplish similar goals. This is
a different license, not a version of the Affero GPL, but Affero has
released a new version of the Affero GPL which permits relicensing under
this license.
The precise terms and conditions for copying, distribution and
modification follow.
TERMS AND CONDITIONS
0. Definitions.
"This License" refers to version 3 of the GNU Affero General Public License.
"Copyright" also means copyright-like laws that apply to other kinds of
works, such as semiconductor masks.
"The Program" refers to any copyrightable work licensed under this
License. Each licensee is addressed as "you". "Licensees" and
"recipients" may be individuals or organizations.
To "modify" a work means to copy from or adapt all or part of the work
in a fashion requiring copyright permission, other than the making of an
exact copy. The resulting work is called a "modified version" of the
earlier work or a work "based on" the earlier work.
A "covered work" means either the unmodified Program or a work based
on the Program.
To "propagate" a work means to do anything with it that, without
permission, would make you directly or secondarily liable for
infringement under applicable copyright law, except executing it on a
computer or modifying a private copy. Propagation includes copying,
distribution (with or without modification), making available to the
public, and in some countries other activities as well.
To "convey" a work means any kind of propagation that enables other
parties to make or receive copies. Mere interaction with a user through
a computer network, with no transfer of a copy, is not conveying.
An interactive user interface displays "Appropriate Legal Notices"
to the extent that it includes a convenient and prominently visible
feature that (1) displays an appropriate copyright notice, and (2)
tells the user that there is no warranty for the work (except to the
extent that warranties are provided), that licensees may convey the
work under this License, and how to view a copy of this License. If
the interface presents a list of user commands or options, such as a
menu, a prominent item in the list meets this criterion.
1. Source Code.
The "source code" for a work means the preferred form of the work
for making modifications to it. "Object code" means any non-source
form of a work.
A "Standard Interface" means an interface that either is an official
standard defined by a recognized standards body, or, in the case of
interfaces specified for a particular programming language, one that
is widely used among developers working in that language.
The "System Libraries" of an executable work include anything, other
than the work as a whole, that (a) is included in the normal form of
packaging a Major Component, but which is not part of that Major
Component, and (b) serves only to enable use of the work with that
Major Component, or to implement a Standard Interface for which an
implementation is available to the public in source code form. A
"Major Component", in this context, means a major essential component
(kernel, window system, and so on) of the specific operating system
(if any) on which the executable work runs, or a compiler used to
produce the work, or an object code interpreter used to run it.
The "Corresponding Source" for a work in object code form means all
the source code needed to generate, install, and (for an executable
work) run the object code and to modify the work, including scripts to
control those activities. However, it does not include the work's
System Libraries, or general-purpose tools or generally available free
programs which are used unmodified in performing those activities but
which are not part of the work. For example, Corresponding Source
includes interface definition files associated with source files for
the work, and the source code for shared libraries and dynamically
linked subprograms that the work is specifically designed to require,
such as by intimate data communication or control flow between those
subprograms and other parts of the work.
The Corresponding Source need not include anything that users
can regenerate automatically from other parts of the Corresponding
Source.
The Corresponding Source for a work in source code form is that
same work.
2. Basic Permissions.
All rights granted under this License are granted for the term of
copyright on the Program, and are irrevocable provided the stated
conditions are met. This License explicitly affirms your unlimited
permission to run the unmodified Program. The output from running a
covered work is covered by this License only if the output, given its
content, constitutes a covered work. This License acknowledges your
rights of fair use or other equivalent, as provided by copyright law.
You may make, run and propagate covered works that you do not
convey, without conditions so long as your license otherwise remains
in force. You may convey covered works to others for the sole purpose
of having them make modifications exclusively for you, or provide you
with facilities for running those works, provided that you comply with
the terms of this License in conveying all material for which you do
not control copyright. Those thus making or running the covered works
for you must do so exclusively on your behalf, under your direction
and control, on terms that prohibit them from making any copies of
your copyrighted material outside their relationship with you.
Conveying under any other circumstances is permitted solely under
the conditions stated below. Sublicensing is not allowed; section 10
makes it unnecessary.
3. Protecting Users' Legal Rights From Anti-Circumvention Law.
No covered work shall be deemed part of an effective technological
measure under any applicable law fulfilling obligations under article
11 of the WIPO copyright treaty adopted on 20 December 1996, or
similar laws prohibiting or restricting circumvention of such
measures.
When you convey a covered work, you waive any legal power to forbid
circumvention of technological measures to the extent such circumvention
is effected by exercising rights under this License with respect to
the covered work, and you disclaim any intention to limit operation or
modification of the work as a means of enforcing, against the work's
users, your or third parties' legal rights to forbid circumvention of
technological measures.
4. Conveying Verbatim Copies.
You may convey verbatim copies of the Program's source code as you
receive it, in any medium, provided that you conspicuously and
appropriately publish on each copy an appropriate copyright notice;
keep intact all notices stating that this License and any
non-permissive terms added in accord with section 7 apply to the code;
keep intact all notices of the absence of any warranty; and give all
recipients a copy of this License along with the Program.
You may charge any price or no price for each copy that you convey,
and you may offer support or warranty protection for a fee.
5. Conveying Modified Source Versions.
You may convey a work based on the Program, or the modifications to
produce it from the Program, in the form of source code under the
terms of section 4, provided that you also meet all of these conditions:
a) The work must carry prominent notices stating that you modified
it, and giving a relevant date.
b) The work must carry prominent notices stating that it is
released under this License and any conditions added under section
7. This requirement modifies the requirement in section 4 to
"keep intact all notices".
c) You must license the entire work, as a whole, under this
License to anyone who comes into possession of a copy. This
License will therefore apply, along with any applicable section 7
additional terms, to the whole of the work, and all its parts,
regardless of how they are packaged. This License gives no
permission to license the work in any other way, but it does not
invalidate such permission if you have separately received it.
d) If the work has interactive user interfaces, each must display
Appropriate Legal Notices; however, if the Program has interactive
interfaces that do not display Appropriate Legal Notices, your
work need not make them do so.
A compilation of a covered work with other separate and independent
works, which are not by their nature extensions of the covered work,
and which are not combined with it such as to form a larger program,
in or on a volume of a storage or distribution medium, is called an
"aggregate" if the compilation and its resulting copyright are not
used to limit the access or legal rights of the compilation's users
beyond what the individual works permit. Inclusion of a covered work
in an aggregate does not cause this License to apply to the other
parts of the aggregate.
6. Conveying Non-Source Forms.
You may convey a covered work in object code form under the terms
of sections 4 and 5, provided that you also convey the
machine-readable Corresponding Source under the terms of this License,
in one of these ways:
a) Convey the object code in, or embodied in, a physical product
(including a physical distribution medium), accompanied by the
Corresponding Source fixed on a durable physical medium
customarily used for software interchange.
b) Convey the object code in, or embodied in, a physical product
(including a physical distribution medium), accompanied by a
written offer, valid for at least three years and valid for as
long as you offer spare parts or customer support for that product
model, to give anyone who possesses the object code either (1) a
copy of the Corresponding Source for all the software in the
product that is covered by this License, on a durable physical
medium customarily used for software interchange, for a price no
more than your reasonable cost of physically performing this
conveying of source, or (2) access to copy the
Corresponding Source from a network server at no charge.
c) Convey individual copies of the object code with a copy of the
written offer to provide the Corresponding Source. This
alternative is allowed only occasionally and noncommercially, and
only if you received the object code with such an offer, in accord
with subsection 6b.
d) Convey the object code by offering access from a designated
place (gratis or for a charge), and offer equivalent access to the
Corresponding Source in the same way through the same place at no
further charge. You need not require recipients to copy the
Corresponding Source along with the object code. If the place to
copy the object code is a network server, the Corresponding Source
may be on a different server (operated by you or a third party)
that supports equivalent copying facilities, provided you maintain
clear directions next to the object code saying where to find the
Corresponding Source. Regardless of what server hosts the
Corresponding Source, you remain obligated to ensure that it is
available for as long as needed to satisfy these requirements.
e) Convey the object code using peer-to-peer transmission, provided
you inform other peers where the object code and Corresponding
Source of the work are being offered to the general public at no
charge under subsection 6d.
A separable portion of the object code, whose source code is excluded
from the Corresponding Source as a System Library, need not be
included in conveying the object code work.
A "User Product" is either (1) a "consumer product", which means any
tangible personal property which is normally used for personal, family,
or household purposes, or (2) anything designed or sold for incorporation
into a dwelling. In determining whether a product is a consumer product,
doubtful cases shall be resolved in favor of coverage. For a particular
product received by a particular user, "normally used" refers to a
typical or common use of that class of product, regardless of the status
of the particular user or of the way in which the particular user
actually uses, or expects or is expected to use, the product. A product
is a consumer product regardless of whether the product has substantial
commercial, industrial or non-consumer uses, unless such uses represent
the only significant mode of use of the product.
"Installation Information" for a User Product means any methods,
procedures, authorization keys, or other information required to install
and execute modified versions of a covered work in that User Product from
a modified version of its Corresponding Source. The information must
suffice to ensure that the continued functioning of the modified object
code is in no case prevented or interfered with solely because
modification has been made.
If you convey an object code work under this section in, or with, or
specifically for use in, a User Product, and the conveying occurs as
part of a transaction in which the right of possession and use of the
User Product is transferred to the recipient in perpetuity or for a
fixed term (regardless of how the transaction is characterized), the
Corresponding Source conveyed under this section must be accompanied
by the Installation Information. But this requirement does not apply
if neither you nor any third party retains the ability to install
modified object code on the User Product (for example, the work has
been installed in ROM).
The requirement to provide Installation Information does not include a
requirement to continue to provide support service, warranty, or updates
for a work that has been modified or installed by the recipient, or for
the User Product in which it has been modified or installed. Access to a
network may be denied when the modification itself materially and
adversely affects the operation of the network or violates the rules and
protocols for communication across the network.
Corresponding Source conveyed, and Installation Information provided,
in accord with this section must be in a format that is publicly
documented (and with an implementation available to the public in
source code form), and must require no special password or key for
unpacking, reading or copying.
7. Additional Terms.
"Additional permissions" are terms that supplement the terms of this
License by making exceptions from one or more of its conditions.
Additional permissions that are applicable to the entire Program shall
be treated as though they were included in this License, to the extent
that they are valid under applicable law. If additional permissions
apply only to part of the Program, that part may be used separately
under those permissions, but the entire Program remains governed by
this License without regard to the additional permissions.
When you convey a copy of a covered work, you may at your option
remove any additional permissions from that copy, or from any part of
it. (Additional permissions may be written to require their own
removal in certain cases when you modify the work.) You may place
additional permissions on material, added by you to a covered work,
for which you have or can give appropriate copyright permission.
Notwithstanding any other provision of this License, for material you
add to a covered work, you may (if authorized by the copyright holders of
that material) supplement the terms of this License with terms:
a) Disclaiming warranty or limiting liability differently from the
terms of sections 15 and 16 of this License; or
b) Requiring preservation of specified reasonable legal notices or
author attributions in that material or in the Appropriate Legal
Notices displayed by works containing it; or
c) Prohibiting misrepresentation of the origin of that material, or
requiring that modified versions of such material be marked in
reasonable ways as different from the original version; or
d) Limiting the use for publicity purposes of names of licensors or
authors of the material; or
e) Declining to grant rights under trademark law for use of some
trade names, trademarks, or service marks; or
f) Requiring indemnification of licensors and authors of that
material by anyone who conveys the material (or modified versions of
it) with contractual assumptions of liability to the recipient, for
any liability that these contractual assumptions directly impose on
those licensors and authors.
All other non-permissive additional terms are considered "further
restrictions" within the meaning of section 10. If the Program as you
received it, or any part of it, contains a notice stating that it is
governed by this License along with a term that is a further
restriction, you may remove that term. If a license document contains
a further restriction but permits relicensing or conveying under this
License, you may add to a covered work material governed by the terms
of that license document, provided that the further restriction does
not survive such relicensing or conveying.
If you add terms to a covered work in accord with this section, you
must place, in the relevant source files, a statement of the
additional terms that apply to those files, or a notice indicating
where to find the applicable terms.
Additional terms, permissive or non-permissive, may be stated in the
form of a separately written license, or stated as exceptions;
the above requirements apply either way.
8. Termination.
You may not propagate or modify a covered work except as expressly
provided under this License. Any attempt otherwise to propagate or
modify it is void, and will automatically terminate your rights under
this License (including any patent licenses granted under the third
paragraph of section 11).
However, if you cease all violation of this License, then your
license from a particular copyright holder is reinstated (a)
provisionally, unless and until the copyright holder explicitly and
finally terminates your license, and (b) permanently, if the copyright
holder fails to notify you of the violation by some reasonable means
prior to 60 days after the cessation.
Moreover, your license from a particular copyright holder is
reinstated permanently if the copyright holder notifies you of the
violation by some reasonable means, this is the first time you have
received notice of violation of this License (for any work) from that
copyright holder, and you cure the violation prior to 30 days after
your receipt of the notice.
Termination of your rights under this section does not terminate the
licenses of parties who have received copies or rights from you under
this License. If your rights have been terminated and not permanently
reinstated, you do not qualify to receive new licenses for the same
material under section 10.
9. Acceptance Not Required for Having Copies.
You are not required to accept this License in order to receive or
run a copy of the Program. Ancillary propagation of a covered work
occurring solely as a consequence of using peer-to-peer transmission
to receive a copy likewise does not require acceptance. However,
nothing other than this License grants you permission to propagate or
modify any covered work. These actions infringe copyright if you do
not accept this License. Therefore, by modifying or propagating a
covered work, you indicate your acceptance of this License to do so.
10. Automatic Licensing of Downstream Recipients.
Each time you convey a covered work, the recipient automatically
receives a license from the original licensors, to run, modify and
propagate that work, subject to this License. You are not responsible
for enforcing compliance by third parties with this License.
An "entity transaction" is a transaction transferring control of an
organization, or substantially all assets of one, or subdividing an
organization, or merging organizations. If propagation of a covered
work results from an entity transaction, each party to that
transaction who receives a copy of the work also receives whatever
licenses to the work the party's predecessor in interest had or could
give under the previous paragraph, plus a right to possession of the
Corresponding Source of the work from the predecessor in interest, if
the predecessor has it or can get it with reasonable efforts.
You may not impose any further restrictions on the exercise of the
rights granted or affirmed under this License. For example, you may
not impose a license fee, royalty, or other charge for exercise of
rights granted under this License, and you may not initiate litigation
(including a cross-claim or counterclaim in a lawsuit) alleging that
any patent claim is infringed by making, using, selling, offering for
sale, or importing the Program or any portion of it.
11. Patents.
A "contributor" is a copyright holder who authorizes use under this
License of the Program or a work on which the Program is based. The
work thus licensed is called the contributor's "contributor version".
A contributor's "essential patent claims" are all patent claims
owned or controlled by the contributor, whether already acquired or
hereafter acquired, that would be infringed by some manner, permitted
by this License, of making, using, or selling its contributor version,
but do not include claims that would be infringed only as a
consequence of further modification of the contributor version. For
purposes of this definition, "control" includes the right to grant
patent sublicenses in a manner consistent with the requirements of
this License.
Each contributor grants you a non-exclusive, worldwide, royalty-free
patent license under the contributor's essential patent claims, to
make, use, sell, offer for sale, import and otherwise run, modify and
propagate the contents of its contributor version.
In the following three paragraphs, a "patent license" is any express
agreement or commitment, however denominated, not to enforce a patent
(such as an express permission to practice a patent or covenant not to
sue for patent infringement). To "grant" such a patent license to a
party means to make such an agreement or commitment not to enforce a
patent against the party.
If you convey a covered work, knowingly relying on a patent license,
and the Corresponding Source of the work is not available for anyone
to copy, free of charge and under the terms of this License, through a
publicly available network server or other readily accessible means,
then you must either (1) cause the Corresponding Source to be so
available, or (2) arrange to deprive yourself of the benefit of the
patent license for this particular work, or (3) arrange, in a manner
consistent with the requirements of this License, to extend the patent
license to downstream recipients. "Knowingly relying" means you have
actual knowledge that, but for the patent license, your conveying the
covered work in a country, or your recipient's use of the covered work
in a country, would infringe one or more identifiable patents in that
country that you have reason to believe are valid.
If, pursuant to or in connection with a single transaction or
arrangement, you convey, or propagate by procuring conveyance of, a
covered work, and grant a patent license to some of the parties
receiving the covered work authorizing them to use, propagate, modify
or convey a specific copy of the covered work, then the patent license
you grant is automatically extended to all recipients of the covered
work and works based on it.
A patent license is "discriminatory" if it does not include within
the scope of its coverage, prohibits the exercise of, or is
conditioned on the non-exercise of one or more of the rights that are
specifically granted under this License. You may not convey a covered
work if you are a party to an arrangement with a third party that is
in the business of distributing software, under which you make payment
to the third party based on the extent of your activity of conveying
the work, and under which the third party grants, to any of the
parties who would receive the covered work from you, a discriminatory
patent license (a) in connection with copies of the covered work
conveyed by you (or copies made from those copies), or (b) primarily
for and in connection with specific products or compilations that
contain the covered work, unless you entered into that arrangement,
or that patent license was granted, prior to 28 March 2007.
Nothing in this License shall be construed as excluding or limiting
any implied license or other defenses to infringement that may
otherwise be available to you under applicable patent law.
12. No Surrender of Others' Freedom.
If conditions are imposed on you (whether by court order, agreement or
otherwise) that contradict the conditions of this License, they do not
excuse you from the conditions of this License. If you cannot convey a
covered work so as to satisfy simultaneously your obligations under this
License and any other pertinent obligations, then as a consequence you may
not convey it at all. For example, if you agree to terms that obligate you
to collect a royalty for further conveying from those to whom you convey
the Program, the only way you could satisfy both those terms and this
License would be to refrain entirely from conveying the Program.
13. Remote Network Interaction; Use with the GNU General Public License.
Notwithstanding any other provision of this License, if you modify the
Program, your modified version must prominently offer all users
interacting with it remotely through a computer network (if your version
supports such interaction) an opportunity to receive the Corresponding
Source of your version by providing access to the Corresponding Source
from a network server at no charge, through some standard or customary
means of facilitating copying of software. This Corresponding Source
shall include the Corresponding Source for any work covered by version 3
of the GNU General Public License that is incorporated pursuant to the
following paragraph.
Notwithstanding any other provision of this License, you have
permission to link or combine any covered work with a work licensed
under version 3 of the GNU General Public License into a single
combined work, and to convey the resulting work. The terms of this
License will continue to apply to the part which is the covered work,
but the work with which it is combined will remain governed by version
3 of the GNU General Public License.
14. Revised Versions of this License.
The Free Software Foundation may publish revised and/or new versions of
the GNU Affero General Public License from time to time. Such new versions
will be similar in spirit to the present version, but may differ in detail to
address new problems or concerns.
Each version is given a distinguishing version number. If the
Program specifies that a certain numbered version of the GNU Affero General
Public License "or any later version" applies to it, you have the
option of following the terms and conditions either of that numbered
version or of any later version published by the Free Software
Foundation. If the Program does not specify a version number of the
GNU Affero General Public License, you may choose any version ever published
by the Free Software Foundation.
If the Program specifies that a proxy can decide which future
versions of the GNU Affero General Public License can be used, that proxy's
public statement of acceptance of a version permanently authorizes you
to choose that version for the Program.
Later license versions may give you additional or different
permissions. However, no additional obligations are imposed on any
author or copyright holder as a result of your choosing to follow a
later version.
15. Disclaimer of Warranty.
THERE IS NO WARRANTY FOR THE PROGRAM, TO THE EXTENT PERMITTED BY
APPLICABLE LAW. EXCEPT WHEN OTHERWISE STATED IN WRITING THE COPYRIGHT
HOLDERS AND/OR OTHER PARTIES PROVIDE THE PROGRAM "AS IS" WITHOUT WARRANTY
OF ANY KIND, EITHER EXPRESSED OR IMPLIED, INCLUDING, BUT NOT LIMITED TO,
THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR
PURPOSE. THE ENTIRE RISK AS TO THE QUALITY AND PERFORMANCE OF THE PROGRAM
IS WITH YOU. SHOULD THE PROGRAM PROVE DEFECTIVE, YOU ASSUME THE COST OF
ALL NECESSARY SERVICING, REPAIR OR CORRECTION.
16. Limitation of Liability.
IN NO EVENT UNLESS REQUIRED BY APPLICABLE LAW OR AGREED TO IN WRITING
WILL ANY COPYRIGHT HOLDER, OR ANY OTHER PARTY WHO MODIFIES AND/OR CONVEYS
THE PROGRAM AS PERMITTED ABOVE, BE LIABLE TO YOU FOR DAMAGES, INCLUDING ANY
GENERAL, SPECIAL, INCIDENTAL OR CONSEQUENTIAL DAMAGES ARISING OUT OF THE
USE OR INABILITY TO USE THE PROGRAM (INCLUDING BUT NOT LIMITED TO LOSS OF
DATA OR DATA BEING RENDERED INACCURATE OR LOSSES SUSTAINED BY YOU OR THIRD
PARTIES OR A FAILURE OF THE PROGRAM TO OPERATE WITH ANY OTHER PROGRAMS),
EVEN IF SUCH HOLDER OR OTHER PARTY HAS BEEN ADVISED OF THE POSSIBILITY OF
SUCH DAMAGES.
17. Interpretation of Sections 15 and 16.
If the disclaimer of warranty and limitation of liability provided
above cannot be given local legal effect according to their terms,
reviewing courts shall apply local law that most closely approximates
an absolute waiver of all civil liability in connection with the
Program, unless a warranty or assumption of liability accompanies a
copy of the Program in return for a fee.
END OF TERMS AND CONDITIONS
How to Apply These Terms to Your New Programs
If you develop a new program, and you want it to be of the greatest
possible use to the public, the best way to achieve this is to make it
free software which everyone can redistribute and change under these terms.
To do so, attach the following notices to the program. It is safest
to attach them to the start of each source file to most effectively
state the exclusion of warranty; and each file should have at least
the "copyright" line and a pointer to where the full notice is found.
<one line to give the program's name and a brief idea of what it does.>
Copyright (C) <year> <name of author>
This program is free software: you can redistribute it and/or modify
it under the terms of the GNU Affero General Public License as published by
the Free Software Foundation, either version 3 of the License, or
(at your option) any later version.
This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
GNU Affero General Public License for more details.
You should have received a copy of the GNU Affero General Public License
along with this program. If not, see <https://www.gnu.org/licenses/>.
Also add information on how to contact you by electronic and paper mail.
If your software can interact with users remotely through a computer
network, you should also make sure that it provides a way for users to
get its source. For example, if your program is a web application, its
interface could display a "Source" link that leads users to an archive
of the code. There are many ways you could offer source, and different
solutions will be better for different programs; see section 13 for the
specific requirements.
You should also get your employer (if you work as a programmer) or school,
if any, to sign a "copyright disclaimer" for the program, if necessary.
For more information on this, and how to apply and follow the GNU AGPL, see
<https://www.gnu.org/licenses/>.
+22
View File
@@ -0,0 +1,22 @@
The MIT License (MIT)
Copyright (c) 2025 sigoden
Copyright (c) 2025 Alexander J. Clarke
Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:
The above copyright notice and this permission notice shall be included in all
copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
SOFTWARE.
+24
View File
@@ -0,0 +1,24 @@
Coyote
Copyright (c) 2025 Alexander J. Clarke
This project as a whole is licensed under the GNU Affero General Public
License, version 3.0 only (AGPL-3.0-only). The full text of that license is
provided in the LICENSE file.
--------------------------------------------------------------------------------
Upstream / third-party notices
--------------------------------------------------------------------------------
Coyote began as a fork of AIChat (https://github.com/sigoden/aichat),
Copyright (c) sigoden, which is distributed under the MIT License. Substantial
portions of Coyote are derived from AIChat and remain available under the terms
of the MIT License. The MIT License text and its required copyright and
permission notices are preserved in the LICENSE-MIT file.
As permitted by the MIT License, these portions have been incorporated into a
larger work that is distributed under the AGPL-3.0-only license. When you
receive Coyote as a combined work, your rights and obligations for the work as
a whole are governed by the AGPL-3.0-only license; the MIT notice is retained
to satisfy the attribution requirements of the MIT-licensed portions.
See CREDITS.md for additional background and attribution.
+16
View File
@@ -6,6 +6,7 @@
![Crate.io downloads](https://img.shields.io/crates/d/coyote-ai?label=Crate%20downloads) ![Crate.io downloads](https://img.shields.io/crates/d/coyote-ai?label=Crate%20downloads)
[![GitHub Downloads](https://img.shields.io/github/downloads/Dark-Alex-17/coyote/total.svg?label=GitHub%20downloads)](https://github.com/Dark-Alex-17/coyote/releases) [![GitHub Downloads](https://img.shields.io/github/downloads/Dark-Alex-17/coyote/total.svg?label=GitHub%20downloads)](https://github.com/Dark-Alex-17/coyote/releases)
![Docker pulls](https://img.shields.io/docker/pulls/darkalex17/coyote?label=Docker%20downloads) ![Docker pulls](https://img.shields.io/docker/pulls/darkalex17/coyote?label=Docker%20downloads)
[![License: AGPL v3](https://img.shields.io/badge/License-AGPL_v3-blue.svg)](./LICENSE)
Coyote is an all-in-one, batteries-included, LLM CLI tool featuring Shell Assistant, CLI & REPL Mode, RAG, AI Tools & Coyote is an all-in-one, batteries-included, LLM CLI tool featuring Shell Assistant, CLI & REPL Mode, RAG, AI Tools &
Agents, and More. Agents, and More.
@@ -69,6 +70,8 @@ Coyote requires the following tools to be installed on your system:
* **Cargo:** `cargo install ast-grep --locked` * **Cargo:** `cargo install ast-grep --locked`
* **npm:** `npm i -g @ast-grep/cli` * **npm:** `npm i -g @ast-grep/cli`
* Optional: if `ast-grep` is not installed, the `ast_grep` tool reports it and agents fall back to `fs_grep` * Optional: if `ast-grep` is not installed, the `ast_grep` tool reports it and agents fall back to `fs_grep`
* [duckdb](https://duckdb.org/) (for fast, local RAGs)
* `curl https://install.duckdb.org | sh`
These tools are used to provide various functionalities within Coyote, such as document processing, JSON manipulation, These tools are used to provide various functionalities within Coyote, such as document processing, JSON manipulation,
etc., and they are used within agents and tools. etc., and they are used within agents and tools.
@@ -331,3 +334,16 @@ See [CREDITS.md](./CREDITS.md) for full attribution and background.
## Creator ## Creator
* [Alex Clarke](https://github.com/Dark-Alex-17) * [Alex Clarke](https://github.com/Dark-Alex-17)
---
## License
Coyote is licensed under the [GNU Affero General Public License v3.0](./LICENSE)
(AGPL-3.0-only).
Coyote began as a fork of [AIChat](https://github.com/sigoden/aichat)
(Copyright (c) sigoden), which is licensed under the MIT License. Substantial
portions of Coyote are derived from AIChat and remain available under the MIT
License, preserved in [LICENSE-MIT](./LICENSE-MIT). See [NOTICE](./NOTICE) and
[CREDITS.md](./CREDITS.md) for details.
+97 -21
View File
@@ -40,15 +40,57 @@ _write_project_cache() {
_detect_heuristic() { _detect_heuristic() {
local dir="$1" local dir="$1"
local runner="" runner_type="" runner_targets=""
if [[ -f "${dir}/Taskfile.yml" || -f "${dir}/Taskfile.yaml" || -f "${dir}/taskfile.yml" || -f "${dir}/taskfile.yaml" ]]; then
runner="task" runner_type="taskfile"
runner_targets=$( (cd "${dir}" && task --list-all 2>/dev/null | sed -n 's/^\* \([^:[:space:]]*\):.*/\1/p') || true)
elif [[ -f "${dir}/justfile" || -f "${dir}/Justfile" ]]; then
runner="just" runner_type="just"
runner_targets=$( (cd "${dir}" && just --summary 2>/dev/null | tr ' ' '\n') || true)
elif [[ -f "${dir}/Makefile" || -f "${dir}/makefile" || -f "${dir}/GNUmakefile" ]]; then
runner="make" runner_type="make"
local mk mkfiles=()
for mk in Makefile makefile GNUmakefile; do
[[ -f "${dir}/${mk}" ]] && mkfiles+=("${dir}/${mk}")
done
runner_targets=$(sed -n 's/^\([A-Za-z0-9_][A-Za-z0-9_.-]*\):\([^=].*\|\)$/\1/p' "${mkfiles[@]}" 2>/dev/null | sort -u || true)
fi
if [[ -n "${runner}" && -n "${runner_targets}" ]]; then
_pick_target() {
local c
for c in "$@"; do
if grep -qx "${c}" <<<"${runner_targets}"; then
echo "${runner} ${c}"
return 0
fi
done
echo ""
}
local r_build r_test r_check r_lint r_fmt
r_build=$(_pick_target build compile)
r_test=$(_pick_target test tests unit)
r_check=$(_pick_target check vet typecheck build)
r_lint=$(_pick_target lint fmt-check)
r_fmt=$(_pick_target fmt format)
if [[ -n "${r_build}${r_test}${r_check}${r_lint}${r_fmt}" ]]; then
echo "{\"type\":\"${runner_type}\",\"build\":\"${r_build}\",\"test\":\"${r_test}\",\"check\":\"${r_check}\",\"lint\":\"${r_lint}\",\"fmt\":\"${r_fmt}\"}"
return 0
fi
fi
# Rust # Rust
if [[ -f "${dir}/Cargo.toml" ]]; then if [[ -f "${dir}/Cargo.toml" ]]; then
echo '{"type":"rust","build":"cargo build","test":"cargo test","check":"cargo check"}' echo '{"type":"rust","build":"cargo build","test":"cargo test","check":"cargo check","lint":"cargo clippy --no-deps -- -D warnings","fmt":"cargo fmt"}'
return 0 return 0
fi fi
# Go # Go
if [[ -f "${dir}/go.mod" ]]; then if [[ -f "${dir}/go.mod" ]]; then
echo '{"type":"go","build":"go build ./...","test":"go test ./...","check":"go vet ./..."}' local go_lint=""
if compgen -G "${dir}/.golangci.*" &>/dev/null && command -v golangci-lint &>/dev/null; then
go_lint="golangci-lint run"
fi
echo "{\"type\":\"go\",\"build\":\"go build ./...\",\"test\":\"go test ./...\",\"check\":\"go vet ./...\",\"lint\":\"${go_lint}\",\"fmt\":\"gofmt -w .\"}"
return 0 return 0
fi fi
@@ -65,7 +107,25 @@ _detect_heuristic() {
[[ -f "${dir}/pnpm-lock.yaml" ]] && pm="pnpm" [[ -f "${dir}/pnpm-lock.yaml" ]] && pm="pnpm"
[[ -f "${dir}/yarn.lock" ]] && pm="yarn" [[ -f "${dir}/yarn.lock" ]] && pm="yarn"
echo "{\"type\":\"nodejs\",\"build\":\"${pm} run build\",\"test\":\"${pm} test\",\"check\":\"${pm} run lint\"}" # Emit only scripts the manifest actually declares (same introspection
# contract as the runner tier: never guess a target into existence).
_pkg_script() {
local s
for s in "$@"; do
if jq -e --arg s "$s" '.scripts[$s] // empty' "${dir}/package.json" &>/dev/null; then
echo "${pm} run ${s}"
return 0
fi
done
echo ""
}
local p_build p_test p_check p_lint p_fmt
p_build=$(_pkg_script build compile)
p_test=$(_pkg_script test)
p_check=$(_pkg_script check typecheck tsc)
p_lint=$(_pkg_script lint)
p_fmt=$(_pkg_script fmt format prettier)
echo "{\"type\":\"nodejs\",\"build\":\"${p_build}\",\"test\":\"${p_test}\",\"check\":\"${p_check}\",\"lint\":\"${p_lint}\",\"fmt\":\"${p_fmt}\"}"
return 0 return 0
fi fi
@@ -82,7 +142,7 @@ _detect_heuristic() {
check_cmd="uv run ruff check ." check_cmd="uv run ruff check ."
fi fi
echo "{\"type\":\"python\",\"build\":\"\",\"test\":\"${test_cmd}\",\"check\":\"${check_cmd}\"}" echo "{\"type\":\"python\",\"build\":\"\",\"test\":\"${test_cmd}\",\"check\":\"${check_cmd}\",\"lint\":\"${check_cmd}\",\"fmt\":\"ruff format .\"}"
return 0 return 0
fi fi
@@ -144,17 +204,6 @@ _detect_heuristic() {
return 0 return 0
fi fi
# Generic build systems (last resort before LLM)
if [[ -f "${dir}/justfile" ]] || [[ -f "${dir}/Justfile" ]]; then
echo '{"type":"just","build":"just build","test":"just test","check":"just lint"}'
return 0
fi
if [[ -f "${dir}/Makefile" ]] || [[ -f "${dir}/makefile" ]] || [[ -f "${dir}/GNUmakefile" ]]; then
echo '{"type":"make","build":"make build","test":"make test","check":"make lint"}'
return 0
fi
return 1 return 1
} }
@@ -218,7 +267,9 @@ _detect_with_llm() {
local prompt local prompt
prompt=$(cat <<-EOF prompt=$(cat <<-EOF
Analyze this project directory and determine the project type, primary language, and the correct shell commands to build, test, and check (lint/typecheck) it. Analyze this project directory and determine the project type, primary language, and the correct shell commands to build, test, check (typecheck/vet), lint, and format it.
PRIORITY RULE: if the project declares its own task-runner interface (a Taskfile, justfile, Makefile, package.json scripts, or similar), those declared targets ARE the correct commands — prefer them over generic ecosystem defaults, and never invent a target the interface does not declare.
EOF EOF
) )
@@ -226,12 +277,12 @@ _detect_with_llm() {
prompt+=$(cat <<-EOF prompt+=$(cat <<-EOF
Respond with ONLY a valid JSON object. No markdown fences, no explanation, no extra text. Respond with ONLY a valid JSON object. No markdown fences, no explanation, no extra text.
The JSON must have exactly these 4 keys: The JSON must have exactly these 6 keys:
{"type":"<language>","build":"<build command>","test":"<test command>","check":"<lint or typecheck command>"} {"type":"<language>","build":"<build command>","test":"<test command>","check":"<typecheck/vet command>","lint":"<lint command>","fmt":"<format command>"}
Rules: Rules:
- "type" must be a single lowercase word (e.g. rust, go, python, nodejs, java, ruby, elixir, cpp, c, zig, haskell, scala, kotlin, dart, swift, php, dotnet, etc.) - "type" must be a single lowercase word (e.g. rust, go, python, nodejs, java, ruby, elixir, cpp, c, zig, haskell, scala, kotlin, dart, swift, php, dotnet, etc.)
- If a command doesn't apply to this project, use an empty string, "" - If a command doesn't apply to this project, use an empty string, "" — NEVER guess a command that might not exist; a wrongly-guessed command is worse than an empty one
- Use the most standard/common commands for the detected ecosystem - Use the most standard/common commands for the detected ecosystem
- If you detect a package manager lockfile, use that package manager (e.g. pnpm over npm) - If you detect a package manager lockfile, use that package manager (e.g. pnpm over npm)
EOF EOF
@@ -244,7 +295,7 @@ _detect_with_llm() {
llm_response=$(echo "${llm_response}" | grep -o '{[^}]*}' | head -1) llm_response=$(echo "${llm_response}" | grep -o '{[^}]*}' | head -1)
if echo "${llm_response}" | jq -e '.type and .build != null and .test != null and .check != null' &>/dev/null; then if echo "${llm_response}" | jq -e '.type and .build != null and .test != null and .check != null' &>/dev/null; then
echo "${llm_response}" | jq -c '{type: (.type // "unknown"), build: (.build // ""), test: (.test // ""), check: (.check // "")}' echo "${llm_response}" | jq -c '{type: (.type // "unknown"), build: (.build // ""), test: (.test // ""), check: (.check // ""), lint: (.lint // ""), fmt: (.fmt // "")}'
return 0 return 0
fi fi
@@ -258,7 +309,7 @@ detect_project() {
local cached local cached
if cached=$(_read_project_cache "${dir}"); then if cached=$(_read_project_cache "${dir}"); then
echo "${cached}" | jq -c '{type, build, test, check}' echo "${cached}" | jq -c '{type, build, test, check, lint: (.lint // ""), fmt: (.fmt // "")}'
return 0 return 0
fi fi
@@ -286,6 +337,31 @@ detect_project() {
echo '{"type":"unknown","build":"","test":"","check":""}' echo '{"type":"unknown","build":"","test":"","check":""}'
} }
# resolve_gate_dir maps a workspace root to the directory verification gates
# must run in. A delivery-repo worker's workspace root holds only dotfiles
# plus the clone, so gates aimed at the root detect nothing and silently
# no-op. When the root has no project markers and exactly ONE first-level
# git repo exists, gates run inside it; anything ambiguous stays at the root.
resolve_gate_dir() {
local dir="${1:-.}"
local m
for m in Taskfile.yml Taskfile.yaml taskfile.yml Cargo.toml go.mod package.json pyproject.toml setup.py pom.xml build.gradle mix.exs Gemfile composer.json Makefile justfile Justfile CMakeLists.txt; do
if [[ -e "${dir}/${m}" ]]; then
echo "${dir}"
return 0
fi
done
local repos=() d
for d in "${dir}"/*/; do
[[ -d "${d}/.git" ]] && repos+=("${d}")
done
if [[ ${#repos[@]} -eq 1 ]]; then
echo "${repos[0]%/}"
return 0
fi
echo "${dir}"
}
########################### ###########################
## FILE SEARCH UTILITIES ## ## FILE SEARCH UTILITIES ##
########################### ###########################
+1 -1
View File
@@ -50,7 +50,7 @@ instructions: |
You are given: You are given:
1. **The diff** — pasted inline, or run `get_diff` (optionally `--base <ref>`) if told to fetch it. 1. **The diff** — pasted inline, or run `get_diff` (optionally `--base <ref>`) if told to fetch it.
2. **The plan** — the task's Objective, Tasks, and especially its **Acceptance criteria**, pasted 2. **The plan** — the task's Objective, Tasks, and especially its **Acceptance criteria**, pasted
inline (e.g. a BCP task `index.md` body + the relevant `PLAN-*.md` section), or a path to read. inline (e.g. a task file's What/Steps/Acceptance criteria + the relevant plan section), or a path to read.
If the plan / acceptance criteria are missing, STOP and say so: conformance cannot be judged If the plan / acceptance criteria are missing, STOP and say so: conformance cannot be judged
without a spec. Do not invent criteria or guess intent. without a spec. Do not invent criteria or guess intent.
+182
View File
@@ -0,0 +1,182 @@
# Architect
A **design-doc orchestrator for any project**. Give it one high-level design doc; it decomposes the
doc into a quality-gated plan and ~1-engineer-day task files, spawns **one
[Sisyphus](../sisyphus/README.md) per task** on a single run branch, verifies each task with an
adversarial plan-conformance check, and finishes with **one draft PR** (CI checks watched to green)
plus tracked follow-up tasks for the manual work the code can't do for itself.
Architect does **not** write feature code itself. It owns the *process*; Sisyphus owns each *task*.
## The pipeline it drives
```mermaid
flowchart TD
user([Design doc]) --> architect["Architect<br/>design-doc orchestrator"]
architect --> orient["Phase A — Orient<br/>project conventions · build/test commands · design doc"]
orient --> design["Phase B — design-session<br/>plans_dir/PLAN-&lt;slug&gt;.md + 1-day task breakdown"]
design -. "grounding" .-> explore[["explore<br/>codebase grep<br/>× parallel"]]
design -. "unfamiliar libraries" .-> librarian[["librarian<br/>docs + OSS grep"]]
explore -. "findings ground<br/>the breakdown" .-> design
librarian -. "findings ground<br/>the breakdown" .-> design
design --> gatekeeper[["gatekeeper<br/>self-containedness audit<br/>(docker-container test)"]]
gatekeeper --> g1{"PLAN_GATE?"}
g1 -->|"LEAKY (≤ 2 cycles)"| amend["Answer the missing questions<br/>via explore / librarian / docs<br/>(user__ask only for business rules)<br/>→ amend the plan"]
amend --> gatekeeper
g1 -->|"LEAKY after 2 cycles"| escalate
g1 -->|"SEALED"| oracle[["oracle<br/>plan-review<br/>(executability)"]]
oracle --> g2{"PLAN_REVIEW?"}
g2 -->|"REJECT — fix complaints,<br/>re-submit SAME session"| oracle
g2 -->|"OKAY"| tasks["Phase D — materialize tasks<br/>plans_dir/tasks/TASK-NNN-*/ (task-tracking)"]
tasks --> branch["Phase E — run branch<br/>feat/PLAN-&lt;slug&gt; off base_branch"]
branch --> claim["Claim task (sequential, dependency order)<br/>status: in-progress + base SHA"]
claim --> sisyphus[["sisyphus<br/>implement ONE task on the run branch<br/>commit + push — NO PR"]]
sisyphus --> adversary[["adversary<br/>conformance check<br/>diff vs task base SHA"]]
adversary --> verdict{"ADVERSARIAL_REVIEW?"}
verdict -->|"DIVERGES — resume<br/>SAME sisyphus session (once)"| sisyphus
verdict -->|"still DIVERGES"| escalate
verdict -->|"CONFORMS"| taskdone["Close task<br/>status: complete · log commits + follow-ups"]
taskdone --> more{"More tasks?"}
more -->|"yes"| claim
more -->|"no"| finish["Phase F — full build + tests<br/>on the integrated run branch"]
finish --> pr["ONE DRAFT PR: run branch → base_branch<br/>(never marked ready — user reviews first)<br/>body: task checklist + Follow-up / manual actions"]
pr --> checks{"PR runs/checks<br/>green?"}
checks -->|"failure — resume responsible<br/>sisyphus session, fix, push"| checks
checks -->|"external flake /<br/>broken base branch"| escalate
checks -->|"green"| followups["Create follow-up task files<br/>(type: followup, pending)<br/>→ picked up by the user post-merge"]
followups --> backfill["Backfill PR link into PLAN + task logs<br/>PLAN status: implemented"]
backfill --> validate["task-tracking consistency checks"]
validate --> done([Run complete])
escalate([user__ask — escalate to user])
branch -. "parallel_tasks=1 (opt-in):<br/>per-task worktrees + task branches,<br/>merged one at a time with<br/>integration tests after every merge" .-> claim
```
## Where state lives
Everything is file-based in **`plans_dir`** (default `plans/`, resolved against the project):
```
<plans_dir>/
PLAN-<slug>.md # problem / approach / alternatives / task breakdown
tasks/TASK-NNN-<slug>/
index.md # What / Steps / Acceptance criteria; status in frontmatter
log.md # append-only audit trail (branch, commits, follow-ups, PR)
```
- `plans_dir` **inside the repo** (default) → planning files ride the run branch and land in the PR
(self-documenting review).
- `plans_dir` **absolute, outside the repo** (e.g. a common runs directory) → nothing planning-related
is ever committed.
Disk is the durable store: task statuses, logs, and follow-ups survive context compression; chat
history does not.
## The three review gates
| Gate | Agent | Question | When |
|------|-------|----------|------|
| Self-containedness | [`gatekeeper`](../gatekeeper/README.md) | "Can a context-free LLM implement from this plan alone?" | Before tasks exist |
| Executability | `oracle` + `plan-review` | "Is the approach sound, verifiable, correctly ordered?" | After sealing |
| Conformance | [`adversary`](../adversary/README.md) | "Is the built code what the plan asked for?" | After each task |
## Key conventions it enforces
- **One task = one engineer-day** — anything larger gets decomposed at the design stage.
- **Task state on disk** — `status:` frontmatter lifecycle per the `task-tracking` skill; no state
lives only in chat.
- **One run branch, one draft PR** — `feat/PLAN-<slug>` off `base_branch`; the PR is never opened
per-task, never non-draft, never marked ready-for-review (you flip it yourself).
- **CI checks watched to green** — failures are routed back to the responsible Sisyphus session; the
run isn't done with red or pending checks.
- **No plan references in code comments** — comments never cite the design doc, plan, phases, steps,
or TASK numbers (docs drift; comments rot). Plan references live in commit messages only.
- **`.env` never lands in a repo** — only `.env.example` with placeholder keys; real values become a
follow-up.
- **Follow-ups are tracked, never dropped** — every manual action (secrets, cloud roles, console
steps, cross-repo changes) is reported per task, logged durably, rolled into the PR's
`## Follow-up / manual actions` section (pre-merge items first), and materialized as
`type: followup` task files for you to pick up post-merge.
## Usage
```sh
# From the target project root (default autonomy: full)
coyote -a architect --agent-variable design_doc docs/design/my-feature.md \
"Implement this design doc end to end"
# Approve the task breakdown once, then run autonomously
coyote -a architect \
--agent-variable design_doc docs/design/my-feature.md \
--agent-variable autonomy plan-gate \
"Decompose and implement"
# Different project / plans outside the repo / PR against a non-main base
coyote -a architect \
--agent-variable project_dir ~/code/my-service \
--agent-variable plans_dir ~/architect-runs/my-service \
--agent-variable base_branch develop \
--agent-variable design_doc ~/docs/big-refactor.md \
"Run the pipeline"
```
### Variables
| Variable | Default | Meaning |
|----------|---------|---------|
| `project_dir` | `.` | The target repo — the only WRITE target for feature code. |
| `plans_dir` | `plans` | Where PLAN + task files live. Relative → in-repo (rides the PR); absolute → outside git. |
| `design_doc` | *(empty)* | Path to the design doc; asked for if unset. |
| `base_branch` | `main` | Branch the run branch forks from and the PR targets. |
| `autonomy` | `full` | `full` (no gates) · `plan-gate` (approve breakdown once) · `phase-gate` (approve each task). |
| `parallel_tasks` | `0` | `0` = sequential (default) · `1` = opt-in worktree-parallel execution for eligible tasks. |
| `auto_confirm` | `1` | Skip the shell confirm guard (needed for non-interactive autonomous runs). |
## Autonomy
Fully autonomous end-to-end by default — it halts only for genuine blockers: scope-changing
ambiguity or unresolved design questions, a task that fails after Sisyphus's own recovery (consults
Oracle, then escalates), and any destructive/irreversible action. Use `plan-gate` or `phase-gate`
to insert approval checkpoints.
## Parallel task execution (opt-in)
By default (`parallel_tasks: 0`) tasks run **sequentially** on the single run branch. Setting
`parallel_tasks: 1` enables worktree-based parallelism:
- Eligible tasks (mutually unblocked, plan-declared file-disjoint, max 3 concurrent) each get an
isolated `git worktree` + task branch forked from the run branch tip.
- Tasks touching **migrations, generated code, or dependency manifests/lockfiles** are never
parallel-eligible — shared hotspots collide even when the plan calls tasks independent.
- Architect integrates: completed task branches merge into the run branch **one at a time**, with a
full build + test run after every merge. Conflicts go back to that task's Sisyphus session to
rebase and re-verify.
- Worktrees and task branches are cleaned up after each clean merge. Phase F (single draft PR +
CI-check watch) is unchanged in both modes.
## Sub-agents it spawns
| Agent | Used for |
|-------|----------|
| [`sisyphus`](../sisyphus/README.md) | Implement ONE task's code (its own explore→coder→verify→review loop). One per task. |
| [`gatekeeper`](../gatekeeper/README.md) | Plan self-containedness gate (`PLAN_GATE: SEALED/LEAKY`). |
| [`adversary`](../adversary/README.md) | Per-task plan-conformance verdict (`ADVERSARIAL_REVIEW: CONFORMS/DIVERGES`). |
| [`oracle`](../oracle/README.md) | Plan review (`plan-review`); diagnosis when a task fails after Sisyphus recovery. |
| [`explore`](../explore/README.md) | Ground the design/plan in real code; read other local repos for library usage and call sites. |
| [`librarian`](../librarian/README.md) | External docs / OSS examples for unfamiliar libraries. |
## Related skills
- [`design-session`](../../skills/design-session/SKILL.md) — design doc → grounded proposal → PLAN + sized breakdown.
- [`task-tracking`](../../skills/task-tracking/SKILL.md) — the task-file schema, lifecycle, and consistency checks.
- [`plan-gatekeeping`](../../skills/plan-gatekeeping/SKILL.md) — the gatekeeper's self-containedness manifest.
- [`plan-authoring`](../../skills/plan-authoring/SKILL.md) / [`plan-review`](../../skills/plan-review/SKILL.md) — plan schema + oracle's executability review.
- [`adversarial-review`](../../skills/adversarial-review/SKILL.md) — the adversary's conformance methodology.
+460
View File
@@ -0,0 +1,460 @@
name: architect
description: |
Design-doc orchestrator for any project. Consumes a high-level design doc, decomposes it into a
gated plan (gatekeeper self-containedness + oracle plan-review) and ~1-engineer-day task files,
spawns one Sisyphus per task on a single run branch, verifies each with an adversarial
plan-conformance check, and finishes with ONE draft PR (CI checks watched to green) plus tracked
follow-up tasks. Task state lives on disk in a plans directory, so runs survive context compression.
version: 2.0.0
agent_session: temp
auto_continue: true
max_auto_continues: 100
inject_todo_instructions: true
can_spawn_agents: true
spawnable_agents:
- sisyphus
- oracle
- explore
- librarian
- adversary
- gatekeeper
max_concurrent_agents: 10
max_agent_depth: 10
inject_spawn_instructions: true
summarization_threshold: 100000
skills_enabled: true
enabled_skills:
- design-session
- task-tracking
- plan-authoring
- delegation-protocol
- git-master
- parallel-research
variables:
- name: project_dir
description: Absolute path to the target project repo — the ONLY write target for feature code
default: '.'
- name: plans_dir
description: Where the PLAN file and task dirs live. Relative paths resolve against project_dir (and then ride the run branch into the PR); an absolute path outside the repo keeps planning files out of git entirely.
default: 'plans'
- name: design_doc
description: Path to the high-level design doc to implement (absolute, or relative to project_dir)
default: ''
- name: base_branch
description: The branch the run branch forks from and the PR targets
default: 'main'
- name: autonomy
description: 'How autonomous the run is: full (no gates), plan-gate (approve breakdown once, then autonomous), phase-gate (approve each task)'
default: full
- name: auto_confirm
description: Auto-confirm command execution (1 = skip the shell guard_operation TTY prompt, needed for non-interactive autonomous runs)
default: '1'
- name: parallel_tasks
description: 'Opt-in worktree-based parallel task execution: 0 = sequential (default, one task at a time on the run branch), 1 = eligible tasks run as concurrent Sisyphus agents in isolated git worktrees, merged back one at a time'
default: '0'
global_tools:
- ast_grep.sh
- fs_read.sh
- fs_grep.sh
- fs_glob.sh
- fs_ls.sh
- fs_write.sh
- fs_patch.sh
- fs_mkdir.sh
- execute_command.sh
instructions: |
You are **Architect** — an orchestrator that takes a single high-level design doc and drives it
end-to-end to implementation on ANY project. You do NOT write feature code yourself. You decompose,
gate the plan, delegate one task to one **Sisyphus** sub-agent, verify conformance, track state on
disk, and finish with a single draft PR — repeating until the entire design doc is implemented.
## Ground rules — READ BEFORE ANYTHING
**Write target.** ALL feature code goes in {{project_dir}}. You and your sub-agents MAY freely READ
other local repos/directories (internal libraries, legacy patterns, call sites, shared contracts)
— reading is encouraged; WRITING anywhere but {{project_dir}} is a scope violation. If the design
genuinely requires writing outside {{project_dir}}, STOP and escalate; likely it's a follow-up.
**Git model — one run branch, one draft PR.** All work lands on a single RUN BRANCH
(`feat/PLAN-<slug>`, forked from {{base_branch}}), and exactly ONE DRAFT PR is opened at the END of
the run (Phase F) covering the entire design doc — NEVER one PR per task, NEVER a push to
{{base_branch}}. Before any `git push`/branch/PR, confirm you are in {{project_dir}}
(`git remote get-url origin`).
**Task state lives on disk.** {{plans_dir}} (relative → resolved against {{project_dir}}, riding
the run branch into the PR; absolute → outside git entirely) holds `PLAN-<slug>.md` and
`tasks/TASK-NNN-*/`. The `task-tracking` skill defines the schema and lifecycle — load it before
touching task files. Disk is your durable store; chat history is not.
**Read the project's own conventions at startup** — `CLAUDE.md` / `AGENTS.md` / `CONTRIBUTING.md`
at the project root. When this prompt and those files disagree on project conventions, the
project's files win; note the discrepancy to the user.
## Autonomy mode: {{autonomy}}
- **full** — run the entire pipeline with no approval gates. Only stop for a genuine blocker
(ambiguity that changes scope, a task that fails after Sisyphus's own recovery, missing critical
info, any destructive action). This is the default.
- **plan-gate** — after the breakdown is SEALED + OKAY'd, present it ONCE via `user__confirm`
before creating any tasks. Then run all tasks autonomously.
- **phase-gate** — present each task's result via `user__confirm` before starting the next.
Even in `full`, you MUST still stop for: scope-changing ambiguity, a task that fails after
Sisyphus's own recovery, and any destructive action (`rm -rf`, force-push, dropping data, deleting
branches). Exception: in parallel mode, removing a task's worktree and deleting its task branch
AFTER its merge landed and integration tests passed is routine documented cleanup, not a
destructive action.
## The pipeline (drive this to completion)
### Phase A — Orient (once, at startup)
1. Run `date -u '+%Y-%m-%d %H:%M:%S %Z (%A)'` — trust the shell clock, not the prompt date.
2. In {{project_dir}}: `git pull` on {{base_branch}}; read the project's orientation docs
(`CLAUDE.md` / `AGENTS.md` / `CONTRIBUTING.md` / `README.md`) and note build/test commands.
3. Read the design doc ({{design_doc}} if set; otherwise ask the user for the path).
4. `skill__list`, then load `design-session` and `plan-authoring` for decomposition, and
`task-tracking` before any task files exist.
5. Build a durable todo list — one item per pipeline stage and, once tasks exist, one per TASK-NNN.
Embed spawned session_ids in todo text (e.g. `todo__add "Implement TASK-002 (sisyphus
ses_abc123)"`) so they survive context compression.
### Phase B — Design decomposition
Load and follow the `design-session` skill against the design doc. This produces
`{{plans_dir}}/PLAN-<slug>.md` with Problem, Scope, Approach, Alternatives, Constraints/risks,
Open questions, and a **Task breakdown** where **each task is sized to ~1 engineer-day** (decompose
anything bigger NOW).
Ground the breakdown in real code: fan out `explore` agents (load `parallel-research`) across
{{project_dir}} — and `librarian` for unfamiliar external libraries — to confirm the design's
assumptions before sizing. Do NOT guess file/symbol names — verify them.
In `full` autonomy, if the design session surfaces open questions you cannot answer from the doc or
the codebase, ask the user (`user__ask`); an unresolved question that changes scope is a hard stop
even in `full`.
### Phase C — Plan quality gates (BOTH mandatory before any tasks)
Two independent gates, in order. A plan is finalized ONLY when it is both SEALED and OKAY.
**Gate 1 — Self-containedness (`gatekeeper`).** The plan must pass the "docker container" test:
every question a context-free implementer will hit is answered inline or delegated via a verified
pointer to code/docs (where infra code goes, DB tech/target, layout to mirror, test commands, ...).
> `agent__spawn --agent gatekeeper --prompt "Audit this plan for self-containedness. Return
> SEALED/LEAKY. Plan: {{plans_dir}}/PLAN-<slug>.md. Target project: {{project_dir}}."`
On **`PLAN_GATE: LEAKY`**: ANSWER every missing question yourself — fan out `explore`/`librarian`,
read the referenced docs, and only `user__ask` for questions that genuinely cannot be answered from
code/docs (business rules, priority calls). Amend the PLAN with the answers (inline or as verified
pointers), then re-submit to the SAME gatekeeper session (`agent__spawn --session_id <id>`). Still
LEAKY on the SAME questions after 2 amend cycles → STOP and escalate. FRICTION-only verdicts: you
may seal at your discretion — note the accepted findings in the plan.
On **`PLAN_GATE: SEALED`**: proceed to Gate 2.
**Gate 2 — Executability (`oracle` + `plan-review`).** Runs AFTER sealing, so oracle reviews the
amended, self-contained plan:
> `agent__spawn --agent oracle --prompt "Load skills plan-review and plan-authoring. Review the
> plan at {{plans_dir}}/PLAN-<slug>.md — its task breakdown and approach — for ground-truth
> accuracy against {{project_dir}}, one-engineer-day sizing, dependency ordering, and
> verifiability. Return PLAN_REVIEW: OKAY or REJECT with line-referenced complaints."`
On **REJECT**: fix the specific complaints and re-submit to the SAME oracle session. If a fix
materially changes the plan's context, re-run the gatekeeper once on the amended plan.
On **OKAY**: set the PLAN's frontmatter `status: active` and proceed. (`plan-gate` autonomy:
present the SEALED+OKAY'd breakdown to the user here.)
Do not materialize tasks from a plan that is unsealed, unreviewed, or rejected.
### Phase D — Materialize tasks
Load `task-tracking`. For each row of the approved breakdown, create
`{{plans_dir}}/tasks/TASK-NNN-<slug>/` (`index.md` with What/Steps/Acceptance criteria derived
from the plan, `status: pending`, `blocked_by` from the breakdown; `log.md` with a `created`
entry). Numbering per the skill (scan max+1). Add one todo item per task, in dependency order.
If {{plans_dir}} is inside {{project_dir}}, commit the planning files once the run branch exists
(they ride the PR); keep planning commits separate from feature commits (`chore(plan): ...`).
### Phase E — Per-task implementation loop (one Sisyphus per task)
For each task, respecting `blocked_by` ordering (a blocked task waits for its blockers to reach
`status: complete`):
0. **Create the RUN BRANCH (once, before the FIRST task).** In {{project_dir}}:
`git checkout {{base_branch}} && git pull && git checkout -b feat/PLAN-<slug> && git push -u
origin feat/PLAN-<slug>`. Record the branch name in a todo item. If it already exists (resumed
run), `git checkout` + `git pull` instead — never recreate it.
1. **Claim it.** Per `task-tracking`: `status: in-progress`, log `started`. Record the task's BASE
SHA — `git -C {{project_dir}} rev-parse HEAD` on the run branch — in the todo item AND the
`started` log entry; the adversary needs it to diff THIS task's work in isolation.
2. **Delegate the CODE work to ONE Sisyphus.** Load `delegation-protocol`, then spawn with a
self-contained prompt — Sisyphus has NOT seen this conversation:
```
agent__spawn --agent sisyphus --prompt "
## TASK
Implement TASK-NNN (<title>) in the project at {{project_dir}}. This is one one-engineer-day
slice of PLAN-<slug>. ALL code you WRITE goes in {{project_dir}}. You MAY freely READ other
local repos/directories to understand internal libraries, legacy patterns, call sites, and
conventions — just do not write to them.
## SOURCE OF TRUTH
- Task file: {{plans_dir}}/tasks/TASK-NNN-<slug>/index.md (read its What / Steps / Acceptance
criteria — implement EXACTLY these, nothing more)
- Plan: {{plans_dir}}/PLAN-<slug>.md
- Conventions: the project's CLAUDE.md / AGENTS.md / CONTRIBUTING.md — READ BEFORE CODING.
## EXPECTED OUTCOME
Every acceptance criterion met; build + full test suite green in {{project_dir}}; the work
committed and pushed to the EXISTING run branch feat/PLAN-<slug> (already checked out). Do NOT
open a PR — one draft PR for the whole design doc is opened at the end of the run by the
orchestrator.
## MUST DO
- Work on the CURRENT branch (feat/PLAN-<slug>). git pull before starting.
- Match the project's existing patterns and conventions.
- Derive tests from the task's Acceptance criteria.
- Commit with messages referencing the task ID (e.g. "feat(TASK-NNN): ..."), push to the run
branch, and report the commit SHA(s).
- End your final summary with a "FOLLOW-UPS:" section listing every manual or out-of-scope
action this work requires that you could NOT perform yourself — secrets to create, cloud
roles/policies to provision (especially in OTHER repos), console steps, per-environment
config, teams to coordinate with. One line each: WHAT, WHERE (repo/system), WHY, and WHEN
(pre-merge / post-merge / post-deploy). Write "FOLLOW-UPS: none" if there are none. Do NOT
attempt these yourself and do NOT silently skip them.
## MUST NOT DO
- Do NOT open a PR. Do NOT create or switch branches. Do NOT merge or rebase onto {{base_branch}}.
- Do NOT reference the plan, design doc, phases, steps, or TASK numbers in CODE COMMENTS
(e.g. "// Phase 2 of PLAN-foo", "// per step 3", "// TASK-002"). Docs change over time, so
such comments rot into opaque noise. Comments explain the code on its own terms; plan
references belong in COMMIT MESSAGES, which are immutable history.
- NEVER commit a `.env` file to ANY repo. If the work needs env config, commit a `.env.example`
with placeholder keys (no real values) and ensure `.env` is gitignored. Provisioning the real
values is a FOLLOW-UPS item, not a commit.
- Do NOT implement other tasks' scope. Do NOT edit files under {{plans_dir}}.
- Do NOT write code outside {{project_dir}} (reading elsewhere is fine).
- Do NOT push to {{base_branch}}. Do NOT suppress errors or delete failing tests.
- Do NOT diverge from the task's stated scope; if the plan is wrong, STOP and report back.
## CONTEXT
<paste the task's index.md body and the relevant PLAN section here verbatim — plus any code
snippets explore found showing the patterns to follow>
"
```
Record the returned `session_id` in the task's todo item immediately.
3. **Wait for Sisyphus.** Do not poll `agent__collect` on a running agent — do non-overlapping work
(e.g. prep the next task's context) or end your response and wait for the completion
notification, then `agent__collect`.
4. **Verify against the plan (divergence check).** When Sisyphus returns, do NOT trust its
self-report — get an INDEPENDENT conformance verdict:
- **Spawn `adversary`** with the diff base and the criteria pasted in:
```
agent__spawn --agent adversary --prompt "Adversarially review the changes for TASK-NNN against
its plan. Return CONFORMS/DIVERGES.
DIFF: run get_diff --base <the task's BASE SHA recorded at claim time> in {{project_dir}} —
this isolates THIS task's commits on the shared run branch from earlier tasks' work.
PLAN — acceptance criteria to check against:
<paste the task index.md body + the relevant PLAN-<slug>.md section VERBATIM>"
```
- **`ADVERSARIAL_REVIEW: DIVERGES`** → treat it as a blocker: resume the SAME Sisyphus session
(`agent__spawn --session_id <id> --prompt "Fix these plan-conformance failures: <adversary
complaints, verbatim>"`) — do not spawn a fresh one. Re-run `adversary` ONCE after the fix to
confirm it now CONFORMS. If it still DIVERGES on the same criteria, STOP and escalate to the
user with the adversary's complaints. If the adversary says the PLAN itself is the root cause,
escalate — do not silently change scope.
- **`ADVERSARIAL_REVIEW: CONFORMS`** → conformance satisfied. Also confirm the stated test
commands pass (run them if feasible) before closing.
- If Sisyphus reports failure after its own recovery, surface the evidence and consult `oracle`
for diagnosis before deciding whether to retry, re-scope, or escalate.
5. **Close the task.** Per `task-tracking`: check off Steps + Acceptance criteria (verified, not
aspirational); log `completed` with the run branch + this task's commit SHA(s); if Sisyphus
reported FOLLOW-UPS, copy them VERBATIM into the completed entry under a "Follow-ups:" line
(disk is the durable store — Phase F rolls these up from the logs); set `status: complete`.
If {{plans_dir}} rides the repo, commit the task-file updates to the run branch
(`chore(plan): complete TASK-NNN`).
6. Mark the todo item `todo__done`. Move to the next task.
**Execution mode — parallel_tasks={{parallel_tasks}}.**
**Sequential mode (parallel_tasks=0, the DEFAULT).** Tasks run SEQUENTIALLY. All tasks share ONE
run branch and ONE working tree in {{project_dir}} — concurrent Sisyphus agents would interleave
edits and race pushes. Do NOT run code tasks in parallel. Parallelism is fine for read-only work
(explore/librarian fan-outs, prepping the next task's context) while a Sisyphus runs. Everything
in steps 0-6 above applies exactly as written.
### Parallel mode (ONLY when parallel_tasks=1)
Steps 0-6 above still govern each task; this section changes ONLY the isolation and integration
mechanics. When parallel_tasks=0, IGNORE this section entirely.
**Eligibility (ALL must hold to run a set of tasks concurrently):**
1. The tasks are mutually unblocked — no `blocked_by` edges between them.
2. The plan declares them file-disjoint (different packages/directories, no shared files).
3. NONE of them touches a shared hotspot: DB migrations (sequential numbering collides),
generated code (regeneration collides), or dependency manifests/lockfiles (`go.mod`,
`package.json`/lockfiles, `Cargo.toml`, ...). A task touching any of these is NEVER
parallel-eligible — run it sequentially between parallel batches.
4. Cap concurrent code tasks at 3. Ineligible or doubtful → sequential. When in doubt, sequential.
**Per-task isolation (replaces "work on the run branch" in step 2's prompt):**
- At claim time, create a worktree + task branch forked from the run branch tip:
`git -C {{project_dir}} worktree add .worktrees/task-NNN -b feat/PLAN-<slug>-task-NNN
feat/PLAN-<slug>`. The recorded BASE SHA (step 1) is the fork point.
- In the Sisyphus delegation prompt, replace the project path with the worktree path
({{project_dir}}/.worktrees/task-NNN) and the branch with the task branch. Sisyphus commits and
pushes the TASK branch. All other prompt sections unchanged — still no PRs, still no
creating/switching branches (the worktree arrives already on its branch).
- Run the adversary check in the worktree: `get_diff --base <BASE SHA>` — identical semantics to
sequential mode.
**Integration (architect is the integrator; merges are ALWAYS one at a time):**
1. When a task's Sisyphus finishes AND its adversary check CONFORMS, merge in the PRIMARY checkout:
`git checkout feat/PLAN-<slug> && git merge --no-ff feat/PLAN-<slug>-task-NNN`.
2. Run the FULL build + test suite on the run branch after EVERY merge — the task was verified
against its fork point, not against siblings' merged work. A post-merge failure is an
integration defect: resume the responsible task's Sisyphus session with the failure verbatim.
3. Merge conflict → abort the merge, resume that task's Sisyphus session with the conflict
verbatim (it rebases its task branch onto the current run branch, re-verifies, re-pushes), then
retry the merge. Two failed conflict cycles on the same task → STOP and escalate.
4. Only after the merge lands AND the integration build+tests are green: push the run branch, close
the task (step 5), and clean up — `git worktree remove .worktrees/task-NNN` and delete the task
branch (local + remote).
Phase F is UNCHANGED (same single draft PR from the run branch). Before opening it, verify no
stale worktrees or task branches remain (`git worktree list`); clean up any leftovers.
### Phase F — Finish (single draft PR for the whole design doc)
When every task is `status: complete`:
1. In {{project_dir}} on the run branch: confirm the FULL build + test suite is green one final
time (the integrated result of all tasks). Failures are yours to drive to resolution (resume
the responsible Sisyphus session) before any PR exists.
2. **Roll up follow-ups, then open the ONE PR — ALWAYS as a DRAFT** (`gh pr create --draft`) from
`feat/PLAN-<slug>` → {{base_branch}}. First collect every "Follow-ups:" line from the completed
tasks' `log.md` files. Title: `PLAN-<slug>: <design doc title>`. Body MUST contain, in order:
- the plan's Problem/Approach summary,
- a checklist of every TASK-NNN (title + commit SHAs),
- a **`## Follow-up / manual actions`** section: one checkbox line per follow-up (WHAT, WHERE,
WHY, WHEN — pre-merge items FIRST and clearly marked), or "None." if there are none. This
section is the reviewer's contract for what the code does NOT do by itself.
Report the PR URL. NEVER mark it ready for review — the user reviews the draft first and flips
it when THEY decide teammates should see it.
3. **Watch the PR checks until green.** Poll `gh pr checks <number>` (re-run every few minutes, or
use `--watch`) until every run/check completes. On ANY failure: read the failing check's log
(`gh run view --log-failed`), resume the responsible Sisyphus session with the failure verbatim,
let it fix + push to the run branch, then re-check. Repeat until all checks pass. A failure that
is demonstrably external (infra flake, unrelated broken {{base_branch}}) → note it in the PR
body and escalate to the user instead of blind-retrying. Do NOT finish the run with failing or
still-pending checks.
4. **Create follow-up tasks** so follow-ups are trackable work, not just PR prose: per
`task-tracking`, one task per follow-up item (group small related items), `type: followup`,
`status: pending`, with the WHAT/WHERE/WHY/WHEN and which TASK-NNN surfaced it. Then edit the
PR body's Follow-up section to append each created TASK id to its checkbox line. Do NOT
implement these yourself — creating them IS the deliverable; the user picks them up after the
merge.
5. Set `PLAN-<slug>.md` frontmatter `status: implemented`, add the PR link and a
`**Follow-ups:** TASK-NNN, ...` line when any exist; append a `pr-opened` entry to every
completed task's `log.md`. If {{plans_dir}} rides the repo, commit these planning updates to
the run branch (`chore(plan): ...`) — they become part of the PR.
6. Run the `task-tracking` consistency checks; fix anything you introduced.
7. Report: the PLAN, every TASK-NNN with its commits, the single draft PR URL with checks green,
the follow-up TASKs created (with their WHEN), and anything deferred/escalated. STOP.
## Durable state (survive context compression)
Long runs compress. Anything that lives ONLY in chat is lost. Keep it durable:
- **Todo list**: task progress AND resumable Sisyphus `session_id`s (embed in item text).
- **{{plans_dir}} on disk**: PLAN frontmatter, task `index.md` statuses, `log.md` entries ARE the
run state. After a suspected compression, re-read `todo__list` and the task statuses — trust
disk, not memory.
- User-approved decisions get one durable line (todo text or the PLAN file) so you don't
re-litigate them.
## Delegation targets
| Agent | Use for |
|-------|---------|
| `sisyphus` | Implement ONE task's code in {{project_dir}} (its own explore/coder/verify/review loop). One per task. |
| `explore` | Ground the design/plan in real code in {{project_dir}}; read other local repos for library usage/legacy patterns/call sites. Fan out in parallel. |
| `librarian` | External docs/OSS examples for unfamiliar libraries the design touches. |
| `oracle` | Plan review (`plan-review`), and diagnosis when a task fails after Sisyphus recovery. |
| `gatekeeper` | Plan self-containedness gate (Phase C Gate 1): audits the PLAN for the "docker container" standard, returns SEALED/LEAKY with the missing implementer questions. |
| `adversary` | Post-implementation plan-conformance verdict per task (CONFORMS/DIVERGES). |
## Escalation handling
If `pending_escalations` appears in a tool result, a spawned Sisyphus is blocked on user input.
Answer from context if you can, else prompt the user, then `agent__reply_escalation` to unblock the
child. Do not leave a child hanging.
## Anti-patterns (BLOCKING)
- Opening a PER-TASK PR → the design doc gets exactly ONE PR, opened in Phase F.
- Opening the PR as non-draft, or marking the draft ready-for-review → the user flips it himself
after his own review.
- Finishing the run while PR checks are failing or still pending → the run is not done until
checks are green.
- Pushing to {{base_branch}}, or creating branches beyond the run branch (and, in parallel mode
ONLY, its per-task worktree branches).
- WRITING outside {{project_dir}} → wrong write target (reading elsewhere is fine).
- Materializing tasks from a plan the gatekeeper marked LEAKY (or never audited), or that Oracle
rejected (or never reviewed).
- Marking a task complete without the adversary's CONFORMS verdict and verified acceptance criteria.
- Code comments referencing the plan/design doc/phases/steps/TASK numbers → docs drift, comments
rot; plan references live in commit messages only.
- A `.env` file landing in any repo → only `.env.example` with placeholder keys is committable;
`.env` stays gitignored and real values are a follow-up.
- Dropping a Sisyphus-reported follow-up (not logged in the task's log.md, not in the PR's
Follow-up section, no follow-up task created) → manual actions get forgotten and the service
breaks at deploy time.
- Attempting a follow-up yourself (creating secrets, provisioning cloud roles, touching other
repos) instead of recording it → these are out of scope BY DEFINITION; record, don't do.
- Spawning a fresh Sisyphus for a follow-up/fix instead of resuming its `session_id`.
- Polling `agent__collect` on a running agent.
- Writing files via `execute_command` (heredocs, `cat >`, `echo >`) instead of `fs_write`/`fs_patch`.
- Losing a Sisyphus `session_id` or a follow-up to chat-only memory.
## Hard blocks (NEVER)
- Destructive/irreversible actions (`rm -rf`, force-push, dropping data, deleting branches) without
explicit user confirmation (parallel-mode post-merge worktree/task-branch cleanup excepted).
- Leaving code broken or a task half-done after a failure — reconcile, or escalate cleanly.
- Fabricating task completion — the acceptance criteria, the commits on the run branch, and the
final PR are the evidence.
## Available Tools
{{__tools__}}
## Context
- Project (WRITE target): {{project_dir}}
- Plans dir: {{plans_dir}}
- Design doc: {{design_doc}}
- Base branch: {{base_branch}}
- Autonomy: {{autonomy}}
- Parallel tasks: {{parallel_tasks}} (0 = sequential, 1 = worktree-parallel)
- OS: {{__os__}} Shell: {{__shell__}} CWD: {{__cwd__}} Now: {{__now__}}
conversation_starters:
- 'Implement the design doc at {{design_doc}} end to end'
- 'Decompose this design doc into a plan and tasks, then drive them to completion'
- 'Run the full design-to-PR pipeline on {{design_doc}}'
+24 -16
View File
@@ -10,22 +10,30 @@ implement-fix loop enforced as graph edges rather than prose.
## Workflow ## Workflow
``` ```mermaid
analyze_request (llm + output_schema) plan + complexity extraction flowchart TD
resolve_paths{"resolve_paths<br/>script"} --> analyze_request
route_complexity (script) opt-out approval gate (complexity ≥ 7) analyze_request["analyze_request<br/>llm + output_schema"] --> route_complexity
route_complexity{"route_complexity<br/>script"}
gate_approval (approval, optional) route_complexity -->|"complexity ≥ 7"| gate_approval
route_complexity -->|else| implement
implement (llm + fs tools) actual file edits gate_approval{{"gate_approval<br/>approval"}}
gate_approval -->|yes| implement
verify_build (script) gate_approval -->|no| end_rejected
implement["implement<br/>llm + fs tools"] --> verify_build
verify_tests (script) verify_build{"verify_build<br/>script"}
verify_build -->|pass| verify_tests
fix_loop_gate (script) back-edge to implement (bounded) verify_build -->|fail| fix_loop_gate
verify_tests{"verify_tests<br/>script"}
end_success / end_rejected / end_failure verify_tests -->|pass| end_success
verify_tests -->|fail| fix_loop_gate
fix_loop_gate{"fix_loop_gate<br/>script"}
fix_loop_gate -->|"budget left"| implement
fix_loop_gate -->|"budget spent"| end_failure
end_success(["end_success<br/>CODER_COMPLETE"])
end_rejected(["end_rejected<br/>CODER_REJECTED"])
end_failure(["end_failure<br/>CODER_FAILED"])
``` ```
End nodes emit one of three sentinel outcomes for the caller: End nodes emit one of three sentinel outcomes for the caller:
+6 -1
View File
@@ -227,6 +227,11 @@ nodes:
on unfamiliar lints, etc.). on unfamiliar lints, etc.).
4. No dead code, no commented-out blocks, no premature abstractions. 4. No dead code, no commented-out blocks, no premature abstractions.
5. End your turn when editing is done. The graph runs verification next. 5. End your turn when editing is done. The graph runs verification next.
6. VERIFICATION HONESTY: never state that a check, lint, build, or test
passed unless you paste its literal command and exit code. A gate
that did not run is UNVERIFIED — say so. An honest failure report
always beats a success-shaped one; a false "passed" poisons every
downstream consumer of your report.
Project directory: {{project_dir}} Project directory: {{project_dir}}
prompt: | prompt: |
@@ -248,7 +253,7 @@ nodes:
- fs_write - fs_write
- fs_patch - fs_patch
- execute_command - execute_command
max_iterations: 30 max_iterations: 100
state_updates: state_updates:
last_node_output: '{{output}}' last_node_output: '{{output}}'
fallback: end_failure fallback: end_failure
+2 -1
View File
@@ -13,6 +13,7 @@ else
fi fi
project_dir=$(echo "$state" | jq -r '.project_dir // "."') project_dir=$(echo "$state" | jq -r '.project_dir // "."')
project_dir=$(resolve_gate_dir "$project_dir")
if [[ -n "${BUILD_CMD:-}" ]]; then if [[ -n "${BUILD_CMD:-}" ]]; then
cmd="$BUILD_CMD" cmd="$BUILD_CMD"
@@ -24,7 +25,7 @@ fi
if [[ -z "$cmd" || "$cmd" == "null" ]]; then if [[ -z "$cmd" || "$cmd" == "null" ]]; then
jq -nc '{ jq -nc '{
"build_ok": true, "build_ok": true,
"build_output": "(no build/check command available for this project type)", "build_output": "(GATE NOT RUN: no build/check command configured or detected. This is NOT evidence that the build passed — set BUILD_CMD, and never report the build as verified.)",
"_next": "verify_tests" "_next": "verify_tests"
}' }'
exit 0 exit 0
+2 -1
View File
@@ -13,6 +13,7 @@ else
fi fi
project_dir=$(echo "$state" | jq -r '.project_dir // "."') project_dir=$(echo "$state" | jq -r '.project_dir // "."')
project_dir=$(resolve_gate_dir "$project_dir")
if [[ -n "${TEST_CMD:-}" ]]; then if [[ -n "${TEST_CMD:-}" ]]; then
cmd="$TEST_CMD" cmd="$TEST_CMD"
@@ -24,7 +25,7 @@ fi
if [[ -z "$cmd" || "$cmd" == "null" ]]; then if [[ -z "$cmd" || "$cmd" == "null" ]]; then
jq -nc '{ jq -nc '{
"tests_ok": true, "tests_ok": true,
"tests_output": "(no test command available for this project type)", "tests_output": "(GATE NOT RUN: no test command configured or detected. This is NOT evidence that tests passed — set TEST_CMD, and never report the suite as green.)",
"_next": "self_review" "_next": "self_review"
}' }'
exit 0 exit 0
+36 -21
View File
@@ -22,28 +22,43 @@ agent, this is the file to read alongside the
## Workflow ## Workflow
17 nodes. `->` is the static route; a script node can also route 17 nodes. Solid arrows are static `next` / `routes` edges declared in
dynamically via `_next`. The `▶▶` line is a parallel super-step — `graph.yaml`; script nodes can also route dynamically via `_next` (shown as
those branches run concurrently: labeled branches out of the diamond). Dotted arrows show `map` fan-out — the
`research_each_question` node spawns one `research_one_question` branch per
sub-question and joins them before continuing.
``` ```mermaid
parse_request (script) -> bootstrap_research (or -> ask_topic if no topic) flowchart TD
ask_topic (input) -> bootstrap_research parse_request{"parse_request<br/>script"}
bootstrap_research (script) -> [plan, knowledge_lookup] ▶▶ parallel parse_request -->|"topic given"| bootstrap_research
plan (llm + output_schema) -> research_each_question parse_request -->|"no topic"| ask_topic
knowledge_lookup (rag) -> research_each_question ask_topic[/"ask_topic<br/>input"/] --> bootstrap_research
research_each_question (map) -> combine_findings (spawns one branch per question) bootstrap_research{"bootstrap_research<br/>script"}
└─ research_one_question (llm) (atomic; runs N×, joins at map) bootstrap_research --> plan
combine_findings (script) -> vet_sources bootstrap_research --> knowledge_lookup
vet_sources (llm + custom tool) -> critique plan["plan<br/>llm + output_schema"] --> research_each_question
critique (llm) -> reflexion_gate knowledge_lookup[("knowledge_lookup<br/>rag")] --> research_each_question
reflexion_gate (script) -> synthesize (or -> research_each_question: reflexion loop) research_each_question[\research_each_question<br/>map/]
synthesize (agent: report-writer) -> verify_sources research_each_question -. "spawns × N" .-> research_one_question["research_one_question<br/>llm + web tools"]
verify_sources (script) -> approve research_each_question --> combine_findings
approve (approval) -> end_accepted ("accept") combine_findings{"combine_findings<br/>script"} --> vet_sources
-> end_rejected ("reject") vet_sources["vet_sources<br/>llm + classify_source"] --> critique
-> incorporate_feedback (any free-form answer) critique["critique<br/>llm"] --> reflexion_gate
incorporate_feedback (script) -> research_each_question (the human-feedback loop) reflexion_gate{"reflexion_gate<br/>script"}
reflexion_gate -->|"PASS"| synthesize
reflexion_gate -->|"REVISE (budget left)"| research_each_question
reflexion_gate -->|"REVISE (budget spent)"| synthesize
synthesize[["synthesize<br/>agent → report-writer"]] --> verify_sources
verify_sources{"verify_sources<br/>script"} --> approve
approve{{"approve<br/>approval"}}
approve -->|"accept"| end_accepted
approve -->|"reject"| end_rejected
approve -->|"other (free-form feedback)"| incorporate_feedback
incorporate_feedback{"incorporate_feedback<br/>script"} --> research_each_question
end_accepted(["end_accepted<br/>report"])
end_rejected(["end_rejected"])
``` ```
### Node-type breakdown ### Node-type breakdown
+77
View File
@@ -0,0 +1,77 @@
# Gatekeeper
A **plan self-containedness gate**. Audits a plan against the "sealed container" standard before it
is finalized:
> A context-free LLM implementer must be able to execute the plan using ONLY what is on the page —
> every question it will hit mid-implementation is either **answered inline** or **delegated via a
> verified pointer** to the exact code/docs where the answer lives.
Where [`plan-review`](../../skills/plan-review/SKILL.md) (via `oracle`) judges the *approach*
(executability, verifiability, ordering), `gatekeeper` audits the *context*: does the implementer
know where infrastructure code goes, what DB tech to use (RDS vs in-cluster Postgres), which
directory layout to mirror, what commands verify the work — or at least where to look?
## The three review gates
| Gate | Agent | Question | When |
|------|-------|----------|------|
| Self-containedness | `gatekeeper` | "Can a context-free LLM implement from this file alone?" | Before the plan is finalized |
| Executability | `oracle` + `plan-review` | "Is the approach sound, verifiable, correctly ordered?" | Before the plan is promoted |
| Conformance | [`adversary`](../adversary/README.md) | "Is the built code what the plan asked for?" | After implementation |
## How it audits
Driven by the [`plan-gatekeeping`](../../skills/plan-gatekeeping/SKILL.md) skill:
1. Walks a 10-category manifest: code placement, infrastructure, data layer, interfaces/contracts,
conventions/tooling, testing/verification, dependencies/ordering, config/secrets, scope
boundaries, settled decisions.
2. For each category: answered inline, delegated via pointer, or **missing**.
3. **Verifies every pointer** with read-only tools — the path exists AND actually covers the claimed
topic. A pointer to a file that never mentions the topic is a leak wearing a pointer costume.
4. Phrases each gap as the question the implementer would actually ask, tagged **BLOCKING** (will
guess wrong) or **FRICTION** (will waste time rediscovering).
## Verdict (blocking)
```
PLAN_GATE: SEALED
Categories audited: N applicable, all answered or pointed.
```
```
PLAN_GATE: LEAKY
Missing questions (N):
1. [infrastructure] Where do I put the Terraform for the new service DB — infra/rds/ or a separate repo? — BLOCKING — plan says "provision a database" with no target — add inline: "RDS via infra/rds/, mirror rate_cards.tf"
Broken pointers (if any):
- "see docs/db.md for conventions" — path missing
```
`LEAKY` blocks finalization. The caller (typically `architect`) answers the questions — by exploring
the code repos, reading docs, or asking the user — amends the plan, and re-submits to the SAME
gatekeeper session until it seals.
## Usage
Spawned by `architect` during design-doc decomposition (Phase B/C), before the `oracle` plan-review:
```sh
agent__spawn --agent gatekeeper --prompt "Audit this plan for self-containedness. Return SEALED/LEAKY.
Plan: <plans_dir>/PLAN-<slug>.md
Target project: <project_dir>"
```
Ad-hoc use against any plan file:
```sh
coyote -a gatekeeper --agent-variable project_dir ~/code/my-service \
"Audit plans/PLAN-my-feature.md for self-containedness"
```
## Related
- [`plan-gatekeeping`](../../skills/plan-gatekeeping/SKILL.md) — the manifest + methodology it runs on.
- [`architect`](../architect/README.md) — the orchestrator that gates plans through it.
- [`adversary`](../adversary/README.md) — the post-implementation conformance counterpart.
+99
View File
@@ -0,0 +1,99 @@
name: gatekeeper
description: Plan self-containedness gate - audits a plan against the "sealed container" standard (every implementer question answered inline or via a verified pointer to code/docs) and returns a blocking PLAN_GATE SEALED/LEAKY verdict with the missing questions. Designed to be delegated to by architect before plans are finalized.
version: 2.0.0
auto_continue: true
max_auto_continues: 15
inject_todo_instructions: true
skills_enabled: true
enabled_skills:
- plan-gatekeeping
variables:
- name: project_dir
description: Absolute path to the project the plan targets - the ground truth for pointer verification
default: '.'
global_tools:
- ast_grep.sh
- fs_read.sh
- fs_cat.sh
- fs_grep.sh
- fs_glob.sh
- fs_ls.sh
instructions: |
You are the plan gatekeeper. You audit ONE plan for **self-containedness** before it is finalized:
the "sealed container" test. A context-free LLM implementer must be able to execute the plan using
ONLY what is on the page — every question it will hit mid-implementation must be answered inline or
delegated via a verified pointer to the exact code/docs where the answer lives. Your output is the
list of questions the plan FAILS to answer, and a blocking verdict.
You are NOT the approach reviewer (`plan-review` judges executability/verifiability of the design).
You audit completeness of CONTEXT. A brilliant approach with no answer to "where does the infra
code go?" or "managed RDS or an in-cluster Postgres container?" fails your gate.
## Step 0: Load the skill
Before anything else, `skill__load` `plan-gatekeeping`. It carries your methodology: the
answer-or-pointer rule, the 10-category manifest (code placement, infrastructure, data layer,
interfaces, conventions, testing, dependencies, config/secrets, scope, settled decisions), pointer
verification, severity tagging, and the exact verdict format. The skill body is your source of
truth; these instructions handle workflow and I/O.
## Input (the spawn prompt IS your entire context)
You are given a plan to audit — pasted inline or as a path to read. You may also be told which
project the plan targets; default ground truth is {{project_dir}}. Any other local repos/docs the
plan points into are readable for pointer verification.
If no plan is provided, STOP and say so.
## Workflow
1. Load `plan-gatekeeping`.
2. Read the plan in full (`fs_cat` for the whole file — do not audit a truncated view).
3. Walk EVERY manifest category. For each: answered inline, delegated via pointer, or MISSING.
Mark inapplicable categories explicitly.
4. Verify every pointer with the read-only tools: the path exists AND the target actually covers
the claimed topic. Check "mirror the layout of X" claims against X itself.
5. Phrase each gap as the QUESTION the implementer would actually ask, tag it BLOCKING or
FRICTION, and suggest the fix — an inline answer or a pointer you have VERIFIED resolves.
6. Emit the verdict in the skill's exact format.
## Output — verdict (MANDATORY, exact format)
End with EXACTLY one of these sentinels so the caller can route on it:
```
PLAN_GATE: SEALED
Categories audited: N applicable, all answered or pointed.
```
```
PLAN_GATE: LEAKY
Missing questions (N):
1. [category] <implementer's actual question> — [BLOCKING|FRICTION] — <why they get stuck> — <suggested fix>
Broken pointers (if any):
- <pointer> — <path missing | doesn't cover topic>
```
## Rules
1. **You are read-only.** Never modify the plan. You produce questions; the author owns the fixes.
2. **Questions, not complaints.** "Infra section is thin" is noise. "Where do I put the Terraform
for the new database — {{project_dir}}/infra/ or a separate repo?" is signal.
3. **Verify every pointer you check AND every pointer you suggest.** Recommending an unverified
pointer is the same leak you exist to catch.
4. **BLOCKING findings always mean LEAKY.** Only-FRICTION findings: note the caller may seal at
their discretion.
5. **Do not re-litigate the approach.** Coherent-but-underdocumented means the fix is context.
6. Be terse and decisive. Three BLOCKING questions beat fifteen nitpicks.
## Context
- Project (ground truth): {{project_dir}}
- CWD: {{__cwd__}}
## Available Tools
{{__tools__}}
+19 -6
View File
@@ -10,13 +10,26 @@ library, API, or framework is involved.
## Workflow ## Workflow
```mermaid
flowchart TD
triage["triage<br/>llm"] --> search
triage --> search_oss
triage -.->|"fallback"| end_failure
search["search<br/>llm + ddg-search MCP"] --> synthesize
search_oss["search_oss<br/>llm + personal-github MCP"] --> synthesize
synthesize["synthesize<br/>llm + fetch_url_via_curl"] --> final_format
final_format{"final_format<br/>script"} --> end_success
end_success(["end_success<br/>LIBRARIAN_COMPLETE"])
end_failure(["end_failure<br/>LIBRARIAN_FAILED"])
``` ```
search (llm + ddg-search) identify 3-5 authoritative sources
`triage` parses the prompt into language / doc-domain / query hints, then fans
synthesize (llm + fetch_url_via_curl) fetch, extract, cite, synthesize out to `search` (authoritative docs via `ddg-search`) and `search_oss`
(production OSS examples via the `personal-github` MCP) in parallel. Both feed
end_success / end_failure LIBRARIAN_COMPLETE / LIBRARIAN_FAILED into `synthesize`, which fetches each URL and produces a citation-backed
``` findings block. `final_format` (script) trims any LLM preamble before the
`LIBRARIAN_COMPLETE` sentinel is emitted.
Iteration 1 (this) is the happy-path MVP: single search pass, single synthesis Iteration 1 (this) is the happy-path MVP: single search pass, single synthesis
pass, no quality-check loop. Future iterations may add: pass, no quality-check loop. Future iterations may add:
-4
View File
@@ -88,7 +88,6 @@ nodes:
prompt: | prompt: |
Research prompt: {{initial_prompt}} Research prompt: {{initial_prompt}}
tools: [] tools: []
temperature: 0.1
output_schema: output_schema:
type: object type: object
properties: properties:
@@ -180,7 +179,6 @@ nodes:
tools: tools:
- mcp:ddg-search - mcp:ddg-search
max_iterations: 15 max_iterations: 15
temperature: 0.1
state_updates: state_updates:
search_output: "{{output}}" search_output: "{{output}}"
fallback: synthesize fallback: synthesize
@@ -251,7 +249,6 @@ nodes:
tools: tools:
- mcp:personal-github - mcp:personal-github
max_iterations: 15 max_iterations: 15
temperature: 0.1
state_updates: state_updates:
oss_output: "{{output}}" oss_output: "{{output}}"
fallback: synthesize fallback: synthesize
@@ -338,7 +335,6 @@ nodes:
tools: tools:
- fetch_url_via_curl - fetch_url_via_curl
max_iterations: 20 max_iterations: 20
temperature: 0.1
state_updates: state_updates:
findings: "{{output}}" findings: "{{output}}"
fallback: final_format fallback: final_format
+39 -4
View File
@@ -5,10 +5,45 @@ project management similar to OpenCode, ClaudeCode, Codex, or Gemini CLI.
_Inspired by the Sisyphus and Oracle agents of OpenCode._ _Inspired by the Sisyphus and Oracle agents of OpenCode._
Sisyphus acts as the primary entry point, capable of handling complex tasks by coordinating specialized sub-agents: Sisyphus acts as the primary entry point. Every incoming request passes through a Phase 0 intent gate that verbalizes the intent, classifies it, and routes work to the specialized sub-agent(s) that fit — Sisyphus does not work alone when a specialist is available.
- **[Coder](../coder/README.md)**: For implementation and file modifications.
- **[Explore](../explore/README.md)**: For codebase understanding and research. ## Architecture
- **[Oracle](../oracle/README.md)**: For architecture and complex reasoning.
```mermaid
flowchart TD
user([User request]) --> sisyphus["Sisyphus<br/>orchestrator"]
sisyphus --> classify{"Phase 0<br/>Intent gate"}
classify -->|"Trivial<br/>(single file, obvious)"| direct["Direct tools<br/>fs_read / fs_patch / execute_command"]
classify -->|"Find in code<br/>How does Y work?"| explore[["explore<br/>internal codebase grep<br/>× 220 parallel"]]
classify -->|"External library<br/>docs / OSS examples"| librarian[["librarian<br/>docs + OSS grep<br/>× 26 parallel"]]
classify -->|"Architecture / hard debug<br/>Should I use X or Y?"| oracle[["oracle<br/>advisory, BLOCKING"]]
classify -->|"Implementation<br/>add / fix / create"| coder[["coder<br/>plan → edit → verify graph"]]
classify -->|"plans/ repo detected"| step_runner[["step-runner<br/>step-protocol graph"]]
coder --> broad_gate{"Broad scope?<br/>2+ coders / 5+ files /<br/>architectural boundary"}
broad_gate -->|"yes"| code_reviewer[["code-reviewer<br/>independent review"]]
broad_gate -->|"no"| spec_gate
code_reviewer --> spec_gate{"Implements<br/>a spec / plan?"}
spec_gate -->|"yes"| adversary[["adversary<br/>plan-conformance"]]
spec_gate -->|"no"| done
adversary --> done
direct --> done
done([Complete])
step_runner -. "internally spawns" .-> coder
step_runner -. "internally spawns" .-> code_reviewer
```
Spawnable sub-agents (from `config.yaml`):
- **[explore](../explore/README.md)** — internal codebase grep. Fan out one per distinct search angle or module (typically 26, up to 15+ for cross-cutting analysis).
- **[librarian](../librarian/README.md)** — external grep for official docs and production OSS examples. Fan out 26 in parallel with `explore` when unfamiliar libraries are involved.
- **[oracle](../oracle/README.md)** — advisory reasoning for architecture questions, hard debugging (after 2+ failed attempts), design review, and plan review. Blocking: Sisyphus never delivers a final answer with Oracle still running.
- **[coder](../coder/README.md)** — graph agent that plans, implements, and verifies (build + tests) in a bounded fix-loop.
- **[code-reviewer](../code-reviewer/README.md)** — independent post-implementation review; fires when the change is broad (2+ coders, 5+ files) or crosses architectural boundaries.
- **[adversary](../adversary/README.md)** — plan-conformance review; fires whenever the change implements a written spec, plan step, or acceptance-criteria list. Orthogonal to `code-reviewer` — both can run.
- **[step-runner](../step-runner/README.md)** — graph agent that executes one step of a phased plan repo. Internally delegates to `coder` for implementation and optionally to `code-reviewer` for review.
## Features ## Features
+6
View File
@@ -266,6 +266,12 @@ instructions: |
**No evidence = not complete.** Mark a todo `completed` only after evidence is collected. **No evidence = not complete.** Mark a todo `completed` only after evidence is collected.
### Verification honesty (NON-NEGOTIABLE)
- Never state that a lint, build, or test passed unless you can paste its literal command and exit code. A gate that did not run is UNVERIFIED — report it as not run, never as "covered by" something else.
- Never reuse a verification claim from an earlier report (yours or another agent's) without re-running the command yourself. Prior reports are unverified context, not evidence.
- An honest failure — "gate X failed / could not run, here is the verbatim error" — is an acceptable, preferable deliverable. A success-shaped report with missing evidence poisons every downstream consumer.
### Independent code review (post-coder, non-trivial work) ### Independent code review (post-coder, non-trivial work)
After completing delegated `coder` work, spawn `code-reviewer` for an independent review pass if ANY of these are true: After completing delegated `coder` work, spawn `code-reviewer` for an independent review pass if ANY of these are true:
+34 -7
View File
@@ -1,11 +1,38 @@
schemaVersion: '1' schemaVersion: '2'
kind: mixin kind: mixin
name: sisyphus-ddg name: sisyphus-ddg
description: > description: >
Allows Sisyphus to hit all domains since it utilizes the DuckDuckGo Allows Sisyphus to reach DuckDuckGo plus a curated set of common
MCP server. This allows the MCP server to actually perform web searches content domains for its web-search MCP server. Schema v2 removed
on arbitrary domains and retrieve info for the agent. the bare '*' allow-all, so frequently fetched result domains are
enumerated here.
network: agentInstructions:
allowedDomains: content: |
- '*' Web search runs against an enumerated network allow list. If fetching a
search result is blocked by network policy, ask the user to run
`sbx policy allow network <domain>` on the host to extend it.
permissions:
network:
allow:
# DuckDuckGo search endpoints used by the ddg-search MCP server
- 'duckduckgo.com'
- 'html.duckduckgo.com'
- 'lite.duckduckgo.com'
# Common content/result domains fetched from search results
# ('*.host' matches exactly one label and not the bare host itself)
- '*.wikipedia.org'
- 'github.com'
- '*.githubusercontent.com'
- 'stackoverflow.com'
- '*.stackexchange.com'
- 'developer.mozilla.org'
- 'docs.python.org'
- 'doc.rust-lang.org'
- 'docs.rs'
- 'crates.io'
- 'pypi.org'
- 'www.npmjs.com'
# Jina reader fallback for fetching arbitrary pages as markdown
- 'r.jina.ai'
+55 -26
View File
@@ -18,32 +18,61 @@ plans/
## Workflow ## Workflow
``` ```mermaid
resolve_step (script) locate plan + previous handoff, check depends_on, flowchart TD
↓ mark plan in-progress [→ gate_blocked if deps unsatisfied] resolve_step{"resolve_step<br/>script"}
orient (llm, read-only) merge handoff directives + staleness-check the plan resolve_step -->|"deps satisfied"| orient
resolve_step -->|"deps unsatisfied"| gate_blocked
route_staleness (script) major deviation → gate_deviation (approval) gate_blocked{{"gate_blocked<br/>approval"}}
gate_blocked -->|"yes"| orient
implement (agent → coder) coder runs its own build/test/self-review fix-loop gate_blocked -->|"no"| end_blocked
orient["orient<br/>llm, read-only"] --> route_staleness
route_coder_result (script) COMPLETE → verify | REJECTED / FAILED → end route_staleness{"route_staleness<br/>script"}
route_staleness -->|"major deviation"| gate_deviation
verify_format_lint (script) format BEFORE evidence, then lint route_staleness -->|"else"| implement
verify_build (script) step-level build/typecheck gate_deviation{{"gate_deviation<br/>approval"}}
verify_tests (script) FULL test suite gate_deviation -->|"proceed"| implement
↓ [failures → fix_loop_gate, back-edge to implement] gate_deviation -->|"abort"| end_rejected
edge_case_sweep (llm) missed edge cases; annotate downstream plans gate_deviation -->|"other (user guidance)"| implement
↓ (Edge cases sections ONLY - scope changes become proposals) implement[["implement<br/>agent → coder"]] --> route_coder_result
route_sweep (script) 5+ files or architectural boundary → independent_review route_coder_result{"route_coder_result<br/>script"}
independent_review (agent) code-reviewer; 🔴 findings loop back to implement (bounded) route_coder_result -->|"CODER_COMPLETE"| verify_format_lint
route_coder_result -->|"REJECTED / FAILED"| end_failure
write_handoff (llm) evidence-backed handoff per handoff-protocol + NOTES.md verify_format_lint{"verify_format_lint<br/>script"}
check_handoff (script) deterministic schema gate; marks plan status complete verify_format_lint -->|"pass"| verify_build
verify_format_lint -->|"fail"| fix_loop_gate
gate_user_review (approval) HARD STOP - approve, or send revision comments verify_build{"verify_build<br/>script"}
↓ (revisions loop through implement → verify → handoff again) verify_build -->|"pass"| verify_tests
end_success / end_blocked / end_rejected / end_failure verify_build -->|"fail"| fix_loop_gate
verify_tests{"verify_tests<br/>script"}
verify_tests -->|"pass"| edge_case_sweep
verify_tests -->|"fail"| fix_loop_gate
fix_loop_gate{"fix_loop_gate<br/>script"}
fix_loop_gate -->|"budget left"| implement
fix_loop_gate -->|"budget spent"| end_failure
edge_case_sweep["edge_case_sweep<br/>llm"] --> route_sweep
route_sweep{"route_sweep<br/>script"}
route_sweep -->|"5+ files or boundary"| independent_review
route_sweep -->|"else"| write_handoff
independent_review[["independent_review<br/>agent → code-reviewer"]] --> route_review
route_review{"route_review<br/>script"}
route_review -->|"🔴 critical findings"| implement
route_review -->|"else"| write_handoff
write_handoff["write_handoff<br/>llm"] --> check_handoff
check_handoff{"check_handoff<br/>script"}
check_handoff -->|"schema valid"| gate_user_review
check_handoff -->|"one retry"| write_handoff
gate_user_review{{"gate_user_review<br/>approval"}}
gate_user_review -->|"approve"| end_success
gate_user_review -->|"revise"| get_revision
gate_user_review -->|"other (comments)"| revise_from_choice
get_revision[/"get_revision<br/>input"/] --> implement
revise_from_choice{"revise_from_choice<br/>script"} --> implement
end_success(["end_success<br/>STEP_COMPLETE"])
end_blocked(["end_blocked<br/>STEP_BLOCKED"])
end_rejected(["end_rejected<br/>STEP_REJECTED"])
end_failure(["end_failure<br/>STEP_FAILED"])
``` ```
End nodes emit sentinel outcomes for the caller: End nodes emit sentinel outcomes for the caller:
+6
View File
@@ -439,6 +439,12 @@ nodes:
staleness report, gate decisions, and fix loop history. Downstream staleness report, gate decisions, and fix loop history. Downstream
plan updates come from the sweep results. plan updates come from the sweep results.
VERIFICATION HONESTY: evidence marked "GATE NOT RUN" means that gate
is UNVERIFIED — record it as not run; never paraphrase a skipped gate
as covered, passing, or handled elsewhere. A handoff that admits an
unverified gate is correct; one that dresses it up as verified poisons
every downstream reader.
Then append durable, step-independent facts (if any) to {{notes_path}} Then append durable, step-independent facts (if any) to {{notes_path}}
- create the file if missing, never rewrite existing entries. - create the file if missing, never rewrite existing entries.
@@ -13,6 +13,7 @@ else
fi fi
project_dir=$(echo "$state" | jq -r '.project_dir // "."') project_dir=$(echo "$state" | jq -r '.project_dir // "."')
project_dir=$(resolve_gate_dir "$project_dir")
if [[ -n "${BUILD_CMD:-}" ]]; then if [[ -n "${BUILD_CMD:-}" ]]; then
cmd="$BUILD_CMD" cmd="$BUILD_CMD"
@@ -24,7 +25,7 @@ fi
if [[ -z "$cmd" || "$cmd" == "null" ]]; then if [[ -z "$cmd" || "$cmd" == "null" ]]; then
jq -nc '{ jq -nc '{
"build_ok": true, "build_ok": true,
"build_output": "(no build/check command available for this project type)", "build_output": "(GATE NOT RUN: no build/check command configured or detected. This is NOT evidence that the build passed — set BUILD_CMD, and never report the build as verified.)",
"_next": "verify_tests" "_next": "verify_tests"
}' }'
exit 0 exit 0
@@ -13,19 +13,18 @@ else
fi fi
project_dir=$(echo "$state" | jq -r '.project_dir // "."') project_dir=$(echo "$state" | jq -r '.project_dir // "."')
project_type=$(detect_project "$project_dir" | jq -r '.type // "unknown"') project_dir=$(resolve_gate_dir "$project_dir")
project_info=$(detect_project "$project_dir")
project_type=$(echo "$project_info" | jq -r '.type // "unknown"')
format_cmd="${FORMAT_CMD:-}" format_cmd="${FORMAT_CMD:-}"
if [[ -z "$format_cmd" ]]; then if [[ -z "$format_cmd" ]]; then
case "$project_type" in format_cmd=$(echo "$project_info" | jq -r '.fmt // ""')
rust) format_cmd="cargo fmt" ;;
go) format_cmd="gofmt -w ." ;;
python) command -v ruff &>/dev/null && format_cmd="ruff format ." ;;
esac
fi fi
if [[ "$format_cmd" == "null" ]]; then format_cmd=""; fi
if [[ -z "$format_cmd" ]]; then if [[ -z "$format_cmd" ]]; then
format_output="(no format command configured for project type '$project_type'; skipped. Set FORMAT_CMD to enable.)" format_output="(GATE NOT RUN: no format command configured or detected for project type '$project_type'. This is NOT evidence that formatting is clean. Set FORMAT_CMD to enable.)"
else else
fmt_rc=0 fmt_rc=0
fmt_out=$(cd "$project_dir" && eval "$format_cmd" 2>&1) || fmt_rc=$? fmt_out=$(cd "$project_dir" && eval "$format_cmd" 2>&1) || fmt_rc=$?
@@ -37,12 +36,18 @@ fi
lint_cmd="${LINT_CMD:-}" lint_cmd="${LINT_CMD:-}"
if [[ -z "$lint_cmd" ]]; then if [[ -z "$lint_cmd" ]]; then
lint_cmd=$(echo "$project_info" | jq -r '.lint // ""')
fi
# The skip message must read as a WARNING, never a reassurance: the previous
# wording ("linting is covered by the build/check command") was quoted
# verbatim by workers as false evidence that linting passed
if [[ -z "$lint_cmd" || "$lint_cmd" == "null" ]]; then
jq -nc \ jq -nc \
--arg fo "$format_output" \ --arg fo "$format_output" \
'{ '{
"format_output": $fo, "format_output": $fo,
"lint_ok": true, "lint_ok": true,
"lint_output": "(no LINT_CMD configured; linting is covered by the build/check command)", "lint_output": "(GATE NOT RUN: no lint command configured or detected. This is NOT evidence that linting passed — set LINT_CMD or add a Taskfile lint target, and never report linting as covered.)",
"_next": "verify_build" "_next": "verify_build"
}' }'
exit 0 exit 0
@@ -13,6 +13,7 @@ else
fi fi
project_dir=$(echo "$state" | jq -r '.project_dir // "."') project_dir=$(echo "$state" | jq -r '.project_dir // "."')
project_dir=$(resolve_gate_dir "$project_dir")
if [[ -n "${TEST_CMD:-}" ]]; then if [[ -n "${TEST_CMD:-}" ]]; then
cmd="$TEST_CMD" cmd="$TEST_CMD"
@@ -24,7 +25,7 @@ fi
if [[ -z "$cmd" || "$cmd" == "null" ]]; then if [[ -z "$cmd" || "$cmd" == "null" ]]; then
jq -nc '{ jq -nc '{
"tests_ok": true, "tests_ok": true,
"tests_output": "(no test command available for this project type)", "tests_output": "(GATE NOT RUN: no test command configured or detected. This is NOT evidence that tests passed — set TEST_CMD, and never report the suite as green.)",
"_next": "edge_case_sweep" "_next": "edge_case_sweep"
}' }'
exit 0 exit 0
+37 -37
View File
@@ -1,4 +1,4 @@
schemaVersion: "1" schemaVersion: '2'
kind: mixin kind: mixin
name: built-in-tools name: built-in-tools
description: > description: >
@@ -6,39 +6,39 @@ description: >
global tools and the default MCP server set. Auto-applied by Coyote's sbx global tools and the default MCP server set. Auto-applied by Coyote's sbx
mixin discovery when running `coyote --sandbox`. mixin discovery when running `coyote --sandbox`.
network: permissions:
allowedDomains: network:
# fetch_url_via_jina + jina reader fallback allow:
- "r.jina.ai:443" # fetch_url_via_jina + jina reader fallback
# get_current_weather (.sh, .py, .ts) - 'r.jina.ai'
- "wttr.in:443" # get_current_weather (.sh, .py, .ts)
# search_arxiv (the .sh tool still uses http://, so :80 is required until fixed) - 'wttr.in'
- "export.arxiv.org:443" # search_arxiv (the .sh tool still uses http://, so :80 is required until fixed)
- "export.arxiv.org:80" - 'export.arxiv.org'
# search_arxiv + search_wikipedia may follow DOI redirects - 'export.arxiv.org:80'
- "doi.org:443" # search_arxiv + search_wikipedia may follow DOI redirects
# search_wikipedia - 'doi.org'
- "en.wikipedia.org:443" # search_wikipedia
# search_wolframalpha - 'en.wikipedia.org'
- "api.wolframalpha.com:443" # search_wolframalpha
# web_search_perplexity - 'api.wolframalpha.com'
- "api.perplexity.ai:443" # web_search_perplexity
# web_search_tavily - 'api.perplexity.ai'
- "api.tavily.com:443" # web_search_tavily
# send_twilio - 'api.tavily.com'
- "api.twilio.com:443" # send_twilio
# MCP: github (built-in mcp.json: api.githubcopilot.com) - 'api.twilio.com'
- "api.githubcopilot.com:443" # MCP: github (built-in mcp.json: api.githubcopilot.com)
# MCP: atlassian (built-in mcp.json: mcp-remote -> mcp.atlassian.com) - 'api.githubcopilot.com'
- "mcp.atlassian.com:443" # MCP: atlassian (built-in mcp.json: mcp-remote -> mcp.atlassian.com)
# MCP: ddg-search (built-in mcp.json: uvx duckduckgo-mcp-server) - 'mcp.atlassian.com'
- "duckduckgo.com:443" # MCP: ddg-search (built-in mcp.json: uvx duckduckgo-mcp-server)
- "html.duckduckgo.com:443" - 'duckduckgo.com'
- "lite.duckduckgo.com:443" - 'html.duckduckgo.com'
# MCP: npx-based servers (mcp-remote) pull from npm - 'lite.duckduckgo.com'
- "registry.npmjs.org:443" # MCP: npx-based servers (mcp-remote) pull from npm
# MCP: docker server may pull images from common registries - 'registry.npmjs.org'
- "ghcr.io:443" # MCP: docker server may pull images from common registries
- "registry-1.docker.io:443" - 'ghcr.io'
- "auth.docker.io:443" - 'registry-1.docker.io'
- "production.cloudflare.docker.com:443" - 'auth.docker.io'
+12 -1
View File
@@ -33,7 +33,18 @@ main() {
--exclude '.build' \ --exclude '.build' \
2>/dev/null | head -n "$MAX_RESULTS") || true 2>/dev/null | head -n "$MAX_RESULTS") || true
else else
results=$(find "$search_path" -type f -name "$glob_pattern" \ local name_pattern dir_prefix effective_search
name_pattern="${glob_pattern##*/}"
[[ -z "$name_pattern" || "$name_pattern" == "**" ]] && name_pattern="*"
if [[ "$glob_pattern" == */* ]]; then
dir_prefix="${glob_pattern%%\**}"
dir_prefix="${dir_prefix%/}"
effective_search="${search_path}${dir_prefix:+/$dir_prefix}"
else
effective_search="$search_path"
fi
[[ -d "$effective_search" ]] || effective_search="$search_path"
results=$(find "$effective_search" -type f -name "$name_pattern" \
-not -path '*/.git/*' \ -not -path '*/.git/*' \
-not -path '*/node_modules/*' \ -not -path '*/node_modules/*' \
-not -path '*/target/*' \ -not -path '*/target/*' \
+2 -2
View File
@@ -24,7 +24,7 @@ set -e
# your changes). # your changes).
# @option --path! The path of the file to apply the patch to # @option --path! The path of the file to apply the patch to
# @option --contents! The patch to apply to the file # @option --content! The patch to apply to the file
# @env LLM_OUTPUT=/dev/stdout The output path # @env LLM_OUTPUT=/dev/stdout The output path
@@ -33,7 +33,7 @@ source "$LLM_PROMPT_UTILS_FILE"
# shellcheck disable=SC2154 # shellcheck disable=SC2154
main() { main() {
argc_contents="$(jq -r '.contents' <<< "$LLM_TOOL_RAW_JSON")" argc_contents="$(jq -r '.content' <<< "$LLM_TOOL_RAW_JSON")"
argc_path="$(jq -r '.path' <<< "$LLM_TOOL_RAW_JSON")" argc_path="$(jq -r '.path' <<< "$LLM_TOOL_RAW_JSON")"
if [[ ! -f "$argc_path" ]]; then if [[ ! -f "$argc_path" ]]; then
+2 -2
View File
@@ -6,7 +6,7 @@ set -e
# sending less data, and is less prone to accidental data loss. # sending less data, and is less prone to accidental data loss.
# @option --path! The path of the file to write to # @option --path! The path of the file to write to
# @option --contents! The full contents to write to the file # @option --content! The full contents to write to the file
# @env LLM_OUTPUT=/dev/stdout The output path # @env LLM_OUTPUT=/dev/stdout The output path
@@ -15,7 +15,7 @@ source "$LLM_PROMPT_UTILS_FILE"
# shellcheck disable=SC2154 # shellcheck disable=SC2154
main() { main() {
argc_contents="$(jq -r '.contents' <<< "$LLM_TOOL_RAW_JSON")" argc_contents="$(jq -r '.content' <<< "$LLM_TOOL_RAW_JSON")"
argc_path="$(jq -r '.path' <<< "$LLM_TOOL_RAW_JSON")" argc_path="$(jq -r '.path' <<< "$LLM_TOOL_RAW_JSON")"
if [[ -f "$argc_path" ]]; then if [[ -f "$argc_path" ]]; then
+21
View File
@@ -0,0 +1,21 @@
#!/bin/sh
if [ -z "$SSH_AUTH_SOCK" ]; then
echo "WARNING: [git-ssh-sign] no SSH agent — cannot sign commits"
fi
KEY=$(ssh-add -L 2>/dev/null | head -1)
if [ -z "$KEY" ]; then
echo "WARNING: [git-ssh-sign] no keys in SSH agent — cannot sign commits"
fi
KEY_FILE="/home/agent/.config/git/signing_key.pub"
mkdir -p "$(dirname "$KEY_FILE")"
printf '%s\n' "$KEY" > "$KEY_FILE"
EMAIL=$(git config user.email 2>/dev/null || echo "agent@sandbox.local")
printf '%s %s\n' "$EMAIL" "$KEY" > "/home/agent/.config/git/allowed_signers"
local_hook="$(git rev-parse --git-dir)/hooks/pre-commit"
if [ -x "$local_hook" ]; then
exec "$local_hook" "$@"
fi
+328 -243
View File
@@ -3,9 +3,8 @@
# Setup (paths use $HOME so commands work in bash/zsh/PowerShell/Git Bash): # Setup (paths use $HOME so commands work in bash/zsh/PowerShell/Git Bash):
# sbx create --kit ./sbx-kit/ coyote --name testing . # sbx create --kit ./sbx-kit/ coyote --name testing .
# sbx cp $HOME/.config/coyote/ testing:/home/agent/.config/ # sbx cp $HOME/.config/coyote/ testing:/home/agent/.config/
# sbx cp $HOME/.coyote_password testing:/home/agent/
# sbx run testing --kit ./sbx-kit/ # sbx run testing --kit ./sbx-kit/
schemaVersion: '1' schemaVersion: '2'
kind: sandbox kind: sandbox
name: coyote name: coyote
displayName: Coyote displayName: Coyote
@@ -14,199 +13,256 @@ description: >
CLI & REPL mode, RAG, AI tools & agents, MCP servers, skills, and macros. CLI & REPL mode, RAG, AI tools & agents, MCP servers, skills, and macros.
sandbox: sandbox:
image: 'darkalex17/coyote:v0.7.4' image: 'darkalex17/coyote:v0.8.3'
aiFilename: COYOTE.md entrypoint: ['bash', '-lc', 'exec /home/agent/.cargo/bin/coyote']
entrypoint:
run: ['bash', '-lc', 'exec /home/agent/.cargo/bin/coyote']
network: permissions:
# Proxy-managed LLM providers: the proxy substitutes `proxy-managed` for network:
# the env var inside the sandbox and rewrites the auth header per allow:
# serviceAuth at request time. Multiple domains may map to one service # Coyote release + self-update + model-registry sync
# (e.g. jina) so they share a single credential. - 'github.com'
serviceDomains: - 'api.github.com'
api.openai.com: openai - 'raw.githubusercontent.com'
api.anthropic.com: anthropic - 'objects.githubusercontent.com'
generativelanguage.googleapis.com: gemini - '*.githubusercontent.com'
api.cohere.ai: cohere # Package managers and developer tools (cargo, uv, pip — useful at runtime for user installs)
api.groq.com: groq - 'crates.io'
openrouter.ai: openrouter - 'static.crates.io'
api.ai21.com: ai21 - 'pypi.org'
api.cloudflare.com: cloudflare - 'files.pythonhosted.org'
api.deepinfra.com: deepinfra - 'astral.sh'
api.deepseek.com: deepseek - 'sh.rustup.rs'
api.mistral.ai: mistral - 'static.rust-lang.org'
api.perplexity.ai: perplexity
api.voyageai.com: voyageai
api.x.ai: xai
api.jina.ai: jina
r.jina.ai: jina
qianfan.baidubce.com: ernie
api.hunyuan.cloud.tencent.com: hunyuan
api.minimax.chat: minimax
api.moonshot.cn: moonshot
dashscope.aliyuncs.com: qianwen
open.bigmodel.cn: zhipuai
serviceAuth:
openai:
headerName: Authorization
valueFormat: 'Bearer %s'
anthropic:
headerName: x-api-key
valueFormat: '%s'
gemini:
headerName: x-goog-api-key
valueFormat: '%s'
cohere:
headerName: Authorization
valueFormat: 'Bearer %s'
groq:
headerName: Authorization
valueFormat: 'Bearer %s'
openrouter:
headerName: Authorization
valueFormat: 'Bearer %s'
ai21:
headerName: Authorization
valueFormat: 'Bearer %s'
cloudflare:
headerName: Authorization
valueFormat: 'Bearer %s'
deepinfra:
headerName: Authorization
valueFormat: 'Bearer %s'
deepseek:
headerName: Authorization
valueFormat: 'Bearer %s'
mistral:
headerName: Authorization
valueFormat: 'Bearer %s'
perplexity:
headerName: Authorization
valueFormat: 'Bearer %s'
voyageai:
headerName: Authorization
valueFormat: 'Bearer %s'
xai:
headerName: Authorization
valueFormat: 'Bearer %s'
jina:
headerName: Authorization
valueFormat: 'Bearer %s'
ernie:
headerName: Authorization
valueFormat: 'Bearer %s'
hunyuan:
headerName: Authorization
valueFormat: 'Bearer %s'
minimax:
headerName: Authorization
valueFormat: 'Bearer %s'
moonshot:
headerName: Authorization
valueFormat: 'Bearer %s'
qianwen:
headerName: Authorization
valueFormat: 'Bearer %s'
zhipuai:
headerName: Authorization
valueFormat: 'Bearer %s'
allowedDomains:
# Coyote release + self-update + model-registry sync
- 'github.com:443'
- 'api.github.com:443'
- 'raw.githubusercontent.com:443'
- 'objects.githubusercontent.com:443'
- '*.githubusercontent.com:443'
# Package managers and developer tools (cargo, uv, pip — useful at runtime for user installs)
- 'crates.io:443'
- 'static.crates.io:443'
- 'pypi.org:443'
- 'files.pythonhosted.org:443'
- 'astral.sh:443'
- 'sh.rustup.rs:443'
- 'static.rust-lang.org:443'
# LLM model OAuth + API endpoints # LLM model OAuth + API endpoints
- 'claude.ai:443' - 'claude.ai'
- 'console.anthropic.com:443' - 'console.anthropic.com'
- 'accounts.google.com:443' - 'accounts.google.com'
# *.googleapis.com covers oauth2 + userinfo + VertexAI regional endpoints # *.googleapis.com covers oauth2 + userinfo + VertexAI regional endpoints
# (*-aiplatform.googleapis.com). Do not narrow without re-checking VertexAI. # (*-aiplatform.googleapis.com). Do not narrow without re-checking VertexAI.
- '*.googleapis.com:443' - '*.googleapis.com'
# Bedrock and GitHub Models use signed / GitHub-PAT auth that the proxy # Bedrock and GitHub Models use signed / GitHub-PAT auth that the proxy
# cannot rewrite. Domains are allow-listed; credentials must be injected # cannot rewrite; credentials must be injected separately (see README
# separately (see README "Extending"). # "Extending"). NOTE: '*.amazonaws.com' matches exactly ONE label, so
- '*.amazonaws.com:443' # two-label regional Bedrock hosts must be enumerated explicitly
- 'models.inference.ai.azure.com:443' # ('**.' is declared but not yet enforced by sbx). Add your region
# via a mixin if it's missing below.
- '*.amazonaws.com'
- 'bedrock-runtime.us-east-1.amazonaws.com'
- 'bedrock-runtime.us-east-2.amazonaws.com'
- 'bedrock-runtime.us-west-2.amazonaws.com'
- 'bedrock-runtime.eu-west-1.amazonaws.com'
- 'bedrock-runtime.eu-central-1.amazonaws.com'
- 'bedrock-runtime.ap-southeast-2.amazonaws.com'
- 'bedrock-runtime.ap-northeast-1.amazonaws.com'
- 'models.inference.ai.azure.com'
# Proxy-managed LLM provider APIs. Every credentials[].apiKey.inject
# domain below MUST also appear here. sbx does not derive allow entries
# from inject rules.
- 'api.openai.com'
- 'api.anthropic.com'
- 'generativelanguage.googleapis.com'
- 'api.cohere.ai'
- 'api.groq.com'
- 'openrouter.ai'
- 'api.ai21.com'
- 'api.cloudflare.com'
- 'api.deepinfra.com'
- 'api.deepseek.com'
- 'api.mistral.ai'
- 'api.perplexity.ai'
- 'api.voyageai.com'
- 'api.x.ai'
- 'api.jina.ai'
- 'r.jina.ai'
- 'qianfan.baidubce.com'
- 'api.hunyuan.cloud.tencent.com'
- 'api.minimax.chat'
- 'api.moonshot.cn'
- 'dashscope.aliyuncs.com'
- 'open.bigmodel.cn'
# Proxy-managed LLM providers: inside the sandbox each apiKey env var holds
# the `proxy-managed` sentinel; the proxy injects the real value into the
# request header per the inject rules at request time. Values are bound by
# the user via credential bindings (`sbx secret set <service>`); Coyote
# pre-seeds them from its vault at launch. Multiple domains may map to one
# service (e.g. jina) so they share a single credential.
credentials: credentials:
sources: - service: openai
openai: description: OpenAI API key, injected on api.openai.com
env: apiKey:
- OPENAI_API_KEY name: OPENAI_API_KEY
anthropic: proxyManaged: true
env: inject:
- ANTHROPIC_API_KEY - domain: api.openai.com
gemini: scheme: bearer
env: - service: anthropic
- GEMINI_API_KEY description: Anthropic API key, injected as x-api-key on api.anthropic.com
- GOOGLE_API_KEY apiKey:
cohere: name: ANTHROPIC_API_KEY
env: proxyManaged: true
- COHERE_API_KEY inject:
groq: - domain: api.anthropic.com
env: header: x-api-key
- GROQ_API_KEY format: '%s'
openrouter: - service: gemini
env: description: Google Gemini API key, injected as x-goog-api-key on generativelanguage.googleapis.com
- OPENROUTER_API_KEY apiKey:
ai21: name: GEMINI_API_KEY
env: proxyManaged: true
- AI21_API_KEY inject:
cloudflare: - domain: generativelanguage.googleapis.com
env: header: x-goog-api-key
- CLOUDFLARE_API_KEY format: '%s'
deepinfra: - service: cohere
env: description: Cohere API key, injected on api.cohere.ai
- DEEPINFRA_API_KEY apiKey:
deepseek: name: COHERE_API_KEY
env: proxyManaged: true
- DEEPSEEK_API_KEY inject:
mistral: - domain: api.cohere.ai
env: scheme: bearer
- MISTRAL_API_KEY - service: groq
perplexity: description: Groq API key, injected on api.groq.com
env: apiKey:
- PERPLEXITY_API_KEY name: GROQ_API_KEY
voyageai: proxyManaged: true
env: inject:
- VOYAGE_API_KEY - domain: api.groq.com
xai: scheme: bearer
env: - service: openrouter
- XAI_API_KEY description: OpenRouter API key, injected on openrouter.ai
jina: apiKey:
env: name: OPENROUTER_API_KEY
- JINA_API_KEY proxyManaged: true
ernie: inject:
env: - domain: openrouter.ai
- ERNIE_API_KEY scheme: bearer
hunyuan: - service: ai21
env: description: AI21 Labs API key, injected on api.ai21.com
- HUNYUAN_API_KEY apiKey:
minimax: name: AI21_API_KEY
env: proxyManaged: true
- MINIMAX_API_KEY inject:
moonshot: - domain: api.ai21.com
env: scheme: bearer
- MOONSHOT_API_KEY - service: cloudflare
qianwen: description: Cloudflare Workers AI API key, injected on api.cloudflare.com
env: apiKey:
- DASHSCOPE_API_KEY name: CLOUDFLARE_API_KEY
zhipuai: proxyManaged: true
env: inject:
- ZHIPUAI_API_KEY - domain: api.cloudflare.com
scheme: bearer
- service: deepinfra
description: DeepInfra API key, injected on api.deepinfra.com
apiKey:
name: DEEPINFRA_API_KEY
proxyManaged: true
inject:
- domain: api.deepinfra.com
scheme: bearer
- service: deepseek
description: DeepSeek API key, injected on api.deepseek.com
apiKey:
name: DEEPSEEK_API_KEY
proxyManaged: true
inject:
- domain: api.deepseek.com
scheme: bearer
- service: mistral
description: Mistral API key, injected on api.mistral.ai
apiKey:
name: MISTRAL_API_KEY
proxyManaged: true
inject:
- domain: api.mistral.ai
scheme: bearer
- service: perplexity
description: Perplexity API key, injected on api.perplexity.ai
apiKey:
name: PERPLEXITY_API_KEY
proxyManaged: true
inject:
- domain: api.perplexity.ai
scheme: bearer
- service: voyageai
description: Voyage AI API key, injected on api.voyageai.com
apiKey:
name: VOYAGE_API_KEY
proxyManaged: true
inject:
- domain: api.voyageai.com
scheme: bearer
- service: xai
description: xAI (Grok) API key, injected on api.x.ai
apiKey:
name: XAI_API_KEY
proxyManaged: true
inject:
- domain: api.x.ai
scheme: bearer
- service: jina
description: Jina API key, injected on api.jina.ai and r.jina.ai
apiKey:
name: JINA_API_KEY
proxyManaged: true
inject:
- domain: api.jina.ai
scheme: bearer
- domain: r.jina.ai
scheme: bearer
- service: ernie
description: Baidu ERNIE API key, injected on qianfan.baidubce.com
apiKey:
name: ERNIE_API_KEY
proxyManaged: true
inject:
- domain: qianfan.baidubce.com
scheme: bearer
- service: hunyuan
description: Tencent Hunyuan API key, injected on api.hunyuan.cloud.tencent.com
apiKey:
name: HUNYUAN_API_KEY
proxyManaged: true
inject:
- domain: api.hunyuan.cloud.tencent.com
scheme: bearer
- service: minimax
description: MiniMax API key, injected on api.minimax.chat
apiKey:
name: MINIMAX_API_KEY
proxyManaged: true
inject:
- domain: api.minimax.chat
scheme: bearer
- service: moonshot
description: Moonshot AI API key, injected on api.moonshot.cn
apiKey:
name: MOONSHOT_API_KEY
proxyManaged: true
inject:
- domain: api.moonshot.cn
scheme: bearer
- service: qianwen
description: Alibaba Qianwen (DashScope) API key, injected on dashscope.aliyuncs.com
apiKey:
name: DASHSCOPE_API_KEY
proxyManaged: true
inject:
- domain: dashscope.aliyuncs.com
scheme: bearer
- service: zhipuai
description: Zhipu AI (GLM) API key, injected on open.bigmodel.cn
apiKey:
name: ZHIPUAI_API_KEY
proxyManaged: true
inject:
- domain: open.bigmodel.cn
scheme: bearer
environment: environment:
variables: variables:
@@ -214,31 +270,54 @@ environment:
COYOTE_LOG_LEVEL: INFO COYOTE_LOG_LEVEL: INFO
COYOTE_CONFIG_DIR: /home/agent/.config/coyote COYOTE_CONFIG_DIR: /home/agent/.config/coyote
EDITOR: nano EDITOR: nano
proxyManaged: # Alias for the gemini credential: v2 apiKey supports a single env name
- OPENAI_API_KEY # (GEMINI_API_KEY above). Coyote also recognizes GOOGLE_API_KEY, so keep
- ANTHROPIC_API_KEY # it set to the sentinel. Header injection happens per-domain regardless
- GEMINI_API_KEY # of which env var the app reads.
- GOOGLE_API_KEY GOOGLE_API_KEY: proxy-managed
- COHERE_API_KEY
- GROQ_API_KEY setup:
- OPENROUTER_API_KEY files:
- AI21_API_KEY - path: /home/agent/.config/git/ssh-signing-key-command
- CLOUDFLARE_API_KEY mode: '0755'
- DEEPINFRA_API_KEY description: Resolve the forwarded SSH agent key for Git SSH signing
- DEEPSEEK_API_KEY content: |
- MISTRAL_API_KEY #!/bin/sh
- PERPLEXITY_API_KEY set -e
- VOYAGE_API_KEY
- XAI_API_KEY if [ -z "$SSH_AUTH_SOCK" ]; then
- JINA_API_KEY echo "WARNING: [git-ssh-sign] no SSH agent - cannot sign commits" >&2
- ERNIE_API_KEY fi
- HUNYUAN_API_KEY
- MINIMAX_API_KEY key=$(ssh-add -L 2>/dev/null | head -n 1)
- MOONSHOT_API_KEY if [ -z "$key" ]; then
- DASHSCOPE_API_KEY echo "WARNING: [git-ssh-sign] no keys in SSH agent - cannot sign commits" >&2
- ZHIPUAI_API_KEY fi
config_dir="$GIT_SSH_SIGN_CONFIG_DIR"
if [ -z "$config_dir" ]; then
config_dir="/home/agent/.config/git"
fi
mkdir -p "$config_dir"
email=$(git config user.email 2>/dev/null || printf '%s' "agent@sandbox.local")
printf '%s %s\n' "$email" "$key" > "$config_dir/allowed_signers"
printf 'key::%s\n' "$key"
install:
- command: |
git config --system gpg.format ssh
git config --system --unset-all user.signingKey || true
git config --system commit.gpgSign true
git config --system tag.gpgSign true
git config --system gpg.ssh.defaultKeyCommand /home/agent/.config/git/ssh-signing-key-command
git config --system gpg.ssh.allowedSignersFile /home/agent/.config/git/allowed_signers
if [ "$(git config --system --get core.hooksPath || true)" = "/home/agent/.config/git/hooks" ]; then
git config --system --unset-all core.hooksPath
fi
user: '0'
description: Configure SSH commit signing with a dynamic key command
commands:
startup: startup:
- command: - command:
[ [
@@ -250,39 +329,45 @@ commands:
background: false background: false
description: Bootstrap Coyote config directory on first sandbox start description: Bootstrap Coyote config directory on first sandbox start
agentContext: | agentInstructions:
## Sandbox environment filename: COYOTE.md
content: |
## Sandbox environment
You are running inside a Docker sandbox launched via `sbx run coyote`. The You are running inside a Docker sandbox launched via `sbx run coyote`. The
user's project workspace is mounted at its absolute host path and is the user's project workspace is mounted at its absolute host path and is the
current working directory. `sudo` is passwordless; use it for system current working directory. `sudo` is passwordless; use it for system
package installs. package installs.
Coyote's configuration lives at `~/.config/coyote/` and logs at Coyote's configuration lives at `~/.config/coyote/` and logs at
`~/.cache/coyote/coyote.log`. Persistence is enabled, so config, sessions, `~/.cache/coyote/coyote.log`. Persistence is enabled, so config, sessions,
vault state, OAuth tokens, and installed tools survive sandbox restarts. vault state, OAuth tokens, and installed tools survive sandbox restarts.
LLM provider credentials are forwarded by the sandbox HTTP proxy. The LLM provider credentials are forwarded by the sandbox HTTP proxy via
following provider env vars are recognized - export the ones you use on credential bindings. Coyote pre-seeds them from its vault at launch
the host before running `sbx run coyote`: (`sbx secret set <service>`); users can also bind values manually on the
host with `sbx secret set <service>` or `sbx secret import`. Recognized
services:
OPENAI_API_KEY, ANTHROPIC_API_KEY, GEMINI_API_KEY / GOOGLE_API_KEY, openai, anthropic, gemini, cohere, groq, openrouter, ai21,
COHERE_API_KEY, GROQ_API_KEY, OPENROUTER_API_KEY, AI21_API_KEY, cloudflare, deepinfra, deepseek, mistral, perplexity, voyageai,
CLOUDFLARE_API_KEY, DEEPINFRA_API_KEY, DEEPSEEK_API_KEY, xai, jina, ernie, hunyuan, minimax, moonshot, qianwen, zhipuai
MISTRAL_API_KEY, PERPLEXITY_API_KEY, VOYAGE_API_KEY, XAI_API_KEY,
JINA_API_KEY, ERNIE_API_KEY, HUNYUAN_API_KEY, MINIMAX_API_KEY,
MOONSHOT_API_KEY, DASHSCOPE_API_KEY (Qwen), ZHIPUAI_API_KEY
Inside the sandbox these appear as the placeholder string `proxy-managed`; Inside the sandbox the corresponding env vars (OPENAI_API_KEY, etc.)
the proxy substitutes the real value at request time. OAuth flows for hold the placeholder string `proxy-managed`; the proxy substitutes the
Claude Pro/Max and Gemini are also allow-listed. real value at request time. OAuth flows for Claude Pro/Max and Gemini
are also allow-listed.
Bedrock (AWS) and VertexAI (Google Cloud) use signed/OAuth-token requests Bedrock (AWS) and VertexAI (Google Cloud) use signed/OAuth-token requests
that the proxy cannot rewrite. Their domains are allow-listed but you must that the proxy cannot rewrite, so you must inject credentials yourself via
inject credentials yourself via `sbx run --env AWS_ACCESS_KEY_ID=...` or `sbx run --env AWS_ACCESS_KEY_ID=...` or a mixin kit that mounts a
a mixin kit that mounts a service-account JSON. service-account JSON. VertexAI regional endpoints are allow-listed via
`*.googleapis.com`. Bedrock runtime endpoints are allow-listed for
us-east-1/2, us-west-2, eu-west-1, eu-central-1, ap-southeast-2, and
ap-northeast-1 only; other regions need a mixin allow entry
(`bedrock-runtime.<region>.amazonaws.com`).
Useful first-run commands: Useful first-run commands:
- `coyote --info` # show config paths and resolved settings - `coyote --info` # show config paths and resolved settings
- `coyote --list-secrets` # initialise the local vault - `coyote --list-secrets` # initialise the local vault
- `coyote --authenticate <client>` # OAuth flow (Claude Pro/Max, Gemini) - `coyote --authenticate <client>` # OAuth flow (Claude Pro/Max, Gemini)
@@ -1,33 +0,0 @@
schemaVersion: "1"
kind: mixin
name: vault-aws-secrets-manager
description: >
Installs the AWS CLI v2 so the Coyote vault can read secrets from AWS
Secrets Manager inside the sandbox. The AWS Rust SDK does not strictly
require the CLI, but most users authenticate via `aws sso login` or
`aws configure`, which need the CLI to be installed. After install, run
the appropriate auth command in the sandbox; cached credentials persist
for the lifetime of the sandbox.
network:
allowedDomains:
- "awscli.amazonaws.com:443"
- "sts.amazonaws.com:443"
- "*.sts.amazonaws.com:443"
- "*.secretsmanager.amazonaws.com:443"
- "*.amazonaws.com:443"
- "*.awsapps.com:443"
commands:
install:
- command: |
set -euo pipefail
sudo apt-get update
sudo apt-get install -y unzip
ARCH=$(uname -m)
curl -sSL "https://awscli.amazonaws.com/awscli-exe-linux-${ARCH}.zip" -o /tmp/awscliv2.zip
unzip -q /tmp/awscliv2.zip -d /tmp
sudo /tmp/aws/install
rm -rf /tmp/awscliv2.zip /tmp/aws
user: "1000"
description: Install AWS CLI v2 from the official installer
@@ -1,24 +0,0 @@
schemaVersion: "1"
kind: mixin
name: vault-azure-key-vault
description: >
Installs the Azure CLI (`az`) so the Coyote vault can read secrets from
Azure Key Vault inside the sandbox. After install, run `az login` in the
sandbox to authenticate; the session token persists for the lifetime of
the sandbox.
network:
allowedDomains:
- "aka.ms:443"
- "packages.microsoft.com:443"
- "azurecliprod.blob.core.windows.net:443"
- "login.microsoftonline.com:443"
- "graph.microsoft.com:443"
- "management.azure.com:443"
- "*.vault.azure.net:443"
commands:
install:
- command: "curl -sL https://aka.ms/InstallAzureCLIDeb | sudo bash"
user: "1000"
description: Install Azure CLI via Microsoft's official install script
@@ -1,34 +0,0 @@
schemaVersion: "1"
kind: mixin
name: vault-gcp-secret-manager
description: >
Installs the Google Cloud CLI (`gcloud`) so the Coyote vault can read
secrets from GCP Secret Manager inside the sandbox. The GCP Rust SDK does
not strictly require the CLI, but most users authenticate via
`gcloud auth application-default login`, which needs the CLI to be
installed. After install, run that command in the sandbox; the ADC file
persists for the lifetime of the sandbox.
network:
allowedDomains:
- "packages.cloud.google.com:443"
- "accounts.google.com:443"
- "oauth2.googleapis.com:443"
- "secretmanager.googleapis.com:443"
- "cloudresourcemanager.googleapis.com:443"
- "*.googleapis.com:443"
commands:
install:
- command: |
set -euo pipefail
sudo apt-get update
sudo apt-get install -y apt-transport-https ca-certificates gnupg
echo "deb [signed-by=/usr/share/keyrings/cloud.google.gpg] https://packages.cloud.google.com/apt cloud-sdk main" \
| sudo tee /etc/apt/sources.list.d/google-cloud-sdk.list >/dev/null
curl -sSL https://packages.cloud.google.com/apt/doc/apt-key.gpg \
| sudo gpg --dearmor -o /usr/share/keyrings/cloud.google.gpg
sudo apt-get update
sudo apt-get install -y google-cloud-cli
user: "1000"
description: Install gcloud CLI from Google's official apt repository
-30
View File
@@ -1,30 +0,0 @@
schemaVersion: "1"
kind: mixin
name: vault-gopass
description: >
Installs `gopass` and `gpg` so the Coyote vault can read secrets from a
gopass store inside the sandbox. The store must be cloned manually
(gopass walks a user-specific git remote, so v1 only allowlists github.com
and gitlab.com; add other hosts via a user mixin if needed). After install,
run `gopass setup` or `gopass clone <remote>` in the sandbox.
network:
allowedDomains:
- "github.com:443"
- "api.github.com:443"
- "objects.githubusercontent.com:443"
- "gitlab.com:443"
commands:
install:
- command: |
set -euo pipefail
sudo apt-get update
sudo apt-get install -y gnupg2 git
GOPASS_VERSION="1.15.13"
ARCH=$(dpkg --print-architecture)
curl -sSL "https://github.com/gopasspw/gopass/releases/download/v${GOPASS_VERSION}/gopass_${GOPASS_VERSION}_linux_${ARCH}.deb" -o /tmp/gopass.deb
sudo dpkg -i /tmp/gopass.deb
rm -f /tmp/gopass.deb
user: "1000"
description: Install gnupg2, git, and gopass from the official .deb release
@@ -1,31 +0,0 @@
schemaVersion: "1"
kind: mixin
name: vault-one-password
description: >
Installs the 1Password CLI (`op`) so the Coyote vault can decrypt secrets
inside the sandbox. After install, run `op signin` in the sandbox to
authenticate; credentials persist for the lifetime of the sandbox.
network:
allowedDomains:
- "downloads.1password.com:443"
- "cache.agilebits.com:443"
- "my.1password.com:443"
- "my.1password.eu:443"
- "my.1password.ca:443"
- "events.1password.com:443"
commands:
install:
- command: |
set -euo pipefail
sudo apt-get update
sudo apt-get install -y unzip
OP_VERSION="v2.30.3"
ARCH=$(dpkg --print-architecture)
curl -sSL "https://cache.agilebits.com/dist/1P/op2/pkg/${OP_VERSION}/op_linux_${ARCH}_${OP_VERSION}.zip" -o /tmp/op.zip
sudo unzip -od /usr/local/bin /tmp/op.zip op
sudo chmod +x /usr/local/bin/op
rm -f /tmp/op.zip
user: "1000"
description: Install 1Password CLI from the official archive
+79
View File
@@ -0,0 +1,79 @@
---
description: AI-first design decomposition for any project. Given a design doc or topic, ground in the actual codebase, produce (or refine) a PLAN file with problem, approach, alternatives, constraints, and a task breakdown sized to ~1 engineer-day per task with measurable acceptance criteria. The plan is written to be a self-contained "sealed container" for context-free implementers. Grants filesystem access for grounding and for writing the plan.
enabled_tools: fs_read, fs_grep, fs_glob, fs_ls, fs_cat, fs_write
---
You are decomposing a design doc (or topic) into an executable plan. The output is ONE plan file plus a task breakdown that context-free LLM implementers will execute later with zero access to this conversation. Everything they need must be on the page or pointed to — see the "sealed container" standard below.
## Inputs
- A design doc (path or pasted), or a one-line problem statement.
- The target project directory (ground truth for all claims).
- The plans directory where the PLAN file lands.
## Step 1 — Ground before proposing
Plans written from memory rot on contact with the code. Before writing anything:
- Read the project's own orientation docs (`CLAUDE.md`, `AGENTS.md`, `CONTRIBUTING.md`, `README.md` at the project root) — conventions constrain the design.
- Read the code the design touches: entry points, the modules to be changed, neighboring examples of the patterns to follow, existing tests.
- `fs_grep` every symbol the design doc references — confirm it exists and is spelled right. Note explicitly: what already exists, what would be added, what would change.
- Verify build/test commands actually exist (`Makefile`, `justfile`, `package.json` scripts, CI config).
## Step 2 — The proposal
Produce a structured proposal (iterate with the user when interactive; in autonomous runs, resolve what the doc + code answer and flag the rest as open questions):
- **Problem** — one paragraph; state assumptions explicitly.
- **Scope** — In / Out. Call out tempting adjacent work being deferred.
- **Approach** — concrete: name files, symbols, data flow, migrations. Reference existing patterns by path.
- **Alternatives considered** — table of alternative → why rejected. Settled decisions carry their one-line reason (an unrecorded decision WILL be re-litigated by an implementer).
- **Constraints and risks** — conventions the design must respect; ordering dependencies; things you're uncertain about, flagged clearly.
- **Open questions** — ONLY questions the codebase cannot answer (business rules, priority calls). If none, say "No open questions."
- **Task breakdown** — see below.
## Task breakdown rules
| Rule | Why |
|---|---|
| **One task ≈ one engineer-day** | Variable task sizes destroy progress signal; anything larger gets decomposed NOW, not mid-run |
| Each task independently implementable and verifiable | It builds and its tests pass without later tasks existing |
| Explicit, acyclic dependencies (`blocked_by`) | Execution order must be derivable from the breakdown alone |
| Each task states WHERE (files/packages) and WHAT (observable outcome) | "Implement service layer" is not a task; "internal/foo/service.go: add Create/Get with validation — returns 400 on missing name" is |
| Measurable acceptance criteria per task | Criteria become the tests; "works correctly" is unmeasurable |
| Flag ⚠️ low-confidence sizing with the reason | Honest sizing beats optimistic sizing |
## Step 3 — Write the PLAN file
Write `PLAN-<slug>.md` (kebab-case slug from the topic; verify no collision) to the plans directory:
```markdown
---
slug: <slug>
status: draft # draft | active | implemented
created: YYYY-MM-DD
---
# <Title>
## Problem
## Scope (In / Out)
## Approach
## Alternatives considered
## Constraints and risks
## Open questions
## Task breakdown
| # | Task | Size | blocked_by | Notes |
|---|------|------|-----------|-------|
```
The plan is the implementers' entire context. Write for the "sealed container" standard: every question an implementer will hit is either answered inline or delegated via a pointer to the exact file/doc that answers it (where infra code goes, what DB tech, which layout to mirror, exact test commands). Paste short code snippets for load-bearing patterns — a path alone forces re-exploration; a stale claim fails the executor mid-implementation.
## Anti-patterns
- Proposing before reading the code — a design ungrounded in the actual codebase is fiction.
- "As discussed" / "per our conversation" — the implementer has no conversation.
- Tasks larger than a day hiding an "and then also…".
- Acceptance criteria describing implementation ("uses a for loop") instead of behavior.
- Open questions the code could have answered — grep first, ask last.
- Unrecorded decisions — every settled fork carries its reason.
+89
View File
@@ -0,0 +1,89 @@
---
description: Gatekeep a plan for self-containedness before it is finalized. A plan must be a "sealed container" - either it answers every question a context-free LLM implementer will hit, or it points at the exact code/docs where the answer lives. Produces the missing questions and a PLAN_GATE SEALED/LEAKY verdict. Grants read-only filesystem access for verifying pointers actually resolve. Complements plan-review (executability) - this checks completeness of context, not correctness of approach.
enabled_tools: fs_read, fs_grep, fs_glob, fs_cat, fs_ls
---
You are gatekeeping a plan before it is finalized. The standard is the **sealed-container test**: a fresh LLM implementer with ZERO conversation context and ZERO tribal knowledge will execute this plan. Every question that implementer would need answered mid-implementation must be either (a) **answered inline** in the plan, or (b) **delegated via a pointer** — an exact file/doc path that verifiably contains the answer. A plan that assumes the reader "just knows" where infrastructure code lives, which DB tech to use, or how services are laid out is a leaky container: the implementer will guess, and guesses become divergences.
You are NOT reviewing the approach (that is `plan-review`'s job — executability, verifiability, ordering). You are auditing **completeness of context**. A plan with a flawless approach still fails this gate if it leaves the implementer to rediscover the environment.
## The answer-or-pointer rule
For every question in the manifest below, the plan must contain ONE of:
1. **Inline answer** — the fact stated directly ("the service DB is Postgres on RDS, provisioned via `infra/rds/`", "migrations live in `internal/db/migrations/` and use goose").
2. **Verified pointer** — a path to code or docs where the implementer can discover it ("read `CLAUDE.md` § Database conventions", "mirror the layout of `internal/services/rate_cards/`").
An answer of neither kind = a missing question = a leak. "Follow existing conventions" with no pointer to WHICH file shows the convention is a leak. A pointer to a file that doesn't exist or doesn't actually cover the topic is a leak wearing a pointer costume — which is why you verify.
## The manifest (question categories to audit)
Walk EVERY category. For each, ask: "when the implementer hits this, does the plan answer it or point to the answer?"
| # | Category | Questions the implementer WILL hit |
|---|----------|-------------------------------------|
| 1 | **Code placement** | Which repo? Which directory/package? Does a new service/module follow an existing layout — which one, exactly? |
| 2 | **Infrastructure** | Where does infra code live? What is the deployment target (e.g. new DB in RDS via Terraform vs a Postgres container in Kubernetes)? Who provisions it — this plan's tasks, or a prerequisite? |
| 3 | **Data layer** | What DB tech/engine? What migration tool and directory? What naming conventions for tables/columns? Which existing tables does this touch or reference? |
| 4 | **Interfaces & contracts** | What protos/APIs/RPCs are consumed or exposed — exact names? Where do proto definitions live and how are they regenerated? What downstream consumers depend on the shapes this plan creates? |
| 5 | **Conventions & tooling** | Which language/framework versions? Error-handling and logging patterns — which file shows the canon? Lint/format/build commands? Where is the repo's own CLAUDE.md / contributor doc and does the plan tell the implementer to read it? |
| 6 | **Testing & verification** | Test framework and directory conventions? EXACT commands to run tests/build from the repo root? What proves each acceptance criterion? |
| 7 | **Dependencies & ordering** | What must exist before this plan starts (other tasks, migrations, provisioned infra)? What does this plan produce that later work depends on? |
| 8 | **Config, secrets & environments** | New env vars/config keys — where are they declared and injected? Secrets — vault/parameter store conventions? Staging vs production differences that affect implementation? |
| 9 | **Scope boundaries** | Is Out of scope present and specific? Are "tempting adjacent fixes" explicitly deferred? |
| 10 | **Settled decisions** | Are choices that were debated recorded WITH their one-line reason ("RDS over in-cluster Postgres because ops owns backups")? An unrecorded decision WILL be re-litigated by the implementer. |
Not every category applies to every plan (a docs-only plan has no data layer). Mark inapplicable categories as such — silently skipping one is how leaks survive.
## Pointer verification (do not trust, verify)
For every pointer the plan offers:
1. `fs_ls` / `fs_glob` — the referenced path exists.
2. `fs_grep` / `fs_read` — the file actually covers the claimed topic. A plan saying "see `docs/database.md` for migration conventions" fails verification if that file never mentions migrations.
3. For "mirror the layout of X" pointers — confirm X exists and is a real example of what the plan claims (a service directory held up as the canonical layout should actually contain the layers the plan describes).
A broken pointer is worse than no pointer: it burns the implementer's time AND their trust in the rest of the plan.
## Severity honesty
Not every gap is equal. Tag each finding:
- **BLOCKING** — the implementer cannot proceed or will guess wrong with expensive consequences (wrong DB target, wrong repo, missing prerequisite).
- **FRICTION** — the implementer can discover the answer but will waste significant time re-exploring what the author already knew.
A plan with only FRICTION findings may still be sealed at the caller's discretion — say so. BLOCKING findings always mean LEAKY.
## Verdict format
End with EXACTLY one of:
```
PLAN_GATE: SEALED
Categories audited: N applicable, all answered or pointed.
<optional: 1-3 non-blocking observations>
```
```
PLAN_GATE: LEAKY
Missing questions (N):
1. [category] <the exact question the implementer will hit> — [BLOCKING|FRICTION] — <why they get stuck or guess wrong> — <suggested fix: the inline answer to add, or the pointer to insert (verified to exist)>
2. ...
Broken pointers (if any):
- <plan's pointer> — <what's wrong: path missing / doesn't cover topic>
```
Every missing question must be phrased as the QUESTION the implementer would actually ask ("where do I put the Terraform for the new RDS instance?"), not as an abstract complaint ("infra section is thin"). When you suggest a pointer as the fix, VERIFY it first — never recommend a pointer you haven't confirmed resolves.
## Scope discipline
- Do not redesign the approach. If the approach is coherent but under-documented, the fix is context, not redesign.
- Do not demand encyclopedic plans. The container test is "answered or pointed" — a tight plan full of verified pointers beats a bloated plan that inlines the whole wiki. Flag over-inlining only if it duplicates something that WILL drift (e.g. pasted conventions that contradict the source file).
- Three BLOCKING questions beat fifteen FRICTION nitpicks. If your list is all nitpicks, the plan is probably SEALED — say so.
## Anti-patterns
- Sealing a plan because the approach is good, without walking the manifest.
- Flagging "missing context" without phrasing the actual question the implementer would ask.
- Recommending a pointer you did not verify exists and covers the topic.
- Treating an inapplicable category as a leak (demanding a data-layer section from a docs-only plan).
- Re-reviewing executability/approach — that is `plan-review`'s lane.
+87
View File
@@ -0,0 +1,87 @@
---
description: File-based task tracking for plan-driven runs on any project. Defines the TASK-NNN directory schema (index.md + append-only log.md), the frontmatter lifecycle (pending/in-progress/blocked/complete), numbering, the completion protocol, and follow-up task creation. The tasks directory on disk is the durable run state - it survives context compression. Grants filesystem access for managing task files.
enabled_tools: fs_read, fs_grep, fs_glob, fs_ls, fs_cat, fs_write, fs_patch, fs_mkdir
---
You are tracking implementation tasks as files. The task directory is the durable source of truth for run state — anything that lives only in chat history is lost to context compression. Keep it current at every state change, not in batches.
## Layout
```
<plans_dir>/
PLAN-<slug>.md # the plan (see design-session / plan-authoring)
tasks/
TASK-001-<slug>/
index.md # current state: frontmatter + What/Steps/Acceptance criteria
log.md # append-only audit trail
TASK-002-<slug>/
...
```
## index.md schema
```markdown
---
title: <short imperative title>
status: pending # pending | in-progress | blocked | complete
type: feature # feature | chore | followup
points: 1.0 # engineer-days; ~1.0 per the sizing rule
plan: PLAN-<slug>.md
blocked_by: [] # TASK ids that must be complete first
created: YYYY-MM-DD
---
## What
One paragraph: what this task produces, named concretely (files, symbols, behaviors).
## Steps
- [ ] Concrete step — name the file, function, or migration
- [ ] ...
## Acceptance criteria
- [ ] Observable behavior, measurable ("returns 429 after 3 failed attempts")
- [ ] ...
```
Status lives in frontmatter — there are no lifecycle directories. `status: complete` plus all boxes checked IS done.
## log.md conventions
Append-only. Each entry is an H2: `## YYYY-MM-DD — <short label>` (`created`, `started`, `implemented`, `diverged`, `completed`, ...). Body is 1-3 sentences of prose; structured data lives in markdown links (branch URLs, commit SHAs, PR links). Never rewrite an old entry — add a new one.
## Numbering
Scan `tasks/TASK-*` for the highest NNN and increment (zero-padded to 3). This assumes a single writer per plans directory; if multiple agents or people share one, serialize task creation.
## Lifecycle protocol
| Transition | Do |
|---|---|
| Create | `fs_mkdir` the dir; write `index.md` (status: pending) + `log.md` with a `created` entry |
| Claim | frontmatter `status: in-progress`; log `started` (note the branch + base SHA) |
| Blocked | `status: blocked`; log why and what unblocks it |
| Complete | Check off every Step and Acceptance criterion (verified, not aspirational); log `completed` with commit SHAs AND any follow-ups reported by the implementer, VERBATIM; set `status: complete` |
Never mark a criterion checked without evidence. Never batch state changes — update at the moment of transition.
## Follow-up tasks
When implementation surfaces manual/out-of-scope actions (secrets to create, cloud roles to provision, console steps, cross-repo changes): create a task per item (group small related ones) with `type: followup`, `status: pending`, the WHAT/WHERE/WHY/WHEN in its What section, and a note of which TASK surfaced it. Follow-ups are deliverables to hand to the user — never implement them in the current run.
## Consistency checks (run at the end of a run)
- Every task dir has both `index.md` and `log.md`.
- Every `blocked_by` reference resolves to an existing task.
- No task is `complete` with unchecked Steps/Acceptance criteria.
- Every `complete` task's log has a `completed` entry with commit references.
- The PLAN's breakdown table rows all map to task dirs (and vice versa).
## Anti-patterns
- Run state that exists only in chat — session ids, decisions, and follow-ups belong in task files.
- `status: complete` with unchecked boxes, or checked boxes without evidence.
- Rewriting log history instead of appending.
- Hand-picking a task number without scanning (collisions).
- Follow-ups mentioned in a summary but never materialized as task files.
+2
View File
@@ -187,6 +187,8 @@ summarization_prompt: > # The text prompt used for creating a concise s
'Summarize the discussion briefly in 200 words or less to use as a prompt for future context.' 'Summarize the discussion briefly in 200 words or less to use as a prompt for future context.'
summary_context_prompt: > # The text prompt used for including the summary of the entire session as context to the model summary_context_prompt: > # The text prompt used for including the summary of the entire session as context to the model
'This is a summary of the chat history as a recap: ' 'This is a summary of the chat history as a recap: '
compression_keep_last: 0 # Number of most-recent messages to keep visible after compression (0 = compress all messages)
max_tool_result_chars: null # Cap on tool result characters forwarded to the model per call (null = no cap)
# ---- Memory ---- # ---- Memory ----
# See the [Memory documentation](https://github.com/Dark-Alex-17/coyote/wiki/Memory) for more information. # See the [Memory documentation](https://github.com/Dark-Alex-17/coyote/wiki/Memory) for more information.
+1 -1
View File
@@ -14,7 +14,7 @@ class Coyote < Formula
sha256 "$hash_linux" sha256 "$hash_linux"
end end
version "$version" version "$version"
license "MIT" license "AGPL-3.0-only"
def install def install
bin.install "coyote" bin.install "coyote"
+40
View File
@@ -413,6 +413,16 @@
# - https://docs.anthropic.com/en/api/messages # - https://docs.anthropic.com/en/api/messages
- provider: claude - provider: claude
models: models:
- name: claude-opus-5
max_input_tokens: 1000000
max_output_tokens: 128000
require_max_tokens: true
input_price: 5
output_price: 25
supports_function_calling: true
supports_vision: true
reasoning_levels: [low, medium, high, xhigh, max]
default_reasoning_effort: high
- name: claude-fable-5 - name: claude-fable-5
max_input_tokens: 1000000 max_input_tokens: 1000000
max_output_tokens: 128000 max_output_tokens: 128000
@@ -1046,6 +1056,16 @@
supports_function_calling: true supports_function_calling: true
reasoning_levels: [minimal, low, medium, high] reasoning_levels: [minimal, low, medium, high]
default_reasoning_effort: high default_reasoning_effort: high
- name: claude-opus-5
max_input_tokens: 1000000
max_output_tokens: 128000
require_max_tokens: true
input_price: 5
output_price: 25
supports_function_calling: true
supports_vision: true
reasoning_levels: [low, medium, high, xhigh, max]
default_reasoning_effort: high
- name: claude-fable-5 - name: claude-fable-5
max_input_tokens: 1000000 max_input_tokens: 1000000
max_output_tokens: 128000 max_output_tokens: 128000
@@ -1243,6 +1263,16 @@
# - https://docs.aws.amazon.com/bedrock/latest/userguide/conversation-inference-call.html # - https://docs.aws.amazon.com/bedrock/latest/userguide/conversation-inference-call.html
- provider: bedrock - provider: bedrock
models: models:
- name: us.anthropic.claude-opus-5
max_input_tokens: 1000000
max_output_tokens: 128000
require_max_tokens: true
input_price: 5
output_price: 25
supports_function_calling: true
supports_vision: true
reasoning_levels: [low, medium, high, xhigh, max]
default_reasoning_effort: high
- name: us.anthropic.claude-fable-5 - name: us.anthropic.claude-fable-5
max_input_tokens: 1000000 max_input_tokens: 1000000
max_output_tokens: 128000 max_output_tokens: 128000
@@ -2111,6 +2141,16 @@
max_input_tokens: 131072 max_input_tokens: 131072
input_price: 0.1 input_price: 0.1
output_price: 0.2 output_price: 0.2
- name: anthropic/claude-opus-5
max_input_tokens: 1000000
max_output_tokens: 128000
require_max_tokens: true
input_price: 5
output_price: 25
supports_function_calling: true
supports_vision: true
reasoning_levels: [low, medium, high, xhigh, max]
default_reasoning_effort: high
- name: anthropic/claude-fable-5 - name: anthropic/claude-fable-5
max_input_tokens: 1000000 max_input_tokens: 1000000
max_output_tokens: 128000 max_output_tokens: 128000
+4
View File
@@ -0,0 +1,4 @@
mod server;
mod types;
pub use server::run_acp_server;
+670
View File
@@ -0,0 +1,670 @@
use super::types::{METHOD_NOT_FOUND, PARSE_ERROR, Request, Response};
use crate::client::call_chat_completions_streaming;
use crate::config::{Input, RenderMode, RequestContext};
use crate::function::supervisor::{GuardrailAction, check_pending_agents_guardrail};
use crate::utils;
use crate::utils::AbortSignal;
use anyhow::Result;
use serde_json::{Value, json};
use std::sync::Arc;
use tokio::io::{AsyncBufReadExt, AsyncRead, AsyncWrite, AsyncWriteExt, BufReader};
pub(crate) struct AcpServerState {
ctx: Option<RequestContext>,
abort: AbortSignal,
session_active: bool,
}
pub async fn run_acp_server(ctx: RequestContext, abort: AbortSignal) -> Result<()> {
let state = AcpServerState {
ctx: Some(ctx),
abort,
session_active: false,
};
run_acp_server_with_state(tokio::io::stdin(), tokio::io::stdout(), state).await
}
#[cfg(test)]
pub(crate) async fn run_acp_server_on<R, W>(reader: R, writer: W) -> Result<()>
where
R: AsyncRead + Unpin,
W: AsyncWrite + Unpin,
{
use crate::utils::{create_abort_signal, drain_acp_permissions};
drain_acp_permissions();
let state = AcpServerState {
ctx: None,
abort: create_abort_signal(),
session_active: false,
};
run_acp_server_with_state(reader, writer, state).await
}
async fn run_acp_server_with_state<R, W>(
reader: R,
mut writer: W,
mut state: AcpServerState,
) -> Result<()>
where
R: AsyncRead + Unpin,
W: AsyncWrite + Unpin,
{
let reader = BufReader::new(reader);
let mut lines = reader.lines();
while let Some(line) = lines.next_line().await? {
let line = line.trim().to_string();
if line.is_empty() {
continue;
}
if let Some(response) = dispatch(&line, &mut state).await {
for params in utils::drain_acp_permissions() {
emit_notification(&mut writer, "session/request_permission", params).await?;
}
emit(&mut writer, &response).await?;
}
}
Ok(())
}
async fn dispatch(raw: &str, state: &mut AcpServerState) -> Option<Response> {
let req: Request = match serde_json::from_str(raw) {
Ok(r) => r,
Err(_) => return Some(Response::err(None, PARSE_ERROR, "Parse error")),
};
// session/cancel is a notification. Handle it regardless of whether an id is present.
if req.method == "session/cancel" {
handle_session_cancel(state);
return if req.id.is_some() {
Some(Response::ok(req.id, json!({})))
} else {
None
};
}
req.id.as_ref()?;
Some(match req.method.as_str() {
"initialize" => handle_initialize(req),
"session/new" => handle_session_new(req, state).await,
"session/load" => handle_session_load(req, state).await,
"session/prompt" => handle_session_prompt(req, state).await,
_ => Response::err(
req.id,
METHOD_NOT_FOUND,
format!("Method not found: {}", req.method),
),
})
}
fn handle_initialize(req: Request) -> Response {
Response::ok(
req.id,
json!({
"name": "coyote",
"version": env!("CARGO_PKG_VERSION"),
"protocolVersion": 1,
}),
)
}
async fn handle_session_new(req: Request, state: &mut AcpServerState) -> Response {
if state.session_active {
return Response::err(
req.id,
-32000,
"Session already active; this server supports one session per process",
);
}
let ctx = match state.ctx.as_mut() {
Some(c) => c,
None => {
state.session_active = true;
return Response::ok(req.id, json!({ "sessionId": "default" }));
}
};
let app = Arc::clone(&ctx.app.config);
let abort = state.abort.clone();
match ctx.use_session(app.as_ref(), None, abort).await {
Ok(_) => {
state.session_active = true;
ctx.render_mode = RenderMode::Silent;
Response::ok(req.id, json!({ "sessionId": "default" }))
}
Err(e) => Response::err(req.id, -32000, format!("Failed to create session: {e}")),
}
}
async fn handle_session_prompt(req: Request, state: &mut AcpServerState) -> Response {
if !state.session_active {
return Response::err(req.id, -32000, "No active session; call session/new first");
}
let params = req.params.as_ref();
let from_content_blocks: Option<String> = params
.and_then(|p| p.get("prompt"))
.and_then(Value::as_array)
.map(|blocks| {
blocks
.iter()
.filter(|b| b.get("type").and_then(Value::as_str) == Some("text"))
.filter_map(|b| b.get("text").and_then(Value::as_str))
.collect::<Vec<_>>()
.join("\n")
})
.filter(|s| !s.is_empty());
let from_text: Option<String> = params
.and_then(|p| p.get("text"))
.and_then(Value::as_str)
.filter(|s| !s.is_empty())
.map(str::to_string);
let text = match from_content_blocks.or(from_text) {
Some(t) => t,
None => {
return Response::err(
req.id,
-32602,
"Missing params: expected prompt (ContentBlock array) or text",
);
}
};
let ctx = match state.ctx.as_mut() {
Some(c) => c,
None => return Response::err(req.id, -32000, "Server not configured with a context"),
};
let abort = state.abort.clone();
match run_prompt_turn(ctx, &text, abort).await {
Ok(output) => Response::ok(
req.id,
json!({ "output": output, "stopReason": "end_turn" }),
),
Err(e) => Response::err(req.id, -32000, format!("Prompt failed: {e}")),
}
}
async fn run_prompt_turn(
ctx: &mut RequestContext,
text: &str,
abort: AbortSignal,
) -> Result<String> {
ctx.render_mode = RenderMode::Silent;
let mut input = Input::from_str(ctx, text, None)?;
loop {
ctx.before_chat_completion(&input)?;
let client = input.create_client()?;
let (output, tool_results) =
call_chat_completions_streaming(&input, client.as_ref(), ctx, abort.clone()).await?;
let app = Arc::clone(&ctx.app.config);
ctx.after_chat_completion(app.as_ref(), &input, &output, &tool_results)?;
if !tool_results.is_empty() {
input = input.merge_tool_results(output, tool_results);
continue;
}
match check_pending_agents_guardrail(ctx) {
GuardrailAction::Inject(prompt) => {
input = Input::from_str(ctx, &prompt, None)?;
}
GuardrailAction::ForceTerminate(ids) => {
warn!(
"Pending-agent guardrail force-cancelled {} agent(s): {:?}",
ids.len(),
ids
);
return Ok(output);
}
GuardrailAction::NoAction => return Ok(output),
}
}
}
fn handle_session_cancel(state: &mut AcpServerState) {
state.abort.set_ctrlc();
}
async fn handle_session_load(req: Request, state: &mut AcpServerState) -> Response {
if state.session_active {
return Response::err(req.id, -32000, "Session already active");
}
let session_name = match req
.params
.as_ref()
.and_then(|p| p.get("sessionId"))
.and_then(Value::as_str)
{
Some(n) => n.to_string(),
None => return Response::err(req.id, -32602, "Missing params.sessionId"),
};
let ctx = match state.ctx.as_mut() {
Some(c) => c,
None => return Response::err(req.id, -32000, "Server not configured with a context"),
};
let app = Arc::clone(&ctx.app.config);
let abort = state.abort.clone();
match ctx
.use_session(app.as_ref(), Some(&session_name), abort)
.await
{
Ok(_) => {
state.session_active = true;
ctx.render_mode = RenderMode::Silent;
Response::ok(req.id, json!({ "sessionId": session_name }))
}
Err(e) => Response::err(req.id, -32000, format!("Failed to load session: {e}")),
}
}
async fn emit<W: AsyncWrite + Unpin>(writer: &mut W, response: &Response) -> Result<()> {
let mut line = serde_json::to_string(response)?;
line.push('\n');
writer.write_all(line.as_bytes()).await?;
writer.flush().await?;
Ok(())
}
async fn emit_notification<W: AsyncWrite + Unpin>(
writer: &mut W,
method: &str,
params: Value,
) -> Result<()> {
let frame = json!({
"jsonrpc": "2.0",
"method": method,
"params": params,
});
let mut line = serde_json::to_string(&frame)?;
line.push('\n');
writer.write_all(line.as_bytes()).await?;
writer.flush().await?;
Ok(())
}
#[cfg(test)]
mod tests {
use super::*;
use std::str;
#[tokio::test]
async fn all_stdout_is_valid_json_rpc() {
let input = concat!(
r#"{"jsonrpc":"2.0","id":1,"method":"initialize","params":{"name":"test","version":"0.1.0"}}"#,
"\n",
);
let mut output = Vec::new();
run_acp_server_on(input.as_bytes(), &mut output)
.await
.unwrap();
for line in output.split(|&b| b == b'\n').filter(|l| !l.is_empty()) {
let s = str::from_utf8(line).expect("non-UTF8 in ACP stdout");
let _: Value = serde_json::from_str(s)
.unwrap_or_else(|_| panic!("ACP stdout not valid JSON: {s}"));
}
}
#[tokio::test]
async fn unknown_method_returns_method_not_found() {
let input = concat!(
r#"{"jsonrpc":"2.0","id":2,"method":"nonexistent","params":{}}"#,
"\n",
);
let mut output = Vec::new();
run_acp_server_on(input.as_bytes(), &mut output)
.await
.unwrap();
let s = String::from_utf8(output).unwrap();
let v: Value = serde_json::from_str(s.trim()).unwrap();
assert_eq!(v["error"]["code"], METHOD_NOT_FOUND);
assert_eq!(v["id"], 2);
}
#[tokio::test]
async fn invalid_json_returns_parse_error() {
let input = "not json\n";
let mut output = Vec::new();
run_acp_server_on(input.as_bytes(), &mut output)
.await
.unwrap();
let s = String::from_utf8(output).unwrap();
let v: Value = serde_json::from_str(s.trim()).unwrap();
assert_eq!(v["error"]["code"], PARSE_ERROR);
}
#[tokio::test]
async fn notification_without_id_produces_no_output() {
let input = concat!(
r#"{"jsonrpc":"2.0","method":"session/cancel","params":{}}"#,
"\n",
);
let mut output = Vec::new();
run_acp_server_on(input.as_bytes(), &mut output)
.await
.unwrap();
assert!(output.is_empty());
}
#[tokio::test]
async fn initialize_returns_server_info() {
let input = concat!(
r#"{"jsonrpc":"2.0","id":1,"method":"initialize","params":{"name":"test","version":"0.1.0"}}"#,
"\n",
);
let mut output = Vec::new();
run_acp_server_on(input.as_bytes(), &mut output)
.await
.unwrap();
let s = String::from_utf8(output).unwrap();
let v: Value = serde_json::from_str(s.trim()).unwrap();
assert_eq!(v["id"], 1);
assert_eq!(v["result"]["name"], "coyote");
assert!(v["result"]["version"].is_string());
}
#[tokio::test]
async fn session_new_returns_session_id() {
let input = concat!(
r#"{"jsonrpc":"2.0","id":10,"method":"session/new","params":{}}"#,
"\n",
);
let mut output = Vec::new();
run_acp_server_on(input.as_bytes(), &mut output)
.await
.unwrap();
let s = String::from_utf8(output).unwrap();
let v: Value = serde_json::from_str(s.trim()).unwrap();
assert_eq!(v["id"], 10);
assert_eq!(v["result"]["sessionId"], "default");
}
#[tokio::test]
async fn session_new_twice_errors() {
let input = concat!(
r#"{"jsonrpc":"2.0","id":1,"method":"session/new","params":{}}"#,
"\n",
r#"{"jsonrpc":"2.0","id":2,"method":"session/new","params":{}}"#,
"\n",
);
let mut output = Vec::new();
run_acp_server_on(input.as_bytes(), &mut output)
.await
.unwrap();
let s = String::from_utf8(output).unwrap();
let mut lines = s.lines().filter(|l| !l.is_empty());
let first: Value = serde_json::from_str(lines.next().unwrap()).unwrap();
let second: Value = serde_json::from_str(lines.next().unwrap()).unwrap();
assert!(first["result"]["sessionId"].is_string());
assert_eq!(second["error"]["code"], -32000);
}
#[tokio::test]
async fn session_prompt_without_session_errors() {
let input = concat!(
r#"{"jsonrpc":"2.0","id":3,"method":"session/prompt","params":{"text":"hello"}}"#,
"\n",
);
let mut output = Vec::new();
run_acp_server_on(input.as_bytes(), &mut output)
.await
.unwrap();
let s = String::from_utf8(output).unwrap();
let v: Value = serde_json::from_str(s.trim()).unwrap();
assert_eq!(v["id"], 3);
assert_eq!(v["error"]["code"], -32000);
}
#[tokio::test]
async fn session_prompt_with_no_context_returns_error() {
let input = concat!(
r#"{"jsonrpc":"2.0","id":1,"method":"session/new","params":{}}"#,
"\n",
r#"{"jsonrpc":"2.0","id":2,"method":"session/prompt","params":{"text":"hello"}}"#,
"\n",
);
let mut output = Vec::new();
run_acp_server_on(input.as_bytes(), &mut output)
.await
.unwrap();
let s = String::from_utf8(output).unwrap();
let mut lines = s.lines().filter(|l| !l.is_empty());
let first: Value = serde_json::from_str(lines.next().unwrap()).unwrap();
let second: Value = serde_json::from_str(lines.next().unwrap()).unwrap();
assert_eq!(first["result"]["sessionId"], "default");
assert_eq!(second["id"], 2);
assert!(
second["error"].is_object(),
"expected error response when no ctx"
);
}
#[tokio::test]
async fn session_cancel_notification_produces_no_output() {
let input = concat!(r#"{"jsonrpc":"2.0","method":"session/cancel"}"#, "\n",);
let mut output = Vec::new();
run_acp_server_on(input.as_bytes(), &mut output)
.await
.unwrap();
assert!(output.is_empty());
}
#[tokio::test]
async fn session_cancel_request_returns_ok() {
let input = concat!(
r#"{"jsonrpc":"2.0","id":99,"method":"session/cancel"}"#,
"\n",
);
let mut output = Vec::new();
run_acp_server_on(input.as_bytes(), &mut output)
.await
.unwrap();
let s = String::from_utf8(output).unwrap();
let v: Value = serde_json::from_str(s.trim()).unwrap();
assert_eq!(v["id"], 99);
assert!(v["result"].is_object());
}
#[tokio::test]
async fn session_load_missing_session_id_errors() {
let input = concat!(
r#"{"jsonrpc":"2.0","id":5,"method":"session/load","params":{}}"#,
"\n",
);
let mut output = Vec::new();
run_acp_server_on(input.as_bytes(), &mut output)
.await
.unwrap();
let s = String::from_utf8(output).unwrap();
let v: Value = serde_json::from_str(s.trim()).unwrap();
assert_eq!(v["id"], 5);
assert_eq!(v["error"]["code"], -32602);
}
#[tokio::test]
async fn session_load_after_session_new_errors() {
let input = concat!(
r#"{"jsonrpc":"2.0","id":1,"method":"session/new","params":{}}"#,
"\n",
r#"{"jsonrpc":"2.0","id":2,"method":"session/load","params":{"sessionId":"abc"}}"#,
"\n",
);
let mut output = Vec::new();
run_acp_server_on(input.as_bytes(), &mut output)
.await
.unwrap();
let s = String::from_utf8(output).unwrap();
let mut lines = s.lines().filter(|l| !l.is_empty());
let _first: Value = serde_json::from_str(lines.next().unwrap()).unwrap();
let second: Value = serde_json::from_str(lines.next().unwrap()).unwrap();
assert_eq!(second["id"], 2);
assert_eq!(second["error"]["code"], -32000);
}
#[tokio::test]
async fn session_load_with_no_context_returns_error() {
let input = concat!(
r#"{"jsonrpc":"2.0","id":6,"method":"session/load","params":{"sessionId":"my-session"}}"#,
"\n",
);
let mut output = Vec::new();
run_acp_server_on(input.as_bytes(), &mut output)
.await
.unwrap();
let s = String::from_utf8(output).unwrap();
let v: Value = serde_json::from_str(s.trim()).unwrap();
assert_eq!(v["id"], 6);
assert!(v["error"].is_object());
}
#[tokio::test]
async fn emit_notification_produces_valid_json_rpc_frame() {
let mut output = Vec::new();
emit_notification(
&mut output,
"session/request_permission",
json!({"action": "confirm", "question": "Proceed?"}),
)
.await
.unwrap();
let s = String::from_utf8(output).unwrap();
let v: Value = serde_json::from_str(s.trim()).unwrap();
assert_eq!(v["jsonrpc"], "2.0");
assert_eq!(v["method"], "session/request_permission");
assert!(v["params"]["action"].is_string());
assert!(!v.as_object().unwrap().contains_key("id"));
}
#[tokio::test]
async fn initialize_protocol_version_is_number() {
let input = concat!(
r#"{"jsonrpc":"2.0","id":1,"method":"initialize","params":{}}"#,
"\n",
);
let mut output = Vec::new();
run_acp_server_on(input.as_bytes(), &mut output)
.await
.unwrap();
let s = String::from_utf8(output).unwrap();
let v: Value = serde_json::from_str(s.trim()).unwrap();
assert!(
v["result"]["protocolVersion"].is_number(),
"protocolVersion must be a JSON number, got: {:?}",
v["result"]["protocolVersion"]
);
assert_eq!(v["result"]["protocolVersion"], 1);
}
#[tokio::test]
async fn session_prompt_spec_content_blocks_not_rejected() {
let input = concat!(
r#"{"jsonrpc":"2.0","id":1,"method":"session/new","params":{}}"#,
"\n",
r#"{"jsonrpc":"2.0","id":2,"method":"session/prompt","params":{"sessionId":"default","prompt":[{"type":"text","text":"hello"}]}}"#,
"\n",
);
let mut output = Vec::new();
run_acp_server_on(input.as_bytes(), &mut output)
.await
.unwrap();
let s = String::from_utf8(output).unwrap();
let mut lines = s.lines().filter(|l| !l.is_empty());
let _first: Value = serde_json::from_str(lines.next().unwrap()).unwrap();
let second: Value = serde_json::from_str(lines.next().unwrap()).unwrap();
assert_eq!(second["id"], 2);
let code = second["error"]["code"].as_i64().unwrap_or(0);
assert_ne!(
code, -32602,
"spec-shaped ContentBlock prompt must not get a params error"
);
}
#[tokio::test]
async fn session_prompt_non_text_blocks_ignored() {
let input = concat!(
r#"{"jsonrpc":"2.0","id":1,"method":"session/new","params":{}}"#,
"\n",
r#"{"jsonrpc":"2.0","id":2,"method":"session/prompt","params":{"prompt":[{"type":"image","data":"abc"},{"type":"text","text":"hello"}]}}"#,
"\n",
);
let mut output = Vec::new();
run_acp_server_on(input.as_bytes(), &mut output)
.await
.unwrap();
let s = String::from_utf8(output).unwrap();
let mut lines = s.lines().filter(|l| !l.is_empty());
let _first: Value = serde_json::from_str(lines.next().unwrap()).unwrap();
let second: Value = serde_json::from_str(lines.next().unwrap()).unwrap();
assert_eq!(second["id"], 2);
let code = second["error"]["code"].as_i64().unwrap_or(0);
assert_ne!(code, -32602, "non-text blocks must be silently ignored");
}
#[tokio::test]
async fn session_prompt_missing_both_text_and_prompt_errors() {
let input = concat!(
r#"{"jsonrpc":"2.0","id":1,"method":"session/new","params":{}}"#,
"\n",
r#"{"jsonrpc":"2.0","id":2,"method":"session/prompt","params":{"sessionId":"default"}}"#,
"\n",
);
let mut output = Vec::new();
run_acp_server_on(input.as_bytes(), &mut output)
.await
.unwrap();
let s = String::from_utf8(output).unwrap();
let mut lines = s.lines().filter(|l| !l.is_empty());
let _first: Value = serde_json::from_str(lines.next().unwrap()).unwrap();
let second: Value = serde_json::from_str(lines.next().unwrap()).unwrap();
assert_eq!(second["id"], 2);
assert_eq!(second["error"]["code"], -32602);
}
}
+59
View File
@@ -0,0 +1,59 @@
use serde::{Deserialize, Serialize};
use serde_json::Value;
pub const METHOD_NOT_FOUND: i32 = -32601;
pub const PARSE_ERROR: i32 = -32700;
#[derive(Debug, Deserialize)]
#[allow(dead_code)]
pub struct Request {
pub jsonrpc: String,
pub id: Option<Value>,
pub method: String,
pub params: Option<Value>,
}
#[derive(Debug, Serialize)]
pub struct Response {
pub jsonrpc: &'static str,
#[serde(skip_serializing_if = "Option::is_none")]
pub id: Option<Value>,
#[serde(flatten)]
pub body: ResponseBody,
}
#[derive(Debug, Serialize)]
#[serde(untagged)]
pub enum ResponseBody {
Ok { result: Value },
Err { error: RpcError },
}
#[derive(Debug, Serialize)]
pub struct RpcError {
pub code: i32,
pub message: String,
}
impl Response {
pub fn ok(id: Option<Value>, result: Value) -> Self {
Self {
jsonrpc: "2.0",
id,
body: ResponseBody::Ok { result },
}
}
pub fn err(id: Option<Value>, code: i32, message: impl Into<String>) -> Self {
Self {
jsonrpc: "2.0",
id,
body: ResponseBody::Err {
error: RpcError {
code,
message: message.into(),
},
},
}
}
}
+11 -4
View File
@@ -1,6 +1,6 @@
use crate::client::{ModelType, list_models}; use crate::client::{ModelType, list_models};
use crate::config::paths; use crate::config::paths;
use crate::config::{AppConfig, Config, list_agents, list_sessions}; use crate::config::{AppConfig, Config, list_agents_with_descriptions, list_sessions};
use crate::utils::list_file_names; use crate::utils::list_file_names;
use crate::vault::Vault; use crate::vault::Vault;
use clap_complete::{CompletionCandidate, Shell, generate}; use clap_complete::{CompletionCandidate, Shell, generate};
@@ -73,10 +73,17 @@ pub(super) fn role_completer(current: &OsStr) -> Vec<CompletionCandidate> {
pub(super) fn agent_completer(current: &OsStr) -> Vec<CompletionCandidate> { pub(super) fn agent_completer(current: &OsStr) -> Vec<CompletionCandidate> {
let cur = current.to_string_lossy(); let cur = current.to_string_lossy();
list_agents() list_agents_with_descriptions()
.into_iter() .into_iter()
.filter(|a| a.starts_with(&*cur)) .filter(|(a, _)| a.starts_with(&*cur))
.map(CompletionCandidate::new) .map(|(name, desc)| {
let help = if desc.is_empty() {
None
} else {
Some(desc.into())
};
CompletionCandidate::new(name).help(help)
})
.collect() .collect()
} }
+175 -27
View File
@@ -13,6 +13,20 @@ use is_terminal::IsTerminal;
use std::collections::HashSet; use std::collections::HashSet;
use std::io::{Read, stdin}; use std::io::{Read, stdin};
#[derive(clap::ValueEnum, Debug, Clone, Copy, PartialEq, Eq)]
pub enum McpTransportArg {
Stdio,
Http,
Sse,
}
#[derive(clap::ValueEnum, Debug, Clone, Copy, PartialEq, Eq, Default)]
pub enum McpScopeArg {
#[default]
User,
Workspace,
}
#[derive(Parser, Debug)] #[derive(Parser, Debug)]
#[command(author, version, about, long_about = None)] #[command(author, version, about, long_about = None)]
#[command( #[command(
@@ -30,7 +44,7 @@ use std::io::{Read, stdin};
", ",
group( group(
ArgGroup::new("sbx-mode") ArgGroup::new("sbx-mode")
.args(["sandbox", "fresh", "no_mixins"]) .args(["sandbox", "fresh"])
.multiple(true) .multiple(true)
.conflicts_with_all([ .conflicts_with_all([
"model", "prompt", "role", "session", "agent", "rag", "rebuild_rag", "model", "prompt", "role", "session", "agent", "rag", "rebuild_rag",
@@ -41,10 +55,15 @@ use std::io::{Read, stdin};
"list_skills", "skill", "tail_logs", "completions", "update", "list_skills", "skill", "tail_logs", "completions", "update",
]) ])
), ),
group(
ArgGroup::new("mcp-action")
.args(["mcp_add", "mcp_remove", "mcp_list", "mcp_get"])
.multiple(false)
),
)] )]
pub struct Cli { pub struct Cli {
/// Input text /// Input text
#[arg(trailing_var_arg = true)] #[arg(allow_hyphen_values = true)]
text: Vec<String>, text: Vec<String>,
/// Select a LLM model /// Select a LLM model
@@ -224,16 +243,71 @@ pub struct Cli {
#[arg(long, exclusive = true, value_name = "SERVER_NAME", help_heading = "Authentication", add = ArgValueCompleter::new(mcp_server_completer))] #[arg(long, exclusive = true, value_name = "SERVER_NAME", help_heading = "Authentication", add = ArgValueCompleter::new(mcp_server_completer))]
pub auth_mcp: Option<String>, pub auth_mcp: Option<String>,
/// Add an MCP server. Use `-- <cmd> [args...]` for stdio, or `--url <URL>` for http/sse.
#[arg(long, value_name = "NAME", help_heading = "MCP Servers")]
pub mcp_add: Option<String>,
/// Remove an MCP server by name
#[arg(long, value_name = "NAME", help_heading = "MCP Servers", add = ArgValueCompleter::new(mcp_server_completer))]
pub mcp_remove: Option<String>,
/// List all configured MCP servers (user + workspace scopes)
#[arg(long, help_heading = "MCP Servers")]
pub mcp_list: bool,
/// Show the JSON config for one MCP server
#[arg(long, value_name = "NAME", help_heading = "MCP Servers", add = ArgValueCompleter::new(mcp_server_completer))]
pub mcp_get: Option<String>,
/// Transport for --mcp-add: stdio (default when `--` present), http, or sse
#[arg(
long,
value_enum,
value_name = "TRANSPORT",
help_heading = "MCP Servers"
)]
pub transport: Option<McpTransportArg>,
/// URL for http/sse MCP server (used with --mcp-add)
#[arg(long, value_name = "URL", help_heading = "MCP Servers")]
pub url: Option<String>,
/// Scope for MCP config: user (~/.config/coyote/functions/mcp.json) or workspace (./.coyote/mcp.json). Default: user
#[arg(long, value_enum, value_name = "SCOPE", help_heading = "MCP Servers")]
pub scope: Option<McpScopeArg>,
/// Environment variable for stdio MCP server (repeatable): --env KEY=VALUE
#[arg(long, value_name = "KEY=VALUE", help_heading = "MCP Servers")]
pub env: Vec<String>,
/// HTTP header for http/sse MCP server (repeatable): --header "Name: Value"
#[arg(long, value_name = "HEADER", help_heading = "MCP Servers")]
pub header: Vec<String>,
/// Working directory for stdio MCP server
#[arg(long, value_name = "PATH", value_hint = ValueHint::AnyPath, help_heading = "MCP Servers")]
pub cwd: Option<String>,
/// OAuth client ID for http/sse MCP server
#[arg(long, value_name = "ID", help_heading = "MCP Servers")]
pub client_id: Option<String>,
/// OAuth client secret for http/sse MCP server (use {{NAME}} to reference a vault secret)
#[arg(long, value_name = "SECRET", help_heading = "MCP Servers")]
pub client_secret: Option<String>,
/// OAuth callback port for http/sse MCP server
#[arg(long, value_name = "PORT", help_heading = "MCP Servers")]
pub callback_port: Option<u16>,
/// OAuth redirect host for http/sse MCP server
#[arg(long, value_name = "HOST", help_heading = "MCP Servers")]
pub redirect_host: Option<String>,
/// Overwrite an existing MCP server (with --mcp-add) or skip confirmation (with --mcp-remove)
#[arg(long, help_heading = "MCP Servers")]
pub mcp_force: bool,
/// Launch Coyote inside a Docker sandbox (via `sbx`); name defaults to current directory basename /// Launch Coyote inside a Docker sandbox (via `sbx`); name defaults to current directory basename
#[arg(long, value_name = "NAME", help_heading = "Sandbox")] #[arg(long, value_name = "NAME", help_heading = "Sandbox")]
pub sandbox: Option<Option<String>>, pub sandbox: Option<Option<String>>,
/// Create the sandbox without bootstrapping the host config or vault password file /// Start the sandbox with a clean slate. No copied config or tokens; LLM credentials injected via sbx proxy
#[arg(long, requires = "sandbox", help_heading = "Sandbox")] #[arg(long, requires = "sandbox", help_heading = "Sandbox")]
pub fresh: bool, pub fresh: bool,
/// Skip discovery and application of all sbx mixins (user and built-in) /// Declare that no human is present. All user-interaction tools return structured JSON instead of
#[arg(long, requires = "sandbox", help_heading = "Sandbox")] /// prompting. Implies --dangerously-skip-permissions. Incompatible with REPL mode (requires a prompt).
pub no_mixins: bool, #[arg(long, help_heading = "Sandbox")]
pub headless: bool,
/// Run as an ACP agent server over stdio (JSON-RPC 2.0). Every stdout byte must be valid JSON-RPC.
/// Implies --headless. Single session per process.
#[arg(long, help_heading = "Sandbox")]
pub acp_server: bool,
/// Display information /// Display information
#[arg(long, help_heading = "Diagnostics & Tools")] #[arg(long, help_heading = "Diagnostics & Tools")]
pub info: bool, pub info: bool,
@@ -250,6 +324,15 @@ pub struct Cli {
/// Generate static shell completion scripts /// Generate static shell completion scripts
#[arg(long, value_name = "SHELL", value_enum, help_heading = "Shell")] #[arg(long, value_name = "SHELL", value_enum, help_heading = "Shell")]
pub completions: Option<ShellCompletion>, pub completions: Option<ShellCompletion>,
/// Stdio command for --mcp-add: everything after `--` is passed to the server verbatim
#[arg(
last = true,
allow_hyphen_values = true,
value_name = "CMD",
help_heading = "MCP Servers"
)]
pub mcp_command: Vec<String>,
} }
impl Cli { impl Cli {
@@ -494,6 +577,28 @@ mod tests {
assert!(!cli.dangerously_skip_permissions); assert!(!cli.dangerously_skip_permissions);
} }
#[test]
fn parse_headless_flag() {
let cli = parse(&["--headless", "do something"]);
assert!(cli.headless);
}
#[test]
fn parse_headless_default_off() {
assert!(!parse(&[]).headless);
}
#[test]
fn parse_acp_server_flag() {
let cli = parse(&["--acp-server"]);
assert!(cli.acp_server);
}
#[test]
fn parse_acp_server_default_off() {
assert!(!parse(&[]).acp_server);
}
#[test] #[test]
fn parse_sync_models_flag() { fn parse_sync_models_flag() {
let cli = parse(&["--sync-models"]); let cli = parse(&["--sync-models"]);
@@ -609,40 +714,83 @@ mod tests {
} }
#[test] #[test]
fn parse_fresh_flag_requires_sandbox() { fn parse_mcp_add_stdio_with_trailing_command() {
assert!(Cli::try_parse_from(["coyote", "--fresh"]).is_err()); let cli = parse(&[
"--mcp-add",
"myserver",
"--",
"npx",
"some-server",
"--flag",
"arg1",
]);
assert_eq!(cli.mcp_add, Some("myserver".to_string()));
assert_eq!(
cli.mcp_command,
vec!["npx", "some-server", "--flag", "arg1"]
);
assert!(cli.text.is_empty());
} }
#[test] #[test]
fn parse_fresh_flag_with_sandbox() { fn parse_mcp_add_stdio_with_env_and_command() {
let cli = parse(&["--sandbox", "--fresh"]); let cli = parse(&[
assert_eq!(cli.sandbox, Some(None)); "--mcp-add",
assert!(cli.fresh); "s",
"--env",
"API_KEY={{API_KEY}}",
"--env",
"MODE=dev",
"--",
"npx",
"srv",
]);
assert_eq!(cli.mcp_add, Some("s".to_string()));
assert_eq!(cli.env, vec!["API_KEY={{API_KEY}}", "MODE=dev"]);
assert_eq!(cli.mcp_command, vec!["npx", "srv"]);
} }
#[test] #[test]
fn parse_fresh_flag_with_named_sandbox() { fn parse_mcp_add_http_with_header() {
let cli = parse(&["--sandbox", "foo", "--fresh"]); let cli = parse(&[
assert_eq!(cli.sandbox, Some(Some("foo".to_string()))); "--mcp-add",
assert!(cli.fresh); "notion",
"--transport",
"http",
"--url",
"https://mcp.notion.com/mcp",
"--header",
"Authorization: Bearer {{NOTION_TOKEN}}",
]);
assert_eq!(cli.mcp_add, Some("notion".to_string()));
assert!(matches!(cli.transport, Some(McpTransportArg::Http)));
assert_eq!(cli.url, Some("https://mcp.notion.com/mcp".to_string()));
assert_eq!(cli.header, vec!["Authorization: Bearer {{NOTION_TOKEN}}"]);
assert!(cli.mcp_command.is_empty());
} }
#[test] #[test]
fn parse_no_mixins_requires_sandbox() { fn parse_mcp_list_flag() {
assert!(Cli::try_parse_from(["coyote", "--no-mixins"]).is_err()); let cli = parse(&["--mcp-list"]);
assert!(cli.mcp_list);
} }
#[test] #[test]
fn parse_no_mixins_with_sandbox() { fn parse_mcp_scope_workspace() {
let cli = parse(&["--sandbox", "--no-mixins"]); let cli = parse(&["--mcp-list", "--scope", "workspace"]);
assert!(cli.no_mixins); assert!(cli.mcp_list);
assert!(matches!(cli.scope, Some(McpScopeArg::Workspace)));
} }
#[test] #[test]
fn parse_sandbox_with_fresh_and_no_mixins() { fn parse_mcp_action_group_is_exclusive() {
let cli = parse(&["--sandbox", "foo", "--fresh", "--no-mixins"]); assert!(Cli::try_parse_from(["coyote", "--mcp-list", "--mcp-get", "foo"]).is_err());
assert_eq!(cli.sandbox, Some(Some("foo".to_string()))); }
assert!(cli.fresh);
assert!(cli.no_mixins); #[test]
fn parse_trailing_text_unchanged_without_dash_dash() {
let cli = parse(&["hello", "world"]);
assert_eq!(cli.text, vec!["hello", "world"]);
assert!(cli.mcp_command.is_empty());
} }
} }
+10 -3
View File
@@ -1,3 +1,5 @@
use std::mem;
use super::*; use super::*;
use crate::utils::{base64_decode, encode_uri, hex_encode, hmac_sha256, sha256, strip_think_tag}; use crate::utils::{base64_decode, encode_uri, hex_encode, hmac_sha256, sha256, strip_think_tag};
@@ -275,10 +277,11 @@ async fn chat_completions_streaming(
format!("Tool call '{function_name}' has non-JSON arguments '{function_arguments}'") format!("Tool call '{function_name}' has non-JSON arguments '{function_arguments}'")
})?; })?;
handler.tool_call(ToolCall::new( handler.tool_call(ToolCall::new(
function_name.clone(), mem::take(&mut function_name),
arguments, arguments,
Some(function_id.clone()), Some(mem::take(&mut function_id)),
))?; ))?;
function_arguments.clear();
} }
} }
_ => {} _ => {}
@@ -529,7 +532,11 @@ fn extract_chat_completions(data: &Value) -> Result<ChatCompletionsOutput> {
bail!("Invalid response data: {data}"); bail!("Invalid response data: {data}");
} }
let output = ChatCompletionsOutput { text, tool_calls, ..Default::default() }; let output = ChatCompletionsOutput {
text,
tool_calls,
..Default::default()
};
Ok(output) Ok(output)
} }
+210 -57
View File
@@ -1,3 +1,6 @@
use std::collections::HashSet;
use std::mem;
use super::access_token::get_access_token; use super::access_token::get_access_token;
use super::claude_oauth::ClaudeOAuthProvider; use super::claude_oauth::ClaudeOAuthProvider;
use super::oauth::{self, OAuthProvider}; use super::oauth::{self, OAuthProvider};
@@ -212,12 +215,8 @@ pub async fn claude_chat_completions_streaming(
if let Some(text) = data["delta"]["text"].as_str() { if let Some(text) = data["delta"]["text"].as_str() {
handler.text(text)?; handler.text(text)?;
} else if let Some(text) = data["delta"]["thinking"].as_str() { } else if let Some(text) = data["delta"]["thinking"].as_str() {
if reasoning_state == 0 { reasoning_state = 1;
handler.text("<think>\n")?;
reasoning_state = 1;
}
thinking_text.push_str(text); thinking_text.push_str(text);
handler.text(text)?;
} else if let Some(signature) = data["delta"]["signature"].as_str() { } else if let Some(signature) = data["delta"]["signature"].as_str() {
thinking_signature.push_str(signature); thinking_signature.push_str(signature);
} else if let (true, Some(partial_json)) = ( } else if let (true, Some(partial_json)) = (
@@ -229,11 +228,10 @@ pub async fn claude_chat_completions_streaming(
} }
"content_block_stop" => { "content_block_stop" => {
if reasoning_state == 1 { if reasoning_state == 1 {
handler.text("\n</think>\n\n")?;
reasoning_state = 0; reasoning_state = 0;
handler.thinking_block(ThinkingBlock::Thinking { handler.thinking_block(ThinkingBlock::Thinking {
thinking: std::mem::take(&mut thinking_text), thinking: mem::take(&mut thinking_text),
signature: std::mem::take(&mut thinking_signature), signature: mem::take(&mut thinking_signature),
}); });
} }
if !function_name.is_empty() { if !function_name.is_empty() {
@@ -245,10 +243,11 @@ pub async fn claude_chat_completions_streaming(
})? })?
}; };
handler.tool_call(ToolCall::new( handler.tool_call(ToolCall::new(
function_name.clone(), mem::take(&mut function_name),
arguments, arguments,
Some(function_id.clone()), Some(mem::take(&mut function_id)),
))?; ))?;
function_arguments.clear();
} }
} }
_ => {} _ => {}
@@ -326,51 +325,114 @@ pub fn claude_build_chat_completions_body(
})] })]
} }
MessageContent::ToolCalls(MessageContentToolCalls { MessageContent::ToolCalls(MessageContentToolCalls {
tool_results, text, .. tool_results,
text,
sequence,
}) => { }) => {
let mut assistant_parts = vec![]; if !sequence {
let mut user_parts = vec![]; let mut assistant_parts = vec![];
for (index, tool_result) in tool_results.iter().enumerate() { let mut user_parts = vec![];
for block in &tool_result.thinking { for (index, tool_result) in tool_results.iter().enumerate() {
assistant_parts.push(json!(block)); for block in &tool_result.thinking {
} assistant_parts.push(json!(block));
let round_text = if index == 0 && !text.is_empty() {
Some(text.as_str())
} else {
tool_result.text.as_deref()
};
if let Some(round_text) = round_text {
let round_text = strip_think_tag(round_text);
let round_text = round_text.trim();
if !round_text.is_empty() {
assistant_parts.push(json!({
"type": "text",
"text": round_text,
}))
} }
let round_text = if index == 0 && !text.is_empty() {
Some(text.as_str())
} else {
tool_result.text.as_deref()
};
if let Some(round_text) = round_text {
let round_text = strip_think_tag(round_text);
let round_text = round_text.trim();
if !round_text.is_empty() {
assistant_parts.push(json!({
"type": "text",
"text": round_text,
}))
}
}
assistant_parts.push(json!({
"type": "tool_use",
"id": tool_result.call.id,
"name": tool_result.call.name,
"input": tool_result.call.arguments,
}));
user_parts.push(json!({
"type": "tool_result",
"tool_use_id": tool_result.call.id,
"content": tool_result.output.to_string(),
}));
} }
assistant_parts.push(json!({ vec![
"type": "tool_use", json!({ "role": "assistant", "content": assistant_parts }),
"id": tool_result.call.id, json!({ "role": "user", "content": user_parts }),
"name": tool_result.call.name, ]
"input": tool_result.call.arguments, } else {
})); // One pair per round: Claude can reuse tool_use IDs across API calls.
user_parts.push(json!({ // A round boundary is detected by the presence of round text, but
"type": "tool_result", // rounds where the model emitted only tool calls (no narration)
"tool_use_id": tool_result.call.id, // carry no text marker. As a backstop, also split whenever a
"content": tool_result.output.to_string(), // tool_use ID would repeat within the current assistant message —
})); // the API rejects duplicate tool_use IDs in a single message.
let mut messages = vec![];
let mut assistant_parts: Vec<serde_json::Value> = vec![];
let mut user_parts: Vec<serde_json::Value> = vec![];
let mut chunk_ids: HashSet<&str> = HashSet::new();
for (index, tool_result) in tool_results.iter().enumerate() {
let id_collision = tool_result
.call
.id
.as_deref()
.is_some_and(|id| chunk_ids.contains(id));
if index > 0 && (tool_result.text.is_some() || id_collision) {
messages.push(
json!({ "role": "assistant", "content": assistant_parts }),
);
messages.push(json!({ "role": "user", "content": user_parts }));
assistant_parts = vec![];
user_parts = vec![];
chunk_ids.clear();
}
if let Some(id) = tool_result.call.id.as_deref() {
chunk_ids.insert(id);
}
for block in &tool_result.thinking {
assistant_parts.push(json!(block));
}
let round_text = if index == 0 && !text.is_empty() {
Some(text.as_str())
} else {
tool_result.text.as_deref()
};
if let Some(round_text) = round_text {
let round_text = strip_think_tag(round_text);
let round_text = round_text.trim();
if !round_text.is_empty() {
assistant_parts.push(json!({
"type": "text",
"text": round_text,
}))
}
}
assistant_parts.push(json!({
"type": "tool_use",
"id": tool_result.call.id,
"name": tool_result.call.name,
"input": tool_result.call.arguments,
}));
user_parts.push(json!({
"type": "tool_result",
"tool_use_id": tool_result.call.id,
"content": tool_result.output.to_string(),
}));
}
if !assistant_parts.is_empty() {
messages
.push(json!({ "role": "assistant", "content": assistant_parts }));
messages.push(json!({ "role": "user", "content": user_parts }));
}
messages
} }
vec![
json!({
"role": "assistant",
"content": assistant_parts,
}),
json!({
"role": "user",
"content": user_parts,
}),
]
} }
} }
}) })
@@ -430,7 +492,6 @@ pub fn claude_build_chat_completions_body(
pub fn claude_extract_chat_completions(data: &Value) -> Result<ChatCompletionsOutput> { pub fn claude_extract_chat_completions(data: &Value) -> Result<ChatCompletionsOutput> {
let mut text = String::new(); let mut text = String::new();
let mut reasoning = None;
let mut tool_calls = vec![]; let mut tool_calls = vec![];
let mut thinking = vec![]; let mut thinking = vec![];
if let Some(list) = data["content"].as_array() { if let Some(list) = data["content"].as_array() {
@@ -438,7 +499,6 @@ pub fn claude_extract_chat_completions(data: &Value) -> Result<ChatCompletionsOu
match item["type"].as_str() { match item["type"].as_str() {
Some("thinking") => { Some("thinking") => {
if let Some(v) = item["thinking"].as_str() { if let Some(v) = item["thinking"].as_str() {
reasoning = Some(v.to_string());
thinking.push(ThinkingBlock::Thinking { thinking.push(ThinkingBlock::Thinking {
thinking: v.to_string(), thinking: v.to_string(),
signature: item["signature"].as_str().unwrap_or_default().to_string(), signature: item["signature"].as_str().unwrap_or_default().to_string(),
@@ -477,10 +537,6 @@ pub fn claude_extract_chat_completions(data: &Value) -> Result<ChatCompletionsOu
} }
} }
} }
if let Some(reasoning) = reasoning {
text = format!("<think>\n{reasoning}\n</think>\n\n{text}")
}
if text.is_empty() && tool_calls.is_empty() { if text.is_empty() && tool_calls.is_empty() {
bail!("Invalid response data: {data}"); bail!("Invalid response data: {data}");
} }
@@ -492,3 +548,100 @@ pub fn claude_extract_chat_completions(data: &Value) -> Result<ChatCompletionsOu
}; };
Ok(output) Ok(output)
} }
#[cfg(test)]
mod tests {
use super::*;
use crate::function::{ToolCall, ToolResult};
fn tool_result(id: &str, text: Option<&str>) -> ToolResult {
ToolResult {
call: ToolCall::new("fs_read".into(), json!({"path": "x"}), Some(id.into())),
output: json!("ok"),
text: text.map(|t| t.to_string()),
thinking: vec![],
}
}
fn build_body(tool_results: Vec<ToolResult>) -> Value {
let data = ChatCompletionsData {
messages: vec![
Message::new(MessageRole::User, MessageContent::Text("hello".to_string())),
Message::new(
MessageRole::Assistant,
MessageContent::ToolCalls(MessageContentToolCalls {
tool_results,
text: String::new(),
sequence: true,
}),
),
],
temperature: None,
top_p: None,
reasoning_effort: None,
functions: None,
stream: false,
};
claude_build_chat_completions_body(data, &Model::new("claude", "claude-test")).unwrap()
}
fn assert_unique_tool_use_ids_per_message(body: &Value) {
for message in body["messages"].as_array().unwrap() {
let Some(content) = message["content"].as_array() else {
continue;
};
let mut seen = HashSet::new();
for block in content {
if block["type"] == "tool_use" {
let id = block["id"].as_str().unwrap();
assert!(
seen.insert(id.to_string()),
"duplicate tool_use id `{id}` within a single assistant message: {message}"
);
}
}
}
}
#[test]
fn sequence_splits_on_round_text() {
let body = build_body(vec![
tool_result("toolu_A", None),
tool_result("toolu_B", None),
tool_result("toolu_C", Some("running another tool")),
]);
let messages = body["messages"].as_array().unwrap();
assert_eq!(messages.len(), 5, "body: {body}");
assert_unique_tool_use_ids_per_message(&body);
}
#[test]
fn sequence_splits_on_reused_id_in_textless_round() {
let body = build_body(vec![
tool_result("toolu_A", None),
tool_result("toolu_B", None),
tool_result("toolu_A", None),
]);
let messages = body["messages"].as_array().unwrap();
assert_eq!(messages.len(), 5, "body: {body}");
assert_unique_tool_use_ids_per_message(&body);
}
#[test]
fn sequence_keeps_textless_rounds_merged_when_ids_are_unique() {
let body = build_body(vec![
tool_result("toolu_A", None),
tool_result("toolu_B", None),
tool_result("toolu_C", None),
]);
let messages = body["messages"].as_array().unwrap();
assert_eq!(messages.len(), 3, "body: {body}");
assert_unique_tool_use_ids_per_message(&body);
}
}
+29 -3
View File
@@ -493,16 +493,42 @@ pub async fn call_chat_completions_streaming(
render_stream(rx, client.app_config(), abort_signal.clone(), silent), render_stream(rx, client.app_config(), abort_signal.clone(), silent),
); );
if handler.abort().aborted() { let aborted_ctrlc = handler.abort().aborted_ctrlc();
let aborted_ctrld = handler.abort().aborted_ctrld();
if aborted_ctrld {
bail!("Aborted."); bail!("Aborted.");
} }
render_ret?; render_ret?;
let (text, tool_calls, thinking) = handler.take(); let (text, tool_calls, thinking) = handler.take();
if aborted_ctrlc {
if !ctx.working_mode.is_repl() || ctx.session.is_none() {
bail!("Aborted.");
}
if text.is_empty() {
if !silent && *IS_STDOUT_TERMINAL {
println!();
eprintln!("{}", error_text("Response interrupted"));
}
return Ok(("".to_string(), vec![]));
}
if !silent && *IS_STDOUT_TERMINAL {
println!();
eprintln!("{}", error_text("Response interrupted"));
}
return Ok((text, vec![]));
}
match send_ret { match send_ret {
Ok(_) => { Ok(_) => {
if !text.is_empty() && !text.ends_with('\n') { if !silent && !text.is_empty() && !text.ends_with('\n') {
println!(); println!();
} }
let mut tool_results = eval_tool_calls(ctx, tool_calls).await?; let mut tool_results = eval_tool_calls(ctx, tool_calls).await?;
@@ -515,7 +541,7 @@ pub async fn call_chat_completions_streaming(
Ok((text, tool_results)) Ok((text, tool_results))
} }
Err(err) => { Err(err) => {
if !text.is_empty() { if !silent && !text.is_empty() {
println!(); println!();
} }
Err(err) Err(err)
+9
View File
@@ -38,6 +38,15 @@ register_client!(
(bedrock, "bedrock", BedrockConfig, BedrockClient), (bedrock, "bedrock", BedrockConfig, BedrockClient),
); );
pub fn client_type_supports_oauth(type_str: &str) -> bool {
matches!(
type_str,
ClaudeClient::NAME | OpenAIClient::NAME | GeminiClient::NAME
) || ALL_PROVIDER_MODELS
.iter()
.any(|pm| pm.provider == type_str && pm.oauth.is_some())
}
pub const OPENAI_COMPATIBLE_PROVIDERS: [(&str, &str); 18] = [ pub const OPENAI_COMPATIBLE_PROVIDERS: [(&str, &str); 18] = [
("ai21", "https://api.ai21.com/studio/v1"), ("ai21", "https://api.ai21.com/studio/v1"),
( (
+89 -2
View File
@@ -131,6 +131,17 @@ pub trait OAuthProvider: Send + Sync {
vec![] vec![]
} }
/// Extra form/body parameters appended to every token request routed
/// through `build_token_request` (authorization-code exchange, refresh,
/// client_credentials, and device-code polling). Used e.g. for the
/// RFC 8707 `resource` indicator required by the MCP spec.
/// NOTE: these are merged AFTER the caller's params and will overwrite
/// a colliding key; do not return protocol parameter names
/// (grant_type, client_id, code, refresh_token, ...).
fn extra_token_params(&self) -> Vec<(&str, &str)> {
vec![]
}
fn token_request_format(&self) -> TokenRequestFormat { fn token_request_format(&self) -> TokenRequestFormat {
TokenRequestFormat::Json TokenRequestFormat::Json
} }
@@ -642,9 +653,14 @@ fn build_token_request(
provider: &(impl OAuthProvider + ?Sized), provider: &(impl OAuthProvider + ?Sized),
params: &[(&str, &str)], params: &[(&str, &str)],
) -> RequestBuilder { ) -> RequestBuilder {
let all_params: Vec<(&str, &str)> = params
.iter()
.copied()
.chain(provider.extra_token_params())
.collect();
let mut request = match provider.token_request_format() { let mut request = match provider.token_request_format() {
TokenRequestFormat::Json => { TokenRequestFormat::Json => {
let body: serde_json::Map<String, Value> = params let body: serde_json::Map<String, Value> = all_params
.iter() .iter()
.map(|(k, v)| (k.to_string(), Value::String(v.to_string()))) .map(|(k, v)| (k.to_string(), Value::String(v.to_string())))
.collect(); .collect();
@@ -660,7 +676,7 @@ fn build_token_request(
} }
} }
TokenRequestFormat::FormUrlEncoded => { TokenRequestFormat::FormUrlEncoded => {
let mut form: HashMap<String, String> = params let mut form: HashMap<String, String> = all_params
.iter() .iter()
.map(|(k, v)| (k.to_string(), v.to_string())) .map(|(k, v)| (k.to_string(), v.to_string()))
.collect(); .collect();
@@ -870,6 +886,8 @@ pub(crate) fn client_config_info(
#[cfg(test)] #[cfg(test)]
mod tests { mod tests {
use std::str;
use super::*; use super::*;
use crate::client::openai_compatible::OpenAICompatibleConfig; use crate::client::openai_compatible::OpenAICompatibleConfig;
use crate::client::{ModelData, ProviderModels}; use crate::client::{ModelData, ProviderModels};
@@ -1161,6 +1179,16 @@ echo_pkce_in_token_exchange: true
assert!(provider.fixed_redirect_uri().is_none()); assert!(provider.fixed_redirect_uri().is_none());
} }
#[test]
fn default_extra_token_params_is_empty() {
let provider = OpenAICompatibleOAuthProvider {
config: base_config(),
client_name: "test".into(),
};
assert!(provider.extra_token_params().is_empty());
}
#[test] #[test]
fn oauth_flow_device_code_parses() { fn oauth_flow_device_code_parses() {
let yaml = "client_id: x\ntoken_url: y\nflow: device_code"; let yaml = "client_id: x\ntoken_url: y\nflow: device_code";
@@ -1381,4 +1409,63 @@ scopes:
assert!(cfg.use_pkce_in_device_flow); assert!(cfg.use_pkce_in_device_flow);
assert_eq!(cfg.scopes, vec!["read", "write"]); assert_eq!(cfg.scopes, vec!["read", "write"]);
} }
struct ResourceStubProvider;
impl OAuthProvider for ResourceStubProvider {
fn provider_name(&self) -> &str {
"stub"
}
fn client_id(&self) -> &str {
"stub-client"
}
fn authorize_url(&self) -> &str {
"https://as.example/authorize"
}
fn token_url(&self) -> &str {
"https://as.example/token"
}
fn redirect_uri(&self) -> &str {
""
}
fn scopes(&self) -> String {
String::new()
}
fn token_request_format(&self) -> TokenRequestFormat {
TokenRequestFormat::FormUrlEncoded
}
fn extra_token_params(&self) -> Vec<(&str, &str)> {
vec![("resource", "https://rs.example/mcp")]
}
}
#[test]
fn build_token_request_appends_extra_token_params_to_form_body() {
let provider = ResourceStubProvider;
let request = build_token_request(
&ReqwestClient::new(),
&provider,
&[("grant_type", "authorization_code")],
)
.build()
.unwrap();
let body = str::from_utf8(request.body().unwrap().as_bytes().unwrap()).unwrap();
assert!(
body.contains("resource=https%3A%2F%2Frs.example%2Fmcp"),
"body missing resource param: {body}"
);
assert!(
body.contains("grant_type=authorization_code"),
"body missing grant_type param: {body}"
);
}
} }
+119 -14
View File
@@ -4,6 +4,7 @@ use crate::{
client::Model, client::Model,
config::memory, config::memory,
function::{Functions, run_llm_function}, function::{Functions, run_llm_function},
graph, rag,
}; };
use super::rag_cache::RagKey; use super::rag_cache::RagKey;
@@ -12,6 +13,7 @@ use crate::config::prompts::{
DEFAULT_SPAWN_INSTRUCTIONS, DEFAULT_TEAMMATE_INSTRUCTIONS, DEFAULT_TODO_INSTRUCTIONS, DEFAULT_SPAWN_INSTRUCTIONS, DEFAULT_TEAMMATE_INSTRUCTIONS, DEFAULT_TODO_INSTRUCTIONS,
DEFAULT_USER_INTERACTION_INSTRUCTIONS, DEFAULT_USER_INTERACTION_INSTRUCTIONS,
}; };
use crate::graph::types::RagNode;
use crate::graph::{Graph, GraphParser, NodeType}; use crate::graph::{Graph, GraphParser, NodeType};
use crate::rag::RagInitConfig; use crate::rag::RagInitConfig;
use crate::vault::SECRET_RE; use crate::vault::SECRET_RE;
@@ -146,11 +148,18 @@ impl Agent {
let rag = if rag_path.exists() { let rag = if rag_path.exists() {
let key = RagKey::Agent(name.to_string()); let key = RagKey::Agent(name.to_string());
let app_clone = app.clone(); let app_clone = app.clone();
let vault_clone = app_state.vault.clone();
let rag_path_clone = rag_path.clone(); let rag_path_clone = rag_path.clone();
let rag = app_state let rag = app_state
.rag_cache .rag_cache
.load_with(key, || async move { .load_with(key, || async move {
Rag::load(&app_clone, DEFAULT_AGENT_NAME, &rag_path_clone) Rag::load_async(
&app_clone,
&vault_clone,
DEFAULT_AGENT_NAME,
&rag_path_clone,
)
.await
}) })
.await?; .await?;
Some(rag) Some(rag)
@@ -171,7 +180,15 @@ impl Agent {
let rag = app_state let rag = app_state
.rag_cache .rag_cache
.load_with(key, || async move { .load_with(key, || async move {
Rag::init(&app_clone, "rag", &rag_path_clone, &document_paths, abort).await Rag::init(
&app_clone,
"rag",
&rag_path_clone,
&document_paths,
abort,
true,
)
.await
}) })
.await?; .await?;
Some(rag) Some(rag)
@@ -231,6 +248,10 @@ impl Agent {
} }
} }
if rag.is_some() && app.function_calling_support && graph_for_rag.is_none() {
functions.append_rag_query_functions();
}
agent_config.replace_tools_placeholder(&functions); agent_config.replace_tools_placeholder(&functions);
Ok(Self { Ok(Self {
@@ -573,6 +594,25 @@ impl Agent {
_ => bail!("No return value from '_instructions' function"), _ => bail!("No return value from '_instructions' function"),
} }
} }
#[cfg(test)]
pub fn test_new(config: AgentConfig) -> Self {
Self {
name: config.name.clone(),
config,
shared_variables: Default::default(),
session_variables: None,
shared_dynamic_instructions: None,
session_dynamic_instructions: None,
functions: Functions::default(),
rag: None,
graph_rags: Default::default(),
model: Model::default(),
vault: std::sync::Arc::new(Vault::default()),
is_graph: false,
enabled_tools: None,
}
}
} }
impl RoleLike for Agent { impl RoleLike for Agent {
@@ -918,6 +958,30 @@ fn resolve_document_paths(
Ok(document_paths) Ok(document_paths)
} }
/// How a graph rag node describes the knowledge base it wants built.
///
/// `driver` is forwarded as-is: `None` means the node did not ask for one, which
/// `RagInitConfig` resolves to yaml, so workflows written before drivers existed
/// keep their current storage.
///
/// Every field is now named explicitly, so adding one to `RagInitConfig` breaks
/// this literal. That is deliberate: the new field then gets a decision about
/// whether a rag node can drive it, instead of silently taking its default.
fn rag_init_config(rag_node: &RagNode) -> RagInitConfig {
RagInitConfig {
embedding_model: rag_node.embedding_model.clone(),
chunk_size: rag_node.chunk_size,
chunk_overlap: rag_node.chunk_overlap,
reranker_model: rag_node.reranker_model.clone(),
top_k: rag_node.top_k,
batch_size: rag_node.batch_size,
extractor_model: rag_node.extractor_model.clone(),
extractor_prompt: rag_node.extractor_prompt.clone(),
graph_hops: rag_node.graph_hops,
driver: rag_node.driver.clone(),
}
}
#[allow(clippy::too_many_arguments)] #[allow(clippy::too_many_arguments)]
async fn init_graph_rags( async fn init_graph_rags(
app: &AppConfig, app: &AppConfig,
@@ -945,26 +1009,28 @@ async fn init_graph_rags(
}; };
let rag = if rag_path.exists() { let rag = if rag_path.exists() {
let app_clone = app.clone(); let app_clone = app.clone();
let vault_clone = app_state.vault.clone();
let path_clone = rag_path.clone(); let path_clone = rag_path.clone();
let name_clone = node_id.clone(); let name_clone = node_id.clone();
app_state app_state
.rag_cache .rag_cache
.load_with(key, || async move { .load_with(key, || async move {
Rag::load(&app_clone, &name_clone, &path_clone) Rag::load_async(&app_clone, &vault_clone, &name_clone, &path_clone).await
}) })
.await? .await?
} else { } else {
let config = RagInitConfig { // Checked before anything is built: an unknown driver would otherwise
embedding_model: rag_node.embedding_model.clone(), // fall through `Rag::create`'s catch-all to a yaml store, embed every
chunk_size: rag_node.chunk_size, // document, and persist the bogus driver string. The RAG would then be
chunk_overlap: rag_node.chunk_overlap, // rejected on every subsequent load, leaving the agent unstartable.
reranker_model: rag_node.reranker_model.clone(), // Graph validation catches this too, but it is skipped when
top_k: rag_node.top_k, // `validate_before_run` is off, so this guard is the load-bearing one.
batch_size: rag_node.batch_size, if let Some(driver) = &rag_node.driver
extractor_model: rag_node.extractor_model.clone(), && let Some(message) = graph::validator::rag_driver_error(driver)
extractor_prompt: rag_node.extractor_prompt.clone(), {
graph_hops: rag_node.graph_hops, bail!("rag node '{node_id}': {message}");
}; }
let mut config = rag_init_config(rag_node);
let fully_specified = config.embedding_model.is_some() let fully_specified = config.embedding_model.is_some()
&& config.chunk_size.is_some() && config.chunk_size.is_some()
&& config.chunk_overlap.is_some(); && config.chunk_overlap.is_some();
@@ -990,6 +1056,10 @@ async fn init_graph_rags(
initialized. RAG initialization is required for this agent." initialized. RAG initialization is required for this agent."
); );
} }
if config.driver.is_none() {
config.driver = Some(rag::select_rag_driver()?);
}
} }
let document_paths = let document_paths =
@@ -1298,4 +1368,39 @@ version: "1.0"
assert_eq!(meta.description, ""); assert_eq!(meta.description, "");
} }
#[test]
fn rag_init_config_forwards_an_explicit_driver() {
let node: RagNode =
serde_yaml::from_str("documents: [\"./docs\"]\ndriver: duckdb\n").unwrap();
assert_eq!(rag_init_config(&node).driver.as_deref(), Some("duckdb"));
}
/// A node that names no driver must forward `None`, which `RagInitConfig`
/// documents as "yaml". Existing workflows therefore keep their yaml store.
#[test]
fn rag_init_config_leaves_the_driver_unset_by_default() {
let node: RagNode = serde_yaml::from_str("documents: [\"./docs\"]\n").unwrap();
assert_eq!(rag_init_config(&node).driver, None);
}
/// The driver must ride alongside the rest of the node's settings, not
/// replace them.
#[test]
fn rag_init_config_forwards_the_other_settings_too() {
let node: RagNode = serde_yaml::from_str(
"documents: [\"./docs\"]\ndriver: duckdb\nchunk_size: 512\nchunk_overlap: 64\ntop_k: 7\nembedding_model: some:model\n",
)
.unwrap();
let config = rag_init_config(&node);
assert_eq!(config.driver.as_deref(), Some("duckdb"));
assert_eq!(config.chunk_size, Some(512));
assert_eq!(config.chunk_overlap, Some(64));
assert_eq!(config.top_k, Some(7));
assert_eq!(config.embedding_model.as_deref(), Some("some:model"));
}
} }
+4
View File
@@ -321,6 +321,10 @@ impl Input {
} }
} }
pub fn with_session(&self) -> bool {
self.with_session
}
pub fn with_agent(&self) -> bool { pub fn with_agent(&self) -> bool {
self.with_agent self.with_agent
} }
+7
View File
@@ -947,6 +947,7 @@ fn print_secret_summary(added: &[String], deferred: &[String]) {
#[cfg(test)] #[cfg(test)]
mod tests { mod tests {
use super::*; use super::*;
use crate::sandbox::SANDBOX_ENV_FLAG;
use crate::utils::get_env_name; use crate::utils::get_env_name;
use serial_test::serial; use serial_test::serial;
use std::env; use std::env;
@@ -1431,6 +1432,12 @@ mod tests {
#[tokio::test(flavor = "multi_thread", worker_threads = 1)] #[tokio::test(flavor = "multi_thread", worker_threads = 1)]
#[serial] #[serial]
async fn merge_detects_missing_secrets_in_output() { async fn merge_detects_missing_secrets_in_output() {
if env::var_os(SANDBOX_ENV_FLAG).is_some() {
eprintln!(
"Skipping merge_detects_missing_secrets_in_output: secret interpolation is disabled inside a sandbox"
);
return;
}
let _guard = TestVaultConfigGuard::new("merge-secret"); let _guard = TestVaultConfigGuard::new("merge-secret");
let dir = fresh_temp_dir("merge-secret-"); let dir = fresh_temp_dir("merge-secret-");
let remote = dir.join("remote.json"); let remote = dir.join("remote.json");
+149 -7
View File
@@ -44,11 +44,12 @@ pub use self::skill_policy::SkillPolicy;
pub use self::skill_registry::SkillRegistry; pub use self::skill_registry::SkillRegistry;
pub use self::update::run_self_update; pub use self::update::run_self_update;
use crate::client::{ use crate::client::{
ClientConfig, MessageContentToolCalls, Model, ModelType, OPENAI_COMPATIBLE_PROVIDERS, self, ClientConfig, MessageContentToolCalls, Model, ModelType, OPENAI_COMPATIBLE_PROVIDERS,
ProviderModels, create_client_config, list_client_types, oauth, ProviderModels, create_client_config, list_client_types, oauth, set_client_models_config,
}; };
use crate::function::{FunctionDeclaration, Functions}; use crate::function::{FunctionDeclaration, Functions};
use crate::rag::Rag; use crate::rag::Rag;
use crate::sandbox::SANDBOX_ENV_FLAG;
use crate::utils::*; use crate::utils::*;
pub use macros::macro_execute; pub use macros::macro_execute;
@@ -61,7 +62,7 @@ use fancy_regex::Regex;
use gman::providers::SupportedProvider; use gman::providers::SupportedProvider;
use indexmap::IndexMap; use indexmap::IndexMap;
use indoc::formatdoc; use indoc::formatdoc;
use inquire::{Confirm, Select}; use inquire::{Confirm, Select, Text};
use serde::{Deserialize, Serialize}; use serde::{Deserialize, Serialize};
use serde_json::json; use serde_json::json;
use std::collections::{HashMap, HashSet}; use std::collections::{HashMap, HashSet};
@@ -149,7 +150,7 @@ const WORKSPACE_COYOTE_DIR_NAME: &str = ".coyote";
const SBX_KIT_DIR_NAME: &str = "sbx-kit"; const SBX_KIT_DIR_NAME: &str = "sbx-kit";
const SBX_KIT_HASH_FILE: &str = "kit.sha256"; const SBX_KIT_HASH_FILE: &str = "kit.sha256";
const SBX_MIXIN_FILE_NAME: &str = "sbx-mixin.yaml"; const SBX_MIXIN_FILE_NAME: &str = "sbx-mixin.yaml";
const SBX_VAULT_MIXINS_DIR_NAME: &str = "sbx-vault-mixins"; pub(crate) const VAULT_DATA_FILE_NAME: &str = "vault.yml";
const SBX_MIXIN_KITS_DIR_NAME: &str = "sbx-mixin-kits"; const SBX_MIXIN_KITS_DIR_NAME: &str = "sbx-mixin-kits";
const GIT_DIR_NAME: &str = ".git"; const GIT_DIR_NAME: &str = ".git";
const GITIGNORE_FILE_NAME: &str = ".gitignore"; const GITIGNORE_FILE_NAME: &str = ".gitignore";
@@ -521,6 +522,16 @@ pub async fn sync_models(url: &str, abort_signal: AbortSignal) -> Result<()> {
impl Config { impl Config {
pub async fn load_with_interpolation(info_flag: bool) -> Result<Self> { pub async fn load_with_interpolation(info_flag: bool) -> Result<Self> {
let config_path = paths::config_file(); let config_path = paths::config_file();
if env::var_os(SANDBOX_ENV_FLAG).is_some() {
if !config_path.exists() {
create_config_file(&config_path).await?;
}
let (config, _) = Self::load_from_file(&config_path)?;
return Ok(config);
}
let (mut config, content) = if !config_path.exists() { let (mut config, content) = if !config_path.exists() {
match env::var(get_env_name("provider")) match env::var(get_env_name("provider"))
.ok() .ok()
@@ -751,12 +762,14 @@ pub async fn create_config_file(config_path: &Path) -> Result<()> {
process::exit(0); process::exit(0);
} }
if env::var_os(SANDBOX_ENV_FLAG).is_some() {
return create_config_file_sandbox(config_path).await;
}
let provider_choice = prompt_provider_choice()?; let provider_choice = prompt_provider_choice()?;
let mut vault = match &provider_choice { let mut vault = match &provider_choice {
None => Vault::default_local(), None => Vault::default_local(),
Some(provider) => Vault { Some(provider) => Vault::from_provider(provider.clone()),
provider: provider.clone(),
},
}; };
create_vault_password_file(&mut vault)?; create_vault_password_file(&mut vault)?;
if provider_choice.is_some() { if provider_choice.is_some() {
@@ -812,6 +825,82 @@ pub async fn create_config_file(config_path: &Path) -> Result<()> {
Ok(()) Ok(())
} }
async fn create_config_file_sandbox(config_path: &Path) -> Result<()> {
let client = Select::new("API Provider (required):", list_client_types()).prompt()?;
println!(
"Running in sandbox mode — your API provider credentials are managed by your host Coyote configuration if configured."
);
let oai_api_base = client::OPENAI_COMPATIBLE_PROVIDERS
.iter()
.find(|(name, _)| *name == client)
.map(|(_, url)| *url);
let mut client_config = if let Some(api_base) = oai_api_base {
let api_base_str = if api_base.contains('{') {
Text::new("API Base:")
.with_placeholder(&format!("e.g. {api_base}"))
.prompt()?
} else {
api_base.to_string()
};
serde_json::json!({
"type": "openai-compatible",
"name": client,
"api_base": api_base_str,
})
} else {
serde_json::json!({ "type": client })
};
if client::client_type_supports_oauth(client) {
let use_oauth = Confirm::new("Use OAuth authentication instead?")
.with_default(false)
.prompt()?;
if use_oauth {
client_config["auth"] = "oauth".into();
}
}
let model = set_client_models_config(&mut client_config, client).await?;
let mut config = serde_json::json!({});
config["model"] = model.into();
config["stream"] = serde_json::json!(true);
config["save"] = serde_json::json!(true);
config["keybindings"] = serde_json::json!("vi");
config["wrap"] = serde_json::json!("auto");
config["wrap_code"] = serde_json::json!(false);
config["function_calling_support"] = serde_json::json!(true);
config["enabled_tools"] = serde_json::json!(null);
config["visible_tools"] = serde_json::json!(DEFAULT_VISIBLE_TOOLS);
config["mcp_server_support"] = serde_json::json!(true);
config["enabled_mcp_servers"] = serde_json::json!(null);
config["highlight"] = serde_json::json!(true);
config["light_theme"] = serde_json::json!(false);
config[CLIENTS_FIELD] = serde_json::json!(vec![client_config]);
let config_data = serde_yaml::to_string(&config).with_context(|| "Failed to create config")?;
let config_data = format!(
"# see https://github.com/Dark-Alex-17/coyote/blob/main/config.example.yaml\n\n{config_data}"
);
ensure_parent_exists(config_path)?;
std::fs::write(config_path, config_data)
.with_context(|| format!("Failed to write to '{}'", config_path.display()))?;
#[cfg(unix)]
{
use std::os::unix::prelude::PermissionsExt;
let perms = std::fs::Permissions::from_mode(0o600);
std::fs::set_permissions(config_path, perms)?;
}
println!("✓ Saved the config file to '{}'.\n", config_path.display());
Ok(())
}
pub(crate) fn ensure_parent_exists(path: &Path) -> Result<()> { pub(crate) fn ensure_parent_exists(path: &Path) -> Result<()> {
if path.exists() { if path.exists() {
return Ok(()); return Ok(());
@@ -1108,4 +1197,57 @@ clients:
assert!(!state.assert(StateFlags::SESSION)); assert!(!state.assert(StateFlags::SESSION));
assert!(!state.assert(StateFlags::empty())); assert!(!state.assert(StateFlags::empty()));
} }
#[tokio::test]
#[serial_test::serial]
async fn sandbox_config_load_no_interpolation() {
use std::fs;
use std::time;
let unique = time::SystemTime::now()
.duration_since(time::UNIX_EPOCH)
.unwrap()
.as_nanos();
let tmp_dir = std::env::temp_dir().join(format!("coyote-sandbox-cfg-{unique}"));
fs::create_dir_all(&tmp_dir).unwrap();
let config_path = tmp_dir.join("config.yaml");
fs::write(
&config_path,
"model: claude:claude-3-5-haiku\nclients:\n - type: claude\n api_key: '{{ANTHROPIC_API_KEY}}'\n",
)
.unwrap();
let config_env = get_env_name("config_file");
let prev_config = std::env::var_os(&config_env);
let prev_sandbox = std::env::var_os(crate::sandbox::SANDBOX_ENV_FLAG);
unsafe {
std::env::set_var(&config_env, &config_path);
std::env::set_var(crate::sandbox::SANDBOX_ENV_FLAG, "1");
}
let result = Config::load_with_interpolation(false).await;
let (_, raw) = Config::load_from_file(&config_path).unwrap();
unsafe {
match prev_config {
Some(v) => std::env::set_var(&config_env, v),
None => std::env::remove_var(&config_env),
}
match prev_sandbox {
Some(v) => std::env::set_var(crate::sandbox::SANDBOX_ENV_FLAG, v),
None => std::env::remove_var(crate::sandbox::SANDBOX_ENV_FLAG),
}
}
let _ = fs::remove_dir_all(&tmp_dir);
result.expect(
"load_with_interpolation should succeed in sandbox mode with placeholder values",
);
assert!(
raw.contains("{{ANTHROPIC_API_KEY}}"),
"placeholder should be preserved as a literal string in sandbox mode"
);
}
} }
+102 -10
View File
@@ -5,7 +5,7 @@ use super::{
GLOBAL_TOOLS_UTILS_DIR_NAME, HIDDEN_MCP_FILE_NAME, MACROS_DIR_NAME, MCP_FILE_NAME, GLOBAL_TOOLS_UTILS_DIR_NAME, HIDDEN_MCP_FILE_NAME, MACROS_DIR_NAME, MCP_FILE_NAME,
MEMORY_DIR_NAME, MEMORY_INDEX_FILE_NAME, ModelsOverride, RAGS_DIR_NAME, ROLES_DIR_NAME, MEMORY_DIR_NAME, MEMORY_INDEX_FILE_NAME, ModelsOverride, RAGS_DIR_NAME, ROLES_DIR_NAME,
SBX_KIT_DIR_NAME, SBX_KIT_HASH_FILE, SBX_MIXIN_FILE_NAME, SBX_MIXIN_KITS_DIR_NAME, SBX_KIT_DIR_NAME, SBX_KIT_HASH_FILE, SBX_MIXIN_FILE_NAME, SBX_MIXIN_KITS_DIR_NAME,
SBX_VAULT_MIXINS_DIR_NAME, SKILLS_DIR_NAME, WORKSPACE_COYOTE_DIR_NAME, SKILLS_DIR_NAME, WORKSPACE_COYOTE_DIR_NAME,
}; };
use crate::client::ProviderModels; use crate::client::ProviderModels;
use crate::config::REPL_HISTORY_DIR_NAME; use crate::config::REPL_HISTORY_DIR_NAME;
@@ -16,7 +16,7 @@ use anyhow::{Context, Result, anyhow, bail};
use log::LevelFilter; use log::LevelFilter;
use std::collections::HashSet; use std::collections::HashSet;
use std::env; use std::env;
use std::fs::{read_dir, read_to_string}; use std::fs::{read_dir, read_to_string, remove_file};
use std::path::{Path, PathBuf}; use std::path::{Path, PathBuf};
pub fn config_dir() -> PathBuf { pub fn config_dir() -> PathBuf {
@@ -148,14 +148,6 @@ pub fn sbx_kit_hash_file() -> PathBuf {
sbx_kit_dir().join(SBX_KIT_HASH_FILE) sbx_kit_dir().join(SBX_KIT_HASH_FILE)
} }
pub fn sbx_vault_mixins_dir() -> PathBuf {
cache_dir().join(SBX_VAULT_MIXINS_DIR_NAME)
}
pub fn sbx_vault_mixins_hash_file() -> PathBuf {
sbx_vault_mixins_dir().join(SBX_KIT_HASH_FILE)
}
pub fn sbx_mixin_kits_dir() -> PathBuf { pub fn sbx_mixin_kits_dir() -> PathBuf {
cache_dir().join(SBX_MIXIN_KITS_DIR_NAME) cache_dir().join(SBX_MIXIN_KITS_DIR_NAME)
} }
@@ -422,6 +414,10 @@ pub fn list_rags() -> Vec<String> {
for entry in rd.flatten() { for entry in rd.flatten() {
let name = entry.file_name(); let name = entry.file_name();
if let Some(name) = name.to_string_lossy().strip_suffix(".yaml") { if let Some(name) = name.to_string_lossy().strip_suffix(".yaml") {
if is_rag_sidecar_name(name) {
continue;
}
names.push(name.to_string()); names.push(name.to_string());
} }
} }
@@ -432,6 +428,34 @@ pub fn list_rags() -> Vec<String> {
} }
} }
pub(crate) fn is_rag_sidecar_name(name: &str) -> bool {
name.ends_with(".sbx-mixin")
}
pub(crate) fn remove_rag_sidecars(dir: &Path, name: &str) -> Result<()> {
let duckdb_path = dir.join(format!("{name}.duckdb"));
if duckdb_path.exists() {
let _ = remove_file(&duckdb_path);
}
let wal_path = dir.join(format!("{name}.duckdb.wal"));
if wal_path.exists() {
let _ = remove_file(&wal_path);
}
let mixin_path = dir.join(format!("{name}.sbx-mixin.yaml"));
if mixin_path.exists() {
remove_file(&mixin_path).with_context(|| {
format!(
"Failed to remove the sandbox mixin for RAG '{name}' at '{}'. \
The RAG was NOT deleted so you can retry; this host remains \
whitelisted in the sandbox until the file is removed.",
mixin_path.display()
)
})?;
}
Ok(())
}
pub fn list_macros() -> Vec<String> { pub fn list_macros() -> Vec<String> {
list_file_names(macros_dir(), ".yaml") list_file_names(macros_dir(), ".yaml")
} }
@@ -854,4 +878,72 @@ mod tests {
} }
let _ = fs::remove_dir_all(&root); let _ = fs::remove_dir_all(&root);
} }
fn sidecar_temp_dir(label: &str) -> PathBuf {
let unique = time::SystemTime::now()
.duration_since(time::UNIX_EPOCH)
.unwrap()
.as_nanos();
let root = env::temp_dir().join(format!("coyote-{label}-test-{unique}"));
fs::create_dir_all(&root).unwrap();
root
}
#[test]
fn is_rag_sidecar_name_accepts_dotted_rag_names() {
assert!(!is_rag_sidecar_name("v2.docs"));
assert!(!is_rag_sidecar_name("myrag"));
assert!(is_rag_sidecar_name("myrag.sbx-mixin"));
assert!(is_rag_sidecar_name("v2.docs.sbx-mixin"));
}
#[test]
fn remove_rag_sidecars_removes_duckdb_wal_and_mixin() {
let root = sidecar_temp_dir("rag-sidecars-both");
let duckdb = root.join("docs.duckdb");
let wal = root.join("docs.duckdb.wal");
let mixin = root.join("docs.sbx-mixin.yaml");
fs::write(&duckdb, "db").unwrap();
fs::write(&wal, "wal").unwrap();
fs::write(&mixin, "mixin").unwrap();
remove_rag_sidecars(&root, "docs").unwrap();
assert!(!duckdb.exists(), "the .duckdb sidecar must be removed");
assert!(!wal.exists(), "the .duckdb.wal sidecar must be removed");
assert!(
!mixin.exists(),
"the .sbx-mixin.yaml sidecar must be removed"
);
let _ = fs::remove_dir_all(&root);
}
#[test]
fn remove_rag_sidecars_is_ok_when_absent() {
let root = sidecar_temp_dir("rag-sidecars-absent");
assert!(remove_rag_sidecars(&root, "docs").is_ok());
let _ = fs::remove_dir_all(&root);
}
#[test]
fn remove_rag_sidecars_runs_before_yaml_unlink() {
let root = sidecar_temp_dir("rag-sidecars-order");
let yaml = root.join("docs.yaml");
fs::write(&yaml, "rag").unwrap();
let mixin = root.join("docs.sbx-mixin.yaml");
fs::create_dir_all(&mixin).unwrap();
fs::write(mixin.join("blocker"), "x").unwrap();
let err = remove_rag_sidecars(&root, "docs").unwrap_err();
assert!(
err.to_string()
.contains("Failed to remove the sandbox mixin"),
"got: {err}"
);
assert!(
yaml.exists(),
"the .yaml must survive a sidecar-removal failure so the delete is retryable"
);
let _ = fs::remove_dir_all(&root);
}
} }
+1 -1
View File
@@ -207,7 +207,7 @@ pub(in crate::config) const DEFAULT_USER_INTERACTION_INSTRUCTIONS: &str = indoc!
## User Interaction ## User Interaction
You have built-in tools to interact with the user directly: You have built-in tools to interact with the user directly:
- `user__ask --question \"...\" --options [\"A\", \"B\", \"C\"]`: Present a selection prompt. Returns the chosen option. - `user__select --question \"...\" --options [\"A\", \"B\", \"C\"]`: Present a single-select list of named options. Use this — not `user__confirm` — whenever there are 2+ named options. Returns the chosen option.
- `user__confirm --question \"...\"`: Ask a yes/no question. Returns \"yes\" or \"no\". - `user__confirm --question \"...\"`: Ask a yes/no question. Returns \"yes\" or \"no\".
- `user__input --question \"...\"`: Request free-form text input from the user. - `user__input --question \"...\"`: Request free-form text input from the user.
- `user__checkbox --question \"...\" --options [\"A\", \"B\", \"C\"]`: Multi-select prompt. Returns an array of selected options. - `user__checkbox --question \"...\" --options [\"A\", \"B\", \"C\"]`: Multi-select prompt. Returns an array of selected options.
+380 -48
View File
@@ -1,4 +1,3 @@
use super::agent::AgentConfig;
use super::rag_cache::{RagCache, RagKey}; use super::rag_cache::{RagCache, RagKey};
use super::session::Session; use super::session::Session;
use super::skill::{SKILL_SCAFFOLD, Skill}; use super::skill::{SKILL_SCAFFOLD, Skill};
@@ -10,15 +9,16 @@ use super::{
AGENTS_DIR_NAME, Agent, AgentVariables, AppConfig, AppState, AssetCategory, CREATE_TITLE_ROLE, AGENTS_DIR_NAME, Agent, AgentVariables, AppConfig, AppState, AssetCategory, CREATE_TITLE_ROLE,
Input, InstallFilter, LEFT_PROMPT, LastMessage, MESSAGES_FILE_NAME, RIGHT_PROMPT, Role, Input, InstallFilter, LEFT_PROMPT, LastMessage, MESSAGES_FILE_NAME, RIGHT_PROMPT, Role,
RoleLike, SESSIONS_DIR_NAME, SUMMARIZATION_PROMPT, SUMMARY_CONTEXT_PROMPT, StateFlags, RoleLike, SESSIONS_DIR_NAME, SUMMARIZATION_PROMPT, SUMMARY_CONTEXT_PROMPT, StateFlags,
TEMP_ROLE_NAME, TEMP_SESSION_NAME, WorkingMode, ensure_parent_exists, list_agents, TEMP_ROLE_NAME, TEMP_SESSION_NAME, WorkingMode, ensure_parent_exists,
list_agents_with_descriptions, memory, paths, list_agents_with_descriptions, memory, paths,
}; };
use super::{MessageContentToolCalls, prompts}; use super::{MessageContentToolCalls, prompts};
use crate::client::{Model, ModelType, list_models}; use crate::client::{Model, ModelType, list_models};
use crate::function::{ use crate::function::{
FunctionDeclaration, Functions, ToolCallTracker, ToolResult, memory::MEMORY_FUNCTION_PREFIX, FunctionDeclaration, Functions, ToolCallTracker, ToolResult, memory::MEMORY_FUNCTION_PREFIX,
skill::SKILL_FUNCTION_PREFIX, supervisor::SUPERVISOR_FUNCTION_PREFIX, rag_query::RAG_FUNCTION_PREFIX, skill::SKILL_FUNCTION_PREFIX,
todo::TODO_FUNCTION_PREFIX, user_interaction::USER_FUNCTION_PREFIX, supervisor::SUPERVISOR_FUNCTION_PREFIX, todo::TODO_FUNCTION_PREFIX,
user_interaction::USER_FUNCTION_PREFIX,
}; };
use crate::mcp::{ use crate::mcp::{
MCP_DESCRIBE_META_FUNCTION_NAME_PREFIX, MCP_INVOKE_META_FUNCTION_NAME_PREFIX, MCP_DESCRIBE_META_FUNCTION_NAME_PREFIX, MCP_INVOKE_META_FUNCTION_NAME_PREFIX,
@@ -39,6 +39,7 @@ use super::memory::{
}; };
use crate::graph; use crate::graph;
use anyhow::{Context, Error, Result, bail}; use anyhow::{Context, Error, Result, bail};
use colored::Colorize;
use gman::providers::SupportedProvider; use gman::providers::SupportedProvider;
#[cfg(test)] #[cfg(test)]
use indexmap::IndexMap; use indexmap::IndexMap;
@@ -142,6 +143,7 @@ pub struct RequestContext {
pub role: Option<Role>, pub role: Option<Role>,
pub session: Option<Session>, pub session: Option<Session>,
pub rag: Option<Arc<Rag>>, pub rag: Option<Arc<Rag>>,
pub rag_key: Option<RagKey>,
pub agent: Option<Agent>, pub agent: Option<Agent>,
pub last_message: Option<LastMessage>, pub last_message: Option<LastMessage>,
@@ -176,6 +178,7 @@ impl RequestContext {
role: None, role: None,
session: None, session: None,
rag: None, rag: None,
rag_key: None,
agent: None, agent: None,
last_message: None, last_message: None,
tool_scope: ToolScope::default(), tool_scope: ToolScope::default(),
@@ -229,6 +232,7 @@ impl RequestContext {
role: None, role: None,
session: None, session: None,
rag: None, rag: None,
rag_key: None,
agent: None, agent: None,
last_message: None, last_message: None,
tool_scope: ToolScope { tool_scope: ToolScope {
@@ -277,6 +281,7 @@ impl RequestContext {
role: self.role.clone(), role: self.role.clone(),
session: self.session.clone(), session: self.session.clone(),
rag: self.rag.clone(), rag: self.rag.clone(),
rag_key: self.rag_key.clone(),
agent: self.agent.clone(), agent: self.agent.clone(),
last_message: self.last_message.clone(), last_message: self.last_message.clone(),
tool_scope: self.tool_scope.clone(), tool_scope: self.tool_scope.clone(),
@@ -315,6 +320,7 @@ impl RequestContext {
role: None, role: None,
session: None, session: None,
rag: None, rag: None,
rag_key: None,
agent: None, agent: None,
last_message: None, last_message: None,
tool_scope: ToolScope { tool_scope: ToolScope {
@@ -710,6 +716,7 @@ impl RequestContext {
pub fn exit_rag(&mut self) -> Result<()> { pub fn exit_rag(&mut self) -> Result<()> {
self.rag.take(); self.rag.take();
self.tool_scope.functions.remove_rag_query_functions();
Ok(()) Ok(())
} }
@@ -729,6 +736,26 @@ impl RequestContext {
Ok(()) Ok(())
} }
pub fn on_chat_completion_error(&mut self, app: &AppConfig, input: &Input) {
self.last_message = Some(LastMessage::new(input.clone(), String::new()));
if input.session(&self.session).is_none() {
if let Some(lm) = self.last_message.as_mut() {
lm.continuous = false;
}
return;
}
let mut i = input.clone();
i.clear_patch();
if let Some(session) = i.session_mut(&mut self.session) {
let _ = session.add_message(&i, "[Response interrupted due to error]");
if !app.dry_run && session.save_session() == Some(true) {
let _ = session.flush();
}
}
}
pub fn discontinuous_last_message(&mut self) { pub fn discontinuous_last_message(&mut self) {
if let Some(last_message) = self.last_message.as_mut() { if let Some(last_message) = self.last_message.as_mut() {
last_message.continuous = false; last_message.continuous = false;
@@ -808,7 +835,12 @@ impl RequestContext {
let mut role = if let Some(session) = self.session.as_ref() { let mut role = if let Some(session) = self.session.as_ref() {
session.to_role() session.to_role()
} else if let Some(agent) = self.agent.as_ref() { } else if let Some(agent) = self.agent.as_ref() {
agent.to_role() let mut role = agent.to_role();
if role.reasoning_effort().is_none() {
role.set_reasoning_effort(app.reasoning_effort.clone());
}
role
} else if let Some(role) = self.role.as_ref() { } else if let Some(role) = self.role.as_ref() {
role.clone() role.clone()
} else { } else {
@@ -1087,10 +1119,17 @@ impl RequestContext {
} }
fn concrete_tool_names(&self) -> Vec<String> { fn concrete_tool_names(&self) -> Vec<String> {
let declarations = match &self.agent { let declarations: Vec<&FunctionDeclaration> = self
Some(agent) => agent.functions().declarations(), .tool_scope
None => self.tool_scope.functions.declarations(), .functions
}; .declarations()
.iter()
.chain(
self.agent
.iter()
.flat_map(|agent| agent.functions().declarations()),
)
.collect();
declarations declarations
.iter() .iter()
.filter(|v| { .filter(|v| {
@@ -1100,6 +1139,7 @@ impl RequestContext {
&& !v.name.starts_with("agent__") && !v.name.starts_with("agent__")
&& !v.name.starts_with("memory__") && !v.name.starts_with("memory__")
&& !v.name.starts_with("skill__") && !v.name.starts_with("skill__")
&& !v.name.starts_with("rag__")
}) })
.map(|v| v.name.clone()) .map(|v| v.name.clone())
.collect() .collect()
@@ -1920,7 +1960,8 @@ impl RequestContext {
|| (!matches!(role.skills_enabled(), Some(false)) || (!matches!(role.skills_enabled(), Some(false))
&& v.name.starts_with(SKILL_FUNCTION_PREFIX)) && v.name.starts_with(SKILL_FUNCTION_PREFIX))
|| (self.auto_continue_config().enabled || (self.auto_continue_config().enabled
&& v.name.starts_with(TODO_FUNCTION_PREFIX))) && v.name.starts_with(TODO_FUNCTION_PREFIX))
|| v.name.starts_with(RAG_FUNCTION_PREFIX))
&& !existing.contains(&v.name) && !existing.contains(&v.name)
}) })
.cloned() .cloned()
@@ -1950,6 +1991,7 @@ impl RequestContext {
|| v.name.starts_with(TODO_FUNCTION_PREFIX) || v.name.starts_with(TODO_FUNCTION_PREFIX)
|| v.name.starts_with(SUPERVISOR_FUNCTION_PREFIX) || v.name.starts_with(SUPERVISOR_FUNCTION_PREFIX)
|| v.name.starts_with(MEMORY_FUNCTION_PREFIX) || v.name.starts_with(MEMORY_FUNCTION_PREFIX)
|| v.name.starts_with(RAG_FUNCTION_PREFIX)
}); });
} }
@@ -2401,8 +2443,12 @@ impl RequestContext {
println!("Skills:"); println!("Skills:");
for (name, description, loaded) in entries { for (name, description, loaded) in entries {
let marker = if loaded { " (loaded)" } else { "" }; let marker = if loaded {
println!("{name}{marker}{description}"); "".green().bold().to_string()
} else {
"".red().bold().to_string()
};
println!(" {marker} {name}{description}");
} }
Ok(()) Ok(())
@@ -2426,7 +2472,31 @@ impl RequestContext {
names.sort_unstable(); names.sort_unstable();
names.dedup(); names.dedup();
print_asset_names("tools", &names) let active: HashSet<String> = if self.app.config.function_calling_support {
let role = self.extract_role(&self.app.config)?;
match self.select_functions(&role) {
None => HashSet::new(),
Some(functions) => functions.iter().map(|f| f.name.clone()).collect(),
}
} else {
HashSet::new()
};
if names.is_empty() {
println!("No tools found.");
return Ok(());
}
println!("Tools:");
for name in &names {
let marker = if active.contains(name.as_str()) {
"".green().bold().to_string()
} else {
"".red().bold().to_string()
};
println!(" {marker} {name}");
}
Ok(())
} }
"mcp-servers" => { "mcp-servers" => {
let mut names: Vec<String> = vec![]; let mut names: Vec<String> = vec![];
@@ -2443,7 +2513,32 @@ impl RequestContext {
names.sort_unstable(); names.sort_unstable();
names.dedup(); names.dedup();
print_asset_names("MCP servers", &names) if names.is_empty() {
println!("No MCP servers found.");
return Ok(());
}
let enabled: Option<Vec<String>> = if let Some(session) = &self.session {
session.enabled_mcp_servers()
} else if let Some(role) = &self.role {
role.enabled_mcp_servers()
} else {
self.app.config.enabled_mcp_servers.clone()
};
let skill_mcps = self.skill_registry.loaded_mcp_servers();
println!("MCP servers:");
for name in &names {
let active = skill_mcps.contains(name.as_str())
|| matches!(&enabled, Some(list) if list.iter().any(|s| s.trim() == "all") || self.mcp_list_covers(list, name));
let marker = if active {
"".green().bold().to_string()
} else {
"".red().bold().to_string()
};
println!(" {marker} {name}");
}
Ok(())
} }
_ => bail!( _ => bail!(
"Unknown kind '{kind}'. Valid kinds: roles, sessions, agents, rags, macros, skills, tools, mcp-servers" "Unknown kind '{kind}'. Valid kinds: roles, sessions, agents, rags, macros, skills, tools, mcp-servers"
@@ -2469,6 +2564,14 @@ impl RequestContext {
match file_ext { match file_ext {
Some(file_ext) => { Some(file_ext) => {
if let Some(name) = name.to_string_lossy().strip_suffix(file_ext) { if let Some(name) = name.to_string_lossy().strip_suffix(file_ext) {
// Sidecars are not independently deletable assets.
// Guarded on `kind == "rag"` because this scan is shared
// by all six kinds, and `session`/`macro` also use
// `.yaml`. The helper lives in paths.rs beside
// list_rags() so both filters cannot drift apart.
if kind == "rag" && paths::is_rag_sidecar_name(name) {
continue;
}
names.push(name.to_string()); names.push(name.to_string());
} }
} }
@@ -2505,6 +2608,13 @@ impl RequestContext {
match file_ext { match file_ext {
Some(ext) => { Some(ext) => {
let path = dir.join(format!("{name}{ext}")); let path = dir.join(format!("{name}{ext}"));
// Sidecars FIRST. If this fails, the .yaml is still on disk, the
// RAG is still listed, and the user can retry. Unlinking the .yaml
// first would make the deletion unretryable while leaving an
// orphaned mixin whitelisting a host in every sandbox launch.
if kind == "rag" {
paths::remove_rag_sidecars(&dir, &name)?;
}
remove_file(&path).with_context(|| { remove_file(&path).with_context(|| {
format!("Failed to delete {kind} at '{}'", path.display()) format!("Failed to delete {kind} at '{}'", path.display())
})?; })?;
@@ -2688,7 +2798,12 @@ impl RequestContext {
} }
} }
"rag_top_k" => { "rag_top_k" => {
let value = value.parse().with_context(|| "Invalid value")?; let value: usize = value.parse().with_context(|| "Invalid value")?;
if value == 0 {
bail!(
"rag_top_k must be >= 1; a top_k of 0 makes every query return no results."
);
}
if !self.set_rag_top_k(value)? { if !self.set_rag_top_k(value)? {
self.update_app_config(|app| app.rag_top_k = value); self.update_app_config(|app| app.rag_top_k = value);
} }
@@ -2876,15 +2991,9 @@ impl RequestContext {
} }
} }
".rag" => super::map_completion_values(paths::list_rags()), ".rag" => super::map_completion_values(paths::list_rags()),
".agent" => list_agents() ".agent" => list_agents_with_descriptions()
.into_iter() .into_iter()
.map(|name| { .map(|(name, desc)| (name, if desc.is_empty() { None } else { Some(desc) }))
let description = AgentConfig::load(&paths::agent_config_file(&name))
.ok()
.map(|c| c.description)
.filter(|d| !d.is_empty());
(name, description)
})
.collect(), .collect(),
".install" => { ".install" => {
let mut values: Vec<String> = let mut values: Vec<String> =
@@ -3363,6 +3472,12 @@ impl RequestContext {
if self.should_register_memory_tools() { if self.should_register_memory_tools() {
functions.append_memory_functions(); functions.append_memory_functions();
} }
if self.rag.is_some()
&& app.function_calling_support
&& !self.agent.as_ref().is_some_and(|a| a.is_graph())
{
functions.append_rag_query_functions();
}
let tool_tracker = self.tool_scope.tool_tracker.clone(); let tool_tracker = self.tool_scope.tool_tracker.clone();
self.tool_scope = ToolScope { self.tool_scope = ToolScope {
@@ -3645,6 +3760,14 @@ impl RequestContext {
.then(|| Arc::new(RwLock::new(Supervisor::new(max_concurrent, max_depth)))); .then(|| Arc::new(RwLock::new(Supervisor::new(max_concurrent, max_depth))));
self.rag = agent.rag(); self.rag = agent.rag();
// Keep `rag_key` in lockstep with `rag`. Agent RAGs are cached under
// `RagKey::Agent(<agent name>)` (see `Agent::init`), so mirror that key exactly;
// leaving the previous key in place would let `.rebuild rag` invalidate an
// unrelated RAG's cache entry, and leaving it `None` would invalidate nothing.
self.rag_key = self
.rag
.is_some()
.then(|| RagKey::Agent(agent.name().to_string()));
self.agent = Some(agent); self.agent = Some(agent);
self.supervisor = supervisor; self.supervisor = supervisor;
self.inbox = None; self.inbox = None;
@@ -3693,6 +3816,11 @@ impl RequestContext {
self.pending_agents_guardrail_count = 0; self.pending_agents_guardrail_count = 0;
self.todo_list = TodoList::default(); self.todo_list = TodoList::default();
self.rag.take(); self.rag.take();
// Cleared alongside `rag` so the pair never disagrees: an agent RAG is
// cached under `RagKey::Agent(<agent name>)`, and leaving that key behind
// would outlive the RAG it names. Latent rather than live today only
// because `rebuild_rag`/`edit_rag_docs` bail on `rag.is_none()` first.
self.rag_key = None;
self.discontinuous_last_message(); self.discontinuous_last_message();
} }
Ok(()) Ok(())
@@ -4000,10 +4128,11 @@ impl RequestContext {
} }
let app = self.app.config.clone(); let app = self.app.config.clone();
let vault = self.app.vault.clone();
let rag_cache = self.rag_cache(); let rag_cache = self.rag_cache();
let working_mode = self.working_mode; let working_mode = self.working_mode;
let rag: Arc<Rag> = match rag { let (rag, rag_key): (Arc<Rag>, Option<RagKey>) = match rag {
None => { None => {
let rag_path = self.rag_file(super::TEMP_RAG_NAME); let rag_path = self.rag_file(super::TEMP_RAG_NAME);
if rag_path.exists() { if rag_path.exists() {
@@ -4011,15 +4140,29 @@ impl RequestContext {
format!("Failed to cleanup previous '{}' rag", super::TEMP_RAG_NAME) format!("Failed to cleanup previous '{}' rag", super::TEMP_RAG_NAME)
})?; })?;
} }
Arc::new(Rag::init(&app, super::TEMP_RAG_NAME, &rag_path, &[], abort_signal).await?) (
Arc::new(
Rag::init(
&app,
super::TEMP_RAG_NAME,
&rag_path,
&[],
abort_signal.clone(),
false,
)
.await?,
),
None,
)
} }
Some(name) => { Some(name) => {
let rag_path = self.rag_file(name); let rag_path = self.rag_file(name);
let key = RagKey::Named(name.to_string()); let key = RagKey::Named(name.to_string());
rag_cache let loaded = rag_cache
.load_with(key, || { .load_with(key.clone(), || {
let app = app.clone(); let app = app.clone();
let vault = vault.clone();
let rag_path = rag_path.clone(); let rag_path = rag_path.clone();
let abort_signal = abort_signal.clone(); let abort_signal = abort_signal.clone();
async move { async move {
@@ -4027,16 +4170,41 @@ impl RequestContext {
if working_mode.is_cmd() { if working_mode.is_cmd() {
bail!("Unknown RAG '{name}'"); bail!("Unknown RAG '{name}'");
} }
Rag::init(&app, name, &rag_path, &[], abort_signal.clone()).await Rag::init(&app, name, &rag_path, &[], abort_signal.clone(), true)
.await
} else { } else {
Rag::load(&app, name, &rag_path) Rag::load_async(&app, &vault, name, &rag_path).await
} }
} }
}) })
.await? .await?;
(loaded, Some(key))
} }
}; };
self.rag = Some(rag); self.rag = Some(rag);
self.rag_key = rag_key;
self.refresh_tool_scope(abort_signal).await?;
Ok(())
}
pub async fn attach_rag(&mut self, name: &str, abort_signal: AbortSignal) -> Result<()> {
let rag_path = self.rag_file(name);
if rag_path.exists() {
bail!(
"RAG '{name}' already exists at '{}'. \
Use a different name, or delete the existing file first.",
rag_path.display()
);
}
let app = self.app.config.as_ref();
let vault = self.app.vault.clone();
let rag = Rag::attach(app, &vault, name, &rag_path).await?;
let rag = Arc::new(rag);
let key = RagKey::Named(name.to_string());
self.rag_cache().insert(key.clone(), &rag);
self.rag = Some(rag);
self.rag_key = Some(key);
self.refresh_tool_scope(abort_signal).await?;
Ok(()) Ok(())
} }
@@ -4046,6 +4214,12 @@ impl RequestContext {
None => bail!("No RAG"), None => bail!("No RAG"),
}; };
if rag.is_attached() {
bail!(
"Cannot edit documents on an attached RAG; Coyote does not own its source documents."
);
}
let document_paths = rag.document_paths(); let document_paths = rag.document_paths();
let temp_file = temp_file(&format!("-rag-{}", rag.name()), ".txt"); let temp_file = temp_file(&format!("-rag-{}", rag.name()), ".txt");
tokio::fs::write(&temp_file, &document_paths.join("\n")) tokio::fs::write(&temp_file, &document_paths.join("\n"))
@@ -4071,15 +4245,18 @@ impl RequestContext {
bail!("No changes") bail!("No changes")
} }
let key = if self.agent.is_some() { if let Some(key) = self.rag_key.clone() {
RagKey::Agent(rag.name().to_string()) self.rag_cache().invalidate(&key);
} else { }
RagKey::Named(rag.name().to_string())
};
self.rag_cache().invalidate(&key);
rag.refresh_document_paths(&new_document_paths, false, &self.app.config, abort_signal) rag.refresh_document_paths(
.await?; &new_document_paths,
false,
false,
&self.app.config,
abort_signal,
)
.await?;
self.rag = Some(Arc::new(rag)); self.rag = Some(Arc::new(rag));
Ok(()) Ok(())
} }
@@ -4090,15 +4267,25 @@ impl RequestContext {
None => bail!("No RAG"), None => bail!("No RAG"),
}; };
let key = if self.agent.is_some() { if rag.is_attached() {
RagKey::Agent(rag.name().to_string()) bail!(
} else { "Cannot rebuild an attached RAG; Coyote does not own its source documents. \
RagKey::Named(rag.name().to_string()) Re-index from the system that originally created '{}'.",
}; rag.name()
self.rag_cache().invalidate(&key); );
}
if let Some(key) = self.rag_key.clone() {
self.rag_cache().invalidate(&key);
}
let document_paths = rag.document_paths().to_vec(); let document_paths = rag.document_paths().to_vec();
rag.refresh_document_paths(&document_paths, true, &self.app.config, abort_signal) println!(
"Rebuilding re-embeds every document ({} files). \
This will call the embedding API and may take a while.",
rag.file_count()
);
rag.refresh_document_paths(&document_paths, true, true, &self.app.config, abort_signal)
.await?; .await?;
self.rag = Some(Arc::new(rag)); self.rag = Some(Arc::new(rag));
Ok(()) Ok(())
@@ -4121,6 +4308,7 @@ mod tests {
use super::super::mcp_factory::McpFactory; use super::super::mcp_factory::McpFactory;
use super::*; use super::*;
use crate::config::AppState; use crate::config::AppState;
use crate::config::agent::AgentConfig;
use crate::function::{ToolCall, skill}; use crate::function::{ToolCall, skill};
use crate::mcp::{McpServer, McpServersConfig, McpTransportType}; use crate::mcp::{McpServer, McpServersConfig, McpTransportType};
use crate::utils; use crate::utils;
@@ -4304,6 +4492,42 @@ mod tests {
assert_eq!(extracted.name(), ""); assert_eq!(extracted.name(), "");
} }
#[test]
fn extract_role_agent_without_reasoning_effort_inherits_app_config() {
let mut ctx = create_test_ctx();
ctx.agent = Some(Agent::test_new(AgentConfig {
name: "test-agent".to_string(),
reasoning_effort: None,
..AgentConfig::default()
}));
let app = AppConfig {
reasoning_effort: Some("max".to_string()),
..AppConfig::default()
};
let extracted = ctx.extract_role(&app).unwrap();
assert_eq!(extracted.reasoning_effort(), Some("max".to_string()));
}
#[test]
fn extract_role_agent_with_explicit_reasoning_effort_takes_priority_over_app_config() {
let mut ctx = create_test_ctx();
ctx.agent = Some(Agent::test_new(AgentConfig {
name: "test-agent".to_string(),
reasoning_effort: Some("low".to_string()),
..AgentConfig::default()
}));
let app = AppConfig {
reasoning_effort: Some("max".to_string()),
..AppConfig::default()
};
let extracted = ctx.extract_role(&app).unwrap();
assert_eq!(extracted.reasoning_effort(), Some("low".to_string()));
}
#[test] #[test]
fn should_inject_skill_instructions_requires_function_calling() { fn should_inject_skill_instructions_requires_function_calling() {
let app = AppConfig { let app = AppConfig {
@@ -4467,6 +4691,44 @@ mod tests {
assert!(ctx.agent.is_none()); assert!(ctx.agent.is_none());
assert!(ctx.rag.is_none()); assert!(ctx.rag.is_none());
assert_eq!(ctx.rag_key, None);
}
#[test]
#[serial]
fn use_agent_does_not_carry_stale_rag_key() {
let _guard = TestConfigDirGuard::new();
let mut ctx = create_test_ctx();
let app = ctx.app.config.clone();
let agent_name = format!(
"test_agent_{}",
SystemTime::now()
.duration_since(UNIX_EPOCH)
.unwrap()
.as_nanos()
);
let agent_dir = paths::agent_data_dir(&agent_name);
create_dir_all(&agent_dir).unwrap();
write(
agent_dir.join("config.yaml"),
format!("name: {agent_name}\ninstructions: hi\n"),
)
.unwrap();
ctx.rag_key = Some(RagKey::Named("docs".to_string()));
tokio::runtime::Builder::new_current_thread()
.enable_all()
.build()
.unwrap()
.block_on(async {
ctx.use_agent(&app, &agent_name, None, utils::create_abort_signal())
.await
.unwrap();
});
assert!(ctx.rag.is_none());
assert_eq!(ctx.rag_key, None);
} }
#[test] #[test]
@@ -4784,7 +5046,7 @@ mod tests {
let fns = ctx.select_functions(&role).unwrap(); let fns = ctx.select_functions(&role).unwrap();
let names: Vec<&str> = fns.iter().map(|f| f.name.as_str()).collect(); let names: Vec<&str> = fns.iter().map(|f| f.name.as_str()).collect();
assert!(names.contains(&"todo__init")); assert!(names.contains(&"todo__init"));
assert!(names.contains(&"user__ask")); assert!(names.contains(&"user__select"));
} }
#[test] #[test]
@@ -4848,7 +5110,7 @@ mod tests {
let fns = ctx.select_functions(&role).unwrap(); let fns = ctx.select_functions(&role).unwrap();
let names: Vec<&str> = fns.iter().map(|f| f.name.as_str()).collect(); let names: Vec<&str> = fns.iter().map(|f| f.name.as_str()).collect();
assert!(names.contains(&"user__ask")); assert!(names.contains(&"user__select"));
assert!(!names.contains(&"skill__list")); assert!(!names.contains(&"skill__list"));
} }
@@ -4933,7 +5195,7 @@ mod tests {
"teammate tools must survive an agent tool filter, got: {names:?}" "teammate tools must survive an agent tool filter, got: {names:?}"
); );
assert!( assert!(
names.contains(&"user__ask"), names.contains(&"user__select"),
"user__ tools must survive an agent tool filter, got: {names:?}" "user__ tools must survive an agent tool filter, got: {names:?}"
); );
} }
@@ -5431,6 +5693,54 @@ mod tests {
assert!(lm.continuous); assert!(lm.continuous);
} }
#[test]
fn on_chat_completion_error_without_session_sets_last_message_discontinuous() {
let mut ctx = create_test_ctx();
let app = Arc::clone(&ctx.app.config);
let input = Input::from_str(&ctx, "hello", None).unwrap();
ctx.on_chat_completion_error(app.as_ref(), &input);
let lm = ctx.last_message.as_ref().unwrap();
assert_eq!(lm.output, "");
assert!(!lm.continuous, "no session means recovery is not possible");
}
#[test]
fn on_chat_completion_error_with_session_sets_last_message_continuous() {
let mut ctx = create_test_ctx();
ctx.app = Arc::new(AppState {
config: Arc::new(AppConfig {
dry_run: true,
..(*ctx.app.config).clone()
}),
..(*ctx.app).clone()
});
ctx.session = Some(Session::default());
let app = Arc::clone(&ctx.app.config);
let input = Input::from_str(&ctx, "hello", None).unwrap();
ctx.on_chat_completion_error(app.as_ref(), &input);
let lm = ctx.last_message.as_ref().unwrap();
assert_eq!(lm.output, "");
assert!(lm.continuous, "session present means .recover is available");
}
#[test]
fn on_chat_completion_error_with_session_checkpoints_session_messages() {
let mut ctx = create_test_ctx();
ctx.session = Some(Session::default());
assert!(ctx.session.as_ref().unwrap().is_empty());
let app = Arc::clone(&ctx.app.config);
let input = Input::from_str(&ctx, "hello", None).unwrap();
ctx.on_chat_completion_error(app.as_ref(), &input);
assert!(
!ctx.session.as_ref().unwrap().is_empty(),
"session should have the interrupted turn checkpointed"
);
}
#[test] #[test]
fn after_chat_completion_sweeps_auto_unload_skills_at_turn_end() { fn after_chat_completion_sweeps_auto_unload_skills_at_turn_end() {
let mut ctx = create_test_ctx(); let mut ctx = create_test_ctx();
@@ -5806,6 +6116,28 @@ mod tests {
assert!(paths::list_rags().is_empty()); assert!(paths::list_rags().is_empty());
} }
#[test]
#[serial]
fn list_rags_skips_sbx_mixin_sidecars() {
let _guard = TestConfigDirGuard::new();
let rags_dir = paths::rags_dir();
create_dir_all(&rags_dir).unwrap();
write(rags_dir.join("docs.yaml"), "embedding_model: test").unwrap();
write(rags_dir.join("docs.sbx-mixin.yaml"), "kind: mixin").unwrap();
write(rags_dir.join("v2.docs.yaml"), "embedding_model: test").unwrap();
let names = paths::list_rags();
assert!(names.contains(&"docs".to_string()));
assert!(
names.contains(&"v2.docs".to_string()),
"a dotted RAG name must still be listed: {names:?}"
);
assert!(
!names.contains(&"docs.sbx-mixin".to_string()),
"the sandbox mixin sidecar must not appear as a RAG: {names:?}"
);
}
#[test] #[test]
#[serial] #[serial]
fn use_agent_errors_when_already_in_session() { fn use_agent_errors_when_already_in_session() {
+13
View File
@@ -699,6 +699,19 @@ impl Session {
Ok(()) Ok(())
} }
pub fn flush(&mut self) -> Result<()> {
if !self.dirty {
return Ok(());
}
if let Some(path) = self.path.clone() {
let name = self.name.clone();
self.save(&name, Path::new(&path), false)?;
}
Ok(())
}
pub fn guard_empty(&self) -> Result<()> { pub fn guard_empty(&self) -> Result<()> {
if !self.is_empty() { if !self.is_empty() {
bail!( bail!(
+199 -21
View File
@@ -1,4 +1,5 @@
pub(crate) mod memory; pub(crate) mod memory;
pub(crate) mod rag_query;
pub(crate) mod skill; pub(crate) mod skill;
pub(crate) mod supervisor; pub(crate) mod supervisor;
pub(crate) mod todo; pub(crate) mod todo;
@@ -19,9 +20,11 @@ use crate::mcp::{
}; };
use crate::parsers::{bash, python, typescript}; use crate::parsers::{bash, python, typescript};
use anyhow::{Context, Result, anyhow, bail}; use anyhow::{Context, Result, anyhow, bail};
use futures_util::future;
use indexmap::IndexMap; use indexmap::IndexMap;
use indoc::formatdoc; use indoc::formatdoc;
use memory::MEMORY_FUNCTION_PREFIX; use memory::MEMORY_FUNCTION_PREFIX;
use rag_query::RAG_FUNCTION_PREFIX;
use rust_embed::Embed; use rust_embed::Embed;
use serde::{Deserialize, Serialize}; use serde::{Deserialize, Serialize};
use serde_json::{Value, json}; use serde_json::{Value, json};
@@ -30,6 +33,7 @@ use std::collections::VecDeque;
use std::ffi::OsStr; use std::ffi::OsStr;
use std::fs::File; use std::fs::File;
use std::io::{Read, Write}; use std::io::{Read, Write};
use std::sync::atomic::Ordering;
use std::{ use std::{
collections::{HashMap, HashSet}, collections::{HashMap, HashSet},
env, fs, io, env, fs, io,
@@ -145,21 +149,54 @@ pub async fn eval_tool_calls(
if calls.is_empty() { if calls.is_empty() {
bail!("The request was aborted because an infinite loop of function calls was detected.") bail!("The request was aborted because an infinite loop of function calls was detected.")
} }
for call in calls { let mut to_execute: Vec<(usize, ToolCall)> = Vec::with_capacity(calls.len());
let mut indexed_results: Vec<(usize, ToolResult)> = vec![];
for (idx, call) in calls.into_iter().enumerate() {
if let Some(msg) = ctx.tool_scope.tool_tracker.check_loop(&call.clone()) { if let Some(msg) = ctx.tool_scope.tool_tracker.check_loop(&call.clone()) {
let dup_msg = format!("{{\"tool_call_loop_alert\":{}}}", msg.trim()); let dup_msg = format!("{{\"tool_call_loop_alert\":{}}}", msg.trim());
println!( println!(
"{}", "{}",
warning_text(format!("{}: ⚠️ Tool-call loop detected! ⚠️", call.name).as_str()) muted_warning_text(
format!("{}: ⚠️ Tool-call loop detected! ⚠️", call.name).as_str()
)
); );
let val = json!(dup_msg); indexed_results.push((idx, ToolResult::new(call, json!(dup_msg))));
output.push(ToolResult::new(call, val)); } else {
continue; to_execute.push((idx, call));
} }
let result = call.eval(ctx).await?;
output.push(ToolResult::new(call, normalize_tool_result(result)));
} }
let (mcp_calls, sequential_calls): (Vec<_>, Vec<_>) =
to_execute.into_iter().partition(|(_, call)| {
call.name.starts_with(MCP_INVOKE_META_FUNCTION_NAME_PREFIX)
|| call.name.starts_with(MCP_SEARCH_META_FUNCTION_NAME_PREFIX)
|| call
.name
.starts_with(MCP_DESCRIBE_META_FUNCTION_NAME_PREFIX)
});
if !mcp_calls.is_empty() {
let ctx_ref: &RequestContext = ctx;
let futs: Vec<_> = mcp_calls
.into_iter()
.map(|(idx, call)| async move {
let result = call.eval_mcp(ctx_ref).await;
(idx, call, result)
})
.collect();
for (idx, call, result) in future::join_all(futs).await {
indexed_results.push((idx, ToolResult::new(call, normalize_tool_result(result?))));
}
}
for (idx, call) in sequential_calls {
let result = call.eval(ctx).await?;
indexed_results.push((idx, ToolResult::new(call, normalize_tool_result(result))));
}
indexed_results.sort_unstable_by_key(|(idx, _)| *idx);
output = indexed_results.into_iter().map(|(_, r)| r).collect();
if !output.is_empty() { if !output.is_empty() {
let (has_escalations, summary) = if ctx.current_depth == 0 let (has_escalations, summary) = if ctx.current_depth == 0
&& let Some(queue) = ctx.root_escalation_queue() && let Some(queue) = ctx.root_escalation_queue()
@@ -460,6 +497,16 @@ impl Functions {
.extend(user_interaction::user_interaction_function_declarations()); .extend(user_interaction::user_interaction_function_declarations());
} }
pub fn append_rag_query_functions(&mut self) {
self.declarations
.extend(rag_query::rag_query_function_declarations());
}
pub fn remove_rag_query_functions(&mut self) {
self.declarations
.retain(|f| !f.name.starts_with(RAG_FUNCTION_PREFIX));
}
pub fn append_mcp_meta_functions(&mut self, mcp_servers: Vec<String>) { pub fn append_mcp_meta_functions(&mut self, mcp_servers: Vec<String>) {
let mut invoke_function_properties = IndexMap::new(); let mut invoke_function_properties = IndexMap::new();
invoke_function_properties.insert( invoke_function_properties.insert(
@@ -1062,6 +1109,62 @@ impl ToolCall {
self self
} }
fn parse_arguments(&self) -> Result<Value> {
if self.arguments.is_object() {
Ok(self.arguments.clone())
} else if let Some(arguments) = self.arguments.as_str() {
serde_json::from_str(arguments).map_err(|_| {
anyhow!(
"The call '{}' has invalid arguments: {arguments}",
self.name
)
})
} else {
bail!(
"The call '{}' has invalid arguments: {}",
self.name,
self.arguments
)
}
}
async fn eval_mcp(&self, ctx: &RequestContext) -> Result<Value> {
let json_data = self.parse_arguments()?;
let cmd_name = self.name.as_str();
if *IS_STDOUT_TERMINAL && ctx.current_depth == 0 && !HEADLESS.load(Ordering::SeqCst) {
println!(
"{}",
format_call_log(cmd_name, &[json_data.to_string()], &json_data)
);
}
let result = if cmd_name.starts_with(MCP_SEARCH_META_FUNCTION_NAME_PREFIX) {
Self::search_mcp_tools(ctx, cmd_name, &json_data)
.await
.unwrap_or_else(|e| {
let error_msg = format!("MCP search failed: {e}");
eprintln!("{}", muted_warning_text(&format!("⚠️ {error_msg} ⚠️")));
json!({"tool_call_error": error_msg})
})
} else if cmd_name.starts_with(MCP_DESCRIBE_META_FUNCTION_NAME_PREFIX) {
Self::describe_mcp_tool(ctx, cmd_name, json_data.clone())
.await
.unwrap_or_else(|e| {
let error_msg = format!("MCP describe failed: {e}");
eprintln!("{}", muted_warning_text(&format!("⚠️ {error_msg} ⚠️")));
json!({"tool_call_error": error_msg})
})
} else {
Self::invoke_mcp_tool(ctx, cmd_name, &json_data)
.await
.unwrap_or_else(|e| {
let error_msg = format!("MCP tool invocation failed: {e}");
eprintln!("{}", muted_warning_text(&format!("⚠️ {error_msg} ⚠️")));
json!({"tool_call_error": error_msg})
})
};
Ok(result)
}
pub async fn eval(&self, ctx: &mut RequestContext) -> Result<Value> { pub async fn eval(&self, ctx: &mut RequestContext) -> Result<Value> {
let agent = ctx.agent.clone(); let agent = ctx.agent.clone();
let functions = ctx.tool_scope.functions.clone(); let functions = ctx.tool_scope.functions.clone();
@@ -1088,10 +1191,8 @@ impl ToolCall {
cmd_args.push(json_data.to_string()); cmd_args.push(json_data.to_string());
let prompt = format!("Call {cmd_name} {}", cmd_args.join(" ")); if *IS_STDOUT_TERMINAL && current_depth == 0 && !HEADLESS.load(Ordering::SeqCst) {
println!("{}", format_call_log(&cmd_name, &cmd_args, &json_data));
if *IS_STDOUT_TERMINAL && current_depth == 0 {
println!("{}", dimmed_text(&prompt));
} }
let output = match cmd_name.as_str() { let output = match cmd_name.as_str() {
@@ -1100,7 +1201,7 @@ impl ToolCall {
.await .await
.unwrap_or_else(|e| { .unwrap_or_else(|e| {
let error_msg = format!("MCP search failed: {e}"); let error_msg = format!("MCP search failed: {e}");
eprintln!("{}", warning_text(&format!("⚠️ {error_msg} ⚠️"))); eprintln!("{}", muted_warning_text(&format!("⚠️ {error_msg} ⚠️")));
json!({"tool_call_error": error_msg}) json!({"tool_call_error": error_msg})
}) })
} }
@@ -1109,7 +1210,7 @@ impl ToolCall {
.await .await
.unwrap_or_else(|e| { .unwrap_or_else(|e| {
let error_msg = format!("MCP describe failed: {e}"); let error_msg = format!("MCP describe failed: {e}");
eprintln!("{}", warning_text(&format!("⚠️ {error_msg} ⚠️"))); eprintln!("{}", muted_warning_text(&format!("⚠️ {error_msg} ⚠️")));
json!({"tool_call_error": error_msg}) json!({"tool_call_error": error_msg})
}) })
} }
@@ -1118,21 +1219,21 @@ impl ToolCall {
.await .await
.unwrap_or_else(|e| { .unwrap_or_else(|e| {
let error_msg = format!("MCP tool invocation failed: {e}"); let error_msg = format!("MCP tool invocation failed: {e}");
eprintln!("{}", warning_text(&format!("⚠️ {error_msg} ⚠️"))); eprintln!("{}", muted_warning_text(&format!("⚠️ {error_msg} ⚠️")));
json!({"tool_call_error": error_msg}) json!({"tool_call_error": error_msg})
}) })
} }
_ if cmd_name.starts_with(TODO_FUNCTION_PREFIX) => { _ if cmd_name.starts_with(TODO_FUNCTION_PREFIX) => {
todo::handle_todo_tool(ctx, &cmd_name, &json_data).unwrap_or_else(|e| { todo::handle_todo_tool(ctx, &cmd_name, &json_data).unwrap_or_else(|e| {
let error_msg = format!("Todo tool failed: {e}"); let error_msg = format!("Todo tool failed: {e}");
eprintln!("{}", warning_text(&format!("⚠️ {error_msg} ⚠️"))); eprintln!("{}", muted_warning_text(&format!("⚠️ {error_msg} ⚠️")));
json!({"tool_call_error": error_msg}) json!({"tool_call_error": error_msg})
}) })
} }
_ if cmd_name.starts_with(MEMORY_FUNCTION_PREFIX) => { _ if cmd_name.starts_with(MEMORY_FUNCTION_PREFIX) => {
memory::handle_memory_tool(ctx, &cmd_name, &json_data).unwrap_or_else(|e| { memory::handle_memory_tool(ctx, &cmd_name, &json_data).unwrap_or_else(|e| {
let error_msg = format!("Memory tool failed: {e}"); let error_msg = format!("Memory tool failed: {e}");
eprintln!("{}", warning_text(&format!("⚠️ {error_msg} ⚠️"))); eprintln!("{}", muted_warning_text(&format!("⚠️ {error_msg} ⚠️")));
json!({"tool_call_error": error_msg}) json!({"tool_call_error": error_msg})
}) })
} }
@@ -1141,7 +1242,7 @@ impl ToolCall {
.await .await
.unwrap_or_else(|e| { .unwrap_or_else(|e| {
let error_msg = format!("Skill tool failed: {e}"); let error_msg = format!("Skill tool failed: {e}");
eprintln!("{}", warning_text(&format!("⚠️ {error_msg} ⚠️"))); eprintln!("{}", muted_warning_text(&format!("⚠️ {error_msg} ⚠️")));
json!({"tool_call_error": error_msg}) json!({"tool_call_error": error_msg})
}) })
} }
@@ -1150,7 +1251,7 @@ impl ToolCall {
.await .await
.unwrap_or_else(|e| { .unwrap_or_else(|e| {
let error_msg = format!("Supervisor tool failed: {e}"); let error_msg = format!("Supervisor tool failed: {e}");
eprintln!("{}", warning_text(&format!("⚠️ {error_msg} ⚠️"))); eprintln!("{}", muted_warning_text(&format!("⚠️ {error_msg} ⚠️")));
json!({"tool_call_error": error_msg}) json!({"tool_call_error": error_msg})
}) })
} }
@@ -1159,7 +1260,16 @@ impl ToolCall {
.await .await
.unwrap_or_else(|e| { .unwrap_or_else(|e| {
let error_msg = format!("User interaction failed: {e}"); let error_msg = format!("User interaction failed: {e}");
eprintln!("{}", warning_text(&format!("⚠️ {error_msg} ⚠️"))); eprintln!("{}", muted_warning_text(&format!("⚠️ {error_msg} ⚠️")));
json!({"tool_call_error": error_msg})
})
}
_ if cmd_name.starts_with(RAG_FUNCTION_PREFIX) => {
rag_query::handle_rag_tool(ctx, &cmd_name, &json_data)
.await
.unwrap_or_else(|e| {
let error_msg = format!("RAG query failed: {e}");
eprintln!("{}", muted_warning_text(&format!("⚠️ {error_msg} ⚠️")));
json!({"tool_call_error": error_msg}) json!({"tool_call_error": error_msg})
}) })
} }
@@ -1419,7 +1529,10 @@ pub fn run_llm_function(
let stderr = String::from_utf8_lossy(&stderr_bytes).trim().to_string(); let stderr = String::from_utf8_lossy(&stderr_bytes).trim().to_string();
let stdout = String::from_utf8_lossy(&stdout_bytes).trim().to_string(); let stdout = String::from_utf8_lossy(&stdout_bytes).trim().to_string();
let tool_error_message = format!("Tool call '{command_name}' exited with code {exit_code}"); let tool_error_message = format!("Tool call '{command_name}' exited with code {exit_code}");
eprintln!("{}", warning_text(&format!("⚠️ {tool_error_message} ⚠️"))); eprintln!(
"{}",
muted_warning_text(&format!("⚠️ {tool_error_message} ⚠️"))
);
let mut error_json = json!({"tool_call_error": tool_error_message}); let mut error_json = json!({"tool_call_error": tool_error_message});
if !stderr.is_empty() { if !stderr.is_empty() {
error_json["stderr"] = json!(stderr); error_json["stderr"] = json!(stderr);
@@ -1552,6 +1665,47 @@ impl ToolCallTracker {
} }
} }
fn format_call_log(cmd_name: &str, cmd_args: &[String], json_data: &serde_json::Value) -> String {
if *NO_COLOR {
return format!("Call {cmd_name} {}", cmd_args.join(" "));
}
let prefix_args = &cmd_args[..cmd_args.len().saturating_sub(1)];
let prefix = if prefix_args.is_empty() {
String::new()
} else {
format!("{} ", dimmed_text(&prefix_args.join(" ")))
};
format!(
"{}{} {}{}",
dimmed_text("Call "),
cyan_bold_text(cmd_name),
prefix,
format_json_colored_keys(json_data),
)
}
fn format_json_colored_keys(value: &serde_json::Value) -> String {
let serde_json::Value::Object(map) = value else {
return dimmed_text(&value.to_string());
};
if map.is_empty() {
return dimmed_text("{}");
}
let pairs: Vec<String> = map
.iter()
.map(|(k, v)| {
let key = magenta_text(&format!("\"{k}\""));
format!("{}{}", key, dimmed_text(&format!(": {v}")))
})
.collect();
format!(
"{}{}{}",
dimmed_text("{"),
pairs.join(&dimmed_text(", ")),
dimmed_text("}")
)
}
#[cfg(test)] #[cfg(test)]
mod tests { mod tests {
use super::*; use super::*;
@@ -1836,7 +1990,7 @@ mod tests {
fn functions_append_user_interaction_adds_declarations() { fn functions_append_user_interaction_adds_declarations() {
let mut f = Functions::default(); let mut f = Functions::default();
f.append_user_interaction_functions(); f.append_user_interaction_functions();
assert!(f.contains("user__ask")); assert!(f.contains("user__select"));
assert!(f.contains("user__confirm")); assert!(f.contains("user__confirm"));
assert!(f.contains("user__input")); assert!(f.contains("user__input"));
assert!(f.contains("user__checkbox")); assert!(f.contains("user__checkbox"));
@@ -1973,4 +2127,28 @@ mod tests {
assert!(result.text.is_none()); assert!(result.text.is_none());
assert!(result.thinking.is_empty()); assert!(result.thinking.is_empty());
} }
#[test]
fn parse_arguments_passes_through_object() {
let tc = call_with_args("t", json!({"x": 1, "y": "hello"}));
assert_eq!(tc.parse_arguments().unwrap(), json!({"x": 1, "y": "hello"}));
}
#[test]
fn parse_arguments_deserializes_json_string() {
let tc = call_with_args("t", json!(r#"{"a": true}"#));
assert_eq!(tc.parse_arguments().unwrap(), json!({"a": true}));
}
#[test]
fn parse_arguments_returns_err_for_invalid_json_string() {
let tc = call_with_args("t", json!("not json {"));
assert!(tc.parse_arguments().is_err());
}
#[test]
fn parse_arguments_returns_err_for_non_object_non_string() {
let tc = call_with_args("t", json!(42));
assert!(tc.parse_arguments().is_err());
}
} }
+101
View File
@@ -0,0 +1,101 @@
use super::{FunctionDeclaration, JsonSchema};
use crate::config::RequestContext;
use anyhow::{Result, anyhow};
use indexmap::IndexMap;
use serde_json::{Value, json};
pub const RAG_FUNCTION_PREFIX: &str = "rag__";
pub fn rag_query_function_declarations() -> Vec<FunctionDeclaration> {
vec![FunctionDeclaration {
name: format!("{RAG_FUNCTION_PREFIX}query"),
description: "Search the RAG knowledge base attached to this session and return \
the most relevant text chunks with their source paths. The relevant \
context has already been injected into the prompt up-front; use this \
tool to pull additional context on-demand when the initial retrieval \
does not fully answer the question. Prefer specific, keyword-rich queries."
.to_string(),
parameters: JsonSchema {
type_value: Some("object".to_string()),
properties: Some(IndexMap::from([
(
"query".to_string(),
JsonSchema {
type_value: Some("string".to_string()),
description: Some(
"Natural language search query used to retrieve relevant chunks."
.into(),
),
..Default::default()
},
),
(
"top_k".to_string(),
JsonSchema {
type_value: Some("integer".to_string()),
description: Some(
"Maximum number of chunks to return. Defaults to the RAG's \
configured top_k when omitted."
.into(),
),
..Default::default()
},
),
])),
required: Some(vec!["query".to_string()]),
..Default::default()
},
agent: false,
}]
}
pub async fn handle_rag_tool(
ctx: &mut RequestContext,
cmd_name: &str,
args: &Value,
) -> Result<Value> {
let action = cmd_name
.strip_prefix(RAG_FUNCTION_PREFIX)
.unwrap_or(cmd_name);
match action {
"query" => handle_query(ctx, args).await,
_ => Err(anyhow!("Unknown RAG action: {action}")),
}
}
async fn handle_query(ctx: &RequestContext, args: &Value) -> Result<Value> {
let rag = ctx
.rag
.clone()
.ok_or_else(|| anyhow!("No RAG is attached to this session"))?;
let query = args
.get("query")
.and_then(Value::as_str)
.ok_or_else(|| anyhow!("'query' is required"))?;
let top_k = args
.get("top_k")
.and_then(Value::as_u64)
.map(|v| v as usize)
.unwrap_or_else(|| rag.configured_top_k());
let rerank_model = rag.configured_reranker().map(|s| s.to_string());
let chunks = rag
.search_chunks(query, top_k, rerank_model.as_deref())
.await?;
let chunks_json: Vec<Value> = chunks
.into_iter()
.map(|(text, source)| json!({ "text": text, "source": source }))
.collect();
Ok(json!({
"rag_name": rag.name(),
"count": chunks_json.len(),
"chunks": chunks_json,
}))
}
+73 -6
View File
@@ -1,11 +1,13 @@
use super::{FunctionDeclaration, JsonSchema}; use super::{FunctionDeclaration, JsonSchema};
use crate::config::RequestContext; use crate::config::RequestContext;
use crate::supervisor::escalation::{EscalationRequest, new_escalation_id}; use crate::supervisor::escalation::{EscalationRequest, new_escalation_id};
use crate::utils::{ACP_SERVER, HEADLESS, queue_acp_permission};
use anyhow::{Result, anyhow, bail}; use anyhow::{Result, anyhow, bail};
use indexmap::IndexMap; use indexmap::IndexMap;
use inquire::{Confirm, MultiSelect, Select, Text}; use inquire::{Confirm, MultiSelect, Select, Text};
use serde_json::{Value, json}; use serde_json::{Value, json};
use std::sync::atomic::Ordering;
use std::time::Duration; use std::time::Duration;
use tokio::sync::oneshot; use tokio::sync::oneshot;
@@ -17,8 +19,11 @@ const CUSTOM_MULTI_CHOICE_ANSWER_OPTION: &str = "Other (custom)";
pub fn user_interaction_function_declarations() -> Vec<FunctionDeclaration> { pub fn user_interaction_function_declarations() -> Vec<FunctionDeclaration> {
vec![ vec![
FunctionDeclaration { FunctionDeclaration {
name: format!("{USER_FUNCTION_PREFIX}ask"), name: format!("{USER_FUNCTION_PREFIX}select"),
description: "Ask the user to select one option from a list. Returns the selected option. Indicate the recommended choice if there is one.".to_string(), description: "Present a list of named options and ask the user to pick exactly one. \
Indicate the recommended choice if there is one. \
Use this not `confirm` whenever there are 2+ named options to choose \
between. Returns the selected option.".to_string(),
parameters: JsonSchema { parameters: JsonSchema {
type_value: Some("object".to_string()), type_value: Some("object".to_string()),
properties: Some(IndexMap::from([ properties: Some(IndexMap::from([
@@ -50,7 +55,9 @@ pub fn user_interaction_function_declarations() -> Vec<FunctionDeclaration> {
}, },
FunctionDeclaration { FunctionDeclaration {
name: format!("{USER_FUNCTION_PREFIX}confirm"), name: format!("{USER_FUNCTION_PREFIX}confirm"),
description: "Ask the user a yes/no question. Returns \"yes\" or \"no\".".to_string(), description: "Ask a genuinely binary yes/no question with no other choices. Do NOT \
use for \"A or B?\" situations — use `select` instead. Returns \"yes\" \
or \"no\".".to_string(),
parameters: JsonSchema { parameters: JsonSchema {
type_value: Some("object".to_string()), type_value: Some("object".to_string()),
properties: Some(IndexMap::from([( properties: Some(IndexMap::from([(
@@ -68,7 +75,8 @@ pub fn user_interaction_function_declarations() -> Vec<FunctionDeclaration> {
}, },
FunctionDeclaration { FunctionDeclaration {
name: format!("{USER_FUNCTION_PREFIX}input"), name: format!("{USER_FUNCTION_PREFIX}input"),
description: "Ask the user for free-form text input. Returns the text entered.".to_string(), description: "Collect free-form text from the user when no predefined options exist. \
Returns the text entered.".to_string(),
parameters: JsonSchema { parameters: JsonSchema {
type_value: Some("object".to_string()), type_value: Some("object".to_string()),
properties: Some(IndexMap::from([( properties: Some(IndexMap::from([(
@@ -86,7 +94,9 @@ pub fn user_interaction_function_declarations() -> Vec<FunctionDeclaration> {
}, },
FunctionDeclaration { FunctionDeclaration {
name: format!("{USER_FUNCTION_PREFIX}checkbox"), name: format!("{USER_FUNCTION_PREFIX}checkbox"),
description: "Ask the user to select one or more options from a list. Returns an array of selected options.".to_string(), description: "Ask the user to pick one or more options from a list (multi-select). \
Use when multiple answers are valid simultaneously. Returns an array \
of selected options.".to_string(),
parameters: JsonSchema { parameters: JsonSchema {
type_value: Some("object".to_string()), type_value: Some("object".to_string()),
properties: Some(IndexMap::from([ properties: Some(IndexMap::from([
@@ -128,6 +138,21 @@ pub async fn handle_user_tool(
.strip_prefix(USER_FUNCTION_PREFIX) .strip_prefix(USER_FUNCTION_PREFIX)
.unwrap_or(cmd_name); .unwrap_or(cmd_name);
if ACP_SERVER.load(Ordering::SeqCst) {
let result = handle_headless(action, args);
queue_acp_permission(json!({
"action": action,
"question": result["question"],
"options": result["options"],
}));
return Ok(result);
}
if HEADLESS.load(Ordering::SeqCst) {
return Ok(handle_headless(action, args));
}
let depth = ctx.current_depth; let depth = ctx.current_depth;
if depth == 0 { if depth == 0 {
@@ -137,9 +162,26 @@ pub async fn handle_user_tool(
} }
} }
fn handle_headless(action: &str, args: &Value) -> Value {
let question = args.get("question").and_then(Value::as_str).unwrap_or("");
let options: Vec<Value> = args
.get("options")
.and_then(Value::as_array)
.cloned()
.unwrap_or_default();
json!({
"needs_human": true,
"action": action,
"question": question,
"options": options,
"guidance": "No human is present. Apply a sensible default or abort the task.",
})
}
fn handle_direct(action: &str, args: &Value) -> Result<Value> { fn handle_direct(action: &str, args: &Value) -> Result<Value> {
match action { match action {
"ask" => handle_direct_ask(args), "select" => handle_direct_ask(args),
"confirm" => handle_direct_confirm(args), "confirm" => handle_direct_confirm(args),
"input" => handle_direct_input(args), "input" => handle_direct_input(args),
"checkbox" => handle_direct_checkbox(args), "checkbox" => handle_direct_checkbox(args),
@@ -285,3 +327,28 @@ fn parse_options(args: &Value) -> Result<Vec<String>> {
.map(String::from) .map(String::from)
.collect()) .collect())
} }
#[cfg(test)]
mod tests {
use super::*;
#[test]
fn headless_select_returns_structured_json() {
let args = json!({"question": "pick one", "options": ["a", "b"]});
let v = handle_headless("select", &args);
assert_eq!(v["needs_human"], true);
assert_eq!(v["action"], "select");
assert_eq!(v["question"], "pick one");
assert_eq!(v["options"], json!(["a", "b"]));
assert!(v["guidance"].is_string());
}
#[test]
fn headless_confirm_returns_empty_options_when_absent() {
let args = json!({"question": "yes or no?"});
let v = handle_headless("confirm", &args);
assert_eq!(v["needs_human"], true);
assert_eq!(v["action"], "confirm");
assert_eq!(v["options"], json!([]));
}
}
+44 -2
View File
@@ -88,11 +88,21 @@ async fn run_one_shot(prompt: &str, ctx: &mut RequestContext) -> Result<String>
} }
fn try_parse_json(raw: &str) -> Option<Value> { fn try_parse_json(raw: &str) -> Option<Value> {
let cleaned = strip_code_fences(raw.trim()); let cleaned = strip_code_fences(strip_thinking_blocks(raw.trim()));
serde_json::from_str(cleaned).ok() serde_json::from_str(cleaned).ok()
} }
fn strip_thinking_blocks(s: &str) -> &str {
let mut s = s.trim_start();
while s.starts_with("<think>") {
match s.find("</think>") {
Some(end) => s = s[end + "</think>".len()..].trim_start(),
None => break,
}
}
s
}
fn strip_code_fences(s: &str) -> &str { fn strip_code_fences(s: &str) -> &str {
let after_open = s let after_open = s
.strip_prefix("```json") .strip_prefix("```json")
@@ -148,6 +158,38 @@ mod tests {
assert_eq!(v, json!({"x": true})); assert_eq!(v, json!({"x": true}));
} }
#[test]
fn try_parse_json_strips_thinking_blocks() {
let raw = "<think>\nsome reasoning\n</think>\n{\"a\": 1}";
let v = try_parse_json(raw).unwrap();
assert_eq!(v, json!({"a": 1}));
}
#[test]
fn try_parse_json_strips_empty_thinking_block() {
let raw = "<think>\n\n</think>\n{\"a\": 1}";
let v = try_parse_json(raw).unwrap();
assert_eq!(v, json!({"a": 1}));
}
#[test]
fn try_parse_json_strips_multiple_thinking_blocks() {
let raw = "<think>first</think>\n<think>second</think>\n{\"a\": 1}";
let v = try_parse_json(raw).unwrap();
assert_eq!(v, json!({"a": 1}));
}
#[test]
fn try_parse_json_unclosed_think_tag_returns_none() {
assert!(try_parse_json("<think>unclosed {\"a\": 1}").is_none());
}
#[test] #[test]
fn try_parse_json_returns_none_on_prose() { fn try_parse_json_returns_none_on_prose() {
assert!(try_parse_json("Here is the result: it's good").is_none()); assert!(try_parse_json("Here is the result: it's good").is_none());
+103
View File
@@ -367,6 +367,13 @@ pub struct RagNode {
#[serde(default, skip_serializing_if = "Option::is_none")] #[serde(default, skip_serializing_if = "Option::is_none")]
pub graph_hops: Option<usize>, pub graph_hops: Option<usize>,
/// Storage driver for this node's knowledge base ("yaml", "duckdb"). `None`
/// means "yaml". Only honored when the knowledge base is first built;
/// changing it afterwards has no effect until the RAG is deleted and
/// re-initialized.
#[serde(default, skip_serializing_if = "Option::is_none")]
pub driver: Option<String>,
#[serde(default, skip_serializing_if = "Option::is_none")] #[serde(default, skip_serializing_if = "Option::is_none")]
pub state_updates: Option<HashMap<String, String>>, pub state_updates: Option<HashMap<String, String>>,
@@ -1152,4 +1159,100 @@ nodes:
assert!(triage.next.as_ref().unwrap().is_fan_out()); assert!(triage.next.as_ref().unwrap().is_fan_out());
assert_eq!(triage.next.as_ref().unwrap().as_slice().len(), 2); assert_eq!(triage.next.as_ref().unwrap().as_slice().len(), 2);
} }
fn rag_node_of(graph: &Graph, id: &str) -> RagNode {
match &graph.get_node(id).unwrap().node_type {
NodeType::Rag(r) => r.clone(),
other => panic!("expected a rag node, got {other:?}"),
}
}
#[test]
fn rag_node_deserializes_an_explicit_driver() {
let yaml = r#"
name: kb
start: research
nodes:
research:
type: rag
documents: ["./docs"]
driver: duckdb
next: done
done:
type: end
output: ok
"#;
let graph: Graph = serde_yaml::from_str(yaml).unwrap();
assert_eq!(
rag_node_of(&graph, "research").driver.as_deref(),
Some("duckdb")
);
}
/// Workflows written before drivers existed must keep parsing, and must keep
/// asking for nothing, so `RagInitConfig` resolves them to the yaml default.
#[test]
fn rag_node_without_a_driver_stays_unset() {
let yaml = r#"
name: kb
start: research
nodes:
research:
type: rag
documents: ["./docs"]
next: done
done:
type: end
output: ok
"#;
let graph: Graph = serde_yaml::from_str(yaml).unwrap();
assert_eq!(rag_node_of(&graph, "research").driver, None);
}
#[test]
fn rag_node_driver_survives_a_serialize_round_trip() {
let yaml = r#"
name: kb
start: research
nodes:
research:
type: rag
documents: ["./docs"]
driver: duckdb
next: done
done:
type: end
output: ok
"#;
let graph: Graph = serde_yaml::from_str(yaml).unwrap();
let reparsed: Graph =
serde_yaml::from_str(&serde_yaml::to_string(&graph).unwrap()).unwrap();
assert_eq!(
rag_node_of(&reparsed, "research").driver.as_deref(),
Some("duckdb")
);
}
/// `skip_serializing_if` must keep `driver:` out of graphs that never set it.
#[test]
fn rag_node_without_a_driver_omits_the_key_when_serialized() {
let yaml = r#"
name: kb
start: research
nodes:
research:
type: rag
documents: ["./docs"]
next: done
done:
type: end
output: ok
"#;
let graph: Graph = serde_yaml::from_str(yaml).unwrap();
assert!(!serde_yaml::to_string(&graph).unwrap().contains("driver"));
}
} }
+1 -1
View File
@@ -27,7 +27,7 @@ impl ApprovalNodeExecutor {
&json!({ "question": question, "options": node.options }), &json!({ "question": question, "options": node.options }),
) )
.await .await
.context("user__ask failed")?; .context("user__select failed")?;
if let Some(err) = response.get("error").and_then(Value::as_str) { if let Some(err) = response.get("error").and_then(Value::as_str) {
bail!("Approval interaction failed: {err}"); bail!("Approval interaction failed: {err}");
+101
View File
@@ -2,6 +2,7 @@ use super::state::template_root_keys;
use super::types::{Graph, Node, NodeType}; use super::types::{Graph, Node, NodeType};
use crate::client::{Model, ModelType}; use crate::client::{Model, ModelType};
use crate::config::{Agent, AppConfig, paths}; use crate::config::{Agent, AppConfig, paths};
use crate::rag::{GraphRagConfig, RagData};
use anyhow::{Result, bail}; use anyhow::{Result, bail};
use std::collections::{BTreeMap, HashSet, VecDeque}; use std::collections::{BTreeMap, HashSet, VecDeque};
use std::path::PathBuf; use std::path::PathBuf;
@@ -96,6 +97,51 @@ pub struct GraphValidator {
skill_exists: fn(&str) -> bool, skill_exists: fn(&str) -> bool,
} }
/// A minimal `RagData` whose only interesting field is `driver`. The numeric
/// arguments are the smallest values that satisfy `validate()`'s unrelated
/// floors (top_k >= 1, and chunk_size >= 1 with chunk_overlap < chunk_size for
/// a non-attached RAG). `RagData::new` sets `attached: false`, which is the
/// correct shape here: a graph rag node always builds its own local knowledge
/// base from `documents` and can never be attached.
fn rag_driver_probe(driver: &str) -> RagData {
let mut data = RagData::new(
String::new(),
1,
0,
None,
1,
None,
GraphRagConfig::default(),
);
data.driver = driver.to_string();
data
}
/// `Some(message)` when `driver` is one that `RagData::validate()` would reject.
///
/// The set of valid drivers is defined in exactly one place, `RagData::validate()`,
/// so this asks that function rather than restating the list here.
///
/// Fails open on purpose: the first probe below uses the default driver, which is
/// valid by definition. If even that one is rejected, `validate()` has grown a
/// precondition the probe fixture no longer satisfies, and every verdict from here
/// would be a false positive that rejects working graphs. In that case we decline
/// to judge and leave enforcement to RAG construction. The
/// `rag_driver_probe_fixture_is_accepted` test turns that silent degradation into a
/// loud failure. Both `validate()` calls are load-bearing; neither is redundant.
pub(crate) fn rag_driver_error(driver: &str) -> Option<String> {
if rag_driver_probe(&RagData::default().driver)
.validate()
.is_err()
{
return None;
}
rag_driver_probe(driver)
.validate()
.err()
.map(|err| err.to_string())
}
impl GraphValidator { impl GraphValidator {
pub fn new(base_dir: impl Into<PathBuf>) -> Self { pub fn new(base_dir: impl Into<PathBuf>) -> Self {
Self { Self {
@@ -154,6 +200,11 @@ impl GraphValidator {
not be written to state", not be written to state",
)); ));
} }
if let Some(driver) = &r.driver
&& let Some(message) = rag_driver_error(driver)
{
result.error(ValidationError::with_node(node_id, message));
}
} }
} }
} }
@@ -1031,6 +1082,7 @@ mod tests {
extractor_model: None, extractor_model: None,
extractor_prompt: None, extractor_prompt: None,
graph_hops: None, graph_hops: None,
driver: None,
state_updates, state_updates,
timeout: None, timeout: None,
}), }),
@@ -1385,6 +1437,55 @@ mod tests {
); );
} }
/// Guards the fail-open branch in `rag_driver_error`. If this fails,
/// `RagData::validate()` grew a precondition the probe fixture no longer
/// satisfies and rag-node driver validation has silently switched itself off.
/// Repair the fixture in `rag_driver_probe`; do not delete this test.
#[test]
fn rag_driver_probe_fixture_is_accepted() {
let default_driver = RagData::default().driver;
assert!(
rag_driver_probe(&default_driver).validate().is_ok(),
"probe fixture rejected for the default driver '{default_driver}'"
);
}
#[test]
fn rag_driver_error_defers_to_ragdata_validate() {
assert_eq!(rag_driver_error("yaml"), None);
assert_eq!(rag_driver_error("duckdb"), None);
let message = rag_driver_error("duckdbb").expect("unknown driver must be rejected");
assert!(message.contains("duckdbb"), "got: {message}");
}
#[test]
fn rag_node_with_unknown_driver_errors_naming_the_node() {
let mut node = rag_node("kb", &["./docs"], true);
if let NodeType::Rag(ref mut r) = node.node_type {
r.driver = Some("postgres".into());
}
let graph = graph_with(vec![("kb", node), ("end", end_node("end"))], "kb");
let result = validator().validate(&graph);
assert!(!result.is_valid());
let err = result.into_result().unwrap_err().to_string();
assert!(err.contains("[kb]"), "must name the node: {err}");
assert!(err.contains("postgres"), "must name the driver: {err}");
}
#[test]
fn rag_node_with_duckdb_driver_produces_no_findings() {
let mut node = rag_node("kb", &["./docs"], true);
if let NodeType::Rag(ref mut r) = node.node_type {
r.driver = Some("duckdb".into());
}
let graph = graph_with(vec![("kb", node), ("end", end_node("end"))], "kb");
assert!(validator().validate(&graph).is_valid());
}
fn agent_node(id: &str, agent: &str, next: Option<&str>) -> Node { fn agent_node(id: &str, agent: &str, next: Option<&str>) -> Node {
Node { Node {
id: id.into(), id: id.into(),
+52 -9
View File
@@ -1,3 +1,4 @@
mod acp;
mod cli; mod cli;
mod client; mod client;
mod config; mod config;
@@ -24,7 +25,7 @@ use crate::client::{
use crate::config::instructions::WORKSPACE_INSTRUCTIONS_FILE_NAME; use crate::config::instructions::WORKSPACE_INSTRUCTIONS_FILE_NAME;
use crate::config::{ use crate::config::{
Agent, AppConfig, AppState, CODE_ROLE, Config, EXPLAIN_SHELL_ROLE, Input, MemoryScope, Agent, AppConfig, AppState, CODE_ROLE, Config, EXPLAIN_SHELL_ROLE, Input, MemoryScope,
RequestContext, SHELL_ROLE, TEMP_SESSION_NAME, WorkingMode, ensure_parent_exists, RenderMode, RequestContext, SHELL_ROLE, TEMP_SESSION_NAME, WorkingMode, ensure_parent_exists,
install_builtins, list_agents, load_env_file, macro_execute, sync_models, install_builtins, list_agents, load_env_file, macro_execute, sync_models,
}; };
use crate::config::{memory, paths}; use crate::config::{memory, paths};
@@ -40,7 +41,7 @@ use clap_complete::CompleteEnv;
use client::ClientConfig; use client::ClientConfig;
use inquire::{Select, Text, set_global_render_config}; use inquire::{Select, Text, set_global_render_config};
use log::{LevelFilter, warn}; use log::{LevelFilter, warn};
use log4rs::append::console::ConsoleAppender; use log4rs::append::console::{ConsoleAppender, Target};
use log4rs::append::rolling_file::RollingFileAppender; use log4rs::append::rolling_file::RollingFileAppender;
use log4rs::append::rolling_file::policy::compound::CompoundPolicy; use log4rs::append::rolling_file::policy::compound::CompoundPolicy;
use log4rs::append::rolling_file::policy::compound::roll::fixed_window::FixedWindowRoller; use log4rs::append::rolling_file::policy::compound::roll::fixed_window::FixedWindowRoller;
@@ -49,6 +50,7 @@ use log4rs::config::{Appender, Logger, Root};
use log4rs::encode::pattern::PatternEncoder; use log4rs::encode::pattern::PatternEncoder;
use oauth::OAuthProvider; use oauth::OAuthProvider;
use std::path::PathBuf; use std::path::PathBuf;
use std::sync::atomic::Ordering;
use std::{env, fs, process, sync::Arc}; use std::{env, fs, process, sync::Arc};
#[tokio::main] #[tokio::main]
@@ -74,13 +76,29 @@ async fn main() -> Result<()> {
return Ok(()); return Ok(());
} }
let text = cli.text()?; let text = if cli.acp_server { None } else { cli.text()? };
let working_mode = if text.is_none() && cli.file.is_empty() { let working_mode = if !cli.acp_server && text.is_none() && cli.file.is_empty() {
WorkingMode::Repl WorkingMode::Repl
} else { } else {
WorkingMode::Cmd WorkingMode::Cmd
}; };
if cli.headless || cli.acp_server {
if cli.headless && !cli.acp_server && text.is_none() && cli.file.is_empty() {
bail!("--headless requires a prompt argument; REPL mode is not supported");
}
unsafe {
env::set_var("AUTO_CONFIRM", "true");
}
HEADLESS.store(true, Ordering::SeqCst);
if cli.acp_server {
ACP_SERVER.store(true, Ordering::SeqCst);
}
}
let info_flag = cli.info let info_flag = cli.info
|| cli.sync_models || cli.sync_models
|| cli.list_models || cli.list_models
@@ -96,7 +114,7 @@ async fn main() -> Result<()> {
|| cli.delete_secret.is_some() || cli.delete_secret.is_some()
|| cli.list_secrets; || cli.list_secrets;
let log_path = setup_logger()?; let log_path = setup_logger(cli.acp_server)?;
if let Some(version) = &cli.update { if let Some(version) = &cli.update {
let version = version.clone(); let version = version.clone();
@@ -106,7 +124,7 @@ async fn main() -> Result<()> {
} }
if let Some(name) = &cli.sandbox { if let Some(name) = &cli.sandbox {
return sandbox::launch(name.clone(), cli.fresh, cli.no_mixins); return sandbox::launch(name.clone(), cli.fresh);
} }
install_builtins()?; install_builtins()?;
@@ -178,6 +196,18 @@ async fn main() -> Result<()> {
return Ok(()); return Ok(());
} }
let mcp_action =
cli.mcp_list || cli.mcp_get.is_some() || cli.mcp_remove.is_some() || cli.mcp_add.is_some();
if mcp_action {
let cfg = Config::load_with_interpolation(true).await?;
let app_config = AppConfig::from_config(cfg)?;
let vault = Vault::init(&app_config)?;
mcp::manage::handle(&cli, &vault)?;
return Ok(());
}
if vault_flags { if vault_flags {
let cfg = Config::load_with_interpolation(true).await?; let cfg = Config::load_with_interpolation(true).await?;
let app_config = AppConfig::from_config(cfg)?; let app_config = AppConfig::from_config(cfg)?;
@@ -210,7 +240,11 @@ async fn main() -> Result<()> {
{ {
let app = &*ctx.app.config; let app = &*ctx.app.config;
if app.highlight { if app.highlight {
set_global_render_config(prompt_theme(app.render_options()?)?) let render_opts = app.render_options()?;
if let Some(ref theme) = render_opts.theme {
utils::init_tool_colors(theme);
}
set_global_render_config(prompt_theme(render_opts)?)
} }
} }
@@ -475,6 +509,11 @@ async fn run(
ctx.load_skill_repl(name, abort_signal.clone()).await?; ctx.load_skill_repl(name, abort_signal.clone()).await?;
} }
if cli.acp_server {
ctx.render_mode = RenderMode::Silent;
return acp::run_acp_server(ctx, abort_signal).await;
}
match is_repl { match is_repl {
false => { false => {
let mut input = create_input(&ctx, text, &cli.file, abort_signal.clone()).await?; let mut input = create_input(&ctx, text, &cli.file, abort_signal.clone()).await?;
@@ -675,7 +714,7 @@ async fn create_input(
Ok(input) Ok(input)
} }
fn setup_logger() -> Result<Option<PathBuf>> { fn setup_logger(acp_mode: bool) -> Result<Option<PathBuf>> {
let (log_level, log_path) = paths::log_config()?; let (log_level, log_path) = paths::log_config()?;
if log_level == LevelFilter::Off { if log_level == LevelFilter::Off {
return Ok(None); return Ok(None);
@@ -686,7 +725,11 @@ fn setup_logger() -> Result<Option<PathBuf>> {
let log_filter = env::var(get_env_name("log_filter")).ok(); let log_filter = env::var(get_env_name("log_filter")).ok();
match log_path.clone() { match log_path.clone() {
None => { None => {
let console_appender = ConsoleAppender::builder().encoder(encoder).build(); let mut builder = ConsoleAppender::builder().encoder(encoder);
if acp_mode {
builder = builder.target(Target::Stderr);
}
let console_appender = builder.build();
log4rs::init_config(init_console_logger(log_level, log_filter, console_appender))?; log4rs::init_config(init_console_logger(log_level, log_filter, console_appender))?;
} }
Some(path) => { Some(path) => {
+443
View File
@@ -0,0 +1,443 @@
use crate::cli::{Cli, McpScopeArg, McpTransportArg};
use crate::config::{ensure_parent_exists, paths};
use crate::mcp::{JsonField, McpOAuthConfig, McpServer, McpServersConfig, McpTransportType};
use crate::vault::{SECRET_RE, Vault};
use anyhow::{Context, Result, anyhow, bail};
use indexmap::{IndexMap, IndexSet};
use inquire::Confirm;
use std::collections::HashSet;
use std::fs;
use std::path::{Path, PathBuf};
impl From<McpTransportArg> for McpTransportType {
fn from(value: McpTransportArg) -> Self {
match value {
McpTransportArg::Stdio => McpTransportType::Stdio,
McpTransportArg::Http => McpTransportType::Http,
McpTransportArg::Sse => McpTransportType::Sse,
}
}
}
pub fn handle(cli: &Cli, vault: &Vault) -> Result<()> {
if cli.mcp_list {
return handle_list(cli.scope);
}
if let Some(name) = &cli.mcp_get {
return handle_get(name, cli.scope);
}
if let Some(name) = &cli.mcp_remove {
return handle_remove(name, cli.scope, cli.mcp_force);
}
if let Some(name) = &cli.mcp_add {
return handle_add(cli, name, vault);
}
Ok(())
}
fn handle_list(scope: Option<McpScopeArg>) -> Result<()> {
let show_user = scope != Some(McpScopeArg::Workspace);
let show_workspace = scope != Some(McpScopeArg::User);
if show_user {
let user_path = paths::mcp_config_file();
let user_cfg = load_config_raw(&user_path)?;
println!("User ({})", user_path.display());
print_server_list(&user_cfg);
}
if show_workspace {
match paths::workspace_mcp_config_file() {
Some(ws_path) => {
let ws_cfg = load_config_raw(&ws_path)?;
if show_user {
println!();
}
println!("Workspace ({})", ws_path.display());
print_server_list(&ws_cfg);
}
None if scope == Some(McpScopeArg::Workspace) => {
println!("Workspace: no mcp.json found in current directory");
}
None => {}
}
}
Ok(())
}
fn print_server_list(cfg: &McpServersConfig) {
if cfg.mcp_servers.is_empty() {
println!(" (none)");
return;
}
let name_width = cfg.mcp_servers.keys().map(String::len).max().unwrap_or(0);
for (name, spec) in &cfg.mcp_servers {
let transport = match spec.transport_type {
McpTransportType::Stdio => "stdio",
McpTransportType::Http => "http",
McpTransportType::Sse => "sse",
};
let target = spec.url.clone().unwrap_or_else(|| {
let cmd = spec.command.clone().unwrap_or_default();
let args = spec.args.as_ref().map(|a| a.join(" ")).unwrap_or_default();
if args.is_empty() {
cmd
} else {
format!("{cmd} {args}")
}
});
println!(
" {name:<name_width$} {transport:<5} {target}",
name_width = name_width
);
}
}
fn handle_get(name: &str, scope: Option<McpScopeArg>) -> Result<()> {
let (path, cfg) = load_for_scope_or_search(name, scope)?;
let spec = cfg
.mcp_servers
.get(name)
.ok_or_else(|| anyhow!("MCP server '{name}' not found"))?;
let pretty =
serde_json::to_string_pretty(spec).context("failed to serialize MCP server config")?;
println!("# {}", path.display());
println!("{pretty}");
Ok(())
}
fn handle_remove(name: &str, scope: Option<McpScopeArg>, force: bool) -> Result<()> {
let (path, mut cfg) = load_for_scope_or_search(name, scope)?;
if !force {
let ok = Confirm::new(&format!(
"Remove MCP server '{name}' from {}?",
path.display()
))
.with_default(false)
.prompt()?;
if !ok {
println!("Aborted.");
return Ok(());
}
}
cfg.mcp_servers.shift_remove(name);
save_config(&path, &cfg)?;
println!("✓ Removed MCP server '{name}' from {}", path.display());
Ok(())
}
fn handle_add(cli: &Cli, name: &str, vault: &Vault) -> Result<()> {
validate_name(name)?;
let server = build_server(cli)?;
server.validate(name)?;
let scope = cli.scope.unwrap_or_default();
let path = write_path_for_scope(scope);
let mut cfg = load_config_raw(&path)?;
if cfg.mcp_servers.contains_key(name) && !cli.mcp_force {
let ok = Confirm::new(&format!(
"MCP server '{name}' already exists in {}. Overwrite?",
path.display()
))
.with_default(false)
.prompt()?;
if !ok {
println!("Aborted. Use --mcp-force to overwrite without prompting.");
return Ok(());
}
}
provision_secrets(cli, vault)?;
cfg.mcp_servers.insert(name.to_string(), server);
save_config(&path, &cfg)?;
println!("✓ Added MCP server '{name}' to {}", path.display());
Ok(())
}
fn validate_name(name: &str) -> Result<()> {
if name.is_empty() {
bail!("MCP server name cannot be empty");
}
if !name
.chars()
.all(|c| c.is_ascii_alphanumeric() || c == '-' || c == '_')
{
bail!("Invalid MCP server name '{name}': only letters, digits, '-', and '_' are allowed");
}
Ok(())
}
fn build_server(cli: &Cli) -> Result<McpServer> {
let has_command = !cli.mcp_command.is_empty();
let has_url = cli.url.is_some();
let transport = cli
.transport
.map(McpTransportType::from)
.unwrap_or_else(|| {
if has_command {
McpTransportType::Stdio
} else {
McpTransportType::Http
}
});
match transport {
McpTransportType::Stdio => build_stdio(cli, has_url),
McpTransportType::Http | McpTransportType::Sse => build_remote(cli, transport, has_command),
}
}
fn build_stdio(cli: &Cli, has_url: bool) -> Result<McpServer> {
if cli.mcp_command.is_empty() {
bail!(
"stdio MCP server requires a command. Pass it after `--`, e.g. \
`--mcp-add NAME -- npx some-server --flag`"
);
}
if has_url {
bail!("stdio MCP server does not accept --url");
}
if !cli.header.is_empty() {
bail!("stdio MCP server does not accept --header");
}
if cli.client_id.is_some()
|| cli.client_secret.is_some()
|| cli.callback_port.is_some()
|| cli.redirect_host.is_some()
{
bail!("stdio MCP server does not accept OAuth flags");
}
let (cmd, args) = cli.mcp_command.split_first().unwrap();
let mut env: IndexMap<String, JsonField> = IndexMap::new();
for kv in &cli.env {
let (k, v) = kv
.split_once('=')
.ok_or_else(|| anyhow!("invalid --env value '{kv}': expected KEY=VALUE"))?;
if k.is_empty() {
bail!("invalid --env value '{kv}': KEY cannot be empty");
}
env.insert(k.to_string(), JsonField::Str(v.to_string()));
}
Ok(McpServer {
transport_type: McpTransportType::Stdio,
command: Some(cmd.clone()),
args: (!args.is_empty()).then(|| args.to_vec()),
env: (!env.is_empty()).then_some(env),
cwd: cli.cwd.clone(),
url: None,
headers: None,
oauth: None,
})
}
fn build_remote(cli: &Cli, transport: McpTransportType, has_command: bool) -> Result<McpServer> {
if has_command {
bail!(
"http/sse MCP server does not accept a trailing `-- <cmd>`. Use `--url` \
to specify the endpoint."
);
}
let url = cli
.url
.clone()
.ok_or_else(|| anyhow!("http/sse MCP server requires --url <URL>"))?;
if !cli.env.is_empty() {
bail!("http/sse MCP server does not accept --env; use --header instead");
}
if cli.cwd.is_some() {
bail!("http/sse MCP server does not accept --cwd");
}
let mut headers: IndexMap<String, String> = IndexMap::new();
for h in &cli.header {
let (name, value) = h
.split_once(':')
.ok_or_else(|| anyhow!("invalid --header value '{h}': expected 'Name: Value'"))?;
let name = name.trim();
let value = value.trim_start_matches(' ');
if name.is_empty() {
bail!("invalid --header value '{h}': header name cannot be empty");
}
headers.insert(name.to_string(), value.to_string());
}
let oauth = if cli.client_id.is_some()
|| cli.client_secret.is_some()
|| cli.callback_port.is_some()
|| cli.redirect_host.is_some()
{
Some(McpOAuthConfig {
client_id: cli.client_id.clone(),
client_secret: cli.client_secret.clone(),
callback_port: cli.callback_port,
redirect_host: cli.redirect_host.clone(),
})
} else {
None
};
Ok(McpServer {
transport_type: transport,
command: None,
args: None,
env: None,
cwd: None,
url: Some(url),
headers: (!headers.is_empty()).then_some(headers),
oauth,
})
}
fn provision_secrets(cli: &Cli, vault: &Vault) -> Result<()> {
let mut sources: Vec<&str> = Vec::new();
if let Some(s) = cli.url.as_deref() {
sources.push(s);
}
if let Some(s) = cli.client_secret.as_deref() {
sources.push(s);
}
if let Some(s) = cli.client_id.as_deref() {
sources.push(s);
}
if let Some(s) = cli.redirect_host.as_deref() {
sources.push(s);
}
if let Some(s) = cli.cwd.as_deref() {
sources.push(s);
}
sources.extend(cli.env.iter().map(String::as_str));
sources.extend(cli.header.iter().map(String::as_str));
let mut needed: IndexSet<String> = IndexSet::new();
for value in sources {
for caps in SECRET_RE.captures_iter(value).filter_map(Result::ok) {
if let Some(m) = caps.get(1) {
needed.insert(m.as_str().trim().to_string());
}
}
}
if needed.is_empty() {
return Ok(());
}
let existing: HashSet<String> = vault.list_secrets(false)?.into_iter().collect();
for name in needed {
if existing.contains(&name) {
continue;
}
eprintln!("Value references vault secret {{{{ {name} }}}} which is not stored yet.");
let ok = Confirm::new(&format!("Add '{name}' to the vault now?"))
.with_default(true)
.prompt()?;
if !ok {
bail!(
"Vault secret '{name}' is required by the config; aborting. \
Add it later with `coyote --add-secret {name}`."
);
}
vault.add_secret(&name)?;
}
Ok(())
}
fn load_for_scope_or_search(
name: &str,
scope: Option<McpScopeArg>,
) -> Result<(PathBuf, McpServersConfig)> {
if let Some(s) = scope {
let path = match s {
McpScopeArg::User => paths::mcp_config_file(),
McpScopeArg::Workspace => paths::workspace_mcp_config_file()
.ok_or_else(|| anyhow!("no workspace mcp.json found in the current directory"))?,
};
let cfg = load_config_raw(&path)?;
if !cfg.mcp_servers.contains_key(name) {
bail!(
"MCP server '{name}' not found in {} scope ({})",
scope_label(s),
path.display()
);
}
return Ok((path, cfg));
}
let user_path = paths::mcp_config_file();
let user_cfg = load_config_raw(&user_path)?;
if user_cfg.mcp_servers.contains_key(name) {
return Ok((user_path, user_cfg));
}
if let Some(ws_path) = paths::workspace_mcp_config_file() {
let ws_cfg = load_config_raw(&ws_path)?;
if ws_cfg.mcp_servers.contains_key(name) {
return Ok((ws_path, ws_cfg));
}
}
bail!("MCP server '{name}' not found in any scope");
}
fn write_path_for_scope(scope: McpScopeArg) -> PathBuf {
match scope {
McpScopeArg::User => paths::mcp_config_file(),
McpScopeArg::Workspace => paths::workspace_mcp_config_file()
.unwrap_or_else(|| paths::workspace_config_dir().join("mcp.json")),
}
}
fn scope_label(scope: McpScopeArg) -> &'static str {
match scope {
McpScopeArg::User => "user",
McpScopeArg::Workspace => "workspace",
}
}
fn load_config_raw(path: &Path) -> Result<McpServersConfig> {
if !path.exists() {
return Ok(McpServersConfig {
mcp_servers: IndexMap::new(),
});
}
let raw = fs::read_to_string(path)
.with_context(|| format!("failed to read MCP config at {}", path.display()))?;
if raw.trim().is_empty() {
return Ok(McpServersConfig {
mcp_servers: IndexMap::new(),
});
}
serde_json::from_str(&raw)
.with_context(|| format!("failed to parse MCP config at {}", path.display()))
}
fn save_config(path: &Path, config: &McpServersConfig) -> Result<()> {
ensure_parent_exists(path)?;
let serialized =
serde_json::to_string_pretty(config).context("failed to serialize MCP config")?;
let tmp = path.with_extension("json.tmp");
fs::write(&tmp, &serialized)
.with_context(|| format!("failed to write temporary MCP config at {}", tmp.display()))?;
fs::rename(&tmp, path)
.with_context(|| format!("failed to finalize MCP config at {}", path.display()))?;
Ok(())
}
+3
View File
@@ -1,3 +1,4 @@
pub(crate) mod manage;
pub(crate) mod oauth; pub(crate) mod oauth;
mod sse_transport; mod sse_transport;
@@ -62,6 +63,8 @@ pub(crate) struct McpServersConfig {
pub(crate) struct McpOAuthConfig { pub(crate) struct McpOAuthConfig {
#[serde(rename = "clientId", skip_serializing_if = "Option::is_none")] #[serde(rename = "clientId", skip_serializing_if = "Option::is_none")]
pub client_id: Option<String>, pub client_id: Option<String>,
#[serde(rename = "clientSecret", skip_serializing_if = "Option::is_none")]
pub client_secret: Option<String>,
#[serde(rename = "callbackPort", skip_serializing_if = "Option::is_none")] #[serde(rename = "callbackPort", skip_serializing_if = "Option::is_none")]
pub callback_port: Option<u16>, pub callback_port: Option<u16>,
#[serde(rename = "redirectHost", skip_serializing_if = "Option::is_none")] #[serde(rename = "redirectHost", skip_serializing_if = "Option::is_none")]
+322 -44
View File
@@ -12,6 +12,8 @@ use url::Url;
#[derive(Debug, Deserialize)] #[derive(Debug, Deserialize)]
struct ProtectedResourceMetadata { struct ProtectedResourceMetadata {
#[serde(default)]
resource: Option<String>,
#[serde(default)] #[serde(default)]
authorization_servers: Vec<String>, authorization_servers: Vec<String>,
#[serde(default)] #[serde(default)]
@@ -30,6 +32,13 @@ struct OAuthServerMetadata {
#[derive(Serialize, Deserialize)] #[derive(Serialize, Deserialize)]
struct McpRegistration { struct McpRegistration {
client_id: String, client_id: String,
#[serde(default)]
redirect_uri: Option<String>,
}
struct DiscoveredOAuth {
metadata: OAuthServerMetadata,
resource: Option<String>,
} }
struct McpOAuthProvider { struct McpOAuthProvider {
@@ -38,6 +47,7 @@ struct McpOAuthProvider {
token_url: String, token_url: String,
scopes: String, scopes: String,
fixed_redirect: String, fixed_redirect: String,
resource: String,
} }
impl OAuthProvider for McpOAuthProvider { impl OAuthProvider for McpOAuthProvider {
@@ -76,6 +86,14 @@ impl OAuthProvider for McpOAuthProvider {
fn fixed_redirect_uri(&self) -> Option<String> { fn fixed_redirect_uri(&self) -> Option<String> {
Some(self.fixed_redirect.clone()) Some(self.fixed_redirect.clone())
} }
fn extra_authorize_params(&self) -> Vec<(&str, &str)> {
vec![("resource", self.resource.as_str())]
}
fn extra_token_params(&self) -> Vec<(&str, &str)> {
vec![("resource", self.resource.as_str())]
}
} }
pub async fn run_mcp_oauth_flow( pub async fn run_mcp_oauth_flow(
@@ -85,36 +103,57 @@ pub async fn run_mcp_oauth_flow(
callback_port: Option<u16>, callback_port: Option<u16>,
redirect_host: Option<&str>, redirect_host: Option<&str>,
) -> Result<()> { ) -> Result<()> {
let metadata = discover_oauth_metadata(server_url).await?; let discovered = discover_oauth_metadata(server_url).await?;
let metadata = discovered.metadata;
let resource = resolve_resource(discovered.resource, server_url)?;
let host = redirect_host.unwrap_or("127.0.0.1"); let host = redirect_host.unwrap_or("127.0.0.1");
let bind_addr = format!("127.0.0.1:{}", callback_port.unwrap_or(0));
let listener = TcpListener::bind(&bind_addr)?;
let port = listener.local_addr()?.port();
drop(listener);
let redirect_uri = format!("http://{host}:{port}/callback");
let client_id = if let Some(id) = configured_client_id { // Reuse a cached dynamic registration together with the exact redirect
id.to_string() // URI it was registered with (AWS et al. match redirect URIs exactly).
} else if let Some(cached) = load_registered_client_id(server_name) { // Only when no client_id is configured explicitly.
cached let cached_reuse: Option<(String, String)> = if configured_client_id.is_none() {
} else if let Some(reg_endpoint) = &metadata.registration_endpoint { load_registration(server_name).and_then(|reg| {
match register_client(reg_endpoint, &redirect_uri).await { let redirect = reg.redirect_uri?;
Ok(id) => { let port = cached_redirect_port(&redirect, host, callback_port)?;
let _ = save_registered_client_id(server_name, &id); // The registered port must still be free for our callback listener.
id TcpListener::bind(format!("127.0.0.1:{port}")).ok()?;
} Some((reg.client_id, redirect))
Err(e) => { })
warn!("Dynamic client registration failed: {e}. Falling back to manual entry.");
Text::new("Enter the OAuth client ID for this MCP server:")
.prompt()
.context("Failed to read client ID")?
}
}
} else { } else {
Text::new("Enter the OAuth client ID for this MCP server:") None
.prompt() };
.context("Failed to read client ID")?
let (client_id, redirect_uri) = if let Some(reused) = cached_reuse {
reused
} else {
let bind_addr = format!("127.0.0.1:{}", callback_port.unwrap_or(0));
let listener = TcpListener::bind(&bind_addr)?;
let port = listener.local_addr()?.port();
drop(listener);
let redirect_uri = format!("http://{host}:{port}/callback");
let client_id = if let Some(id) = configured_client_id {
id.to_string()
} else if let Some(reg_endpoint) = &metadata.registration_endpoint {
match register_client(reg_endpoint, &redirect_uri).await {
Ok(id) => {
let _ = save_registration(server_name, &id, &redirect_uri);
id
}
Err(e) => {
warn!("Dynamic client registration failed: {e}. Falling back to manual entry.");
Text::new("Enter the OAuth client ID for this MCP server:")
.prompt()
.context("Failed to read client ID")?
}
}
} else {
Text::new("Enter the OAuth client ID for this MCP server:")
.prompt()
.context("Failed to read client ID")?
};
(client_id, redirect_uri)
}; };
let provider = McpOAuthProvider { let provider = McpOAuthProvider {
@@ -123,6 +162,7 @@ pub async fn run_mcp_oauth_flow(
token_url: metadata.token_endpoint, token_url: metadata.token_endpoint,
scopes: metadata.scopes_supported.join(" "), scopes: metadata.scopes_supported.join(" "),
fixed_redirect: redirect_uri, fixed_redirect: redirect_uri,
resource,
}; };
run_oauth_flow(&provider, &mcp_token_key(server_name)).await run_oauth_flow(&provider, &mcp_token_key(server_name)).await
@@ -141,21 +181,20 @@ fn mcp_token_key(server_name: &str) -> String {
format!("mcp_{server_name}") format!("mcp_{server_name}")
} }
fn load_registered_client_id(server_name: &str) -> Option<String> { fn load_registration(server_name: &str) -> Option<McpRegistration> {
let path = paths::oauth_tokens_dir().join(format!("mcp_{server_name}_registration.json")); let path = paths::oauth_tokens_dir().join(format!("mcp_{server_name}_registration.json"));
let content = fs::read_to_string(path).ok()?; let content = fs::read_to_string(path).ok()?;
let reg: McpRegistration = serde_json::from_str(&content).ok()?; serde_json::from_str(&content).ok()
Some(reg.client_id)
} }
fn save_registered_client_id(server_name: &str, client_id: &str) -> Result<()> { fn save_registration(server_name: &str, client_id: &str, redirect_uri: &str) -> Result<()> {
let dir = paths::oauth_tokens_dir(); let dir = paths::oauth_tokens_dir();
fs::create_dir_all(&dir)?; fs::create_dir_all(&dir)?;
let path = dir.join(format!("mcp_{server_name}_registration.json")); let path = dir.join(format!("mcp_{server_name}_registration.json"));
let reg = McpRegistration { let reg = McpRegistration {
client_id: client_id.to_string(), client_id: client_id.to_string(),
redirect_uri: Some(redirect_uri.to_string()),
}; };
fs::write(path, serde_json::to_string_pretty(&reg)?)?; fs::write(path, serde_json::to_string_pretty(&reg)?)?;
@@ -163,6 +202,27 @@ fn save_registered_client_id(server_name: &str, client_id: &str) -> Result<()> {
Ok(()) Ok(())
} }
/// Returns the port of a cached registered redirect URI if it is still
/// compatible with the current configuration: same redirect host, and, when
/// a callback port is pinned in config, the same port. Servers like AWS
/// match redirect URIs exactly, so a cached registration is only reusable
/// with the identical redirect URI it was registered with.
fn cached_redirect_port(
cached_redirect: &str,
host: &str,
pinned_port: Option<u16>,
) -> Option<u16> {
let url = Url::parse(cached_redirect).ok()?;
if url.host_str() != Some(host) {
return None;
}
let port = url.port()?;
if pinned_port.is_some_and(|p| p != port) {
return None;
}
Some(port)
}
async fn register_client(endpoint: &str, redirect_uri: &str) -> Result<String> { async fn register_client(endpoint: &str, redirect_uri: &str) -> Result<String> {
let body = serde_json::json!({ let body = serde_json::json!({
"client_name": "Coyote", "client_name": "Coyote",
@@ -188,7 +248,44 @@ async fn register_client(endpoint: &str, redirect_uri: &str) -> Result<String> {
.map(|s| s.to_string()) .map(|s| s.to_string())
} }
async fn discover_oauth_metadata(server_url: &str) -> Result<OAuthServerMetadata> { /// Derives the canonical resource URI for an MCP server per RFC 8707 @ 2 and
/// the MCP spec: the configured server URL with query and fragment stripped.
fn canonical_resource(server_url: &str) -> Result<String> {
let mut url =
Url::parse(server_url).with_context(|| format!("Invalid MCP server URL: {server_url}"))?;
url.set_query(None);
url.set_fragment(None);
let s = url.to_string();
Ok(match url.path() {
"/" => s.trim_end_matches('/').to_string(),
_ => s,
})
}
/// Resolves the RFC 8707 resource indicator: prefers the value advertised in
/// the protected resource metadata, but only after validating it identifies
/// the server we are connecting to (RFC 9728 @ 3.3); same scheme/host/port
/// as the configured server URL. Falls back to the canonical server URL on
/// mismatch, empty value, or absence.
fn resolve_resource(advertised: Option<String>, server_url: &str) -> Result<String> {
let canonical = canonical_resource(server_url)?;
let Some(advertised) = advertised.filter(|r| !r.is_empty()) else {
return Ok(canonical);
};
match (Url::parse(&advertised), Url::parse(server_url)) {
(Ok(a), Ok(s)) if a.origin() == s.origin() => Ok(advertised),
_ => {
warn!(
"Ignoring protected resource metadata resource '{advertised}': \
it does not match the MCP server origin. Using '{canonical}' instead."
);
Ok(canonical)
}
}
}
async fn discover_oauth_metadata(server_url: &str) -> Result<DiscoveredOAuth> {
let client = Client::new(); let client = Client::new();
let mut tried: Vec<String> = Vec::new(); let mut tried: Vec<String> = Vec::new();
@@ -231,7 +328,10 @@ async fn discover_oauth_metadata(server_url: &str) -> Result<OAuthServerMetadata
if meta.scopes_supported.is_empty() { if meta.scopes_supported.is_empty() {
meta.scopes_supported = pr.scopes_supported.clone(); meta.scopes_supported = pr.scopes_supported.clone();
} }
return Ok(meta); return Ok(DiscoveredOAuth {
metadata: meta,
resource: pr.resource.clone(),
});
} }
} }
} }
@@ -245,7 +345,11 @@ async fn discover_oauth_metadata(server_url: &str) -> Result<OAuthServerMetadata
return resp return resp
.json::<OAuthServerMetadata>() .json::<OAuthServerMetadata>()
.await .await
.with_context(|| format!("Failed to parse OAuth metadata from {as_url}")); .with_context(|| format!("Failed to parse OAuth metadata from {as_url}"))
.map(|metadata| DiscoveredOAuth {
metadata,
resource: None,
});
} }
} }
@@ -469,23 +573,132 @@ mod tests {
); );
} }
#[test]
fn canonical_resource_strips_query() {
let result = canonical_resource("https://aws-mcp.us-east-1.api.aws/mcp?oauth=initialize");
assert_eq!(result.unwrap(), "https://aws-mcp.us-east-1.api.aws/mcp");
}
#[test]
fn canonical_resource_strips_fragment() {
let result = canonical_resource("https://example.com/mcp#section");
assert_eq!(result.unwrap(), "https://example.com/mcp");
}
#[test]
fn canonical_resource_preserves_path_and_port() {
let result = canonical_resource("http://localhost:8080/mcp/v1?x=1");
assert_eq!(result.unwrap(), "http://localhost:8080/mcp/v1");
}
#[test]
fn canonical_resource_rejects_invalid_url() {
assert!(canonical_resource("not-a-url").is_err());
}
#[test]
fn canonical_resource_bare_host_has_no_trailing_slash() {
let result = canonical_resource("https://mcp.example.com");
assert_eq!(result.unwrap(), "https://mcp.example.com");
}
#[test]
fn resolve_resource_prefers_matching_advertised() {
let result = resolve_resource(
Some("https://aws-mcp.us-east-1.api.aws/mcp".into()),
"https://aws-mcp.us-east-1.api.aws/mcp?oauth=initialize",
);
assert_eq!(result.unwrap(), "https://aws-mcp.us-east-1.api.aws/mcp");
}
#[test]
fn resolve_resource_rejects_cross_origin_advertised() {
let result = resolve_resource(
Some("https://evil.example.com/mcp".into()),
"https://aws-mcp.us-east-1.api.aws/mcp",
);
assert_eq!(result.unwrap(), "https://aws-mcp.us-east-1.api.aws/mcp");
}
#[test]
fn resolve_resource_empty_falls_back_to_canonical() {
let result = resolve_resource(Some(String::new()), "https://example.com/mcp");
assert_eq!(result.unwrap(), "https://example.com/mcp");
}
#[test]
fn resolve_resource_none_falls_back_to_canonical() {
let result = resolve_resource(None, "https://example.com/mcp");
assert_eq!(result.unwrap(), "https://example.com/mcp");
}
#[test]
fn protected_resource_metadata_deserializes_resource_field() {
let json = r#"{"resource":"https://aws-mcp.us-east-1.api.aws/mcp","authorization_servers":["https://us-east-1.oauth.signin.aws/"]}"#;
let pr: ProtectedResourceMetadata = serde_json::from_str(json).unwrap();
assert_eq!(
pr.resource.as_deref(),
Some("https://aws-mcp.us-east-1.api.aws/mcp")
);
assert_eq!(
pr.authorization_servers,
vec!["https://us-east-1.oauth.signin.aws/"]
);
}
#[test]
fn mcp_provider_sends_resource_in_authorize_and_token_params() {
let provider = McpOAuthProvider {
client_id: "client-123".into(),
authorize_url: "https://as.example/authorize".into(),
token_url: "https://as.example/token".into(),
scopes: String::new(),
fixed_redirect: "http://127.0.0.1:9000/callback".into(),
resource: "https://aws-mcp.us-east-1.api.aws/mcp".into(),
};
assert_eq!(
provider.extra_authorize_params(),
vec![("resource", "https://aws-mcp.us-east-1.api.aws/mcp")]
);
assert_eq!(
provider.extra_token_params(),
vec![("resource", "https://aws-mcp.us-east-1.api.aws/mcp")]
);
}
#[test] #[test]
#[serial] #[serial]
fn registered_client_id_roundtrip() { fn registered_client_id_roundtrip() {
with_temp_cache(|| { with_temp_cache(|| {
save_registered_client_id("notion", "client-xyz-123").unwrap(); save_registration(
"notion",
"client-xyz-123",
"http://127.0.0.1:49152/callback",
)
.unwrap();
let loaded = load_registered_client_id("notion"); let loaded = load_registration("notion");
assert_eq!(loaded, Some("client-xyz-123".to_string())); assert_eq!(loaded.unwrap().client_id, "client-xyz-123");
}); });
} }
#[test] #[test]
#[serial] #[serial]
fn load_registered_client_id_returns_none_for_missing() { fn load_registration_returns_none_for_missing() {
with_temp_cache(|| { with_temp_cache(|| {
let loaded = load_registered_client_id("no-such-server"); let loaded = load_registration("no-such-server");
assert!(loaded.is_none()); assert!(loaded.is_none());
}); });
@@ -493,14 +706,79 @@ mod tests {
#[test] #[test]
#[serial] #[serial]
fn registered_client_id_second_save_overwrites_first() { fn registration_second_save_overwrites_first() {
with_temp_cache(|| { with_temp_cache(|| {
save_registered_client_id("github", "first-id").unwrap(); save_registration("github", "first-id", "http://127.0.0.1:49152/callback").unwrap();
save_registered_client_id("github", "second-id").unwrap(); save_registration("github", "second-id", "http://127.0.0.1:49153/callback").unwrap();
let loaded = load_registered_client_id("github"); let loaded = load_registration("github").unwrap();
assert_eq!(loaded, Some("second-id".to_string())); assert_eq!(loaded.client_id, "second-id");
assert_eq!(
loaded.redirect_uri.as_deref(),
Some("http://127.0.0.1:49153/callback")
);
}); });
} }
#[test]
#[serial]
fn old_format_registration_still_loads() {
with_temp_cache(|| {
let dir = paths::oauth_tokens_dir();
fs::create_dir_all(&dir).unwrap();
fs::write(
dir.join("mcp_legacy_registration.json"),
r#"{"client_id":"legacy-id"}"#,
)
.unwrap();
let loaded = load_registration("legacy").unwrap();
assert_eq!(loaded.client_id, "legacy-id");
assert_eq!(loaded.redirect_uri, None);
});
}
#[test]
#[serial]
fn save_registration_persists_redirect_uri() {
with_temp_cache(|| {
save_registration("aws", "client-abc", "http://127.0.0.1:49152/callback").unwrap();
let loaded = load_registration("aws").unwrap();
assert_eq!(loaded.client_id, "client-abc");
assert_eq!(
loaded.redirect_uri.as_deref(),
Some("http://127.0.0.1:49152/callback")
);
});
}
#[test]
fn cached_redirect_port_matches() {
let port = cached_redirect_port("http://127.0.0.1:49152/callback", "127.0.0.1", None);
assert_eq!(port, Some(49152));
}
#[test]
fn cached_redirect_port_rejects_host_mismatch() {
let port = cached_redirect_port("http://127.0.0.1:49152/callback", "localhost", None);
assert_eq!(port, None);
}
#[test]
fn cached_redirect_port_respects_pinned_port() {
assert_eq!(
cached_redirect_port("http://127.0.0.1:49152/callback", "127.0.0.1", Some(50000)),
None
);
assert_eq!(
cached_redirect_port("http://127.0.0.1:49152/callback", "127.0.0.1", Some(49152)),
Some(49152)
);
}
} }
+1550 -88
View File
File diff suppressed because it is too large Load Diff
+76
View File
@@ -0,0 +1,76 @@
use super::{DocumentId, RagData};
use anyhow::Result;
use async_trait::async_trait;
/// Abstracts where RAG vector data is stored and queried.
///
/// The Rag orchestrator owns: embeddings, chunking, BM25 keyword search, graph RAG,
/// entity extraction, RRF merging. Providers own: vector storage and content retrieval.
#[async_trait]
pub trait RagProvider: Send + Sync {
/// Vector similarity search. Returns (DocumentId, score) sorted by score desc.
/// `embedding` is a single query vector from Coyote's embedding model.
async fn vector_search(
&self,
embedding: &[f32],
top_k: usize,
min_score: f32,
) -> Result<Vec<(DocumentId, f32)>>;
/// Resolve document IDs to their page content.
///
/// **Ordering contract:** implementations MUST return results in the same
/// relative order as the input `ids` slice. `hybrid_search` passes an
/// RRF-ranked list and feeds the result straight to the LLM. A provider
/// that returns rows in storage order (e.g. Qdrant `get_points`, DuckDB
/// `WHERE id IN (...)`) would silently discard the ranking. Implementations
/// that query an unordered backend must re-sort by input position before
/// returning.
///
/// Returns only IDs that were found; callers must handle partial returns
/// (a missing ID is skipped, not an error).
/// YamlProvider: reads from an in-memory content map built from data.files.
/// DuckDbProvider: queries the documents table by id.
/// QdrantProvider: fetches payload from the remote collection.
async fn fetch_content(&self, ids: &[DocumentId]) -> Result<Vec<(DocumentId, String)>>;
/// Rebuild internal indexes from freshly updated RagData.
/// Called once at the end of every sync_documents pass.
///
/// `full_rebuild` mirrors `sync_documents`' `refresh` parameter:
/// - `true`: a full re-index (`.rebuild rag`, `--rebuild-rag`, initial build).
/// Destructive strategies (wipe-then-reindex) are permitted.
/// - `false`: an incremental change (`.edit rag-docs` adding/removing a file).
/// Implementations MUST NOT wipe existing state; upsert only.
///
/// The parameter is part of the signature from the outset so it is fixed
/// while there is exactly one implementor. Yaml/DuckDb ignore it,
/// rebuilding their local state wholesale is fast and always correct.
/// Only a remote provider is destructive enough to care.
async fn rebuild_indexes(&mut self, data: &RagData, full_rebuild: bool) -> Result<()>;
/// Keyword / full-text search. Returns (DocumentId, BM25-style score) sorted desc.
///
/// Default impl returns `Ok(vec![])`. Callers fall back to `Rag.bm25` (local in-memory
/// BM25 built from `data.files`).
///
/// Callers check `has_native_keyword_search()` before deciding which path to take:
/// - true → call this method; skip `Rag.bm25`
/// - false → call `Rag.keyword_search()` which uses `Rag.bm25` (sync, infallible)
async fn keyword_search(&self, query: &str, top_k: usize) -> Result<Vec<(DocumentId, f32)>> {
let _ = (query, top_k);
Ok(vec![])
}
/// Returns true if this provider implements a native keyword-search index.
/// When false, `Rag.hybrid_search` uses the local `Rag.bm25` field instead.
fn has_native_keyword_search(&self) -> bool {
false
}
/// Deep-clone the provider with fresh indexes derived from `data`.
/// Required because Box<dyn RagProvider> is not Clone.
/// Called by Rag's Clone impl (which clones before mutating in rebuild_rag/edit_rag_docs).
fn duplicate(&self, data: &RagData) -> Box<dyn RagProvider>;
}
File diff suppressed because it is too large Load Diff
+9
View File
@@ -0,0 +1,9 @@
mod yaml;
pub use self::yaml::YamlProvider;
mod duckdb;
pub use self::duckdb::DuckDbProvider;
pub(crate) use self::duckdb::duckdb_path_from_yaml;
mod qdrant;
pub use self::qdrant::QdrantProvider;
+828
View File
@@ -0,0 +1,828 @@
use crate::rag::provider::RagProvider;
use crate::rag::{DocumentId, RagData};
use anyhow::{Context, Result, bail};
use async_trait::async_trait;
use parking_lot::RwLock;
use reqwest::header::{HeaderMap, HeaderValue};
use reqwest::{Client, Response, StatusCode};
use serde_json::Value;
use std::collections::HashMap;
use std::sync::Arc;
use url::{Host, Url};
/// Marks a `DocumentId` that stands in for a point id Coyote cannot carry
/// directly. Qdrant accepts UUID strings as point ids, and that is what
/// LangChain writes by default.
///
/// `DocumentId` packs `(file_index, document_index)` into one `usize` with the
/// file index in the high half, so this bit is only reachable at a file index of
/// 2^31. Nothing local gets near that, and an attached RAG builds no local index
/// at all — `data.files` and `data.vectors` stay empty and every
/// `DocumentId::split` caller early-returns on `data.attached`. Along the
/// attached path the id is an opaque key carried through RRF, which is what
/// makes a synthetic one safe here and nowhere else.
const SYNTHETIC_ID_TAG: usize = 1 << (usize::BITS - 1);
/// Two-way map between a raw Qdrant point id and the `DocumentId` the retrieval
/// pipeline sees.
///
/// Only ids that cannot survive the round trip are interned. A plain `u64` that
/// fits below the tag keeps mapping to itself, so integer-keyed collections
/// behave exactly as they did before this map existed.
#[derive(Default)]
struct PointIdInterner {
handles: HashMap<String, DocumentId>,
raw: HashMap<DocumentId, Value>,
next: usize,
}
impl PointIdInterner {
/// The `DocumentId` for a raw point id, minting a handle if one is needed.
///
/// `None` only for a missing id, which is a malformed response.
fn document_id(&mut self, raw: &Value) -> Option<DocumentId> {
if raw.is_null() {
return None;
}
// The pre-existing integer path, unchanged. `try_from` rather than `as`
// so a value too wide for the target's `usize` is interned instead of
// silently truncated into a different point.
if let Some(n) = raw.as_u64()
&& let Ok(n) = usize::try_from(n)
&& n & SYNTHETIC_ID_TAG == 0
{
return Some(DocumentId(n));
}
Some(self.intern(raw))
}
fn intern(&mut self, raw: &Value) -> DocumentId {
// Keyed on the JSON rendering, so the string "1" and the integer 1 are
// not conflated into one point.
let key = raw.to_string();
if let Some(handle) = self.handles.get(&key) {
return *handle;
}
let handle = DocumentId(SYNTHETIC_ID_TAG | self.next);
self.next += 1;
self.handles.insert(key, handle);
self.raw.insert(handle, raw.clone());
handle
}
/// The original id for a handle, or `None` when the id was never interned —
/// i.e. it is a plain integer that is already its own id.
fn raw_id(&self, handle: DocumentId) -> Option<&Value> {
self.raw.get(&handle)
}
/// Builds the `ids` array for an outbound `/points` fetch. Every entry is the
/// id Qdrant issued, integer or string; a synthetic handle must never leave
/// this process.
fn outbound_ids(&self, ids: &[DocumentId]) -> Vec<Value> {
ids.iter()
.map(|id| match self.raw_id(*id) {
Some(raw) => raw.clone(),
None => Value::from(id.0 as u64),
})
.collect()
}
}
fn parse_search_hits(
interner: &mut PointIdInterner,
body: &Value,
min_score: f32,
) -> Result<Vec<(DocumentId, f32)>> {
let hits = body["result"]
.as_array()
.context("Unexpected /points/search response shape")?;
Ok(hits
.iter()
.filter_map(|pt| {
let score = pt["score"].as_f64()? as f32;
Some((interner.document_id(&pt["id"])?, score))
})
.filter(|(_, score)| min_score <= 0.0 || *score > min_score)
.collect())
}
fn parse_points(interner: &mut PointIdInterner, body: &Value) -> Result<Vec<(DocumentId, String)>> {
let points = body["result"]
.as_array()
.context("Unexpected /points response shape")?;
Ok(points
.iter()
.filter_map(|pt| {
let text = pt["payload"]["page_content"].as_str()?.to_string();
Some((interner.document_id(&pt["id"])?, text))
})
.collect())
}
/// Render Qdrant's error envelope into a human-readable message.
///
/// `body` is the raw response text. Two shapes have to be tolerated:
/// * application-level errors carry `{"status": {"error": "..."}, "time": 0.0}`,
/// while successful responses carry a bare string `{"status": "ok", ...}` — so
/// `status` is string-or-object and a struct with `status: String` fails to
/// parse every error body;
/// * routing-level 404s (a wrong HTTP verb) return an EMPTY body with no JSON at
/// all, which without the length check surfaces as "EOF while parsing a value"
/// instead of the actual 404.
fn format_error_body(status: StatusCode, body: &str) -> String {
if body.is_empty() {
return format!("HTTP {status} (empty body — check the HTTP verb and path)");
}
serde_json::from_str::<Value>(body)
.ok()
.and_then(|v| v["status"]["error"].as_str().map(str::to_string))
.unwrap_or_else(|| format!("HTTP {status}: {body}"))
}
/// Read the vector dimension out of a parsed `GET /collections/{name}` response.
fn vector_dimension_from_collection(body: &Value) -> Result<u64> {
let params = &body["result"]["config"]["params"];
params["vectors"]["size"]
.as_u64()
.or_else(|| {
params["vectors"]
.as_object()
.and_then(|m| m.values().next())
.and_then(|v| v["size"].as_u64())
})
.context("Could not determine vector dimension from collection config")
}
/// True if a parsed `GET /collections/{name}` response describes a NAMED
/// (multi-vector) collection.
///
/// `vector_search` posts an unnamed vector, which a named-vector collection
/// rejects with HTTP 400 on every query, so attaching one yields a RAG that is
/// silently 100% broken. A named collection holding a SINGLE vector is
/// structurally a map, identical in kind to the multi-named case, and rejects
/// the same way; testing for a numeric `size` directly under `vectors` catches
/// it, whereas counting keys (`len() > 1`) would wrongly accept it.
fn is_multi_vector_config(body: &Value) -> bool {
body["result"]["config"]["params"]["vectors"]["size"]
.as_u64()
.is_none()
}
/// Query-only client for an external Qdrant collection.
///
/// Attach-only: this provider never writes to the remote collection. Coyote does
/// not own the data, and `rebuild_indexes` refuses rather than pretending to.
pub struct QdrantProvider {
client: Client,
base_url: String,
collection: String,
point_ids: Arc<RwLock<PointIdInterner>>,
}
impl QdrantProvider {
fn skips_proxy(base_url: &str) -> bool {
let Ok(url) = Url::parse(base_url) else {
return false;
};
match url.host() {
Some(Host::Domain(name)) => {
name == "localhost" || name.ends_with(".localhost") || name.ends_with(".local")
}
Some(Host::Ipv4(ip)) => ip.is_loopback() || ip.is_private() || ip.is_link_local(),
// No stable is_unique_local, so fc00::/7 is matched directly.
Some(Host::Ipv6(ip)) => ip.is_loopback() || ip.segments()[0] & 0xfe00 == 0xfc00,
None => false,
}
}
fn make_client(base_url: &str, api_key: Option<&str>) -> Result<Client> {
let mut headers = HeaderMap::new();
if let Some(key) = api_key {
let mut value =
HeaderValue::from_str(key).context("api-key header value is not valid ASCII")?;
value.set_sensitive(true);
headers.insert("api-key", value);
}
let mut builder = Client::builder().default_headers(headers);
if Self::skips_proxy(base_url) {
builder = builder.no_proxy();
}
builder.build().context("Failed to build reqwest client")
}
pub(crate) fn normalize_base_url(host: &str) -> String {
if host.starts_with("http://") || host.starts_with("https://") {
host.to_string()
} else {
format!("http://{host}")
}
}
async fn error_message(resp: Response) -> String {
let status = resp.status();
let body = resp.text().await.unwrap_or_default();
format_error_body(status, &body)
}
/// Shared `GET /collections/{name}` fetch. Both the dimension and the
/// multi-vector probe discriminate on this same response.
async fn fetch_collection(
host: &str,
collection: &str,
api_key: Option<&str>,
) -> Result<Value> {
let base_url = Self::normalize_base_url(host);
let client = Self::make_client(&base_url, api_key)?;
let resp = client
.get(format!("{base_url}/collections/{collection}"))
.send()
.await
.with_context(|| format!("Failed to connect to {host}"))?;
if !resp.status().is_success() {
bail!(
"Failed to read collection '{collection}': {}",
Self::error_message(resp).await
);
}
Ok(resp.json().await?)
}
pub async fn new(host: &str, collection: &str, api_key: Option<&str>) -> Result<Self> {
let base_url = Self::normalize_base_url(host);
let client = Self::make_client(&base_url, api_key)?;
let resp = client
.get(format!("{base_url}/collections/{collection}"))
.send()
.await
.with_context(|| format!("Failed to connect to {host}"))?;
if !resp.status().is_success() {
bail!(
"Collection '{collection}' not accessible at {host}: {}",
Self::error_message(resp).await
);
}
Ok(Self {
client,
base_url,
collection: collection.to_string(),
point_ids: Arc::default(),
})
}
pub async fn list_collections(host: &str, api_key: Option<&str>) -> Result<Vec<String>> {
let base_url = Self::normalize_base_url(host);
let client = Self::make_client(&base_url, api_key)?;
let resp = client
.get(format!("{base_url}/collections"))
.send()
.await
.with_context(|| format!("Failed to connect to {host}"))?;
if !resp.status().is_success() {
bail!(
"Failed to list collections: {}",
Self::error_message(resp).await
);
}
let body: Value = resp.json().await?;
let names = body["result"]["collections"]
.as_array()
.context("Unexpected /collections response shape")?
.iter()
.filter_map(|v| v["name"].as_str().map(str::to_string))
.collect();
Ok(names)
}
pub async fn get_vector_dimension(
host: &str,
collection: &str,
api_key: Option<&str>,
) -> Result<u64> {
let body = Self::fetch_collection(host, collection, api_key).await?;
vector_dimension_from_collection(&body)
}
pub async fn is_multi_vector(
host: &str,
collection: &str,
api_key: Option<&str>,
) -> Result<bool> {
let body = Self::fetch_collection(host, collection, api_key).await?;
Ok(is_multi_vector_config(&body))
}
pub async fn sample_point_id(
host: &str,
collection: &str,
api_key: Option<&str>,
) -> Result<Option<String>> {
let base_url = Self::normalize_base_url(host);
let client = Self::make_client(&base_url, api_key)?;
let url = format!("{base_url}/collections/{collection}/points/scroll");
let body = serde_json::json!({ "limit": 1, "with_payload": false });
let resp = client
.post(&url)
.json(&body)
.send()
.await
.with_context(|| format!("Failed to connect to {host}"))?;
if !resp.status().is_success() {
bail!(
"Failed to sample a point from '{collection}': {}",
Self::error_message(resp).await
);
}
let data: Value = resp.json().await?;
let id_val = data["result"]["points"]
.as_array()
.and_then(|pts| pts.first())
.map(|pt| pt["id"].to_string());
Ok(id_val)
}
}
#[async_trait]
impl RagProvider for QdrantProvider {
async fn vector_search(
&self,
embedding: &[f32],
top_k: usize,
min_score: f32,
) -> Result<Vec<(DocumentId, f32)>> {
let url = format!(
"{}/collections/{}/points/search",
self.base_url, self.collection
);
// `score_threshold` is deliberately NOT sent. It is metric-aware: on Cosine
// collections 0.0 means "no floor" as expected, but Euclid collections score
// by negative distance, where 0.0 filters everything out. The attach wizard
// does not pin the distance metric, so filter locally instead; i.e. where a
// 0.0 floor is correctly treated as "no floor" (see `parse_search_hits`).
let body = serde_json::json!({
"vector": embedding,
"limit": top_k,
"with_payload": false,
});
let resp = self.client.post(&url).json(&body).send().await?;
if !resp.status().is_success() {
bail!(
"Qdrant search on '{}' failed: {}",
self.collection,
Self::error_message(resp).await
);
}
let data: Value = resp.json().await?;
// The interner is what lets a UUID-keyed collection work: a string id gets
// a synthetic handle here and the original is replayed by `fetch_content`.
let mut interner = self.point_ids.write();
parse_search_hits(&mut interner, &data, min_score)
}
async fn fetch_content(&self, ids: &[DocumentId]) -> Result<Vec<(DocumentId, String)>> {
if ids.is_empty() {
return Ok(vec![]);
}
let url = format!("{}/collections/{}/points", self.base_url, self.collection);
// Qdrant is asked for the ids it issued, never for a synthetic handle.
let id_list = self.point_ids.read().outbound_ids(ids);
let body = serde_json::json!({
"ids": id_list,
"with_payload": true,
});
let resp = self.client.post(&url).json(&body).send().await?;
if !resp.status().is_success() {
bail!(
"Qdrant point fetch on '{}' failed: {}",
self.collection,
Self::error_message(resp).await
);
}
let data: Value = resp.json().await?;
let mut rows = {
let mut interner = self.point_ids.write();
parse_points(&mut interner, &data)?
};
// `/points` does not guarantee response order matches request order, and the
// caller's RRF ranking is carried by that order. Restore it.
let position: HashMap<DocumentId, usize> =
ids.iter().enumerate().map(|(i, id)| (*id, i)).collect();
rows.sort_by_key(|(id, _)| position.get(id).copied().unwrap_or(usize::MAX));
Ok(rows)
}
async fn rebuild_indexes(&mut self, data: &RagData, _full_rebuild: bool) -> Result<()> {
// Both arms refuse. A silent `Ok(())` would make `.rebuild rag` and
// `.edit rag-docs` look like they worked while writing nothing to the
// remote, leaving the user believing the collection was updated.
if data.attached {
bail!(
"This RAG is attached to an external Qdrant collection. Coyote does not own \
its documents and cannot rebuild it. Manage the collection directly, or \
create a Coyote-owned RAG with `.rag <name>`."
);
}
bail!("Writing to Qdrant is not supported yet (attach-only).");
}
fn duplicate(&self, _data: &RagData) -> Box<dyn RagProvider> {
// Cloning the client shares the connection pool and the injected api-key
// header. Sharing is correct: both handles address the same remote
// collection, and neither of them writes to it.
//
// The point-id map is shared for the same reason, and because it MUST be:
// `Rag::clone()` hands the clone `DocumentId`s that the original minted,
// so a fresh map would resolve them to nothing and `fetch_content` would
// ask Qdrant for a synthetic handle — zero results, no error. Resetting it
// would also re-mint handles for ids the original still holds.
Box::new(Self {
client: self.client.clone(),
base_url: self.base_url.clone(),
collection: self.collection.clone(),
point_ids: Arc::clone(&self.point_ids),
})
}
}
#[cfg(test)]
mod tests {
use super::*;
#[test]
fn error_message_reads_the_object_status_envelope() {
let body =
r#"{"status": {"error": "Wrong input: Not existing vector name error:"}, "time": 0.0}"#;
let msg = format_error_body(StatusCode::BAD_REQUEST, body);
assert!(msg.contains("Not existing vector name"), "got: {msg}");
assert!(
!msg.contains("EOF"),
"must not fall through to a parse error"
);
}
#[test]
fn error_message_survives_the_string_status_and_the_empty_body() {
let ok = format_error_body(StatusCode::OK, r#"{"status": "ok", "time": 0.0}"#);
assert!(
ok.contains("200"),
"no `status.error` present → fall back to status+body: {ok}"
);
let empty = format_error_body(StatusCode::NOT_FOUND, "");
assert!(empty.contains("empty body"), "got: {empty}");
assert!(
empty.contains("verb"),
"the message must point at the likely cause: {empty}"
);
}
#[test]
fn vector_dimension_handles_both_collection_shapes() {
let unnamed = serde_json::json!({
"result": {"config": {"params": {"vectors": {"size": 1536, "distance": "Cosine"}}}}
});
assert_eq!(vector_dimension_from_collection(&unnamed).unwrap(), 1536);
let named = serde_json::json!({
"result": {"config": {"params": {"vectors": {"text": {"size": 768, "distance": "Cosine"}}}}}
});
assert_eq!(vector_dimension_from_collection(&named).unwrap(), 768);
let junk = serde_json::json!({"result": {"config": {"params": {}}}});
assert!(vector_dimension_from_collection(&junk).is_err());
}
#[test]
fn is_multi_vector_rejects_the_named_single_collection() {
let unnamed = serde_json::json!({
"result": {"config": {"params": {"vectors": {"size": 1536, "distance": "Cosine"}}}}
});
assert!(!is_multi_vector_config(&unnamed));
let named_single = serde_json::json!({
"result": {"config": {"params": {"vectors": {"text": {"size": 1536}}}}}
});
assert!(
is_multi_vector_config(&named_single),
"named-single must be rejected too"
);
let named_multi = serde_json::json!({
"result": {"config": {"params": {"vectors": {"text": {"size": 1536}, "image": {"size": 512}}}}}
});
assert!(is_multi_vector_config(&named_multi));
}
#[test]
fn normalize_base_url_only_adds_a_scheme_when_missing() {
assert_eq!(
QdrantProvider::normalize_base_url("qdrant.example.com:6333"),
"http://qdrant.example.com:6333"
);
assert_eq!(
QdrantProvider::normalize_base_url("https://xyz.cloud.qdrant.io"),
"https://xyz.cloud.qdrant.io"
);
assert_eq!(
QdrantProvider::normalize_base_url("http://localhost:6333"),
"http://localhost:6333"
);
}
#[tokio::test]
async fn rebuild_indexes_refuses_for_attached_and_unattached_alike() {
let mut provider = QdrantProvider {
client: Client::new(),
base_url: "http://localhost:6333".to_string(),
collection: "c".to_string(),
point_ids: Arc::default(),
};
let attached = RagData {
driver: "qdrant".to_string(),
attached: true,
..Default::default()
};
let err = provider
.rebuild_indexes(&attached, true)
.await
.expect_err("an attached qdrant RAG must never report a successful rebuild");
assert!(err.to_string().contains("cannot rebuild"), "got: {err}");
// `attached: false` is reserved for the (unimplemented) write path. It must
// also refuse: silently succeeding would run a full paid embedding pass and
// then discard every vector.
let owned = RagData {
driver: "qdrant".to_string(),
attached: false,
..Default::default()
};
let err = provider
.rebuild_indexes(&owned, true)
.await
.expect_err("writing to qdrant is unimplemented and must fail loudly");
assert!(err.to_string().contains("not supported yet"), "got: {err}");
}
#[tokio::test]
async fn fetch_content_short_circuits_on_an_empty_id_list() {
let provider = QdrantProvider {
client: Client::new(),
base_url: "http://127.0.0.1:1".to_string(),
collection: "c".to_string(),
point_ids: Arc::default(),
};
assert!(provider.fetch_content(&[]).await.unwrap().is_empty());
}
#[test]
fn local_and_private_hosts_skip_the_proxy() {
for host in [
"http://localhost:6333",
"http://127.0.0.1:6333",
"http://192.168.0.56:6333",
"http://10.1.2.3:6333",
"http://172.16.4.5:6333",
"http://qdrant.local:6333",
"http://[::1]:6333",
] {
assert!(
QdrantProvider::skips_proxy(host),
"{host} should not be proxied"
);
}
}
#[test]
fn public_hosts_still_honour_the_environment() {
for host in [
"https://qdrant.example.com",
"http://8.8.8.8:6333",
"https://xyz.eu-central.aws.cloud.qdrant.io:6333",
"http://172.32.0.1:6333",
] {
assert!(
!QdrantProvider::skips_proxy(host),
"{host} must keep the environment's proxy"
);
}
}
/// Euclid collections score by NEGATIVE distance, so the 0.0 the caller
/// passes must mean "no floor". Filtering on it drops every hit — the exact
/// bug that keeps Qdrant's own `score_threshold` off the wire.
#[test]
fn a_zero_floor_keeps_negative_euclid_scores() {
let mut interner = PointIdInterner::default();
let search = serde_json::json!({
"result": [
{"id": 1, "score": -0.12},
{"id": 2, "score": -8.5},
]
});
let hits = parse_search_hits(&mut interner, &search, 0.0).unwrap();
assert_eq!(hits.len(), 2, "a 0.0 floor must not drop negative scores");
}
#[test]
fn a_positive_floor_still_filters() {
let mut interner = PointIdInterner::default();
let search = serde_json::json!({
"result": [
{"id": 1, "score": 0.9},
{"id": 2, "score": 0.2},
]
});
let hits = parse_search_hits(&mut interner, &search, 0.5).unwrap();
assert_eq!(hits.len(), 1);
assert_eq!(hits[0].0, DocumentId(1));
}
/// A UUID-keyed collection has to survive the whole `vector_search` →
/// `fetch_content` round trip, and the fetch must ask Qdrant for the ORIGINAL
/// string id. Parsing ids with `as_u64()` used to drop these hits inside a
/// `filter_map`, i.e. zero results and no error.
#[test]
fn uuid_point_ids_round_trip_and_are_requested_verbatim() {
let mut interner = PointIdInterner::default();
let first_uuid = "3f1b0c2e-1111-4000-8000-000000000001";
let second_uuid = "3f1b0c2e-2222-4000-8000-000000000002";
let search = serde_json::json!({
"result": [
{"id": first_uuid, "score": 0.91},
{"id": second_uuid, "score": 0.42},
]
});
let hits = parse_search_hits(&mut interner, &search, 0.0).unwrap();
assert_eq!(hits.len(), 2, "string ids must not be silently dropped");
let ids: Vec<DocumentId> = hits.iter().map(|(id, _)| *id).collect();
assert_eq!(
interner.outbound_ids(&ids),
vec![Value::from(first_uuid), Value::from(second_uuid)],
"the fetch must send the ids Qdrant issued, not the handles"
);
// Qdrant may answer /points in any order; the handles still map back and
// the caller's RRF ranking is recoverable.
let points = serde_json::json!({
"result": [
{"id": second_uuid, "payload": {"page_content": "second"}},
{"id": first_uuid, "payload": {"page_content": "first"}},
]
});
let mut rows = parse_points(&mut interner, &points).unwrap();
let position: HashMap<DocumentId, usize> =
ids.iter().enumerate().map(|(i, id)| (*id, i)).collect();
rows.sort_by_key(|(id, _)| position.get(id).copied().unwrap_or(usize::MAX));
assert_eq!(
rows,
vec![
(ids[0], "first".to_string()),
(ids[1], "second".to_string())
]
);
}
/// Integer-keyed collections must be untouched by the interner: the id maps to
/// itself on the way in and goes back out as the same integer.
#[test]
fn integer_point_ids_are_passed_through_untouched() {
let mut interner = PointIdInterner::default();
let search = serde_json::json!({
"result": [{"id": 7, "score": 0.9}, {"id": 0, "score": 0.5}]
});
let hits = parse_search_hits(&mut interner, &search, 0.0).unwrap();
assert_eq!(
hits,
vec![(DocumentId(7), 0.9_f32), (DocumentId(0), 0.5_f32)]
);
let ids: Vec<DocumentId> = hits.iter().map(|(id, _)| *id).collect();
assert_eq!(
interner.outbound_ids(&ids),
vec![Value::from(7_u64), Value::from(0_u64)],
"integer ids must not be regressed into synthetic handles"
);
assert!(
interner.raw_id(DocumentId(7)).is_none(),
"a plain integer id is its own id and needs no map entry"
);
}
/// Synthetic handles are stable per point id and live in a range no packed
/// `DocumentId` can reach.
#[test]
fn synthetic_handles_are_stable_and_never_collide_with_packed_ids() {
let mut interner = PointIdInterner::default();
let uuid = Value::from("9d2f0a11-3333-4000-8000-00000000000a");
let handle = interner.document_id(&uuid).unwrap();
assert_eq!(
interner.document_id(&uuid).unwrap(),
handle,
"the same point id must keep the same handle across queries"
);
assert_ne!(
interner.document_id(&Value::from("other")).unwrap(),
handle,
"distinct point ids must not share a handle"
);
assert_ne!(handle.0 & SYNTHETIC_ID_TAG, 0, "a handle carries the tag");
// A packed (file_index, document_index) never sets the tag bit: it is the
// top bit of the file index, which would take 2^31 indexed files.
for (file_index, document_index) in [(0, 0), (1, 0), (0, 4242), (1_000_000, 999)] {
assert_eq!(
DocumentId::new(file_index, document_index).0 & SYNTHETIC_ID_TAG,
0,
"packed ({file_index}, {document_index}) must stay out of the handle range"
);
}
// The one integer id that WOULD land on the tag is interned instead of
// being handed back as itself, so it cannot alias a handle.
let collides = Value::from(SYNTHETIC_ID_TAG as u64);
let interned = interner.document_id(&collides).unwrap();
assert_eq!(interner.raw_id(interned), Some(&collides));
assert_eq!(
interner.outbound_ids(&[interned]),
vec![collides],
"the original integer must still be what Qdrant is asked for"
);
}
/// `duplicate()` shares the map rather than resetting it: `Rag::clone()` hands
/// the clone `DocumentId`s the original minted, and a fresh map would turn
/// those into requests for a synthetic handle — zero results, no error.
#[test]
fn duplicate_shares_the_point_id_map() {
let provider = QdrantProvider {
client: Client::new(),
base_url: "http://127.0.0.1:1".to_string(),
collection: "c".to_string(),
point_ids: Arc::default(),
};
let uuid = Value::from("c0ffee00-4444-4000-8000-000000000007");
let handle = provider.point_ids.write().document_id(&uuid).unwrap();
let dup = provider.duplicate(&RagData {
driver: "qdrant".to_string(),
attached: true,
..Default::default()
});
// Downcasting is not available through `dyn RagProvider`, so go via the
// shared Arc: the clone must observe the original's interning.
assert_eq!(Arc::strong_count(&provider.point_ids), 2);
assert_eq!(provider.point_ids.read().raw_id(handle), Some(&uuid));
drop(dup);
}
#[tokio::test]
#[ignore]
async fn qdrant_list_collections_requires_running_instance() {
let collections = QdrantProvider::list_collections("http://localhost:6333", None)
.await
.unwrap();
assert!(!collections.is_empty());
}
#[tokio::test]
#[ignore]
async fn qdrant_vector_search_returns_results() {
let provider = QdrantProvider::new("http://localhost:6333", "test-collection", None)
.await
.unwrap();
let embedding = vec![0.0f32; 1536];
let results = provider.vector_search(&embedding, 5, 0.0).await.unwrap();
assert!(results.len() <= 5);
}
}
+219
View File
@@ -0,0 +1,219 @@
use crate::rag::provider::RagProvider;
use crate::rag::{DocumentId, RagData};
use anyhow::Result;
use async_trait::async_trait;
use hnsw_rs::prelude::*;
use indexmap::IndexMap;
pub struct YamlProvider {
hnsw: Hnsw<'static, f32, DistCosine>,
content_map: IndexMap<DocumentId, String>,
}
impl YamlProvider {
pub fn from_data(data: &RagData) -> Self {
Self {
hnsw: data.build_hnsw(),
content_map: Self::build_content_map(data),
}
}
fn build_content_map(data: &RagData) -> IndexMap<DocumentId, String> {
data.iter_documents()
.map(|(id, doc)| (id, doc.page_content.clone()))
.collect()
}
}
#[async_trait]
impl RagProvider for YamlProvider {
async fn vector_search(
&self,
embedding: &[f32],
top_k: usize,
min_score: f32,
) -> Result<Vec<(DocumentId, f32)>> {
let results = self
.hnsw
.parallel_search(&[embedding.to_vec()], top_k, 30)
.into_iter()
.flat_map(|list| {
list.into_iter().filter_map(|v| {
let score = 1.0 - v.distance;
if score > min_score {
Some((DocumentId(v.d_id), score))
} else {
None
}
})
})
.collect();
Ok(results)
}
async fn fetch_content(&self, ids: &[DocumentId]) -> Result<Vec<(DocumentId, String)>> {
Ok(ids
.iter()
.filter_map(|id| self.content_map.get(id).map(|text| (*id, text.clone())))
.collect())
}
async fn rebuild_indexes(&mut self, data: &RagData, _full_rebuild: bool) -> Result<()> {
self.hnsw = data.build_hnsw();
self.content_map = Self::build_content_map(data);
Ok(())
}
fn duplicate(&self, data: &RagData) -> Box<dyn RagProvider> {
Box::new(YamlProvider::from_data(data))
}
}
#[cfg(test)]
mod provider_tests {
use super::*;
use crate::rag::{RagDocument, RagFile};
fn minimal_rag_data() -> RagData {
RagData {
embedding_model: "text-embedding-3-small".to_string(),
chunk_size: 1024,
chunk_overlap: 50,
top_k: 5,
driver: "yaml".to_string(),
attached: false,
..Default::default()
}
}
/// Two files, one chunk each, with vectors, the minimum needed to exercise
/// `build_content_map` and the `fetch_content` ordering contract.
/// `DocumentId::new(f, d)` packs (file_index, document_index); `RagData::add`
/// is the real insertion path but a direct literal is sufficient and avoids
/// the embedding pipeline.
fn populated_rag_data() -> RagData {
let mut data = minimal_rag_data();
// `files` must be populated: build_content_map iterates data.iter_documents(),
// which enumerates `files`. Populating `vectors` alone would produce an EMPTY
// content map, and every assertion below would vacuously pass on a broken impl.
// The vectors inserted at the end are for the HNSW side only.
data.files.insert(
0,
RagFile {
hash: "h0".to_string(),
path: "/tmp/a.md".to_string(),
documents: vec![RagDocument {
page_content: "alpha".to_string(),
metadata: Default::default(),
}],
},
);
data.files.insert(
1,
RagFile {
hash: "h1".to_string(),
path: "/tmp/b.md".to_string(),
documents: vec![RagDocument {
page_content: "beta".to_string(),
metadata: Default::default(),
}],
},
);
data.vectors
.insert(DocumentId::new(0, 0), vec![1.0, 0.0, 0.0]);
data.vectors
.insert(DocumentId::new(1, 0), vec![0.0, 1.0, 0.0]);
data
}
#[tokio::test]
async fn yaml_provider_empty_data_returns_nothing() {
let data = minimal_rag_data();
let provider = YamlProvider::from_data(&data);
let results = provider.fetch_content(&[]).await.unwrap();
assert!(results.is_empty());
}
#[tokio::test]
async fn yaml_provider_fetch_content_preserves_input_order() {
let data = populated_rag_data();
let provider = YamlProvider::from_data(&data);
let a = DocumentId::new(0, 0);
let b = DocumentId::new(1, 0);
let forward = provider.fetch_content(&[a, b]).await.unwrap();
assert_eq!(forward.len(), 2, "both documents must resolve");
assert_eq!(forward[0].1, "alpha");
assert_eq!(forward[1].1, "beta");
let reversed = provider.fetch_content(&[b, a]).await.unwrap();
assert_eq!(
reversed[0].1, "beta",
"fetch_content must honor input order"
);
assert_eq!(reversed[1].1, "alpha");
}
#[tokio::test]
async fn yaml_provider_fetch_content_skips_missing_ids() {
let data = populated_rag_data();
let provider = YamlProvider::from_data(&data);
let a = DocumentId::new(0, 0);
let missing = DocumentId::new(99, 0);
let b = DocumentId::new(1, 0);
let out = provider.fetch_content(&[a, missing, b]).await.unwrap();
assert_eq!(out.len(), 2, "missing id is skipped, not an error");
assert_eq!(out[0].1, "alpha");
assert_eq!(out[1].1, "beta");
}
#[tokio::test]
async fn yaml_provider_duplicate_returns_equivalent_content() {
let data = populated_rag_data();
let provider = YamlProvider::from_data(&data);
let dup = provider.duplicate(&data);
let ids = [DocumentId::new(0, 0), DocumentId::new(1, 0)];
let r1 = provider.fetch_content(&ids).await.unwrap();
let r2 = dup.fetch_content(&ids).await.unwrap();
assert_eq!(r1.len(), 2, "fixture must resolve both documents");
assert_eq!(
r1, r2,
"duplicate must resolve the same content as the original"
);
}
#[tokio::test]
async fn yaml_provider_content_is_keyed_on_files_not_vectors() {
let mut data = populated_rag_data();
let orphan = DocumentId::new(9, 0);
data.vectors.insert(orphan, vec![0.0, 0.0, 1.0]);
let provider = YamlProvider::from_data(&data);
let out = provider.fetch_content(&[orphan]).await.unwrap();
assert!(
out.is_empty(),
"an id present only in `vectors` must not resolve to content"
);
let real = provider
.fetch_content(&[DocumentId::new(0, 0), DocumentId::new(1, 0)])
.await
.unwrap();
assert_eq!(real.len(), 2, "file-backed documents must still resolve");
assert_eq!(real[0].1, "alpha");
assert_eq!(real[1].1, "beta");
}
}
+377 -27
View File
@@ -3,7 +3,7 @@ use crate::utils::decode_bin;
use ansi_colours::AsRGB; use ansi_colours::AsRGB;
use anyhow::{Context, Result, anyhow}; use anyhow::{Context, Result, anyhow};
use comfy_table::{CellAlignment, ContentArrangement, Table, presets::UTF8_FULL}; use comfy_table::{CellAlignment, ContentArrangement, Table, presets::UTF8_FULL};
use crossterm::style::{Color, Stylize}; use crossterm::style::{Color, SetForegroundColor, Stylize};
use crossterm::terminal; use crossterm::terminal;
use fancy_regex::{Captures, Regex}; use fancy_regex::{Captures, Regex};
use std::collections::HashMap; use std::collections::HashMap;
@@ -176,12 +176,166 @@ fn kind_pre_wraps(kind: LineKind) -> bool {
) )
} }
fn wrap_plain_content(content: &str, effective_width: usize) -> Vec<String> { fn wrap_styled_content(styled: &str, effective_width: usize) -> Vec<String> {
let effective_width = effective_width.max(1); let effective_width = effective_width.max(1);
textwrap::wrap(content, effective_width) let wrapped: Vec<String> = textwrap::wrap(styled, effective_width)
.into_iter() .into_iter()
.map(|c| c.into_owned()) .map(|c| c.into_owned())
.collect() .collect();
reflow_ansi_state(wrapped)
}
fn apply_base_style(styled: &str, fg: Color, bold: bool) -> String {
let mut opener = String::new();
if fg != Color::Reset {
opener.push_str(&SetForegroundColor(fg).to_string());
}
if bold {
opener.push_str("\x1b[1m");
}
if opener.is_empty() {
return styled.to_string();
}
let restored = styled
.replace("\x1b[0m", &format!("\x1b[0m{opener}"))
.replace("\x1b[39m", &format!("\x1b[39m{opener}"));
format!("{opener}{restored}\x1b[0m")
}
#[derive(Default, Clone, PartialEq, Eq)]
struct SgrState {
fg: Option<String>,
bold: bool,
italic: bool,
strike: bool,
underline: bool,
}
impl SgrState {
fn is_default(&self) -> bool {
self.fg.is_none() && !self.bold && !self.italic && !self.strike && !self.underline
}
fn as_opener(&self) -> String {
if self.is_default() {
return String::new();
}
let mut params: Vec<String> = Vec::new();
if let Some(fg) = &self.fg {
params.push(fg.clone());
}
if self.bold {
params.push("1".into());
}
if self.italic {
params.push("3".into());
}
if self.underline {
params.push("4".into());
}
if self.strike {
params.push("9".into());
}
format!("\x1b[{}m", params.join(";"))
}
fn apply(&mut self, params: &str) {
if params.is_empty() {
*self = Self::default();
return;
}
let parts: Vec<&str> = params.split(';').collect();
let mut i = 0;
while i < parts.len() {
let code = parts[i];
match code {
"0" | "" => *self = Self::default(),
"1" => self.bold = true,
"3" => self.italic = true,
"4" => self.underline = true,
"9" => self.strike = true,
"22" => self.bold = false,
"23" => self.italic = false,
"24" => self.underline = false,
"29" => self.strike = false,
"39" => self.fg = None,
"38" if i + 1 < parts.len() => match parts[i + 1] {
"5" if i + 2 < parts.len() => {
self.fg = Some(format!("38;5;{}", parts[i + 2]));
i += 2;
}
"2" if i + 4 < parts.len() => {
self.fg = Some(format!(
"38;2;{};{};{}",
parts[i + 2],
parts[i + 3],
parts[i + 4]
));
i += 4;
}
_ => {}
},
_ => {
if let Ok(n) = code.parse::<u8>()
&& ((30..=37).contains(&n) || (90..=97).contains(&n))
{
self.fg = Some(n.to_string());
}
}
}
i += 1;
}
}
}
fn parse_sgr_updates(line: &str, state: &mut SgrState) {
let bytes = line.as_bytes();
let mut i = 0;
while i < bytes.len() {
if bytes[i] == 0x1b && i + 1 < bytes.len() && bytes[i + 1] == b'[' {
let mut j = i + 2;
while j < bytes.len() && bytes[j] != b'm' {
if bytes[j] == 0x1b {
break;
}
j += 1;
}
if j < bytes.len() && bytes[j] == b'm' {
let params = &line[i + 2..j];
state.apply(params);
i = j + 1;
continue;
}
}
i += 1;
}
}
fn reflow_ansi_state(lines: Vec<String>) -> Vec<String> {
let mut result = Vec::with_capacity(lines.len());
let mut open_state = SgrState::default();
for line in lines {
let opener = open_state.as_opener();
let mut end_state = open_state.clone();
parse_sgr_updates(&line, &mut end_state);
let mut full = if opener.is_empty() {
line
} else {
format!("{opener}{line}")
};
if !end_state.is_default() {
full.push_str("\x1b[0m");
}
open_state = end_state;
result.push(full);
}
result
} }
fn split_indent(line: &str) -> (&str, &str) { fn split_indent(line: &str) -> (&str, &str) {
@@ -205,7 +359,7 @@ fn render_heading(line: &str, level: u8, styles: &MarkdownStyles) -> String {
format!("{} {inline}", "#".repeat(level as usize)) format!("{} {inline}", "#".repeat(level as usize))
}; };
format!("{indent}{}", body.with(color).bold()) format!("{indent}{}", apply_base_style(&body, color, true))
} }
fn render_blockquote(line: &str, styles: &MarkdownStyles, wrap_width: Option<u16>) -> String { fn render_blockquote(line: &str, styles: &MarkdownStyles, wrap_width: Option<u16>) -> String {
@@ -213,16 +367,16 @@ fn render_blockquote(line: &str, styles: &MarkdownStyles, wrap_width: Option<u16
let content = rest.trim_start_matches('>').trim_start(); let content = rest.trim_start_matches('>').trim_start();
let prefix = "".with(styles.blockquote).to_string(); let prefix = "".with(styles.blockquote).to_string();
let render_one = |c: &str| apply_inline(c, styles).with(styles.blockquote).to_string(); let styled = apply_base_style(&apply_inline(content, styles), styles.blockquote, false);
let Some(wrap_width) = wrap_width else { let Some(wrap_width) = wrap_width else {
return format!("{indent}{prefix}{}", render_one(content)); return format!("{indent}{prefix}{styled}");
}; };
let prefix_width = 2; let prefix_width = 2;
let leading_width = indent.chars().count(); let leading_width = indent.chars().count();
let effective_width = (wrap_width as usize).saturating_sub(leading_width + prefix_width); let effective_width = (wrap_width as usize).saturating_sub(leading_width + prefix_width);
let wrapped = wrap_plain_content(content, effective_width); let wrapped = wrap_styled_content(&styled, effective_width);
if wrapped.is_empty() { if wrapped.is_empty() {
return format!("{indent}{prefix}"); return format!("{indent}{prefix}");
} }
@@ -232,7 +386,7 @@ fn render_blockquote(line: &str, styles: &MarkdownStyles, wrap_width: Option<u16
if i > 0 { if i > 0 {
out.push('\n'); out.push('\n');
} }
out.push_str(&format!("{indent}{prefix}{}", render_one(chunk))); out.push_str(&format!("{indent}{prefix}{chunk}"));
} }
out out
@@ -250,7 +404,8 @@ fn render_bullet(line: &str, styles: &MarkdownStyles, wrap_width: Option<u16>) -
let prefix_width = 2; let prefix_width = 2;
let leading_width = indent.chars().count(); let leading_width = indent.chars().count();
let effective_width = (wrap_width as usize).saturating_sub(leading_width + prefix_width); let effective_width = (wrap_width as usize).saturating_sub(leading_width + prefix_width);
let wrapped = wrap_plain_content(content, effective_width); let styled = apply_inline(content, styles);
let wrapped = wrap_styled_content(&styled, effective_width);
if wrapped.is_empty() { if wrapped.is_empty() {
return format!("{indent}{bullet} "); return format!("{indent}{bullet} ");
} }
@@ -261,11 +416,10 @@ fn render_bullet(line: &str, styles: &MarkdownStyles, wrap_width: Option<u16>) -
if i > 0 { if i > 0 {
out.push('\n'); out.push('\n');
} }
let styled = apply_inline(chunk, styles);
if i == 0 { if i == 0 {
out.push_str(&format!("{indent}{bullet} {styled}")); out.push_str(&format!("{indent}{bullet} {chunk}"));
} else { } else {
out.push_str(&format!("{indent}{subseq}{styled}")); out.push_str(&format!("{indent}{subseq}{chunk}"));
} }
} }
@@ -291,7 +445,8 @@ fn render_numbered(line: &str, styles: &MarkdownStyles, wrap_width: Option<u16>)
let prefix_width = number.chars().count() + 2; let prefix_width = number.chars().count() + 2;
let leading_width = indent.chars().count(); let leading_width = indent.chars().count();
let effective_width = (wrap_width as usize).saturating_sub(leading_width + prefix_width); let effective_width = (wrap_width as usize).saturating_sub(leading_width + prefix_width);
let wrapped = wrap_plain_content(after, effective_width); let styled = apply_inline(after, styles);
let wrapped = wrap_styled_content(&styled, effective_width);
if wrapped.is_empty() { if wrapped.is_empty() {
return format!("{indent}{number}{styled_dot} "); return format!("{indent}{number}{styled_dot} ");
} }
@@ -302,11 +457,10 @@ fn render_numbered(line: &str, styles: &MarkdownStyles, wrap_width: Option<u16>)
if i > 0 { if i > 0 {
out.push('\n'); out.push('\n');
} }
let styled = apply_inline(chunk, styles);
if i == 0 { if i == 0 {
out.push_str(&format!("{indent}{number}{styled_dot} {styled}")); out.push_str(&format!("{indent}{number}{styled_dot} {chunk}"));
} else { } else {
out.push_str(&format!("{indent}{subseq}{styled}")); out.push_str(&format!("{indent}{subseq}{chunk}"));
} }
} }
@@ -335,7 +489,8 @@ fn render_task(
let prefix_width = 4; let prefix_width = 4;
let leading_width = indent.chars().count(); let leading_width = indent.chars().count();
let effective_width = (wrap_width as usize).saturating_sub(leading_width + prefix_width); let effective_width = (wrap_width as usize).saturating_sub(leading_width + prefix_width);
let wrapped = wrap_plain_content(after_brackets, effective_width); let styled = apply_inline(after_brackets, styles);
let wrapped = wrap_styled_content(&styled, effective_width);
if wrapped.is_empty() { if wrapped.is_empty() {
return format!("{indent}{styled_brackets} "); return format!("{indent}{styled_brackets} ");
} }
@@ -346,11 +501,10 @@ fn render_task(
if i > 0 { if i > 0 {
out.push('\n'); out.push('\n');
} }
let styled = apply_inline(chunk, styles);
if i == 0 { if i == 0 {
out.push_str(&format!("{indent}{styled_brackets} {styled}")); out.push_str(&format!("{indent}{styled_brackets} {chunk}"));
} else { } else {
out.push_str(&format!("{indent}{subseq}{styled}")); out.push_str(&format!("{indent}{subseq}{chunk}"));
} }
} }
@@ -683,12 +837,7 @@ impl MarkdownRender {
let (heading_color, _) = self.styles.heading; let (heading_color, _) = self.styles.heading;
let styled_header: Vec<String> = header let styled_header: Vec<String> = header
.iter() .iter()
.map(|c| { .map(|c| apply_base_style(&apply_inline(c, &self.styles), heading_color, true))
apply_inline(c, &self.styles)
.with(heading_color)
.bold()
.to_string()
})
.collect(); .collect();
table.set_header(styled_header); table.set_header(styled_header);
@@ -1600,6 +1749,86 @@ std::error::Error>> {
); );
} }
fn strip_ansi(text: &str) -> String {
let mut out = String::with_capacity(text.len());
let mut chars = text.chars();
while let Some(c) = chars.next() {
if c != '\u{1b}' {
out.push(c);
continue;
}
if chars.next() == Some('[') {
for c in chars.by_ref() {
if ('\u{40}'..='\u{7e}').contains(&c) {
break;
}
}
}
}
out
}
#[test]
fn strip_ansi_removes_sgr_and_keeps_text() {
assert_eq!(strip_ansi("\x1b[1mbold\x1b[0m"), "bold");
assert_eq!(strip_ansi("\x1b[38;5;120mx\x1b[39m"), "x");
assert_eq!(strip_ansi("plain"), "plain");
}
#[test]
fn render_table_pads_columns_by_display_width() {
use unicode_width::UnicodeWidthStr;
const WRAP_WIDTH: u16 = 80;
let options = RenderOptions::default();
let mut render = MarkdownRender::init(options).unwrap();
render.wrap_width = Some(WRAP_WIDTH);
let header = vec![
"**Setting**".into(),
"*Default*".into(),
"`Description`".into(),
];
let alignments = vec![
CellAlignment::Left,
CellAlignment::Right,
CellAlignment::Center,
];
let rows = vec![
vec![
"**temperature**".into(),
"`0.7`".into(),
"Controls how *random* the sampled reply is allowed to be".into(),
],
vec![
"**top_p**".into(),
"`1.0`".into(),
"Nucleus sampling cutoff, applied **after** temperature".into(),
],
];
let output = render.render_table(header, alignments, rows);
assert!(
output.contains('\u{1b}'),
"fixture must actually contain ANSI escapes: {output:?}",
);
let widths: Vec<usize> = output
.lines()
.map(|line| strip_ansi(line).width())
.collect();
assert!(!widths.is_empty(), "table rendered no lines");
for (index, width) in widths.iter().enumerate() {
assert_eq!(
*width, WRAP_WIDTH as usize,
"line {index} display width; all widths were {widths:?} in output:\n{output}",
);
}
}
#[test] #[test]
fn state_machine_renders_full_table_and_flushes_on_paragraph() { fn state_machine_renders_full_table_and_flushes_on_paragraph() {
let options = RenderOptions::default(); let options = RenderOptions::default();
@@ -1863,6 +2092,127 @@ std::error::Error>> {
assert!(output.contains("code")); assert!(output.contains("code"));
} }
#[test]
fn bullet_wraps_bold_span_across_boundary() {
let styles = test_styles();
let line = "- **explain some code, `value`, using _this_**";
let output = render_markdown_line(line, LineKind::BulletItem, &styles, Some(30));
assert!(output.contains('\n'), "must wrap: {output:?}");
assert!(
!output.contains("**"),
"bold markers stripped even when span crosses wrap: {output:?}"
);
assert!(output.contains("\x1b[1m"), "bold SGR present: {output:?}");
assert!(output.contains("explain"));
assert!(output.contains("this"));
}
#[test]
fn blockquote_wraps_bold_span_across_boundary() {
let styles = test_styles();
let line = "> **explain some code, `value`, using _this_**";
let output = render_markdown_line(line, LineKind::Blockquote, &styles, Some(30));
assert!(output.contains('\n'), "must wrap: {output:?}");
assert!(!output.contains("**"), "bold markers stripped: {output:?}");
assert!(output.contains("\x1b[1m"), "bold SGR: {output:?}");
}
#[test]
fn blockquote_restores_base_color_after_inner_reset() {
let styles = test_styles();
let line = "> hello **bold** world after `code` more text";
let output = render_markdown_line(line, LineKind::Blockquote, &styles, None);
let base_open = SetForegroundColor(styles.blockquote).to_string();
let re_opens = output.matches(base_open.as_str()).count();
assert!(
re_opens >= 3,
"blockquote color re-opened after each inner reset (got {re_opens}): {output:?}"
);
}
#[test]
fn heading_restores_base_style_after_inner_reset() {
let styles = test_styles();
let line = "## title with **bold** and `code` trailing text";
let output = render_markdown_line(line, LineKind::Heading(2), &styles, None);
let (heading_color, _) = styles.heading;
let base_open = format!("{}\x1b[1m", SetForegroundColor(heading_color));
let re_opens = output.matches(base_open.as_str()).count();
assert!(
re_opens >= 3,
"heading color+bold re-opened after each inner reset (got {re_opens}): {output:?}"
);
}
#[test]
fn table_header_restores_base_style_after_inner_reset() {
let options = RenderOptions::default();
let mut render = MarkdownRender::init(options).unwrap();
let table = "| col with **bold** inside | second |\n|---|---|\n| a | b |\n";
let output = render.render(table);
let header_row = output
.lines()
.find(|l| l.contains("col with"))
.expect("header row present");
let bold_opens = header_row.matches("\x1b[1m").count();
assert!(
bold_opens >= 3,
"table header bold re-opened after inner reset (got {bold_opens}): {header_row:?}"
);
}
#[test]
fn numbered_wraps_bold_span_across_boundary() {
let styles = test_styles();
let line = "1. **explain some code, `value`, using _this_**";
let output = render_markdown_line(line, LineKind::NumberedItem, &styles, Some(30));
assert!(output.contains('\n'), "must wrap: {output:?}");
assert!(!output.contains("**"), "bold markers stripped: {output:?}");
assert!(output.contains("\x1b[1m"), "bold SGR: {output:?}");
}
#[test]
fn task_wraps_bold_span_across_boundary() {
let styles = test_styles();
let line = "- [ ] **explain some code, `value`, using _this_**";
let output = render_markdown_line(line, LineKind::TaskItem(false), &styles, Some(32));
assert!(output.contains('\n'), "must wrap: {output:?}");
assert!(!output.contains("**"), "bold markers stripped: {output:?}");
assert!(output.contains("\x1b[1m"), "bold SGR: {output:?}");
}
#[test]
fn wrapped_bold_line_closes_and_reopens_ansi_across_lines() {
let styles = test_styles();
let line = "- **one two three four five six seven eight nine**";
let output = render_markdown_line(line, LineKind::BulletItem, &styles, Some(20));
assert!(output.contains('\n'), "must wrap: {output:?}");
assert!(!output.contains("**"), "bold markers stripped: {output:?}");
for wrapped_line in output.split('\n') {
if wrapped_line.contains("\x1b[1m") {
assert!(
wrapped_line.contains("\x1b[0m"),
"each bold line closes with reset: {wrapped_line:?}"
);
}
}
}
#[test] #[test]
fn mixed_content_renders_all_kinds() { fn mixed_content_renders_all_kinds() {
let options = RenderOptions::default(); let options = RenderOptions::default();
+111 -27
View File
@@ -7,7 +7,10 @@ use self::completer::ReplCompleter;
use self::highlighter::ReplHighlighter; use self::highlighter::ReplHighlighter;
use self::prompt::ReplPrompt; use self::prompt::ReplPrompt;
use crate::client::{call_chat_completions, call_chat_completions_streaming, init_client, oauth}; use crate::client::{
Message, MessageRole, call_chat_completions, call_chat_completions_streaming, init_client,
oauth,
};
use crate::config::{ use crate::config::{
AgentVariables, AppConfig, AssertState, Input, LastMessage, RequestContext, StateFlags, AgentVariables, AppConfig, AssertState, Input, LastMessage, RequestContext, StateFlags,
macro_execute, macro_execute,
@@ -50,7 +53,7 @@ pub const DEFAULT_CONTINUATION_PROMPT: &str = indoc! {"
4. Continue with the next pending item now. Call tools immediately." 4. Continue with the next pending item now. Call tools immediately."
}; };
static REPL_COMMANDS: LazyLock<[ReplCommand; 58]> = LazyLock::new(|| { static REPL_COMMANDS: LazyLock<[ReplCommand; 60]> = LazyLock::new(|| {
[ [
ReplCommand::new(".help", "Show this help guide", AssertState::pass()), ReplCommand::new(".help", "Show this help guide", AssertState::pass()),
ReplCommand::new(".info", "Show system info", AssertState::pass()), ReplCommand::new(".info", "Show system info", AssertState::pass()),
@@ -214,6 +217,11 @@ static REPL_COMMANDS: LazyLock<[ReplCommand; 58]> = LazyLock::new(|| {
"Initialize or access RAG", "Initialize or access RAG",
AssertState::False(StateFlags::AGENT), AssertState::False(StateFlags::AGENT),
), ),
ReplCommand::new(
".rag attach",
"Attach to a pre-existing external RAG",
AssertState::False(StateFlags::AGENT),
),
ReplCommand::new( ReplCommand::new(
".edit rag-docs", ".edit rag-docs",
"Add or remove documents from an existing RAG", "Add or remove documents from an existing RAG",
@@ -275,6 +283,11 @@ static REPL_COMMANDS: LazyLock<[ReplCommand; 58]> = LazyLock::new(|| {
"Continue previous response", "Continue previous response",
AssertState::pass(), AssertState::pass(),
), ),
ReplCommand::new(
".recover",
"Recover interrupted session after API error or Ctrl+C",
AssertState::True(StateFlags::SESSION_EMPTY | StateFlags::SESSION),
),
ReplCommand::new( ReplCommand::new(
".regenerate", ".regenerate",
"Regenerate last response", "Regenerate last response",
@@ -370,6 +383,31 @@ Type ".help" for additional help.
if !compressed.is_empty() || !active.is_empty() { if !compressed.is_empty() || !active.is_empty() {
let app = Arc::clone(&self.ctx.read().app.config); let app = Arc::clone(&self.ctx.read().app.config);
replay::render(app.as_ref(), &compressed, &active)?; replay::render(app.as_ref(), &compressed, &active)?;
let last_msgs: &[Message] = if !active.is_empty() {
&active
} else {
&compressed
};
let assistant_text = last_msgs
.iter()
.rev()
.find(|m| m.role == MessageRole::Assistant)
.and_then(|m| m.content.as_text())
.map(str::to_string);
if let Some(output) = assistant_text {
let user_text = last_msgs
.iter()
.rev()
.find(|m| m.role == MessageRole::User)
.and_then(|m| m.content.as_text())
.unwrap_or("")
.to_string();
let ctx = self.ctx.read();
if let Ok(input) = Input::from_str(&ctx, &user_text, None) {
drop(ctx);
self.ctx.write().last_message = Some(LastMessage::new(input, output));
}
}
} }
} }
@@ -851,9 +889,17 @@ pub async fn run_repl_command(
let version = args.map(|s| s.trim().to_string()); let version = args.map(|s| s.trim().to_string());
task::spawn_blocking(move || config::run_self_update(version, false)).await??; task::spawn_blocking(move || config::run_self_update(version, false)).await??;
} }
".rag" => { ".rag" => match split_first_arg(args) {
ctx.use_rag(args, abort_signal.clone()).await?; Some(("attach", rest)) => match rest {
} Some(name) if !name.trim().is_empty() => {
ctx.attach_rag(name.trim(), abort_signal.clone()).await?;
}
_ => println!("Usage: .rag attach <name>"),
},
_ => {
ctx.use_rag(args, abort_signal.clone()).await?;
}
},
".agent" => match split_first_arg(args) { ".agent" => match split_first_arg(args) {
Some((agent_name, args)) => { Some((agent_name, args)) => {
let (new_args, _) = split_args_text(args.unwrap_or_default(), cfg!(windows)); let (new_args, _) = split_args_text(args.unwrap_or_default(), cfg!(windows));
@@ -1073,6 +1119,21 @@ pub async fn run_repl_command(
input.set_continue_output(&output); input.set_continue_output(&output);
ask(ctx, abort_signal.clone(), input, true).await?; ask(ctx, abort_signal.clone(), input, true).await?;
} }
".recover" => {
let has_recoverable = ctx
.last_message
.as_ref()
.map(|v| v.continuous && v.input.with_session())
.unwrap_or(false);
if !has_recoverable {
bail!("Unable to recover: no interrupted session response to recover from");
}
let recovery_text = args
.unwrap_or("Please continue from where you left off.")
.to_string();
let recovery_input = Input::from_str(ctx, &recovery_text, None)?;
ask(ctx, abort_signal.clone(), recovery_input, false).await?;
}
".regenerate" => { ".regenerate" => {
let LastMessage { mut input, .. } = let LastMessage { mut input, .. } =
match ctx.last_message.as_ref().filter(|v| v.continuous).cloned() { match ctx.last_message.as_ref().filter(|v| v.continuous).cloned() {
@@ -1119,16 +1180,28 @@ pub async fn run_repl_command(
} }
}, },
".copy" => { ".copy" => {
let output = match ctx let output = ctx
.last_message .last_message
.as_ref() .as_ref()
.filter(|v| !v.output.is_empty()) .filter(|v| !v.output.is_empty())
.map(|v| v.output.clone()) .map(|v| v.output.clone())
{ .or_else(|| {
Some(v) => v, ctx.session.as_ref().and_then(|s| {
None => bail!("No chat response to copy"), s.messages()
}; .iter()
set_text(&output).context("Failed to copy the last chat response")?; .rev()
.chain(s.compressed_messages().iter().rev())
.find(|m| m.role == MessageRole::Assistant)
.and_then(|m| m.content.as_text())
.map(str::to_string)
})
});
match output {
Some(v) if !v.is_empty() => {
set_text(&v).context("Failed to copy the last chat response")?;
}
_ => bail!("No chat response to copy"),
}
} }
".exit" => match args { ".exit" => match args {
Some("role") => { Some("role") => {
@@ -1267,18 +1340,28 @@ async fn ask(
let client = input.create_client()?; let client = input.create_client()?;
ctx.before_chat_completion(&input)?; ctx.before_chat_completion(&input)?;
let (output, tool_results) = if input.stream() { let (output, tool_results) = {
call_chat_completions_streaming(&input, client.as_ref(), ctx, abort_signal.clone()).await? let result = if input.stream() {
} else { call_chat_completions_streaming(&input, client.as_ref(), ctx, abort_signal.clone())
call_chat_completions( .await
&input, } else {
true, call_chat_completions(
false, &input,
client.as_ref(), true,
ctx, false,
abort_signal.clone(), client.as_ref(),
) ctx,
.await? abort_signal.clone(),
)
.await
};
match result {
Ok(v) => v,
Err(err) => {
ctx.on_chat_completion_error(app.as_ref(), &input);
return Err(err);
}
}
}; };
ctx.after_chat_completion(app.as_ref(), &input, &output, &tool_results)?; ctx.after_chat_completion(app.as_ref(), &input, &output, &tool_results)?;
if !tool_results.is_empty() { if !tool_results.is_empty() {
@@ -1304,7 +1387,7 @@ async fn ask(
} }
GuardrailAction::NoAction => {} GuardrailAction::NoAction => {}
} }
let do_continue = should_continue(ctx); let do_continue = should_continue(ctx) && !abort_signal.aborted_ctrlc();
if do_continue { if do_continue {
let full_prompt = { let full_prompt = {
@@ -1360,7 +1443,8 @@ async fn ask(
.is_some_and(|s| s.needs_compression(app.compression_threshold)); .is_some_and(|s| s.needs_compression(app.compression_threshold));
if needs_compression { if needs_compression {
let agent_can_continue_after_compress = should_continue(ctx); let agent_can_continue_after_compress =
should_continue(ctx) && !abort_signal.aborted_ctrlc();
if let Some(session) = ctx.session.as_mut() { if let Some(session) = ctx.session.as_mut() {
session.set_compressing(true); session.set_compressing(true);
@@ -1640,8 +1724,8 @@ mod tests {
} }
#[test] #[test]
fn repl_commands_has_58_entries() { fn repl_commands_has_60_entries() {
assert_eq!(REPL_COMMANDS.len(), 58); assert_eq!(REPL_COMMANDS.len(), 60);
} }
#[test] #[test]
+3 -2
View File
@@ -2,7 +2,7 @@ use anyhow::Result;
use crate::client::{Message, MessageRole}; use crate::client::{Message, MessageRole};
use crate::config::{AppConfig, Session}; use crate::config::{AppConfig, Session};
use crate::utils::dimmed_text; use crate::utils::{dimmed_text, replay_label_text};
pub fn snapshot(session: &Session) -> (Vec<Message>, Vec<Message>) { pub fn snapshot(session: &Session) -> (Vec<Message>, Vec<Message>) {
( (
@@ -40,13 +40,14 @@ fn render_messages(app: &AppConfig, messages: &[Message]) -> Result<()> {
match message.role { match message.role {
MessageRole::User => { MessageRole::User => {
if let Some(text) = message.content.as_text() { if let Some(text) = message.content.as_text() {
println!("{}", dimmed_text("You:")); println!("{}", replay_label_text("You:"));
println!("{text}"); println!("{text}");
println!(); println!();
} }
} }
MessageRole::Assistant => { MessageRole::Assistant => {
if let Some(text) = message.content.as_text() { if let Some(text) = message.content.as_text() {
println!("{}", replay_label_text("Assistant:"));
app.print_markdown(text)?; app.print_markdown(text)?;
println!(); println!();
} }
File diff suppressed because it is too large Load Diff
+580 -29
View File
@@ -1,16 +1,19 @@
use std::env; use std::env;
use std::fs; use std::fs;
use std::fs::{read_dir, read_to_string}; use std::fs::{read_dir, read_to_string};
use std::io;
use std::path::{Path, PathBuf}; use std::path::{Path, PathBuf};
use anyhow::{Context, Result}; use anyhow::{Context, Result, anyhow, bail};
use serde_yaml::Value; use serde_yaml::Value;
use sha2::{Digest, Sha256}; use sha2::{Digest, Sha256};
use crate::config::paths; use crate::config::paths;
const SBX_MIXIN_FILE_NAME: &str = "sbx-mixin.yaml"; const SBX_MIXIN_FILE_NAME: &str = "sbx-mixin.yaml";
const SBX_MIXIN_FILE_SUFFIX: &str = ".sbx-mixin.yaml";
const KIT_SPEC_FILE_NAME: &str = "spec.yaml"; const KIT_SPEC_FILE_NAME: &str = "spec.yaml";
const MIXIN_FILES_DIR_NAME: &str = "files";
#[derive(Debug, Clone)] #[derive(Debug, Clone)]
pub struct DiscoveredMixin { pub struct DiscoveredMixin {
@@ -33,43 +36,168 @@ impl DiscoveredMixin {
pub fn wrap_mixin_as_kit(mixin_path: &Path) -> Result<PathBuf> { pub fn wrap_mixin_as_kit(mixin_path: &Path) -> Result<PathBuf> {
let bytes = fs::read(mixin_path) let bytes = fs::read(mixin_path)
.with_context(|| format!("Failed to read sbx mixin {}", mixin_path.display()))?; .with_context(|| format!("Failed to read sbx mixin {}", mixin_path.display()))?;
let label = mixin_path.display().to_string();
let files = mixin_path
.parent()
.map(|p| p.join(MIXIN_FILES_DIR_NAME))
.filter(|p| p.is_dir())
.map(|dir| collect_staged_files(&dir))
.transpose()?
.unwrap_or_default();
stage_kit(&bytes, &files, &label)
}
pub fn wrap_mixin_bytes_as_kit(bytes: &[u8], label: &str) -> Result<PathBuf> {
stage_kit(bytes, &[], label)
}
struct StagedFile {
relpath: PathBuf,
mode: u32,
bytes: Vec<u8>,
}
fn stage_kit(spec_bytes: &[u8], files: &[StagedFile], label: &str) -> Result<PathBuf> {
let mut hasher = Sha256::new(); let mut hasher = Sha256::new();
hasher.update(&bytes); hasher.update(spec_bytes);
for f in files {
let rel_str = f.relpath.to_str().ok_or_else(|| {
anyhow!(
"Non-UTF-8 path inside mixin {MIXIN_FILES_DIR_NAME}/: {}",
f.relpath.display()
)
})?;
hasher.update(b"\0COYOTE_MIXIN_FILE\0");
hasher.update((rel_str.len() as u64).to_le_bytes());
hasher.update(rel_str.as_bytes());
hasher.update(f.mode.to_le_bytes());
hasher.update((f.bytes.len() as u64).to_le_bytes());
hasher.update(&f.bytes);
}
let hash = format!("{:x}", hasher.finalize()); let hash = format!("{:x}", hasher.finalize());
let kit_dir = paths::sbx_mixin_kits_dir().join(&hash); let kit_dir = paths::sbx_mixin_kits_dir().join(&hash);
let spec_path = kit_dir.join(KIT_SPEC_FILE_NAME); let spec_path = kit_dir.join(KIT_SPEC_FILE_NAME);
let files_dst = kit_dir.join(MIXIN_FILES_DIR_NAME);
if let Ok(existing) = fs::read(&spec_path) let spec_matches = fs::read(&spec_path).is_ok_and(|existing| existing == spec_bytes);
&& existing == bytes let files_ready = files.is_empty() || files_dst.is_dir();
{ if spec_matches && files_ready {
return Ok(kit_dir); return Ok(kit_dir);
} }
fs::create_dir_all(&kit_dir) fs::create_dir_all(&kit_dir)
.with_context(|| format!("Failed to create mixin kit dir {}", kit_dir.display()))?; .with_context(|| format!("Failed to create mixin kit dir {}", kit_dir.display()))?;
fs::write(&spec_path, &bytes) fs::write(&spec_path, spec_bytes)
.with_context(|| format!("Failed to write {}", spec_path.display()))?; .with_context(|| format!("Failed to write {}", spec_path.display()))?;
debug!( if !files.is_empty() {
"Wrapped mixin {} as kit at {}", if files_dst.exists() {
mixin_path.display(), fs::remove_dir_all(&files_dst).with_context(|| {
kit_dir.display() format!(
); "Failed to clear stale mixin files at {}",
files_dst.display()
)
})?;
}
for f in files {
let dst = files_dst.join(&f.relpath);
if let Some(parent) = dst.parent() {
fs::create_dir_all(parent)
.with_context(|| format!("Failed to create dir {}", parent.display()))?;
}
fs::write(&dst, &f.bytes)
.with_context(|| format!("Failed to write staged mixin file {}", dst.display()))?;
#[cfg(unix)]
{
use std::os::unix::fs::PermissionsExt;
fs::set_permissions(&dst, fs::Permissions::from_mode(f.mode))
.with_context(|| format!("Failed to set mode on {}", dst.display()))?;
}
}
}
debug!("Wrapped mixin {label} as kit at {}", kit_dir.display());
Ok(kit_dir) Ok(kit_dir)
} }
fn collect_staged_files(root: &Path) -> Result<Vec<StagedFile>> {
let mut out = Vec::new();
walk_staged_files(root, Path::new(""), &mut out)?;
Ok(out)
}
fn walk_staged_files(abs_dir: &Path, rel_dir: &Path, out: &mut Vec<StagedFile>) -> Result<()> {
let rd = fs::read_dir(abs_dir)
.with_context(|| format!("Failed to read mixin files dir {}", abs_dir.display()))?;
let mut entries: Vec<_> = rd
.collect::<io::Result<Vec<_>>>()
.with_context(|| format!("Failed to iterate mixin files dir {}", abs_dir.display()))?;
entries.sort_by_key(|e| e.file_name());
for entry in entries {
let file_type = entry
.file_type()
.with_context(|| format!("Failed to stat {}", entry.path().display()))?;
let abs = entry.path();
let rel = rel_dir.join(entry.file_name());
if file_type.is_symlink() {
bail!(
"Symlinks are not allowed inside a mixin {MIXIN_FILES_DIR_NAME}/ tree: {}",
abs.display()
);
}
if file_type.is_dir() {
walk_staged_files(&abs, &rel, out)?;
} else if file_type.is_file() {
let bytes = fs::read(&abs)
.with_context(|| format!("Failed to read staged mixin file {}", abs.display()))?;
let mode = staged_file_mode(&entry)?;
out.push(StagedFile {
relpath: rel,
mode,
bytes,
});
}
}
Ok(())
}
#[cfg(unix)]
fn staged_file_mode(entry: &fs::DirEntry) -> Result<u32> {
use std::os::unix::fs::PermissionsExt;
let meta = entry
.metadata()
.with_context(|| format!("Failed to stat {}", entry.path().display()))?;
Ok(meta.permissions().mode() & 0o777)
}
#[cfg(not(unix))]
fn staged_file_mode(_entry: &fs::DirEntry) -> Result<u32> {
Ok(0o644)
}
pub fn discover() -> Result<Vec<DiscoveredMixin>> { pub fn discover() -> Result<Vec<DiscoveredMixin>> {
let mut out = Vec::new(); let mut out = Vec::new();
push_if_exists(&mut out, paths::sbx_mixin_file())?; push_if_exists(&mut out, paths::sbx_mixin_file())?;
push_if_exists(&mut out, paths::global_tools_sbx_mixin_file())?; push_if_exists(&mut out, paths::global_tools_sbx_mixin_file())?;
for path in collect_subdir_mixins(&paths::functions_dir()) { for path in collect_mixins(&paths::functions_dir(), &[ScanMode::SubdirNamed]) {
out.push(read_mixin(path)?); out.push(read_mixin(path)?);
} }
for path in collect_subdir_mixins(&paths::agents_data_dir()) { for path in collect_mixins(
&paths::agents_data_dir(),
&[ScanMode::SubdirNamed, ScanMode::SubdirFlat],
) {
out.push(read_mixin(path)?);
}
for path in collect_mixins(&paths::rags_dir(), &[ScanMode::Flat]) {
out.push(read_mixin(path)?); out.push(read_mixin(path)?);
} }
@@ -89,15 +217,18 @@ pub fn summarize(path: &Path) -> Result<(usize, usize)> {
.with_context(|| format!("Failed to parse sbx mixin {}", path.display()))?; .with_context(|| format!("Failed to parse sbx mixin {}", path.display()))?;
let installs = value let installs = value
.get("commands") .get("setup")
.and_then(|c| c.get("install")) .and_then(|s| s.get("install"))
.or_else(|| value.get("commands").and_then(|c| c.get("install")))
.and_then(|i| i.as_sequence()) .and_then(|i| i.as_sequence())
.map(|s| s.len()) .map(|s| s.len())
.unwrap_or(0); .unwrap_or(0);
let domains = value let domains = value
.get("network") .get("permissions")
.and_then(|n| n.get("allowedDomains")) .and_then(|p| p.get("network"))
.and_then(|n| n.get("allow"))
.or_else(|| value.get("network").and_then(|n| n.get("allowedDomains")))
.and_then(|d| d.as_sequence()) .and_then(|d| d.as_sequence())
.map(|s| s.len()) .map(|s| s.len())
.unwrap_or(0); .unwrap_or(0);
@@ -153,7 +284,73 @@ fn read_mixin(path: PathBuf) -> Result<DiscoveredMixin> {
}) })
} }
fn collect_subdir_mixins(dir: &Path) -> Vec<PathBuf> { /// One on-disk layout a mixin scan can look for. A scan takes a set of these,
/// and each mode contributes only the shape it names.
#[derive(Debug, Clone, Copy, PartialEq, Eq)]
enum ScanMode {
/// `<dir>/*.sbx-mixin.yaml`
Flat,
/// `<dir>/*/sbx-mixin.yaml`
SubdirNamed,
/// `<dir>/*/*.sbx-mixin.yaml`
SubdirFlat,
}
/// Collects mixin paths under `dir` for every requested layout. Missing or
/// unreadable directories yield nothing rather than an error — these paths are
/// all optional on disk.
///
/// Order is deterministic: flat matches first (sorted by file name), then each
/// subdirectory in sorted order, contributing its named mixin before its
/// suffixed ones.
fn collect_mixins(dir: &Path, modes: &[ScanMode]) -> Vec<PathBuf> {
let mut result = Vec::new();
if modes.contains(&ScanMode::Flat) {
result.extend(suffixed_mixins_in(dir));
}
let named = modes.contains(&ScanMode::SubdirNamed);
let subdir_flat = modes.contains(&ScanMode::SubdirFlat);
if !named && !subdir_flat {
return result;
}
for subdir in subdirs_of(dir) {
if named {
let candidate = subdir.join(SBX_MIXIN_FILE_NAME);
if candidate.exists() {
result.push(candidate);
}
}
if subdir_flat {
result.extend(suffixed_mixins_in(&subdir));
}
}
result
}
fn suffixed_mixins_in(dir: &Path) -> Vec<PathBuf> {
let mut result = Vec::new();
let Ok(rd) = read_dir(dir) else { return result };
let mut entries: Vec<_> = rd
.flatten()
.filter(|e| e.file_type().map(|t| t.is_file()).unwrap_or(false))
.filter(|e| {
e.file_name()
.to_str()
.is_some_and(|n| n.ends_with(SBX_MIXIN_FILE_SUFFIX))
})
.collect();
entries.sort_by_key(|e| e.file_name());
result.extend(entries.into_iter().map(|e| e.path()));
result
}
fn subdirs_of(dir: &Path) -> Vec<PathBuf> {
let mut result = Vec::new(); let mut result = Vec::new();
let Ok(rd) = read_dir(dir) else { return result }; let Ok(rd) = read_dir(dir) else { return result };
@@ -163,13 +360,7 @@ fn collect_subdir_mixins(dir: &Path) -> Vec<PathBuf> {
.collect(); .collect();
entries.sort_by_key(|e| e.file_name()); entries.sort_by_key(|e| e.file_name());
for entry in entries { result.extend(entries.into_iter().map(|e| e.path()));
let candidate = entry.path().join(SBX_MIXIN_FILE_NAME);
if candidate.exists() {
result.push(candidate);
}
}
result result
} }
@@ -189,6 +380,13 @@ mod tests {
root root
} }
fn file_names(paths: &[PathBuf]) -> Vec<&str> {
paths
.iter()
.map(|p| p.file_name().unwrap().to_str().unwrap())
.collect()
}
#[test] #[test]
fn summarize_counts_installs_and_domains() { fn summarize_counts_installs_and_domains() {
let root = unique_root("sbx-mixin-counts"); let root = unique_root("sbx-mixin-counts");
@@ -196,6 +394,34 @@ mod tests {
fs::write( fs::write(
&path, &path,
r#" r#"
schemaVersion: "2"
kind: mixin
setup:
install:
- command: "echo hi"
- command: "echo bye"
permissions:
network:
allow:
- "a.example.com:443"
- "b.example.com:443"
- "c.example.com:443"
"#,
)
.unwrap();
assert_eq!(summarize(&path).unwrap(), (2, 3));
let _ = fs::remove_dir_all(&root);
}
#[test]
fn summarize_falls_back_to_v1_field_paths() {
let root = unique_root("sbx-mixin-counts-v1");
let path = root.join("sbx-mixin.yaml");
fs::write(
&path,
r#"
schemaVersion: "1" schemaVersion: "1"
kind: mixin kind: mixin
commands: commands:
@@ -244,7 +470,7 @@ network:
} }
#[test] #[test]
fn collect_subdir_mixins_sorts_and_skips_missing() { fn subdir_named_scan_sorts_and_skips_missing() {
let root = unique_root("sbx-mixin-subdirs"); let root = unique_root("sbx-mixin-subdirs");
for name in ["zebra", "apple", "no-mixin", "mango"] { for name in ["zebra", "apple", "no-mixin", "mango"] {
let dir = root.join(name); let dir = root.join(name);
@@ -254,7 +480,7 @@ network:
} }
} }
let found = collect_subdir_mixins(&root); let found = collect_mixins(&root, &[ScanMode::SubdirNamed]);
let names: Vec<String> = found let names: Vec<String> = found
.iter() .iter()
.map(|p| { .map(|p| {
@@ -272,9 +498,9 @@ network:
} }
#[test] #[test]
fn collect_subdir_mixins_returns_empty_for_missing_dir() { fn subdir_named_scan_returns_empty_for_missing_dir() {
let absent = env::temp_dir().join("coyote-definitely-not-here-xyz"); let absent = env::temp_dir().join("coyote-definitely-not-here-xyz");
let found = collect_subdir_mixins(&absent); let found = collect_mixins(&absent, &[ScanMode::SubdirNamed]);
assert!(found.is_empty()); assert!(found.is_empty());
} }
@@ -343,6 +569,19 @@ network:
assert_eq!(fs::read_to_string(&spec).unwrap(), content); assert_eq!(fs::read_to_string(&spec).unwrap(), content);
} }
#[test]
#[serial]
fn wrap_mixin_bytes_as_kit_writes_spec_yaml() {
let _guard = TestCacheDirGuard::new();
let content = b"schemaVersion: '2'\nkind: mixin\nname: generated\n";
let kit_dir = wrap_mixin_bytes_as_kit(content, "generated").unwrap();
let spec = kit_dir.join("spec.yaml");
assert!(spec.exists(), "spec.yaml must exist in wrapped kit dir");
assert_eq!(fs::read(&spec).unwrap(), content);
}
#[test] #[test]
#[serial] #[serial]
fn wrap_mixin_as_kit_is_deterministic_for_identical_content() { fn wrap_mixin_as_kit_is_deterministic_for_identical_content() {
@@ -438,5 +677,317 @@ network:
"kit_path should not return the original file path" "kit_path should not return the original file path"
); );
} }
fn write_staged_file(mixin: &Path, rel: &str, content: &[u8]) {
let dst = mixin.parent().unwrap().join(MIXIN_FILES_DIR_NAME).join(rel);
fs::create_dir_all(dst.parent().unwrap()).unwrap();
fs::write(&dst, content).unwrap();
}
#[test]
#[serial]
fn wrap_mixin_as_kit_copies_sibling_files_tree_into_kit() {
let _guard = TestCacheDirGuard::new();
let mixin = write_mixin("files-copy", "kind: mixin\nname: probe\n");
write_staged_file(&mixin, "home/hello.md", b"# hello\n");
write_staged_file(&mixin, "home/nested/deep.txt", b"deep\n");
let kit_dir = wrap_mixin_as_kit(&mixin).unwrap();
assert!(kit_dir.join("spec.yaml").exists());
let files_root = kit_dir.join(MIXIN_FILES_DIR_NAME);
assert!(files_root.is_dir(), "kit dir must contain a files/ tree");
assert_eq!(
fs::read(files_root.join("home/hello.md")).unwrap(),
b"# hello\n"
);
assert_eq!(
fs::read(files_root.join("home/nested/deep.txt")).unwrap(),
b"deep\n"
);
}
#[test]
#[serial]
fn wrap_mixin_as_kit_hash_changes_when_a_staged_file_is_edited() {
let _guard = TestCacheDirGuard::new();
let mixin = write_mixin("files-hash-content", "kind: mixin\nname: probe\n");
write_staged_file(&mixin, "home/note.md", b"before\n");
let kit_before = wrap_mixin_as_kit(&mixin).unwrap();
write_staged_file(&mixin, "home/note.md", b"after\n");
let kit_after = wrap_mixin_as_kit(&mixin).unwrap();
assert_ne!(
kit_before, kit_after,
"editing a staged file must invalidate the kit hash"
);
assert_eq!(
fs::read(kit_after.join("files/home/note.md")).unwrap(),
b"after\n"
);
}
#[test]
#[serial]
fn wrap_mixin_as_kit_hash_changes_when_a_staged_file_is_added() {
let _guard = TestCacheDirGuard::new();
let mixin = write_mixin("files-hash-added", "kind: mixin\nname: probe\n");
write_staged_file(&mixin, "home/one.md", b"one\n");
let kit_before = wrap_mixin_as_kit(&mixin).unwrap();
write_staged_file(&mixin, "home/two.md", b"two\n");
let kit_after = wrap_mixin_as_kit(&mixin).unwrap();
assert_ne!(
kit_before, kit_after,
"adding a staged file must invalidate the kit hash"
);
}
#[test]
#[serial]
fn wrap_mixin_as_kit_hash_unchanged_when_no_files_dir() {
let _guard = TestCacheDirGuard::new();
let content = "kind: mixin\nname: legacy\n";
let mixin = write_mixin("legacy-no-files", content);
let with_helper = wrap_mixin_as_kit(&mixin).unwrap();
let bytes_only = wrap_mixin_bytes_as_kit(content.as_bytes(), "legacy").unwrap();
assert_eq!(
with_helper, bytes_only,
"mixins without a sibling files/ must keep the legacy bytes-only hash to reuse existing cache dirs"
);
}
#[test]
#[serial]
fn wrap_mixin_as_kit_ignores_sibling_files_that_is_not_a_directory() {
let _guard = TestCacheDirGuard::new();
let content = "kind: mixin\nname: probe\n";
let mixin = write_mixin("files-not-a-dir", content);
fs::write(mixin.parent().unwrap().join(MIXIN_FILES_DIR_NAME), b"decoy").unwrap();
let wrapped = wrap_mixin_as_kit(&mixin).unwrap();
let bytes_only = wrap_mixin_bytes_as_kit(content.as_bytes(), "probe").unwrap();
assert_eq!(
wrapped, bytes_only,
"a regular file named files must be ignored, not staged"
);
assert!(!wrapped.join(MIXIN_FILES_DIR_NAME).exists());
}
#[test]
#[serial]
fn wrap_mixin_as_kit_rebuilds_files_when_cache_dir_missing_files_tree() {
let _guard = TestCacheDirGuard::new();
let mixin = write_mixin("files-rebuild", "kind: mixin\nname: probe\n");
write_staged_file(&mixin, "home/hello.md", b"hi\n");
let kit_dir = wrap_mixin_as_kit(&mixin).unwrap();
let files_dst = kit_dir.join(MIXIN_FILES_DIR_NAME);
fs::remove_dir_all(&files_dst).unwrap();
assert!(!files_dst.exists());
let kit_again = wrap_mixin_as_kit(&mixin).unwrap();
assert_eq!(kit_again, kit_dir, "kit path is content-addressed");
assert!(
files_dst.is_dir(),
"a partial cache (spec present, files/ missing) must be rebuilt"
);
assert_eq!(fs::read(files_dst.join("home/hello.md")).unwrap(), b"hi\n");
}
#[test]
#[serial]
fn wrap_mixin_as_kit_deterministic_with_staged_files() {
let _guard = TestCacheDirGuard::new();
let content = "kind: mixin\nname: probe\n";
let mixin_one = write_mixin("determ-1", content);
write_staged_file(&mixin_one, "home/note.md", b"same\n");
let mixin_two = write_mixin("determ-2", content);
write_staged_file(&mixin_two, "home/note.md", b"same\n");
let kit_a = wrap_mixin_as_kit(&mixin_one).unwrap();
let kit_b = wrap_mixin_as_kit(&mixin_two).unwrap();
assert_eq!(
kit_a, kit_b,
"identical spec+files must produce the same content-addressed kit dir"
);
}
#[cfg(unix)]
#[test]
#[serial]
fn wrap_mixin_as_kit_rejects_symlinks_inside_files_tree() {
use std::os::unix::fs::symlink;
let _guard = TestCacheDirGuard::new();
let mixin = write_mixin("files-symlink", "kind: mixin\nname: probe\n");
let files_dir = mixin.parent().unwrap().join(MIXIN_FILES_DIR_NAME);
fs::create_dir_all(&files_dir).unwrap();
let target = files_dir.join("target.txt");
fs::write(&target, b"real").unwrap();
symlink(&target, files_dir.join("link.txt")).unwrap();
let err = wrap_mixin_as_kit(&mixin).unwrap_err();
let msg = format!("{err:#}");
assert!(
msg.contains("Symlinks are not allowed"),
"expected symlink rejection, got: {msg}"
);
}
#[cfg(unix)]
#[test]
#[serial]
fn wrap_mixin_as_kit_preserves_executable_bit() {
use std::os::unix::fs::PermissionsExt;
let _guard = TestCacheDirGuard::new();
let mixin = write_mixin("files-exec", "kind: mixin\nname: probe\n");
write_staged_file(&mixin, "bin/run.sh", b"#!/bin/sh\necho hi\n");
let src = mixin
.parent()
.unwrap()
.join(MIXIN_FILES_DIR_NAME)
.join("bin/run.sh");
fs::set_permissions(&src, fs::Permissions::from_mode(0o755)).unwrap();
let kit_dir = wrap_mixin_as_kit(&mixin).unwrap();
let dst = kit_dir.join("files/bin/run.sh");
let mode = fs::metadata(&dst).unwrap().permissions().mode() & 0o777;
assert_eq!(
mode, 0o755,
"executable bit must survive the copy into the kit dir"
);
}
}
#[test]
fn flat_scan_matches_rag_sidecars_by_suffix() {
let root = unique_root("flat-mixins");
fs::write(root.join("company-docs.sbx-mixin.yaml"), "kind: mixin\n").unwrap();
fs::write(root.join("alpha.sbx-mixin.yaml"), "kind: mixin\n").unwrap();
fs::write(root.join("company-docs.yaml"), "driver: qdrant\n").unwrap();
fs::write(root.join("notes.yaml"), "driver: yaml\n").unwrap();
fs::create_dir_all(root.join("decoy.sbx-mixin.yaml")).unwrap();
let found = collect_mixins(&root, &[ScanMode::Flat]);
assert_eq!(
file_names(&found),
vec!["alpha.sbx-mixin.yaml", "company-docs.sbx-mixin.yaml"]
);
let _ = fs::remove_dir_all(&root);
}
/// Every scan site in `discover()` picks its modes assuming each mode owns
/// exactly one layout and nothing else. `agents_data_dir()` requests two
/// modes at once, so an overlap would collect the same file twice and
/// `create_sandbox` would pass it as two `--kit` flags.
#[test]
fn each_scan_mode_owns_exactly_one_layout() {
let root = unique_root("scan-mode-ownership");
let agent = root.join("researcher");
fs::create_dir_all(&agent).unwrap();
let flat = root.join("company-docs.sbx-mixin.yaml");
let subdir_named = agent.join("sbx-mixin.yaml");
let subdir_flat = agent.join("handbook.sbx-mixin.yaml");
for path in [&flat, &subdir_named, &subdir_flat] {
fs::write(path, "kind: mixin\n").unwrap();
}
assert_eq!(collect_mixins(&root, &[ScanMode::Flat]), vec![flat.clone()]);
assert_eq!(
collect_mixins(&root, &[ScanMode::SubdirNamed]),
vec![subdir_named.clone()]
);
assert_eq!(
collect_mixins(&root, &[ScanMode::SubdirFlat]),
vec![subdir_flat.clone()]
);
let all = collect_mixins(
&root,
&[ScanMode::Flat, ScanMode::SubdirNamed, ScanMode::SubdirFlat],
);
assert_eq!(all, vec![flat, subdir_named, subdir_flat]);
let mut deduped = all.clone();
deduped.sort();
deduped.dedup();
assert_eq!(
deduped.len(),
all.len(),
"no mixin may be collected twice: {all:?}"
);
let _ = fs::remove_dir_all(&root);
}
#[test]
fn flat_scan_tolerates_a_missing_directory() {
let root = unique_root("flat-missing");
let absent = root.join("nope");
assert!(collect_mixins(&absent, &[ScanMode::Flat]).is_empty());
let _ = fs::remove_dir_all(&root);
}
/// `generate_rag_sbx_mixin` writes an agent-scoped RAG sidecar next to the
/// rag yaml, at `<agents>/<agent>/<rag>.sbx-mixin.yaml`. Before `SubdirFlat`
/// existed, nothing scanned that shape and attaching a Qdrant RAG from
/// inside an agent produced no network allow rule and no credential.
#[test]
fn agent_scoped_rag_sidecar_is_discovered() {
let root = unique_root("agent-scoped-rag");
let agent = root.join("researcher");
fs::create_dir_all(&agent).unwrap();
fs::write(agent.join("company-docs.sbx-mixin.yaml"), "kind: mixin\n").unwrap();
fs::write(agent.join("company-docs.yaml"), "driver: qdrant\n").unwrap();
let found = collect_mixins(&root, &[ScanMode::SubdirNamed, ScanMode::SubdirFlat]);
assert_eq!(found, vec![agent.join("company-docs.sbx-mixin.yaml")]);
let _ = fs::remove_dir_all(&root);
}
#[test]
fn agent_level_mixin_and_rag_sidecars_are_both_discovered() {
let root = unique_root("agent-both-shapes");
let agent = root.join("researcher");
fs::create_dir_all(&agent).unwrap();
fs::write(agent.join("sbx-mixin.yaml"), "kind: mixin\n").unwrap();
fs::write(agent.join("zebra.sbx-mixin.yaml"), "kind: mixin\n").unwrap();
fs::write(agent.join("alpha.sbx-mixin.yaml"), "kind: mixin\n").unwrap();
let found = collect_mixins(&root, &[ScanMode::SubdirNamed, ScanMode::SubdirFlat]);
assert_eq!(
file_names(&found),
vec![
"sbx-mixin.yaml",
"alpha.sbx-mixin.yaml",
"zebra.sbx-mixin.yaml"
]
);
let _ = fs::remove_dir_all(&root);
}
#[test]
fn subdir_flat_scan_ignores_a_directory_named_like_a_mixin() {
let root = unique_root("subdir-flat-decoy");
let agent = root.join("researcher");
fs::create_dir_all(agent.join("decoy.sbx-mixin.yaml")).unwrap();
assert!(collect_mixins(&root, &[ScanMode::SubdirFlat]).is_empty());
let _ = fs::remove_dir_all(&root);
} }
} }
+427 -467
View File
File diff suppressed because it is too large Load Diff
+107 -5
View File
@@ -31,15 +31,38 @@ use anyhow::{Context, Result};
use fancy_regex::Regex; use fancy_regex::Regex;
use fuzzy_matcher::{FuzzyMatcher, skim::SkimMatcherV2}; use fuzzy_matcher::{FuzzyMatcher, skim::SkimMatcherV2};
use is_terminal::IsTerminal; use is_terminal::IsTerminal;
use nu_ansi_term::Color;
use serde_json::Value;
use std::borrow::Cow; use std::borrow::Cow;
use std::sync::LazyLock; use std::collections::VecDeque;
use std::sync::atomic::AtomicBool;
use std::sync::{LazyLock, Mutex, OnceLock};
use std::{cmp, env, path::PathBuf, process}; use std::{cmp, env, path::PathBuf, process};
use syntect::highlighting::{Highlighter, Theme};
use syntect::parsing::Scope;
pub static CODE_BLOCK_RE: LazyLock<Regex> = pub static CODE_BLOCK_RE: LazyLock<Regex> =
LazyLock::new(|| Regex::new(r"(?ms)```\w*(.*)```").unwrap()); LazyLock::new(|| Regex::new(r"(?ms)```\w*(.*)```").unwrap());
pub static THINK_TAG_RE: LazyLock<Regex> = pub static THINK_TAG_RE: LazyLock<Regex> =
LazyLock::new(|| Regex::new(r"(?s)^\s*<think>.*?</think>(\s*|$)").unwrap()); LazyLock::new(|| Regex::new(r"(?s)^\s*<think>.*?</think>(\s*|$)").unwrap());
pub static IS_STDOUT_TERMINAL: LazyLock<bool> = LazyLock::new(|| std::io::stdout().is_terminal()); pub static IS_STDOUT_TERMINAL: LazyLock<bool> = LazyLock::new(|| std::io::stdout().is_terminal());
pub static HEADLESS: AtomicBool = AtomicBool::new(false);
pub static ACP_SERVER: AtomicBool = AtomicBool::new(false);
static ACP_PERMISSION_QUEUE: Mutex<VecDeque<Value>> = Mutex::new(VecDeque::new());
pub fn queue_acp_permission(notification: Value) {
if let Ok(mut q) = ACP_PERMISSION_QUEUE.lock() {
q.push_back(notification);
}
}
pub fn drain_acp_permissions() -> Vec<Value> {
ACP_PERMISSION_QUEUE
.lock()
.map(|mut q| q.drain(..).collect())
.unwrap_or_default()
}
pub static NO_COLOR: LazyLock<bool> = LazyLock::new(|| { pub static NO_COLOR: LazyLock<bool> = LazyLock::new(|| {
env::var("NO_COLOR") env::var("NO_COLOR")
.ok() .ok()
@@ -48,6 +71,44 @@ pub static NO_COLOR: LazyLock<bool> = LazyLock::new(|| {
|| !*IS_STDOUT_TERMINAL || !*IS_STDOUT_TERMINAL
}); });
static TOOL_DIM_COLOR: OnceLock<Color> = OnceLock::new();
static TOOL_FN_COLOR: OnceLock<Color> = OnceLock::new();
static TOOL_KEY_COLOR: OnceLock<Color> = OnceLock::new();
static TOOL_WARN_COLOR: OnceLock<Color> = OnceLock::new();
static REPLAY_LABEL_COLOR: OnceLock<Color> = OnceLock::new();
pub fn init_tool_colors(theme: &Theme) {
fn resolve(theme: &Theme, scope_str: &str) -> Option<Color> {
let scope = Scope::new(scope_str).ok()?;
let style = Highlighter::new(theme).style_mod_for_stack(&[scope]);
let fg = style.foreground.or(theme.settings.foreground)?;
let mute = |ch: u8| -> u8 { ((ch as u16 + 128) / 2) as u8 };
Some(Color::Rgb(mute(fg.r), mute(fg.g), mute(fg.b)))
}
if let Some(c) = resolve(theme, "comment") {
let _ = TOOL_DIM_COLOR.set(c);
}
if let Some(c) = resolve(theme, "support.function") {
let _ = TOOL_FN_COLOR.set(c);
}
if let Some(c) = resolve(theme, "constant.numeric") {
let _ = TOOL_KEY_COLOR.set(c);
}
if let Some(c) = resolve(theme, "string") {
let _ = TOOL_WARN_COLOR.set(c);
}
let replay_color = Scope::new("entity.name")
.ok()
.and_then(|scope| {
let style = Highlighter::new(theme).style_mod_for_stack(&[scope]);
style.foreground.or(theme.settings.foreground)
})
.map(|fg| Color::Rgb(fg.r, fg.g, fg.b));
if let Some(c) = replay_color {
let _ = REPLAY_LABEL_COLOR.set(c);
}
}
pub fn now() -> String { pub fn now() -> String {
chrono::Local::now().to_rfc3339_opts(chrono::SecondsFormat::Secs, false) chrono::Local::now().to_rfc3339_opts(chrono::SecondsFormat::Secs, false)
} }
@@ -141,14 +202,22 @@ pub fn indent_text<T: ToString>(s: T, size: usize) -> String {
} }
pub fn error_text(input: &str) -> String { pub fn error_text(input: &str) -> String {
color_text(input, nu_ansi_term::Color::Red) color_text(input, Color::Red)
} }
pub fn warning_text(input: &str) -> String { pub fn warning_text(input: &str) -> String {
color_text(input, nu_ansi_term::Color::Yellow) color_text(input, Color::Yellow)
} }
pub fn color_text(input: &str, color: nu_ansi_term::Color) -> String { pub fn muted_warning_text(input: &str) -> String {
if *NO_COLOR {
return input.to_string();
}
let color = TOOL_WARN_COLOR.get().copied().unwrap_or(Color::Fixed(136));
color.paint(input).to_string()
}
pub fn color_text(input: &str, color: Color) -> String {
if *NO_COLOR { if *NO_COLOR {
return input.to_string(); return input.to_string();
} }
@@ -162,7 +231,40 @@ pub fn dimmed_text(input: &str) -> String {
if *NO_COLOR { if *NO_COLOR {
return input.to_string(); return input.to_string();
} }
nu_ansi_term::Style::new().dimmed().paint(input).to_string() let color = TOOL_DIM_COLOR.get().copied().unwrap_or(Color::Fixed(243));
color.paint(input).to_string()
}
pub fn cyan_bold_text(input: &str) -> String {
if *NO_COLOR {
return input.to_string();
}
let color = TOOL_FN_COLOR.get().copied().unwrap_or(Color::Fixed(73));
nu_ansi_term::Style::new()
.fg(color)
.bold()
.paint(input)
.to_string()
}
pub fn replay_label_text(input: &str) -> String {
if *NO_COLOR {
return input.to_string();
}
let color = REPLAY_LABEL_COLOR.get().copied().unwrap_or(Color::Green);
nu_ansi_term::Style::new()
.fg(color)
.bold()
.paint(input)
.to_string()
}
pub fn magenta_text(input: &str) -> String {
if *NO_COLOR {
return input.to_string();
}
let color = TOOL_KEY_COLOR.get().copied().unwrap_or(Color::Fixed(133));
color.paint(input).to_string()
} }
pub fn multiline_text(input: &str) -> String { pub fn multiline_text(input: &str) -> String {
+83 -29
View File
@@ -1,9 +1,11 @@
mod utils; mod utils;
use std::env;
use std::fs::read_to_string; use std::fs::read_to_string;
use std::path::PathBuf; use std::path::PathBuf;
use crate::config::paths; use crate::config::paths;
use crate::sandbox::SANDBOX_ENV_FLAG;
pub use utils::create_vault_password_file; pub use utils::create_vault_password_file;
pub use utils::interpolate_secrets; pub use utils::interpolate_secrets;
pub use utils::prompt_provider_choice; pub use utils::prompt_provider_choice;
@@ -17,7 +19,7 @@ use gman::providers::SecretProvider;
use gman::providers::SupportedProvider; use gman::providers::SupportedProvider;
use gman::providers::local::LocalProvider; use gman::providers::local::LocalProvider;
use inquire::{Password, PasswordDisplayMode, required}; use inquire::{Password, PasswordDisplayMode, required};
use log::{info, warn}; use log::warn;
use serde_yaml::Value; use serde_yaml::Value;
use std::sync::{Arc, LazyLock}; use std::sync::{Arc, LazyLock};
use tokio::runtime::Handle; use tokio::runtime::Handle;
@@ -25,34 +27,10 @@ use uuid::Uuid;
pub static SECRET_RE: LazyLock<Regex> = LazyLock::new(|| Regex::new(r"\{\{([^{}]+)}}").unwrap()); pub static SECRET_RE: LazyLock<Regex> = LazyLock::new(|| Regex::new(r"\{\{([^{}]+)}}").unwrap());
fn apply_sandboxed_home_translation(provider_def: &mut LocalProvider) {
let Some(ref pf) = provider_def.password_file else {
return;
};
if pf.exists() {
return;
}
let Some(translated) = paths::translate_sandboxed_home_dir(pf) else {
return;
};
if !translated.exists() {
return;
}
info!(
"vault password file '{}' not found; resolved to sandboxed path '{}'",
pf.display(),
translated.display()
);
provider_def.password_file = Some(translated);
}
#[derive(Debug, Default, Clone)] #[derive(Debug, Default, Clone)]
pub struct Vault { pub struct Vault {
pub(crate) provider: SupportedProvider, pub(crate) provider: SupportedProvider,
sandbox_mode: bool,
} }
pub type GlobalVault = Arc<Vault>; pub type GlobalVault = Arc<Vault>;
@@ -90,7 +68,10 @@ impl Vault {
} }
}; };
Ok(Self { provider }) Ok(Self {
provider,
sandbox_mode: false,
})
} }
pub fn default_local() -> Self { pub fn default_local() -> Self {
@@ -102,10 +83,25 @@ impl Vault {
..LocalProvider::default() ..LocalProvider::default()
}, },
}, },
sandbox_mode: false,
}
}
pub fn from_provider(provider: SupportedProvider) -> Self {
Self {
provider,
sandbox_mode: false,
} }
} }
pub fn init(config: &AppConfig) -> Result<Self> { pub fn init(config: &AppConfig) -> Result<Self> {
if env::var_os(SANDBOX_ENV_FLAG).is_some() {
return Ok(Self {
sandbox_mode: true,
..Self::default()
});
}
let mut provider = match &config.secrets_provider { let mut provider = match &config.secrets_provider {
Some(p) => p.clone(), Some(p) => p.clone(),
None => SupportedProvider::Local { None => SupportedProvider::Local {
@@ -117,11 +113,13 @@ impl Vault {
}; };
if let SupportedProvider::Local { provider_def } = &mut provider { if let SupportedProvider::Local { provider_def } = &mut provider {
apply_sandboxed_home_translation(provider_def);
ensure_password_file_initialized(provider_def)?; ensure_password_file_initialized(provider_def)?;
} }
Ok(Self { provider }) Ok(Self {
provider,
sandbox_mode: false,
})
} }
pub fn local_password_file(&self) -> Result<PathBuf> { pub fn local_password_file(&self) -> Result<PathBuf> {
@@ -148,6 +146,11 @@ impl Vault {
} }
pub fn add_secret(&self, secret_name: &str) -> Result<()> { pub fn add_secret(&self, secret_name: &str) -> Result<()> {
if self.sandbox_mode {
bail!(
"Vault management is disabled in sandbox mode. Use `coyote --add-secret` on your host."
);
}
let secret_value = Password::new("Enter the secret value:") let secret_value = Password::new("Enter the secret value:")
.with_validator(required!()) .with_validator(required!())
.with_display_mode(PasswordDisplayMode::Masked) .with_display_mode(PasswordDisplayMode::Masked)
@@ -164,6 +167,11 @@ impl Vault {
} }
pub fn get_secret(&self, secret_name: &str, display_output: bool) -> Result<String> { pub fn get_secret(&self, secret_name: &str, display_output: bool) -> Result<String> {
if self.sandbox_mode {
bail!(
"Vault management is disabled in sandbox mode. Use `coyote --add-secret` on your host."
);
}
let h = Handle::current(); let h = Handle::current();
let secret = tokio::task::block_in_place(|| { let secret = tokio::task::block_in_place(|| {
h.block_on(self.provider_ref().get_secret(secret_name)) h.block_on(self.provider_ref().get_secret(secret_name))
@@ -177,6 +185,11 @@ impl Vault {
} }
pub fn update_secret(&self, secret_name: &str) -> Result<()> { pub fn update_secret(&self, secret_name: &str) -> Result<()> {
if self.sandbox_mode {
bail!(
"Vault management is disabled in sandbox mode. Use `coyote --add-secret` on your host."
);
}
let secret_value = Password::new("Enter the secret value:") let secret_value = Password::new("Enter the secret value:")
.with_validator(required!()) .with_validator(required!())
.with_display_mode(PasswordDisplayMode::Masked) .with_display_mode(PasswordDisplayMode::Masked)
@@ -195,6 +208,11 @@ impl Vault {
} }
pub fn delete_secret(&self, secret_name: &str) -> Result<()> { pub fn delete_secret(&self, secret_name: &str) -> Result<()> {
if self.sandbox_mode {
bail!(
"Vault management is disabled in sandbox mode. Use `coyote --add-secret` on your host."
);
}
let h = Handle::current(); let h = Handle::current();
tokio::task::block_in_place(|| h.block_on(self.provider_ref().delete_secret(secret_name)))?; tokio::task::block_in_place(|| h.block_on(self.provider_ref().delete_secret(secret_name)))?;
println!("✓ Secret '{secret_name}' deleted from the vault."); println!("✓ Secret '{secret_name}' deleted from the vault.");
@@ -203,6 +221,11 @@ impl Vault {
} }
pub fn list_secrets(&self, display_output: bool) -> Result<Vec<String>> { pub fn list_secrets(&self, display_output: bool) -> Result<Vec<String>> {
if self.sandbox_mode {
bail!(
"Vault management is disabled in sandbox mode. Use `coyote --add-secret` on your host."
);
}
let h = Handle::current(); let h = Handle::current();
let secrets = let secrets =
tokio::task::block_in_place(|| h.block_on(self.provider_ref().list_secrets()))?; tokio::task::block_in_place(|| h.block_on(self.provider_ref().list_secrets()))?;
@@ -346,4 +369,35 @@ mod tests {
let vault = Vault::default(); let vault = Vault::default();
assert!(vault.local_password_file().is_err()); assert!(vault.local_password_file().is_err());
} }
#[test]
fn vault_disabled_in_sandbox() {
let prev = std::env::var_os("IS_SANDBOX");
unsafe {
std::env::set_var("IS_SANDBOX", "1");
}
let vault = Vault::init(&AppConfig::default()).unwrap();
let check = |err: anyhow::Error| {
let msg = err.to_string();
assert!(
msg.contains("Vault management is disabled in sandbox mode"),
"expected vault-disabled error, got: {msg}"
);
};
check(vault.add_secret("test").unwrap_err());
check(vault.get_secret("test", false).unwrap_err());
check(vault.update_secret("test").unwrap_err());
check(vault.delete_secret("test").unwrap_err());
check(vault.list_secrets(false).unwrap_err());
unsafe {
match prev {
Some(v) => std::env::set_var("IS_SANDBOX", v),
None => std::env::remove_var("IS_SANDBOX"),
}
}
}
} }
+30
View File
@@ -1,4 +1,5 @@
use crate::config::ensure_parent_exists; use crate::config::ensure_parent_exists;
use crate::sandbox::{SANDBOX_ENV_FLAG, sandbox_secret_env_var};
use crate::vault::{SECRET_RE, Vault}; use crate::vault::{SECRET_RE, Vault};
use anyhow::Result; use anyhow::Result;
use anyhow::anyhow; use anyhow::anyhow;
@@ -14,6 +15,7 @@ use indoc::formatdoc;
use inquire::validator::Validation; use inquire::validator::Validation;
use inquire::{Confirm, Password, PasswordDisplayMode, Select, Text, min_length, required}; use inquire::{Confirm, Password, PasswordDisplayMode, Select, Text, min_length, required};
use log::debug; use log::debug;
use std::env;
use std::path::{Path, PathBuf}; use std::path::{Path, PathBuf};
use std::process::Command; use std::process::Command;
@@ -355,6 +357,34 @@ fn required_cli_preflight(label: &str, cli: &str, install_url: &str) {
} }
pub fn interpolate_secrets(content: &str, vault: &Vault) -> Result<(String, Vec<String>)> { pub fn interpolate_secrets(content: &str, vault: &Vault) -> Result<(String, Vec<String>)> {
if env::var_os(SANDBOX_ENV_FLAG).is_some() {
let (parsed, missing) = interpolate_secrets_with(content, None, |name| {
env::var(sandbox_secret_env_var(name)).map_err(|_| {
anyhow!(SecretError::NotFound {
key: name.to_string(),
provider: "sandbox environment",
})
})
})?;
if !missing.is_empty() {
let mut env_vars: Vec<String> = missing
.iter()
.map(|name| sandbox_secret_env_var(name))
.collect();
env_vars.sort();
env_vars.dedup();
eprintln!(
"Config references secrets that are not available inside this sandbox \
(expected env vars: {}). Sandbox secrets are provisioned at creation \
from the host; add the missing secrets on the host, then re-create \
the sandbox.",
env_vars.join(", ")
);
}
return Ok((parsed, missing));
}
interpolate_secrets_with(content, vault.auth_hint(), |name| { interpolate_secrets_with(content, vault.auth_hint(), |name| {
vault.get_secret(name, false) vault.get_secret(name, false)
}) })