Dark-Alex-17/coyote
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Compare commits

base: Dark-Alex-17/coyote:7a732436aa5b59c05f241467fec06d0693cc3089
Branches Tags
Dark-Alex-17/coyote:main Dark-Alex-17/coyote:feat/rag-phase5-qdrant-write Dark-Alex-17/coyote:feat/rag-driver-abstraction-v3 Dark-Alex-17/coyote:graph-based-rag Dark-Alex-17/coyote:skills Dark-Alex-17/coyote:develop Dark-Alex-17/coyote:graph-agents Dark-Alex-17/coyote:config-refactor Dark-Alex-17/coyote:restful-api-docs Dark-Alex-17/coyote:langchain
Dark-Alex-17/coyote:v0.7.3 Dark-Alex-17/coyote:v0.7.2 Dark-Alex-17/coyote:v0.7.1 Dark-Alex-17/coyote:v0.7.0 Dark-Alex-17/coyote:v0.6.0 Dark-Alex-17/coyote:v0.5.0 Dark-Alex-17/coyote:v0.4.0 Dark-Alex-17/coyote:v0.3.0 Dark-Alex-17/coyote:v0.2.0 Dark-Alex-17/coyote:v0.1.3 Dark-Alex-17/coyote:v0.1.2 Dark-Alex-17/coyote:v0.1.1
...
compare: Dark-Alex-17/coyote:feat/rag-driver-abstraction-v3
Branches Tags
Dark-Alex-17/coyote:main Dark-Alex-17/coyote:feat/rag-phase5-qdrant-write Dark-Alex-17/coyote:feat/rag-driver-abstraction-v3 Dark-Alex-17/coyote:graph-based-rag Dark-Alex-17/coyote:skills Dark-Alex-17/coyote:develop Dark-Alex-17/coyote:graph-agents Dark-Alex-17/coyote:config-refactor Dark-Alex-17/coyote:restful-api-docs Dark-Alex-17/coyote:langchain
Dark-Alex-17/coyote:v0.7.3 Dark-Alex-17/coyote:v0.7.2 Dark-Alex-17/coyote:v0.7.1 Dark-Alex-17/coyote:v0.7.0 Dark-Alex-17/coyote:v0.6.0 Dark-Alex-17/coyote:v0.5.0 Dark-Alex-17/coyote:v0.4.0 Dark-Alex-17/coyote:v0.3.0 Dark-Alex-17/coyote:v0.2.0 Dark-Alex-17/coyote:v0.1.3 Dark-Alex-17/coyote:v0.1.2 Dark-Alex-17/coyote:v0.1.1
18 Commits
7a732436aa ... feat/rag-driver-abstraction-v3
Author SHA1 Message Date
Dark-Alex-17 d6c114fe58 feat: simplified the duckdb selection prompt 2026-08-11 16:50:50 -06:00
Dark-Alex-17 912e00a627 docs(rag): correct the duckdb concurrency note in the driver prompt 
The driver picker still told users a duckdb RAG can only be open in one Coyote
process at a time. That stopped being true once the store began opening
read-only for queries: any number of processes can now query it concurrently.

The restriction that remains is narrower and only bites while writing, so the
prompt now states that instead — several processes can query at once, but an
ingest or rebuild locks the others out until it finishes.
 2026-08-11 16:38:55 -06:00
Dark-Alex-17 e006e29ff1 feat(rag): let several Coyote processes query one duckdb RAG at once 
The DuckDB store was always opened read-write, which takes an exclusive file
lock, so a second Coyote process could not even read the RAG. Querying does not
write, and DuckDB permits many concurrent readers as long as no writer is
attached, so the store is now opened read-only whenever it already carries a
complete schema.

Creating or initializing the store still writes, as does rebuilding, so those
paths take the exclusive handle. The rebuild path upgrades a read-only
connection in place, which every clone of the handle observes because the mode
lives behind the shared mutex rather than beside it. Extension loads and the
HNSW persistence setting are per-connection and are re-established on the
upgraded connection.

An upgrade that loses the race for the write lock reports that another process
holds the RAG and that nothing was written, then reopens read-only so the
session can keep querying. A read-only handle also refuses writes outright, so a
missed upgrade cannot silently discard an ingest.
 2026-08-11 14:58:18 -06:00
Dark-Alex-17 c0067d387c refactor(rag): drop the hardcoded embedding model hint from attach 
The attach wizard mapped a collection's vector dimension to a hardcoded list of
model names and printed them as likely candidates. The list was never checked
against the models the user actually has configured, so it could recommend a
model they cannot select, and one entry was a parenthetical note rather than a
model id and so could never match anything. Any list like this rots as models
are released.

The dimension itself comes from the server and is worth stating, so it is still
printed, as is the warning that a mismatched embedding model returns bad
results. Deriving real candidates would need a dimension recorded against each
configured model, which the model config does not carry today.
 2026-08-11 14:06:57 -06:00
Dark-Alex-17 118c346345 feat(rag): support string and UUID Qdrant point IDs 
Point ids were read with as_u64() inside a filter_map, so a string id was
silently dropped and a UUID-keyed collection returned zero hits with no error.
The attach wizard therefore refused such collections and told the user to
rebuild with integer ids, which defeats the purpose of attaching to a
collection someone else already built. LangChain, a common way to populate
Qdrant, uses UUIDs by default.

The integer id was never load-bearing for this driver. DocumentId is a packed
(file, chunk) pair used positionally by the local drivers, but an attached RAG
holds no local files or vectors and every positional consumer already returns
early on it, so the id only has to survive the round trip from search back to
the content fetch. Ids that cannot make that trip as a u64 are interned behind
a synthetic handle and restored when the fetch is issued, leaving collections
that already use integer ids on exactly the path they used before.
 2026-08-11 14:05:38 -06:00
Dark-Alex-17 dc677a2529 fix(sandbox): discover agent-scoped RAG mixin sidecars 
An agent-scoped RAG writes its config to <data>/agents/<agent>/<rag>.yaml, so
its sbx mixin sidecar lands beside it as <rag>.sbx-mixin.yaml. Discovery scanned
the agents directory only for a file named exactly sbx-mixin.yaml, and scanned
for suffixed sidecars only in the top-level rags directory, so a RAG attached
while an agent was active contributed no network allow rule and no credential to
the sandbox. The failure was silent: the sandbox launched and the RAG was simply
unreachable from inside it.

The two collectors differed only in the filename shape they matched, so they are
now one scan that takes the set of layouts to look for. The agents directory
asks for both its own sbx-mixin.yaml and the suffixed sidecars one level in,
which is the shape that was missing. Discovery order is unchanged, and it is
load-bearing: each mixin becomes a --kit in list order and later ones layer over
earlier ones, so the workspace mixin must stay last.
 2026-08-11 14:05:31 -06:00
Dark-Alex-17 5e2b9c98ad fix(rag): stop the attach wizard from silently accepting an empty collection 
`sample_point_id` returns `None` for a collection with no points, so the
UUID guard's `if let Some(..)` fell straight through and the wizard attached
happily. The result is a RAG that answers every query with zero hits and
never says why.

Sample once, then check for emptiness explicitly. This warns and asks rather
than hard-failing: an empty collection is not necessarily a mistake, since
another tool may be about to populate it, and none of the wizard's remaining
probes can distinguish that from a misconfiguration. The confirmation
defaults to "no" so it cannot be walked past by accident, and `attach`
already refuses to run non-interactively, so no unattended path reaches it.
 2026-08-11 13:55:59 -06:00
Dark-Alex-17 c458ca93a9 feat(rag): create the API key secret inline in the attach wizard 
The wizard hard-errored with "Secret 'X' not found in vault. Run
`coyote --add-secret X` first.", throwing away every answer the user had
already given it. Offer to create the secret in place instead, deferring to
`Vault::add_secret` for the masked prompt, the provider write and the
confirmation line, then read it back.

Only a genuine `SecretError::NotFound` triggers the offer. An auth failure,
a provider outage, or the vault being disabled inside a sandbox all
propagate with their own message, because prompting for a value that cannot
be stored would fail one step later and bury the real cause. Declining the
offer fails with both ways out spelled: add the secret up front, or answer
"no" to the API-key question.
 2026-08-11 13:50:33 -06:00
Dark-Alex-17 860566bf50 feat: let workflow rag nodes select a RAG driver 
`RagNode` gains an optional `driver`, forwarded into `RagInitConfig` so a
graph node can build its knowledge base on duckdb instead of yaml. Nodes
that name no driver forward `None`, which still resolves to yaml, so
existing workflows are unaffected.

An unknown driver is rejected up front rather than at construction time.
`Rag::create` dispatches unknown drivers to its yaml catch-all, so a typo
would otherwise embed every document and persist the bogus string, after
which every subsequent load fails validation and the agent cannot start.
The check asks `RagData::validate()` through a probe value instead of
restating the list of valid drivers, so the two cannot drift.
 2026-08-11 13:46:59 -06:00
Dark-Alex-17 7f90710427 refactor(rag): interpolate every driver_config value, not just api_key 
Only `driver_config["api_key"]` was interpolated, so any credential-bearing
driver field added later would have shipped its raw `{{PLACEHOLDER}}` to the
server. Resolve every value instead, via `resolve_driver_config`.

Resolution still happens into a function-local copy and never touches
`RagData`: `save()` serializes `self.data` and is called by `.set rag_top_k`
and friends, so a resolved credential parked there would be written to the
RAG's YAML in plaintext. The literal `{{NAME}}` also has to survive on disk
because sandbox credential provisioning parses it back out to learn which
vault secret to bind. Scope stays `driver_config` deliberately: the rest of
a RAG file is ingested document text, where `{{...}}` is ordinary content.

Covered by a test that saves after a load and asserts the placeholder, not
the secret, is what reaches the file.
 2026-08-11 13:46:32 -06:00
Dark-Alex-17 93a934439b fix(rag): fail loudly when a RAG's vault secret is missing 
`interpolate_secrets` does not error on a secret the vault cannot resolve:
it substitutes the empty string and returns the name in its second tuple
element. `load_async` discarded that vec, so a typo'd or deleted vault
secret produced `api_key = ""` and an unexplained 401 from Qdrant.

Bail instead, naming the RAG and the missing secrets, matching what global
config loading already does.
 2026-08-11 13:44:00 -06:00
Dark-Alex-17 ecda258d3a style: Cleaned up some minor styling issues 2026-08-11 13:04:27 -06:00
Dark-Alex-17 3e598065f8 fix(rag): serialize DuckDB extension installs to stop a Windows race 
`ensure_extension` fell back to `INSTALL` whenever `LOAD` failed. With a cold
extension cache every thread's `LOAD` fails at once, so every thread ran
`INSTALL` concurrently for the same extension. DuckDB installs by downloading
to a temp file and then MOVING it into `~/.duckdb/extensions/...`; POSIX allows
replacing a file other handles hold open, so Linux and macOS survived, but
Windows rejects that move with "Access is denied" and the losing threads failed.

Guard the install step with a process-global mutex and re-check `LOAD` after
acquiring it. The re-check is what bounds the work to a single install: without
it every thread queued behind the winner would still run a redundant `INSTALL`
and repeat the same move over a file that is now open.

`LOAD` is per-connection, so it still runs on every connection; only `INSTALL`
is serialized. An already-installed extension takes the pre-lock fast path and
costs neither a lock nor network. The lock is never held across the connection
mutex, so it cannot invert lock order.

Traced with strace on a cold cache under default test parallelism: before, 17
threads moved files into the store (13 racing on vss alone); after, exactly one
rename per extension.
 2026-08-10 16:13:36 -06:00
Dark-Alex-17 3abc30d633 fix(rag): emit an sbx kit v2 mixin and declare RAG credentials to the proxy 
The RAG attach sidecar was written against the sbx kit v1 spec and still emitted schemaVersion "1" with network.allowedDomains, network.serviceDomains, network.serviceAuth, credentials.sources.<n>.env and environment.proxyManaged. Every one of those keys was removed in kit v2. Coyote does not validate mixins, it copies them byte-for-byte into spec.yaml, so the invalid document surfaced only as an opaque sbx failure with no indication of which mixin caused it.

generate_rag_sbx_mixin now builds the document from the shared serializer structs instead of a format! string, which is how the envelope drifted unnoticed in the first place. render_mixin_yaml and the RAG sidecar both go through a new render_mixin_document, giving one definition of the envelope and one enforcement point for the rule that every inject domain must also appear in permissions.network.allow.

Fix an auth bug the port exposed: inject_rag_secrets bound the API key with sbx secret set, but nothing ever emitted a matching credentials entry, so the proxy held a value with no inject rule and never rewrote the auth header. An attached RAG credential silently did not work inside the sandbox. The sidecar now declares that credential; a RAG with no API key declares none while still receiving egress.

Fix the service id: the bind passed the raw file stem instead of routing it through secret_service_id, so a RAG named My_Docs produced an illegal id. The bind and the generated credentials service now share that derivation and cannot disagree.

Retire sbx_domain_forms in favour of allow_entry_for_url, now pub(crate). It emitted both a bare host and host:port because v1 serviceDomains needed a bare key; v2 has no such need, so the extra entry is simply wrong. It also defaulted a schemeless host to port 6333 while normalize_base_url resolves it to http and port 80, meaning the allow entry named a port the client never dialled.
 2026-08-10 15:58:40 -06:00
Dark-Alex-17 f68937611e fix(rag): install DuckDB vss and fts extensions when they are missing 
The DuckDB schema init loaded the vss and fts extensions but nothing ever
installed them, so any machine without them already present failed with
'IO Error: Extension "vss.duckdb_extension" not found'. This surfaced as 13
failing tests in CI while passing locally, because local runs had the
extensions installed already.

Loading is attempted first so an extension that is already present costs
nothing and never touches the network; INSTALL is reached only once, on a
machine seeing the extension for the first time, and reports an actionable
message if it cannot download.

CI cached the extension directory but nothing populated it, so the cache
saved an empty directory forever. The cache key now derives from Cargo.lock
rather than a hardcoded DuckDB version, and a step on cache miss installs the
extensions so the post-job save has something to store.
 2026-08-10 15:47:15 -06:00
Dark-Alex-17 a12cf84eb6 Merge remote-tracking branch 'refs/remotes/origin/main' 2026-08-10 15:41:50 -06:00
Dark-Alex-17 2b45e3a9b8 feat: improved wording and heuristic detection for sisyphus suite of agents
CI / All (ubuntu-latest) (push) Failing after 25s
Details
CI / All (macos-latest) (push) Has been cancelled
Details
CI / All (windows-latest) (push) Has been cancelled
Details
2026-08-10 15:37:53 -06:00
Dark-Alex-17 91dbaf5533 feat: upgraded to sbx kit v2 spec for improved integration
CI / All (ubuntu-latest) (push) Failing after 25s
Details
CI / All (macos-latest) (push) Has been cancelled
Details
CI / All (windows-latest) (push) Has been cancelled
Details
2026-08-10 15:26:16 -06:00
31 changed files with 4046 additions and 1232 deletions
Expand all files Collapse all files
.github/workflows/ci.yaml
+6 -1
View File
@@ -37,10 +37,15 @@ jobs:
- uses: Swatinem/rust-cache@v2 - uses: Swatinem/rust-cache@v2
- name: Cache DuckDB Extensions - name: Cache DuckDB Extensions
id: duckdb-extensions
uses: actions/cache@v4 uses: actions/cache@v4
with: with:
path: ~/.duckdb/extensions path: ~/.duckdb/extensions
key: duckdb-ext-${{ matrix.os }}-v1.5.5 key: duckdb-ext-${{ matrix.os }}-${{ hashFiles('Cargo.lock') }}
- name: Install DuckDB Extensions
if: steps.duckdb-extensions.outputs.cache-hit != 'true'
run: cargo test --all duckdb
- name: Test - name: Test
run: cargo test --all run: cargo test --all
Cargo.lock
Generated
+144 -120
View File
@@ -58,9 +58,9 @@ checksum = "683d7910e743518b0e34f1186f92494becacb047c7b6bf616c96772180fef923"
[[package]] [[package]]
name = "android_system_properties" name = "android_system_properties"
version = "0.1.5" version = "0.1.6"
source = "registry+https://github.com/rust-lang/crates.io-index" source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "819e7219dbd41043ac279b19830f2efc897156490d7fd6ea916720117ee66311" checksum = "ae221649c9976a6f6c56ae1facf410f3ddb33cc661c4b7b61020a912d4237fbc"
dependencies = [ dependencies = [
"libc", "libc",
] ]
@@ -209,7 +209,7 @@ source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "17060e608fbc0809d62a996a65cdee9e7c441a979f40f2d1d2fbdce9eef60dad" checksum = "17060e608fbc0809d62a996a65cdee9e7c441a979f40f2d1d2fbdce9eef60dad"
dependencies = [ dependencies = [
"anyhow", "anyhow",
"base64", "base64 0.22.1",
"convert_case 0.11.0", "convert_case 0.11.0",
"dirs", "dirs",
"either", "either",
@@ -321,7 +321,7 @@ dependencies = [
"arrow-schema", "arrow-schema",
"arrow-select", "arrow-select",
"atoi", "atoi",
"base64", "base64 0.22.1",
"chrono", "chrono",
"comfy-table", "comfy-table",
"half", "half",
@@ -445,9 +445,9 @@ dependencies = [
[[package]] [[package]]
name = "async-trait" name = "async-trait"
version = "0.1.91" version = "0.1.92"
source = "registry+https://github.com/rust-lang/crates.io-index" source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "ae36dc4177970ef04fde5178d3e2429882def40e57a451f919c098f72baa6cec" checksum = "82f6aeea286b8eb4dd3431a1be1b59d290ace00f5bfd8e2a159bc2a05e2c1667"
dependencies = [ dependencies = [
"proc-macro2", "proc-macro2",
"quote", "quote",
@@ -520,11 +520,11 @@ dependencies = [
[[package]] [[package]]
name = "aws-lc-rs" name = "aws-lc-rs"
version = "1.17.3" version = "1.18.0"
source = "registry+https://github.com/rust-lang/crates.io-index" source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "00bdb5da18dac48ca2cc7cd4a98e533e8635a58e2361d13a1a4ee3888e0d72f1" checksum = "ce2b2dcc879c3bae0d371e77c99f2238400ef24ec001394befa67b6e543add9e"
dependencies = [ dependencies = [
"aws-lc-sys 0.43.0", "aws-lc-sys 0.44.0",
"zeroize", "zeroize",
] ]
@@ -543,9 +543,9 @@ dependencies = [
[[package]] [[package]]
name = "aws-lc-sys" name = "aws-lc-sys"
version = "0.43.0" version = "0.44.0"
source = "registry+https://github.com/rust-lang/crates.io-index" source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "43103168cc76fe62678a375e722fc9cb3a0146159ac5828bc4f0dfd755c2224c" checksum = "f09fae7be8bb3174e05c6afdb34199e6dc0c7c04ba9fa237b1967adfbde27483"
dependencies = [ dependencies = [
"cc", "cc",
"cmake", "cmake",
@@ -1052,6 +1052,12 @@ version = "0.22.1"
source = "registry+https://github.com/rust-lang/crates.io-index" source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "72b3254f16251a8381aa12e40e3c4d2f0199f8c6508fbecb9d91f575e0fbb8c6" checksum = "72b3254f16251a8381aa12e40e3c4d2f0199f8c6508fbecb9d91f575e0fbb8c6"
[[package]]
name = "base64"
version = "0.23.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "ac07cdecf99051d9a5238b80f35af32cdeba5b336e55d957b318b50137e18da5"
[[package]] [[package]]
name = "base64-simd" name = "base64-simd"
version = "0.8.0" version = "0.8.0"
@@ -1248,7 +1254,7 @@ dependencies = [
"cached_proc_macro_types", "cached_proc_macro_types",
"hashbrown 0.15.5", "hashbrown 0.15.5",
"once_cell", "once_cell",
"thiserror 2.0.19", "thiserror 2.0.20",
"web-time", "web-time",
] ]
@@ -1278,9 +1284,9 @@ checksum = "37b2a672a2cb129a2e41c10b1224bb368f9f37a2b16b612598138befd7b37eb5"
[[package]] [[package]]
name = "cc" name = "cc"
version = "1.4.0" version = "1.4.2"
source = "registry+https://github.com/rust-lang/crates.io-index" source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "5add81bb678e6cb321aff7fa0dc7689ad82b112dbc032cea19f91d6b8e3582b9" checksum = "5d262e149917187838d5b42777c8253bcb64500067342904e7d429499a6f277e"
dependencies = [ dependencies = [
"find-msvc-tools", "find-msvc-tools",
"jobserver", "jobserver",
@@ -1382,9 +1388,9 @@ dependencies = [
[[package]] [[package]]
name = "clap" name = "clap"
version = "4.6.5" version = "4.6.6"
source = "registry+https://github.com/rust-lang/crates.io-index" source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "301b56658598e48f3648647ac6fc887be7e7108eddfa4e9b63fcf3ec58c0cadf" checksum = "473c7e07f409a8d772161724aa8db6a765a2532a70f9667eeb7b49d3d02fbdca"
dependencies = [ dependencies = [
"clap_builder", "clap_builder",
"clap_derive", "clap_derive",
@@ -1392,9 +1398,9 @@ dependencies = [
[[package]] [[package]]
name = "clap_builder" name = "clap_builder"
version = "4.6.5" version = "4.6.6"
source = "registry+https://github.com/rust-lang/crates.io-index" source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "94a65403d1a1bd28f7dc68eb8506e8874808ee5eecb59298de588e2e1407a078" checksum = "7b48fea5a88e9ae728a2dcbedbfc0e730f7d60da42e1cb049a83c9fb8b789889"
dependencies = [ dependencies = [
"anstream", "anstream",
"anstyle", "anstyle",
@@ -1405,9 +1411,9 @@ dependencies = [
[[package]] [[package]]
name = "clap_complete" name = "clap_complete"
version = "4.6.8" version = "4.6.9"
source = "registry+https://github.com/rust-lang/crates.io-index" source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "b1f84a88507dbd05c695f2cb5e8558e747179134005e9893882dec964190ed89" checksum = "3be2ad0423bdbbb0e25bc89add796f3559706d4a95e1bc98e4d9662a957b6a19"
dependencies = [ dependencies = [
"clap", "clap",
"clap_lex", "clap_lex",
@@ -1417,9 +1423,9 @@ dependencies = [
[[package]] [[package]]
name = "clap_complete_nushell" name = "clap_complete_nushell"
version = "4.6.1" version = "4.6.2"
source = "registry+https://github.com/rust-lang/crates.io-index" source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "933b05d5d83ff65fd7eaf5d106c792f2264908790a2642aca57429767b762ce2" checksum = "ffb66bc82eb9c92b1727310ae2c5868df22ae7cf46185bc5c544a4fa71955e49"
dependencies = [ dependencies = [
"clap", "clap",
"clap_complete", "clap_complete",
@@ -1532,7 +1538,7 @@ dependencies = [
"lazy_static", "lazy_static",
"serde", "serde",
"serde_yaml", "serde_yaml",
"thiserror 2.0.19", "thiserror 2.0.20",
] ]
[[package]] [[package]]
@@ -1612,9 +1618,9 @@ dependencies = [
[[package]] [[package]]
name = "cookie" name = "cookie"
version = "0.18.1" version = "0.18.2"
source = "registry+https://github.com/rust-lang/crates.io-index" source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "4ddef33a339a91ea89fb53151bd0a4689cfce27055c291dfa69945475d22c747" checksum = "1a373e3602691c3cdea496d2f0ee5935151e6168fe87739483c463db1b2f2f87"
dependencies = [ dependencies = [
"percent-encoding", "percent-encoding",
"time", "time",
@@ -1667,7 +1673,7 @@ dependencies = [
"async-trait", "async-trait",
"aws-smithy-eventstream", "aws-smithy-eventstream",
"aws-smithy-types", "aws-smithy-types",
"base64", "base64 0.22.1",
"bincode 2.0.1", "bincode 2.0.1",
"bitflags 2.13.1", "bitflags 2.13.1",
"bm25", "bm25",
@@ -2033,7 +2039,7 @@ version = "1.0.0"
source = "registry+https://github.com/rust-lang/crates.io-index" source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "10d60334b3b2e7c9d91ef8150abfb6fa4c1c39ebbcf4a81c2e346aad939fee3e" checksum = "10d60334b3b2e7c9d91ef8150abfb6fa4c1c39ebbcf4a81c2e346aad939fee3e"
dependencies = [ dependencies = [
"thiserror 2.0.19", "thiserror 2.0.20",
] ]
[[package]] [[package]]
@@ -2487,9 +2493,9 @@ dependencies = [
[[package]] [[package]]
name = "find-msvc-tools" name = "find-msvc-tools"
version = "0.1.9" version = "0.1.10"
source = "registry+https://github.com/rust-lang/crates.io-index" source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "5baebc0774151f905a1a2cc41989300b1e6fbb29aff0ceffa1064fdd3088d582" checksum = "26b73573e6edcd2af0cdf47bd6cb58f0b3839491263c314eaad1ccf24430e1de"
[[package]] [[package]]
name = "fixedbitset" name = "fixedbitset"
@@ -2562,9 +2568,9 @@ dependencies = [
[[package]] [[package]]
name = "futures" name = "futures"
version = "0.3.33" version = "0.3.34"
source = "registry+https://github.com/rust-lang/crates.io-index" source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "a88cf1f829d945f548cf8fec32c61b1f202b6d93b45848602fc02af4b12ad218" checksum = "9a31d2a3fbaaeb2af2368bbdd904aa8e812d3c04a1ee10d3171f52d556e5d0a3"
dependencies = [ dependencies = [
"futures-channel", "futures-channel",
"futures-core", "futures-core",
@@ -2577,9 +2583,9 @@ dependencies = [
[[package]] [[package]]
name = "futures-channel" name = "futures-channel"
version = "0.3.33" version = "0.3.34"
source = "registry+https://github.com/rust-lang/crates.io-index" source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "262590f4fe6afeb0bc83be1daa64e52657fe185690a958af7f3ad0e92085c5ae" checksum = "b1f9e3d69d39e4862ffed03ed071a76f9a13ba1d9109d355b0f0aa6b15e393c4"
dependencies = [ dependencies = [
"futures-core", "futures-core",
"futures-sink", "futures-sink",
@@ -2587,15 +2593,15 @@ dependencies = [
[[package]] [[package]]
name = "futures-core" name = "futures-core"
version = "0.3.33" version = "0.3.34"
source = "registry+https://github.com/rust-lang/crates.io-index" source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "2cd50c473c80f6d7c3670a752354b8e569b1a7cbfdc0419ec88e5edad85e0dc7" checksum = "92d699e522242e69e3003b94ecc1f960f3a5e015aa7c5d7486e65ad01dd94f5e"
[[package]] [[package]]
name = "futures-executor" name = "futures-executor"
version = "0.3.33" version = "0.3.34"
source = "registry+https://github.com/rust-lang/crates.io-index" source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "6754879cc9f2c66f88c6e5c35344bb0bdb0708b0352b1201815667c7eabc7458" checksum = "031b47cf1a3c6cc8bc2fc76cd437f521619387907d469316e7c0bc278f1f5432"
dependencies = [ dependencies = [
"futures-core", "futures-core",
"futures-task", "futures-task",
@@ -2604,38 +2610,38 @@ dependencies = [
[[package]] [[package]]
name = "futures-io" name = "futures-io"
version = "0.3.33" version = "0.3.34"
source = "registry+https://github.com/rust-lang/crates.io-index" source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "4577ecaa3c4f96589d473f679a71b596316f6641bc350038b962a5daf0085d7a" checksum = "53c0fa8157de1303bfffdaa1cc2a673bfffb60102f76b0ef4441659124373fed"
[[package]] [[package]]
name = "futures-macro" name = "futures-macro"
version = "0.3.33" version = "0.3.34"
source = "registry+https://github.com/rust-lang/crates.io-index" source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "2d6d3cde68c518367be28956066ddfef33813991b77a55005a69dae04bf3b10b" checksum = "9fb9654ba8355388abeb8dcb4fc62f511300867002afc858860463bdd9fe0c44"
dependencies = [ dependencies = [
"proc-macro2", "proc-macro2",
"quote", "quote",
"syn 2.0.119", "syn 3.0.3",
] ]
[[package]] [[package]]
name = "futures-sink" name = "futures-sink"
version = "0.3.33" version = "0.3.34"
source = "registry+https://github.com/rust-lang/crates.io-index" source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "e34418ac499d6305c2fb5ad0ed2f6ac998c5f8ca209b4510f7f94242c647e307" checksum = "1944426bf7d03f1d14f708785e4b33efd750b36d48a157b836b3efc15ede8e1d"
[[package]] [[package]]
name = "futures-task" name = "futures-task"
version = "0.3.33" version = "0.3.34"
source = "registry+https://github.com/rust-lang/crates.io-index" source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "b231ed28831efb4a61a08580c4bc233ec56bc009f4cd8f52da2c3cb97df0c109" checksum = "cd417de3d1d015fc3bfd2b1ea46dfc7bab72ef86f1cc7cc9c78e728b34a6d1fd"
[[package]] [[package]]
name = "futures-util" name = "futures-util"
version = "0.3.33" version = "0.3.34"
source = "registry+https://github.com/rust-lang/crates.io-index" source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "a77a90a256fce34da66415271e30f94ee91c57b04b8a2c042d9cf3220179deaa" checksum = "0d50a92467f8ba5dd6e3ee5d4bd04d73ab2e4e1c44474a0674821dfce14b79bc"
dependencies = [ dependencies = [
"futures-channel", "futures-channel",
"futures-core", "futures-core",
@@ -2781,7 +2787,7 @@ dependencies = [
"azure_identity", "azure_identity",
"azure_security_keyvault_secrets", "azure_security_keyvault_secrets",
"backtrace", "backtrace",
"base64", "base64 0.22.1",
"chacha20poly1305", "chacha20poly1305",
"chrono", "chrono",
"clap", "clap",
@@ -2807,7 +2813,7 @@ dependencies = [
"serde_with", "serde_with",
"serde_yaml", "serde_yaml",
"tempfile", "tempfile",
"thiserror 2.0.19", "thiserror 2.0.20",
"tokio", "tokio",
"validator", "validator",
"which", "which",
@@ -3214,7 +3220,7 @@ version = "0.1.20"
source = "registry+https://github.com/rust-lang/crates.io-index" source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "96547c2556ec9d12fb1578c4eaf448b04993e7fb79cbaad930a656880a6bdfa0" checksum = "96547c2556ec9d12fb1578c4eaf448b04993e7fb79cbaad930a656880a6bdfa0"
dependencies = [ dependencies = [
"base64", "base64 0.22.1",
"bytes", "bytes",
"futures-channel", "futures-channel",
"futures-util", "futures-util",
@@ -3528,10 +3534,12 @@ dependencies = [
"defmt", "defmt",
"jiff-core", "jiff-core",
"jiff-static", "jiff-static",
"jiff-tzdb-platform",
"log", "log",
"portable-atomic", "portable-atomic",
"portable-atomic-util", "portable-atomic-util",
"serde_core", "serde_core",
"windows-link",
] ]
[[package]] [[package]]
@@ -3555,6 +3563,21 @@ dependencies = [
"syn 2.0.119", "syn 2.0.119",
] ]
[[package]]
name = "jiff-tzdb"
version = "0.1.8"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "142bd39932ad231f10513df9ab62661fead8719872150b7ad02a2df79f4e141e"
[[package]]
name = "jiff-tzdb-platform"
version = "0.1.3"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "875a5a69ac2bab1a891711cf5eccbec1ce0341ea805560dcd90b7a2e925132e8"
dependencies = [
"jiff-tzdb",
]
[[package]] [[package]]
name = "jni" name = "jni"
version = "0.22.4" version = "0.22.4"
@@ -3567,7 +3590,7 @@ dependencies = [
"jni-sys", "jni-sys",
"log", "log",
"simd_cesu8", "simd_cesu8",
"thiserror 2.0.19", "thiserror 2.0.20",
"walkdir", "walkdir",
"windows-link", "windows-link",
] ]
@@ -3616,9 +3639,9 @@ dependencies = [
[[package]] [[package]]
name = "js-sys" name = "js-sys"
version = "0.3.103" version = "0.3.104"
source = "registry+https://github.com/rust-lang/crates.io-index" source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "53b44bfcdb3f8d5837a46dae1ca9660a837176eee74a28b229bc626816589102" checksum = "0e0c1080212aad755ea003d18543e8768dd432c48819efd73a7bf1e39b7a5a3a"
dependencies = [ dependencies = [
"cfg-if", "cfg-if",
"futures-util", "futures-util",
@@ -3634,7 +3657,7 @@ dependencies = [
"jsonptr", "jsonptr",
"serde", "serde",
"serde_json", "serde_json",
"thiserror 2.0.19", "thiserror 2.0.20",
] ]
[[package]] [[package]]
@@ -3653,7 +3676,7 @@ version = "10.4.0"
source = "registry+https://github.com/rust-lang/crates.io-index" source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "eba32bfb4ffdeaca3e34431072faf01745c9b26d25504aa7a6cf5684334fc4fc" checksum = "eba32bfb4ffdeaca3e34431072faf01745c9b26d25504aa7a6cf5684334fc4fc"
dependencies = [ dependencies = [
"base64", "base64 0.22.1",
"getrandom 0.2.17", "getrandom 0.2.17",
"js-sys", "js-sys",
"pem", "pem",
@@ -3845,7 +3868,7 @@ dependencies = [
"serde-value", "serde-value",
"serde_json", "serde_json",
"serde_yaml", "serde_yaml",
"thiserror 2.0.19", "thiserror 2.0.20",
"thread-id", "thread-id",
"typemap-ors", "typemap-ors",
"unicode-segmentation", "unicode-segmentation",
@@ -3992,7 +4015,7 @@ dependencies = [
"mach2", "mach2",
"nix 0.30.1", "nix 0.30.1",
"sysctl", "sysctl",
"thiserror 2.0.19", "thiserror 2.0.20",
"widestring", "widestring",
"windows 0.48.0", "windows 0.48.0",
] ]
@@ -4389,9 +4412,9 @@ checksum = "c08d65885ee38876c4f86fa503fb49d7b507c2b62552df7c70b2fce627e06381"
[[package]] [[package]]
name = "open" name = "open"
version = "5.4.0" version = "5.4.1"
source = "registry+https://github.com/rust-lang/crates.io-index" source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "a0b3d059e795d52b8a72fef45658620edd4d9c359b338564aa14391ffa511ed5" checksum = "f9cfef937e9c486488c7e3d949ae31c0f1d06bdacd75b99c086cb35356e30408"
dependencies = [ dependencies = [
"is-wsl", "is-wsl",
"libc", "libc",
@@ -4566,7 +4589,7 @@ version = "3.0.6"
source = "registry+https://github.com/rust-lang/crates.io-index" source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "1d30c53c26bc5b31a98cd02d20f25a7c8567146caf63ed593a9d87b2775291be" checksum = "1d30c53c26bc5b31a98cd02d20f25a7c8567146caf63ed593a9d87b2775291be"
dependencies = [ dependencies = [
"base64", "base64 0.22.1",
"serde_core", "serde_core",
] ]
@@ -4705,7 +4728,7 @@ version = "1.10.0"
source = "registry+https://github.com/rust-lang/crates.io-index" source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "7da1d65da6dd5d1e44199ac0f58712d241c0f439f80adea8924d832384087f85" checksum = "7da1d65da6dd5d1e44199ac0f58712d241c0f439f80adea8924d832384087f85"
dependencies = [ dependencies = [
"base64", "base64 0.22.1",
"indexmap 2.14.0", "indexmap 2.14.0",
"quick-xml 0.41.0", "quick-xml 0.41.0",
"serde", "serde",
@@ -4725,9 +4748,9 @@ dependencies = [
[[package]] [[package]]
name = "portable-atomic" name = "portable-atomic"
version = "1.14.0" version = "1.15.0"
source = "registry+https://github.com/rust-lang/crates.io-index" source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "3d20d5497ef88037a52ff98267d066e7f11fcc5e99bbfbd58a42336193aacec3" checksum = "05c8b63e8d9609db387f0324918f81d68fe27748f084ef092fb35954d0539a85"
[[package]] [[package]]
name = "portable-atomic-util" name = "portable-atomic-util"
@@ -4912,7 +4935,7 @@ dependencies = [
"rustc-hash", "rustc-hash",
"rustls 0.23.43", "rustls 0.23.43",
"socket2 0.6.5", "socket2 0.6.5",
"thiserror 2.0.19", "thiserror 2.0.20",
"tokio", "tokio",
"tracing", "tracing",
"web-time", "web-time",
@@ -4935,7 +4958,7 @@ dependencies = [
"rustls 0.23.43", "rustls 0.23.43",
"rustls-pki-types", "rustls-pki-types",
"slab", "slab",
"thiserror 2.0.19", "thiserror 2.0.20",
"tinyvec", "tinyvec",
"tracing", "tracing",
"web-time", "web-time",
@@ -5086,7 +5109,7 @@ checksum = "a4e608c6638b9c18977b00b475ac1f28d14e84b27d8d42f70e0bf1e3dec127ac"
dependencies = [ dependencies = [
"getrandom 0.2.17", "getrandom 0.2.17",
"libredox", "libredox",
"thiserror 2.0.19", "thiserror 2.0.20",
] ]
[[package]] [[package]]
@@ -5103,7 +5126,7 @@ dependencies = [
"serde", "serde",
"strip-ansi-escapes", "strip-ansi-escapes",
"strum", "strum",
"thiserror 2.0.19", "thiserror 2.0.20",
"unicase", "unicase",
"unicode-segmentation", "unicode-segmentation",
"unicode-width", "unicode-width",
@@ -5143,9 +5166,9 @@ dependencies = [
[[package]] [[package]]
name = "regex-automata" name = "regex-automata"
version = "0.4.16" version = "0.4.18"
source = "registry+https://github.com/rust-lang/crates.io-index" source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "8fcfdb36bda0c880c5931cdc7a2bcdc8ba4556847b9d912bca70bc94708711ad" checksum = "ad8553b9b26413251cbf30e620595c7a41b3887f03da04579c0e6b0d6a06b4b2"
dependencies = [ dependencies = [
"aho-corasick", "aho-corasick",
"memchr", "memchr",
@@ -5170,7 +5193,7 @@ version = "0.12.28"
source = "registry+https://github.com/rust-lang/crates.io-index" source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "eddd3ca559203180a307f12d114c268abf583f59b03cb906fd0b3ff8646c1147" checksum = "eddd3ca559203180a307f12d114c268abf583f59b03cb906fd0b3ff8646c1147"
dependencies = [ dependencies = [
"base64", "base64 0.22.1",
"bytes", "bytes",
"futures-core", "futures-core",
"futures-util", "futures-util",
@@ -5214,7 +5237,7 @@ version = "0.13.4"
source = "registry+https://github.com/rust-lang/crates.io-index" source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "219c5811de6525e5416c7d5d53bb656d3afdbc6c5af816e0802bcfa42dbdc1c3" checksum = "219c5811de6525e5416c7d5d53bb656d3afdbc6c5af816e0802bcfa42dbdc1c3"
dependencies = [ dependencies = [
"base64", "base64 0.22.1",
"bytes", "bytes",
"futures-channel", "futures-channel",
"futures-core", "futures-core",
@@ -5285,7 +5308,7 @@ source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "1d1f571c72940a19d9532fe52dbea8bc9912bf1d766c2970bb824056b86f3f59" checksum = "1d1f571c72940a19d9532fe52dbea8bc9912bf1d766c2970bb824056b86f3f59"
dependencies = [ dependencies = [
"async-trait", "async-trait",
"base64", "base64 0.22.1",
"chrono", "chrono",
"futures", "futures",
"http 1.5.0", "http 1.5.0",
@@ -5298,7 +5321,7 @@ dependencies = [
"serde", "serde",
"serde_json", "serde_json",
"sse-stream", "sse-stream",
"thiserror 2.0.19", "thiserror 2.0.20",
"tokio", "tokio",
"tokio-stream", "tokio-stream",
"tokio-util", "tokio-util",
@@ -5460,7 +5483,7 @@ dependencies = [
"once_cell", "once_cell",
"ring", "ring",
"rustls-pki-types", "rustls-pki-types",
"rustls-webpki 0.103.13", "rustls-webpki 0.103.14",
"subtle", "subtle",
"zeroize", "zeroize",
] ]
@@ -5501,7 +5524,7 @@ dependencies = [
"rustls 0.23.43", "rustls 0.23.43",
"rustls-native-certs", "rustls-native-certs",
"rustls-platform-verifier-android", "rustls-platform-verifier-android",
"rustls-webpki 0.103.13", "rustls-webpki 0.103.14",
"security-framework", "security-framework",
"security-framework-sys", "security-framework-sys",
"webpki-root-certs", "webpki-root-certs",
@@ -5526,9 +5549,9 @@ dependencies = [
[[package]] [[package]]
name = "rustls-webpki" name = "rustls-webpki"
version = "0.103.13" version = "0.103.14"
source = "registry+https://github.com/rust-lang/crates.io-index" source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "61c429a8649f110dddef65e2a5ad240f747e85f7758a6bccc7e5777bd33f756e" checksum = "0527518605e68109d875e248ea259b6758801cf165e4b2c2733ae3b51f12535a"
dependencies = [ dependencies = [
"aws-lc-rs", "aws-lc-rs",
"ring", "ring",
@@ -5828,16 +5851,17 @@ dependencies = [
[[package]] [[package]]
name = "serde_with" name = "serde_with"
version = "3.21.0" version = "3.22.0"
source = "registry+https://github.com/rust-lang/crates.io-index" source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "76a5c54c7310e7b8b9577c286d7e399ddd876c3e12b3ed917a8aabc4b96e9e8c" checksum = "ee78f1fbe43ac4a0e47aadb3dbd357b69eb0d3793e948624cd03dd2750ab1c0a"
dependencies = [ dependencies = [
"base64", "base64 0.22.1",
"bs58", "bs58",
"chrono", "chrono",
"hex", "hex",
"indexmap 1.9.3", "indexmap 1.9.3",
"indexmap 2.14.0", "indexmap 2.14.0",
"jiff",
"schemars 0.9.0", "schemars 0.9.0",
"schemars 1.2.2", "schemars 1.2.2",
"serde_core", "serde_core",
@@ -5848,9 +5872,9 @@ dependencies = [
[[package]] [[package]]
name = "serde_with_macros" name = "serde_with_macros"
version = "3.21.0" version = "3.22.0"
source = "registry+https://github.com/rust-lang/crates.io-index" source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "84d57bc0c8b9a17920c178daa6bb924850d54a9c97ab45194bb8c17ad66bb660" checksum = "8705578779c2b6bd90d84d66eb2e206b708b1a4d7b9f17641b293545bf1c7e46"
dependencies = [ dependencies = [
"darling 0.23.0", "darling 0.23.0",
"proc-macro2", "proc-macro2",
@@ -6055,7 +6079,7 @@ checksum = "0d585997b0ac10be3c5ee635f1bab02d512760d14b7c468801ac8a01d9ae5f1d"
dependencies = [ dependencies = [
"num-bigint", "num-bigint",
"num-traits", "num-traits",
"thiserror 2.0.19", "thiserror 2.0.20",
"time", "time",
] ]
@@ -6283,7 +6307,7 @@ dependencies = [
"serde", "serde",
"serde_derive", "serde_derive",
"serde_json", "serde_json",
"thiserror 2.0.19", "thiserror 2.0.20",
"walkdir", "walkdir",
] ]
@@ -6417,11 +6441,11 @@ dependencies = [
[[package]] [[package]]
name = "thiserror" name = "thiserror"
version = "2.0.19" version = "2.0.20"
source = "registry+https://github.com/rust-lang/crates.io-index" source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "09a43598840e33d5b0331f38c5e30d13bb11c11210a4b58f0d9b18a5a5eefcd9" checksum = "ec86235f5fcc2a73650310756d2ac5b138a5780bbbdfae3eeccec992c435ba4f"
dependencies = [ dependencies = [
"thiserror-impl 2.0.19", "thiserror-impl 2.0.20",
] ]
[[package]] [[package]]
@@ -6437,9 +6461,9 @@ dependencies = [
[[package]] [[package]]
name = "thiserror-impl" name = "thiserror-impl"
version = "2.0.19" version = "2.0.20"
source = "registry+https://github.com/rust-lang/crates.io-index" source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "43cbfe0cf76104d42a574802844187e84a305e531ed54455f11fbde0f10541cd" checksum = "bc04cd3e1236dd4a98afca4569f2deb3f120e5422a4023be2cb683f8486292af"
dependencies = [ dependencies = [
"proc-macro2", "proc-macro2",
"quote", "quote",
@@ -6658,7 +6682,7 @@ checksum = "ac2a5518c70fa84342385732db33fb3f44bc4cc748936eb5833d2df34d6445ef"
dependencies = [ dependencies = [
"async-trait", "async-trait",
"axum", "axum",
"base64", "base64 0.22.1",
"bytes", "bytes",
"h2 0.4.15", "h2 0.4.15",
"http 1.5.0", "http 1.5.0",
@@ -6779,9 +6803,9 @@ dependencies = [
[[package]] [[package]]
name = "tree-sitter" name = "tree-sitter"
version = "0.26.11" version = "0.26.12"
source = "registry+https://github.com/rust-lang/crates.io-index" source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "af1c71c1c4cc0920b20d6b0f6572e7682cd07a6a2faec71067a31fa394c586df" checksum = "83c567a8e18ae93f20982c90370b16fd24023aeaf52f6052b96957ab253a0fec"
dependencies = [ dependencies = [
"cc", "cc",
"regex", "regex",
@@ -6855,7 +6879,7 @@ version = "0.11.0"
source = "registry+https://github.com/rust-lang/crates.io-index" source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "4dd1eb4a538c1ab3d5c05437129bc16891296146b23c9b0bb3f5df99f5b3a18d" checksum = "4dd1eb4a538c1ab3d5c05437129bc16891296146b23c9b0bb3f5df99f5b3a18d"
dependencies = [ dependencies = [
"base64", "base64 0.22.1",
"bytes", "bytes",
"futures", "futures",
"serde", "serde",
@@ -6870,7 +6894,7 @@ source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "e632235c99ae896a3c451d1ead00cea11a2219aeda1b35a74027fe99ea3f3b72" checksum = "e632235c99ae896a3c451d1ead00cea11a2219aeda1b35a74027fe99ea3f3b72"
dependencies = [ dependencies = [
"async-trait", "async-trait",
"base64", "base64 0.22.1",
"dyn-clone", "dyn-clone",
"futures", "futures",
"getrandom 0.3.4", "getrandom 0.3.4",
@@ -6981,11 +7005,11 @@ checksum = "6d49784317cd0d1ee7ec5c716dd598ec5b4483ea832a2dced265471cc0f690ae"
[[package]] [[package]]
name = "ureq" name = "ureq"
version = "3.3.0" version = "3.4.0"
source = "registry+https://github.com/rust-lang/crates.io-index" source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "dea7109cdcd5864d4eeb1b58a1648dc9bf520360d7af16ec26d0a9354bafcfc0" checksum = "972d7902c8735f2695410b8aed7df6ed12a47394aa1c8d7af49f0497b731a94d"
dependencies = [ dependencies = [
"base64", "base64 0.23.1",
"cookie_store", "cookie_store",
"encoding_rs", "encoding_rs",
"flate2", "flate2",
@@ -7003,11 +7027,11 @@ dependencies = [
[[package]] [[package]]
name = "ureq-proto" name = "ureq-proto"
version = "0.6.0" version = "0.6.1"
source = "registry+https://github.com/rust-lang/crates.io-index" source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "e994ba84b0bd1b1b0cf92878b7ef898a5c1760108fe7b6010327e274917a808c" checksum = "da5f78b09e6941e1a0f2e30e695e4b120377b54d5e0aec11b594bb57b3971613"
dependencies = [ dependencies = [
"base64", "base64 0.23.1",
"http 1.5.0", "http 1.5.0",
"httparse", "httparse",
"log", "log",
@@ -7159,9 +7183,9 @@ dependencies = [
[[package]] [[package]]
name = "wasm-bindgen" name = "wasm-bindgen"
version = "0.2.126" version = "0.2.127"
source = "registry+https://github.com/rust-lang/crates.io-index" source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "4b067c0c11094aef6b7a801c1e34a26affafdf3d051dba08456b868789aaf9a4" checksum = "1b70935747edd64d89de3efa29d73789b806c15798f8e7dca4d8ac356b50ce70"
dependencies = [ dependencies = [
"cfg-if", "cfg-if",
"once_cell", "once_cell",
@@ -7172,9 +7196,9 @@ dependencies = [
[[package]] [[package]]
name = "wasm-bindgen-futures" name = "wasm-bindgen-futures"
version = "0.4.76" version = "0.4.77"
source = "registry+https://github.com/rust-lang/crates.io-index" source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "c62df1340f32221cb9c54d6a27b030e3dba64361d4a95bed55f9aacb44da291d" checksum = "6b7777d5cc23d0e91404e53ce2d5e8ec7acae3026b16233dba62cd3246457950"
dependencies = [ dependencies = [
"js-sys", "js-sys",
"wasm-bindgen", "wasm-bindgen",
@@ -7182,9 +7206,9 @@ dependencies = [
[[package]] [[package]]
name = "wasm-bindgen-macro" name = "wasm-bindgen-macro"
version = "0.2.126" version = "0.2.127"
source = "registry+https://github.com/rust-lang/crates.io-index" source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "167ce5e579f6bcf889c4f7175a8a5a585de84e8ff93976ce393efa5f2837aab1" checksum = "77775f8f3f7217702089053b94958f8f54061a3f663417df76e19cbdcca29bc1"
dependencies = [ dependencies = [
"quote", "quote",
"wasm-bindgen-macro-support", "wasm-bindgen-macro-support",
@@ -7192,9 +7216,9 @@ dependencies = [
[[package]] [[package]]
name = "wasm-bindgen-macro-support" name = "wasm-bindgen-macro-support"
version = "0.2.126" version = "0.2.127"
source = "registry+https://github.com/rust-lang/crates.io-index" source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "f3997c7839262f4ef12cf90b818d6340c18e80f263f1a94bf157d0ec4420380e" checksum = "e11d33f857dc2fb11b8bc75aee111aa9cbeb12cd9f25efd3d4c2a3dd4e235284"
dependencies = [ dependencies = [
"bumpalo", "bumpalo",
"proc-macro2", "proc-macro2",
@@ -7205,9 +7229,9 @@ dependencies = [
[[package]] [[package]]
name = "wasm-bindgen-shared" name = "wasm-bindgen-shared"
version = "0.2.126" version = "0.2.127"
source = "registry+https://github.com/rust-lang/crates.io-index" source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "dc1b4cb0cc549fcf58d7dfc081778139b3d283a081644e833e84682ad71cea24" checksum = "7ef64dbcc55df09c7e5a46182d181c2cfa3e925f3da937ea764728b4bbb9dcbf"
dependencies = [ dependencies = [
"unicode-ident", "unicode-ident",
] ]
@@ -7310,9 +7334,9 @@ dependencies = [
[[package]] [[package]]
name = "web-sys" name = "web-sys"
version = "0.3.103" version = "0.3.104"
source = "registry+https://github.com/rust-lang/crates.io-index" source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "8622dcb61c0bcc9fffa6938bed81210af2da9a7e4a1a834b2e37a59b6dfb6141" checksum = "c435338968042f4f59a557f690a253676d47ce13ceb55d70100e7facf6620a30"
dependencies = [ dependencies = [
"js-sys", "js-sys",
"wasm-bindgen", "wasm-bindgen",
@@ -7749,7 +7773,7 @@ dependencies = [
"log", "log",
"os_pipe", "os_pipe",
"rustix 1.1.4", "rustix 1.1.4",
"thiserror 2.0.19", "thiserror 2.0.20",
"tree_magic_mini", "tree_magic_mini",
"wayland-backend", "wayland-backend",
"wayland-client", "wayland-client",
@@ -7844,18 +7868,18 @@ dependencies = [
[[package]] [[package]]
name = "zerocopy" name = "zerocopy"
version = "0.8.55" version = "0.8.56"
source = "registry+https://github.com/rust-lang/crates.io-index" source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "b5a105cd7b140f6eeec8acff2ea38135d3cab283ada58540f629fe51e46696eb" checksum = "556764e583adb45a9f8d413c2a147fa7e8d821e48e12b14fd560b607998b75eb"
dependencies = [ dependencies = [
"zerocopy-derive", "zerocopy-derive",
] ]
[[package]] [[package]]
name = "zerocopy-derive" name = "zerocopy-derive"
version = "0.8.55" version = "0.8.56"
source = "registry+https://github.com/rust-lang/crates.io-index" source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "0fe976fb70c78cd64cccfe3a6fc142244e8a77b70959b30faf9d0ac37ee228eb" checksum = "f2ab42fc20575779bd240faa45f94a74256f755c0fa9e89f0ede20d91d0cdfc1"
dependencies = [ dependencies = [
"proc-macro2", "proc-macro2",
"quote", "quote",
@@ -7957,9 +7981,9 @@ version = "0.1.5"
source = "registry+https://github.com/rust-lang/crates.io-index" source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "dba6063ff82cdbd9a765add16d369abe81e520f836054e997c2db217ceca40c0" checksum = "dba6063ff82cdbd9a765add16d369abe81e520f836054e997c2db217ceca40c0"
dependencies = [ dependencies = [
"base64", "base64 0.22.1",
"ed25519-dalek", "ed25519-dalek",
"thiserror 2.0.19", "thiserror 2.0.20",
] ]
[[package]] [[package]]
assets/agents/.shared/utils.sh
+97 -21
View File
@@ -40,15 +40,57 @@ _write_project_cache() {
_detect_heuristic() { _detect_heuristic() {
local dir="$1" local dir="$1"
local runner="" runner_type="" runner_targets=""
if [[ -f "${dir}/Taskfile.yml" || -f "${dir}/Taskfile.yaml" || -f "${dir}/taskfile.yml" || -f "${dir}/taskfile.yaml" ]]; then
runner="task" runner_type="taskfile"
runner_targets=$( (cd "${dir}" && task --list-all 2>/dev/null | sed -n 's/^\* \([^:[:space:]]*\):.*/\1/p') || true)
elif [[ -f "${dir}/justfile" || -f "${dir}/Justfile" ]]; then
runner="just" runner_type="just"
runner_targets=$( (cd "${dir}" && just --summary 2>/dev/null | tr ' ' '\n') || true)
elif [[ -f "${dir}/Makefile" || -f "${dir}/makefile" || -f "${dir}/GNUmakefile" ]]; then
runner="make" runner_type="make"
local mk mkfiles=()
for mk in Makefile makefile GNUmakefile; do
[[ -f "${dir}/${mk}" ]] && mkfiles+=("${dir}/${mk}")
done
runner_targets=$(sed -n 's/^\([A-Za-z0-9_][A-Za-z0-9_.-]*\):\([^=].*\|\)$/\1/p' "${mkfiles[@]}" 2>/dev/null | sort -u || true)
fi
if [[ -n "${runner}" && -n "${runner_targets}" ]]; then
_pick_target() {
local c
for c in "$@"; do
if grep -qx "${c}" <<<"${runner_targets}"; then
echo "${runner} ${c}"
return 0
fi
done
echo ""
}
local r_build r_test r_check r_lint r_fmt
r_build=$(_pick_target build compile)
r_test=$(_pick_target test tests unit)
r_check=$(_pick_target check vet typecheck build)
r_lint=$(_pick_target lint fmt-check)
r_fmt=$(_pick_target fmt format)
if [[ -n "${r_build}${r_test}${r_check}${r_lint}${r_fmt}" ]]; then
echo "{\"type\":\"${runner_type}\",\"build\":\"${r_build}\",\"test\":\"${r_test}\",\"check\":\"${r_check}\",\"lint\":\"${r_lint}\",\"fmt\":\"${r_fmt}\"}"
return 0
fi
fi
# Rust # Rust
if [[ -f "${dir}/Cargo.toml" ]]; then if [[ -f "${dir}/Cargo.toml" ]]; then
echo '{"type":"rust","build":"cargo build","test":"cargo test","check":"cargo check"}' echo '{"type":"rust","build":"cargo build","test":"cargo test","check":"cargo check","lint":"cargo clippy --no-deps -- -D warnings","fmt":"cargo fmt"}'
return 0 return 0
fi fi
# Go # Go
if [[ -f "${dir}/go.mod" ]]; then if [[ -f "${dir}/go.mod" ]]; then
echo '{"type":"go","build":"go build ./...","test":"go test ./...","check":"go vet ./..."}' local go_lint=""
if compgen -G "${dir}/.golangci.*" &>/dev/null && command -v golangci-lint &>/dev/null; then
go_lint="golangci-lint run"
fi
echo "{\"type\":\"go\",\"build\":\"go build ./...\",\"test\":\"go test ./...\",\"check\":\"go vet ./...\",\"lint\":\"${go_lint}\",\"fmt\":\"gofmt -w .\"}"
return 0 return 0
fi fi
@@ -65,7 +107,25 @@ _detect_heuristic() {
[[ -f "${dir}/pnpm-lock.yaml" ]] && pm="pnpm" [[ -f "${dir}/pnpm-lock.yaml" ]] && pm="pnpm"
[[ -f "${dir}/yarn.lock" ]] && pm="yarn" [[ -f "${dir}/yarn.lock" ]] && pm="yarn"
echo "{\"type\":\"nodejs\",\"build\":\"${pm} run build\",\"test\":\"${pm} test\",\"check\":\"${pm} run lint\"}" # Emit only scripts the manifest actually declares (same introspection
# contract as the runner tier: never guess a target into existence).
_pkg_script() {
local s
for s in "$@"; do
if jq -e --arg s "$s" '.scripts[$s] // empty' "${dir}/package.json" &>/dev/null; then
echo "${pm} run ${s}"
return 0
fi
done
echo ""
}
local p_build p_test p_check p_lint p_fmt
p_build=$(_pkg_script build compile)
p_test=$(_pkg_script test)
p_check=$(_pkg_script check typecheck tsc)
p_lint=$(_pkg_script lint)
p_fmt=$(_pkg_script fmt format prettier)
echo "{\"type\":\"nodejs\",\"build\":\"${p_build}\",\"test\":\"${p_test}\",\"check\":\"${p_check}\",\"lint\":\"${p_lint}\",\"fmt\":\"${p_fmt}\"}"
return 0 return 0
fi fi
@@ -82,7 +142,7 @@ _detect_heuristic() {
check_cmd="uv run ruff check ." check_cmd="uv run ruff check ."
fi fi
echo "{\"type\":\"python\",\"build\":\"\",\"test\":\"${test_cmd}\",\"check\":\"${check_cmd}\"}" echo "{\"type\":\"python\",\"build\":\"\",\"test\":\"${test_cmd}\",\"check\":\"${check_cmd}\",\"lint\":\"${check_cmd}\",\"fmt\":\"ruff format .\"}"
return 0 return 0
fi fi
@@ -144,17 +204,6 @@ _detect_heuristic() {
return 0 return 0
fi fi
# Generic build systems (last resort before LLM)
if [[ -f "${dir}/justfile" ]] || [[ -f "${dir}/Justfile" ]]; then
echo '{"type":"just","build":"just build","test":"just test","check":"just lint"}'
return 0
fi
if [[ -f "${dir}/Makefile" ]] || [[ -f "${dir}/makefile" ]] || [[ -f "${dir}/GNUmakefile" ]]; then
echo '{"type":"make","build":"make build","test":"make test","check":"make lint"}'
return 0
fi
return 1 return 1
} }
@@ -218,7 +267,9 @@ _detect_with_llm() {
local prompt local prompt
prompt=$(cat <<-EOF prompt=$(cat <<-EOF
Analyze this project directory and determine the project type, primary language, and the correct shell commands to build, test, and check (lint/typecheck) it. Analyze this project directory and determine the project type, primary language, and the correct shell commands to build, test, check (typecheck/vet), lint, and format it.
PRIORITY RULE: if the project declares its own task-runner interface (a Taskfile, justfile, Makefile, package.json scripts, or similar), those declared targets ARE the correct commands — prefer them over generic ecosystem defaults, and never invent a target the interface does not declare.
EOF EOF
) )
@@ -226,12 +277,12 @@ _detect_with_llm() {
prompt+=$(cat <<-EOF prompt+=$(cat <<-EOF
Respond with ONLY a valid JSON object. No markdown fences, no explanation, no extra text. Respond with ONLY a valid JSON object. No markdown fences, no explanation, no extra text.
The JSON must have exactly these 4 keys: The JSON must have exactly these 6 keys:
{"type":"<language>","build":"<build command>","test":"<test command>","check":"<lint or typecheck command>"} {"type":"<language>","build":"<build command>","test":"<test command>","check":"<typecheck/vet command>","lint":"<lint command>","fmt":"<format command>"}
Rules: Rules:
- "type" must be a single lowercase word (e.g. rust, go, python, nodejs, java, ruby, elixir, cpp, c, zig, haskell, scala, kotlin, dart, swift, php, dotnet, etc.) - "type" must be a single lowercase word (e.g. rust, go, python, nodejs, java, ruby, elixir, cpp, c, zig, haskell, scala, kotlin, dart, swift, php, dotnet, etc.)
- If a command doesn't apply to this project, use an empty string, "" - If a command doesn't apply to this project, use an empty string, "" — NEVER guess a command that might not exist; a wrongly-guessed command is worse than an empty one
- Use the most standard/common commands for the detected ecosystem - Use the most standard/common commands for the detected ecosystem
- If you detect a package manager lockfile, use that package manager (e.g. pnpm over npm) - If you detect a package manager lockfile, use that package manager (e.g. pnpm over npm)
EOF EOF
@@ -244,7 +295,7 @@ _detect_with_llm() {
llm_response=$(echo "${llm_response}" | grep -o '{[^}]*}' | head -1) llm_response=$(echo "${llm_response}" | grep -o '{[^}]*}' | head -1)
if echo "${llm_response}" | jq -e '.type and .build != null and .test != null and .check != null' &>/dev/null; then if echo "${llm_response}" | jq -e '.type and .build != null and .test != null and .check != null' &>/dev/null; then
echo "${llm_response}" | jq -c '{type: (.type // "unknown"), build: (.build // ""), test: (.test // ""), check: (.check // "")}' echo "${llm_response}" | jq -c '{type: (.type // "unknown"), build: (.build // ""), test: (.test // ""), check: (.check // ""), lint: (.lint // ""), fmt: (.fmt // "")}'
return 0 return 0
fi fi
@@ -258,7 +309,7 @@ detect_project() {
local cached local cached
if cached=$(_read_project_cache "${dir}"); then if cached=$(_read_project_cache "${dir}"); then
echo "${cached}" | jq -c '{type, build, test, check}' echo "${cached}" | jq -c '{type, build, test, check, lint: (.lint // ""), fmt: (.fmt // "")}'
return 0 return 0
fi fi
@@ -286,6 +337,31 @@ detect_project() {
echo '{"type":"unknown","build":"","test":"","check":""}' echo '{"type":"unknown","build":"","test":"","check":""}'
} }
# resolve_gate_dir maps a workspace root to the directory verification gates
# must run in. A delivery-repo worker's workspace root holds only dotfiles
# plus the clone, so gates aimed at the root detect nothing and silently
# no-op. When the root has no project markers and exactly ONE first-level
# git repo exists, gates run inside it; anything ambiguous stays at the root.
resolve_gate_dir() {
local dir="${1:-.}"
local m
for m in Taskfile.yml Taskfile.yaml taskfile.yml Cargo.toml go.mod package.json pyproject.toml setup.py pom.xml build.gradle mix.exs Gemfile composer.json Makefile justfile Justfile CMakeLists.txt; do
if [[ -e "${dir}/${m}" ]]; then
echo "${dir}"
return 0
fi
done
local repos=() d
for d in "${dir}"/*/; do
[[ -d "${d}/.git" ]] && repos+=("${d}")
done
if [[ ${#repos[@]} -eq 1 ]]; then
echo "${repos[0]%/}"
return 0
fi
echo "${dir}"
}
########################### ###########################
## FILE SEARCH UTILITIES ## ## FILE SEARCH UTILITIES ##
########################### ###########################
assets/agents/coder/graph.yaml
+6 -1
View File
@@ -227,6 +227,11 @@ nodes:
on unfamiliar lints, etc.). on unfamiliar lints, etc.).
4. No dead code, no commented-out blocks, no premature abstractions. 4. No dead code, no commented-out blocks, no premature abstractions.
5. End your turn when editing is done. The graph runs verification next. 5. End your turn when editing is done. The graph runs verification next.
6. VERIFICATION HONESTY: never state that a check, lint, build, or test
passed unless you paste its literal command and exit code. A gate
that did not run is UNVERIFIED — say so. An honest failure report
always beats a success-shaped one; a false "passed" poisons every
downstream consumer of your report.
Project directory: {{project_dir}} Project directory: {{project_dir}}
prompt: | prompt: |
@@ -248,7 +253,7 @@ nodes:
- fs_write - fs_write
- fs_patch - fs_patch
- execute_command - execute_command
max_iterations: 30 max_iterations: 100
state_updates: state_updates:
last_node_output: '{{output}}' last_node_output: '{{output}}'
fallback: end_failure fallback: end_failure
assets/agents/coder/scripts/verify_build.sh
+2 -1
View File
@@ -13,6 +13,7 @@ else
fi fi
project_dir=$(echo "$state" | jq -r '.project_dir // "."') project_dir=$(echo "$state" | jq -r '.project_dir // "."')
project_dir=$(resolve_gate_dir "$project_dir")
if [[ -n "${BUILD_CMD:-}" ]]; then if [[ -n "${BUILD_CMD:-}" ]]; then
cmd="$BUILD_CMD" cmd="$BUILD_CMD"
@@ -24,7 +25,7 @@ fi
if [[ -z "$cmd" || "$cmd" == "null" ]]; then if [[ -z "$cmd" || "$cmd" == "null" ]]; then
jq -nc '{ jq -nc '{
"build_ok": true, "build_ok": true,
"build_output": "(no build/check command available for this project type)", "build_output": "(GATE NOT RUN: no build/check command configured or detected. This is NOT evidence that the build passed — set BUILD_CMD, and never report the build as verified.)",
"_next": "verify_tests" "_next": "verify_tests"
}' }'
exit 0 exit 0
assets/agents/coder/scripts/verify_tests.sh
+2 -1
View File
@@ -13,6 +13,7 @@ else
fi fi
project_dir=$(echo "$state" | jq -r '.project_dir // "."') project_dir=$(echo "$state" | jq -r '.project_dir // "."')
project_dir=$(resolve_gate_dir "$project_dir")
if [[ -n "${TEST_CMD:-}" ]]; then if [[ -n "${TEST_CMD:-}" ]]; then
cmd="$TEST_CMD" cmd="$TEST_CMD"
@@ -24,7 +25,7 @@ fi
if [[ -z "$cmd" || "$cmd" == "null" ]]; then if [[ -z "$cmd" || "$cmd" == "null" ]]; then
jq -nc '{ jq -nc '{
"tests_ok": true, "tests_ok": true,
"tests_output": "(no test command available for this project type)", "tests_output": "(GATE NOT RUN: no test command configured or detected. This is NOT evidence that tests passed — set TEST_CMD, and never report the suite as green.)",
"_next": "self_review" "_next": "self_review"
}' }'
exit 0 exit 0
assets/agents/sisyphus/config.yaml
+6
View File
@@ -266,6 +266,12 @@ instructions: |
**No evidence = not complete.** Mark a todo `completed` only after evidence is collected. **No evidence = not complete.** Mark a todo `completed` only after evidence is collected.
### Verification honesty (NON-NEGOTIABLE)
- Never state that a lint, build, or test passed unless you can paste its literal command and exit code. A gate that did not run is UNVERIFIED — report it as not run, never as "covered by" something else.
- Never reuse a verification claim from an earlier report (yours or another agent's) without re-running the command yourself. Prior reports are unverified context, not evidence.
- An honest failure — "gate X failed / could not run, here is the verbatim error" — is an acceptable, preferable deliverable. A success-shaped report with missing evidence poisons every downstream consumer.
### Independent code review (post-coder, non-trivial work) ### Independent code review (post-coder, non-trivial work)
After completing delegated `coder` work, spawn `code-reviewer` for an independent review pass if ANY of these are true: After completing delegated `coder` work, spawn `code-reviewer` for an independent review pass if ANY of these are true:
assets/agents/sisyphus/sbx-mixin.yaml
+33 -6
View File
@@ -1,11 +1,38 @@
schemaVersion: '1' schemaVersion: '2'
kind: mixin kind: mixin
name: sisyphus-ddg name: sisyphus-ddg
description: > description: >
Allows Sisyphus to hit all domains since it utilizes the DuckDuckGo Allows Sisyphus to reach DuckDuckGo plus a curated set of common
MCP server. This allows the MCP server to actually perform web searches content domains for its web-search MCP server. Schema v2 removed
on arbitrary domains and retrieve info for the agent. the bare '*' allow-all, so frequently fetched result domains are
enumerated here.
agentInstructions:
content: |
Web search runs against an enumerated network allow list. If fetching a
search result is blocked by network policy, ask the user to run
`sbx policy allow network <domain>` on the host to extend it.
permissions:
network: network:
allowedDomains: allow:
- '*' # DuckDuckGo search endpoints used by the ddg-search MCP server
- 'duckduckgo.com'
- 'html.duckduckgo.com'
- 'lite.duckduckgo.com'
# Common content/result domains fetched from search results
# ('*.host' matches exactly one label and not the bare host itself)
- '*.wikipedia.org'
- 'github.com'
- '*.githubusercontent.com'
- 'stackoverflow.com'
- '*.stackexchange.com'
- 'developer.mozilla.org'
- 'docs.python.org'
- 'doc.rust-lang.org'
- 'docs.rs'
- 'crates.io'
- 'pypi.org'
- 'www.npmjs.com'
# Jina reader fallback for fetching arbitrary pages as markdown
- 'r.jina.ai'
assets/agents/step-runner/graph.yaml
+6
View File
@@ -439,6 +439,12 @@ nodes:
staleness report, gate decisions, and fix loop history. Downstream staleness report, gate decisions, and fix loop history. Downstream
plan updates come from the sweep results. plan updates come from the sweep results.
VERIFICATION HONESTY: evidence marked "GATE NOT RUN" means that gate
is UNVERIFIED — record it as not run; never paraphrase a skipped gate
as covered, passing, or handled elsewhere. A handoff that admits an
unverified gate is correct; one that dresses it up as verified poisons
every downstream reader.
Then append durable, step-independent facts (if any) to {{notes_path}} Then append durable, step-independent facts (if any) to {{notes_path}}
- create the file if missing, never rewrite existing entries. - create the file if missing, never rewrite existing entries.
assets/agents/step-runner/scripts/verify_build.sh
+2 -1
View File
@@ -13,6 +13,7 @@ else
fi fi
project_dir=$(echo "$state" | jq -r '.project_dir // "."') project_dir=$(echo "$state" | jq -r '.project_dir // "."')
project_dir=$(resolve_gate_dir "$project_dir")
if [[ -n "${BUILD_CMD:-}" ]]; then if [[ -n "${BUILD_CMD:-}" ]]; then
cmd="$BUILD_CMD" cmd="$BUILD_CMD"
@@ -24,7 +25,7 @@ fi
if [[ -z "$cmd" || "$cmd" == "null" ]]; then if [[ -z "$cmd" || "$cmd" == "null" ]]; then
jq -nc '{ jq -nc '{
"build_ok": true, "build_ok": true,
"build_output": "(no build/check command available for this project type)", "build_output": "(GATE NOT RUN: no build/check command configured or detected. This is NOT evidence that the build passed — set BUILD_CMD, and never report the build as verified.)",
"_next": "verify_tests" "_next": "verify_tests"
}' }'
exit 0 exit 0
assets/agents/step-runner/scripts/verify_format_lint.sh
+13 -8
View File
@@ -13,19 +13,18 @@ else
fi fi
project_dir=$(echo "$state" | jq -r '.project_dir // "."') project_dir=$(echo "$state" | jq -r '.project_dir // "."')
project_type=$(detect_project "$project_dir" | jq -r '.type // "unknown"') project_dir=$(resolve_gate_dir "$project_dir")
project_info=$(detect_project "$project_dir")
project_type=$(echo "$project_info" | jq -r '.type // "unknown"')
format_cmd="${FORMAT_CMD:-}" format_cmd="${FORMAT_CMD:-}"
if [[ -z "$format_cmd" ]]; then if [[ -z "$format_cmd" ]]; then
case "$project_type" in format_cmd=$(echo "$project_info" | jq -r '.fmt // ""')
rust) format_cmd="cargo fmt" ;;
go) format_cmd="gofmt -w ." ;;
python) command -v ruff &>/dev/null && format_cmd="ruff format ." ;;
esac
fi fi
if [[ "$format_cmd" == "null" ]]; then format_cmd=""; fi
if [[ -z "$format_cmd" ]]; then if [[ -z "$format_cmd" ]]; then
format_output="(no format command configured for project type '$project_type'; skipped. Set FORMAT_CMD to enable.)" format_output="(GATE NOT RUN: no format command configured or detected for project type '$project_type'. This is NOT evidence that formatting is clean. Set FORMAT_CMD to enable.)"
else else
fmt_rc=0 fmt_rc=0
fmt_out=$(cd "$project_dir" && eval "$format_cmd" 2>&1) || fmt_rc=$? fmt_out=$(cd "$project_dir" && eval "$format_cmd" 2>&1) || fmt_rc=$?
@@ -37,12 +36,18 @@ fi
lint_cmd="${LINT_CMD:-}" lint_cmd="${LINT_CMD:-}"
if [[ -z "$lint_cmd" ]]; then if [[ -z "$lint_cmd" ]]; then
lint_cmd=$(echo "$project_info" | jq -r '.lint // ""')
fi
# The skip message must read as a WARNING, never a reassurance: the previous
# wording ("linting is covered by the build/check command") was quoted
# verbatim by workers as false evidence that linting passed
if [[ -z "$lint_cmd" || "$lint_cmd" == "null" ]]; then
jq -nc \ jq -nc \
--arg fo "$format_output" \ --arg fo "$format_output" \
'{ '{
"format_output": $fo, "format_output": $fo,
"lint_ok": true, "lint_ok": true,
"lint_output": "(no LINT_CMD configured; linting is covered by the build/check command)", "lint_output": "(GATE NOT RUN: no lint command configured or detected. This is NOT evidence that linting passed — set LINT_CMD or add a Taskfile lint target, and never report linting as covered.)",
"_next": "verify_build" "_next": "verify_build"
}' }'
exit 0 exit 0
assets/agents/step-runner/scripts/verify_tests.sh
+2 -1
View File
@@ -13,6 +13,7 @@ else
fi fi
project_dir=$(echo "$state" | jq -r '.project_dir // "."') project_dir=$(echo "$state" | jq -r '.project_dir // "."')
project_dir=$(resolve_gate_dir "$project_dir")
if [[ -n "${TEST_CMD:-}" ]]; then if [[ -n "${TEST_CMD:-}" ]]; then
cmd="$TEST_CMD" cmd="$TEST_CMD"
@@ -24,7 +25,7 @@ fi
if [[ -z "$cmd" || "$cmd" == "null" ]]; then if [[ -z "$cmd" || "$cmd" == "null" ]]; then
jq -nc '{ jq -nc '{
"tests_ok": true, "tests_ok": true,
"tests_output": "(no test command available for this project type)", "tests_output": "(GATE NOT RUN: no test command configured or detected. This is NOT evidence that tests passed — set TEST_CMD, and never report the suite as green.)",
"_next": "edge_case_sweep" "_next": "edge_case_sweep"
}' }'
exit 0 exit 0
assets/functions/sbx-mixin.yaml
+22 -22
View File
@@ -1,4 +1,4 @@
schemaVersion: "1" schemaVersion: '2'
kind: mixin kind: mixin
name: built-in-tools name: built-in-tools
description: > description: >
@@ -6,39 +6,39 @@ description: >
global tools and the default MCP server set. Auto-applied by Coyote's sbx global tools and the default MCP server set. Auto-applied by Coyote's sbx
mixin discovery when running `coyote --sandbox`. mixin discovery when running `coyote --sandbox`.
permissions:
network: network:
allowedDomains: allow:
# fetch_url_via_jina + jina reader fallback # fetch_url_via_jina + jina reader fallback
- "r.jina.ai:443" - 'r.jina.ai'
# get_current_weather (.sh, .py, .ts) # get_current_weather (.sh, .py, .ts)
- "wttr.in:443" - 'wttr.in'
# search_arxiv (the .sh tool still uses http://, so :80 is required until fixed) # search_arxiv (the .sh tool still uses http://, so :80 is required until fixed)
- "export.arxiv.org:443" - 'export.arxiv.org'
- "export.arxiv.org:80" - 'export.arxiv.org:80'
# search_arxiv + search_wikipedia may follow DOI redirects # search_arxiv + search_wikipedia may follow DOI redirects
- "doi.org:443" - 'doi.org'
# search_wikipedia # search_wikipedia
- "en.wikipedia.org:443" - 'en.wikipedia.org'
# search_wolframalpha # search_wolframalpha
- "api.wolframalpha.com:443" - 'api.wolframalpha.com'
# web_search_perplexity # web_search_perplexity
- "api.perplexity.ai:443" - 'api.perplexity.ai'
# web_search_tavily # web_search_tavily
- "api.tavily.com:443" - 'api.tavily.com'
# send_twilio # send_twilio
- "api.twilio.com:443" - 'api.twilio.com'
# MCP: github (built-in mcp.json: api.githubcopilot.com) # MCP: github (built-in mcp.json: api.githubcopilot.com)
- "api.githubcopilot.com:443" - 'api.githubcopilot.com'
# MCP: atlassian (built-in mcp.json: mcp-remote -> mcp.atlassian.com) # MCP: atlassian (built-in mcp.json: mcp-remote -> mcp.atlassian.com)
- "mcp.atlassian.com:443" - 'mcp.atlassian.com'
# MCP: ddg-search (built-in mcp.json: uvx duckduckgo-mcp-server) # MCP: ddg-search (built-in mcp.json: uvx duckduckgo-mcp-server)
- "duckduckgo.com:443" - 'duckduckgo.com'
- "html.duckduckgo.com:443" - 'html.duckduckgo.com'
- "lite.duckduckgo.com:443" - 'lite.duckduckgo.com'
# MCP: npx-based servers (mcp-remote) pull from npm # MCP: npx-based servers (mcp-remote) pull from npm
- "registry.npmjs.org:443" - 'registry.npmjs.org'
# MCP: docker server may pull images from common registries # MCP: docker server may pull images from common registries
- "ghcr.io:443" - 'ghcr.io'
- "registry-1.docker.io:443" - 'registry-1.docker.io'
- "auth.docker.io:443" - 'auth.docker.io'
- "production.cloudflare.docker.com:443"
assets/sbx-kit/spec.yaml
+267 -222
View File
@@ -4,7 +4,7 @@
# sbx create --kit ./sbx-kit/ coyote --name testing . # sbx create --kit ./sbx-kit/ coyote --name testing .
# sbx cp $HOME/.config/coyote/ testing:/home/agent/.config/ # sbx cp $HOME/.config/coyote/ testing:/home/agent/.config/
# sbx run testing --kit ./sbx-kit/ # sbx run testing --kit ./sbx-kit/
schemaVersion: '1' schemaVersion: '2'
kind: sandbox kind: sandbox
name: coyote name: coyote
displayName: Coyote displayName: Coyote
@@ -14,198 +14,255 @@ description: >
sandbox: sandbox:
image: 'darkalex17/coyote:v0.8.3' image: 'darkalex17/coyote:v0.8.3'
aiFilename: COYOTE.md entrypoint: ['bash', '-lc', 'exec /home/agent/.cargo/bin/coyote']
entrypoint:
run: ['bash', '-lc', 'exec /home/agent/.cargo/bin/coyote']
permissions:
network: network:
# Proxy-managed LLM providers: the proxy substitutes `proxy-managed` for allow:
# the env var inside the sandbox and rewrites the auth header per
# serviceAuth at request time. Multiple domains may map to one service
# (e.g. jina) so they share a single credential.
serviceDomains:
api.openai.com: openai
api.anthropic.com: anthropic
generativelanguage.googleapis.com: gemini
api.cohere.ai: cohere
api.groq.com: groq
openrouter.ai: openrouter
api.ai21.com: ai21
api.cloudflare.com: cloudflare
api.deepinfra.com: deepinfra
api.deepseek.com: deepseek
api.mistral.ai: mistral
api.perplexity.ai: perplexity
api.voyageai.com: voyageai
api.x.ai: xai
api.jina.ai: jina
r.jina.ai: jina
qianfan.baidubce.com: ernie
api.hunyuan.cloud.tencent.com: hunyuan
api.minimax.chat: minimax
api.moonshot.cn: moonshot
dashscope.aliyuncs.com: qianwen
open.bigmodel.cn: zhipuai
serviceAuth:
openai:
headerName: Authorization
valueFormat: 'Bearer %s'
anthropic:
headerName: x-api-key
valueFormat: '%s'
gemini:
headerName: x-goog-api-key
valueFormat: '%s'
cohere:
headerName: Authorization
valueFormat: 'Bearer %s'
groq:
headerName: Authorization
valueFormat: 'Bearer %s'
openrouter:
headerName: Authorization
valueFormat: 'Bearer %s'
ai21:
headerName: Authorization
valueFormat: 'Bearer %s'
cloudflare:
headerName: Authorization
valueFormat: 'Bearer %s'
deepinfra:
headerName: Authorization
valueFormat: 'Bearer %s'
deepseek:
headerName: Authorization
valueFormat: 'Bearer %s'
mistral:
headerName: Authorization
valueFormat: 'Bearer %s'
perplexity:
headerName: Authorization
valueFormat: 'Bearer %s'
voyageai:
headerName: Authorization
valueFormat: 'Bearer %s'
xai:
headerName: Authorization
valueFormat: 'Bearer %s'
jina:
headerName: Authorization
valueFormat: 'Bearer %s'
ernie:
headerName: Authorization
valueFormat: 'Bearer %s'
hunyuan:
headerName: Authorization
valueFormat: 'Bearer %s'
minimax:
headerName: Authorization
valueFormat: 'Bearer %s'
moonshot:
headerName: Authorization
valueFormat: 'Bearer %s'
qianwen:
headerName: Authorization
valueFormat: 'Bearer %s'
zhipuai:
headerName: Authorization
valueFormat: 'Bearer %s'
allowedDomains:
# Coyote release + self-update + model-registry sync # Coyote release + self-update + model-registry sync
- 'github.com:443' - 'github.com'
- 'api.github.com:443' - 'api.github.com'
- 'raw.githubusercontent.com:443' - 'raw.githubusercontent.com'
- 'objects.githubusercontent.com:443' - 'objects.githubusercontent.com'
- '*.githubusercontent.com:443' - '*.githubusercontent.com'
# Package managers and developer tools (cargo, uv, pip — useful at runtime for user installs) # Package managers and developer tools (cargo, uv, pip — useful at runtime for user installs)
- 'crates.io:443' - 'crates.io'
- 'static.crates.io:443' - 'static.crates.io'
- 'pypi.org:443' - 'pypi.org'
- 'files.pythonhosted.org:443' - 'files.pythonhosted.org'
- 'astral.sh:443' - 'astral.sh'
- 'sh.rustup.rs:443' - 'sh.rustup.rs'
- 'static.rust-lang.org:443' - 'static.rust-lang.org'
# LLM model OAuth + API endpoints # LLM model OAuth + API endpoints
- 'claude.ai:443' - 'claude.ai'
- 'console.anthropic.com:443' - 'console.anthropic.com'
- 'accounts.google.com:443' - 'accounts.google.com'
# *.googleapis.com covers oauth2 + userinfo + VertexAI regional endpoints # *.googleapis.com covers oauth2 + userinfo + VertexAI regional endpoints
# (*-aiplatform.googleapis.com). Do not narrow without re-checking VertexAI. # (*-aiplatform.googleapis.com). Do not narrow without re-checking VertexAI.
- '*.googleapis.com:443' - '*.googleapis.com'
# Bedrock and GitHub Models use signed / GitHub-PAT auth that the proxy # Bedrock and GitHub Models use signed / GitHub-PAT auth that the proxy
# cannot rewrite. Domains are allow-listed; credentials must be injected # cannot rewrite; credentials must be injected separately (see README
# separately (see README "Extending"). # "Extending"). NOTE: '*.amazonaws.com' matches exactly ONE label, so
- '*.amazonaws.com:443' # two-label regional Bedrock hosts must be enumerated explicitly
- 'models.inference.ai.azure.com:443' # ('**.' is declared but not yet enforced by sbx). Add your region
# via a mixin if it's missing below.
- '*.amazonaws.com'
- 'bedrock-runtime.us-east-1.amazonaws.com'
- 'bedrock-runtime.us-east-2.amazonaws.com'
- 'bedrock-runtime.us-west-2.amazonaws.com'
- 'bedrock-runtime.eu-west-1.amazonaws.com'
- 'bedrock-runtime.eu-central-1.amazonaws.com'
- 'bedrock-runtime.ap-southeast-2.amazonaws.com'
- 'bedrock-runtime.ap-northeast-1.amazonaws.com'
- 'models.inference.ai.azure.com'
# Proxy-managed LLM provider APIs. Every credentials[].apiKey.inject
# domain below MUST also appear here. sbx does not derive allow entries
# from inject rules.
- 'api.openai.com'
- 'api.anthropic.com'
- 'generativelanguage.googleapis.com'
- 'api.cohere.ai'
- 'api.groq.com'
- 'openrouter.ai'
- 'api.ai21.com'
- 'api.cloudflare.com'
- 'api.deepinfra.com'
- 'api.deepseek.com'
- 'api.mistral.ai'
- 'api.perplexity.ai'
- 'api.voyageai.com'
- 'api.x.ai'
- 'api.jina.ai'
- 'r.jina.ai'
- 'qianfan.baidubce.com'
- 'api.hunyuan.cloud.tencent.com'
- 'api.minimax.chat'
- 'api.moonshot.cn'
- 'dashscope.aliyuncs.com'
- 'open.bigmodel.cn'
# Proxy-managed LLM providers: inside the sandbox each apiKey env var holds
# the `proxy-managed` sentinel; the proxy injects the real value into the
# request header per the inject rules at request time. Values are bound by
# the user via credential bindings (`sbx secret set <service>`); Coyote
# pre-seeds them from its vault at launch. Multiple domains may map to one
# service (e.g. jina) so they share a single credential.
credentials: credentials:
sources: - service: openai
openai: description: OpenAI API key, injected on api.openai.com
env: apiKey:
- OPENAI_API_KEY name: OPENAI_API_KEY
anthropic: proxyManaged: true
env: inject:
- ANTHROPIC_API_KEY - domain: api.openai.com
gemini: scheme: bearer
env: - service: anthropic
- GEMINI_API_KEY description: Anthropic API key, injected as x-api-key on api.anthropic.com
- GOOGLE_API_KEY apiKey:
cohere: name: ANTHROPIC_API_KEY
env: proxyManaged: true
- COHERE_API_KEY inject:
groq: - domain: api.anthropic.com
env: header: x-api-key
- GROQ_API_KEY format: '%s'
openrouter: - service: gemini
env: description: Google Gemini API key, injected as x-goog-api-key on generativelanguage.googleapis.com
- OPENROUTER_API_KEY apiKey:
ai21: name: GEMINI_API_KEY
env: proxyManaged: true
- AI21_API_KEY inject:
cloudflare: - domain: generativelanguage.googleapis.com
env: header: x-goog-api-key
- CLOUDFLARE_API_KEY format: '%s'
deepinfra: - service: cohere
env: description: Cohere API key, injected on api.cohere.ai
- DEEPINFRA_API_KEY apiKey:
deepseek: name: COHERE_API_KEY
env: proxyManaged: true
- DEEPSEEK_API_KEY inject:
mistral: - domain: api.cohere.ai
env: scheme: bearer
- MISTRAL_API_KEY - service: groq
perplexity: description: Groq API key, injected on api.groq.com
env: apiKey:
- PERPLEXITY_API_KEY name: GROQ_API_KEY
voyageai: proxyManaged: true
env: inject:
- VOYAGE_API_KEY - domain: api.groq.com
xai: scheme: bearer
env: - service: openrouter
- XAI_API_KEY description: OpenRouter API key, injected on openrouter.ai
jina: apiKey:
env: name: OPENROUTER_API_KEY
- JINA_API_KEY proxyManaged: true
ernie: inject:
env: - domain: openrouter.ai
- ERNIE_API_KEY scheme: bearer
hunyuan: - service: ai21
env: description: AI21 Labs API key, injected on api.ai21.com
- HUNYUAN_API_KEY apiKey:
minimax: name: AI21_API_KEY
env: proxyManaged: true
- MINIMAX_API_KEY inject:
moonshot: - domain: api.ai21.com
env: scheme: bearer
- MOONSHOT_API_KEY - service: cloudflare
qianwen: description: Cloudflare Workers AI API key, injected on api.cloudflare.com
env: apiKey:
- DASHSCOPE_API_KEY name: CLOUDFLARE_API_KEY
zhipuai: proxyManaged: true
env: inject:
- ZHIPUAI_API_KEY - domain: api.cloudflare.com
scheme: bearer
- service: deepinfra
description: DeepInfra API key, injected on api.deepinfra.com
apiKey:
name: DEEPINFRA_API_KEY
proxyManaged: true
inject:
- domain: api.deepinfra.com
scheme: bearer
- service: deepseek
description: DeepSeek API key, injected on api.deepseek.com
apiKey:
name: DEEPSEEK_API_KEY
proxyManaged: true
inject:
- domain: api.deepseek.com
scheme: bearer
- service: mistral
description: Mistral API key, injected on api.mistral.ai
apiKey:
name: MISTRAL_API_KEY
proxyManaged: true
inject:
- domain: api.mistral.ai
scheme: bearer
- service: perplexity
description: Perplexity API key, injected on api.perplexity.ai
apiKey:
name: PERPLEXITY_API_KEY
proxyManaged: true
inject:
- domain: api.perplexity.ai
scheme: bearer
- service: voyageai
description: Voyage AI API key, injected on api.voyageai.com
apiKey:
name: VOYAGE_API_KEY
proxyManaged: true
inject:
- domain: api.voyageai.com
scheme: bearer
- service: xai
description: xAI (Grok) API key, injected on api.x.ai
apiKey:
name: XAI_API_KEY
proxyManaged: true
inject:
- domain: api.x.ai
scheme: bearer
- service: jina
description: Jina API key, injected on api.jina.ai and r.jina.ai
apiKey:
name: JINA_API_KEY
proxyManaged: true
inject:
- domain: api.jina.ai
scheme: bearer
- domain: r.jina.ai
scheme: bearer
- service: ernie
description: Baidu ERNIE API key, injected on qianfan.baidubce.com
apiKey:
name: ERNIE_API_KEY
proxyManaged: true
inject:
- domain: qianfan.baidubce.com
scheme: bearer
- service: hunyuan
description: Tencent Hunyuan API key, injected on api.hunyuan.cloud.tencent.com
apiKey:
name: HUNYUAN_API_KEY
proxyManaged: true
inject:
- domain: api.hunyuan.cloud.tencent.com
scheme: bearer
- service: minimax
description: MiniMax API key, injected on api.minimax.chat
apiKey:
name: MINIMAX_API_KEY
proxyManaged: true
inject:
- domain: api.minimax.chat
scheme: bearer
- service: moonshot
description: Moonshot AI API key, injected on api.moonshot.cn
apiKey:
name: MOONSHOT_API_KEY
proxyManaged: true
inject:
- domain: api.moonshot.cn
scheme: bearer
- service: qianwen
description: Alibaba Qianwen (DashScope) API key, injected on dashscope.aliyuncs.com
apiKey:
name: DASHSCOPE_API_KEY
proxyManaged: true
inject:
- domain: dashscope.aliyuncs.com
scheme: bearer
- service: zhipuai
description: Zhipu AI (GLM) API key, injected on open.bigmodel.cn
apiKey:
name: ZHIPUAI_API_KEY
proxyManaged: true
inject:
- domain: open.bigmodel.cn
scheme: bearer
environment: environment:
variables: variables:
@@ -213,32 +270,14 @@ environment:
COYOTE_LOG_LEVEL: INFO COYOTE_LOG_LEVEL: INFO
COYOTE_CONFIG_DIR: /home/agent/.config/coyote COYOTE_CONFIG_DIR: /home/agent/.config/coyote
EDITOR: nano EDITOR: nano
proxyManaged: # Alias for the gemini credential: v2 apiKey supports a single env name
- OPENAI_API_KEY # (GEMINI_API_KEY above). Coyote also recognizes GOOGLE_API_KEY, so keep
- ANTHROPIC_API_KEY # it set to the sentinel. Header injection happens per-domain regardless
- GEMINI_API_KEY # of which env var the app reads.
- GOOGLE_API_KEY GOOGLE_API_KEY: proxy-managed
- COHERE_API_KEY
- GROQ_API_KEY
- OPENROUTER_API_KEY
- AI21_API_KEY
- CLOUDFLARE_API_KEY
- DEEPINFRA_API_KEY
- DEEPSEEK_API_KEY
- MISTRAL_API_KEY
- PERPLEXITY_API_KEY
- VOYAGE_API_KEY
- XAI_API_KEY
- JINA_API_KEY
- ERNIE_API_KEY
- HUNYUAN_API_KEY
- MINIMAX_API_KEY
- MOONSHOT_API_KEY
- DASHSCOPE_API_KEY
- ZHIPUAI_API_KEY
commands: setup:
initFiles: files:
- path: /home/agent/.config/git/ssh-signing-key-command - path: /home/agent/.config/git/ssh-signing-key-command
mode: '0755' mode: '0755'
description: Resolve the forwarded SSH agent key for Git SSH signing description: Resolve the forwarded SSH agent key for Git SSH signing
@@ -290,7 +329,9 @@ commands:
background: false background: false
description: Bootstrap Coyote config directory on first sandbox start description: Bootstrap Coyote config directory on first sandbox start
agentContext: | agentInstructions:
filename: COYOTE.md
content: |
## Sandbox environment ## Sandbox environment
You are running inside a Docker sandbox launched via `sbx run coyote`. The You are running inside a Docker sandbox launched via `sbx run coyote`. The
@@ -302,25 +343,29 @@ agentContext: |
`~/.cache/coyote/coyote.log`. Persistence is enabled, so config, sessions, `~/.cache/coyote/coyote.log`. Persistence is enabled, so config, sessions,
vault state, OAuth tokens, and installed tools survive sandbox restarts. vault state, OAuth tokens, and installed tools survive sandbox restarts.
LLM provider credentials are forwarded by the sandbox HTTP proxy. The LLM provider credentials are forwarded by the sandbox HTTP proxy via
following provider env vars are recognized - export the ones you use on credential bindings. Coyote pre-seeds them from its vault at launch
the host before running `sbx run coyote`: (`sbx secret set <service>`); users can also bind values manually on the
host with `sbx secret set <service>` or `sbx secret import`. Recognized
services:
OPENAI_API_KEY, ANTHROPIC_API_KEY, GEMINI_API_KEY / GOOGLE_API_KEY, openai, anthropic, gemini, cohere, groq, openrouter, ai21,
COHERE_API_KEY, GROQ_API_KEY, OPENROUTER_API_KEY, AI21_API_KEY, cloudflare, deepinfra, deepseek, mistral, perplexity, voyageai,
CLOUDFLARE_API_KEY, DEEPINFRA_API_KEY, DEEPSEEK_API_KEY, xai, jina, ernie, hunyuan, minimax, moonshot, qianwen, zhipuai
MISTRAL_API_KEY, PERPLEXITY_API_KEY, VOYAGE_API_KEY, XAI_API_KEY,
JINA_API_KEY, ERNIE_API_KEY, HUNYUAN_API_KEY, MINIMAX_API_KEY,
MOONSHOT_API_KEY, DASHSCOPE_API_KEY (Qwen), ZHIPUAI_API_KEY
Inside the sandbox these appear as the placeholder string `proxy-managed`; Inside the sandbox the corresponding env vars (OPENAI_API_KEY, etc.)
the proxy substitutes the real value at request time. OAuth flows for hold the placeholder string `proxy-managed`; the proxy substitutes the
Claude Pro/Max and Gemini are also allow-listed. real value at request time. OAuth flows for Claude Pro/Max and Gemini
are also allow-listed.
Bedrock (AWS) and VertexAI (Google Cloud) use signed/OAuth-token requests Bedrock (AWS) and VertexAI (Google Cloud) use signed/OAuth-token requests
that the proxy cannot rewrite. Their domains are allow-listed but you must that the proxy cannot rewrite, so you must inject credentials yourself via
inject credentials yourself via `sbx run --env AWS_ACCESS_KEY_ID=...` or `sbx run --env AWS_ACCESS_KEY_ID=...` or a mixin kit that mounts a
a mixin kit that mounts a service-account JSON. service-account JSON. VertexAI regional endpoints are allow-listed via
`*.googleapis.com`. Bedrock runtime endpoints are allow-listed for
us-east-1/2, us-west-2, eu-west-1, eu-central-1, ap-southeast-2, and
ap-northeast-1 only; other regions need a mixin allow entry
(`bedrock-runtime.<region>.amazonaws.com`).
Useful first-run commands: Useful first-run commands:
- `coyote --info` # show config paths and resolved settings - `coyote --info` # show config paths and resolved settings
src/config/agent.rs
+72 -15
View File
@@ -12,6 +12,7 @@ use crate::config::prompts::{
DEFAULT_SPAWN_INSTRUCTIONS, DEFAULT_TEAMMATE_INSTRUCTIONS, DEFAULT_TODO_INSTRUCTIONS, DEFAULT_SPAWN_INSTRUCTIONS, DEFAULT_TEAMMATE_INSTRUCTIONS, DEFAULT_TODO_INSTRUCTIONS,
DEFAULT_USER_INTERACTION_INSTRUCTIONS, DEFAULT_USER_INTERACTION_INSTRUCTIONS,
}; };
use crate::graph::types::RagNode;
use crate::graph::{Graph, GraphParser, NodeType}; use crate::graph::{Graph, GraphParser, NodeType};
use crate::rag::RagInitConfig; use crate::rag::RagInitConfig;
use crate::vault::SECRET_RE; use crate::vault::SECRET_RE;
@@ -952,6 +953,30 @@ fn resolve_document_paths(
Ok(document_paths) Ok(document_paths)
} }
/// How a graph rag node describes the knowledge base it wants built.
///
/// `driver` is forwarded as-is: `None` means the node did not ask for one, which
/// `RagInitConfig` resolves to yaml, so workflows written before drivers existed
/// keep their current storage.
///
/// Every field is now named explicitly, so adding one to `RagInitConfig` breaks
/// this literal. That is deliberate: the new field then gets a decision about
/// whether a rag node can drive it, instead of silently taking its default.
fn rag_init_config(rag_node: &RagNode) -> RagInitConfig {
RagInitConfig {
embedding_model: rag_node.embedding_model.clone(),
chunk_size: rag_node.chunk_size,
chunk_overlap: rag_node.chunk_overlap,
reranker_model: rag_node.reranker_model.clone(),
top_k: rag_node.top_k,
batch_size: rag_node.batch_size,
extractor_model: rag_node.extractor_model.clone(),
extractor_prompt: rag_node.extractor_prompt.clone(),
graph_hops: rag_node.graph_hops,
driver: rag_node.driver.clone(),
}
}
#[allow(clippy::too_many_arguments)] #[allow(clippy::too_many_arguments)]
async fn init_graph_rags( async fn init_graph_rags(
app: &AppConfig, app: &AppConfig,
@@ -989,21 +1014,18 @@ async fn init_graph_rags(
}) })
.await? .await?
} else { } else {
let config = RagInitConfig { // Checked before anything is built: an unknown driver would otherwise
embedding_model: rag_node.embedding_model.clone(), // fall through `Rag::create`'s catch-all to a yaml store, embed every
chunk_size: rag_node.chunk_size, // document, and persist the bogus driver string. The RAG would then be
chunk_overlap: rag_node.chunk_overlap, // rejected on every subsequent load, leaving the agent unstartable.
reranker_model: rag_node.reranker_model.clone(), // Graph validation catches this too, but it is skipped when
top_k: rag_node.top_k, // `validate_before_run` is off, so this guard is the load-bearing one.
batch_size: rag_node.batch_size, if let Some(driver) = &rag_node.driver
extractor_model: rag_node.extractor_model.clone(), && let Some(message) = crate::graph::validator::rag_driver_error(driver)
extractor_prompt: rag_node.extractor_prompt.clone(), {
graph_hops: rag_node.graph_hops, bail!("rag node '{node_id}': {message}");
// Graph-node RAGs are yaml-only: `RagNode` has no `driver` field, so }
// there is nothing to forward. The rest-pattern also keeps this literal let config = rag_init_config(rag_node);
// from breaking on future `RagInitConfig` additions.
..Default::default()
};
let fully_specified = config.embedding_model.is_some() let fully_specified = config.embedding_model.is_some()
&& config.chunk_size.is_some() && config.chunk_size.is_some()
&& config.chunk_overlap.is_some(); && config.chunk_overlap.is_some();
@@ -1337,4 +1359,39 @@ version: "1.0"
assert_eq!(meta.description, ""); assert_eq!(meta.description, "");
} }
#[test]
fn rag_init_config_forwards_an_explicit_driver() {
let node: RagNode =
serde_yaml::from_str("documents: [\"./docs\"]\ndriver: duckdb\n").unwrap();
assert_eq!(rag_init_config(&node).driver.as_deref(), Some("duckdb"));
}
/// A node that names no driver must forward `None`, which `RagInitConfig`
/// documents as "yaml". Existing workflows therefore keep their yaml store.
#[test]
fn rag_init_config_leaves_the_driver_unset_by_default() {
let node: RagNode = serde_yaml::from_str("documents: [\"./docs\"]\n").unwrap();
assert_eq!(rag_init_config(&node).driver, None);
}
/// The driver must ride alongside the rest of the node's settings, not
/// replace them.
#[test]
fn rag_init_config_forwards_the_other_settings_too() {
let node: RagNode = serde_yaml::from_str(
"documents: [\"./docs\"]\ndriver: duckdb\nchunk_size: 512\nchunk_overlap: 64\ntop_k: 7\nembedding_model: some:model\n",
)
.unwrap();
let config = rag_init_config(&node);
assert_eq!(config.driver.as_deref(), Some("duckdb"));
assert_eq!(config.chunk_size, Some(512));
assert_eq!(config.chunk_overlap, Some(64));
assert_eq!(config.top_k, Some(7));
assert_eq!(config.embedding_model.as_deref(), Some("some:model"));
}
} }
src/config/paths.rs
+2 -23
View File
@@ -414,11 +414,10 @@ pub fn list_rags() -> Vec<String> {
for entry in rd.flatten() { for entry in rd.flatten() {
let name = entry.file_name(); let name = entry.file_name();
if let Some(name) = name.to_string_lossy().strip_suffix(".yaml") { if let Some(name) = name.to_string_lossy().strip_suffix(".yaml") {
// Sidecars are not RAGs. `.duckdb` files are already excluded by
// the `.yaml` suffix check above; this rejects `<name>.sbx-mixin`.
if is_rag_sidecar_name(name) { if is_rag_sidecar_name(name) {
continue; continue;
} }
names.push(name.to_string()); names.push(name.to_string());
} }
} }
@@ -429,24 +428,10 @@ pub fn list_rags() -> Vec<String> {
} }
} }
/// True for the sidecar YAML files that must never be listed or deleted as RAGs.
/// `name` is the already-stripped stem (i.e. after `strip_suffix(".yaml")`).
/// Uses `ends_with`, not `contains('.')`, so a RAG legitimately named "v2.docs" is
/// not rejected.
pub(crate) fn is_rag_sidecar_name(name: &str) -> bool { pub(crate) fn is_rag_sidecar_name(name: &str) -> bool {
name.ends_with(".sbx-mixin") name.ends_with(".sbx-mixin")
} }
/// Remove every sidecar belonging to RAG `name` in `dir`. Missing files are NOT an
/// error. A failure to remove an EXISTING mixin IS an error and must propagate — a
/// silently-orphaned mixin keeps a sandbox network permission alive after the user
/// believes it is gone. The `.duckdb` orphan is only wasted disk, so its removal
/// failure is ignorable; the asymmetry is deliberate.
///
/// Callers must run this BEFORE unlinking the primary `.yaml`. If the YAML goes first
/// and this then fails, the RAG disappears from `list_rags()` — so the user can no
/// longer select it to retry — while its `allowedDomains` entry keeps being injected
/// into every sandbox launch.
pub(crate) fn remove_rag_sidecars(dir: &Path, name: &str) -> Result<()> { pub(crate) fn remove_rag_sidecars(dir: &Path, name: &str) -> Result<()> {
let duckdb_path = dir.join(format!("{name}.duckdb")); let duckdb_path = dir.join(format!("{name}.duckdb"));
if duckdb_path.exists() { if duckdb_path.exists() {
@@ -463,6 +448,7 @@ pub(crate) fn remove_rag_sidecars(dir: &Path, name: &str) -> Result<()> {
) )
})?; })?;
} }
Ok(()) Ok(())
} }
@@ -889,8 +875,6 @@ mod tests {
let _ = fs::remove_dir_all(&root); let _ = fs::remove_dir_all(&root);
} }
/// Unique temp dir for the sidecar helper tests. These take `dir: &Path` directly,
/// so no env-var mutation and therefore no `#[serial]` is needed.
fn sidecar_temp_dir(label: &str) -> PathBuf { fn sidecar_temp_dir(label: &str) -> PathBuf {
let unique = time::SystemTime::now() let unique = time::SystemTime::now()
.duration_since(time::UNIX_EPOCH) .duration_since(time::UNIX_EPOCH)
@@ -903,7 +887,6 @@ mod tests {
#[test] #[test]
fn is_rag_sidecar_name_accepts_dotted_rag_names() { fn is_rag_sidecar_name_accepts_dotted_rag_names() {
// A RAG legitimately named "v2.docs" must not be mistaken for a sidecar.
assert!(!is_rag_sidecar_name("v2.docs")); assert!(!is_rag_sidecar_name("v2.docs"));
assert!(!is_rag_sidecar_name("myrag")); assert!(!is_rag_sidecar_name("myrag"));
assert!(is_rag_sidecar_name("myrag.sbx-mixin")); assert!(is_rag_sidecar_name("myrag.sbx-mixin"));
@@ -940,8 +923,6 @@ mod tests {
let root = sidecar_temp_dir("rag-sidecars-order"); let root = sidecar_temp_dir("rag-sidecars-order");
let yaml = root.join("docs.yaml"); let yaml = root.join("docs.yaml");
fs::write(&yaml, "rag").unwrap(); fs::write(&yaml, "rag").unwrap();
// A non-empty DIRECTORY at the mixin path makes remove_file fail, standing in
// for any real removal failure (permissions, a busy mount).
let mixin = root.join("docs.sbx-mixin.yaml"); let mixin = root.join("docs.sbx-mixin.yaml");
fs::create_dir_all(&mixin).unwrap(); fs::create_dir_all(&mixin).unwrap();
fs::write(mixin.join("blocker"), "x").unwrap(); fs::write(mixin.join("blocker"), "x").unwrap();
@@ -952,8 +933,6 @@ mod tests {
.contains("Failed to remove the sandbox mixin"), .contains("Failed to remove the sandbox mixin"),
"got: {err}" "got: {err}"
); );
// The whole point of removing sidecars first: the RAG is still on disk, still
// listed, and the deletion is retryable.
assert!( assert!(
yaml.exists(), yaml.exists(),
"the .yaml must survive a sidecar-removal failure so the delete is retryable" "the .yaml must survive a sidecar-removal failure so the delete is retryable"
src/config/request_context.rs
+2 -24
View File
@@ -142,11 +142,6 @@ pub struct RequestContext {
pub role: Option<Role>, pub role: Option<Role>,
pub session: Option<Session>, pub session: Option<Session>,
pub rag: Option<Arc<Rag>>, pub rag: Option<Arc<Rag>>,
/// The cache key `self.rag` was actually inserted under, carried rather than
/// reconstructed. Reconstruction was the bug: the invalidation sites do not have
/// the information needed to rebuild the key (agent RAGs are inserted under the
/// AGENT's name but `rag.name()` is the constant "rag"), so insert and invalidate
/// silently disagreed. `None` for the temp RAG, which bypasses the cache entirely.
pub rag_key: Option<RagKey>, pub rag_key: Option<RagKey>,
pub agent: Option<Agent>, pub agent: Option<Agent>,
@@ -4122,14 +4117,10 @@ impl RequestContext {
} }
let app = self.app.config.clone(); let app = self.app.config.clone();
// Hoisted: `rag_cache` below borrows `self`, so the loader closure cannot
// reach through `self` for the vault. `GlobalVault` is an Arc, so this is cheap.
let vault = self.app.vault.clone(); let vault = self.app.vault.clone();
let rag_cache = self.rag_cache(); let rag_cache = self.rag_cache();
let working_mode = self.working_mode; let working_mode = self.working_mode;
// The key is returned alongside the Rag rather than assigned inside the match:
// `rag_cache` borrows `self`, so writing `self.rag_key` there is E0506.
let (rag, rag_key): (Arc<Rag>, Option<RagKey>) = match rag { let (rag, rag_key): (Arc<Rag>, Option<RagKey>) = match rag {
None => { None => {
let rag_path = self.rag_file(super::TEMP_RAG_NAME); let rag_path = self.rag_file(super::TEMP_RAG_NAME);
@@ -4138,7 +4129,6 @@ impl RequestContext {
format!("Failed to cleanup previous '{}' rag", super::TEMP_RAG_NAME) format!("Failed to cleanup previous '{}' rag", super::TEMP_RAG_NAME)
})?; })?;
} }
// The temp RAG is never inserted into the cache, so it has no key.
( (
Arc::new( Arc::new(
Rag::init( Rag::init(
@@ -4198,13 +4188,9 @@ impl RequestContext {
let vault = self.app.vault.clone(); let vault = self.app.vault.clone();
let rag = Rag::attach(app, &vault, name, &rag_path).await?; let rag = Rag::attach(app, &vault, name, &rag_path).await?;
let rag = Arc::new(rag); let rag = Arc::new(rag);
// Populate the cache so a later `.rag <name>` reuses this instance rather
// than re-running the network preflight. Attach is always a global RAG.
let key = RagKey::Named(name.to_string()); let key = RagKey::Named(name.to_string());
self.rag_cache().insert(key.clone(), &rag); self.rag_cache().insert(key.clone(), &rag);
self.rag = Some(rag); self.rag = Some(rag);
// Carried so invalidation in rebuild_rag()/edit_rag_docs() can find this
// entry; without it a stale Arc would linger in the cache all session.
self.rag_key = Some(key); self.rag_key = Some(key);
Ok(()) Ok(())
} }
@@ -4217,7 +4203,7 @@ impl RequestContext {
if rag.is_attached() { if rag.is_attached() {
bail!( bail!(
"Cannot edit documents on an attached RAG — Coyote does not own its source documents." "Cannot edit documents on an attached RAG; Coyote does not own its source documents."
); );
} }
@@ -4270,7 +4256,7 @@ impl RequestContext {
if rag.is_attached() { if rag.is_attached() {
bail!( bail!(
"Cannot rebuild an attached RAG — Coyote does not own its source documents. \ "Cannot rebuild an attached RAG; Coyote does not own its source documents. \
Re-index from the system that originally created '{}'.", Re-index from the system that originally created '{}'.",
rag.name() rag.name()
); );
@@ -4716,8 +4702,6 @@ mod tests {
) )
.unwrap(); .unwrap();
// Stand in for the state `.rag docs` leaves behind: `use_rag` sets `rag` and
// `rag_key` together, so a named key is live when the agent is entered.
ctx.rag_key = Some(RagKey::Named("docs".to_string())); ctx.rag_key = Some(RagKey::Named("docs".to_string()));
tokio::runtime::Builder::new_current_thread() tokio::runtime::Builder::new_current_thread()
@@ -4730,9 +4714,6 @@ mod tests {
.unwrap(); .unwrap();
}); });
// This agent has no RAG, so `rag` is None and `rag_key` must be None as well.
// Carrying `Named("docs")` across the transition would point `.rebuild rag`
// at an unrelated RAG's cache entry.
assert!(ctx.rag.is_none()); assert!(ctx.rag.is_none());
assert_eq!(ctx.rag_key, None); assert_eq!(ctx.rag_key, None);
} }
@@ -6122,9 +6103,6 @@ mod tests {
assert!(paths::list_rags().is_empty()); assert!(paths::list_rags().is_empty());
} }
/// A `<name>.sbx-mixin.yaml` sidecar must not appear as a phantom RAG in TAB
/// completion or `.list rag`. A RAG whose name legitimately contains a dot must
/// still be listed — the filter uses `ends_with`, not `contains('.')`.
#[test] #[test]
#[serial] #[serial]
fn list_rags_skips_sbx_mixin_sidecars() { fn list_rags_skips_sbx_mixin_sidecars() {
src/graph/types.rs
+103
View File
@@ -367,6 +367,13 @@ pub struct RagNode {
#[serde(default, skip_serializing_if = "Option::is_none")] #[serde(default, skip_serializing_if = "Option::is_none")]
pub graph_hops: Option<usize>, pub graph_hops: Option<usize>,
/// Storage driver for this node's knowledge base ("yaml", "duckdb"). `None`
/// means "yaml". Only honored when the knowledge base is first built;
/// changing it afterwards has no effect until the RAG is deleted and
/// re-initialized.
#[serde(default, skip_serializing_if = "Option::is_none")]
pub driver: Option<String>,
#[serde(default, skip_serializing_if = "Option::is_none")] #[serde(default, skip_serializing_if = "Option::is_none")]
pub state_updates: Option<HashMap<String, String>>, pub state_updates: Option<HashMap<String, String>>,
@@ -1152,4 +1159,100 @@ nodes:
assert!(triage.next.as_ref().unwrap().is_fan_out()); assert!(triage.next.as_ref().unwrap().is_fan_out());
assert_eq!(triage.next.as_ref().unwrap().as_slice().len(), 2); assert_eq!(triage.next.as_ref().unwrap().as_slice().len(), 2);
} }
fn rag_node_of(graph: &Graph, id: &str) -> RagNode {
match &graph.get_node(id).unwrap().node_type {
NodeType::Rag(r) => r.clone(),
other => panic!("expected a rag node, got {other:?}"),
}
}
#[test]
fn rag_node_deserializes_an_explicit_driver() {
let yaml = r#"
name: kb
start: research
nodes:
research:
type: rag
documents: ["./docs"]
driver: duckdb
next: done
done:
type: end
output: ok
"#;
let graph: Graph = serde_yaml::from_str(yaml).unwrap();
assert_eq!(
rag_node_of(&graph, "research").driver.as_deref(),
Some("duckdb")
);
}
/// Workflows written before drivers existed must keep parsing, and must keep
/// asking for nothing, so `RagInitConfig` resolves them to the yaml default.
#[test]
fn rag_node_without_a_driver_stays_unset() {
let yaml = r#"
name: kb
start: research
nodes:
research:
type: rag
documents: ["./docs"]
next: done
done:
type: end
output: ok
"#;
let graph: Graph = serde_yaml::from_str(yaml).unwrap();
assert_eq!(rag_node_of(&graph, "research").driver, None);
}
#[test]
fn rag_node_driver_survives_a_serialize_round_trip() {
let yaml = r#"
name: kb
start: research
nodes:
research:
type: rag
documents: ["./docs"]
driver: duckdb
next: done
done:
type: end
output: ok
"#;
let graph: Graph = serde_yaml::from_str(yaml).unwrap();
let reparsed: Graph =
serde_yaml::from_str(&serde_yaml::to_string(&graph).unwrap()).unwrap();
assert_eq!(
rag_node_of(&reparsed, "research").driver.as_deref(),
Some("duckdb")
);
}
/// `skip_serializing_if` must keep `driver:` out of graphs that never set it.
#[test]
fn rag_node_without_a_driver_omits_the_key_when_serialized() {
let yaml = r#"
name: kb
start: research
nodes:
research:
type: rag
documents: ["./docs"]
next: done
done:
type: end
output: ok
"#;
let graph: Graph = serde_yaml::from_str(yaml).unwrap();
assert!(!serde_yaml::to_string(&graph).unwrap().contains("driver"));
}
} }
src/graph/validator.rs
+101
View File
@@ -2,6 +2,7 @@ use super::state::template_root_keys;
use super::types::{Graph, Node, NodeType}; use super::types::{Graph, Node, NodeType};
use crate::client::{Model, ModelType}; use crate::client::{Model, ModelType};
use crate::config::{Agent, AppConfig, paths}; use crate::config::{Agent, AppConfig, paths};
use crate::rag::{GraphRagConfig, RagData};
use anyhow::{Result, bail}; use anyhow::{Result, bail};
use std::collections::{BTreeMap, HashSet, VecDeque}; use std::collections::{BTreeMap, HashSet, VecDeque};
use std::path::PathBuf; use std::path::PathBuf;
@@ -96,6 +97,51 @@ pub struct GraphValidator {
skill_exists: fn(&str) -> bool, skill_exists: fn(&str) -> bool,
} }
/// A minimal `RagData` whose only interesting field is `driver`. The numeric
/// arguments are the smallest values that satisfy `validate()`'s unrelated
/// floors (top_k >= 1, and chunk_size >= 1 with chunk_overlap < chunk_size for
/// a non-attached RAG). `RagData::new` sets `attached: false`, which is the
/// correct shape here: a graph rag node always builds its own local knowledge
/// base from `documents` and can never be attached.
fn rag_driver_probe(driver: &str) -> RagData {
let mut data = RagData::new(
String::new(),
1,
0,
None,
1,
None,
GraphRagConfig::default(),
);
data.driver = driver.to_string();
data
}
/// `Some(message)` when `driver` is one that `RagData::validate()` would reject.
///
/// The set of valid drivers is defined in exactly one place, `RagData::validate()`,
/// so this asks that function rather than restating the list here.
///
/// Fails open on purpose: the first probe below uses the default driver, which is
/// valid by definition. If even that one is rejected, `validate()` has grown a
/// precondition the probe fixture no longer satisfies, and every verdict from here
/// would be a false positive that rejects working graphs. In that case we decline
/// to judge and leave enforcement to RAG construction. The
/// `rag_driver_probe_fixture_is_accepted` test turns that silent degradation into a
/// loud failure. Both `validate()` calls are load-bearing; neither is redundant.
pub(crate) fn rag_driver_error(driver: &str) -> Option<String> {
if rag_driver_probe(&RagData::default().driver)
.validate()
.is_err()
{
return None;
}
rag_driver_probe(driver)
.validate()
.err()
.map(|err| err.to_string())
}
impl GraphValidator { impl GraphValidator {
pub fn new(base_dir: impl Into<PathBuf>) -> Self { pub fn new(base_dir: impl Into<PathBuf>) -> Self {
Self { Self {
@@ -154,6 +200,11 @@ impl GraphValidator {
not be written to state", not be written to state",
)); ));
} }
if let Some(driver) = &r.driver
&& let Some(message) = rag_driver_error(driver)
{
result.error(ValidationError::with_node(node_id, message));
}
} }
} }
} }
@@ -1031,6 +1082,7 @@ mod tests {
extractor_model: None, extractor_model: None,
extractor_prompt: None, extractor_prompt: None,
graph_hops: None, graph_hops: None,
driver: None,
state_updates, state_updates,
timeout: None, timeout: None,
}), }),
@@ -1385,6 +1437,55 @@ mod tests {
); );
} }
/// Guards the fail-open branch in `rag_driver_error`. If this fails,
/// `RagData::validate()` grew a precondition the probe fixture no longer
/// satisfies and rag-node driver validation has silently switched itself off.
/// Repair the fixture in `rag_driver_probe`; do not delete this test.
#[test]
fn rag_driver_probe_fixture_is_accepted() {
let default_driver = RagData::default().driver;
assert!(
rag_driver_probe(&default_driver).validate().is_ok(),
"probe fixture rejected for the default driver '{default_driver}'"
);
}
#[test]
fn rag_driver_error_defers_to_ragdata_validate() {
assert_eq!(rag_driver_error("yaml"), None);
assert_eq!(rag_driver_error("duckdb"), None);
let message = rag_driver_error("duckdbb").expect("unknown driver must be rejected");
assert!(message.contains("duckdbb"), "got: {message}");
}
#[test]
fn rag_node_with_unknown_driver_errors_naming_the_node() {
let mut node = rag_node("kb", &["./docs"], true);
if let NodeType::Rag(ref mut r) = node.node_type {
r.driver = Some("postgres".into());
}
let graph = graph_with(vec![("kb", node), ("end", end_node("end"))], "kb");
let result = validator().validate(&graph);
assert!(!result.is_valid());
let err = result.into_result().unwrap_err().to_string();
assert!(err.contains("[kb]"), "must name the node: {err}");
assert!(err.contains("postgres"), "must name the driver: {err}");
}
#[test]
fn rag_node_with_duckdb_driver_produces_no_findings() {
let mut node = rag_node("kb", &["./docs"], true);
if let NodeType::Rag(ref mut r) = node.node_type {
r.driver = Some("duckdb".into());
}
let graph = graph_with(vec![("kb", node), ("end", end_node("end"))], "kb");
assert!(validator().validate(&graph).is_valid());
}
fn agent_node(id: &str, agent: &str, next: Option<&str>) -> Node { fn agent_node(id: &str, agent: &str, next: Option<&str>) -> Node {
Node { Node {
id: id.into(), id: id.into(),
src/rag/mod.rs
+552 -270
View File
File diff suppressed because it is too large Load Diff
src/rag/provider.rs
+7 -18
View File
@@ -4,11 +4,6 @@ use async_trait::async_trait;
/// Abstracts where RAG vector data is stored and queried. /// Abstracts where RAG vector data is stored and queried.
/// ///
/// Implementors:
/// - YamlProvider: HNSW in-memory, state derived from RagData.vectors/files
/// - DuckDbProvider: DuckDB on-disk vector index + document store
/// - QdrantProvider: remote Qdrant collection
///
/// The Rag orchestrator owns: embeddings, chunking, BM25 keyword search, graph RAG, /// The Rag orchestrator owns: embeddings, chunking, BM25 keyword search, graph RAG,
/// entity extraction, RRF merging. Providers own: vector storage and content retrieval. /// entity extraction, RRF merging. Providers own: vector storage and content retrieval.
#[async_trait] #[async_trait]
@@ -26,7 +21,7 @@ pub trait RagProvider: Send + Sync {
/// ///
/// **Ordering contract:** implementations MUST return results in the same /// **Ordering contract:** implementations MUST return results in the same
/// relative order as the input `ids` slice. `hybrid_search` passes an /// relative order as the input `ids` slice. `hybrid_search` passes an
/// RRF-ranked list and feeds the result straight to the LLM — a provider /// RRF-ranked list and feeds the result straight to the LLM. A provider
/// that returns rows in storage order (e.g. Qdrant `get_points`, DuckDB /// that returns rows in storage order (e.g. Qdrant `get_points`, DuckDB
/// `WHERE id IN (...)`) would silently discard the ranking. Implementations /// `WHERE id IN (...)`) would silently discard the ranking. Implementations
/// that query an unordered backend must re-sort by input position before /// that query an unordered backend must re-sort by input position before
@@ -43,34 +38,28 @@ pub trait RagProvider: Send + Sync {
/// Called once at the end of every sync_documents pass. /// Called once at the end of every sync_documents pass.
/// ///
/// `full_rebuild` mirrors `sync_documents`' `refresh` parameter: /// `full_rebuild` mirrors `sync_documents`' `refresh` parameter:
/// - `true` — a full re-index (`.rebuild rag`, `--rebuild-rag`, initial build). /// - `true`: a full re-index (`.rebuild rag`, `--rebuild-rag`, initial build).
/// Destructive strategies (wipe-then-reindex) are permitted. /// Destructive strategies (wipe-then-reindex) are permitted.
/// - `false` — an incremental change (`.edit rag-docs` adding/removing a file). /// - `false`: an incremental change (`.edit rag-docs` adding/removing a file).
/// Implementations MUST NOT wipe existing state; upsert only. /// Implementations MUST NOT wipe existing state; upsert only.
/// ///
/// The parameter is part of the signature from the outset so it is fixed /// The parameter is part of the signature from the outset so it is fixed
/// while there is exactly one implementor. Yaml/DuckDb ignore it — /// while there is exactly one implementor. Yaml/DuckDb ignore it,
/// rebuilding their local state wholesale is fast and always correct. /// rebuilding their local state wholesale is fast and always correct.
/// Only a remote provider is destructive enough to care. /// Only a remote provider is destructive enough to care.
///
/// YamlProvider: rebuilds HNSW + content map from data.vectors/files.
/// DuckDbProvider: writes new rows to DuckDB, deletes removed rows.
/// QdrantProvider: no-op while attach-only — remote data is unchanged.
async fn rebuild_indexes(&mut self, data: &RagData, full_rebuild: bool) -> Result<()>; async fn rebuild_indexes(&mut self, data: &RagData, full_rebuild: bool) -> Result<()>;
/// Keyword / full-text search. Returns (DocumentId, BM25-style score) sorted desc. /// Keyword / full-text search. Returns (DocumentId, BM25-style score) sorted desc.
/// ///
/// Default impl returns `Ok(vec![])` — callers fall back to `Rag.bm25` (local in-memory /// Default impl returns `Ok(vec![])`. Callers fall back to `Rag.bm25` (local in-memory
/// BM25 built from `data.files`). DuckDbProvider overrides this with a native FTS query /// BM25 built from `data.files`).
/// (DuckDB's `fts` extension, installed once at schema-creation time).
/// ///
/// Callers check `has_native_keyword_search()` before deciding which path to take: /// Callers check `has_native_keyword_search()` before deciding which path to take:
/// - true → call this method; skip `Rag.bm25` /// - true → call this method; skip `Rag.bm25`
/// - false → call `Rag.keyword_search()` which uses `Rag.bm25` (sync, infallible) /// - false → call `Rag.keyword_search()` which uses `Rag.bm25` (sync, infallible)
///
/// YamlProvider and QdrantProvider do NOT override this (return empty).
async fn keyword_search(&self, query: &str, top_k: usize) -> Result<Vec<(DocumentId, f32)>> { async fn keyword_search(&self, query: &str, top_k: usize) -> Result<Vec<(DocumentId, f32)>> {
let _ = (query, top_k); let _ = (query, top_k);
Ok(vec![]) Ok(vec![])
} }
src/rag/providers/duckdb.rs
+659 -181
View File
File diff suppressed because it is too large Load Diff
src/rag/providers/mod.rs
-7
View File
@@ -1,15 +1,8 @@
mod yaml; mod yaml;
// Use `self::` on every re-export in this file. Once a `mod duckdb;` sits here
// alongside a dependency on the `duckdb` CRATE, a bare `pub use duckdb::...`
// is ambiguous (E0659) — `use` paths resolve against both this module's items
// and the extern prelude, and `use` declarations may not shadow.
pub use self::yaml::YamlProvider; pub use self::yaml::YamlProvider;
mod duckdb; mod duckdb;
pub use self::duckdb::DuckDbProvider; pub use self::duckdb::DuckDbProvider;
// `create()` in rag/mod.rs derives the sidecar path through this. It is `pub(crate)`
// in providers/duckdb.rs, and the re-export must be `pub(crate)` too — a `pub use` of
// a `pub(crate)` item is E0364/E0365.
pub(crate) use self::duckdb::duckdb_path_from_yaml; pub(crate) use self::duckdb::duckdb_path_from_yaml;
mod qdrant; mod qdrant;
src/rag/providers/qdrant.rs
+324 -72
View File
@@ -3,7 +3,124 @@ use crate::rag::{DocumentId, RagData};
use anyhow::{Context, Result, bail}; use anyhow::{Context, Result, bail};
use async_trait::async_trait; use async_trait::async_trait;
use parking_lot::RwLock;
use reqwest::header::{HeaderMap, HeaderValue};
use reqwest::{Client, Response, StatusCode};
use serde_json::Value;
use std::collections::HashMap; use std::collections::HashMap;
use std::sync::Arc;
/// Marks a `DocumentId` that stands in for a point id Coyote cannot carry
/// directly. Qdrant accepts UUID strings as point ids, and that is what
/// LangChain writes by default.
///
/// `DocumentId` packs `(file_index, document_index)` into one `usize` with the
/// file index in the high half, so this bit is only reachable at a file index of
/// 2^31. Nothing local gets near that, and an attached RAG builds no local index
/// at all — `data.files` and `data.vectors` stay empty and every
/// `DocumentId::split` caller early-returns on `data.attached`. Along the
/// attached path the id is an opaque key carried through RRF, which is what
/// makes a synthetic one safe here and nowhere else.
const SYNTHETIC_ID_TAG: usize = 1 << (usize::BITS - 1);
/// Two-way map between a raw Qdrant point id and the `DocumentId` the retrieval
/// pipeline sees.
///
/// Only ids that cannot survive the round trip are interned. A plain `u64` that
/// fits below the tag keeps mapping to itself, so integer-keyed collections
/// behave exactly as they did before this map existed.
#[derive(Default)]
struct PointIdInterner {
handles: HashMap<String, DocumentId>,
raw: HashMap<DocumentId, Value>,
next: usize,
}
impl PointIdInterner {
/// The `DocumentId` for a raw point id, minting a handle if one is needed.
///
/// `None` only for a missing id, which is a malformed response.
fn document_id(&mut self, raw: &Value) -> Option<DocumentId> {
if raw.is_null() {
return None;
}
// The pre-existing integer path, unchanged. `try_from` rather than `as`
// so a value too wide for the target's `usize` is interned instead of
// silently truncated into a different point.
if let Some(n) = raw.as_u64()
&& let Ok(n) = usize::try_from(n)
&& n & SYNTHETIC_ID_TAG == 0
{
return Some(DocumentId(n));
}
Some(self.intern(raw))
}
fn intern(&mut self, raw: &Value) -> DocumentId {
// Keyed on the JSON rendering, so the string "1" and the integer 1 are
// not conflated into one point.
let key = raw.to_string();
if let Some(handle) = self.handles.get(&key) {
return *handle;
}
let handle = DocumentId(SYNTHETIC_ID_TAG | self.next);
self.next += 1;
self.handles.insert(key, handle);
self.raw.insert(handle, raw.clone());
handle
}
/// The original id for a handle, or `None` when the id was never interned —
/// i.e. it is a plain integer that is already its own id.
fn raw_id(&self, handle: DocumentId) -> Option<&Value> {
self.raw.get(&handle)
}
/// Builds the `ids` array for an outbound `/points` fetch. Every entry is the
/// id Qdrant issued, integer or string; a synthetic handle must never leave
/// this process.
fn outbound_ids(&self, ids: &[DocumentId]) -> Vec<Value> {
ids.iter()
.map(|id| match self.raw_id(*id) {
Some(raw) => raw.clone(),
None => Value::from(id.0 as u64),
})
.collect()
}
}
fn parse_search_hits(
interner: &mut PointIdInterner,
body: &Value,
min_score: f32,
) -> Result<Vec<(DocumentId, f32)>> {
let hits = body["result"]
.as_array()
.context("Unexpected /points/search response shape")?;
Ok(hits
.iter()
.filter_map(|pt| {
let score = pt["score"].as_f64()? as f32;
Some((interner.document_id(&pt["id"])?, score))
})
.filter(|(_, score)| *score > min_score)
.collect())
}
fn parse_points(interner: &mut PointIdInterner, body: &Value) -> Result<Vec<(DocumentId, String)>> {
let points = body["result"]
.as_array()
.context("Unexpected /points response shape")?;
Ok(points
.iter()
.filter_map(|pt| {
let text = pt["payload"]["page_content"].as_str()?.to_string();
Some((interner.document_id(&pt["id"])?, text))
})
.collect())
}
/// Render Qdrant's error envelope into a human-readable message. /// Render Qdrant's error envelope into a human-readable message.
/// ///
@@ -15,22 +132,18 @@ use std::collections::HashMap;
/// * routing-level 404s (a wrong HTTP verb) return an EMPTY body with no JSON at /// * routing-level 404s (a wrong HTTP verb) return an EMPTY body with no JSON at
/// all, which without the length check surfaces as "EOF while parsing a value" /// all, which without the length check surfaces as "EOF while parsing a value"
/// instead of the actual 404. /// instead of the actual 404.
fn format_error_body(status: reqwest::StatusCode, body: &str) -> String { fn format_error_body(status: StatusCode, body: &str) -> String {
if body.is_empty() { if body.is_empty() {
return format!("HTTP {status} (empty body — check the HTTP verb and path)"); return format!("HTTP {status} (empty body — check the HTTP verb and path)");
} }
serde_json::from_str::<serde_json::Value>(body) serde_json::from_str::<Value>(body)
.ok() .ok()
.and_then(|v| v["status"]["error"].as_str().map(str::to_string)) .and_then(|v| v["status"]["error"].as_str().map(str::to_string))
.unwrap_or_else(|| format!("HTTP {status}: {body}")) .unwrap_or_else(|| format!("HTTP {status}: {body}"))
} }
/// Read the vector dimension out of a parsed `GET /collections/{name}` response. /// Read the vector dimension out of a parsed `GET /collections/{name}` response.
/// fn vector_dimension_from_collection(body: &Value) -> Result<u64> {
/// Unnamed collections put `size` directly under `vectors`; named ones nest it
/// under the vector's name. Both shapes occur in the wild, so try the flat one
/// first and fall back to the first named entry.
fn vector_dimension_from_collection(body: &serde_json::Value) -> Result<u64> {
let params = &body["result"]["config"]["params"]; let params = &body["result"]["config"]["params"];
params["vectors"]["size"] params["vectors"]["size"]
.as_u64() .as_u64()
@@ -47,12 +160,12 @@ fn vector_dimension_from_collection(body: &serde_json::Value) -> Result<u64> {
/// (multi-vector) collection. /// (multi-vector) collection.
/// ///
/// `vector_search` posts an unnamed vector, which a named-vector collection /// `vector_search` posts an unnamed vector, which a named-vector collection
/// rejects with HTTP 400 on every query — so attaching one yields a RAG that is /// rejects with HTTP 400 on every query, so attaching one yields a RAG that is
/// silently 100% broken. A named collection holding a SINGLE vector is /// silently 100% broken. A named collection holding a SINGLE vector is
/// structurally a map, identical in kind to the multi-named case, and rejects /// structurally a map, identical in kind to the multi-named case, and rejects
/// the same way; testing for a numeric `size` directly under `vectors` catches /// the same way; testing for a numeric `size` directly under `vectors` catches
/// it, whereas counting keys (`len() > 1`) would wrongly accept it. /// it, whereas counting keys (`len() > 1`) would wrongly accept it.
fn is_multi_vector_config(body: &serde_json::Value) -> bool { fn is_multi_vector_config(body: &Value) -> bool {
body["result"]["config"]["params"]["vectors"]["size"] body["result"]["config"]["params"]["vectors"]["size"]
.as_u64() .as_u64()
.is_none() .is_none()
@@ -63,33 +176,28 @@ fn is_multi_vector_config(body: &serde_json::Value) -> bool {
/// Attach-only: this provider never writes to the remote collection. Coyote does /// Attach-only: this provider never writes to the remote collection. Coyote does
/// not own the data, and `rebuild_indexes` refuses rather than pretending to. /// not own the data, and `rebuild_indexes` refuses rather than pretending to.
pub struct QdrantProvider { pub struct QdrantProvider {
/// `reqwest::Client` is Arc-backed, so `clone()` is O(1) and shares both the client: Client,
/// connection pool and the `api-key` default header injected at build time.
client: reqwest::Client,
/// Includes the scheme, e.g. `http://qdrant.example.com:6333`.
base_url: String, base_url: String,
collection: String, collection: String,
point_ids: Arc<RwLock<PointIdInterner>>,
} }
impl QdrantProvider { impl QdrantProvider {
/// The resolved API key is injected as a default header here and is fn make_client(api_key: Option<&str>) -> Result<Client> {
/// deliberately NOT stored on the struct: the plaintext value stays a local let mut headers = HeaderMap::new();
/// of the caller and never outlives it.
fn make_client(api_key: Option<&str>) -> Result<reqwest::Client> {
let mut headers = reqwest::header::HeaderMap::new();
if let Some(key) = api_key { if let Some(key) = api_key {
let mut value = reqwest::header::HeaderValue::from_str(key) let mut value =
.context("api-key header value is not valid ASCII")?; HeaderValue::from_str(key).context("api-key header value is not valid ASCII")?;
value.set_sensitive(true); value.set_sensitive(true);
headers.insert("api-key", value); headers.insert("api-key", value);
} }
reqwest::Client::builder() Client::builder()
.default_headers(headers) .default_headers(headers)
.build() .build()
.context("Failed to build reqwest client") .context("Failed to build reqwest client")
} }
fn normalize_base_url(host: &str) -> String { pub(crate) fn normalize_base_url(host: &str) -> String {
if host.starts_with("http://") || host.starts_with("https://") { if host.starts_with("http://") || host.starts_with("https://") {
host.to_string() host.to_string()
} else { } else {
@@ -97,7 +205,7 @@ impl QdrantProvider {
} }
} }
async fn error_message(resp: reqwest::Response) -> String { async fn error_message(resp: Response) -> String {
let status = resp.status(); let status = resp.status();
let body = resp.text().await.unwrap_or_default(); let body = resp.text().await.unwrap_or_default();
format_error_body(status, &body) format_error_body(status, &body)
@@ -109,7 +217,7 @@ impl QdrantProvider {
host: &str, host: &str,
collection: &str, collection: &str,
api_key: Option<&str>, api_key: Option<&str>,
) -> Result<serde_json::Value> { ) -> Result<Value> {
let base_url = Self::normalize_base_url(host); let base_url = Self::normalize_base_url(host);
let client = Self::make_client(api_key)?; let client = Self::make_client(api_key)?;
let resp = client let resp = client
@@ -123,13 +231,13 @@ impl QdrantProvider {
Self::error_message(resp).await Self::error_message(resp).await
); );
} }
Ok(resp.json().await?) Ok(resp.json().await?)
} }
pub async fn new(host: &str, collection: &str, api_key: Option<&str>) -> Result<Self> { pub async fn new(host: &str, collection: &str, api_key: Option<&str>) -> Result<Self> {
let base_url = Self::normalize_base_url(host); let base_url = Self::normalize_base_url(host);
let client = Self::make_client(api_key)?; let client = Self::make_client(api_key)?;
// Preflight: confirm the collection exists and we may read it.
let resp = client let resp = client
.get(format!("{base_url}/collections/{collection}")) .get(format!("{base_url}/collections/{collection}"))
.send() .send()
@@ -141,10 +249,12 @@ impl QdrantProvider {
Self::error_message(resp).await Self::error_message(resp).await
); );
} }
Ok(Self { Ok(Self {
client, client,
base_url, base_url,
collection: collection.to_string(), collection: collection.to_string(),
point_ids: Arc::default(),
}) })
} }
@@ -162,13 +272,15 @@ impl QdrantProvider {
Self::error_message(resp).await Self::error_message(resp).await
); );
} }
let body: serde_json::Value = resp.json().await?;
let body: Value = resp.json().await?;
let names = body["result"]["collections"] let names = body["result"]["collections"]
.as_array() .as_array()
.context("Unexpected /collections response shape")? .context("Unexpected /collections response shape")?
.iter() .iter()
.filter_map(|v| v["name"].as_str().map(str::to_string)) .filter_map(|v| v["name"].as_str().map(str::to_string))
.collect(); .collect();
Ok(names) Ok(names)
} }
@@ -178,6 +290,7 @@ impl QdrantProvider {
api_key: Option<&str>, api_key: Option<&str>,
) -> Result<u64> { ) -> Result<u64> {
let body = Self::fetch_collection(host, collection, api_key).await?; let body = Self::fetch_collection(host, collection, api_key).await?;
vector_dimension_from_collection(&body) vector_dimension_from_collection(&body)
} }
@@ -187,11 +300,10 @@ impl QdrantProvider {
api_key: Option<&str>, api_key: Option<&str>,
) -> Result<bool> { ) -> Result<bool> {
let body = Self::fetch_collection(host, collection, api_key).await?; let body = Self::fetch_collection(host, collection, api_key).await?;
Ok(is_multi_vector_config(&body)) Ok(is_multi_vector_config(&body))
} }
/// Peek at one point to learn how its ID is typed. Returns the raw JSON
/// rendering, so a string ID comes back quoted and an integer one bare.
pub async fn sample_point_id( pub async fn sample_point_id(
host: &str, host: &str,
collection: &str, collection: &str,
@@ -201,23 +313,27 @@ impl QdrantProvider {
let client = Self::make_client(api_key)?; let client = Self::make_client(api_key)?;
let url = format!("{base_url}/collections/{collection}/points/scroll"); let url = format!("{base_url}/collections/{collection}/points/scroll");
let body = serde_json::json!({ "limit": 1, "with_payload": false }); let body = serde_json::json!({ "limit": 1, "with_payload": false });
let resp = client let resp = client
.post(&url) .post(&url)
.json(&body) .json(&body)
.send() .send()
.await .await
.with_context(|| format!("Failed to connect to {host}"))?; .with_context(|| format!("Failed to connect to {host}"))?;
if !resp.status().is_success() { if !resp.status().is_success() {
bail!( bail!(
"Failed to sample a point from '{collection}': {}", "Failed to sample a point from '{collection}': {}",
Self::error_message(resp).await Self::error_message(resp).await
); );
} }
let data: serde_json::Value = resp.json().await?;
let data: Value = resp.json().await?;
let id_val = data["result"]["points"] let id_val = data["result"]["points"]
.as_array() .as_array()
.and_then(|pts| pts.first()) .and_then(|pts| pts.first())
.map(|pt| pt["id"].to_string()); .map(|pt| pt["id"].to_string());
Ok(id_val) Ok(id_val)
} }
} }
@@ -251,22 +367,12 @@ impl RagProvider for QdrantProvider {
Self::error_message(resp).await Self::error_message(resp).await
); );
} }
let data: serde_json::Value = resp.json().await?; let data: Value = resp.json().await?;
let results = data["result"] // The interner is what lets a UUID-keyed collection work: a string id gets
.as_array() // a synthetic handle here and the original is replayed by `fetch_content`.
.context("Unexpected /points/search response shape")? let mut interner = self.point_ids.write();
.iter()
.filter_map(|pt| { parse_search_hits(&mut interner, &data, min_score)
// String (UUID) IDs yield None here and are dropped. The attach
// wizard rejects such collections up front so this cannot silently
// become "zero results, no error".
let id = pt["id"].as_u64()? as usize;
let score = pt["score"].as_f64()? as f32;
Some((DocumentId(id), score))
})
.filter(|(_, score)| *score > min_score)
.collect();
Ok(results)
} }
async fn fetch_content(&self, ids: &[DocumentId]) -> Result<Vec<(DocumentId, String)>> { async fn fetch_content(&self, ids: &[DocumentId]) -> Result<Vec<(DocumentId, String)>> {
@@ -274,12 +380,15 @@ impl RagProvider for QdrantProvider {
return Ok(vec![]); return Ok(vec![]);
} }
let url = format!("{}/collections/{}/points", self.base_url, self.collection); let url = format!("{}/collections/{}/points", self.base_url, self.collection);
let id_list: Vec<u64> = ids.iter().map(|d| d.0 as u64).collect(); // Qdrant is asked for the ids it issued, never for a synthetic handle.
let id_list = self.point_ids.read().outbound_ids(ids);
let body = serde_json::json!({ let body = serde_json::json!({
"ids": id_list, "ids": id_list,
"with_payload": true, "with_payload": true,
}); });
let resp = self.client.post(&url).json(&body).send().await?; let resp = self.client.post(&url).json(&body).send().await?;
if !resp.status().is_success() { if !resp.status().is_success() {
bail!( bail!(
"Qdrant point fetch on '{}' failed: {}", "Qdrant point fetch on '{}' failed: {}",
@@ -287,29 +396,24 @@ impl RagProvider for QdrantProvider {
Self::error_message(resp).await Self::error_message(resp).await
); );
} }
let data: serde_json::Value = resp.json().await?; let data: Value = resp.json().await?;
let mut rows: Vec<(DocumentId, String)> = data["result"] let mut rows = {
.as_array() let mut interner = self.point_ids.write();
.context("Unexpected /points response shape")? parse_points(&mut interner, &data)?
.iter() };
.filter_map(|pt| {
let id = pt["id"].as_u64()? as usize;
let text = pt["payload"]["page_content"].as_str()?.to_string();
Some((DocumentId(id), text))
})
.collect();
// `/points` does not guarantee response order matches request order, and the // `/points` does not guarantee response order matches request order, and the
// caller's RRF ranking is carried by that order. Restore it. // caller's RRF ranking is carried by that order. Restore it.
let position: HashMap<DocumentId, usize> = let position: HashMap<DocumentId, usize> =
ids.iter().enumerate().map(|(i, id)| (*id, i)).collect(); ids.iter().enumerate().map(|(i, id)| (*id, i)).collect();
rows.sort_by_key(|(id, _)| position.get(id).copied().unwrap_or(usize::MAX)); rows.sort_by_key(|(id, _)| position.get(id).copied().unwrap_or(usize::MAX));
Ok(rows) Ok(rows)
} }
async fn rebuild_indexes(&mut self, data: &RagData, _full_rebuild: bool) -> Result<()> { async fn rebuild_indexes(&mut self, data: &RagData, _full_rebuild: bool) -> Result<()> {
// Both arms refuse. A silent `Ok(())` would make `.rebuild rag` and // Both arms refuse. A silent `Ok(())` would make `.rebuild rag` and
// `.edit rag-docs` look like they worked while writing nothing to the // `.edit rag-docs` look like they worked while writing nothing to the
// remote — leaving the user believing the collection was updated. // remote, leaving the user believing the collection was updated.
if data.attached { if data.attached {
bail!( bail!(
"This RAG is attached to an external Qdrant collection. Coyote does not own \ "This RAG is attached to an external Qdrant collection. Coyote does not own \
@@ -324,10 +428,17 @@ impl RagProvider for QdrantProvider {
// Cloning the client shares the connection pool and the injected api-key // Cloning the client shares the connection pool and the injected api-key
// header. Sharing is correct: both handles address the same remote // header. Sharing is correct: both handles address the same remote
// collection, and neither of them writes to it. // collection, and neither of them writes to it.
//
// The point-id map is shared for the same reason, and because it MUST be:
// `Rag::clone()` hands the clone `DocumentId`s that the original minted,
// so a fresh map would resolve them to nothing and `fetch_content` would
// ask Qdrant for a synthetic handle — zero results, no error. Resetting it
// would also re-mint handles for ids the original still holds.
Box::new(Self { Box::new(Self {
client: self.client.clone(), client: self.client.clone(),
base_url: self.base_url.clone(), base_url: self.base_url.clone(),
collection: self.collection.clone(), collection: self.collection.clone(),
point_ids: Arc::clone(&self.point_ids),
}) })
} }
} }
@@ -340,7 +451,7 @@ mod tests {
fn error_message_reads_the_object_status_envelope() { fn error_message_reads_the_object_status_envelope() {
let body = let body =
r#"{"status": {"error": "Wrong input: Not existing vector name error:"}, "time": 0.0}"#; r#"{"status": {"error": "Wrong input: Not existing vector name error:"}, "time": 0.0}"#;
let msg = format_error_body(reqwest::StatusCode::BAD_REQUEST, body); let msg = format_error_body(StatusCode::BAD_REQUEST, body);
assert!(msg.contains("Not existing vector name"), "got: {msg}"); assert!(msg.contains("Not existing vector name"), "got: {msg}");
assert!( assert!(
!msg.contains("EOF"), !msg.contains("EOF"),
@@ -350,15 +461,14 @@ mod tests {
#[test] #[test]
fn error_message_survives_the_string_status_and_the_empty_body() { fn error_message_survives_the_string_status_and_the_empty_body() {
// Success envelope: `status` is a bare string, so the object lookup misses let ok = format_error_body(StatusCode::OK, r#"{"status": "ok", "time": 0.0}"#);
// and we must fall back rather than panic or invent an error text.
let ok = format_error_body(reqwest::StatusCode::OK, r#"{"status": "ok", "time": 0.0}"#);
assert!( assert!(
ok.contains("200"), ok.contains("200"),
"no `status.error` present → fall back to status+body: {ok}" "no `status.error` present → fall back to status+body: {ok}"
); );
// Routing-level 404 from a wrong HTTP verb: empty body, no JSON at all.
let empty = format_error_body(reqwest::StatusCode::NOT_FOUND, ""); let empty = format_error_body(StatusCode::NOT_FOUND, "");
assert!(empty.contains("empty body"), "got: {empty}"); assert!(empty.contains("empty body"), "got: {empty}");
assert!( assert!(
empty.contains("verb"), empty.contains("verb"),
@@ -384,15 +494,11 @@ mod tests {
#[test] #[test]
fn is_multi_vector_rejects_the_named_single_collection() { fn is_multi_vector_rejects_the_named_single_collection() {
// The only supported shape: a single unnamed vector.
let unnamed = serde_json::json!({ let unnamed = serde_json::json!({
"result": {"config": {"params": {"vectors": {"size": 1536, "distance": "Cosine"}}}} "result": {"config": {"params": {"vectors": {"size": 1536, "distance": "Cosine"}}}}
}); });
assert!(!is_multi_vector_config(&unnamed)); assert!(!is_multi_vector_config(&unnamed));
// Named but SINGLE — structurally a map, and writes to it fail with
// `400 "Wrong input: Not existing vector name error:"`. A `len() > 1` check
// would wrongly accept this one; that is the bug this case exists to catch.
let named_single = serde_json::json!({ let named_single = serde_json::json!({
"result": {"config": {"params": {"vectors": {"text": {"size": 1536}}}}} "result": {"config": {"params": {"vectors": {"text": {"size": 1536}}}}}
}); });
@@ -426,9 +532,10 @@ mod tests {
#[tokio::test] #[tokio::test]
async fn rebuild_indexes_refuses_for_attached_and_unattached_alike() { async fn rebuild_indexes_refuses_for_attached_and_unattached_alike() {
let mut provider = QdrantProvider { let mut provider = QdrantProvider {
client: reqwest::Client::new(), client: Client::new(),
base_url: "http://localhost:6333".to_string(), base_url: "http://localhost:6333".to_string(),
collection: "c".to_string(), collection: "c".to_string(),
point_ids: Arc::default(),
}; };
let attached = RagData { let attached = RagData {
@@ -459,22 +566,165 @@ mod tests {
#[tokio::test] #[tokio::test]
async fn fetch_content_short_circuits_on_an_empty_id_list() { async fn fetch_content_short_circuits_on_an_empty_id_list() {
// No network is touched: the early return happens before any request, which
// is why this can assert against an unreachable host.
let provider = QdrantProvider { let provider = QdrantProvider {
client: reqwest::Client::new(), client: Client::new(),
base_url: "http://127.0.0.1:1".to_string(), base_url: "http://127.0.0.1:1".to_string(),
collection: "c".to_string(), collection: "c".to_string(),
point_ids: Arc::default(),
}; };
assert!(provider.fetch_content(&[]).await.unwrap().is_empty()); assert!(provider.fetch_content(&[]).await.unwrap().is_empty());
} }
/// A UUID-keyed collection has to survive the whole `vector_search` →
/// `fetch_content` round trip, and the fetch must ask Qdrant for the ORIGINAL
/// string id. Parsing ids with `as_u64()` used to drop these hits inside a
/// `filter_map`, i.e. zero results and no error.
#[test]
fn uuid_point_ids_round_trip_and_are_requested_verbatim() {
let mut interner = PointIdInterner::default();
let first_uuid = "3f1b0c2e-1111-4000-8000-000000000001";
let second_uuid = "3f1b0c2e-2222-4000-8000-000000000002";
let search = serde_json::json!({
"result": [
{"id": first_uuid, "score": 0.91},
{"id": second_uuid, "score": 0.42},
]
});
let hits = parse_search_hits(&mut interner, &search, 0.0).unwrap();
assert_eq!(hits.len(), 2, "string ids must not be silently dropped");
let ids: Vec<DocumentId> = hits.iter().map(|(id, _)| *id).collect();
assert_eq!(
interner.outbound_ids(&ids),
vec![Value::from(first_uuid), Value::from(second_uuid)],
"the fetch must send the ids Qdrant issued, not the handles"
);
// Qdrant may answer /points in any order; the handles still map back and
// the caller's RRF ranking is recoverable.
let points = serde_json::json!({
"result": [
{"id": second_uuid, "payload": {"page_content": "second"}},
{"id": first_uuid, "payload": {"page_content": "first"}},
]
});
let mut rows = parse_points(&mut interner, &points).unwrap();
let position: HashMap<DocumentId, usize> =
ids.iter().enumerate().map(|(i, id)| (*id, i)).collect();
rows.sort_by_key(|(id, _)| position.get(id).copied().unwrap_or(usize::MAX));
assert_eq!(
rows,
vec![
(ids[0], "first".to_string()),
(ids[1], "second".to_string())
]
);
}
/// Integer-keyed collections must be untouched by the interner: the id maps to
/// itself on the way in and goes back out as the same integer.
#[test]
fn integer_point_ids_are_passed_through_untouched() {
let mut interner = PointIdInterner::default();
let search = serde_json::json!({
"result": [{"id": 7, "score": 0.9}, {"id": 0, "score": 0.5}]
});
let hits = parse_search_hits(&mut interner, &search, 0.0).unwrap();
assert_eq!(
hits,
vec![(DocumentId(7), 0.9_f32), (DocumentId(0), 0.5_f32)]
);
let ids: Vec<DocumentId> = hits.iter().map(|(id, _)| *id).collect();
assert_eq!(
interner.outbound_ids(&ids),
vec![Value::from(7_u64), Value::from(0_u64)],
"integer ids must not be regressed into synthetic handles"
);
assert!(
interner.raw_id(DocumentId(7)).is_none(),
"a plain integer id is its own id and needs no map entry"
);
}
/// Synthetic handles are stable per point id and live in a range no packed
/// `DocumentId` can reach.
#[test]
fn synthetic_handles_are_stable_and_never_collide_with_packed_ids() {
let mut interner = PointIdInterner::default();
let uuid = Value::from("9d2f0a11-3333-4000-8000-00000000000a");
let handle = interner.document_id(&uuid).unwrap();
assert_eq!(
interner.document_id(&uuid).unwrap(),
handle,
"the same point id must keep the same handle across queries"
);
assert_ne!(
interner.document_id(&Value::from("other")).unwrap(),
handle,
"distinct point ids must not share a handle"
);
assert_ne!(handle.0 & SYNTHETIC_ID_TAG, 0, "a handle carries the tag");
// A packed (file_index, document_index) never sets the tag bit: it is the
// top bit of the file index, which would take 2^31 indexed files.
for (file_index, document_index) in [(0, 0), (1, 0), (0, 4242), (1_000_000, 999)] {
assert_eq!(
DocumentId::new(file_index, document_index).0 & SYNTHETIC_ID_TAG,
0,
"packed ({file_index}, {document_index}) must stay out of the handle range"
);
}
// The one integer id that WOULD land on the tag is interned instead of
// being handed back as itself, so it cannot alias a handle.
let collides = Value::from(SYNTHETIC_ID_TAG as u64);
let interned = interner.document_id(&collides).unwrap();
assert_eq!(interner.raw_id(interned), Some(&collides));
assert_eq!(
interner.outbound_ids(&[interned]),
vec![collides],
"the original integer must still be what Qdrant is asked for"
);
}
/// `duplicate()` shares the map rather than resetting it: `Rag::clone()` hands
/// the clone `DocumentId`s the original minted, and a fresh map would turn
/// those into requests for a synthetic handle — zero results, no error.
#[test]
fn duplicate_shares_the_point_id_map() {
let provider = QdrantProvider {
client: Client::new(),
base_url: "http://127.0.0.1:1".to_string(),
collection: "c".to_string(),
point_ids: Arc::default(),
};
let uuid = Value::from("c0ffee00-4444-4000-8000-000000000007");
let handle = provider.point_ids.write().document_id(&uuid).unwrap();
let dup = provider.duplicate(&RagData {
driver: "qdrant".to_string(),
attached: true,
..Default::default()
});
// Downcasting is not available through `dyn RagProvider`, so go via the
// shared Arc: the clone must observe the original's interning.
assert_eq!(Arc::strong_count(&provider.point_ids), 2);
assert_eq!(provider.point_ids.read().raw_id(handle), Some(&uuid));
drop(dup);
}
#[tokio::test] #[tokio::test]
#[ignore] #[ignore]
async fn qdrant_list_collections_requires_running_instance() { async fn qdrant_list_collections_requires_running_instance() {
let collections = QdrantProvider::list_collections("http://localhost:6333", None) let collections = QdrantProvider::list_collections("http://localhost:6333", None)
.await .await
.unwrap(); .unwrap();
assert!(!collections.is_empty()); assert!(!collections.is_empty());
} }
@@ -485,7 +735,9 @@ mod tests {
.await .await
.unwrap(); .unwrap();
let embedding = vec![0.0f32; 1536]; let embedding = vec![0.0f32; 1536];
let results = provider.vector_search(&embedding, 5, 0.0).await.unwrap(); let results = provider.vector_search(&embedding, 5, 0.0).await.unwrap();
assert!(results.len() <= 5); assert!(results.len() <= 5);
} }
} }
src/rag/providers/yaml.rs
+9 -34
View File
@@ -20,9 +20,6 @@ impl YamlProvider {
} }
fn build_content_map(data: &RagData) -> IndexMap<DocumentId, String> { fn build_content_map(data: &RagData) -> IndexMap<DocumentId, String> {
// Keyed on `files`, NOT `vectors`: this is the exact replacement for the
// per-id document lookup it supersedes, and it must resolve every id that
// BM25 or graph_search can produce — both of which enumerate `files`.
data.iter_documents() data.iter_documents()
.map(|(id, doc)| (id, doc.page_content.clone())) .map(|(id, doc)| (id, doc.page_content.clone()))
.collect() .collect()
@@ -52,12 +49,11 @@ impl RagProvider for YamlProvider {
}) })
}) })
.collect(); .collect();
Ok(results) Ok(results)
} }
async fn fetch_content(&self, ids: &[DocumentId]) -> Result<Vec<(DocumentId, String)>> { async fn fetch_content(&self, ids: &[DocumentId]) -> Result<Vec<(DocumentId, String)>> {
// Iterating `ids` (not `content_map`) satisfies the trait's ordering
// contract for free — output order mirrors input order.
Ok(ids Ok(ids
.iter() .iter()
.filter_map(|id| self.content_map.get(id).map(|text| (*id, text.clone()))) .filter_map(|id| self.content_map.get(id).map(|text| (*id, text.clone())))
@@ -65,10 +61,10 @@ impl RagProvider for YamlProvider {
} }
async fn rebuild_indexes(&mut self, data: &RagData, _full_rebuild: bool) -> Result<()> { async fn rebuild_indexes(&mut self, data: &RagData, _full_rebuild: bool) -> Result<()> {
// Local in-memory state — a wholesale rebuild is fast and always correct,
// so the incremental/full distinction is irrelevant here.
self.hnsw = data.build_hnsw(); self.hnsw = data.build_hnsw();
self.content_map = Self::build_content_map(data); self.content_map = Self::build_content_map(data);
Ok(()) Ok(())
} }
@@ -80,14 +76,9 @@ impl RagProvider for YamlProvider {
#[cfg(test)] #[cfg(test)]
mod provider_tests { mod provider_tests {
use super::*; use super::*;
// `RagFile` and `RagDocument` are not used by the impl above, so they are
// imported here rather than at module scope. Both have private fields, which
// is why these tests must live in-crate rather than under `tests/`.
use crate::rag::{RagDocument, RagFile}; use crate::rag::{RagDocument, RagFile};
fn minimal_rag_data() -> RagData { fn minimal_rag_data() -> RagData {
// `..Default::default()` rather than an exhaustive struct literal so that
// later additions to `RagData` do not break this helper.
RagData { RagData {
embedding_model: "text-embedding-3-small".to_string(), embedding_model: "text-embedding-3-small".to_string(),
chunk_size: 1024, chunk_size: 1024,
@@ -99,7 +90,7 @@ mod provider_tests {
} }
} }
/// Two files, one chunk each, with vectors — the minimum needed to exercise /// Two files, one chunk each, with vectors, the minimum needed to exercise
/// `build_content_map` and the `fetch_content` ordering contract. /// `build_content_map` and the `fetch_content` ordering contract.
/// `DocumentId::new(f, d)` packs (file_index, document_index); `RagData::add` /// `DocumentId::new(f, d)` packs (file_index, document_index); `RagData::add`
/// is the real insertion path but a direct literal is sufficient and avoids /// is the real insertion path but a direct literal is sufficient and avoids
@@ -143,13 +134,12 @@ mod provider_tests {
async fn yaml_provider_empty_data_returns_nothing() { async fn yaml_provider_empty_data_returns_nothing() {
let data = minimal_rag_data(); let data = minimal_rag_data();
let provider = YamlProvider::from_data(&data); let provider = YamlProvider::from_data(&data);
let results = provider.fetch_content(&[]).await.unwrap(); let results = provider.fetch_content(&[]).await.unwrap();
assert!(results.is_empty()); assert!(results.is_empty());
} }
/// `fetch_content` MUST return results in the same relative order as the input
/// ids. The reversed-input case is the one that fails if an implementation ever
/// iterates its own map instead of `ids`.
#[tokio::test] #[tokio::test]
async fn yaml_provider_fetch_content_preserves_input_order() { async fn yaml_provider_fetch_content_preserves_input_order() {
let data = populated_rag_data(); let data = populated_rag_data();
@@ -163,8 +153,8 @@ mod provider_tests {
assert_eq!(forward[0].1, "alpha"); assert_eq!(forward[0].1, "alpha");
assert_eq!(forward[1].1, "beta"); assert_eq!(forward[1].1, "beta");
// Reversed input must produce reversed output — NOT storage order.
let reversed = provider.fetch_content(&[b, a]).await.unwrap(); let reversed = provider.fetch_content(&[b, a]).await.unwrap();
assert_eq!( assert_eq!(
reversed[0].1, "beta", reversed[0].1, "beta",
"fetch_content must honor input order" "fetch_content must honor input order"
@@ -172,8 +162,6 @@ mod provider_tests {
assert_eq!(reversed[1].1, "alpha"); assert_eq!(reversed[1].1, "alpha");
} }
/// A missing id is skipped, not an error, and does not disturb the order of
/// the ids that DO resolve.
#[tokio::test] #[tokio::test]
async fn yaml_provider_fetch_content_skips_missing_ids() { async fn yaml_provider_fetch_content_skips_missing_ids() {
let data = populated_rag_data(); let data = populated_rag_data();
@@ -189,23 +177,16 @@ mod provider_tests {
assert_eq!(out[1].1, "beta"); assert_eq!(out[1].1, "beta");
} }
/// `YamlProvider::duplicate()` rebuilds from `data`, so the clone is a genuine
/// independent snapshot. Providers backed by a shared store deliberately are not.
#[tokio::test] #[tokio::test]
async fn yaml_provider_duplicate_returns_equivalent_content() { async fn yaml_provider_duplicate_returns_equivalent_content() {
// MUST be populated_rag_data(): on minimal_rag_data() both providers hold an
// EMPTY content map, so `assert_eq!(r1, r2)` compares two empty vectors and
// passes against a duplicate() that returns nothing at all.
let data = populated_rag_data(); let data = populated_rag_data();
let provider = YamlProvider::from_data(&data); let provider = YamlProvider::from_data(&data);
let dup = provider.duplicate(&data); let dup = provider.duplicate(&data);
let ids = [DocumentId::new(0, 0), DocumentId::new(1, 0)]; let ids = [DocumentId::new(0, 0), DocumentId::new(1, 0)];
// Query with REAL ids, not `&[]` — an empty slice is answered without ever
// touching the content map, so it would pass against a broken duplicate().
let r1 = provider.fetch_content(&ids).await.unwrap(); let r1 = provider.fetch_content(&ids).await.unwrap();
let r2 = dup.fetch_content(&ids).await.unwrap(); let r2 = dup.fetch_content(&ids).await.unwrap();
// Guard against the vacuous case: if both sides resolved nothing, the equality
// below proves nothing. Assert the fixture actually produced content first.
assert_eq!(r1.len(), 2, "fixture must resolve both documents"); assert_eq!(r1.len(), 2, "fixture must resolve both documents");
assert_eq!( assert_eq!(
r1, r2, r1, r2,
@@ -213,11 +194,6 @@ mod provider_tests {
); );
} }
/// The content store is keyed on `files`, never on `vectors`. A vector may exist
/// for an id with no backing file (a stale entry, or a file dropped mid-sync);
/// keying on `vectors` would surface such an id with empty text instead of
/// dropping it. The shared fixture only ever inserts vectors for ids that also
/// have files, so this case has to be constructed here.
#[tokio::test] #[tokio::test]
async fn yaml_provider_content_is_keyed_on_files_not_vectors() { async fn yaml_provider_content_is_keyed_on_files_not_vectors() {
let mut data = populated_rag_data(); let mut data = populated_rag_data();
@@ -232,7 +208,6 @@ mod provider_tests {
"an id present only in `vectors` must not resolve to content" "an id present only in `vectors` must not resolve to content"
); );
// The file-backed ids still resolve, so the assertion above is not vacuous.
let real = provider let real = provider
.fetch_content(&[DocumentId::new(0, 0), DocumentId::new(1, 0)]) .fetch_content(&[DocumentId::new(0, 0), DocumentId::new(1, 0)])
.await .await
src/render/markdown.rs
-11
View File
@@ -1749,11 +1749,6 @@ std::error::Error>> {
); );
} }
/// Removes CSI escape sequences so only printable content is measured.
///
/// Deliberately tolerant of malformed input: a sequence that was sliced
/// mid-escape swallows the following characters, which is precisely the
/// corruption `render_table_pads_columns_by_display_width` exists to catch.
fn strip_ansi(text: &str) -> String { fn strip_ansi(text: &str) -> String {
let mut out = String::with_capacity(text.len()); let mut out = String::with_capacity(text.len());
let mut chars = text.chars(); let mut chars = text.chars();
@@ -1780,12 +1775,6 @@ std::error::Error>> {
assert_eq!(strip_ansi("plain"), "plain"); assert_eq!(strip_ansi("plain"), "plain");
} }
/// `render_table` hands comfy-table pre-styled cells that already contain
/// ANSI escapes, and `colorize_box_chars` adds more afterwards. Column
/// widths are therefore only correct if the escapes are excluded from the
/// width calculation. When they are not, the table still renders and every
/// other assertion in this file still passes -- only the alignment silently
/// degrades -- so this is the sole guard over that behaviour.
#[test] #[test]
fn render_table_pads_columns_by_display_width() { fn render_table_pads_columns_by_display_width() {
use unicode_width::UnicodeWidthStr; use unicode_width::UnicodeWidthStr;
src/repl/mod.rs
+3 -8
View File
@@ -219,7 +219,7 @@ static REPL_COMMANDS: LazyLock<[ReplCommand; 60]> = LazyLock::new(|| {
), ),
ReplCommand::new( ReplCommand::new(
".rag attach", ".rag attach",
"Attach to a pre-existing external RAG (Qdrant)", "Attach to a pre-existing external RAG",
AssertState::False(StateFlags::AGENT), AssertState::False(StateFlags::AGENT),
), ),
ReplCommand::new( ReplCommand::new(
@@ -889,11 +889,7 @@ pub async fn run_repl_command(
let version = args.map(|s| s.trim().to_string()); let version = args.map(|s| s.trim().to_string());
task::spawn_blocking(move || config::run_self_update(version, false)).await??; task::spawn_blocking(move || config::run_self_update(version, false)).await??;
} }
".rag" => { ".rag" => match split_first_arg(args) {
// `split_first_arg` rather than `starts_with("attach ")`: the latter
// misses a bare `.rag attach`, which would silently create a RAG
// literally named "attach".
match split_first_arg(args) {
Some(("attach", rest)) => match rest { Some(("attach", rest)) => match rest {
Some(name) if !name.trim().is_empty() => { Some(name) if !name.trim().is_empty() => {
ctx.attach_rag(name.trim()).await?; ctx.attach_rag(name.trim()).await?;
@@ -903,8 +899,7 @@ pub async fn run_repl_command(
_ => { _ => {
ctx.use_rag(args, abort_signal.clone()).await?; ctx.use_rag(args, abort_signal.clone()).await?;
} }
} },
}
".agent" => match split_first_arg(args) { ".agent" => match split_first_arg(args) {
Some((agent_name, args)) => { Some((agent_name, args)) => {
let (new_args, _) = split_args_text(args.unwrap_or_default(), cfg!(windows)); let (new_args, _) = split_args_text(args.unwrap_or_default(), cfg!(windows));
src/sandbox/mcp_credentials.rs
+1152
View File
File diff suppressed because it is too large Load Diff
src/sandbox/mixins.rs
+216 -58
View File
@@ -34,8 +34,12 @@ impl DiscoveredMixin {
pub fn wrap_mixin_as_kit(mixin_path: &Path) -> Result<PathBuf> { pub fn wrap_mixin_as_kit(mixin_path: &Path) -> Result<PathBuf> {
let bytes = fs::read(mixin_path) let bytes = fs::read(mixin_path)
.with_context(|| format!("Failed to read sbx mixin {}", mixin_path.display()))?; .with_context(|| format!("Failed to read sbx mixin {}", mixin_path.display()))?;
wrap_mixin_bytes_as_kit(&bytes, &mixin_path.display().to_string())
}
pub fn wrap_mixin_bytes_as_kit(bytes: &[u8], label: &str) -> Result<PathBuf> {
let mut hasher = Sha256::new(); let mut hasher = Sha256::new();
hasher.update(&bytes); hasher.update(bytes);
let hash = format!("{:x}", hasher.finalize()); let hash = format!("{:x}", hasher.finalize());
let kit_dir = paths::sbx_mixin_kits_dir().join(&hash); let kit_dir = paths::sbx_mixin_kits_dir().join(&hash);
@@ -49,14 +53,10 @@ pub fn wrap_mixin_as_kit(mixin_path: &Path) -> Result<PathBuf> {
fs::create_dir_all(&kit_dir) fs::create_dir_all(&kit_dir)
.with_context(|| format!("Failed to create mixin kit dir {}", kit_dir.display()))?; .with_context(|| format!("Failed to create mixin kit dir {}", kit_dir.display()))?;
fs::write(&spec_path, &bytes) fs::write(&spec_path, bytes)
.with_context(|| format!("Failed to write {}", spec_path.display()))?; .with_context(|| format!("Failed to write {}", spec_path.display()))?;
debug!( debug!("Wrapped mixin {label} as kit at {}", kit_dir.display());
"Wrapped mixin {} as kit at {}",
mixin_path.display(),
kit_dir.display()
);
Ok(kit_dir) Ok(kit_dir)
} }
@@ -67,17 +67,16 @@ pub fn discover() -> Result<Vec<DiscoveredMixin>> {
push_if_exists(&mut out, paths::sbx_mixin_file())?; push_if_exists(&mut out, paths::sbx_mixin_file())?;
push_if_exists(&mut out, paths::global_tools_sbx_mixin_file())?; push_if_exists(&mut out, paths::global_tools_sbx_mixin_file())?;
for path in collect_subdir_mixins(&paths::functions_dir()) { for path in collect_mixins(&paths::functions_dir(), &[ScanMode::SubdirNamed]) {
out.push(read_mixin(path)?); out.push(read_mixin(path)?);
} }
for path in collect_subdir_mixins(&paths::agents_data_dir()) { for path in collect_mixins(
&paths::agents_data_dir(),
&[ScanMode::SubdirNamed, ScanMode::SubdirFlat],
) {
out.push(read_mixin(path)?); out.push(read_mixin(path)?);
} }
// RAG sidecars are FLAT files named `<rag>.sbx-mixin.yaml` inside rags/, not for path in collect_mixins(&paths::rags_dir(), &[ScanMode::Flat]) {
// the `<subdir>/sbx-mixin.yaml` shape the two scans above walk. Loaded
// unconditionally, mirroring agents/*: a RAG mixin only adds an outbound
// allowlist entry for that RAG's host and opens no inbound rules.
for path in collect_flat_mixins(&paths::rags_dir()) {
out.push(read_mixin(path)?); out.push(read_mixin(path)?);
} }
@@ -97,15 +96,18 @@ pub fn summarize(path: &Path) -> Result<(usize, usize)> {
.with_context(|| format!("Failed to parse sbx mixin {}", path.display()))?; .with_context(|| format!("Failed to parse sbx mixin {}", path.display()))?;
let installs = value let installs = value
.get("commands") .get("setup")
.and_then(|c| c.get("install")) .and_then(|s| s.get("install"))
.or_else(|| value.get("commands").and_then(|c| c.get("install")))
.and_then(|i| i.as_sequence()) .and_then(|i| i.as_sequence())
.map(|s| s.len()) .map(|s| s.len())
.unwrap_or(0); .unwrap_or(0);
let domains = value let domains = value
.get("network") .get("permissions")
.and_then(|n| n.get("allowedDomains")) .and_then(|p| p.get("network"))
.and_then(|n| n.get("allow"))
.or_else(|| value.get("network").and_then(|n| n.get("allowedDomains")))
.and_then(|d| d.as_sequence()) .and_then(|d| d.as_sequence())
.map(|s| s.len()) .map(|s| s.len())
.unwrap_or(0); .unwrap_or(0);
@@ -161,29 +163,54 @@ fn read_mixin(path: PathBuf) -> Result<DiscoveredMixin> {
}) })
} }
fn collect_subdir_mixins(dir: &Path) -> Vec<PathBuf> { /// One on-disk layout a mixin scan can look for. A scan takes a set of these,
/// and each mode contributes only the shape it names.
#[derive(Debug, Clone, Copy, PartialEq, Eq)]
enum ScanMode {
/// `<dir>/*.sbx-mixin.yaml`
Flat,
/// `<dir>/*/sbx-mixin.yaml`
SubdirNamed,
/// `<dir>/*/*.sbx-mixin.yaml`
SubdirFlat,
}
/// Collects mixin paths under `dir` for every requested layout. Missing or
/// unreadable directories yield nothing rather than an error — these paths are
/// all optional on disk.
///
/// Order is deterministic: flat matches first (sorted by file name), then each
/// subdirectory in sorted order, contributing its named mixin before its
/// suffixed ones.
fn collect_mixins(dir: &Path, modes: &[ScanMode]) -> Vec<PathBuf> {
let mut result = Vec::new(); let mut result = Vec::new();
let Ok(rd) = read_dir(dir) else { return result };
let mut entries: Vec<_> = rd if modes.contains(&ScanMode::Flat) {
.flatten() result.extend(suffixed_mixins_in(dir));
.filter(|e| e.file_type().map(|t| t.is_dir()).unwrap_or(false)) }
.collect();
entries.sort_by_key(|e| e.file_name());
for entry in entries { let named = modes.contains(&ScanMode::SubdirNamed);
let candidate = entry.path().join(SBX_MIXIN_FILE_NAME); let subdir_flat = modes.contains(&ScanMode::SubdirFlat);
if !named && !subdir_flat {
return result;
}
for subdir in subdirs_of(dir) {
if named {
let candidate = subdir.join(SBX_MIXIN_FILE_NAME);
if candidate.exists() { if candidate.exists() {
result.push(candidate); result.push(candidate);
} }
} }
if subdir_flat {
result.extend(suffixed_mixins_in(&subdir));
}
}
result result
} }
/// Mixins stored as flat `<name>.sbx-mixin.yaml` files directly inside `dir`, fn suffixed_mixins_in(dir: &Path) -> Vec<PathBuf> {
/// matched by suffix rather than by exact filename.
fn collect_flat_mixins(dir: &Path) -> Vec<PathBuf> {
let mut result = Vec::new(); let mut result = Vec::new();
let Ok(rd) = read_dir(dir) else { return result }; let Ok(rd) = read_dir(dir) else { return result };
@@ -198,10 +225,21 @@ fn collect_flat_mixins(dir: &Path) -> Vec<PathBuf> {
.collect(); .collect();
entries.sort_by_key(|e| e.file_name()); entries.sort_by_key(|e| e.file_name());
for entry in entries { result.extend(entries.into_iter().map(|e| e.path()));
result.push(entry.path()); result
} }
fn subdirs_of(dir: &Path) -> Vec<PathBuf> {
let mut result = Vec::new();
let Ok(rd) = read_dir(dir) else { return result };
let mut entries: Vec<_> = rd
.flatten()
.filter(|e| e.file_type().map(|t| t.is_dir()).unwrap_or(false))
.collect();
entries.sort_by_key(|e| e.file_name());
result.extend(entries.into_iter().map(|e| e.path()));
result result
} }
@@ -221,6 +259,13 @@ mod tests {
root root
} }
fn file_names(paths: &[PathBuf]) -> Vec<&str> {
paths
.iter()
.map(|p| p.file_name().unwrap().to_str().unwrap())
.collect()
}
#[test] #[test]
fn summarize_counts_installs_and_domains() { fn summarize_counts_installs_and_domains() {
let root = unique_root("sbx-mixin-counts"); let root = unique_root("sbx-mixin-counts");
@@ -228,6 +273,34 @@ mod tests {
fs::write( fs::write(
&path, &path,
r#" r#"
schemaVersion: "2"
kind: mixin
setup:
install:
- command: "echo hi"
- command: "echo bye"
permissions:
network:
allow:
- "a.example.com:443"
- "b.example.com:443"
- "c.example.com:443"
"#,
)
.unwrap();
assert_eq!(summarize(&path).unwrap(), (2, 3));
let _ = fs::remove_dir_all(&root);
}
#[test]
fn summarize_falls_back_to_v1_field_paths() {
let root = unique_root("sbx-mixin-counts-v1");
let path = root.join("sbx-mixin.yaml");
fs::write(
&path,
r#"
schemaVersion: "1" schemaVersion: "1"
kind: mixin kind: mixin
commands: commands:
@@ -276,7 +349,7 @@ network:
} }
#[test] #[test]
fn collect_subdir_mixins_sorts_and_skips_missing() { fn subdir_named_scan_sorts_and_skips_missing() {
let root = unique_root("sbx-mixin-subdirs"); let root = unique_root("sbx-mixin-subdirs");
for name in ["zebra", "apple", "no-mixin", "mango"] { for name in ["zebra", "apple", "no-mixin", "mango"] {
let dir = root.join(name); let dir = root.join(name);
@@ -286,7 +359,7 @@ network:
} }
} }
let found = collect_subdir_mixins(&root); let found = collect_mixins(&root, &[ScanMode::SubdirNamed]);
let names: Vec<String> = found let names: Vec<String> = found
.iter() .iter()
.map(|p| { .map(|p| {
@@ -304,9 +377,9 @@ network:
} }
#[test] #[test]
fn collect_subdir_mixins_returns_empty_for_missing_dir() { fn subdir_named_scan_returns_empty_for_missing_dir() {
let absent = env::temp_dir().join("coyote-definitely-not-here-xyz"); let absent = env::temp_dir().join("coyote-definitely-not-here-xyz");
let found = collect_subdir_mixins(&absent); let found = collect_mixins(&absent, &[ScanMode::SubdirNamed]);
assert!(found.is_empty()); assert!(found.is_empty());
} }
@@ -375,6 +448,19 @@ network:
assert_eq!(fs::read_to_string(&spec).unwrap(), content); assert_eq!(fs::read_to_string(&spec).unwrap(), content);
} }
#[test]
#[serial]
fn wrap_mixin_bytes_as_kit_writes_spec_yaml() {
let _guard = TestCacheDirGuard::new();
let content = b"schemaVersion: '2'\nkind: mixin\nname: generated\n";
let kit_dir = wrap_mixin_bytes_as_kit(content, "generated").unwrap();
let spec = kit_dir.join("spec.yaml");
assert!(spec.exists(), "spec.yaml must exist in wrapped kit dir");
assert_eq!(fs::read(&spec).unwrap(), content);
}
#[test] #[test]
#[serial] #[serial]
fn wrap_mixin_as_kit_is_deterministic_for_identical_content() { fn wrap_mixin_as_kit_is_deterministic_for_identical_content() {
@@ -472,52 +558,124 @@ network:
} }
} }
/// RAG sidecars are flat `<name>.sbx-mixin.yaml` files, matched by SUFFIX.
#[test] #[test]
fn collect_flat_mixins_matches_rag_sidecars_by_suffix() { fn flat_scan_matches_rag_sidecars_by_suffix() {
let root = unique_root("flat-mixins"); let root = unique_root("flat-mixins");
fs::write(root.join("company-docs.sbx-mixin.yaml"), "kind: mixin\n").unwrap(); fs::write(root.join("company-docs.sbx-mixin.yaml"), "kind: mixin\n").unwrap();
fs::write(root.join("alpha.sbx-mixin.yaml"), "kind: mixin\n").unwrap(); fs::write(root.join("alpha.sbx-mixin.yaml"), "kind: mixin\n").unwrap();
// The RAGs themselves must not be picked up, only their sidecars.
fs::write(root.join("company-docs.yaml"), "driver: qdrant\n").unwrap(); fs::write(root.join("company-docs.yaml"), "driver: qdrant\n").unwrap();
fs::write(root.join("notes.yaml"), "driver: yaml\n").unwrap(); fs::write(root.join("notes.yaml"), "driver: yaml\n").unwrap();
// A directory whose name ends in the suffix is not a mixin file.
fs::create_dir_all(root.join("decoy.sbx-mixin.yaml")).unwrap(); fs::create_dir_all(root.join("decoy.sbx-mixin.yaml")).unwrap();
let found = collect_flat_mixins(&root); let found = collect_mixins(&root, &[ScanMode::Flat]);
let names: Vec<_> = found
.iter()
.map(|p| p.file_name().unwrap().to_str().unwrap())
.collect();
// Sorted by file name, so the order is deterministic.
assert_eq!( assert_eq!(
names, file_names(&found),
vec!["alpha.sbx-mixin.yaml", "company-docs.sbx-mixin.yaml"] vec!["alpha.sbx-mixin.yaml", "company-docs.sbx-mixin.yaml"]
); );
let _ = fs::remove_dir_all(&root); let _ = fs::remove_dir_all(&root);
} }
/// Why `collect_flat_mixins` had to be written: the existing collector walks /// Every scan site in `discover()` picks its modes assuming each mode owns
/// SUBDIRECTORIES for a file named exactly `sbx-mixin.yaml`, so it cannot see /// exactly one layout and nothing else. `agents_data_dir()` requests two
/// a flat sidecar. If this ever starts finding them, the new collector is /// modes at once, so an overlap would collect the same file twice and
/// redundant — but until then, removing it silently drops every RAG mixin. /// `create_sandbox` would pass it as two `--kit` flags.
#[test] #[test]
fn collect_subdir_mixins_cannot_see_flat_rag_sidecars() { fn each_scan_mode_owns_exactly_one_layout() {
let root = unique_root("flat-vs-subdir"); let root = unique_root("scan-mode-ownership");
fs::write(root.join("company-docs.sbx-mixin.yaml"), "kind: mixin\n").unwrap(); let agent = root.join("researcher");
fs::create_dir_all(&agent).unwrap();
let flat = root.join("company-docs.sbx-mixin.yaml");
let subdir_named = agent.join("sbx-mixin.yaml");
let subdir_flat = agent.join("handbook.sbx-mixin.yaml");
for path in [&flat, &subdir_named, &subdir_flat] {
fs::write(path, "kind: mixin\n").unwrap();
}
assert!(collect_subdir_mixins(&root).is_empty()); assert_eq!(collect_mixins(&root, &[ScanMode::Flat]), vec![flat.clone()]);
assert_eq!(collect_flat_mixins(&root).len(), 1); assert_eq!(
collect_mixins(&root, &[ScanMode::SubdirNamed]),
vec![subdir_named.clone()]
);
assert_eq!(
collect_mixins(&root, &[ScanMode::SubdirFlat]),
vec![subdir_flat.clone()]
);
let all = collect_mixins(
&root,
&[ScanMode::Flat, ScanMode::SubdirNamed, ScanMode::SubdirFlat],
);
assert_eq!(all, vec![flat, subdir_named, subdir_flat]);
let mut deduped = all.clone();
deduped.sort();
deduped.dedup();
assert_eq!(
deduped.len(),
all.len(),
"no mixin may be collected twice: {all:?}"
);
let _ = fs::remove_dir_all(&root); let _ = fs::remove_dir_all(&root);
} }
#[test] #[test]
fn collect_flat_mixins_tolerates_a_missing_directory() { fn flat_scan_tolerates_a_missing_directory() {
let root = unique_root("flat-missing"); let root = unique_root("flat-missing");
let absent = root.join("nope"); let absent = root.join("nope");
assert!(collect_flat_mixins(&absent).is_empty()); assert!(collect_mixins(&absent, &[ScanMode::Flat]).is_empty());
let _ = fs::remove_dir_all(&root);
}
/// `generate_rag_sbx_mixin` writes an agent-scoped RAG sidecar next to the
/// rag yaml, at `<agents>/<agent>/<rag>.sbx-mixin.yaml`. Before `SubdirFlat`
/// existed, nothing scanned that shape and attaching a Qdrant RAG from
/// inside an agent produced no network allow rule and no credential.
#[test]
fn agent_scoped_rag_sidecar_is_discovered() {
let root = unique_root("agent-scoped-rag");
let agent = root.join("researcher");
fs::create_dir_all(&agent).unwrap();
fs::write(agent.join("company-docs.sbx-mixin.yaml"), "kind: mixin\n").unwrap();
fs::write(agent.join("company-docs.yaml"), "driver: qdrant\n").unwrap();
let found = collect_mixins(&root, &[ScanMode::SubdirNamed, ScanMode::SubdirFlat]);
assert_eq!(found, vec![agent.join("company-docs.sbx-mixin.yaml")]);
let _ = fs::remove_dir_all(&root);
}
#[test]
fn agent_level_mixin_and_rag_sidecars_are_both_discovered() {
let root = unique_root("agent-both-shapes");
let agent = root.join("researcher");
fs::create_dir_all(&agent).unwrap();
fs::write(agent.join("sbx-mixin.yaml"), "kind: mixin\n").unwrap();
fs::write(agent.join("zebra.sbx-mixin.yaml"), "kind: mixin\n").unwrap();
fs::write(agent.join("alpha.sbx-mixin.yaml"), "kind: mixin\n").unwrap();
let found = collect_mixins(&root, &[ScanMode::SubdirNamed, ScanMode::SubdirFlat]);
assert_eq!(
file_names(&found),
vec![
"sbx-mixin.yaml",
"alpha.sbx-mixin.yaml",
"zebra.sbx-mixin.yaml"
]
);
let _ = fs::remove_dir_all(&root);
}
#[test]
fn subdir_flat_scan_ignores_a_directory_named_like_a_mixin() {
let root = unique_root("subdir-flat-decoy");
let agent = root.join("researcher");
fs::create_dir_all(agent.join("decoy.sbx-mixin.yaml")).unwrap();
assert!(collect_mixins(&root, &[ScanMode::SubdirFlat]).is_empty());
let _ = fs::remove_dir_all(&root); let _ = fs::remove_dir_all(&root);
} }
src/sandbox/mod.rs
+159 -54
View File
@@ -10,13 +10,17 @@ use std::path::{Path, PathBuf};
use std::process::{Command, Stdio}; use std::process::{Command, Stdio};
use which::which; use which::which;
pub(crate) mod mcp_credentials;
mod mixins; mod mixins;
pub(crate) use mcp_credentials::sandbox_secret_env_var;
use crate::config::AppConfig; use crate::config::AppConfig;
use crate::config::Config; use crate::config::Config;
use crate::config::VAULT_DATA_FILE_NAME; use crate::config::VAULT_DATA_FILE_NAME;
use crate::config::paths; use crate::config::paths;
use crate::rag::RagData; use crate::rag::RagData;
use crate::sandbox::mcp_credentials::MCP_MIXIN_NAME;
use crate::sandbox::mixins::DiscoveredMixin; use crate::sandbox::mixins::DiscoveredMixin;
use crate::utils::run_command_with_output; use crate::utils::run_command_with_output;
use crate::vault::SECRET_RE; use crate::vault::SECRET_RE;
@@ -51,17 +55,22 @@ pub fn launch(name: Option<String>, fresh: bool) -> Result<()> {
let registered = sbx_registered_services()?; let registered = sbx_registered_services()?;
inject_llm_secret(&config_content, &vault, &registered)?; inject_llm_secret(&config_content, &vault, &registered)?;
if !fresh { if !fresh {
inject_mcp_secrets(&vault, &registered)?;
inject_rag_secrets(&vault, &registered)?; inject_rag_secrets(&vault, &registered)?;
} }
let credentials_mixin = if fresh {
None
} else {
inject_mcp_secrets(&vault, &registered)?
};
let discovered = mixins::discover()?; let discovered = mixins::discover()?;
if sandbox_exists(&name)? { if sandbox_exists(&name)? {
info!("Re-attaching to existing sandbox '{name}'"); info!("Re-attaching to existing sandbox '{name}'");
} else { } else {
mixins::log_discovery(&discovered, false); mixins::log_discovery(&discovered, false);
create_sandbox(&name, &kit_path, &discovered)?; create_sandbox(&name, &kit_path, &discovered, credentials_mixin.as_deref())?;
if !fresh { if !fresh {
copy_host_files(&name)?; copy_host_files(&name)?;
} }
@@ -234,7 +243,7 @@ fn inject_llm_secret(
if registered.contains(&service) { if registered.contains(&service) {
eprintln!( eprintln!(
"Secret for '{service}' already registered with sbx. \ "Secret for '{service}' already registered with sbx. \
To update it, run: sbx secret set -g --force {service}" To update it, run: sbx secret set --force {service}"
); );
continue; continue;
} }
@@ -249,23 +258,14 @@ fn inject_llm_secret(
Ok(()) Ok(())
} }
fn find_secret_placeholder(value: &Value) -> Option<String> { /// Registers one sbx secret per distinct `{{placeholder}}` in the MCP config
match value { /// and returns the generated schema-v2 `coyote-mcp` mixin (network egress for
Value::String(s) => SECRET_RE /// every remote MCP server + credential declarations), or `None` when the MCP
.captures(s) /// config references no remote servers and no secrets.
.ok() fn inject_mcp_secrets(vault: &Vault, registered: &HashSet<String>) -> Result<Option<String>> {
.flatten()
.map(|caps| caps[1].to_string()),
Value::Object(map) => map.values().find_map(find_secret_placeholder),
Value::Array(arr) => arr.iter().find_map(find_secret_placeholder),
_ => None,
}
}
fn inject_mcp_secrets(vault: &Vault, registered: &HashSet<String>) -> Result<()> {
let mcp_path = paths::mcp_config_file(); let mcp_path = paths::mcp_config_file();
if !mcp_path.exists() { if !mcp_path.exists() {
return Ok(()); return Ok(None);
} }
let content = fs::read_to_string(&mcp_path) let content = fs::read_to_string(&mcp_path)
@@ -274,40 +274,46 @@ fn inject_mcp_secrets(vault: &Vault, registered: &HashSet<String>) -> Result<()>
.with_context(|| format!("Failed to parse {}", mcp_path.display()))?; .with_context(|| format!("Failed to parse {}", mcp_path.display()))?;
let Some(servers) = mcp.get("mcpServers").and_then(|v| v.as_object()) else { let Some(servers) = mcp.get("mcpServers").and_then(|v| v.as_object()) else {
return Ok(()); return Ok(None);
}; };
for (server_name, server_config) in servers { let credentials = mcp_credentials::collect_credentials(servers)?;
let Some(secret_name) = find_secret_placeholder(server_config) else { let allow_entries = mcp_credentials::collect_server_allow_entries(servers);
continue; if credentials.is_empty() && allow_entries.is_empty() {
}; return Ok(None);
}
if registered.contains(server_name.as_str()) { for credential in &credentials {
if registered.contains(credential.service_id.as_str()) {
eprintln!( eprintln!(
"Secret for '{server_name}' already registered with sbx. \ "Secret for '{}' already registered with sbx. \
To update it, run: sbx secret set -g --force {server_name}" To update it, run: sbx secret set --force {}",
credential.service_id, credential.service_id
); );
continue; continue;
} }
let secret_value = vault.get_secret(&secret_name, false).with_context(|| { let secret_value = vault
.get_secret(&credential.secret_name, false)
.with_context(|| {
format!( format!(
"Secret '{secret_name}' referenced by MCP server '{server_name}' not found \ "Secret '{}' referenced by MCP server(s) {} not found \
in vault. Add it with: coyote --add-secret {secret_name}" in vault. Add it with: coyote --add-secret {}",
credential.secret_name,
mcp_credentials::quoted_list(&credential.servers),
credential.secret_name
) )
})?; })?;
sbx_secret_set(server_name, &secret_value)?; sbx_secret_set(&credential.service_id, &secret_value)?;
} }
Ok(()) Ok(Some(mcp_credentials::render_mixin_yaml(
&credentials,
&allow_entries,
)?))
} }
/// Registers the API key of every attached RAG with the sbx proxy.
///
/// `launch()` has no notion of an active RAG — that is runtime state set by
/// `--rag` / `.rag` and never persisted — so every attached RAG is scanned
/// unconditionally, exactly as `inject_mcp_secrets` does for MCP servers.
fn inject_rag_secrets(vault: &Vault, registered: &HashSet<String>) -> Result<()> { fn inject_rag_secrets(vault: &Vault, registered: &HashSet<String>) -> Result<()> {
let rags_dir = paths::rags_dir(); let rags_dir = paths::rags_dir();
if !rags_dir.exists() { if !rags_dir.exists() {
@@ -319,7 +325,6 @@ fn inject_rag_secrets(vault: &Vault, registered: &HashSet<String>) -> Result<()>
continue; continue;
} }
let stem = match path.file_stem().and_then(|s| s.to_str()) { let stem = match path.file_stem().and_then(|s| s.to_str()) {
// Skip sidecars ("myrag.sbx-mixin.yaml" has stem "myrag.sbx-mixin").
Some(s) if !paths::is_rag_sidecar_name(s) => s.to_string(), Some(s) if !paths::is_rag_sidecar_name(s) => s.to_string(),
_ => continue, _ => continue,
}; };
@@ -335,19 +340,18 @@ fn inject_rag_secrets(vault: &Vault, registered: &HashSet<String>) -> Result<()>
let Some(placeholder) = data.driver_config.get("api_key") else { let Some(placeholder) = data.driver_config.get("api_key") else {
continue; continue;
}; };
if registered.contains(&stem) { let service_id = mcp_credentials::secret_service_id(&stem);
if service_id.is_empty() || registered.contains(&service_id) {
continue; continue;
} }
let secret_name = placeholder let secret_name = placeholder
.trim_start_matches("{{") .trim_start_matches("{{")
.trim_end_matches("}}") .trim_end_matches("}}")
.trim(); .trim();
// Degrade rather than abort: one stale RAG key must not block the whole
// sandbox launch. Queries to that RAG fail with a 401 at runtime, which
// is recoverable without a restart.
match vault.get_secret(secret_name, false) { match vault.get_secret(secret_name, false) {
Ok(secret_value) => { Ok(secret_value) => {
sbx_secret_set(&stem, &secret_value) sbx_secret_set(&service_id, &secret_value)
.context("Failed to register RAG secret with sbx")?; .context("Failed to register RAG secret with sbx")?;
} }
Err(e) => { Err(e) => {
@@ -359,6 +363,7 @@ fn inject_rag_secrets(vault: &Vault, registered: &HashSet<String>) -> Result<()>
} }
} }
} }
Ok(()) Ok(())
} }
@@ -366,7 +371,7 @@ fn provider_to_sbx_service(provider_type: &str, client_name: Option<&str>) -> St
match provider_type { match provider_type {
"claude" => "anthropic".to_string(), "claude" => "anthropic".to_string(),
"openai" => "openai".to_string(), "openai" => "openai".to_string(),
"gemini" | "vertexai" => "google".to_string(), "gemini" | "vertexai" => "gemini".to_string(),
"openai-compatible" => client_name.unwrap_or("openai-compatible").to_string(), "openai-compatible" => client_name.unwrap_or("openai-compatible").to_string(),
other => client_name.unwrap_or(other).to_string(), other => client_name.unwrap_or(other).to_string(),
} }
@@ -399,25 +404,29 @@ fn sbx_registered_services() -> Result<HashSet<String>> {
fn sbx_secret_set(service: &str, secret_value: &str) -> Result<()> { fn sbx_secret_set(service: &str, secret_value: &str) -> Result<()> {
let mut child = Command::new(SBX_BINARY) let mut child = Command::new(SBX_BINARY)
.args(["secret", "set", "-g", service]) .args(["secret", "set", service])
.stdin(Stdio::piped()) .stdin(Stdio::piped())
.stdout(Stdio::inherit()) .stdout(Stdio::inherit())
.stderr(Stdio::inherit()) .stderr(Stdio::inherit())
.spawn() .spawn()
.context("Failed to spawn `sbx secret set -g`")?; .context("Failed to spawn `sbx secret set`")?;
if let Some(mut stdin_handle) = child.stdin.take() { if let Some(mut stdin_handle) = child.stdin.take() {
stdin_handle stdin_handle
.write_all(secret_value.as_bytes()) .write_all(secret_value.as_bytes())
.context("Failed to write secret to `sbx secret set -g` stdin")?; .context("Failed to write secret to `sbx secret set` stdin")?;
} }
let status = child let status = child
.wait() .wait()
.context("Failed to wait for `sbx secret set -g`")?; .context("Failed to wait for `sbx secret set`")?;
if !status.success() { if !status.success() {
bail!("`sbx secret set -g {service}` exited with {status}"); eprintln!(
"Warning: failed to register sbx secret '{service}' \
(`sbx secret set {service}` exited with {status}). \
Set it manually with: echo '<value>' | sbx secret set {service}"
);
} }
Ok(()) Ok(())
@@ -436,9 +445,17 @@ fn sandbox_exists(name: &str) -> Result<bool> {
.any(|line| line.split_whitespace().next() == Some(name))) .any(|line| line.split_whitespace().next() == Some(name)))
} }
fn create_sandbox(name: &str, kit_path: &Path, mixins: &[DiscoveredMixin]) -> Result<()> { fn create_sandbox(
name: &str,
kit_path: &Path,
mixins: &[DiscoveredMixin],
credentials_mixin: Option<&str>,
) -> Result<()> {
info!("Creating sandbox '{name}'"); info!("Creating sandbox '{name}'");
let args = build_create_args(name, kit_path, mixins)?; let credentials_kit = credentials_mixin
.map(|yaml| mixins::wrap_mixin_bytes_as_kit(yaml.as_bytes(), MCP_MIXIN_NAME))
.transpose()?;
let args = build_create_args(name, kit_path, mixins, credentials_kit.as_deref())?;
debug!("sbx {}", args.join(" ")); debug!("sbx {}", args.join(" "));
let status = Command::new(SBX_BINARY) let status = Command::new(SBX_BINARY)
.args(&args) .args(&args)
@@ -459,6 +476,7 @@ fn build_create_args(
name: &str, name: &str,
kit_path: &Path, kit_path: &Path,
mixins: &[DiscoveredMixin], mixins: &[DiscoveredMixin],
credentials_kit: Option<&Path>,
) -> Result<Vec<String>> { ) -> Result<Vec<String>> {
let kit_str = kit_path let kit_str = kit_path
.to_str() .to_str()
@@ -482,6 +500,15 @@ fn build_create_args(
args.push(mixin_str); args.push(mixin_str);
} }
if let Some(kit) = credentials_kit {
let cred_str = kit
.to_str()
.ok_or_else(|| anyhow!("Credentials kit path is not valid UTF-8: {}", kit.display()))?
.to_string();
args.push("--kit".to_string());
args.push(cred_str);
}
args.push(SANDBOX_AGENT.to_string()); args.push(SANDBOX_AGENT.to_string());
args.push(".".to_string()); args.push(".".to_string());
@@ -619,6 +646,7 @@ fn chown_agent_recursive(sandbox: &str, path: &str) -> Result<()> {
#[cfg(test)] #[cfg(test)]
mod tests { mod tests {
use super::*; use super::*;
use std::time::{SystemTime, UNIX_EPOCH};
#[test] #[test]
fn sanitize_name_lowercases() { fn sanitize_name_lowercases() {
@@ -687,8 +715,8 @@ mod tests {
#[test] #[test]
fn build_create_args_emits_base_kit_before_mixins() { fn build_create_args_emits_base_kit_before_mixins() {
let kit = PathBuf::from("/cache/sbx-kit"); let kit = PathBuf::from("/cache/sbx-kit");
let unique = std::time::SystemTime::now() let unique = SystemTime::now()
.duration_since(std::time::UNIX_EPOCH) .duration_since(UNIX_EPOCH)
.unwrap() .unwrap()
.as_nanos(); .as_nanos();
let dir_a = env::temp_dir().join(format!("coyote-mixin-a-{unique}")); let dir_a = env::temp_dir().join(format!("coyote-mixin-a-{unique}"));
@@ -711,7 +739,7 @@ mod tests {
}, },
]; ];
let args = build_create_args("my-box", &kit, &mixins).unwrap(); let args = build_create_args("my-box", &kit, &mixins, None).unwrap();
assert_eq!( assert_eq!(
args, args,
@@ -737,7 +765,9 @@ mod tests {
#[test] #[test]
fn build_create_args_with_no_mixins_omits_mixin_kits() { fn build_create_args_with_no_mixins_omits_mixin_kits() {
let kit = PathBuf::from("/cache/sbx-kit"); let kit = PathBuf::from("/cache/sbx-kit");
let args = build_create_args("box", &kit, &[]).unwrap();
let args = build_create_args("box", &kit, &[], None).unwrap();
assert_eq!( assert_eq!(
args, args,
vec![ vec![
@@ -751,4 +781,79 @@ mod tests {
] ]
); );
} }
#[test]
fn build_create_args_appends_credentials_kit_after_mixins() {
let kit = PathBuf::from("/cache/sbx-kit");
let credentials_kit = PathBuf::from("/cache/sbx-mixin-kits/abc123");
let args = build_create_args("box", &kit, &[], Some(&credentials_kit)).unwrap();
assert_eq!(
args,
vec![
"create".to_string(),
"--name".to_string(),
"box".to_string(),
"--kit".to_string(),
"/cache/sbx-kit".to_string(),
"--kit".to_string(),
"/cache/sbx-mixin-kits/abc123".to_string(),
"coyote".to_string(),
".".to_string(),
]
);
}
#[test]
fn build_create_args_orders_base_kit_then_mixins_then_credentials_kit() {
let kit = PathBuf::from("/cache/sbx-kit");
let credentials_kit = PathBuf::from("/cache/sbx-mixin-kits/abc123");
let unique = SystemTime::now()
.duration_since(UNIX_EPOCH)
.unwrap()
.as_nanos();
let dir = env::temp_dir().join(format!("coyote-mixin-cred-{unique}"));
fs::create_dir_all(&dir).unwrap();
let mixins = vec![DiscoveredMixin {
path: dir.clone(),
label: "user".into(),
install_count: 0,
domain_count: 0,
}];
let args = build_create_args("box", &kit, &mixins, Some(&credentials_kit)).unwrap();
assert_eq!(
args,
vec![
"create".to_string(),
"--name".to_string(),
"box".to_string(),
"--kit".to_string(),
"/cache/sbx-kit".to_string(),
"--kit".to_string(),
dir.display().to_string(),
"--kit".to_string(),
"/cache/sbx-mixin-kits/abc123".to_string(),
"coyote".to_string(),
".".to_string(),
]
);
let _ = fs::remove_dir_all(&dir);
}
#[test]
fn provider_to_sbx_service_maps_gemini_family_to_gemini() {
assert_eq!(provider_to_sbx_service("gemini", None), "gemini");
assert_eq!(provider_to_sbx_service("vertexai", None), "gemini");
}
#[test]
fn provider_to_sbx_service_maps_known_providers() {
assert_eq!(provider_to_sbx_service("claude", None), "anthropic");
assert_eq!(provider_to_sbx_service("openai", None), "openai");
}
} }
src/vault/utils.rs
+27 -2
View File
@@ -1,5 +1,5 @@
use crate::config::ensure_parent_exists; use crate::config::ensure_parent_exists;
use crate::sandbox::SANDBOX_ENV_FLAG; use crate::sandbox::{SANDBOX_ENV_FLAG, sandbox_secret_env_var};
use crate::vault::{SECRET_RE, Vault}; use crate::vault::{SECRET_RE, Vault};
use anyhow::Result; use anyhow::Result;
use anyhow::anyhow; use anyhow::anyhow;
@@ -358,7 +358,32 @@ fn required_cli_preflight(label: &str, cli: &str, install_url: &str) {
pub fn interpolate_secrets(content: &str, vault: &Vault) -> Result<(String, Vec<String>)> { pub fn interpolate_secrets(content: &str, vault: &Vault) -> Result<(String, Vec<String>)> {
if env::var_os(SANDBOX_ENV_FLAG).is_some() { if env::var_os(SANDBOX_ENV_FLAG).is_some() {
return Ok((content.to_string(), vec![])); let (parsed, missing) = interpolate_secrets_with(content, None, |name| {
env::var(sandbox_secret_env_var(name)).map_err(|_| {
anyhow!(SecretError::NotFound {
key: name.to_string(),
provider: "sandbox environment",
})
})
})?;
if !missing.is_empty() {
let mut env_vars: Vec<String> = missing
.iter()
.map(|name| sandbox_secret_env_var(name))
.collect();
env_vars.sort();
env_vars.dedup();
eprintln!(
"Config references secrets that are not available inside this sandbox \
(expected env vars: {}). Sandbox secrets are provisioned at creation \
from the host; add the missing secrets on the host, then re-create \
the sandbox.",
env_vars.join(", ")
);
}
return Ok((parsed, missing));
} }
interpolate_secrets_with(content, vault.auth_hint(), |name| { interpolate_secrets_with(content, vault.auth_hint(), |name| {
vault.get_secret(name, false) vault.get_secret(name, false)