The user-scope MCP config historically lived at
<config-dir>/functions/mcp.json, a leftover from when MCP support was
part of the llm-functions tooling. It now resolves through a single
choke point with these semantics:
- Preferred location: <config-dir>/mcp.json (created there on first run)
- Historical <config-dir>/functions/mcp.json still honored when the
preferred file does not exist, so existing installs are unchanged
- If both exist, the preferred location wins
--info/.info now reports the resolved location as mcp_config_file, and
the --scope help text plus config.agent.example.yaml reference the new
default. This also removes the asymmetry with the workspace scope,
which already used .coyote/mcp.json directly.
The supervisor registry went kind-generic (TaskHandle::Agent | Job)
earlier in this branch, but the module holding the agent__* handlers
and two model-facing error strings still carried the old name:
- src/function/supervisor.rs -> src/function/agents.rs (it contains
only agent__* tool handlers, pairing with function/jobs.rs; the
kind-generic src/supervisor/ registry keeps its name)
- 'Supervisor tool failed' -> 'Agent tool failed'
- 'Unknown supervisor action' -> 'Unknown agent action'
Graph LLM nodes now own the jobs they start, on every exit path. A new
node_job_scope on RequestContext records job ids started while a node
runs: the turn-end guardrail nags only about the node's own jobs
(parallel branches no longer see each other's), and the node executor
reaps — cancels and deregisters — anything left registered when the
node exits, including error, timeout, and retry-exhaustion paths.
Cross-node job handoff is no longer possible; a crashed node takes its
in-flight jobs with it.
With inheritance gone, job__* declarations are gated on capability:
the family is only declared when at least one declared tool would pass
job__start's whitelist (shared predicate: is_backgroundable_tool). One
carve-out — while a context still owns registered jobs (job started,
tool disabled mid-session), the lifecycle verbs stay declared so a
running job can never become unreachable; job__start alone disappears.
A graph node with tools: [] now sees no job__* tools at all.
Prompt instructions, tool declarations, and graph.example.yaml updated
to the node-local semantics; +7 tests, 8 visibility pins rewritten.
Since the supervisor registry became kind-generic (agents AND jobs),
'supervisor' naming on the agent__* tool plumbing was misleading:
job__* handlers operate on the same supervisor. Rename
SUPERVISOR_FUNCTION_PREFIX -> AGENT_FUNCTION_PREFIX,
supervisor_function_declarations -> agent_function_declarations,
handle_supervisor_tool -> handle_agent_tool. No behavior change.
- add NotificationQueue/SystemNotification: every context owns a fresh
queue (children never inherit the parent's, avoiding first-drainer-wins
races between transcripts)
- job tasks push job_completed/job_failed events on completion, failure,
and timeout; a panic skips the push and is surfaced by the guardrail's
finished-handle enumeration and collect's JoinError mapping instead
- events for jobs already collected or cancelled are dropped at drain time
by filtering against live supervisor registration
- replace inject_escalation_notification with single-pass
merge_system_channel: pending_escalations (root-only) ordered before
system_notifications (any depth) on the last tool result of a batch;
byte-identical output when notifications are empty, proven by the
unmodified pre-merger characterization tests
Reject fast built-in file tools (fs_* / ast_grep) in job__start per the
backgroundable-tools whitelist; clean up env-snapshot temp files on every
exit of run_process_job via a drop guard; bound the output-pump awaits and
abort them on the failure path; treat signal death (no exit code) as a
failure with a teaching message; bound job__collect's post-drain join with
a SIGKILL escalation so a TERM-ignoring process cannot hang collect after
a Ctrl-C teardown; document the unguarded SIGTERM pid-reuse window; give
the injected Background Jobs prompt section a fresh line on both sides;
extract the MCP server name with strip_prefix instead of replace.
Capacity-0 audit for jobs-disabled contexts: REPL displays have no
supervisor consumers (only Ctrl-C/exit cancel_recursive at
repl/mod.rs:460,473, kind-agnostic); session save/load does not persist
supervisor state (src/config/session.rs has no supervisor references) --
nothing to test for either.
Detached tokio::process runner with a frozen JobEnvSnapshot (env-derived
bin dirs, vault-interpolated agent envs, COYOTE_TOOL_TIMEOUT resolved at
start), process_group(0) with pgid-guarded SIGTERM/SIGKILL escalation,
capture-only ring-buffer telemetry, and LLM_OUTPUT read after wait().
MCP jobs snapshot a single-entry McpRuntime holding only the validated
server and render through the same free fn as the foreground path.
job__start enforces its gates synchronously before any spawn:
jobs_enabled, the backgroundable whitelist with directionality teaching
errors, the per-request declared-names stash captured in
before_chat_completion, then capacity (lazy supervisor get-or-init in
plain sessions). job__check/list read the shared JobState cell without
consuming; job__collect blocks with the escalation early-out and applies
a tail-biased char-boundary cap plus optional tail_lines; job__cancel
kills the group with a 5s grace.
Job declarations are injected iff jobs are enabled at agent init, the
plain-session function-init sites, and the exit_agent rebuild; job__ is
carved out of enabled_tools filtering and excluded from
concrete_tool_names so REPL toggles cannot grant or revoke it.
Implements T1 of plans/background-jobs-design.md (§6, R7/R8/R9):
- Supervisor.handles is now HashMap<String, TaskHandle> where
TaskHandle = Agent(AgentHandle) | Job(JobHandle); agent-facing
accessors (active_count, effective_active_count, is_finished, take,
inbox, abort_signal_for, list_agents) match only Agent variants,
preserving all existing external behavior byte-for-byte.
- New JobHandle/JobState/JobStatus/JobResult types with pgid-guarded
process-group kill discipline: Drop and cancel_all/cancel_recursive
kill the group only while state.pgid is still set (pid-reuse guard),
via libc::killpg on unix and JoinHandle::abort elsewhere.
- Per-kind job capacity: Supervisor carries max_concurrent_jobs
(builder-set, default 0); job registration rejects at capacity.
- Cross-kind teaching errors at the four agent-lookup miss sites
(agent__check/collect/cancel/send_message) when the id is a
registered job or job_-prefixed; genuinely-unknown ids keep their
existing messages.
- Supervisor init condition is now can_spawn_agents || jobs_enabled in
use_agent and both child-agent spawn paths, with agent capacity 0 in
jobs-only contexts; use_agent cancels the old supervisor recursively
before replacing it.
- max_concurrent_jobs config plumbing: global Config field, AgentConfig
override + accessor, all four AppConfig touch points including the
COYOTE_MAX_CONCURRENT_JOBS env override; shared
effective_max_concurrent_jobs/jobs_enabled predicates
(agent override -> global -> default 5; 0 disables).
- Stage dependency-free RingBuf (64 KiB default) in src/function/jobs.rs
for the upcoming job output pump.
- New sanctioned dependency: libc 0.2 under cfg(unix).
Completes the .prompt/.temp-role split: --prompt set an ad-hoc system
role, which is what .temp-role now means everywhere. The --prompt name
is left unbound so a future one-shot MCP prompt flag can take it with
properly designed non-interactive semantics. use_prompt follows the
rename as use_temp_role.
BREAKING CHANGE: invocations using --prompt <text> must switch to
--temp-role <text>; clap rejects the old flag loudly.
MCP prompts are live, server-owned catalog entries, not managed assets;
list_prompt_assets/prompt_asset_rows implied otherwise and are now
list_mcp_prompts/mcp_prompt_rows. The .list completer was also missing
the prompts kind that the usage string and unknown-kind error advertise.
The unified install parser accepts flags in any position, so the
first-argument completion list now offers all four flags instead of
only --git-host and --help.
Implements plans/mcp-resources-prompts-design.md §5.1/§5.4 (T6):
- .prompt <server> <name> [key=value ...] fetches an MCP prompt and
submits the result as chat input via Input::from_str + ask(), never
through REPL line parsing; GetPromptResult messages are flattened
into one user-role block with unconditional [user]/[assistant] labels
- missing required prompt arguments are collected interactively
- .list prompts renders server/name/description/args via the unified
catalog (CatalogItem gains an arguments field), degrading per server
- staged live tab-completion: enabled+running+prompts-capable servers
(no RPC), then live prompt names, then key= argument suggestions with
(required) markers; 2s timeout per RPC, all errors degrade to silent
empty suggestions, ctx read guard dropped before blocking
- the enabled-server alias expansion is factored into a shared helper
used by both tool-scope rebuild and completion
- BREAKING: the former .prompt <text> temp-role builtin is renamed to
.temp-role <text> (behavior preserved); .prompt now belongs to MCP
prompts, and a user macro named prompt or temp-role is shadowed
Per-server McpServerFeatures (tools fail-open, resources/prompts
fail-closed) now drive which meta-functions are declared, with
gated_meta_function_prefixes as the single gating seam; read/prompt
declarations land together with their handlers. The server-enablement
sentinel keys on the always-emitted search name so resources-only
servers survive role filtering.
Implements plans/mcp-resources-prompts-design.md §4.4/D7 (T5).
Implements T1 of plans/mcp-resources-prompts-design.md (§4.1, §4.6):
- Replace list_tools(None) with cursor-following list_all_tools() at the
three call sites (start_server catalog build, catalog_items, describe)
so paginating servers no longer silently lose tools past page one.
- Add MCP_READ/MCP_PROMPT prefix constants (declared nowhere yet; wired
in T4/T7) plus centralized helpers MCP_META_FUNCTION_PREFIXES,
is_mcp_meta_function, and mcp_meta_function_names.
- Mechanically replace every hand-rolled 3-prefix starts_with triple
(partition in eval_tool_calls, 3 exclusion triples in
select_enabled_functions, 3 inclusion triples + per-server name
construction in select_enabled_mcp_servers) with the helpers,
preserving the existing lax starts_with matching semantics and the
mcp_invoke_* enablement sentinel (sentinel moves to search in T5).
- Behavior-neutral: dispatch chains keep their 3 arms, emission stays
at exactly 3 meta-functions per server, existing tests unmodified.
- Add unit tests: helper classification, prefix-soundness property,
lax-matching pin, ordered candidate-name construction.
Macro invocations (.name and .macro name) accept leading name=value
assignments before positional args: assignments set declared variables
directly so earlier variables can keep their defaults, remaining
positionals fill unassigned variables in declaration order, and the
free text after -- is never scanned for assignments. Identifier-shaped
keys that match no declared variable error with the declared list to
catch typos; non-identifier tokens containing = stay positional.
MacroVariable gains an optional description field, and tab completion
after a macro name offers name= candidates showing each variable's
description and default until the assignment prefix ends.
.install --help and .uninstall --help print a usage guide covering the
owner/repo shorthand, --git-host, --filter, --force, ref pinning, and
the bundle lifecycle; both usage error lines now point at --help. Tab
completion offers --help for both commands and --git-host on the first
.install argument, and the unified install parser accepts flags in any
argument position so completed flags work wherever they are inserted.
The empty .list bundles message now shows the REPL install form
alongside the CLI one.
The path-escape guard that uninstall applies to recorded paths now also
covers update's obsolete-file deletion through a shared check, so a
tampered store cannot turn either delete site into an arbitrary file
removal. Updates gain a working non-interactive path: --yes now applies
to --update-bundle (locally modified files, obsolete files, and modified
mcp entries are all kept; everything else refreshes), owned mcp entries
whose recorded hash still matches the local entry take the remote side
without prompting, and the non-TTY conflict bails name the flag that
actually works per surface. An update records its new commit and version
only after files and mcp entries land, so an aborted update cannot claim
content it never wrote. The store gains a version field and rejects
stores from newer builds, the corrupt-store error no longer advises the
removal that would forfeit ownership tracking, and duplicate records
tracking one source abort a rename instead of overwriting a record.
Reinstalling from a source URL reclassifies owned unmodified files as
silent refreshes just like updates. git runs with GIT_TERMINAL_PROMPT=0
and a null stdin so private or mistyped URLs fail instead of hanging.
File comparison fills buffers fully before comparing, deleting an
obsolete file prunes emptied directories, mcp.json backfill uses the
fsynced atomic writer, --list-bundles no longer triggers builtin
backfill, bundle-name completion logs store errors instead of swallowing
them and offers --yes, and REPL .uninstall rejects unknown flags.
The user-origin marker on replaced mcp.json entries is now sticky:
re-records and cross-bundle transfers only upgrade replaced to
transferred when the prior record proves bundle origin, so updating a
bundle can no longer make uninstall delete a key the user had before the
bundle replaced it. Canonical source URLs lowercase only the host, since
self-hosted forges treat repository paths as case-sensitive and
collapsing distinct repos misdirects updates and uninstalls. git clone
invocations pass '--' before the URL so a crafted source cannot be
parsed as a git flag. Lifecycle flags (--install, --install-builtins,
--update-bundle, --uninstall) and their companions now conflict
explicitly instead of first-match dispatch silently dropping actions.
--install-from returns as a hidden tombstone that errors with the
replacement instead of feeding the flag to the LLM as prompt text.
--list-bundles dispatches before config load so a pure read no longer
boots MCP servers. write_file_atomic fsyncs before the rename so a crash
cannot persist a truncated store. REPL: .uninstall accepts --yes,
.install rejects trailing tokens after a category, and .install remote
gets a migration hint. Plus polish: host validation rejects '#' and '?',
renamed_to no longer serializes null, derived names get a debug assert
against the validator, completions share DEFAULT_GIT_HOST, README
mentions skills.
--install someuser/repo expands to https://github.com/someuser/repo;
--git-host overrides the default host and forces source interpretation
even when the value matches an installed bundle name. Two or more path
segments are accepted so nested GitLab-style groups work, and #ref
pinning applies to shorthand values. --uninstall resolves owner/repo
against recorded sources: a single match uninstalls, multiple matches
prompt an interactive selection showing each bundle's source, and
non-interactive runs bail instead of guessing.
--install <git-url|name> is the single entry point for remote installs
and updates; the unified .install dispatch likewise replaces
.install remote. Flag completion for .install now applies to the
unified form.
--install now takes a git URL or an installed bundle name: categories
are redirected to the new --install-builtins, installed names become
implicit updates, and source-shaped values install remotely. The old
--install-from keeps its exact behavior as a hidden deprecated alias.
The REPL's .install gains the same unified dispatch while keeping
.install <category> and .install remote <url> back-compat.
Long agent/skill descriptions wrapped badly in the bullet-list format.
Extract a shared asset_table helper (UTF8_FULL + dynamic arrangement,
same style as the markdown renderer and .list macros) and use it for
the agents, skills, and macros listings. The skills loaded marker
keeps its color; comfy-table's custom_styling feature accounts for
ANSI sequences in column widths.
Hand-rolled {:<24} padding broke alignment as soon as a macro name
exceeded the column width. Reuse the comfy-table UTF8_FULL preset with
dynamic content arrangement, matching the markdown renderer's tables.
Mirrors --no-workspace-mcp exactly: a CLI-only flag backed by an
AppConfig field (default false) that disables .coyote/macros in both
the resolved macro policy and Macro::load's workspace-then-global
preference, so the two always agree (custom-commands design §5).
Make the macro isolated field live (design §3, §9 step 5):
isolated: false now runs the interpolated steps via run_repl_command
on the live RequestContext — session-recorded, conversation-visible,
with mutating steps persisting by design — while isolated: true keeps
the forked execution byte-for-byte unchanged.
- Add macro_non_isolated companion field beside macro_flag; both
fork-propagation sites mirror it verbatim.
- RAII MacroModeGuard wraps the whole &mut RequestContext (DerefMut
passthrough) and restores flag+mode on every exit path, including a
failing step; steps remain fail-fast.
- Reject nested macro invocation when the current mode is non-isolated
("nested macros not allowed in non-isolated mode"); an isolated
macro's step may still run a non-isolated macro inline on its fork.
- use_agent now suppresses the agent's default session only for
isolated macros; a non-isolated .agent step engages it as if typed.
Implements the invocation and management surfaces from
plans/custom-commands-design.md §4 and §6:
- Top-level dispatch: an enabled macro <name> now runs as ".<name> [args]"
from the command catch-all; runtime-disabled macros point at
".macro enable <name>", locked macros name the owning config, and
unknown commands keep the existing error verbatim
- .macro enable|disable <name>: runtime toggles over the in-memory
global-level enabled_macros list (disable with no list materializes
all-active-minus-name); toggles error when a role/agent/session
allowlist owns the field
- .set enabled_macros <csv|null> with workspace-then-global existence
validation; .set key completion gains enabled_macros and the
previously missing enabled_skills
- Dynamic completion: enabled macros (with descriptions) join built-ins
on ".<TAB>" without touching the static command registry;
".macro <TAB>" lists invocable macros (incl. built-in-shadowed ones)
plus the enable/disable subcommands; second-arg completion offers
toggle-eligible names
- .list macros: enriched table (name, source, isolated, state,
description) covering every resolver state incl. missing and
shadowed rows; .help gains a custom-commands section
- Session info/render and sysinfo display enabled_macros; Macro::load
resolves workspace-then-global; enable/disable rejected as macro
names in the creator
Distinguish why an OAuth MCP server was not started: never authenticated
(no stored credentials), stored token expired and refresh failed, or the
server rejected a token that looked valid. McpTokenStatus replaces the
Option<String> return of load_or_refresh_mcp_token, and McpAuthRequired
carries the reason across the error boundary via anyhow context.
Adds QdrantProvider as a read-only driver for pre-existing remote Qdrant
collections, an interactive '.rag attach' wizard, and the sandbox credential
and domain-whitelisting wiring that lets an attached RAG work inside a sandbox.
Attach-only by design: rebuild_indexes bails for both the attached and the
unattached case rather than silently succeeding. Coyote never writes to Qdrant
in this change.
Vectors are never hydrated back from Qdrant. Cosine collections L2-normalize
stored vectors on write, so reading them back returns unit-length copies of the
originals; the YAML vector copy is authoritative and the serialization guard
stays scoped to the duckdb driver alone.
Collections keyed by string or UUID point IDs are rejected at attach time. The
read path parses point ids as u64 inside a filter_map, so such a collection
would otherwise yield zero results with no error.
The three response-shape parsers are pure functions over an already-parsed JSON
body, unit-tested against captured fixtures, with the async wrappers delegating
to them rather than duplicating the logic.
'.rag' now splits its first argument, so '.rag attach <name>' no longer tries to
load a RAG literally named 'attach <name>'.
Phase 3 of the RAG driver abstraction. Adds a `DuckDbProvider` that keeps
vectors and document content in a `.duckdb` sidecar next to the existing
YAML metadata, selected by the `driver: duckdb` field.
- `src/rag/providers/duckdb.rs` (new): vector search via the vss extension
and keyword search via fts, an all-or-nothing hydration path (a partial
read is an error, never a shorter map), and an anti-wipe guard that
refuses the destructive `CREATE OR REPLACE TABLE` when `data.vectors` is
empty while `data.files` is not and the store still holds rows.
- `src/rag/mod.rs`: `sync_documents` now refreshes `bm25`/`node_to_docs`
BEFORE the fallible `provider.rebuild_indexes`. `self.data` is already
mutated by that point, so propagating a provider error afterwards would
leave the derived in-memory state describing the previous corpus while
`data` describes the new one. Both rebuilds are pure functions of
`self.data` and cannot fail, so running them first is always safe.
- `src/config/paths.rs`: sidecar path helpers.
- `src/rag/providers/mod.rs`, `src/config/agent.rs`: driver dispatch and
RAG cache keying.
Also keeps `RequestContext::rag_key` in lockstep with `rag` at the two
sites that were still missing it, so that a cache insert and its matching
invalidate are structurally incapable of disagreeing:
- `use_agent` assigned `self.rag` from the agent but never set `rag_key`.
This one was live. Agent RAGs are inserted under `RagKey::Agent(<name>)`,
so with `rag_key == None` the invalidation guards in `rebuild_rag` and
`edit_rag_docs` matched nothing and `.rebuild rag` left the stale cache
entry in place. Worse, a preceding `.rag <name>` left a stale
`Named(<name>)` key attached to the agent's RAG, pointing the
invalidation at an unrelated RAG's cache entry. Now mirrors the insert
key exactly, yielding `None` when the agent has no RAG.
- `exit_agent` cleared `self.rag` but left `rag_key` behind. Latent rather
than live, since `rebuild_rag`/`edit_rag_docs` both bail on
`rag.is_none()` before reaching the invalidate guards, but the guards
that make it unobservable are not the kind of thing to depend on.
Covered by `use_agent_does_not_carry_stale_rag_key`, and by a new
assertion in `exit_agent_clears_all_agent_state`.
Phase 1 of the RAG driver abstraction (design doc sections 5.1-5.5a).
Data model:
- Add `driver: String` (serde default "yaml" via RagData::default_driver) and
`attached: bool` as the first two fields of RagData, so driver metadata sits
at the top of each RAG YAML. Old files without them load unchanged.
- Add `#[serde(default)]` to the non-Option fields so a minimal attached-RAG
YAML deserializes, and add `skip_serializing_if` to `vectors` so an empty
map renders no `vectors:` key.
- Add a hand-written `impl Default for RagData` delegating to `RagData::new()`.
It is deliberately not derived: a derived impl yields `driver: ""`, which is
not a valid driver string.
Validation (the price of the new serde defaults):
- Add `RagData::validate()`, called from `Rag::load()` after deserialization.
It enforces the (driver, attached) matrix and, critically, numeric floors
that the new defaults would otherwise mask: `top_k >= 1` unconditionally
(a 0 makes every query return nothing, silently), and `chunk_size >= 1` plus
`chunk_overlap < chunk_size` when not attached (a 0 chunk_size is a real
divide-by-zero panic while sizing embedding batches).
- Reject `.set rag_top_k 0` at the setter, before the set/update fork. Without
this, the new load-time floor turns one keystroke into an unloadable RAG:
the setter saves immediately and no dot-command can reach the file again.
Rebuild actually re-embeds now:
- `.rebuild rag` and `--rebuild-rag` previously re-scanned paths and re-embedded
nothing, because the content-hash skip fired regardless of the refresh flag.
Extract that decision into a module-level `find_hash_skip()` free function and
thread a `force_reingest` flag through `sync_documents()` and
`refresh_document_paths()`, set true only from `rebuild_rag()`. `.edit rag-docs`
stays incremental. Re-embedding costs time and API spend, so `rebuild_rag()`
now prints a one-line file-count warning first (no prompt: the path is
reachable from a non-interactive CLI flag).
Attached-RAG guards:
- Block `.rebuild rag` / `--rebuild-rag` and `.edit rag-docs` on attached RAGs,
which Coyote did not index and whose source documents it does not own.
- Add `Rag::driver()`, `Rag::is_attached()` and `Rag::file_count()`, and surface
driver/attached through `Rag::export()` so `.info rag` shows them.
Adds 12 unit tests (1299 -> 1311), including the two gate tests pinning that a
forced re-ingest does not hash-skip while an ordinary refresh still does.
- Hang fix: 120s timeout on compression LLM call + raw_stream
channel-close handling (None => break instead of spinning)
- Supervisor effective active count: stop counting finished
JoinHandles as occupying capacity slots
- Universal tool result size cap: truncate_if_needed() on
ToolResult, applied in eval_tool_calls() after escalation block;
configurable via max_tool_result_chars in AppConfig/AgentConfig
- Windowed compression: compression_keep_last config param keeps
the N most recent messages visible after compression
- Fix pre-existing flaky test: add #[serial] to
handle_list_available_unrestricted_when_no_whitelist so it does
not race with TestConfigDirGuard-based tests that temporarily
populate the agents data dir
Dark-Alex-17