Graph agents could only inherit the app-wide job budget; the agent-level
header in graph.yaml now accepts max_concurrent_jobs alongside
model/temperature, flowing through AgentConfig::from_graph into the
run-wide supervisor. Deliberately graph-wide, not per-node: jobs outlive
the node that started them.
Reject fast built-in file tools (fs_* / ast_grep) in job__start per the
backgroundable-tools whitelist; clean up env-snapshot temp files on every
exit of run_process_job via a drop guard; bound the output-pump awaits and
abort them on the failure path; treat signal death (no exit code) as a
failure with a teaching message; bound job__collect's post-drain join with
a SIGKILL escalation so a TERM-ignoring process cannot hang collect after
a Ctrl-C teardown; document the unguarded SIGTERM pid-reuse window; give
the injected Background Jobs prompt section a fresh line on both sides;
extract the MCP server name with strip_prefix instead of replace.
Capacity-0 audit for jobs-disabled contexts: REPL displays have no
supervisor consumers (only Ctrl-C/exit cancel_recursive at
repl/mod.rs:460,473, kind-agnostic); session save/load does not persist
supervisor state (src/config/session.rs has no supervisor references) --
nothing to test for either.
Agents whose function pool includes job__* declarations get a Background
Jobs section teaching start/check/collect/cancel discipline and the
snapshot/no-persistence semantics. Presence of the declarations doubles
as the jobs_enabled predicate, so a context with function calling off or
max_concurrent_jobs 0 sees no job prompt text.
Detached tokio::process runner with a frozen JobEnvSnapshot (env-derived
bin dirs, vault-interpolated agent envs, COYOTE_TOOL_TIMEOUT resolved at
start), process_group(0) with pgid-guarded SIGTERM/SIGKILL escalation,
capture-only ring-buffer telemetry, and LLM_OUTPUT read after wait().
MCP jobs snapshot a single-entry McpRuntime holding only the validated
server and render through the same free fn as the foreground path.
job__start enforces its gates synchronously before any spawn:
jobs_enabled, the backgroundable whitelist with directionality teaching
errors, the per-request declared-names stash captured in
before_chat_completion, then capacity (lazy supervisor get-or-init in
plain sessions). job__check/list read the shared JobState cell without
consuming; job__collect blocks with the escalation early-out and applies
a tail-biased char-boundary cap plus optional tail_lines; job__cancel
kills the group with a 5s grace.
Job declarations are injected iff jobs are enabled at agent init, the
plain-session function-init sites, and the exit_agent rebuild; job__ is
carved out of enabled_tools filtering and excluded from
concrete_tool_names so REPL toggles cannot grant or revoke it.
Implements T1 of plans/background-jobs-design.md (§6, R7/R8/R9):
- Supervisor.handles is now HashMap<String, TaskHandle> where
TaskHandle = Agent(AgentHandle) | Job(JobHandle); agent-facing
accessors (active_count, effective_active_count, is_finished, take,
inbox, abort_signal_for, list_agents) match only Agent variants,
preserving all existing external behavior byte-for-byte.
- New JobHandle/JobState/JobStatus/JobResult types with pgid-guarded
process-group kill discipline: Drop and cancel_all/cancel_recursive
kill the group only while state.pgid is still set (pid-reuse guard),
via libc::killpg on unix and JoinHandle::abort elsewhere.
- Per-kind job capacity: Supervisor carries max_concurrent_jobs
(builder-set, default 0); job registration rejects at capacity.
- Cross-kind teaching errors at the four agent-lookup miss sites
(agent__check/collect/cancel/send_message) when the id is a
registered job or job_-prefixed; genuinely-unknown ids keep their
existing messages.
- Supervisor init condition is now can_spawn_agents || jobs_enabled in
use_agent and both child-agent spawn paths, with agent capacity 0 in
jobs-only contexts; use_agent cancels the old supervisor recursively
before replacing it.
- max_concurrent_jobs config plumbing: global Config field, AgentConfig
override + accessor, all four AppConfig touch points including the
COYOTE_MAX_CONCURRENT_JOBS env override; shared
effective_max_concurrent_jobs/jobs_enabled predicates
(agent override -> global -> default 5; 0 disables).
- Stage dependency-free RingBuf (64 KiB default) in src/function/jobs.rs
for the upcoming job output pump.
- New sanctioned dependency: libc 0.2 under cfg(unix).
Per-server McpServerFeatures (tools fail-open, resources/prompts
fail-closed) now drive which meta-functions are declared, with
gated_meta_function_prefixes as the single gating seam; read/prompt
declarations land together with their handlers. The server-enablement
sentinel keys on the always-emitted search name so resources-only
servers survive role filtering.
Implements plans/mcp-resources-prompts-design.md §4.4/D7 (T5).
Adds src/config/macro_policy.rs: MacroPolicy::effective computes the
visible macro set on demand from the discovered definition files, the
four-level enabled_macros allowlists, and the built-in command names.
- Discovery scans workspace (.coyote/macros/) then global macros dirs on
every resolution; workspace shadows global by name, and the shadowed
global entry is retained and flagged so both stay listable (plan
custom-commands-design.md §5). Workspace scanning is gated on a bool
parameter so the future --no-workspace-macros flag wires in one line.
- Allowlist precedence is session > agent > role > global, first Some
wins, no merging; None falls through, an empty list is an explicit
zero, all-None enables everything (mirrors SkillPolicy).
- Per-macro states per plan §6: enabled, disabled (runtime, global-level
exclusions only), locked (role/agent/session exclusions, recording the
owning level), missing (unknown allowlist names warn instead of
bailing — deliberate divergence from skills), shadowed (built-in name
collisions), and invalid (parse failures and the reserved names
enable/disable). Invalid beats allowlist exclusion beats shadowing.
- Adds enabled_macros() accessors on Role, Session, and Agent alongside
their enabled_skills() counterparts, plus paths::workspace_macros_dir.
- 37 tests: state matrix, pairwise precedence, explicit-zero pinned at
every level, workspace shadowing, reserved names, builtin collisions,
missing rows, invalid YAML, and env-gated discovery (#[serial]).
Mirrors the enabled_skills plumbing per plans/custom-commands-design.md §5:
- global: Config + AppConfig structs, from_config copy, and the
COYOTE_ENABLED_MACROS env-override arm (csv_to_vec parsing)
- role: frontmatter via parse_string_or_array (list or csv string),
plus the export() mirror so Role::save round-trips the field
- agent (non-graph): plain serde on AgentConfig; graph.yaml silently
ignores the key (pinned by test, no field on Graph by design)
- session: plain serde with csv-or-vec deserializer
Empty list/string deserializes to Some([]) (explicit zero), distinct
from absent/null (None) — pinned by tests at every level, including
the env arm (serial-fenced against the from_config tests, which read
the process env via load_envs).
Agent startup and graph rag nodes both run an interactive wizard when their
knowledge base has not been built, but neither offered the driver choice that
interactive named-RAG creation has, so both silently produced a yaml store.
A plain agent was the worse of the two: AgentConfig carries only documents, so
there was no way to get a duckdb RAG for one, interactively or declaratively. A
graph node could at least declare driver: in the workflow.
Agent startup now passes prompt_for_driver, and a rag node whose wizard runs is
asked too. The prompt is skipped when the node already declares a driver, and
sits inside the not-fully-specified branch after the non-interactive bail, so
declarative workflows and headless runs are unchanged. Temp RAGs still pass
false: they are deleted on the next run, so a persistent store would only leave
a sidecar behind.
The prompt moves to select_rag_driver rather than being duplicated.
`RagNode` gains an optional `driver`, forwarded into `RagInitConfig` so a
graph node can build its knowledge base on duckdb instead of yaml. Nodes
that name no driver forward `None`, which still resolves to yaml, so
existing workflows are unaffected.
An unknown driver is rejected up front rather than at construction time.
`Rag::create` dispatches unknown drivers to its yaml catch-all, so a typo
would otherwise embed every document and persist the bogus string, after
which every subsequent load fails validation and the agent cannot start.
The check asks `RagData::validate()` through a probe value instead of
restating the list of valid drivers, so the two cannot drift.
Adds QdrantProvider as a read-only driver for pre-existing remote Qdrant
collections, an interactive '.rag attach' wizard, and the sandbox credential
and domain-whitelisting wiring that lets an attached RAG work inside a sandbox.
Attach-only by design: rebuild_indexes bails for both the attached and the
unattached case rather than silently succeeding. Coyote never writes to Qdrant
in this change.
Vectors are never hydrated back from Qdrant. Cosine collections L2-normalize
stored vectors on write, so reading them back returns unit-length copies of the
originals; the YAML vector copy is authoritative and the serialization guard
stays scoped to the duckdb driver alone.
Collections keyed by string or UUID point IDs are rejected at attach time. The
read path parses point ids as u64 inside a filter_map, so such a collection
would otherwise yield zero results with no error.
The three response-shape parsers are pure functions over an already-parsed JSON
body, unit-tested against captured fixtures, with the async wrappers delegating
to them rather than duplicating the logic.
'.rag' now splits its first argument, so '.rag attach <name>' no longer tries to
load a RAG literally named 'attach <name>'.
Phase 3 of the RAG driver abstraction. Adds a `DuckDbProvider` that keeps
vectors and document content in a `.duckdb` sidecar next to the existing
YAML metadata, selected by the `driver: duckdb` field.
- `src/rag/providers/duckdb.rs` (new): vector search via the vss extension
and keyword search via fts, an all-or-nothing hydration path (a partial
read is an error, never a shorter map), and an anti-wipe guard that
refuses the destructive `CREATE OR REPLACE TABLE` when `data.vectors` is
empty while `data.files` is not and the store still holds rows.
- `src/rag/mod.rs`: `sync_documents` now refreshes `bm25`/`node_to_docs`
BEFORE the fallible `provider.rebuild_indexes`. `self.data` is already
mutated by that point, so propagating a provider error afterwards would
leave the derived in-memory state describing the previous corpus while
`data` describes the new one. Both rebuilds are pure functions of
`self.data` and cannot fail, so running them first is always safe.
- `src/config/paths.rs`: sidecar path helpers.
- `src/rag/providers/mod.rs`, `src/config/agent.rs`: driver dispatch and
RAG cache keying.
Also keeps `RequestContext::rag_key` in lockstep with `rag` at the two
sites that were still missing it, so that a cache insert and its matching
invalidate are structurally incapable of disagreeing:
- `use_agent` assigned `self.rag` from the agent but never set `rag_key`.
This one was live. Agent RAGs are inserted under `RagKey::Agent(<name>)`,
so with `rag_key == None` the invalidation guards in `rebuild_rag` and
`edit_rag_docs` matched nothing and `.rebuild rag` left the stale cache
entry in place. Worse, a preceding `.rag <name>` left a stale
`Named(<name>)` key attached to the agent's RAG, pointing the
invalidation at an unrelated RAG's cache entry. Now mirrors the insert
key exactly, yielding `None` when the agent has no RAG.
- `exit_agent` cleared `self.rag` but left `rag_key` behind. Latent rather
than live, since `rebuild_rag`/`edit_rag_docs` both bail on
`rag.is_none()` before reaching the invalidate guards, but the guards
that make it unobservable are not the kind of thing to depend on.
Covered by `use_agent_does_not_carry_stale_rag_key`, and by a new
assertion in `exit_agent_clears_all_agent_state`.
- Hang fix: 120s timeout on compression LLM call + raw_stream
channel-close handling (None => break instead of spinning)
- Supervisor effective active count: stop counting finished
JoinHandles as occupying capacity slots
- Universal tool result size cap: truncate_if_needed() on
ToolResult, applied in eval_tool_calls() after escalation block;
configurable via max_tool_result_chars in AppConfig/AgentConfig
- Windowed compression: compression_keep_last config param keeps
the N most recent messages visible after compression
- Fix pre-existing flaky test: add #[serial] to
handle_list_available_unrestricted_when_no_whitelist so it does
not race with TestConfigDirGuard-based tests that temporarily
populate the agents data dir
Dark-Alex-17