diff --git a/Model-Catalog.md b/Model-Catalog.md new file mode 100644 index 0000000..616b2cd --- /dev/null +++ b/Model-Catalog.md @@ -0,0 +1,125 @@ +# Model Catalog + +Coyote ships with a built-in catalog of models for every supported provider: context window +sizes, output limits, pricing, capabilities (vision, function calling, reasoning), wire-name +aliases, and per-model request patches. The catalog is what powers the `.model` tab-completion +table in the REPL, `--list-models`, token-budget accounting, and cost display. + +This page explains where that data comes from, how it stays fresh automatically, and how to +control it. + +## How the catalog works + +The catalog has three layers, from most static to most fresh: + +1. **Embedded generated floor:** A full `models.yaml` is generated from live API data + ([models.dev](https://models.dev) + [OpenRouter](https://openrouter.ai)) and embedded into + the Coyote binary at compile time. A scheduled GitHub workflow regenerates it twice a day + and commits the result, so every release carries a recent snapshot. This layer always + exists and is the fallback for everything else. Coyote never has *no* catalog. + +2. **Local override (`/models-override.yaml`):** A synced copy of the catalog + written by `--sync-models` or by the automatic background refresh. When present and valid, + it takes precedence over the embedded floor. The override is **version-pinned** to the + Coyote version that wrote it; an override written by a different version is ignored (see + [Troubleshooting](#troubleshooting)). + +3. **TTL auto-refresh:** On REPL startup, if the override is missing, unreadable, or older + than `sync_models_ttl_hours`, Coyote fires a fire-and-forget background sync (when + `sync_models_auto` is enabled). The running session keeps the catalog it already loaded; + the *next* start picks up the refreshed file. Providers that yield no live data keep their + embedded entries, so a partial upstream outage never shrinks your catalog. + +Models you define inline in your `config.yaml` client blocks always win over all of the above, +and unknown `client:model` names still work. They're created on the fly with family-level +quirks applied (so a brand-new `claude:claude-*` model inherits the right request behavior on +day zero). + +## Configuration + +Three keys in `config.yaml` control syncing: + +```yaml +sync_models_url: null # null = fetch fresh data from models.dev + OpenRouter (API mode) + # URL = mirror a complete models.yaml from that URL (mirror mode) +sync_models_auto: true # refresh the catalog in the background on REPL startup once stale +sync_models_ttl_hours: 24 # hours before the synced catalog is considered stale +``` + +### API mode vs. mirror mode + +- **API mode (`sync_models_url: null`, the default):** `--sync-models` and the auto-refresh + rebuild the catalog from live models.dev + OpenRouter data, applying the same hand-curated + quirks the generator uses. You get new models as soon as the upstream catalogs list them. + +- **Mirror mode (`sync_models_url: `):** Fetches a complete, ready-made `models.yaml` + from the URL instead of talking to the APIs. Useful for air-gapped or policy-restricted + environments, or to pin your whole team to one blessed catalog. For example, to mirror the + latest generated catalog from the Coyote repo: + + ```yaml + sync_models_url: https://raw.githubusercontent.com/Dark-Alex-17/coyote/refs/heads/main/models.yaml + ``` + +## CLI commands + +- `coyote --sync-models`: refresh the catalog now (API or mirror mode per + `sync_models_url`) and write `models-override.yaml`. +- `coyote --list-models`: print every chat model available with your configured clients. +- `coyote --generate-models [PATH]`: regenerate a complete `models.yaml` from live API data + (defaults to `./models.yaml`). This is the maintainer-facing generator the scheduled + workflow runs; it hard-fails unless both upstream sources respond, so a generated file is + never built from partial data. + +## quirks.yaml: The hand-owned overlay + +Not everything about a model is machine-discoverable. `quirks.yaml` (repo root, embedded at +compile time) layers hand-curated knowledge on top of the API-sourced data, per provider: + +- `oauth`: provider OAuth configuration (client IDs, scopes). +- `exclude`: glob patterns for API-listed models Coyote should drop (e.g. image-only models). +- `rules`: field overlays applied to every **API-sourced** model whose name matches a glob + (`*` = any run of characters, `?` = exactly one): request `patch`es, `real_name` wire + aliases, `default_reasoning_effort`, `require_max_tokens`, `no_stream`, + `system_prompt_prefix`, and friends. Only fields explicitly present in the rule are written. +- `variants`: clones of matching models with a suffix appended and fields overlaid (e.g. + `:thinking` variants). +- `models`: fully hand-owned model entries for providers (or individual models) that no + public catalog describes. `rules` never apply to them, so write them self-complete, though + if a hand-owned entry shares its name with an API-sourced model, the two merge field-wise + (the hand-owned field always wins; absent fields inherit the fetched values). + +When the generator or a sync runs, quirks are applied on top of the fetched data, so curated +behavior survives every refresh. + +## Capability glyphs + +Model listings (`.model` tab-completion, `--list-models`) show per-model capability glyphs: + +| Glyph | Meaning | +|-------|---------| +| 👁 | Supports vision (image input) | +| ⚒ | Supports function calling (tools) | +| 🧠 | Supports configurable reasoning effort | + +## Sandboxes + +Coyote's [sandbox](https://github.com/Dark-Alex-17/coyote/wiki/Sandboxes) kit allowlists +`models.dev` and `openrouter.ai`, so API-mode syncing works out of the box inside a Docker +Sandbox; no extra network policy needed. + +## Troubleshooting + +- **Stale or corrupt override:** The catalog self-heals. If `models-override.yaml` fails to + parse (or fails its version check), Coyote silently falls back to the embedded catalog, and + the next REPL startup treats the override as stale and rewrites it in the background. You + can always force a rewrite with `coyote --sync-models`, or just delete the file. + +- **Upgraded Coyote and the catalog "reset"?:** That's the version pin: an override written + by an older Coyote is ignored by a newer one, so you're on the (fresh) embedded catalog + shipped with the new binary until the first background refresh rewrites the override, + usually within one REPL start. No action needed. + +- **No new models appearing:** Check `sync_models_auto` isn't `false`, and remember the + running session keeps its loaded catalog: restart the REPL after a sync completes. In + mirror mode, you only ever get what the mirror serves.