From 78e23f27e4cb121e680f98d5509db9964747321c Mon Sep 17 00:00:00 2001 From: Alex Clarke Date: Fri, 22 May 2026 17:47:32 -0600 Subject: [PATCH] docs: Added additional docs for configuration sharing --- Sharing-Configurations.md | 355 ++++++++++++++++++++++++++++++++++++++ 1 file changed, 355 insertions(+) create mode 100644 Sharing-Configurations.md diff --git a/Sharing-Configurations.md b/Sharing-Configurations.md new file mode 100644 index 0000000..ba56221 --- /dev/null +++ b/Sharing-Configurations.md @@ -0,0 +1,355 @@ +Loki ships with a built-in mechanism for installing and sharing configurations (i.e. agents, roles, macros, tools, and MCP +servers) directly from any git repository. This makes it easy to: + +- Sync your Loki setup across multiple machines. +- Share your work with teammates or the community. +- Bootstrap a new install with a curated set of assets. +- Pin to specific versions for reproducibility. + +The relevant commands: + +- CLI: `loki --install-from ` +- REPL: `.install remote ` + +This page covers the expected repository layout, command-line flags, conflict resolution, secrets handling, and how to +publish your own shareable bundle. + +--- + +## Table of contents + +- [Quick start](#quick-start) +- [Expected repository layout](#expected-repository-layout) +- [Ref pinning](#ref-pinning) +- [Filtering by category](#filtering-by-category) +- [Conflict resolution](#conflict-resolution) +- [MCP configuration merge](#mcp-configuration-merge) +- [Secrets handling](#secrets-handling) +- [Git authentication](#git-authentication) +- [Publishing your own bundle](#publishing-your-own-bundle) +- [What is not shared](#what-is-not-shared) +- [Examples](#examples) +- [Troubleshooting](#troubleshooting) + +--- + +## Quick start + +Install everything from a remote repo: + +```sh +loki --install-from https://github.com/Dark-Alex-17/loki-config-template +``` + +or from inside the Loki REPL: + +``` +.install remote https://github.com/Dark-Alex-17/loki-config-template +``` + +That's it. Loki clones the repo to a temp directory, scans for recognized asset categories, and installs each into the +matching subdirectory of your user config. The temp clone is removed on completion. + +--- + +## Expected repository layout + +Loki recognizes these top-level directories. Anything outside them is ignored. + +``` +/ +├── agents/ +│ └── / # One subdirectory per agent +│ ├── config.yaml # LLM-loop agent +│ │ └── (or graph.yaml) # Graph agent +│ ├── README.md # Optional +│ ├── tools.sh # Optional agent-local tools +│ └── scripts/ # Optional graph-node scripts +├── roles/ +│ └── .md # Markdown with YAML frontmatter + prompt body +├── macros/ +│ └── .yaml # Positional/rest variables + REPL command steps +└── functions/ + ├── tools/ + │ └── *.sh / *.py / *.ts # Global tools (auto chmod +x on install) + └── mcp.json # MCP server config (merged with local, not overwritten) +``` + +A few things to note: + +- **Missing categories are skipped silently.** A repo that only contains `agents/` installs only agents. This means you + do not need to use `--filter` for partial repos. +- **`.git/` is excluded** from the scan automatically. +- **Symlinks are rejected** by the install walker as a defense-in-depth measure. +- **`functions/bin/` and `functions/utils/` are not recognized** (compiled at runtime / not in scope for sharing). +- **The whole `config.yaml`** (global Loki config) is **not** shared (see [What is not shared](#what-is-not-shared)). + +--- + +## Ref pinning + +Pin the install to a specific tag, branch, or commit by appending `#` to the URL. + +```sh +loki --install-from https://github.com//#v1.2.3 +loki --install-from https://github.com//#main +loki --install-from https://github.com//#abc1234 +``` + +How Loki resolves the ref: + +- **Branch or tag names** are passed as `--branch ` to a shallow `git clone --depth 1`. +- **Commit SHAs** (4-40 hex characters) trigger a full clone followed by `git checkout `. + +Validation: refs must match `[A-Za-z0-9._/+-]`, must not start with `-`, and must not contain `..`. These rules prevent +the ref from being interpreted as a CLI flag or escaping the repo via path traversal. + +--- + +## Filtering by category + +Restrict the install to a single asset category with `--filter`: + +| Filter | Installs | +|----------------|------------------------------------------------| +| *(omitted)* | `agents/`, `roles/`, `macros/`, `functions/tools/`, and merges `functions/mcp.json` | +| `agents` | `agents/` only | +| `roles` | `roles/` only | +| `macros` | `macros/` only | +| `functions` | `functions/tools/` only (does **not** include `mcp.json`) | +| `mcp_config` | `functions/mcp.json` only (merged) | + +```sh +loki --install-from --filter agents +loki --install-from --filter mcp_config +``` + +REPL form: + +``` +.install remote --filter agents +``` + +Note that `--filter functions` is intentionally narrow. It installs the global tools under `functions/tools/` and +**not** `mcp.json`. To install just the MCP config, use `--filter mcp_config`. To install both, use no filter. + +--- + +## Conflict resolution + +When an install file would overwrite an existing local file with different contents, you have several options. + +### Plain files (agents, roles, macros, tools) + +In a terminal (TTY), Loki prompts per conflicting file: + +| Option | Effect | +|----------------|------------------------------------------------------------------------------| +| `keep` | Skip this file; keep your local copy. | +| `replace` | Overwrite with the remote file. | +| `keep-all` | Skip this file and all remaining conflicts in this install. | +| `replace-all` | Overwrite this file and all remaining conflicts in this install. | +| `abort` | Stop the install. Files already written stay; they are not rolled back. | + +To skip prompts and replace everything, pass `--install-force` on the CLI or `--force` in the REPL form. In non-TTY +mode (CI, piped, redirected stdin), the install will **abort** rather than silently overwrite. To overwrite +non-interactively, you must pass `--install-force`. + +### Identical content is not a conflict + +If a remote file's bytes match your local copy exactly, Loki silently treats it as `identical` and skips it. Re-running +the same install is idempotent. + +### File-mode (`+x`) handling + +Scripts written into `functions/tools/` are automatically marked executable based on extension. Bash (`.sh`), Python +(`.py`), and TypeScript (`.ts`) files get `chmod 0o755` on Unix. This applies to both new files and replaced conflicts. + +--- + +## MCP configuration merge + +`functions/mcp.json` is the exception to the per-file conflict model. Instead of replacing your local file outright, +Loki **merges** the remote `mcp.json` into your existing one. + +### Merge behavior + +- **New server names** (present in remote, absent locally) are added. +- **Identical server entries** (same JSON shape locally and remotely) are silently kept. +- **Conflicting server entries** prompt with three options: + +| Option | Effect | +|-------------------------------------|---------------------------------------------------------------------| +| `keep local` | Leave your existing entry alone. | +| `take remote` | Overwrite with the remote entry. | +| `rename remote as "-remote"` | Insert the remote entry under a suffixed key. If the suffixed key is also taken, Loki appends `-2`, `-3`, etc. | +| `abort merge` | Stop the merge with an error. No partial write. | + +With `--install-force`, every conflict is resolved by taking the remote entry. In non-TTY mode without +`--install-force`, the merge aborts before writing. + +### Validation + +Each added, replaced, or renamed entry is validated against the MCP server schema (e.g., `stdio` servers must have a +`command`; `http`/`sse` servers must have a `url`) before the merged file is written. A validation failure aborts the +install. Loki will **never** write a partially-validated `mcp.json`. + +### Atomic write + +The merged file is written to a tempfile (`mcp.json.tmp`) and then renamed atomically over the target. This guarantees +that a crash or interrupt mid-write will not corrupt your existing `mcp.json`. + +--- + +## Secrets handling + +MCP server entries (and other config files) can reference vault secrets with `{{SECRET_NAME}}` placeholders. After the +install completes, Loki scans the resulting `mcp.json` for placeholders that are not yet in your vault and either: + +- **In a TTY:** prompts you, one secret at a time, whether to add it to the vault now. On the first "Yes", Loki + initializes the vault password file (if needed). On "No", the secret is deferred and reported at the end. +- **In a non-TTY environment:** skips prompts entirely; lists every missing secret in a final reminder block, with the + commands you can run later (`loki --add-secret ` or `.vault add `). + +Both modes always print a final summary distinguishing secrets added from those deferred. See [Vault](Vault) for the +full secrets workflow. + +--- + +## Git authentication + +Loki shells out to your local `git` binary to clone the remote repository. This means it inherits **your existing git +authentication setup** with no additional configuration: + +- **SSH URLs** (`git@github.com:owner/repo.git`) use your `ssh-agent`, `~/.ssh/config`, and SSH keys. +- **HTTPS URLs** with private repos use your configured `credential.helper` (e.g., osxkeychain on macOS, libsecret on + Linux, the GitHub CLI's helper). +- **Public HTTPS URLs** require no auth. + +If `git` is not on your `PATH`, you'll get a clear error message on the first install attempt. + +--- + +## Publishing your own bundle + +To share your Loki configuration, structure a git repo like the template above and push it to GitHub, GitLab, your +self-hosted Forgejo instance, etc. + +The easiest way to start is to **fork the official template repo**: +[`loki-config-template`](https://github.com/Dark-Alex-17/loki-config-template). It includes a working sample of each +asset type plus a `README` describing the customization workflow. + +### Best practices + +1. **Pin to tagged releases** so consumers can install with `#` for reproducibility. +2. **Keep per-agent logic in `agents//`**. Only put global tools in `functions/tools/`. +3. **Use `{{SECRET}}` placeholders** for any sensitive value in `mcp.json`. Never commit a real API key. +4. **Document expected secrets** in your repo's `README` so users know which vault entries to add. +5. **Avoid name collisions** with Loki's bundled assets (`agents/code-reviewer`, `agents/sql`, etc.) unless you + specifically want to replace them. Pick distinctive names for your shared assets. +6. **Test installs against a clean `LOKI_CONFIG_DIR`** before publishing. You can use: + + ```sh + LOKI_CONFIG_DIR=$(mktemp -d) loki --install-from file:///path/to/your/clone --install-force + ``` + +--- + +## What is not shared + +The following are **intentionally** outside the install-remote feature's scope: + +- **`config.yaml`** (the global Loki config). It holds user-specific things like editor preference, vault password file + path, client API keys, OAuth tokens, and similar. Merging a shared `config.yaml` would risk breaking auth or routing + traffic somewhere unexpected. Settings that genuinely benefit from sharing (like default models) already belong + inside individual agents' or roles' configs. +- **Sessions, RAGs, agent runtime data.** These accumulate as you use Loki and aren't shareable in a meaningful way. +- **`functions/bin/`:** Agent script binaries built at runtime, not part of the source layout. +- **`functions/utils/`:** Utility scripts intentionally not part of the share contract. + +If a team really needs synced global settings, the recommended approach is dotfiles-style symlinks or a sync tool. +Those are designed for that problem and not for `loki --install-from`. + +--- + +## Examples + +### Install everything from a public repo + +```sh +loki --install-from https://github.com/Dark-Alex-17/loki-config-template +``` + +### Pin to a release tag + +```sh +loki --install-from https://github.com/your-org/loki-config#v2.4.0 +``` + +### Install only the agents from a fork + +```sh +loki --install-from https://github.com/your-org/loki-config --filter agents +``` + +### Install only the MCP servers, force-replace conflicts + +```sh +loki --install-from https://github.com/your-org/loki-config --filter mcp_config --install-force +``` + +### From the REPL + +``` +.install remote https://github.com/your-org/loki-config +.install remote https://github.com/your-org/loki-config#main --filter agents +.install remote https://github.com/your-org/loki-config --force +``` + +### From a private SSH URL + +```sh +loki --install-from git@github.com:your-org/private-loki-config.git#v1.0.0 +``` + +### From a local file:// URL (testing your own template) + +```sh +loki --install-from file:///home/you/code/my-loki-config +``` + +--- + +## Troubleshooting + +### "git failed: ..." + +The remote clone failed. Loki passes git's stderr through verbatim. The most common causes: + +- **Authentication required**: `fatal: could not read username/password` or `Permission denied (publickey)`. Verify + your git credentials work for the same URL outside Loki by running `git clone ` manually. +- **Network failure**: `Could not resolve host` or similar. Check connectivity. +- **Invalid ref**: `couldn't find remote ref `. Verify the tag/branch/commit exists in the remote. + +### "No recognized assets found in ``" + +The cloned repo has none of the recognized top-level directories. Make sure the repo's layout matches the +[expected layout](#expected-repository-layout). A `README.md` and a `LICENSE` at the root are fine; they're just +ignored. + +### "Refusing to overwrite local file `` non-interactively" + +You're running in non-TTY mode (CI, piped, redirected stdin) and there's a conflict. Re-run with `--install-force` to +overwrite, or run in an interactive terminal to be prompted. + +### "MCP server '``' already exists locally. Refusing to merge non-interactively." + +Same pattern as the above, but for the MCP merge. Use `--install-force` to take the remote entry for every conflict. + +### Missing secrets after install + +Add them via `loki --add-secret ` (CLI) or `.vault add ` (REPL). See [Vault](Vault) for details. + +### `git` binary not found + +Loki shells out to the system `git` binary. Install git for your platform and ensure it's on your `PATH`.