docs: Added additional docs for configuration sharing

2026-05-22 17:47:32 -06:00
parent 4ebf1be271
commit 78e23f27e4
+355
@@ -0,0 +1,355 @@
Loki ships with a built-in mechanism for installing and sharing configurations (i.e. agents, roles, macros, tools, and MCP
servers) directly from any git repository. This makes it easy to:
- Sync your Loki setup across multiple machines.
- Share your work with teammates or the community.
- Bootstrap a new install with a curated set of assets.
- Pin to specific versions for reproducibility.
The relevant commands:
- CLI: `loki --install-from <git-url>`
- REPL: `.install remote <git-url>`
This page covers the expected repository layout, command-line flags, conflict resolution, secrets handling, and how to
publish your own shareable bundle.
---
## Table of contents
- [Quick start](#quick-start)
- [Expected repository layout](#expected-repository-layout)
- [Ref pinning](#ref-pinning)
- [Filtering by category](#filtering-by-category)
- [Conflict resolution](#conflict-resolution)
- [MCP configuration merge](#mcp-configuration-merge)
- [Secrets handling](#secrets-handling)
- [Git authentication](#git-authentication)
- [Publishing your own bundle](#publishing-your-own-bundle)
- [What is not shared](#what-is-not-shared)
- [Examples](#examples)
- [Troubleshooting](#troubleshooting)
---
## Quick start
Install everything from a remote repo:
```sh
loki --install-from https://github.com/Dark-Alex-17/loki-config-template
```
or from inside the Loki REPL:
```
.install remote https://github.com/Dark-Alex-17/loki-config-template
```
That's it. Loki clones the repo to a temp directory, scans for recognized asset categories, and installs each into the
matching subdirectory of your user config. The temp clone is removed on completion.
---
## Expected repository layout
Loki recognizes these top-level directories. Anything outside them is ignored.
```
<repo>/
├── agents/
│ └── <agent-name>/ # One subdirectory per agent
│ ├── config.yaml # LLM-loop agent
│ │ └── (or graph.yaml) # Graph agent
│ ├── README.md # Optional
│ ├── tools.sh # Optional agent-local tools
│ └── scripts/ # Optional graph-node scripts
├── roles/
│ └── <role-name>.md # Markdown with YAML frontmatter + prompt body
├── macros/
│ └── <macro-name>.yaml # Positional/rest variables + REPL command steps
└── functions/
├── tools/
│ └── *.sh / *.py / *.ts # Global tools (auto chmod +x on install)
└── mcp.json # MCP server config (merged with local, not overwritten)
```
A few things to note:
- **Missing categories are skipped silently.** A repo that only contains `agents/` installs only agents. This means you
do not need to use `--filter` for partial repos.
- **`.git/` is excluded** from the scan automatically.
- **Symlinks are rejected** by the install walker as a defense-in-depth measure.
- **`functions/bin/` and `functions/utils/` are not recognized** (compiled at runtime / not in scope for sharing).
- **The whole `config.yaml`** (global Loki config) is **not** shared (see [What is not shared](#what-is-not-shared)).
---
## Ref pinning
Pin the install to a specific tag, branch, or commit by appending `#<ref>` to the URL.
```sh
loki --install-from https://github.com/<owner>/<repo>#v1.2.3
loki --install-from https://github.com/<owner>/<repo>#main
loki --install-from https://github.com/<owner>/<repo>#abc1234
```
How Loki resolves the ref:
- **Branch or tag names** are passed as `--branch <ref>` to a shallow `git clone --depth 1`.
- **Commit SHAs** (4-40 hex characters) trigger a full clone followed by `git checkout <ref>`.
Validation: refs must match `[A-Za-z0-9._/+-]`, must not start with `-`, and must not contain `..`. These rules prevent
the ref from being interpreted as a CLI flag or escaping the repo via path traversal.
---
## Filtering by category
Restrict the install to a single asset category with `--filter`:
| Filter | Installs |
|----------------|------------------------------------------------|
| *(omitted)* | `agents/`, `roles/`, `macros/`, `functions/tools/`, and merges `functions/mcp.json` |
| `agents` | `agents/` only |
| `roles` | `roles/` only |
| `macros` | `macros/` only |
| `functions` | `functions/tools/` only (does **not** include `mcp.json`) |
| `mcp_config` | `functions/mcp.json` only (merged) |
```sh
loki --install-from <git-url> --filter agents
loki --install-from <git-url> --filter mcp_config
```
REPL form:
```
.install remote <git-url> --filter agents
```
Note that `--filter functions` is intentionally narrow. It installs the global tools under `functions/tools/` and
**not** `mcp.json`. To install just the MCP config, use `--filter mcp_config`. To install both, use no filter.
---
## Conflict resolution
When an install file would overwrite an existing local file with different contents, you have several options.
### Plain files (agents, roles, macros, tools)
In a terminal (TTY), Loki prompts per conflicting file:
| Option | Effect |
|----------------|------------------------------------------------------------------------------|
| `keep` | Skip this file; keep your local copy. |
| `replace` | Overwrite with the remote file. |
| `keep-all` | Skip this file and all remaining conflicts in this install. |
| `replace-all` | Overwrite this file and all remaining conflicts in this install. |
| `abort` | Stop the install. Files already written stay; they are not rolled back. |
To skip prompts and replace everything, pass `--install-force` on the CLI or `--force` in the REPL form. In non-TTY
mode (CI, piped, redirected stdin), the install will **abort** rather than silently overwrite. To overwrite
non-interactively, you must pass `--install-force`.
### Identical content is not a conflict
If a remote file's bytes match your local copy exactly, Loki silently treats it as `identical` and skips it. Re-running
the same install is idempotent.
### File-mode (`+x`) handling
Scripts written into `functions/tools/` are automatically marked executable based on extension. Bash (`.sh`), Python
(`.py`), and TypeScript (`.ts`) files get `chmod 0o755` on Unix. This applies to both new files and replaced conflicts.
---
## MCP configuration merge
`functions/mcp.json` is the exception to the per-file conflict model. Instead of replacing your local file outright,
Loki **merges** the remote `mcp.json` into your existing one.
### Merge behavior
- **New server names** (present in remote, absent locally) are added.
- **Identical server entries** (same JSON shape locally and remotely) are silently kept.
- **Conflicting server entries** prompt with three options:
| Option | Effect |
|-------------------------------------|---------------------------------------------------------------------|
| `keep local` | Leave your existing entry alone. |
| `take remote` | Overwrite with the remote entry. |
| `rename remote as "<name>-remote"` | Insert the remote entry under a suffixed key. If the suffixed key is also taken, Loki appends `-2`, `-3`, etc. |
| `abort merge` | Stop the merge with an error. No partial write. |
With `--install-force`, every conflict is resolved by taking the remote entry. In non-TTY mode without
`--install-force`, the merge aborts before writing.
### Validation
Each added, replaced, or renamed entry is validated against the MCP server schema (e.g., `stdio` servers must have a
`command`; `http`/`sse` servers must have a `url`) before the merged file is written. A validation failure aborts the
install. Loki will **never** write a partially-validated `mcp.json`.
### Atomic write
The merged file is written to a tempfile (`mcp.json.tmp`) and then renamed atomically over the target. This guarantees
that a crash or interrupt mid-write will not corrupt your existing `mcp.json`.
---
## Secrets handling
MCP server entries (and other config files) can reference vault secrets with `{{SECRET_NAME}}` placeholders. After the
install completes, Loki scans the resulting `mcp.json` for placeholders that are not yet in your vault and either:
- **In a TTY:** prompts you, one secret at a time, whether to add it to the vault now. On the first "Yes", Loki
initializes the vault password file (if needed). On "No", the secret is deferred and reported at the end.
- **In a non-TTY environment:** skips prompts entirely; lists every missing secret in a final reminder block, with the
commands you can run later (`loki --add-secret <NAME>` or `.vault add <NAME>`).
Both modes always print a final summary distinguishing secrets added from those deferred. See [Vault](Vault) for the
full secrets workflow.
---
## Git authentication
Loki shells out to your local `git` binary to clone the remote repository. This means it inherits **your existing git
authentication setup** with no additional configuration:
- **SSH URLs** (`git@github.com:owner/repo.git`) use your `ssh-agent`, `~/.ssh/config`, and SSH keys.
- **HTTPS URLs** with private repos use your configured `credential.helper` (e.g., osxkeychain on macOS, libsecret on
Linux, the GitHub CLI's helper).
- **Public HTTPS URLs** require no auth.
If `git` is not on your `PATH`, you'll get a clear error message on the first install attempt.
---
## Publishing your own bundle
To share your Loki configuration, structure a git repo like the template above and push it to GitHub, GitLab, your
self-hosted Forgejo instance, etc.
The easiest way to start is to **fork the official template repo**:
[`loki-config-template`](https://github.com/Dark-Alex-17/loki-config-template). It includes a working sample of each
asset type plus a `README` describing the customization workflow.
### Best practices
1. **Pin to tagged releases** so consumers can install with `#<tag>` for reproducibility.
2. **Keep per-agent logic in `agents/<name>/`**. Only put global tools in `functions/tools/`.
3. **Use `{{SECRET}}` placeholders** for any sensitive value in `mcp.json`. Never commit a real API key.
4. **Document expected secrets** in your repo's `README` so users know which vault entries to add.
5. **Avoid name collisions** with Loki's bundled assets (`agents/code-reviewer`, `agents/sql`, etc.) unless you
specifically want to replace them. Pick distinctive names for your shared assets.
6. **Test installs against a clean `LOKI_CONFIG_DIR`** before publishing. You can use:
```sh
LOKI_CONFIG_DIR=$(mktemp -d) loki --install-from file:///path/to/your/clone --install-force
```
---
## What is not shared
The following are **intentionally** outside the install-remote feature's scope:
- **`config.yaml`** (the global Loki config). It holds user-specific things like editor preference, vault password file
path, client API keys, OAuth tokens, and similar. Merging a shared `config.yaml` would risk breaking auth or routing
traffic somewhere unexpected. Settings that genuinely benefit from sharing (like default models) already belong
inside individual agents' or roles' configs.
- **Sessions, RAGs, agent runtime data.** These accumulate as you use Loki and aren't shareable in a meaningful way.
- **`functions/bin/`:** Agent script binaries built at runtime, not part of the source layout.
- **`functions/utils/`:** Utility scripts intentionally not part of the share contract.
If a team really needs synced global settings, the recommended approach is dotfiles-style symlinks or a sync tool.
Those are designed for that problem and not for `loki --install-from`.
---
## Examples
### Install everything from a public repo
```sh
loki --install-from https://github.com/Dark-Alex-17/loki-config-template
```
### Pin to a release tag
```sh
loki --install-from https://github.com/your-org/loki-config#v2.4.0
```
### Install only the agents from a fork
```sh
loki --install-from https://github.com/your-org/loki-config --filter agents
```
### Install only the MCP servers, force-replace conflicts
```sh
loki --install-from https://github.com/your-org/loki-config --filter mcp_config --install-force
```
### From the REPL
```
.install remote https://github.com/your-org/loki-config
.install remote https://github.com/your-org/loki-config#main --filter agents
.install remote https://github.com/your-org/loki-config --force
```
### From a private SSH URL
```sh
loki --install-from git@github.com:your-org/private-loki-config.git#v1.0.0
```
### From a local file:// URL (testing your own template)
```sh
loki --install-from file:///home/you/code/my-loki-config
```
---
## Troubleshooting
### "git failed: ..."
The remote clone failed. Loki passes git's stderr through verbatim. The most common causes:
- **Authentication required**: `fatal: could not read username/password` or `Permission denied (publickey)`. Verify
your git credentials work for the same URL outside Loki by running `git clone <url>` manually.
- **Network failure**: `Could not resolve host` or similar. Check connectivity.
- **Invalid ref**: `couldn't find remote ref <ref>`. Verify the tag/branch/commit exists in the remote.
### "No recognized assets found in `<url>`"
The cloned repo has none of the recognized top-level directories. Make sure the repo's layout matches the
[expected layout](#expected-repository-layout). A `README.md` and a `LICENSE` at the root are fine; they're just
ignored.
### "Refusing to overwrite local file `<path>` non-interactively"
You're running in non-TTY mode (CI, piped, redirected stdin) and there's a conflict. Re-run with `--install-force` to
overwrite, or run in an interactive terminal to be prompted.
### "MCP server '`<name>`' already exists locally. Refusing to merge non-interactively."
Same pattern as the above, but for the MCP merge. Use `--install-force` to take the remote entry for every conflict.
### Missing secrets after install
Add them via `loki --add-secret <NAME>` (CLI) or `.vault add <NAME>` (REPL). See [Vault](Vault) for details.
### `git` binary not found
Loki shells out to the system `git` binary. Install git for your platform and ensure it's on your `PATH`.