docs: Added additional docs for configuration sharing
@@ -0,0 +1,355 @@
|
||||
Loki ships with a built-in mechanism for installing and sharing configurations (i.e. agents, roles, macros, tools, and MCP
|
||||
servers) directly from any git repository. This makes it easy to:
|
||||
|
||||
- Sync your Loki setup across multiple machines.
|
||||
- Share your work with teammates or the community.
|
||||
- Bootstrap a new install with a curated set of assets.
|
||||
- Pin to specific versions for reproducibility.
|
||||
|
||||
The relevant commands:
|
||||
|
||||
- CLI: `loki --install-from <git-url>`
|
||||
- REPL: `.install remote <git-url>`
|
||||
|
||||
This page covers the expected repository layout, command-line flags, conflict resolution, secrets handling, and how to
|
||||
publish your own shareable bundle.
|
||||
|
||||
---
|
||||
|
||||
## Table of contents
|
||||
|
||||
- [Quick start](#quick-start)
|
||||
- [Expected repository layout](#expected-repository-layout)
|
||||
- [Ref pinning](#ref-pinning)
|
||||
- [Filtering by category](#filtering-by-category)
|
||||
- [Conflict resolution](#conflict-resolution)
|
||||
- [MCP configuration merge](#mcp-configuration-merge)
|
||||
- [Secrets handling](#secrets-handling)
|
||||
- [Git authentication](#git-authentication)
|
||||
- [Publishing your own bundle](#publishing-your-own-bundle)
|
||||
- [What is not shared](#what-is-not-shared)
|
||||
- [Examples](#examples)
|
||||
- [Troubleshooting](#troubleshooting)
|
||||
|
||||
---
|
||||
|
||||
## Quick start
|
||||
|
||||
Install everything from a remote repo:
|
||||
|
||||
```sh
|
||||
loki --install-from https://github.com/Dark-Alex-17/loki-config-template
|
||||
```
|
||||
|
||||
or from inside the Loki REPL:
|
||||
|
||||
```
|
||||
.install remote https://github.com/Dark-Alex-17/loki-config-template
|
||||
```
|
||||
|
||||
That's it. Loki clones the repo to a temp directory, scans for recognized asset categories, and installs each into the
|
||||
matching subdirectory of your user config. The temp clone is removed on completion.
|
||||
|
||||
---
|
||||
|
||||
## Expected repository layout
|
||||
|
||||
Loki recognizes these top-level directories. Anything outside them is ignored.
|
||||
|
||||
```
|
||||
<repo>/
|
||||
├── agents/
|
||||
│ └── <agent-name>/ # One subdirectory per agent
|
||||
│ ├── config.yaml # LLM-loop agent
|
||||
│ │ └── (or graph.yaml) # Graph agent
|
||||
│ ├── README.md # Optional
|
||||
│ ├── tools.sh # Optional agent-local tools
|
||||
│ └── scripts/ # Optional graph-node scripts
|
||||
├── roles/
|
||||
│ └── <role-name>.md # Markdown with YAML frontmatter + prompt body
|
||||
├── macros/
|
||||
│ └── <macro-name>.yaml # Positional/rest variables + REPL command steps
|
||||
└── functions/
|
||||
├── tools/
|
||||
│ └── *.sh / *.py / *.ts # Global tools (auto chmod +x on install)
|
||||
└── mcp.json # MCP server config (merged with local, not overwritten)
|
||||
```
|
||||
|
||||
A few things to note:
|
||||
|
||||
- **Missing categories are skipped silently.** A repo that only contains `agents/` installs only agents. This means you
|
||||
do not need to use `--filter` for partial repos.
|
||||
- **`.git/` is excluded** from the scan automatically.
|
||||
- **Symlinks are rejected** by the install walker as a defense-in-depth measure.
|
||||
- **`functions/bin/` and `functions/utils/` are not recognized** (compiled at runtime / not in scope for sharing).
|
||||
- **The whole `config.yaml`** (global Loki config) is **not** shared (see [What is not shared](#what-is-not-shared)).
|
||||
|
||||
---
|
||||
|
||||
## Ref pinning
|
||||
|
||||
Pin the install to a specific tag, branch, or commit by appending `#<ref>` to the URL.
|
||||
|
||||
```sh
|
||||
loki --install-from https://github.com/<owner>/<repo>#v1.2.3
|
||||
loki --install-from https://github.com/<owner>/<repo>#main
|
||||
loki --install-from https://github.com/<owner>/<repo>#abc1234
|
||||
```
|
||||
|
||||
How Loki resolves the ref:
|
||||
|
||||
- **Branch or tag names** are passed as `--branch <ref>` to a shallow `git clone --depth 1`.
|
||||
- **Commit SHAs** (4-40 hex characters) trigger a full clone followed by `git checkout <ref>`.
|
||||
|
||||
Validation: refs must match `[A-Za-z0-9._/+-]`, must not start with `-`, and must not contain `..`. These rules prevent
|
||||
the ref from being interpreted as a CLI flag or escaping the repo via path traversal.
|
||||
|
||||
---
|
||||
|
||||
## Filtering by category
|
||||
|
||||
Restrict the install to a single asset category with `--filter`:
|
||||
|
||||
| Filter | Installs |
|
||||
|----------------|------------------------------------------------|
|
||||
| *(omitted)* | `agents/`, `roles/`, `macros/`, `functions/tools/`, and merges `functions/mcp.json` |
|
||||
| `agents` | `agents/` only |
|
||||
| `roles` | `roles/` only |
|
||||
| `macros` | `macros/` only |
|
||||
| `functions` | `functions/tools/` only (does **not** include `mcp.json`) |
|
||||
| `mcp_config` | `functions/mcp.json` only (merged) |
|
||||
|
||||
```sh
|
||||
loki --install-from <git-url> --filter agents
|
||||
loki --install-from <git-url> --filter mcp_config
|
||||
```
|
||||
|
||||
REPL form:
|
||||
|
||||
```
|
||||
.install remote <git-url> --filter agents
|
||||
```
|
||||
|
||||
Note that `--filter functions` is intentionally narrow. It installs the global tools under `functions/tools/` and
|
||||
**not** `mcp.json`. To install just the MCP config, use `--filter mcp_config`. To install both, use no filter.
|
||||
|
||||
---
|
||||
|
||||
## Conflict resolution
|
||||
|
||||
When an install file would overwrite an existing local file with different contents, you have several options.
|
||||
|
||||
### Plain files (agents, roles, macros, tools)
|
||||
|
||||
In a terminal (TTY), Loki prompts per conflicting file:
|
||||
|
||||
| Option | Effect |
|
||||
|----------------|------------------------------------------------------------------------------|
|
||||
| `keep` | Skip this file; keep your local copy. |
|
||||
| `replace` | Overwrite with the remote file. |
|
||||
| `keep-all` | Skip this file and all remaining conflicts in this install. |
|
||||
| `replace-all` | Overwrite this file and all remaining conflicts in this install. |
|
||||
| `abort` | Stop the install. Files already written stay; they are not rolled back. |
|
||||
|
||||
To skip prompts and replace everything, pass `--install-force` on the CLI or `--force` in the REPL form. In non-TTY
|
||||
mode (CI, piped, redirected stdin), the install will **abort** rather than silently overwrite. To overwrite
|
||||
non-interactively, you must pass `--install-force`.
|
||||
|
||||
### Identical content is not a conflict
|
||||
|
||||
If a remote file's bytes match your local copy exactly, Loki silently treats it as `identical` and skips it. Re-running
|
||||
the same install is idempotent.
|
||||
|
||||
### File-mode (`+x`) handling
|
||||
|
||||
Scripts written into `functions/tools/` are automatically marked executable based on extension. Bash (`.sh`), Python
|
||||
(`.py`), and TypeScript (`.ts`) files get `chmod 0o755` on Unix. This applies to both new files and replaced conflicts.
|
||||
|
||||
---
|
||||
|
||||
## MCP configuration merge
|
||||
|
||||
`functions/mcp.json` is the exception to the per-file conflict model. Instead of replacing your local file outright,
|
||||
Loki **merges** the remote `mcp.json` into your existing one.
|
||||
|
||||
### Merge behavior
|
||||
|
||||
- **New server names** (present in remote, absent locally) are added.
|
||||
- **Identical server entries** (same JSON shape locally and remotely) are silently kept.
|
||||
- **Conflicting server entries** prompt with three options:
|
||||
|
||||
| Option | Effect |
|
||||
|-------------------------------------|---------------------------------------------------------------------|
|
||||
| `keep local` | Leave your existing entry alone. |
|
||||
| `take remote` | Overwrite with the remote entry. |
|
||||
| `rename remote as "<name>-remote"` | Insert the remote entry under a suffixed key. If the suffixed key is also taken, Loki appends `-2`, `-3`, etc. |
|
||||
| `abort merge` | Stop the merge with an error. No partial write. |
|
||||
|
||||
With `--install-force`, every conflict is resolved by taking the remote entry. In non-TTY mode without
|
||||
`--install-force`, the merge aborts before writing.
|
||||
|
||||
### Validation
|
||||
|
||||
Each added, replaced, or renamed entry is validated against the MCP server schema (e.g., `stdio` servers must have a
|
||||
`command`; `http`/`sse` servers must have a `url`) before the merged file is written. A validation failure aborts the
|
||||
install. Loki will **never** write a partially-validated `mcp.json`.
|
||||
|
||||
### Atomic write
|
||||
|
||||
The merged file is written to a tempfile (`mcp.json.tmp`) and then renamed atomically over the target. This guarantees
|
||||
that a crash or interrupt mid-write will not corrupt your existing `mcp.json`.
|
||||
|
||||
---
|
||||
|
||||
## Secrets handling
|
||||
|
||||
MCP server entries (and other config files) can reference vault secrets with `{{SECRET_NAME}}` placeholders. After the
|
||||
install completes, Loki scans the resulting `mcp.json` for placeholders that are not yet in your vault and either:
|
||||
|
||||
- **In a TTY:** prompts you, one secret at a time, whether to add it to the vault now. On the first "Yes", Loki
|
||||
initializes the vault password file (if needed). On "No", the secret is deferred and reported at the end.
|
||||
- **In a non-TTY environment:** skips prompts entirely; lists every missing secret in a final reminder block, with the
|
||||
commands you can run later (`loki --add-secret <NAME>` or `.vault add <NAME>`).
|
||||
|
||||
Both modes always print a final summary distinguishing secrets added from those deferred. See [Vault](Vault) for the
|
||||
full secrets workflow.
|
||||
|
||||
---
|
||||
|
||||
## Git authentication
|
||||
|
||||
Loki shells out to your local `git` binary to clone the remote repository. This means it inherits **your existing git
|
||||
authentication setup** with no additional configuration:
|
||||
|
||||
- **SSH URLs** (`git@github.com:owner/repo.git`) use your `ssh-agent`, `~/.ssh/config`, and SSH keys.
|
||||
- **HTTPS URLs** with private repos use your configured `credential.helper` (e.g., osxkeychain on macOS, libsecret on
|
||||
Linux, the GitHub CLI's helper).
|
||||
- **Public HTTPS URLs** require no auth.
|
||||
|
||||
If `git` is not on your `PATH`, you'll get a clear error message on the first install attempt.
|
||||
|
||||
---
|
||||
|
||||
## Publishing your own bundle
|
||||
|
||||
To share your Loki configuration, structure a git repo like the template above and push it to GitHub, GitLab, your
|
||||
self-hosted Forgejo instance, etc.
|
||||
|
||||
The easiest way to start is to **fork the official template repo**:
|
||||
[`loki-config-template`](https://github.com/Dark-Alex-17/loki-config-template). It includes a working sample of each
|
||||
asset type plus a `README` describing the customization workflow.
|
||||
|
||||
### Best practices
|
||||
|
||||
1. **Pin to tagged releases** so consumers can install with `#<tag>` for reproducibility.
|
||||
2. **Keep per-agent logic in `agents/<name>/`**. Only put global tools in `functions/tools/`.
|
||||
3. **Use `{{SECRET}}` placeholders** for any sensitive value in `mcp.json`. Never commit a real API key.
|
||||
4. **Document expected secrets** in your repo's `README` so users know which vault entries to add.
|
||||
5. **Avoid name collisions** with Loki's bundled assets (`agents/code-reviewer`, `agents/sql`, etc.) unless you
|
||||
specifically want to replace them. Pick distinctive names for your shared assets.
|
||||
6. **Test installs against a clean `LOKI_CONFIG_DIR`** before publishing. You can use:
|
||||
|
||||
```sh
|
||||
LOKI_CONFIG_DIR=$(mktemp -d) loki --install-from file:///path/to/your/clone --install-force
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## What is not shared
|
||||
|
||||
The following are **intentionally** outside the install-remote feature's scope:
|
||||
|
||||
- **`config.yaml`** (the global Loki config). It holds user-specific things like editor preference, vault password file
|
||||
path, client API keys, OAuth tokens, and similar. Merging a shared `config.yaml` would risk breaking auth or routing
|
||||
traffic somewhere unexpected. Settings that genuinely benefit from sharing (like default models) already belong
|
||||
inside individual agents' or roles' configs.
|
||||
- **Sessions, RAGs, agent runtime data.** These accumulate as you use Loki and aren't shareable in a meaningful way.
|
||||
- **`functions/bin/`:** Agent script binaries built at runtime, not part of the source layout.
|
||||
- **`functions/utils/`:** Utility scripts intentionally not part of the share contract.
|
||||
|
||||
If a team really needs synced global settings, the recommended approach is dotfiles-style symlinks or a sync tool.
|
||||
Those are designed for that problem and not for `loki --install-from`.
|
||||
|
||||
---
|
||||
|
||||
## Examples
|
||||
|
||||
### Install everything from a public repo
|
||||
|
||||
```sh
|
||||
loki --install-from https://github.com/Dark-Alex-17/loki-config-template
|
||||
```
|
||||
|
||||
### Pin to a release tag
|
||||
|
||||
```sh
|
||||
loki --install-from https://github.com/your-org/loki-config#v2.4.0
|
||||
```
|
||||
|
||||
### Install only the agents from a fork
|
||||
|
||||
```sh
|
||||
loki --install-from https://github.com/your-org/loki-config --filter agents
|
||||
```
|
||||
|
||||
### Install only the MCP servers, force-replace conflicts
|
||||
|
||||
```sh
|
||||
loki --install-from https://github.com/your-org/loki-config --filter mcp_config --install-force
|
||||
```
|
||||
|
||||
### From the REPL
|
||||
|
||||
```
|
||||
.install remote https://github.com/your-org/loki-config
|
||||
.install remote https://github.com/your-org/loki-config#main --filter agents
|
||||
.install remote https://github.com/your-org/loki-config --force
|
||||
```
|
||||
|
||||
### From a private SSH URL
|
||||
|
||||
```sh
|
||||
loki --install-from git@github.com:your-org/private-loki-config.git#v1.0.0
|
||||
```
|
||||
|
||||
### From a local file:// URL (testing your own template)
|
||||
|
||||
```sh
|
||||
loki --install-from file:///home/you/code/my-loki-config
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Troubleshooting
|
||||
|
||||
### "git failed: ..."
|
||||
|
||||
The remote clone failed. Loki passes git's stderr through verbatim. The most common causes:
|
||||
|
||||
- **Authentication required**: `fatal: could not read username/password` or `Permission denied (publickey)`. Verify
|
||||
your git credentials work for the same URL outside Loki by running `git clone <url>` manually.
|
||||
- **Network failure**: `Could not resolve host` or similar. Check connectivity.
|
||||
- **Invalid ref**: `couldn't find remote ref <ref>`. Verify the tag/branch/commit exists in the remote.
|
||||
|
||||
### "No recognized assets found in `<url>`"
|
||||
|
||||
The cloned repo has none of the recognized top-level directories. Make sure the repo's layout matches the
|
||||
[expected layout](#expected-repository-layout). A `README.md` and a `LICENSE` at the root are fine; they're just
|
||||
ignored.
|
||||
|
||||
### "Refusing to overwrite local file `<path>` non-interactively"
|
||||
|
||||
You're running in non-TTY mode (CI, piped, redirected stdin) and there's a conflict. Re-run with `--install-force` to
|
||||
overwrite, or run in an interactive terminal to be prompted.
|
||||
|
||||
### "MCP server '`<name>`' already exists locally. Refusing to merge non-interactively."
|
||||
|
||||
Same pattern as the above, but for the MCP merge. Use `--install-force` to take the remote entry for every conflict.
|
||||
|
||||
### Missing secrets after install
|
||||
|
||||
Add them via `loki --add-secret <NAME>` (CLI) or `.vault add <NAME>` (REPL). See [Vault](Vault) for details.
|
||||
|
||||
### `git` binary not found
|
||||
|
||||
Loki shells out to the system `git` binary. Install git for your platform and ensure it's on your `PATH`.
|
||||
Reference in New Issue
Block a user