diff --git a/README.md b/README.md index c3d9cf5..a87a57a 100644 --- a/README.md +++ b/README.md @@ -1,7 +1,6 @@ # Loki: All-in-one, batteries-included LLM CLI Tool ![Test](https://github.com/Dark-Alex-17/loki/actions/workflows/ci.yaml/badge.svg) -![LOC](https://tokei.rs/b1/github/Dark-Alex-17/loki?category=code) [![crates.io link](https://img.shields.io/crates/v/loki-ai.svg)](https://crates.io/crates/loki-ai) ![Release](https://img.shields.io/github/v/release/Dark-Alex-17/loki?color=%23c694ff) ![Crate.io downloads](https://img.shields.io/crates/d/loki-ai?label=Crate%20downloads) @@ -13,36 +12,36 @@ Agents, and More. It is designed to include a number of useful agents, roles, macros, and more so users can get up and running with Loki in as little time as possible. -![Agent example](./docs/images/agents/sql.gif) +![Agent example](https://raw.githubusercontent.com/wiki/Dark-Alex-17/loki/docs/images/agents/sql.gif) -Coming from [AIChat](https://github.com/sigoden/aichat)? Follow the [migration guide](./docs/AICHAT-MIGRATION.md) to get started. +Coming from [AIChat](https://github.com/sigoden/aichat)? Follow the [migration guide](https://github.com/Dark-Alex-17/loki/wiki/AIChat-Migration) to get started. ## Quick Links -* [AIChat Migration Guide](./docs/AICHAT-MIGRATION.md): Coming from AIChat? Follow the migration guide to get started. +* [AIChat Migration Guide](https://github.com/Dark-Alex-17/loki/wiki/AIChat-Migration): Coming from AIChat? Follow the migration guide to get started. * [Installation](#install): Install Loki * [Getting Started](#getting-started): Get started with Loki by doing first-run setup steps. -* [REPL](./docs/REPL.md): Interactive Read-Eval-Print Loop for conversational interactions with LLMs and Loki. - * [Custom REPL Prompt](./docs/REPL-PROMPT.md): Customize the REPL prompt to provide useful contextual information. -* [Vault](./docs/VAULT.md): Securely store and manage sensitive information such as API keys and credentials. -* [Shell Integrations](./docs/SHELL-INTEGRATIONS.md): Seamlessly integrate Loki with your shell environment for enhanced command-line assistance. -* [Function Calling](./docs/function-calling/TOOLS.md#Tools): Leverage function calling capabilities to extend Loki's functionality with custom tools - * [Creating Custom Tools](./docs/function-calling/CUSTOM-TOOLS.md): You can create your own custom tools to enhance Loki's capabilities. - * [Create Custom Python Tools](./docs/function-calling/CUSTOM-TOOLS.md#custom-python-based-tools) - * [Create Custom TypeScript Tools](./docs/function-calling/CUSTOM-TOOLS.md#custom-typescript-based-tools) - * [Create Custom Bash Tools](./docs/function-calling/CUSTOM-BASH-TOOLS.md) - * [Bash Prompt Utilities](./docs/function-calling/BASH-PROMPT-HELPERS.md) -* [First-Class MCP Server Support](./docs/function-calling/MCP-SERVERS.md): Easily connect and interact with MCP servers for advanced functionality. -* [Macros](./docs/MACROS.md): Automate repetitive tasks and workflows with Loki "scripts" (macros). -* [RAG](./docs/RAG.md): Retrieval-Augmented Generation for enhanced information retrieval and generation. -* [Sessions](/docs/SESSIONS.md): Manage and persist conversational contexts and settings across multiple interactions. -* [Roles](./docs/ROLES.md): Customize model behavior for specific tasks or domains. -* [Agents](/docs/AGENTS.md): Leverage AI agents to perform complex tasks and workflows, including sub-agent spawning, teammate messaging, and user interaction tools. - * [Todo System](./docs/TODO-SYSTEM.md): Built-in task tracking for improved agent reliability with smaller models. -* [Environment Variables](./docs/ENVIRONMENT-VARIABLES.md): Override and customize your Loki configuration at runtime with environment variables. -* [Client Configurations](./docs/clients/CLIENTS.md): Configuration instructions for various LLM providers. - * [Authentication (API Key & OAuth)](./docs/clients/CLIENTS.md#authentication): Authenticate with API keys or OAuth for subscription-based access. - * [Patching API Requests](./docs/clients/PATCHES.md): Learn how to patch API requests for advanced customization. -* [Custom Themes](./docs/THEMES.md): Change the look and feel of Loki to your preferences with custom themes. +* [REPL](https://github.com/Dark-Alex-17/loki/wiki/REPL): Interactive Read-Eval-Print Loop for conversational interactions with LLMs and Loki. + * [Custom REPL Prompt](https://github.com/Dark-Alex-17/loki/wiki/REPL-Prompt): Customize the REPL prompt to provide useful contextual information. +* [Vault](https://github.com/Dark-Alex-17/loki/wiki/Vault): Securely store and manage sensitive information such as API keys and credentials. +* [Shell Integrations](https://github.com/Dark-Alex-17/loki/wiki/Shell-Integrations): Seamlessly integrate Loki with your shell environment for enhanced command-line assistance. +* [Function Calling](https://github.com/Dark-Alex-17/loki/wiki/Tools): Leverage function calling capabilities to extend Loki's functionality with custom tools + * [Creating Custom Tools](https://github.com/Dark-Alex-17/loki/wiki/Custom-Tools): You can create your own custom tools to enhance Loki's capabilities. + * [Create Custom Python Tools](https://github.com/Dark-Alex-17/loki/wiki/Custom-Tools#custom-python-based-tools) + * [Create Custom TypeScript Tools](https://github.com/Dark-Alex-17/loki/wiki/Custom-Tools#custom-typescript-based-tools) + * [Create Custom Bash Tools](https://github.com/Dark-Alex-17/loki/wiki/Custom-Bash-Tools) + * [Bash Prompt Utilities](https://github.com/Dark-Alex-17/loki/wiki/Bash-Prompt-Helpers) +* [First-Class MCP Server Support](https://github.com/Dark-Alex-17/loki/wiki/MCP-Servers): Easily connect and interact with MCP servers for advanced functionality. +* [Macros](https://github.com/Dark-Alex-17/loki/wiki/Macros): Automate repetitive tasks and workflows with Loki "scripts" (macros). +* [RAG](https://github.com/Dark-Alex-17/loki/wiki/RAG): Retrieval-Augmented Generation for enhanced information retrieval and generation. +* [Sessions](https://github.com/Dark-Alex-17/loki/wiki/Sessions): Manage and persist conversational contexts and settings across multiple interactions. +* [Roles](https://github.com/Dark-Alex-17/loki/wiki/Roles): Customize model behavior for specific tasks or domains. +* [Agents](https://github.com/Dark-Alex-17/loki/wiki/Agents): Leverage AI agents to perform complex tasks and workflows, including sub-agent spawning, teammate messaging, and user interaction tools. + * [Todo System](https://github.com/Dark-Alex-17/loki/wiki/TODO-System): Built-in task tracking for improved agent reliability with smaller models. +* [Environment Variables](https://github.com/Dark-Alex-17/loki/wiki/Environment-Variables): Override and customize your Loki configuration at runtime with environment variables. +* [Client Configurations](https://github.com/Dark-Alex-17/loki/wiki/Clients): Configuration instructions for various LLM providers. + * [Authentication (API Key & OAuth)](https://github.com/Dark-Alex-17/loki/wiki/Clients#authentication): Authenticate with API keys or OAuth for subscription-based access. + * [Patching API Requests](https://github.com/Dark-Alex-17/loki/wiki/Patches): Learn how to patch API requests for advanced customization. +* [Custom Themes](https://github.com/Dark-Alex-17/loki/wiki/Themes): Change the look and feel of Loki to your preferences with custom themes. * [History](#history): A history of how Loki came to be. ## Prerequisites @@ -154,7 +153,7 @@ loki --list-secrets ### Authentication Each client in your configuration needs authentication (with a few exceptions; e.g. ollama). Most clients use an API key -(set via `api_key` in the config or through the [vault](./docs/VAULT.md)). For providers that support OAuth (e.g. Claude Pro/Max +(set via `api_key` in the config or through the [vault](https://github.com/Dark-Alex-17/loki/wiki/Vault)). For providers that support OAuth (e.g. Claude Pro/Max subscribers, Google Gemini), you can authenticate with your existing subscription instead: ```yaml @@ -170,7 +169,7 @@ loki --authenticate my-claude-oauth # Or via the REPL: .authenticate ``` -For full details, see the [authentication documentation](./docs/clients/CLIENTS.md#authentication). +For full details, see the [authentication documentation](https://github.com/Dark-Alex-17/loki/wiki/Clients#authentication). ### Tab-Completions You can also enable tab completions to make using Loki easier. To do so, add the following to your shell profile: @@ -247,7 +246,7 @@ shown below: | Setting | Description | |-----------------|-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------| -| `repl_prelude` | This setting lets you specify a default `session` or `role` to use when starting Loki in [REPL](./docs/REPL.md) mode.
Values can be | +| `repl_prelude` | This setting lets you specify a default `session` or `role` to use when starting Loki in [REPL](https://github.com/Dark-Alex-17/loki/wiki/REPL) mode.
Values can be | | `cmd_prelude` | This setting lets you specify a default `session` or `role` to use when running one-off queries in Loki via the CLI.
Values can be | | `agent_session` | This setting is used to specify a default session that all agents should start into, unless otherwise specified in the agent configuration. (e.g. `temp`, `default`) | diff --git a/assets/functions/tools/query_jira_issues.sh b/assets/functions/tools/query_jira_issues.sh deleted file mode 100755 index 4160be0..0000000 --- a/assets/functions/tools/query_jira_issues.sh +++ /dev/null @@ -1,11 +0,0 @@ -#!/usr/bin/env bash -set -e - -# @meta require-tools jira -# @describe Query for jira issues using a Jira Query Language (JQL) query -# @option --jql-query! The Jira Query Language query to execute -# @env LLM_OUTPUT=/dev/stdout The output path - -main() { - jira issue ls -q "$argc_jql_query" --plain >> "$LLM_OUTPUT" -} \ No newline at end of file diff --git a/docs/AGENTS.md b/docs/AGENTS.md deleted file mode 100644 index e9782d8..0000000 --- a/docs/AGENTS.md +++ /dev/null @@ -1,775 +0,0 @@ -# Agents - -Agents in Loki follow the same style as OpenAI's GPTs. They consist of 3 parts: - -* [Role](./ROLES.md) - Tell the LLM how to behave -* [RAG](./RAG.md) - Pre-built knowledge bases specifically for the agent -* [Function Calling](./function-calling/TOOLS.md#tools) ([#2](./function-calling/MCP-SERVERS.md)) - Extends the functionality of the LLM through custom functions it can call - -![Agent example](./images/agents/sql.gif) - -Agent configuration files are stored in the `agents` subdirectory of your Loki configuration directory. The location of -this directory varies between systems so you can use the following command to locate yours: - -```shell -loki --info | grep 'agents_dir' | awk '{print $2}' -``` - -If you're looking for more example agents, refer to the [built-in agents](../assets/agents). - -## Quick Links - -- [Directory Structure](#directory-structure) -- [Metadata](#1-metadata) -- [2. Define the Instructions](#2-define-the-instructions) - - [Static Instructions](#static-instructions) - - [Special Variables](#special-variables) - - [User-Defined Variables](#user-defined-variables) - - [Dynamic Instructions](#dynamic-instructions) - - [Variables](#variables) -- [3. Initializing RAG](#3-initializing-rag) -- [4. Building Tools for Agents](#4-building-tools-for-agents) - - [Limitations](#limitations) - - [.env File Support](#env-file-support) - - [Python-Based Agent Tools](#python-based-agent-tools) - - [Bash-Based Agent Tools](#bash-based-agent-tools) - - [TypeScript-Based Agent Tools](#typescript-based-agent-tools) -- [5. Conversation Starters](#5-conversation-starters) -- [6. Todo System & Auto-Continuation](#6-todo-system--auto-continuation) -- [7. Sub-Agent Spawning System](#7-sub-agent-spawning-system) - - [Configuration](#spawning-configuration) - - [Spawning & Collecting Agents](#spawning--collecting-agents) - - [Task Queue with Dependencies](#task-queue-with-dependencies) - - [Active Task Dispatch](#active-task-dispatch) - - [Output Summarization](#output-summarization) - - [Teammate Messaging](#teammate-messaging) - - [Runaway Safeguards](#runaway-safeguards) -- [8. User Interaction Tools](#8-user-interaction-tools) - - [Available Tools](#user-interaction-available-tools) - - [Escalation (Sub-Agent to User)](#escalation-sub-agent-to-user) -- [9. Auto-Injected Prompts](#9-auto-injected-prompts) -- [Built-In Agents](#built-in-agents) - - ---- - -## Directory Structure -Agent configurations often have the following directory structure: - -``` -/agents - └── my-agent - ├── config.yaml - ├── tools.sh - or - ├── tools.py - or - ├── tools.ts -``` - -This means that agent configurations often are only two files: the agent configuration file (`config.yaml`), and the -tool definitions (`agents/my-agent/tools.sh`, `tools.py`, or `tools.ts`). - -To see a full example configuration file, refer to the [example agent config file](../config.agent.example.yaml). - -The best way to understand how an agent is built is to go step by step in the following manner: - ---- - -## 1. Metadata -Agent configurations have the following settings available to customize each agent: - -```yaml -# Model Configuration -model: openai:gpt-4o # Specify the LLM to use -temperature: null # Set default temperature parameter, range (0, 1) -top_p: null # Set default top-p parameter, with a range of (0, 1) or (0, 2), depending on the model -# Agent Metadata Configuration -agent_session: null # Set a session to use when starting the agent. (e.g. temp, default); defaults to globally set agent_session -# Agent Configuration -name: # Name of the agent, used in the UI and logs -description: # Description of the agent, used in the UI -version: 1 # Version of the agent -# Function Calling Configuration -mcp_servers: # Optional list of MCP servers that the agent utilizes - - github # Corresponds to the name of an MCP server in the `/functions/mcp.json` file -global_tools: # Optional list of additional global tools to enable for the agent; i.e. not tools specific to the agent - - web_search - - fs - - python -# Todo System & Auto-Continuation (see "Todo System & Auto-Continuation" section below) -auto_continue: false # Enable automatic continuation when incomplete todos remain -max_auto_continues: 10 # Maximum continuation attempts before stopping -inject_todo_instructions: true # Inject todo tool instructions into system prompt -continuation_prompt: null # Custom prompt for continuations (optional) -# Sub-Agent Spawning (see "Sub-Agent Spawning System" section below) -can_spawn_agents: false # Enable spawning child agents -max_concurrent_agents: 4 # Max simultaneous child agents -max_agent_depth: 3 # Max nesting depth (prevents runaway) -inject_spawn_instructions: true # Inject spawning instructions into system prompt -summarization_model: null # Model for summarizing sub-agent output (e.g. 'openai:gpt-4o-mini') -summarization_threshold: 4000 # Char count above which sub-agent output is summarized -escalation_timeout: 300 # Seconds sub-agents wait for escalated user input (default: 5 min) -``` - -As mentioned previously: Agents utilize function calling to extend a model's capabilities. However, agents operate in -isolated environment, so in order for an agent to use a tool or MCP server that you have defined globally, you must -explicitly state which tools and/or MCP servers the agent uses. Otherwise, it is assumed that the agent doesn't use any -tools outside its own custom defined tools. - -And if you don't define a `agents/my-agent/tools.sh`, `agents/my-agent/tools.py`, or `agents/my-agent/tools.ts`, then the agent is really just a -`role`. - -You'll notice there are no settings for agent-specific tooling. This is because they are handled separately and -automatically. See the [Building Tools for Agents](#4-building-tools-for-agents) section below for more information. - -To see a full example configuration file, refer to the [example agent config file](../config.agent.example.yaml). - -## 2. Define the Instructions -At their heart, agents function similarly to roles in that they tell the model how to behave. Agent configuration files -have the following settings for the instruction definitions: - -```yaml -dynamic_instructions: # Whether to use dynamically generated instructions for the agent; if false, static instructions are used. False by default. -instructions: # Static instructions for the LLM; These are ignored if dynamic instructions are used -variables: # An array of optional variables that the agent expects and uses -``` - -### Static Instructions -By default, Loki agents use statically defined instructions. Think of them as being identical to the instructions for a -[role](./ROLES.md#instructions), because they virtually are. - -**Example:** -```yaml -instructions: | - You are an AI agent designed to demonstrate agentic capabilities -``` - -Just like roles, agents support variable interpolation at runtime. There's two types of variables that can be -interpolated into the instructions at runtime: special variables (like roles have), and user-defined variables. Just -like roles, variables are interpolated into your instructions anywhere Loki sees the `{{variable}}` syntax. - -#### Special Variables -The following special variables are provided by Loki at runtime and can be injected into your agent's instructions: - -| Name | Description | Example | -|-----------------|---------------------------------------------------------------------|----------------------------| -| `__os__` | Operating system name | `linux` | -| `__os_family__` | Operating system family | `unix` | -| `__arch__` | System architecture | `x86_64` | -| `__shell__` | The current user's default shell | `bash` | -| `__locale__` | The current user's preferred language and region settings | `en-US` | -| `__now__` | Current timestamp in ISO 8601 format | `2025-11-07T10:15:44.268Z` | -| `__cwd__` | The current working directory | `/tmp` | -| `__tools__` | A list of the enabled tools (global + mcp servers + agent-specific) | | - -#### User-Defined Variables -Agents also support user-defined variables that can be interpolated into the instructions, and are made available to any -agent-specific tools you define (see [Building Tools for Agents](#4-building-tools-for-agents) for more details on how to -create agent-specific tooling). - -The `variables` setting in an agent's config has the following fields: - -| Field | Required | Description | -|---------------|----------|----------------------------------------------------------------------------------------------------| -| `name` | * | The name of the variable | -| `description` | * | The description of the field | -| `default` | | A default value for the field. If left undefined, the user will be prompted for a value at runtime | - -These variables can be referenced in both the agent's instructions, and in the tool definitions via `LLM_AGENT_VAR_`. - -**Example:** -```yaml -instructions: | - You are an agent who answers questions about a user's system. - - - {{__tools__}} - - - - os: {{__os__}} - os_family: {{__os_family__}} - arch: {{__arch__}} - shell: {{__shell__}} - locale: {{__locale__}} - now: {{__now__}} - cwd: {{__cwd__}} - - - - username: {{username}} - -variables: - - name: username # Accessible from the tool definitions via the `LLM_AGENT_VAR_USERNAME` environment variable - description: Your user name -``` - -### Dynamic Instructions -Sometimes you may find it useful to dynamically generate instructions on startup. Whether that be via a call to Loki -itself to generate them, or by some other means. Loki supports this type of behavior using a special function defined -in your `agents/my-agent/tools.py`, `agents/my-agent/tools.sh`, or `agents/my-agent/tools.ts`. - -**Example: Instructions for a JSON-reader agent that specializes on each JSON input it receives** -`agents/json-reader/tools.py`: -```python -import json -from pathlib import Path -from genson import SchemaBuilder - -def _instructions(): - """Generates instructions for the agent dynamically""" - value = input("Enter a JSON file path OR paste raw JSON: ").strip() - if not value: - raise SystemExit("A file path or JSON string is required.") - - p = Path(value) - if p.exists() and p.is_file(): - json_file_path = str(p.resolve()) - json_text = p.read_text(encoding="utf-8") - else: - try: - json.loads(value) - except json.JSONDecodeError as e: - raise SystemExit(f"Input is neither a file nor valid JSON.\n{e}") - json_file_path = "" - json_text = value - - try: - data = json.loads(json_text) - except json.JSONDecodeError as e: - raise SystemExit(f"Provided content is not valid JSON.\n{e}") - - builder = SchemaBuilder() - builder.add_object(data) - json_schema = builder.to_schema() - return f""" - You are an AI agent that can view and filter JSON data with jq. - - ## Context - json_file_path: {json_file_path} - json_schema: {json.dumps(json_schema, indent=2)} - """ -``` - -or - -`agents/json-reader/tools.sh`: -```bash -#!/usr/bin/env bash -set -e - -# @meta require-tools jq,genson -# @env LLM_OUTPUT=/dev/stdout The output path - -# @cmd Generates instructions for the agent dynamically -_instructions() { - read -r -p "Enter a JSON file path OR paste raw JSON: " value - - if [[ -z "${value}" ]]; then - echo "A file path or JSON string is required" >&2 - exit 1 - fi - json_file_path="" - inline_temp="" - cleanup() { - [[ -n "${inline_temp:-}" && -f "${inline_temp}" ]] && rm -f "${inline_temp}" - } - trap cleanup EXIT - - if [[ -f "${value}" ]]; then - json_file_path="$(realpath "${value}")" - if ! jq empty "${json_file_path}" >/dev/null 2>&1; then - echo "Error: File does not contain valid JSON: ${json_file_path}" >&2 - exit 1 - fi - else - inline_temp="$(mktemp)" - printf "%s" "${value}" > "${inline_temp}" - if ! jq empty "${inline_temp}" >/dev/null 2>&1; then - echo "Error: Input is neither a file nor valid JSON." >&2 - exit 1 - fi - json_file_path="" - fi - - source_file="${json_file_path}" - if [[ "${json_file_path}" == "" ]]; then - source_file="${inline_temp}" - fi - - json_schema="$(genson < "${source_file}" | jq -c '.')" - cat <> "$LLM_OUTPUT" -You are an AI agent that can view and filter JSON data with jq. - -## Context -json_file_path: ${json_file_path} -json_schema: ${json_schema} -EOF -} -``` - -For more information on how to create custom tools for your agent and the structure of the `agent/my-agent/tools.sh`, -`agent/my-agent/tools.py`, or `agent/my-agent/tools.ts` files, refer to the [Building Tools for Agents](#4-building-tools-for-agents) section below. - -#### Variables -All the same variable interpolations supported by static instructions is also supported by dynamic instructions. For -more information on what variables are available and how to use them, refer to the [Special Variables](#special-variables) -and [User-Defined Variables](#user-defined-variables) sections above. - -## 3. Initializing RAG -Each agent you create also has a dedicated knowledge base that adds additional context to your queries and helps the LLM -answer queries effectively. The documents to load into RAG are defined in the `documents` array of your agent -configuration file: - -```yaml -documents: - - https://www.ohdsi.org/data-standardization/ - - https://github.com/OHDSI/Vocabulary-v5.0/wiki/** - - OMOPCDM_ddl.sql # Relative path to agent (i.e. file lives at '/agents/my-agent/OMOPCDM_ddl.sql') -``` - -These documents use the same syntax as those you'd define when constructing RAG normally. To see all the available types -of documents that Loki supports and how to use custom document loaders, refer to the [RAG documentation](./RAG.md#supported-document-sources). - -Anytime your agent starts up, it will automatically be using the RAG you've defined here. - -## 4. Building Tools for Agents -Building tools for agents is virtually identical to building custom tools, with one slight difference: instead of -defining a single function that gets executed at runtime (e.g. `main` for bash tools and `run` for Python tools), agent -tools define a number of *subcommands*. - -### Limitations -You can only utilize one of: a bash-based `/agents/my-agent/tools.sh`, a Python-based -`/agents/my-agent/tools.py`, or a TypeScript-based `/agents/my-agent/tools.ts`. -However, if it's easier to achieve a task in one language vs the other, -you're free to define other scripts in your agent's configuration directory and reference them from the main -tools file. **Any scripts *not* named `tools.{py,sh,ts}` will not be picked up by Loki's compiler**, meaning they -can be used like any other set of scripts. - -It's important to keep in mind the following: - -* **Do not give agents the same name as an executable**. Loki compiles the tools for each agent into a binary that it - temporarily places on your path during execution. If you have a binary with the same name as your agent, then your - shell may execute the existing binary instead of your agent's tools -* **`LLM_ROOT_DIR` points to the agent's configuration directory**. This is where agents differ slightly from normal - tools: The `LLM_ROOT_DIR` environment variable does *not* point to the `functions/tools` directory like it does in - global tools. Instead, it points to the agent's configuration directory, making it easier to source scripts and other - miscellaneous files - -### .env File Support -When Loki loads an agent, it will also search the agent's configuration directory for a `.env` file. If found, all -environment variables defined in the file will be made available to the agent's tools. - -### Python-Based Agent Tools -Python-based tools are defined exactly the same as they are for custom tool definitions. The only difference is that -instead of a single `run` function, you define as many as you like with whatever arguments you like. - -**Example:** -`agents/my-agent/tools.py` -```python -import urllib.request - -def get_ip_info(): - """ - Get your IP information - """ - with urllib.request.urlopen("https://httpbin.org/ip") as response: - data = response.read() - return data.decode('utf-8') - -def get_ip_address_from_aws(): - """ - Find your public IP address using AWS - """ - with urllib.request.urlopen("https://checkip.amazonaws.com") as response: - data = response.read() - return data.decode('utf-8') -``` - -Loki automatically compiles these as separate functions for the LLM to call. No extra work is needed. Just make sure you -follow all the same steps to define each function as you would when creating custom Python tools. - -For more information on how to build tools in Python, refer to the [custom Python tools documentation](./function-calling/CUSTOM-TOOLS.md#custom-python-based-tools) - -### Bash-Based Agent Tools -Bash-based agent tools are virtually identical to custom bash tools, with only one difference. Instead of defining a -single entrypoint via the `main` function, you actually define as many subcommands as you like. - -**Example:** -`agents/my-agent/tools.sh` -```bash -#!/usr/bin/env bash - -# @env LLM_OUTPUT=/dev/stdout The output path -# @describe Discover network information about your computer and its place in the internet - -# Use the `@cmd` annotation to define subcommands for your script. -# @cmd Get your IP information -get_ip_info() { - curl -fsSL https://httpbin.org/ip >> "$LLM_OUTPUT" -} - -# @cmd Find your public IP address using AWS -get_ip_address_from_aws() { - curl -fsSL https://checkip.amazonaws.com >> "$LLM_OUTPUT" -} -``` -To compile the script so it's executable and testable: -```bash -$ loki --build-tools -``` - -Then you can execute your script (assuming your current working directory is `agents/my-agent`): -```bash -$ ./tools.sh get_ip_info -$ ./tools.sh get_ip_address_from_aws -``` - -All other special annotations (`@env`, `@arg`, `@option` `@flags`) apply to subcommands as well, so be sure to follow -the same syntax ad formatting as is used to create custom bash tools globally. - -For more information on how to write, [build and test](function-calling/CUSTOM-BASH-TOOLS.md#execute-and-test-your-bash-tools) tools in bash, refer to the -[custom bash tools documentation](function-calling/CUSTOM-BASH-TOOLS.md). - -### TypeScript-Based Agent Tools -TypeScript-based agent tools work exactly the same as TypeScript global tools. Instead of a single `run` function, -you define as many exported functions as you like. Non-exported functions are private helpers and are invisible to the -LLM. - -**Example:** -`agents/my-agent/tools.ts` -```typescript -/** - * Get your IP information - */ -export async function get_ip_info(): Promise { - const resp = await fetch("https://httpbin.org/ip"); - return await resp.text(); -} - -/** - * Find your public IP address using AWS - */ -export async function get_ip_address_from_aws(): Promise { - const resp = await fetch("https://checkip.amazonaws.com"); - return await resp.text(); -} - -// Non-exported helper — invisible to the LLM -function formatResponse(data: string): string { - return data.trim(); -} -``` - -Loki automatically compiles each exported function as a separate tool for the LLM to call. Just make sure you -follow the same JSDoc and parameter conventions as you would when creating custom TypeScript tools. - -TypeScript agent tools also support dynamic instructions via an exported `_instructions()` function: - -```typescript -import { readFileSync } from "fs"; - -/** - * Generates instructions for the agent dynamically - */ -export function _instructions(): string { - const schema = readFileSync("schema.json", "utf-8"); - return `You are an AI agent that works with the following schema:\n${schema}`; -} -``` - -For more information on how to build tools in TypeScript, refer to the [custom TypeScript tools documentation](function-calling/CUSTOM-TOOLS.md#custom-typescript-based-tools). - -## 5. Conversation Starters -It's often helpful to also have some conversation starters so users know what kinds of things the agent is capable of -doing. These are available in the REPL via the `.starter` command and are selectable. - -They are defined using the `conversation_starters` setting in your agent's configuration file: - -**Example:** -`agents/my-agent/config.yaml`: -```yaml -conversation_starters: - - What is my username? - - What is my current shell? - - What is my ip? - - How much disk space is left on my PC?? - - How to create an agent? -``` - -![Example Conversation Starters](./images/agents/conversation-starters.gif) - -## 6. Todo System & Auto-Continuation - -Loki includes a built-in task tracking system designed to improve the reliability of agents, especially when using -smaller language models. The Todo System helps models: - -- Break complex tasks into manageable steps -- Track progress through multi-step workflows -- Automatically continue work until all tasks are complete - -### Quick Configuration - -```yaml -# agents/my-agent/config.yaml -auto_continue: true # Enable auto-continuation -max_auto_continues: 10 # Max continuation attempts -inject_todo_instructions: true # Include the default todo instructions into prompt -``` - -### How It Works - -1. When `inject_todo_instructions` is enabled, agents receive instructions on using five built-in tools: - - `todo__init`: Initialize a todo list with a goal - - `todo__add`: Add a task to the list - - `todo__done`: Mark a task complete - - `todo__list`: View current todo state - - `todo__clear`: Clear the entire todo list and reset the goal - - These instructions are a reasonable default that detail how to use Loki's To-Do System. If you wish, - you can disable the injection of the default instructions and specify your own instructions for how - to use the To-Do System into your main `instructions` for the agent. - -2. When `auto_continue` is enabled and the model stops with incomplete tasks, Loki automatically sends a - continuation prompt with the current todo state, nudging the model to continue working. - -3. This continues until all tasks are done or `max_auto_continues` is reached. - -### When to Use - -- Multistep tasks where the model might lose track -- Smaller models that need more structure -- Workflows requiring guaranteed completion of all steps - -For complete documentation including all configuration options, tool details, and best practices, see the -[Todo System Guide](./TODO-SYSTEM.md). - -## 7. Sub-Agent Spawning System - -Loki agents can spawn and manage child agents that run **in parallel** as background tasks inside the same process. -This enables orchestrator-style agents that delegate specialized work to other agents, similar to how tools like -Claude Code or OpenCode handle complex multi-step tasks. - -For a working example of an orchestrator agent that uses sub-agent spawning, see the built-in -[sisyphus](../assets/agents/sisyphus) agent. For an example of the teammate messaging pattern with parallel sub-agents, -see the [code-reviewer](../assets/agents/code-reviewer) agent. - -### Spawning Configuration - -| Setting | Type | Default | Description | -|-----------------------------|---------|---------------|--------------------------------------------------------------------------------| -| `can_spawn_agents` | boolean | `false` | Enable this agent to spawn child agents | -| `max_concurrent_agents` | integer | `4` | Maximum number of child agents that can run simultaneously | -| `max_agent_depth` | integer | `3` | Maximum nesting depth for sub-agents (prevents runaway spawning chains) | -| `inject_spawn_instructions` | boolean | `true` | Inject the default spawning instructions into the agent's system prompt | -| `summarization_model` | string | current model | Model to use for summarizing long sub-agent output (e.g. `openai:gpt-4o-mini`) | -| `summarization_threshold` | integer | `4000` | Character count above which sub-agent output is summarized before returning | -| `escalation_timeout` | integer | `300` | Seconds a sub-agent waits for an escalated user interaction response | - -**Example configuration:** -```yaml -# agents/my-orchestrator/config.yaml -can_spawn_agents: true -max_concurrent_agents: 6 -max_agent_depth: 2 -inject_spawn_instructions: true -summarization_model: openai:gpt-4o-mini -summarization_threshold: 3000 -escalation_timeout: 600 -``` - -### Spawning & Collecting Agents - -When `can_spawn_agents` is enabled, the agent receives tools for spawning and managing child agents: - -| Tool | Description | -|------------------|-------------------------------------------------------------------------| -| `agent__spawn` | Spawn a child agent in the background. Returns an agent ID immediately. | -| `agent__check` | Non-blocking check: is the agent done? Returns `PENDING` or the result. | -| `agent__collect` | Blocking wait: wait for an agent to finish, return its output. | -| `agent__list` | List all spawned agents and their status. | -| `agent__cancel` | Cancel a running agent by ID. | - -The core pattern is **Spawn -> Continue -> Collect**: - -``` -# 1. Spawn agents in parallel (returns IDs immediately) -agent__spawn --agent explore --prompt "Find auth middleware patterns in src/" -agent__spawn --agent explore --prompt "Find error handling patterns in src/" - -# 2. Continue your own work while they run - -# 3. Check if done (non-blocking) -agent__check --id agent_explore_a1b2c3d4 - -# 4. Collect results when ready (blocking) -agent__collect --id agent_explore_a1b2c3d4 -agent__collect --id agent_explore_e5f6g7h8 -``` - -Any agent defined in your `/agents/` directory can be spawned as a child. Child agents: -- Run in a fully isolated environment (separate session, config, and tools) -- Have their output suppressed from the terminal (no spinner, no tool call logging) -- Return their accumulated output to the parent when collected - -### Task Queue with Dependencies - -For complex workflows where tasks have ordering requirements, the spawning system includes a dependency-aware -task queue: - -| Tool | Description | -|------------------------|-----------------------------------------------------------------------------| -| `agent__task_create` | Create a task with optional dependencies and auto-dispatch agent. | -| `agent__task_list` | List all tasks with their status, dependencies, and assignments. | -| `agent__task_complete` | Mark a task done. Returns newly unblocked tasks and auto-dispatches agents. | -| `agent__task_fail` | Mark a task as failed. Dependents remain blocked. | - -``` -# Create tasks with dependency ordering -agent__task_create --subject "Explore existing patterns" -agent__task_create --subject "Implement feature" --blocked_by ["task_1"] -agent__task_create --subject "Write tests" --blocked_by ["task_2"] - -# Mark tasks complete to unblock dependents -agent__task_complete --task_id task_1 -``` - -### Active Task Dispatch - -Tasks can optionally specify an agent to auto-spawn when the task becomes runnable: - -``` -agent__task_create \ - --subject "Implement the auth module" \ - --blocked_by ["task_1"] \ - --agent coder \ - --prompt "Implement auth module based on patterns found in task_1" -``` - -When `task_1` completes and the dependent task becomes unblocked, an agent is automatically spawned with the -specified prompt. No manual intervention needed. This enables fully automated multi-step pipelines. - -### Output Summarization - -When a child agent produces long output, it can be automatically summarized before returning to the parent. -This keeps parent context windows manageable. - -- If the output exceeds `summarization_threshold` characters (default: 4000), it is sent through an LLM - summarization pass -- The `summarization_model` setting lets you use a cheaper/faster model for summarization (e.g. `gpt-4o-mini`) -- If `summarization_model` is not set, the parent's current model is used -- The summarization preserves all actionable information: code snippets, file paths, error messages, and - concrete recommendations - -### Teammate Messaging - -All agents (including children) automatically receive tools for **direct sibling-to-sibling messaging**: - -| Tool | Description | -|-----------------------|-----------------------------------------------------| -| `agent__send_message` | Send a text message to another agent's inbox by ID. | -| `agent__check_inbox` | Drain all pending messages from your inbox. | - -This enables coordination patterns where child agents share cross-cutting findings: - -``` -# Agent A discovers something relevant to Agent B -agent__send_message --id agent_reviewer_b1c2d3e4 --message "Found a security issue in auth.rs line 42" - -# Agent B checks inbox before finalizing -agent__check_inbox -``` - -Messages are routed through the parent's supervisor. A parent can message its children, and children can message -their siblings. For a working example of the teammate pattern, see the built-in -[code-reviewer](../assets/agents/code-reviewer) agent, which spawns file-specific reviewers that share -cross-cutting findings with each other. - -### Runaway Safeguards - -The spawning system includes built-in safeguards to prevent runaway agent chains: - -- **`max_concurrent_agents`:** Caps how many agents can run at once (default: 4). Spawn attempts beyond this - limit return an error asking the agent to wait or cancel existing agents. -- **`max_agent_depth`:** Caps nesting depth (default: 3). A child agent spawning its own child increments the - depth counter. Attempts beyond the limit are rejected. -- **`can_spawn_agents`:** Only agents with this flag set to `true` can spawn children. By default, spawning is - disabled. This means child agents cannot spawn their own children unless you explicitly create them with - `can_spawn_agents: true` in their config. - -## 8. User Interaction Tools - -Loki includes built-in tools for agents (and the REPL) to interactively prompt the user for input. These tools -are **always available**. No configuration needed. They are automatically injected into every agent and into -REPL mode when function calling is enabled. - -### User Interaction Available Tools - -| Tool | Description | Returns | -|------------------|-----------------------------------------|----------------------------------| -| `user__ask` | Present a single-select list of options | The selected option string | -| `user__confirm` | Ask a yes/no question | `"yes"` or `"no"` | -| `user__input` | Request free-form text input | The text entered by the user | -| `user__checkbox` | Present a multi-select checkbox list | Array of selected option strings | - -**Parameters:** - -- `user__ask`: `--question "..." --options ["Option A", "Option B", "Option C"]` -- `user__confirm`: `--question "..."` -- `user__input`: `--question "..."` -- `user__checkbox`: `--question "..." --options ["Option A", "Option B", "Option C"]` - -At the top level (depth 0), these tools render interactive terminal prompts directly using arrow-key navigation, -checkboxes, and text input fields. - -### Escalation (Sub-Agent to User) - -When a **child agent** (depth > 0) calls a `user__*` tool, it cannot prompt the terminal directly. Instead, -the request is **automatically escalated** to the root agent: - -1. The child agent calls `user__ask(...)` and **blocks**, waiting for a reply -2. The root agent sees a `pending_escalations` notification in its next tool results -3. The root agent either answers from context or prompts the user itself, then calls - `agent__reply_escalation` to unblock the child -4. The child receives the reply and continues - -The escalation timeout is configurable via `escalation_timeout` in the agent's `config.yaml` (default: 300 -seconds / 5 minutes). If the timeout expires, the child receives a fallback message asking it to use its -best judgment. - -| Tool | Description | -|---------------------------|--------------------------------------------------------------------------| -| `agent__reply_escalation` | Reply to a pending child escalation, unblocking the waiting child agent. | - -This tool is automatically available to any agent with `can_spawn_agents: true`. - -## 9. Auto-Injected Prompts - -Loki automatically appends usage instructions to your agent's system prompt for each enabled built-in system. -These instructions are injected into both **static and dynamic instructions** after your own instructions, -ensuring agents always know how to use their available tools. - -| System | Injected When | Toggle | -|--------------------|----------------------------------------------------------------|-----------------------------| -| Todo tools | `auto_continue: true` AND `inject_todo_instructions: true` | `inject_todo_instructions` | -| Spawning tools | `can_spawn_agents: true` AND `inject_spawn_instructions: true` | `inject_spawn_instructions` | -| Teammate messaging | Always (all agents) | None (always injected) | -| User interaction | Always (all agents) | None (always injected) | - -If you prefer to write your own instructions for a system, set the corresponding `inject_*` flag to `false` -and include your custom instructions in the agent's `instructions` field. The built-in tools will still be -available; only the auto-injected prompt text is suppressed. - -## Built-In Agents -Loki comes packaged with some useful built-in agents: - -* `coder`: An agent to assist you with all your coding tasks -* `code-reviewer`: A [CodeRabbit](https://coderabbit.ai)-style code reviewer that spawns per-file reviewers using the teammate messaging pattern -* `demo`: An example agent to use for reference when learning to create your own agents -* `explore`: An agent designed to help you explore and understand your codebase -* `file-reviewer`: An agent designed to perform code-review on a single file (used by the `code-reviewer` agent) -* `jira-helper`: An agent that assists you with all your Jira-related tasks -* `oracle`: An agent for high-level architecture, design decisions, and complex debugging -* `sisyphus`: A powerhouse orchestrator agent for writing complex code and acting as a natural language interface for your codebase (similar to ClaudeCode, Gemini CLI, Codex, or OpenCode). Uses sub-agent spawning to delegate to `explore`, `coder`, and `oracle`. -* `sql`: A universal SQL agent that enables you to talk to any relational database in natural language diff --git a/docs/AICHAT-MIGRATION.md b/docs/AICHAT-MIGRATION.md deleted file mode 100644 index f48050f..0000000 --- a/docs/AICHAT-MIGRATION.md +++ /dev/null @@ -1,211 +0,0 @@ -# AIChat to Loki Migration Guide -Loki originally started as a fork of AIChat but has since evolved into its own separate project with separate goals. - -As a result, there's some changes you'll need to make to your AIChat configuration to be able to use Loki. - -Be sure you've run `loki` at least once so that the Loki configuration directory and subdirectories exist and is -populated with the built-in defaults. - -## Global Configuration File -You should be able to copy/paste your AIChat configuration file into your Loki configuration directory. Since the -location of the Loki configuration directory varies between systems, you can use the following command to locate your -config directory: - -```shell -loki --info | grep 'config_dir' | awk '{print $2}' -``` - -Then, you'll need to make the following changes: - -* `function_calling` -> `function_calling_support` -* `use_tools` -> `enabled_tools` -* `agent_prelude` -> `agent_session` -* `compress_threshold` -> `compression_threshold` -* `summarize_prompt` -> `summarization_prompt` -* `summary_prompt` -> `summary_context_prompt` - -## Roles -Locate your `roles` directory using the following command: - -```shell -loki --info | grep 'roles_dir' | awk '{print $2}' -``` - -Update any roles that have `use_tools` to `enabled_tools`. - -## Sessions -Locate your `sessions` directory using the following command: - -```shell -loki --info | grep 'sessions_dir' | awk '{print $2}' -``` - -Update the following settings: -* `use_tools` -> `enabled_tools` -* `compress_threshold` -> `compression_threshold` -* `summarize_prompt` -> `summarization_prompt` -* `summary_prompt` -> `summary_context_prompt` - ---- - -# LLM Functions Changes -Probably the most significant difference between AIChat and Loki is how tools are handled. So if you cloned the -[llm-functions](https://github.com/sigoden/llm-functions) repo, you'll need to make the following changes. - -**Note: JavaScript functions are not supported in Loki.** - -The following guide assumes you're using the `llm-functions` repository as your base for custom functions, and thus -follows that directory structure. - -## Agents -Agents are now all handled in one place: the `agents` directory (`/agents`): - -```shell -loki --info | grep 'agents_dir' | awk '{print $2}' -``` - -And instead of separate `index.yaml` and `config.yaml` files, they're now both in a single `config.yaml` file. - -So now for all of your agents, copy all the contents of those directories to the corresponding directory in the Loki -`agents` directory. Then make the following changes: - -* Copy the contents of your `/functions/agents` directory into `/agents//tools.txt` -* No `/agents//index.yaml` - -## Functions -Loki consolidates much of the `llm-functions` repo functionality into one binary. So this means - -* There's no need to have `argc` installed anymore -* No separate repository to manage -* No `tools.txt` -* No `functions.json` -* No `functions/mcp` directory at all -* No `functions/scripts` - -Here's how to migrate your functions over to Loki from the `llm-functions` repository. - -* Copy your AIChat `/functions` directory into your Loki config directory -* Delete the following files and directories from your `/functions` directory: - * `scripts/` - * `agents.txt` - * `functions.json` - * `Argcfile.sh` - * `README.md` (irrelevant now) - * `LICENSE` (irrelevant now) - * `utils/guard_operation.sh` - * `utils/guard_path.sh` - * `utils/patch.awk` -* Everything in `tools.txt` now lives in the global config file under the `visible_tools` setting: - ```text - get_current_weather.sh - execute_command.sh - web_search.sh - #execute_py_code.py - query_jira_issues.sh - ``` - becomes the following in your `/config.yaml` - ```yaml - visible_tools: - - get_current_weather.sh - - execute_command.sh - - web_search.sh - # - web_search.sh - - query_jira_issues.sh - ``` -* If you've defined a `functions/mcp.json` file, you can leave it alone. -* Similarly to agents, if you have any bash `tools.sh` that depend on the utility scripts in the `llm-functions` - repository, they've been replaced by built-in utility scripts. So use the following to replace any matching lines in - your `tools.sh` files: - ```bash - ################## - ## Scripts file ## - ################## - ROOT_DIR="${LLM_ROOT_DIR:-$(cd "$(dirname "${BASH_SOURCE[0]}")/.." && pwd)}" - # replace with - source "$LLM_PROMPT_UTILS_FILE" - - ####################### - ## guard_path script ## - ####################### - "$ROOT_DIR/utils/guard_path.sh" - # replace with - guard_path - - ############################ - ## guard_operation script ## - ############################ - "$ROOT_DIR/utils/guard_operation.sh" - # replace with - guard_operation - - ###################### - ## patch.awk script ## - ###################### - awk -f "$ROOT_DIR/utils/patch.awk" - # replace with - patch_file - ``` - -Refer to the [custom bash tools docs](./function-calling/CUSTOM-BASH-TOOLS.md) to learn how to compile and test bash -tools in Loki without needing to use `argc`. diff --git a/docs/ENVIRONMENT-VARIABLES.md b/docs/ENVIRONMENT-VARIABLES.md deleted file mode 100644 index a9a0a3f..0000000 --- a/docs/ENVIRONMENT-VARIABLES.md +++ /dev/null @@ -1,113 +0,0 @@ -# Environment Variables - -Loki is designed to be highly dynamic and customizable. As a result, Loki utilizes a number of environment variables -that can be used to modify its behavior at runtime without needing to modify the existing configuration files. - -Loki also supports defining environment variables via a `.env` file in the Loki configuration directory. This directory -varies between systems, so you can find the location of your configuration directory using the following command: - -```shell -loki --info | grep 'config_dir' | awk '{print $2}' -``` - -## Quick Links - -- [Global Configuration Related Variables](#global-configuration-related-variables) -- [Client Related Variables](#client-related-variables) -- [Files and Directory Related Variables](#files-and-directory-related-variables) -- [Agent Related Variables](#agent-related-variables) -- [Logging Related Variables](#logging-related-variables) -- [Miscellaneous Variables](#miscellaneous-variables) - - ---- - -## Global Configuration Related Variables -All configuration items in the global config file have environment variables that can be overridden at runtime. To see -all configuration options and more thorough descriptions, refer to the [example config file](../config.example.yaml). - -Below are the most commonly used configuration settings and their corresponding environment variables: - -| Setting | Environment Variable | -|----------------------------|---------------------------------| -| `model` | `LOKI_MODEL` | -| `temperature` | `LOKI_TEMPERATURE` | -| `top_p` | `LOKI_TOP_P` | -| `stream` | `LOKI_STREAM` | -| `save` | `LOKI_SAVE` | -| `editor` | `LOKI_EDITOR` | -| `wrap` | `LOKI_WRAP` | -| `wrap_code` | `LOKI_WRAP_CODE` | -| `save_session` | `LOKI_SAVE_SESSION` | -| `compression_threshold` | `LOKI_COMPRESSION_THRESHOLD` | -| `function_calling_support` | `LOKI_FUNCTION_CALLING_SUPPORT` | -| `enabled_tools` | `LOKI_ENABLED_TOOLS` | -| `mcp_server_support` | `LOKI_MCP_SERVER_SUPPORT` | -| `enabled_mcp_servers` | `LOKI_ENABLED_MCP_SERVERS` | -| `rag_embedding_model` | `LOKI_RAG_EMBEDDING_MODEL` | -| `rag_reranker_model` | `LOKI_RAG_RERANKER_MODEL` | -| `rag_top_k` | `LOKI_RAG_TOP_K` | -| `rag_chunk_size` | `LOKI_RAG_CHUNK_SIZE` | -| `rag_chunk_overlap` | `LOKI_RAG_CHUNK_OVERLAP` | -| `highlight` | `LOKI_HIGHLIGHT` | -| `theme` | `LOKI_THEME` | -| `serve_addr` | `LOKI_SERVE_ADDR` | -| `user_agent` | `LOKI_USER_AGENT` | -| `save_shell_history` | `LOKI_SAVE_SHELL_HISTORY` | -| `sync_models_url` | `LOKI_SYNC_MODELS_URL` | - - -## Client Related Variables -The following environment variables are available for clients in Loki: - -| Environment Variable | Description | -|----------------------------------------|---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------| -| `{client}_API_KEY` | For clients that require an API key, you can define the keys either through environment variables or
using the [vault](./VAULT.md). The variables are named after the client to which they apply;
e.g. `OPENAI_API_KEY`, `GEMINI_API_KEY`, etc. | -| `LOKI_PLATFORM` | Combine with `{client}_API_KEY` to run Loki without a configuration file.
This variable is ignored if a configuration file exists. | -| `LOKI_PATCH_{client}_CHAT_COMPLETIONS` | Patch chat completion requests to models on the corresponding client; Can modify the URL, body,
or headers. | -| `LOKI_SHELL` | Specify the shell that Loki should be using when executing commands | - -## Files and Directory Related Variables -You can also customize the files and directories that Loki loads its configuration files from: - -| Environment Variable | Description | Default Value | -|----------------------|------------------------------------------------------------------------|---------------------------------| -| `LOKI_CONFIG_DIR` | Customize the location of the Loki configuration directory. | `/loki` | -| `LOKI_ENV_FILE` | Customize the location of the `.env` file to load at startup. | `/.env` | -| `LOKI_CONFIG_FILE` | Customize the location of the global `config.yaml` configuration file. | `/config.yaml` | -| `LOKI_ROLES_DIR` | Customize the location of the `roles` directory. | `/roles` | -| `LOKI_SESSIONS_DIR` | Customize the location of the `sessions` directory. | `/sessions` | -| `LOKI_RAGS_DIR` | Customize the location of the `rags` directory. | `/rags` | -| `LOKI_FUNCTIONS_DIR` | Customize the location of the `functions` directory. | `/functions` | - -## Agent Related Variables -You can also customize the location of full agent configurations using the following environment variables: - -| Environment Variable | Description | -|------------------------------|-------------------------------------------------------------------------------------------------------------------------------------| -| `_CONFIG_FILE` | Customize the location of the agent's configuration file; e.g. `SQL_CONFIG_FILE` | -| `_MODEL` | Customize the `model` used for the agent; e.g `SQL_MODEL` | -| `_TEMPERATURE` | Customize the `temperature` used for the agent; e.g. `SQL_TEMPERATURE` | -| `_TOP_P` | Customize the `top_p` used for the agent; e.g. `SQL_TOP_P` | -| `_GLOBAL_TOOLS` | Customize the `global_tools` that are enabled for the agent (a JSON string array); e.g. `SQL_GLOBAL_TOOLS` | -| `_MCP_SERVERS` | Customize the `mcp_servers` that are enabled for the agent (a JSON string array); e.g. `SQL_MCP_SERVERS` | -| `_AGENT_SESSION` | Customize the `agent_session` used with the agent; e.g. `SQL_SESSION` | -| `_INSTRUCTIONS` | Customize the `instructions` for the agent; e.g. `SQL_INSTRUCTIONS` | -| `_VARIABLES` | Customize the `variables` used for the agent (in JSON format of `[{"key1": "value1", "key2": "value2"}]`);
e.g. `SQL_VARIABLES` | - -## Logging Related Variables -The following variables can be used to change the log level of Loki or the location of the log file: - -| Environment Variable | Description | Default Value | -|----------------------|---------------------------------------------|----------------------------------| -| `LOKI_LOG_LEVEL` | Customize the log level of Loki | `INFO` | -| `LOKI_LOG_FILE` | Customize the location of the Loki log file | `/loki/loki.log` | - -**Pro-Tip:** You can always tail the Loki logs using the `--tail-logs` flag. If you need to disable color output, you -can also pass the `--disable-log-colors` flag as well. - -## Miscellaneous Variables -| Environment Variable | Description | Default Value | -|----------------------|--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|---------------| -| `AUTO_CONFIRM` | Bypass all `guard_*` checks in the bash prompt helpers; useful for agent composition and routing | | -| `LLM_TOOL_DATA_FILE` | Set automatically by Loki on Windows. Points to a temporary file containing the JSON tool call data.
Tool scripts (`run-tool.sh`, `run-agent.sh`, etc.) read from this file instead of command-line args
to avoid JSON escaping issues when data passes through `cmd.exe` → bash. **Not intended to be set by users.** | | \ No newline at end of file diff --git a/docs/MACROS.md b/docs/MACROS.md deleted file mode 100644 index a90aafe..0000000 --- a/docs/MACROS.md +++ /dev/null @@ -1,103 +0,0 @@ -# Macros -Macros are essentially Loki "scripts"; that is, a predefined sequence of REPL commands that automate repetitive tasks or -workflows. Macros run in isolated environments, ensuring that the macros don't inherit any pre-existing role, session, -RAG, or agent state, and they will not affect your current context. - -This isolation ensures that your workspace remains clean and unaffected by macro operations. - -![Macro Example](./images/macros/macros-example.gif) - -For more information on Loki's REPL, refer to the [REPL](./REPL.md) documentation. - -## Quick Links - -- [Macro Definition](#macro-definition) - - [Step Definitions](#step-definitions) - - [Macro Variables](#macro-variables) -- [Built-In Macros](#built-in-macros) - - ---- - -## Macro Definition -Macros are defined as YAML files in the `macros` subdirectory of your Loki configuration directory. The Loki configuration -directory can vary between systems, so to find the location of your macros config directory, you can use the following -command: - -```shell -loki --info | grep 'macros_dir' | awk '{print $2}' -``` - -Macro definitions are broken into two parts: the `steps` of the macro, and an optional `variables` section that lets -users pass in variables to alter the behavior of the macro at runtime. - -### Step Definitions -The step definitions for a macro are straightforward: They are simply the exact commands you would otherwise type in the -REPL. - -**Example: Macro to generate a git commit message** -`macros/generate-commit-message.yaml` -```yaml -steps: - - .file `git diff` -- generate git commit message -``` -Usage: -```shell -$ loki --macro generate-commit-message ->> .file `git diff` -- generate a git commit message -Add documentation on macros -``` - -For a full example configuration, refer to the [example macro configuration file](../config.macro.example.yaml) in the root of this project. - -### Macro Variables -Sometimes it's useful to be able to modify the behavior of a macro at runtime. This is achieved with the `variables` -array of the macro definition. - -To pass variables to a macro, since they are just Loki scripts, the syntax is the same as it is for any other scripting -language: You just pass them alongside your invocation. - -**Example:** -```shell -$ loki --macro example-variable-macro first_argument second_argument -``` - -Each variable in the `variables` array has the following properties: -* `name` (Required): the name of the variable, which can be referenced in the actual steps of the macro using the - `{{name}}` syntax. -* `default` (Optional): A default value for the variable if no value is specified. If no default value is defined, and - no value is provided for the variable at runtime, Loki will error out. -* `rest` (Optional, Boolean): When set to `true`, this variable will collect all remaining arguments passed to the - macro. This behavior is only applicable when the variable is the last variable in the list. By default, this is - `false`. - -The `variables` array is order-dependent; that is to say that all arguments passed to the macro are positional. So be -careful about the ordering if that is important to your macro's invocation. - -**Example: Simple variable example to invoke an agent** -`macros/invoke-agent.yaml` -```yaml -variables: - - name: agent # No default value means this must be defined at runtime - - name: args - rest: true # All remaining arguments to the macro are collected into this variable - default: What can you do? # This is used if no value is passed at runtime -steps: - - .agent {{agent}} - - '{{args}}' -``` -Usage: -```shell -$ loki --macro invoke-agent sql -# or -$ loki --macro invoke-agent sql What tables are available? -``` - -For a full example configuration, refer to the [example macro configuration file](../config.macro.example.yaml) in the root of this project. - -## Built-In Macros -Loki comes packaged with some useful built-in macros. These are also good examples if you're looking for more examples -on how to make your own macros, so be sure to check out the [built-in macro definitions](../assets/macros) if you're -looking for more examples. - -* `generate-commit-message` - Generate a Git commit message based on the staged changes in the current directory diff --git a/docs/RAG.md b/docs/RAG.md deleted file mode 100644 index ea3fe9e..0000000 --- a/docs/RAG.md +++ /dev/null @@ -1,307 +0,0 @@ -# RAG -Retrieval Augmented Generation (RAG) is a method of minimizing LLM hallucinations and extending the model's context -without consuming a significant portion of the context length. It uses documents and other additional resources that you -provide to give the model more context for all of your prompts. - -Loki has a built-in vector database and full-text search engine to support RAG knowledge bases for your queries. - -The generated knowledge bases are stored in the `rag` subdirectory of your Loki configuration directory. The location of -this directory varies by system, so you can use the following command to find your RAG directory: - -```shell -loki --info | grep 'rags_dir' | awk '{print $2}' -``` - -## Quick Links - -- [Usage](#usage) - - [Persistent RAG](#persistent-rag) - - [Ephemeral RAG](#ephemeral-rag) -- [How It Works](#how-it-works) - - [1. Build](#1-build) - - [2. Lookup](#2-lookup) - - [2a. Reranking (Optional)](#2a-reranking-optional) - - [3. Prompt](#3-prompt) -- [Supported Document Sources](#supported-document-sources) -- [Document Loaders](#document-loaders) - - [Document Loader Usage](#document-loader-usage) -- [Advanced Customizations](#advanced-customizations) - - [Embedding Model](#embedding-model) - - [Reranker](#reranker) - - [Chunk Size](#chunk-size) - - [Trade-Offs](#chunk-size-trade-offs) - - [Chunk Overlap](#chunk-overlap) - - [Top K](#top-k) - - [Trade-Offs](#top-k-trade-offs) - - [RAG Template](#rag-template) - - ---- - -## Usage -There's two ways to use RAG in Loki: A persistent RAG that can be loaded on-demand for queries, and an ephemeral one for -adding RAG to a single specific query. - -### Persistent RAG -In the REPL, persistent RAG is initialized via the `.rag` command: - -![Persistent RAG example](./images/rag/persistent-rag.gif) - -The generated RAG is then saved to the `rag` subdirectory of the Loki configuration, and can then be loaded whenever you -want that knowledge base via either `.rag ` or `loki --rag `. - -### Ephemeral RAG -Short-lived RAG that is only used for a single session or query is loaded using `.file`/`--file`. - -You can use it to either execute a prompt from a file, or for temporary RAG. The difference is the usage of the `--` -separator. If you only specify a filename and no `--` separator, Loki will know to read the file contents and pass them -as a query to the model. Otherwise, the `--` separator is read to indicate that this is the end of the list of documents -to load into the ephemeral RAG, and what follows is the query to pass to the model. - -```shell -.file prompt.md # Read the file as a prompt -.file %% -- translate the last reply to italian -.file `git diff` -- generate a commit message -``` - -![Ephemeral RAG Example](./images/rag/ephemeral-rag.gif) - -Once the session ends, this RAG will no longer be accessible and is only visible to the current session. - -#### The `%%` Document Type -In addition to the usual documents that can be specified for persistent RAG, ephemeral RAG has a special `%%` value. -This value references the content of the last reply. So you can use it like this: - -```shell -.file %% -- translate the last reply to italian -``` - -The `--` indicates that this is the end of your documents and the beginning of your prompt. - -#### The `cmd` Document Type -Loki also lets you use command outputs for ephemeral RAG input. Simply enclose the command in backticks: - -```shell -.file `git diff` -- generate a commit message -``` - -The `--` indicates that this is the end of your documents and the beginning of your prompt. - -## How It Works -#### 1. Build -When you define RAG, Loki will first "build" the RAG. This means that Loki will consume the documents you specified and -generate [embeddings](https://huggingface.co/spaces/hesamation/primer-llm-embedding) for that text. This essentially just means that Loki translates the document into a language -the LLM can understand. - -These embeddings are then stored in an in-memory vector database. - -#### 2. Lookup -Loki sits between you and the model. So when you submit a prompt to the model, before Loki ever sends it, it will first -convert your prompt into embeddings (LLM language), and look for relevant snippets of text in the vector database. - -Loki then passes the top `n`-snippets of text that it finds in the vector database as additional context to the model -before your prompt. - -#### 2a. Reranking (Optional) -The lookup for relevant snippets of texts uses embeddings to find text that is semantically similar to your prompt, and -returns the top `n`-results. This often works fairly well, however these top results aren't always the most relevant for -answering the specific query. - -Reranking improves these initial results (say, the top 20-100 text snippets) and re-scores them using a more -sophisticated model. The reranker model will rank documents by their actual usefulness for answering the query to ensure -the most relevant context is passed to the model alongside your query. - -This reranking model can be customized for each RAG you build in Loki. See the [Custom Reranker](#reranker) section -below for more details on how to customize this. - -#### 3. Prompt -Finally, the text snippets that were looked up in RAG are passed to the model as additional context to your prompt, -giving the model query-specific context to answer your question. - -## Supported Document Sources -Loki supports a number of document sources that can be used for RAG: - -| Source | Example | Comments | -|--------------------------|-----------------------------------------------------------------------|----------------------------------------------------------------------------------------------------------------------------------------------------------| -| Files | `/tmp/dir1/file1;/tmp/dir1/file2` | | -| Directory | `/tmp/dir` | Picks up all files in a directory and all its subdirectories | -| Directory (extensions} | `/tmp/dir2/**/*.{md,txt}` | Finds all files in all subdirectories with the specified extensions | -| Recursive Filename | `/tmp/*/LOKI.md` | The following files will be picked up:
  • `/tmp/dir1/LOKI.md`
  • `/tmp/dir2/subdir1/LOKI.md`
  • `/tmp/dir2/subdir2/LOKI.md`
| -| URL | `https://www.ohdsi.org/data-standardization/` | Downloads and loads the specified webpage into the
knowledge base | -| Recursive URL (Websites) | `https://github.com/OHDSI/Vocabulary-v5.0/wiki/**` | Crawls all pages under the given URL and loads them
into the knowledge base | -| Document Loader (custom) | `jina:https://cloud.google.com/bigquery/docs/reference/standard-sql/` | Use a custom document loader to parse the given document | - -## Document Loaders -Loki only has built-in support for loading text files. But that functionality can be extended to read all kinds of files -into your knowledge bases. These custom loaders are used by both RAG and for documents specified using the -`.file`/`--file` flags. - -In the global configuration file, you can specify loaders for specific document types using the `document_loaders` -setting. Each loader is defined by specifying a name and then a command that Loki will execute to load the document. - -The following variables are interpolated at runtime by Loki and can be used as placeholders in your command definitions: -* `$1` (Required) - The input file -* `$2` (Optional) - The output file. If omitted, `stdout` is used as the output destination - -**Note:** It is your responsibility to ensure that any tools used to parse documents into text that Loki can read are -installed on your system and are available on your `$PATH`. Loki does not have any built-in way of installing -dependencies for document loaders for you. - -The following are some example loaders: -```yaml -document_loaders: - pdf: 'pdftotext $1 -' # Use pdftotext to convert a PDF file to text - # (see https://poppler.freedesktop.org for details on how to install pdftotext) - docx: 'pandoc --to plain $1' # Use pandoc to convert a .docx file to text - # (see https://pandoc.org for details on how to install pandoc) - jina: 'curl -fsSL https://r.jina.ai/$1 -H "Authorization: Bearer {{JINA_API_KEY}}' # Use Jina to translate a website into text; - # Requires a Jina API key to be added to the Loki vault - git: > # Use yek to load a git repository into the knowledgebase (https://github.com/bodo-run/yek) - sh -c "yek $1 --json | jq 'map({ path: .filename, contents: .content })'" -``` - -### Document Loader Usage -Once you have your loaders defined, you can specify when Loki should use them by prefixing any RAG file/directory/URI -with the name of the loader. - -**Example: Load a git repo into RAG** -![Git Repo Loader Example](./images/rag/git-loader.png) - -**Example: Use pdf loader for ephemeral RAG** -```shell -$ loki --file pdf:some-file.pdf -``` - -## Advanced Customizations -For those familiar with RAG, Loki exposes a handful of advanced global settings that can be used to tweak your default -RAG configurations. - -### Embedding Model -When Loki queries your RAG knowledge bases, it needs to first convert your query into embeddings. By default, Loki uses -the same embedding model that was used to create the knowledge base in the first place. - -This can be customized to any other embedding model available in your configured clients by setting the -`rag_embedding_model` setting in your global Loki configuration file: - -```yaml -rag_embedding_model: null # Specifies the embedding model used for context retrieval -``` - -### Reranker -By default, Loki uses [Reciprocal Rank Fusion (RRF)](https://www.elastic.co/docs/reference/elasticsearch/rest-apis/reciprocal-rank-fusion) to merge vector and keyword search results. - -You can change the default reranker model to any other reranking model in your configured clients. To change the default -reranker model, simply change the value of the `rag_reranker_model` setting in your global configuration file: - -```yaml -rag_reranker_model: null # By default, -``` - -### Chunk Size -In the context of RAG, the chunk size is the maximum length of each text chunk (measured in characters) that is created -when splitting documents. In Loki, this defaults to `2000` characters. - -You can specify a different global default by setting the `rag_chunk_size` property in your global configuration file: - -```yaml -rag_chunk_size: null # Defines the size of chunks for document processing in characters -``` - -#### Chunk Size Trade-Offs -Keep in mind the following trade-offs when changing the chunk size: - -* **Smaller chunks (e.g. 256 characters):** More precise retrieval, better semantic focus, but may lack context or split - important information -* **Larger chunks (e.g. 1024 characters):** More context preserved, fewer chunks to manage, but less precise matching - and more noise in retrieved document - -### Chunk Overlap -Chunk overlap in RAG is the number of characters that overlap between consecutive chunks to maintain continuity. - ---- - -**Example:** If the following sentence is cut off at the end of one chunk - -`I was doing fine until someone brought up` - -You'll ideally want that full sentence to be picked up at the beginning of the next chunk to make sure the full meaning -is captured. So in this example, if your chunk overlap is 42 characters, then the start of the next chunk would look -like this: - -`I was doing fine until someone brought up the game. ` - ---- - -Often, this value is 10%-20% of the chunk size. - -By default, in Loki, this value is 5% the chunk size. You can override this and specify the default chunk overlap (in -characters) that Loki should use as a global default by setting the `rag_chunk_overlap` property in the global Loki -configuration file: - -```yaml -rag_chunk_overlap: null # Defines the overlap between chunks -``` - -### Top K -In RAG, `top_k` represents the top `k`-chunks to return from the vector database query. Think of it like if you search -something on Google and only care about the top 10 results, that's what you'll use for your context. - -In Loki, the default value for this is `5`. You can customize this global default by setting the `rag_top_k` property in -your global configuration file: - -```yaml -rag_top_k: 5 # Specifies the number of documents to retrieve for answering queries -``` - -#### Top K Trade-Offs -When customizing this value, keep in mind the following trade-offs so you get the best performance: - -* **Lower top_k (e.g. 3):** Faster, more focused context, lower cost, but risks missing relevant information -* **Higher top_k (e.g. 10):** More comprehensive coverage, but more noise, higher latency, increased token costs, and - potential context window constraints - -### RAG Template -When you use RAG in Loki, after Loki performs the lookup for relevant chunks of text to add as context to your query, it -will add the retrieved text chunks as context to your query before sending it to the model. The format of this context -is determined by the `rag_template` setting in your global Loki configuration file. - -This template utilizes three placeholders: -* `__INPUT__`: The user's actual query -* `__CONTEXT__`: The context retrieved from RAG -* `__SOURCES__`: A numbered list of the source file paths or URLs that the retrieved context came from - -These placeholders are replaced with the corresponding values into the template and make up what's actually passed to -the model at query-time. The `__SOURCES__` placeholder enables the model to cite which documents its answer is based on, -which is especially useful when building knowledge-base assistants that need to provide verifiable references. - -The default template that Loki uses is the following: - -```text -Answer the query based on the context while respecting the rules. (user query, some textual context and rules, all inside xml tags) - - -__CONTEXT__ - - - -__SOURCES__ - - - -- If you don't know, just say so. -- If you are not sure, ask for clarification. -- Answer in the same language as the user query. -- If the context appears unreadable or of poor quality, tell the user then answer as best as you can. -- If the answer is not in the context but you think you know the answer, explain that to the user then answer with your own knowledge. -- Answer directly and without using xml tags. -- When using information from the context, cite the relevant source from the section. - - - -__INPUT__ - -``` - -You can customize this template by specifying the `rag_template` setting in your global Loki configuration file. Your -template *must* include both the `__INPUT__` and `__CONTEXT__` placeholders in order for it to be valid. The -`__SOURCES__` placeholder is optional. If it is omitted, source references will not be included in the prompt. diff --git a/docs/REPL-PROMPT.md b/docs/REPL-PROMPT.md deleted file mode 100644 index 01acbb9..0000000 --- a/docs/REPL-PROMPT.md +++ /dev/null @@ -1,117 +0,0 @@ -# Customize REPL Prompt - -The prompt you see when you start the Loki REPL can be customized to your liking. This is achieved via the `left_prompt` -and `right_prompt` settings in the global Loki configuration file: - -```yaml -left_prompt: '{color.red}{model}){color.green}{?session {?agent {agent}>}{session}{?role /}}{!session {?agent {agent}>}}{role}{?rag @{rag}}{color.cyan}{?session )}{!session >}{color.reset} ' -right_prompt: '{color.purple}{?session {?consume_tokens {consume_tokens}({consume_percent}%)}{!consume_tokens {consume_tokens}}}{color.reset}' -``` - -The location of the global configuration file differs between systems, so you can use the following command to find your -global configuration file's location: - -```shell -loki --info | grep 'config_file' | awk '{print $2}' -``` - -## Quick Links - -- [Syntax](#syntax) -- [Variables](#variables) - - -## Syntax -The syntax for the prompts consists of plain text and templates contained in `{...}`. The plain text is -printed exactly as given. - -The syntax for the templates `{...}` is as follows: - -* `{variable}` - Replaced with the value of `variable` -* `{?variable