loki/docs/AGENTS.md

Agents

Agents in Loki follow the same style as OpenAI's GPTs. They consist of 3 parts:

• Role - Tell the LLM how to behave
• RAG - Pre-built knowledge bases specifically for the agent
• Function Calling (#2) - Extends the functionality of the LLM through custom functions it can call

Agent configuration files are stored in the agents subdirectory of your Loki configuration directory. The location of this directory varies between systems so you can use the following command to locate yours:

loki --info | grep 'agents_dir' | awk '{print $2}'

If you're looking for more example agents, refer to the built-in agents.

Quick Links

• Directory Structure
• Metadata
• 2. Define the Instructions
  • Static Instructions
    • Special Variables
    • User-Defined Variables
  • Dynamic Instructions
  • Variables
• 3. Initializing RAG
• 4. Building Tools for Agents
  • Limitations
  • .env File Support
  • Python-Based Agent Tools
  • Bash-Based Agent Tools
• 5. Conversation Starters
• 6. Todo System & Auto-Continuation
• Built-In Agents

---

Directory Structure

Agent configurations often have the following directory structure:

<loki-config-dir>/agents
    └── my-agent
        ├── config.yaml
        ├── tools.sh
            or
        ├── tools.py

This means that agent configurations often are only two files: the agent configuration file (config.yaml), and the tool definitions (agents/my-agent/tools.sh or tools.py).

To see a full example configuration file, refer to the example agent config file.

The best way to understand how an agent is built is to go step by step in the following manner:

---

1. Metadata

Agent configurations have the following settings available to customize each agent:

# Model Configuration
model: openai:gpt-4o                 # Specify the LLM to use
temperature: null                    # Set default temperature parameter, range (0, 1)
top_p: null                          # Set default top-p parameter, with a range of (0, 1) or (0, 2), depending on the model
# Agent Metadata Configuration
agent_session: null                  # Set a session to use when starting the agent. (e.g. temp, default); defaults to globally set agent_session
# Agent Configuration
name: <agent-name>                   # Name of the agent, used in the UI and logs
description: <description>           # Description of the agent, used in the UI
version: 1                           # Version of the agent
# Function Calling Configuration
mcp_servers:                         # Optional list of MCP servers that the agent utilizes
  - github                           # Corresponds to the name of an MCP server in the `<loki-config-dir>/functions/mcp.json` file
global_tools:                        # Optional list of additional global tools to enable for the agent; i.e. not tools specific to the agent
  - web_search
  - fs
  - python
# Todo System & Auto-Continuation (see "Todo System & Auto-Continuation" section below)
auto_continue: false                 # Enable automatic continuation when incomplete todos remain
max_auto_continues: 10               # Maximum continuation attempts before stopping
inject_todo_instructions: true       # Inject todo tool instructions into system prompt
continuation_prompt: null            # Custom prompt for continuations (optional)

As mentioned previously: Agents utilize function calling to extend a model's capabilities. However, agents operate in isolated environment, so in order for an agent to use a tool or MCP server that you have defined globally, you must explicitly state which tools and/or MCP servers the agent uses. Otherwise, it is assumed that the agent doesn't use any tools outside its own custom defined tools.

And if you don't define a agents/my-agent/tools.sh or agents/my-agent/tools.py, then the agent is really just a role.

You'll notice there's no settings for agent-specific tooling. This is because they are handled separately and automatically. See the Building Tools for Agents section below for more information.

To see a full example configuration file, refer to the example agent config file.

2. Define the Instructions

At their heart, agents function similarly to roles in that they tell the model how to behave. Agent configuration files have the following settings for the instruction definitions:

dynamic_instructions:     # Whether to use dynamically generated instructions for the agent; if false, static instructions are used. False by default.
instructions:             # Static instructions for the LLM; These are ignored if dynamic instructions are used
variables:                # An array of optional variables that the agent expects and uses

Static Instructions

By default, Loki agents use statically defined instructions. Think of them as being identical to the instructions for a role, because they virtually are.

Example:

instructions: |
  You are an AI agent designed to demonstrate agentic capabilities

Just like roles, agents support variable interpolation at runtime. There's two types of variables that can be interpolated into the instructions at runtime: special variables (like roles have), and user-defined variables. Just like roles, variables are interpolated into your instructions anywhere Loki sees the {{variable}} syntax.

Special Variables

The following special variables are provided by Loki at runtime and can be injected into your agent's instructions:

Name	Description	Example
__os__	Operating system name	linux
__os_family__	Operating system family	unix
__arch__	System architecture	x86_64
__shell__	The current user's default shell	bash
__locale__	The current user's preferred language and region settings	en-US
__now__	Current timestamp in ISO 8601 format	2025-11-07T10:15:44.268Z
__cwd__	The current working directory	/tmp
__tools__	A list of the enabled tools (global + mcp servers + agent-specific)

User-Defined Variables

Agents also support user-defined variables that can be interpolated into the instructions, and are made available to any agent-specific tools you define (see Building Tools for Agents for more details on how to create agent-specific tooling).

The variables setting in an agent's config has the following fields:

Field	Required	Description
name	*	The name of the variable
description	*	The description of the field
default		A default value for the field. If left undefined, the user will be prompted for a value at runtime

These variables can be referenced in both the agent's instructions, and in the tool definitions via LLM_AGENT_VAR_<name>.

Example:

instructions: |
  You are an agent who answers questions about a user's system.

  <tools>
  {{__tools__}}
  </tools>

  <system>
  os: {{__os__}}
  os_family: {{__os_family__}}
  arch: {{__arch__}}
  shell: {{__shell__}}
  locale: {{__locale__}}
  now: {{__now__}}
  cwd: {{__cwd__}}
  </system>

  <user>
  username: {{username}}
  </user>
variables:
  - name: username                 # Accessible from the tool definitions via the `LLM_AGENT_VAR_USERNAME` environment variable
    description: Your user name

Dynamic Instructions

Sometimes you may find it useful to dynamically generate instructions on startup. Whether that be via a call to Loki itself to generate them, or by some other means. Loki supports this type of behavior using a special function defined in your agents/my-agent/tools.py or agents/my-agent/tools.sh.

Example: Instructions for a JSON-reader agent that specializes on each JSON input it receives agents/json-reader/tools.py:

import json
from pathlib import Path
from genson import SchemaBuilder

def _instructions():
    """Generates instructions for the agent dynamically"""
    value = input("Enter a JSON file path OR paste raw JSON: ").strip()
    if not value:
        raise SystemExit("A file path or JSON string is required.")

    p = Path(value)
    if p.exists() and p.is_file():
        json_file_path = str(p.resolve())
        json_text = p.read_text(encoding="utf-8")
    else:
        try:
            json.loads(value)
        except json.JSONDecodeError as e:
            raise SystemExit(f"Input is neither a file nor valid JSON.\n{e}")
        json_file_path = "<provided-inline-json>"
        json_text = value

    try:
        data = json.loads(json_text)
    except json.JSONDecodeError as e:
        raise SystemExit(f"Provided content is not valid JSON.\n{e}")

    builder = SchemaBuilder()
    builder.add_object(data)
    json_schema = builder.to_schema()
    return f"""
        You are an AI agent that can view and filter JSON data with jq.
        
        ## Context
        json_file_path: {json_file_path}
        json_schema: {json.dumps(json_schema, indent=2)}
    """

or

agents/json-reader/tools.sh:

#!/usr/bin/env bash
set -e

# @meta require-tools jq,genson
# @env LLM_OUTPUT=/dev/stdout The output path

# @cmd Generates instructions for the agent dynamically
_instructions() {
	read -r -p "Enter a JSON file path OR paste raw JSON: " value
	
	if [[ -z "${value}" ]]; then
		echo "A file path or JSON string is required" >&2
		exit 1
	fi 
	json_file_path=""
    inline_temp=""
    cleanup() {
      [[ -n "${inline_temp:-}" && -f "${inline_temp}" ]] && rm -f "${inline_temp}"
    }
    trap cleanup EXIT
    
    if [[ -f "${value}" ]]; then
      json_file_path="$(realpath "${value}")"
      if ! jq empty "${json_file_path}" >/dev/null 2>&1; then
        echo "Error: File does not contain valid JSON: ${json_file_path}" >&2
        exit 1
      fi
    else
      inline_temp="$(mktemp)"
      printf "%s" "${value}" > "${inline_temp}"
      if ! jq empty "${inline_temp}" >/dev/null 2>&1; then
        echo "Error: Input is neither a file nor valid JSON." >&2
        exit 1
      fi
      json_file_path="<provided-inline-json>"
    fi
    
    source_file="${json_file_path}"
    if [[ "${json_file_path}" == "<provided-inline-json>" ]]; then
      source_file="${inline_temp}"
    fi
    
    json_schema="$(genson < "${source_file}" | jq -c '.')"
	cat <<EOF >> "$LLM_OUTPUT"
You are an AI agent that can view and filter JSON data with jq.

## Context
json_file_path: ${json_file_path}
json_schema: ${json_schema}
EOF
}

For more information on how to create custom tools for your agent and the structure of the agent/my-agent/tools.sh or agent/my-agent/tools.py files, refer to the Building Tools for Agents section below.

Variables

All the same variable interpolations supported by static instructions is also supported by dynamic instructions. For more information on what variables are available and how to use them, refer to the Special Variables and User-Defined Variables sections above.

3. Initializing RAG

Each agent you create also has a dedicated knowledge base that adds additional context to your queries and helps the LLM answer queries effectively. The documents to load into RAG are defined in the documents array of your agent configuration file:

documents:
  - https://www.ohdsi.org/data-standardization/
  - https://github.com/OHDSI/Vocabulary-v5.0/wiki/**
  - OMOPCDM_ddl.sql       # Relative path to agent (i.e. file lives at '<loki-config-dir>/agents/my-agent/OMOPCDM_ddl.sql')

These documents use the same syntax as those you'd define when constructing RAG normally. To see all the available types of documents that Loki supports and how to use custom document loaders, refer to the RAG documentation.

Anytime your agent starts up, it will automatically be using the RAG you've defined here.

4. Building Tools for Agents

Building tools for agents is virtually identical to building custom tools, with one slight difference: instead of defining a single function that gets executed at runtime (e.g. main for bash tools and run for Python tools), agent tools define a number of subcommands.

Limitations

You can only utilize either a bash-based <loki-config-dir>/agents/my-agent/tools.sh or a Python-based <loki-config-dir>/agents/my-agent/tools.py. However, if it's easier to achieve a task in one language vs the other, you're free to define other scripts in your agent's configuration directory and reference them from the main tools.py/sh file. Any scripts not named tools.{py,sh} will not be picked up by Loki's compiler, meaning they can be used like any other set of scripts.

It's important to keep in mind the following:

• Do not give agents the same name as an executable. Loki compiles the tools for each agent into a binary that it temporarily places on your path during execution. If you have a binary with the same name as your agent, then your shell may execute the existing binary instead of your agent's tools
• LLM_ROOT_DIR points to the agent's configuration directory. This is where agents differ slightly from normal tools: The LLM_ROOT_DIR environment variable does not point to the functions/tools directory like it does in global tools. Instead, it points to the agent's configuration directory, making it easier to source scripts and other miscellaneous files

.env File Support

When Loki loads an agent, it will also search the agent's configuration directory for a .env file. If found, all environment variables defined in the file will be made available to the agent's tools.

Python-Based Agent Tools

Python-based tools are defined exactly the same as they are for custom tool definitions. The only difference is that instead of a single run function, you define as many as you like with whatever arguments you like.

Example: agents/my-agent/tools.py

import urllib.request

def get_ip_info():
  """
  Get your IP information
  """
  with urllib.request.urlopen("https://httpbin.org/ip") as response:
    data = response.read()
    return data.decode('utf-8')

def get_ip_address_from_aws():
    """
    Find your public IP address using AWS
    """
    with urllib.request.urlopen("https://checkip.amazonaws.com") as response:
        data = response.read()
        return data.decode('utf-8')

Loki automatically compiles these as separate functions for the LLM to call. No extra work is needed. Just make sure you follow all the same steps to define each function as you would when creating custom Python tools.

For more information on how to build tools in Python, refer to the custom Python tools documentation

Bash-Based Agent Tools

Bash-based agent tools are virtually identical to custom bash tools, with only one difference. Instead of defining a single entrypoint via the main function, you actually define as many subcommands as you like.

Example: agents/my-agent/tools.sh

#!/usr/bin/env bash

# @env LLM_OUTPUT=/dev/stdout The output path
# @describe Discover network information about your computer and its place in the internet

# Use the `@cmd` annotation to define subcommands for your script.
# @cmd Get your IP information
get_ip_info() {
  curl -fsSL https://httpbin.org/ip >> "$LLM_OUTPUT"
}

# @cmd Find your public IP address using AWS
get_ip_address_from_aws() {
  curl -fsSL https://checkip.amazonaws.com >> "$LLM_OUTPUT"
}

To compile the script so it's executable and testable:

$ loki --build-tools

Then you can execute your script (assuming your current working directory is agents/my-agent):

$ ./tools.sh get_ip_info
$ ./tools.sh get_ip_address_from_aws

All other special annotations (@env, @arg, @option @flags) apply to subcommands as well, so be sure to follow the same syntax ad formatting as is used to create custom bash tools globally.

For more information on how to write, build and test tools in bash, refer to the custom bash tools documentation.

5. Conversation Starters

It's often helpful to also have some conversation starters so users know what kinds of things the agent is capable of doing. These are available in the REPL via the .starter command and are selectable.

They are defined using the conversation_starters setting in your agent's configuration file:

Example: agents/my-agent/config.yaml:

conversation_starters:
  - What is my username?
  - What is my current shell?
  - What is my ip?
  - How much disk space is left on my PC??
  - How to create an agent?

6. Todo System & Auto-Continuation

Loki includes a built-in task tracking system designed to improve the reliability of agents, especially when using smaller language models. The Todo System helps models:

• Break complex tasks into manageable steps
• Track progress through multi-step workflows
• Automatically continue work until all tasks are complete

Quick Configuration

# agents/my-agent/config.yaml
auto_continue: true              # Enable auto-continuation
max_auto_continues: 10           # Max continuation attempts
inject_todo_instructions: true   # Include the default todo instructions into prompt

How It Works

1. When inject_todo_instructions is enabled, agents receive instructions on using four built-in tools:

   • todo__init: Initialize a todo list with a goal
   • todo__add: Add a task to the list
   • todo__done: Mark a task complete
   • todo__list: View current todo state

   These instructions are a reasonable default that detail how to use Loki's To-Do System. If you wish, you can disable the injection of the default instructions and specify your own instructions for how to use the To-Do System into your main instructions for the agent.

2. When auto_continue is enabled and the model stops with incomplete tasks, Loki automatically sends a continuation prompt with the current todo state, nudging the model to continue working.

3. This continues until all tasks are done or max_auto_continues is reached.

When to Use

• Multistep tasks where the model might lose track
• Smaller models that need more structure
• Workflows requiring guaranteed completion of all steps

For complete documentation including all configuration options, tool details, and best practices, see the Todo System Guide.

Built-In Agents

Loki comes packaged with some useful built-in agents:

• coder: An agent to assist you with all your coding tasks
• demo: An example agent to use for reference when learning to create your own agents
• jira-helper: An agent that assists you with all your Jira-related tasks
• sql: A universal SQL agent that enables you to talk to any relational database in natural language