diff --git a/Agents.md b/Agents.md index 9e3ebe5..0ec20f2 100644 --- a/Agents.md +++ b/Agents.md @@ -466,8 +466,8 @@ conversation_starters: # 6. Todo System & Auto-Continuation -Loki includes a built-in task tracking system designed to improve the reliability of agents, especially when using -smaller language models. The Todo System helps models: +Loki includes a built-in task tracking system designed to improve the reliability of LLM interactions, especially +when using smaller language models. The Todo System helps models: - Break complex tasks into manageable steps - Track progress through multi-step workflows @@ -480,6 +480,7 @@ smaller language models. The Todo System helps models: auto_continue: true # Enable auto-continuation max_auto_continues: 10 # Max continuation attempts inject_todo_instructions: true # Include the default todo instructions into prompt +continuation_prompt: null # Custom prompt used when auto-continuing if inject_todo_instructions is 'false' ``` ## How It Works diff --git a/REPL.md b/REPL.md index 9b01c0f..cb2c825 100644 --- a/REPL.md +++ b/REPL.md @@ -95,19 +95,29 @@ For more information on sessions and how to use them in Loki, refer to the [sess Loki lets you build OpenAI GPT-style agents. The following commands let you interact with and manage your agents in Loki: -| Command | Description | -|----------------------|-----------------------------------------------------------------------------------------------| -| `.agent` | Use an agent | -| `.starter` | Display and use conversation starters for the active agent | -| `.clear todo` | Clear the todo list and stop auto-continuation (requires `auto_continue: true` on the agent) | -| `.edit agent-config` | Open the agent configuration in your preferred text editor | -| `.info agent` | Display information about the active agent | -| `.exit agent` | Leave the active agent | +| Command | Description | +|----------------------|-------------------------------------------------------------------| +| `.agent` | Use an agent | +| `.starter` | Display and use conversation starters for the active agent | +| `.edit agent-config` | Open the agent configuration in your preferred text editor | +| `.info agent` | Display information about the active agent | +| `.exit agent` | Leave the active agent | ![agent](./images/agents/sql.gif) For more information on agents in Loki and how to create them, refer to the [agents documentation](Agents). +## `.clear todo` - Clear the todo list +Clears the todo list and stops auto-continuation. This command is available in any context (agents, sessions, +roles, or bare mode) as long as `auto_continue` is enabled. + +This is useful when: +- You gave a custom response that changes or cancels the current task +- The model is stuck in auto-continuation with stale todos +- You want to start fresh + +For more information, see the [Todo System documentation](TODO-System). + ## `.rag` - Chat with documents RAG (Retrieval Augmented Generation) enables you to load documents into the LLM so you can ask questions about it or complete tasks using the documents as additional context. @@ -178,6 +188,28 @@ You can use `.set` to adjust select settings at runtime. This is useful when you to know how they'll affect Loki. To persist the changes you make, be sure to update them in the global configuration file. +The following settings can be adjusted at runtime: + +| Setting | Type | Description | +|----------------------------|---------|--------------------------------------------------------------------------| +| `auto_continue` | boolean | Enable/disable the [Todo System](TODO-System) auto-continuation | +| `max_auto_continues` | integer | Maximum number of automatic continuations | +| `inject_todo_instructions` | boolean | Inject default todo instructions into the system prompt | +| `continuation_prompt` | string | Custom continuation prompt (supports multi-word values; `null` to reset) | +| `temperature` | float | Model temperature parameter | +| `top_p` | float | Model top-p parameter | +| `enabled_tools` | string | Comma-separated list of enabled tools | +| `enabled_mcp_servers` | string | Comma-separated list of enabled MCP servers | +| `save_session` | boolean | Whether to auto-save sessions | +| `compression_threshold` | integer | Token threshold for session compression | +| `max_output_tokens` | integer | Maximum output tokens for the current model | +| `dry_run` | boolean | Enable/disable dry run mode | +| `function_calling_support` | boolean | Enable/disable function calling | +| `mcp_server_support` | boolean | Enable/disable MCP server support | +| `stream` | boolean | Enable/disable streaming | +| `save` | boolean | Enable/disable saving responses | +| `highlight` | boolean | Enable/disable syntax highlighting | + ![set](./images/repl/set.gif) ## `.edit` - Modify configuration files diff --git a/Roles.md b/Roles.md index e9a9489..a0435d5 100644 --- a/Roles.md +++ b/Roles.md @@ -50,15 +50,19 @@ want to feed the model. The following table lists the available configuration settings and their default values (if undefined): -| Setting | Default | Description | -|-----------------------|----------------------------------------------------------------|---------------------------------------------------------------------------------------------------------------| -| `name` | The name of the role markdown file | The name of the role | -| `model` | Default configured model or currently in-use model (REPL mode) | The preferred model to use with this role | -| `temperature` | Default `temperature` for the preferred model | Controls the creativity and randomness of the model's responses | -| `top_p` | Default `top_p` for the preferred model | Alternative way to control the model's output diversity, affecting the
probability distribution of tokens | -| `enabled_tools` | Global setting for `enabled_tools` | The tools that this role utilizes | -| `enabled_mcp_servers` | Global setting for `enabled_mcp_servers` | The MCP servers that this role utilizes | -| `prompt` | `null` | See [Prompt Types](#prompt-types) for detailed usage | +| Setting | Default | Description | +|----------------------------|----------------------------------------------------------------|---------------------------------------------------------------------------------------------------------------| +| `name` | The name of the role markdown file | The name of the role | +| `model` | Default configured model or currently in-use model (REPL mode) | The preferred model to use with this role | +| `temperature` | Default `temperature` for the preferred model | Controls the creativity and randomness of the model's responses | +| `top_p` | Default `top_p` for the preferred model | Alternative way to control the model's output diversity, affecting the
probability distribution of tokens | +| `enabled_tools` | Global setting for `enabled_tools` | The tools that this role utilizes | +| `enabled_mcp_servers` | Global setting for `enabled_mcp_servers` | The MCP servers that this role utilizes | +| `auto_continue` | Global setting for `auto_continue` | Enable the [Todo System](TODO-System) auto-continuation for this role | +| `max_auto_continues` | Global setting for `max_auto_continues` | Maximum number of automatic continuations before stopping | +| `inject_todo_instructions` | Global setting for `inject_todo_instructions` | Inject default todo tool usage instructions into the system prompt | +| `continuation_prompt` | Global setting for `continuation_prompt` | Custom prompt used when auto-continuing | +| `prompt` | `null` | See [Prompt Types](#prompt-types) for detailed usage | ## Instructions The instructions for a role is what you use to tell the model how you want it to behave. This typically consists of one diff --git a/Sessions.md b/Sessions.md index b2aa4b2..f6d1376 100644 --- a/Sessions.md +++ b/Sessions.md @@ -41,3 +41,16 @@ The following settings are available to customize the default behavior of sessio | `compression_threshold` | Defines the token count threshold at which Loki will compress the session to save on the context length | | `summarization_prompt` | This is the prompt that is used to compress the session up to a given point when compression is triggered | | `summary_context_prompt` | This is the prompt that's used to add the summarized conversation generated by the `summarization_prompt` as context to the model | + +## Per-Session Settings +In addition to the global settings above, individual sessions can override the following settings. These can be set +at runtime using the `.set` command or configured in the session's YAML file: + +| Setting | Default | Description | +|----------------------------|-----------------------------------------|---------------------------------------------------------------------------------------------------------------| +| `auto_continue` | Global `auto_continue` value | Enable the [Todo System](TODO-System) auto-continuation for this session. Overrides global and role settings. | +| `max_auto_continues` | Global `max_auto_continues` value | Maximum number of automatic continuations before stopping. Overrides global and role settings. | +| `inject_todo_instructions` | Global `inject_todo_instructions` value | Inject default todo tool usage instructions into the system prompt. Overrides global and role settings. | +| `continuation_prompt` | Global `continuation_prompt` value | Custom prompt used when auto-continuing. Overrides global and role settings. | + +For more information on the Todo System, see the [Todo System documentation](TODO-System). diff --git a/TODO-System.md b/TODO-System.md index f4b58ac..e3b28e4 100644 --- a/TODO-System.md +++ b/TODO-System.md @@ -1,5 +1,5 @@ -Loki's Todo System is a built-in task tracking feature designed to improve the reliability and effectiveness of LLM agents, -especially smaller models. It provides structured task management that helps models: +Loki's Todo System is a built-in task tracking feature designed to improve the reliability and effectiveness of LLM +interactions, especially with smaller models. It provides structured task management that helps models: - Break complex tasks into manageable steps - Track progress through multistep workflows @@ -26,20 +26,79 @@ The system automatically prompts the model to continue when incomplete tasks rem The todo state is preserved across the conversation (and any compressions), and injected into continuation prompts, keeping the model focused on remaining work. +# Where It Works + +The Todo System works in **all contexts** where function calling is enabled: + +| Context | How to Enable | +|----------------|--------------------------------------------------------------------------------| +| **Global** | Set `auto_continue: true` in your `config.yaml` | +| **Roles** | Add `auto_continue: true` to the role's metadata header | +| **Sessions** | Use `.set auto_continue true` or set it in the session's YAML | +| **Agents** | Set `auto_continue: true` in the agent's `config.yaml` (agent-specific config) | +| **REPL** | Use `.set auto_continue true` at runtime | + +When multiple levels are configured, the following precedence applies: + +``` +Agent config > Session config > Role config > Global config > Default (false) +``` + +> **Note:** The Todo System requires `function_calling_support: true` (the default) to work, since todo tracking +> is implemented via tool calls. + # Configuration Options -The Todo System is configured per-agent in `/agents//config.yaml`: -| Setting | Type | Default | Description | -|----------------------------|---------|-------------|---------------------------------------------------------------------------------| -| `auto_continue` | boolean | `false` | Enable the To-Do system for automatic continuation when incomplete todos remain | -| `max_auto_continues` | integer | `10` | Maximum number of automatic continuations before stopping | -| `inject_todo_instructions` | boolean | `true` | Inject the default todo tool usage instructions into the agent's system prompt | -| `continuation_prompt` | string | (see below) | Custom prompt used when auto-continuing | +## Global Configuration +The Todo System can be configured globally in your `config.yaml`: + +| Setting | Type | Default | Description | +|----------------------------|---------|-------------|-------------------------------------------------------------------------| +| `auto_continue` | boolean | `false` | Enable automatic continuation when incomplete todos remain | +| `max_auto_continues` | integer | `10` | Maximum number of automatic continuations before stopping | +| `inject_todo_instructions` | boolean | `true` | Inject the default todo tool usage instructions into the system prompt | +| `continuation_prompt` | string | (see below) | Custom prompt used when auto-continuing | + +### Example Global Configuration +```yaml +# config.yaml +auto_continue: true # Enable auto-continuation globally +max_auto_continues: 15 # Allow up to 15 automatic continuations +inject_todo_instructions: true # Include todo instructions in system prompt +continuation_prompt: null # Use the built-in default (or set a custom string) +``` + +## Role Configuration +Roles can opt into auto-continuation via their metadata header. All four settings can be overridden at the role +level, with unset values falling through to the global config. + +```markdown +--- +auto_continue: true +max_auto_continues: 5 +inject_todo_instructions: true +continuation_prompt: "Continue with the next task." +--- +You are a helpful coding assistant. +``` + +## Session Configuration +Sessions can override all auto-continuation settings. These are persisted in the session's YAML file: + +```yaml +auto_continue: true +max_auto_continues: 10 +inject_todo_instructions: true +continuation_prompt: null +``` + +You can also change these at runtime using `.set` (see [Runtime Settings](#runtime-settings) below). + +## Agent Configuration +Agents have the richest configuration, with all four settings available in `/agents//config.yaml`: -## Example Configuration ```yaml # agents/my-agent/config.yaml -model: openai:gpt-4o auto_continue: true # Enable auto-continuation max_auto_continues: 15 # Allow up to 15 automatic continuations inject_todo_instructions: true # Include todo instructions in system prompt @@ -49,17 +108,40 @@ continuation_prompt: | # Optional: customize the continuation prompt Do not explain; just execute. ``` +When an agent is active, its configuration takes full precedence over session, role, and global settings. + ## Default Continuation Prompt If `continuation_prompt` is not specified, the following default is used: ``` [SYSTEM REMINDER - TODO CONTINUATION] -You have incomplete tasks in your todo list. Continue with the next pending item. -Call tools immediately. Do not explain what you will do. +You have incomplete tasks. Rules: +1. BEFORE marking a todo done: verify the work compiles/works. No premature completion. +2. If a todo is broad (e.g. "implement X and implement Y"): break it into specific subtasks FIRST using todo__add, then work on those. +3. Each todo should be atomic and be "single responsibility" - completable in one focused action. +4. Continue with the next pending item now. Call tools immediately. ``` +## Runtime Settings +You can toggle auto-continuation settings at runtime using the `.set` command in the REPL: + +| Command | Description | +|-----------------------------------------------|--------------------------------------------------------------------| +| `.set auto_continue true` | Enable auto-continuation for the current session (or globally) | +| `.set auto_continue false` | Disable auto-continuation | +| `.set max_auto_continues 20` | Change the maximum number of continuations | +| `.set inject_todo_instructions false` | Disable injection of default todo instructions into system prompt | +| `.set continuation_prompt ` | Set a custom continuation prompt (supports multi-word values) | +| `.set continuation_prompt null` | Reset continuation prompt to the built-in default | + +When a session is active, `.set auto_continue` and `.set max_auto_continues` modify the session config. +Otherwise, they modify the global config for the remainder of the REPL session. + +When you enable `auto_continue` via `.set`, the todo tools are automatically made available without needing to +restart or reload. + # Available Tools -When `inject_todo_instructions` is enabled (the default), agents have access to four built-in todo management tools: +When `inject_todo_instructions` is enabled (the default), the model has access to five built-in todo management tools: ## `todo__init` Initialize a new todo list with a goal. Clears any existing todos. @@ -113,16 +195,16 @@ Clear the entire todo list and reset the goal. Use when the current task has bee ## REPL Command: `.clear todo` You can also clear the todo list manually from the REPL by typing `.clear todo`. This is useful when: - You gave a custom response that changes or cancels the current task -- The agent is stuck in auto-continuation with stale todos -- You want to start fresh without leaving and re-entering the agent +- The model is stuck in auto-continuation with stale todos +- You want to start fresh -**Note:** This command is only available when an agent with `auto_continue: true` is active. If the todo -system isn't enabled for the current agent, the command will display an error message. +This command is available in all contexts (agents, sessions, roles, or bare mode) as long as `auto_continue` +is enabled. If auto-continuation isn't enabled, the command will display an error message. # Auto-Continuation When `auto_continue` is enabled, Loki automatically sends a continuation prompt if: -1. The agent's response completes (model stops generating) +1. The model's response completes (model stops generating) 2. There are incomplete tasks in the todo list 3. The continuation count hasn't exceeded `max_auto_continues` 4. The response isn't identical to the previous continuation (prevents loops) @@ -133,30 +215,39 @@ Each continuation prompt includes: - The current todo list state showing: - The goal - Progress (e.g., "3/5 completed") - - Each task with status (✓ done, ○ pending) + - Each task with status (done, pending) **Example continuation context:** ``` [SYSTEM REMINDER - TODO CONTINUATION] -You have incomplete tasks in your todo list. Continue with the next pending item. -Call tools immediately. Do not explain what you will do. +You have incomplete tasks. Rules: +1. BEFORE marking a todo done: verify the work compiles/works. No premature completion. +2. If a todo is broad: break it into specific subtasks FIRST using todo__add, then work on those. +3. Each todo should be atomic and be "single responsibility" - completable in one focused action. +4. Continue with the next pending item now. Call tools immediately. Goal: Refactor the authentication module Progress: 2/4 completed - ✓ 1. Extract password validation into separate function - ✓ 2. Add unit tests for password validation - ○ 3. Update login handler to use new validation - ○ 4. Update registration handler to use new validation + done 1. Extract password validation into separate function + done 2. Add unit tests for password validation + pending 3. Update login handler to use new validation + pending 4. Update registration handler to use new validation ``` ## Visual Feedback During auto-continuation, you'll see a message in your terminal: ``` -📋 Auto-continuing (3/10): 2 incomplete todo(s) remain +Auto-continuing (3/10): 2 incomplete todo(s) remain ``` # Best Practices +## For Users +1. **Start small**: Try `.set auto_continue true` in the REPL before committing to global config +2. **Set reasonable limits**: Adjust `max_auto_continues` based on typical task complexity +3. **Use roles for recurring patterns**: Create a role with `auto_continue: true` for task-heavy workflows +4. **Customize the prompt**: If the model needs specific continuation behavior, set a custom `continuation_prompt` + ## For Agent Developers 1. **Enable for complex workflows**: Use `auto_continue: true` for agents that handle multistep tasks 2. **Set reasonable limits**: Adjust `max_auto_continues` based on typical task complexity @@ -213,8 +304,9 @@ Here's how a typical todo-driven workflow looks: # Troubleshooting ## Model Not Using Todo Tools -- Verify `inject_todo_instructions: true` in your agent config -- Check that the agent is properly loaded (not just a role) +- Verify `inject_todo_instructions: true` in your config +- Check that `function_calling_support: true` is set (the default) +- Check that `auto_continue: true` is set in the appropriate config level - Some models may need explicit prompting to use the tools ## Too Many Continuations @@ -232,5 +324,6 @@ automatically. If you're seeing loops: # Additional Docs - [Agents](Agents) - Full agent configuration guide +- [Roles](Roles) - Role configuration with auto_continue support +- [Sessions](Sessions) - Session configuration with auto_continue support - [Function Calling](Tools) - How tools work in Loki -- [Sessions](Sessions) - How conversation state is managed diff --git a/_Sidebar.md b/_Sidebar.md index 23cfe5c..e1f36db 100644 --- a/_Sidebar.md +++ b/_Sidebar.md @@ -34,12 +34,12 @@ - [Sub-Agent Spawning](Agents#7-sub-agent-spawning-system) - [User Interaction Tools](Agents#8-user-interaction-tools) - [Built-In Agents](Agents#built-in-agents) -- [TODO System](TODO-System) ## Knowledge & Automation - [RAG](RAG) - [Macros](Macros) - [Roles](Roles) +- [TODO System](TODO-System) ## Configuration - [Environment Variables](Environment-Variables)