diff --git a/Graph-Agents.md b/Graph-Agents.md index 76e8440..ffee1d2 100644 --- a/Graph-Agents.md +++ b/Graph-Agents.md @@ -161,9 +161,8 @@ nodes: is stopped only by `settings.timeout`, an enclosing agent node's `timeout`, or Ctrl-C; if you also set `timeout: 0`, nothing bounds the run but completion. The validator warns on the explicit `0`. The `0 = no bound` - rule applies to `timeout` and `max_loop_iterations` only — - `max_iterations: 0` on an `llm` node runs zero turns and fails, and - `max_concurrency: 0` is a validation error. + rule applies to `timeout`, `max_loop_iterations`, and an `llm` node's + `max_iterations`; `max_concurrency: 0` is a validation error. - **`timeout`:** Wall-clock cap on the entire graph run. Checked before each super-step **and** enforced during super-steps: if the budget elapses while nodes are still running, the in-flight branches are aborted and the run @@ -574,7 +573,7 @@ ask_for_code: ## llm -A one-shot LLM call with an optional bounded tool-call loop. Unlike `agent` +A one-shot LLM call with an optionally bounded tool-call loop. Unlike `agent` nodes, this does NOT spawn a sub-agent; it runs in a fresh isolated context with a caller-supplied system prompt and user prompt. Tool access is strictly opt-in: an `llm` node gets **no tools at all** unless its `tools` field @@ -597,7 +596,7 @@ grade_research: top_p: null reasoning_effort: null # optional override; model-dependent effort level (e.g. low, medium, high) max_attempts: 1 # transient-error retries (default 1) - max_iterations: 10 # tool-call-loop turn cap (default 10) + max_iterations: 10 # tool-call-loop turn cap (default 10; 0 = no cap) fallback: skip # routes here if all attempts fail state_updates: grade: "{{output}}" @@ -680,6 +679,73 @@ failure message containing one of: `timed out`, `rate limit`, `429`, other error fails immediately without consuming further attempts. The default is `1` (no retries). +### Turn cap (`max_iterations`) + +`max_iterations` caps the number of model-call turns in one execution of the +node (default `10`). If the model is still calling tools on the last turn, the +node fails with `llm node hit max_iterations (N) before LLM concluded` and +routes to `fallback` if declared. `0` disables the cap; the validator warns +on the explicit `0`. + +**Watcher pattern (short/medium polls).** Use `max_iterations: 0` for a node +whose job is to poll a condition through tool calls (CI status, a file +appearing, a queue draining) and conclude only when it holds — a loop that +converges on an external signal, not on a turn count. Rules: + +- The *tool* does the waiting (`sleep 30 && check`), not the model. +- Keep tool output tiny (`pending` / `ready`), never a log dump. +- **Set a non-zero `timeout:` on the node.** An `llm` node's `timeout` is + unset by default, which is *no* bound. +- Leave `max_attempts` at `1`: a retry restarts from the original prompt and + replays every tool side effect. + +**What still ends an unbounded node, honestly:** + +- (a) The model concludes (returns without tool calls). +- (b) `timeout:` on the node or `settings.timeout` on the graph. Either fires + only between tool calls, so the effective bound is `timeout + one tool + call`. +- (c) **The context window.** Every turn appends its tool results to the + conversation and nothing compacts them between turns, so input cost grows + quadratically in the number of turns and the node eventually fails with + `Exceed max_input_tokens limit` (routes to `fallback` if declared) — an + error unrelated to anything you configured. +- (d) Ctrl-C, **only while `Generating…` is showing** (an in-flight model + call, on a TTY). During a tool's sleep it is swallowed: the tool exits with + code 130, the model sees a failed poll, and the watcher polls again. When it + does land it fails the *node* (→ `fallback`); it does not abort the graph. + +**For long-lived watchers, prefer a graph-level self-loop:** an `llm` check +node with the default cap, a `script` gate that emits `_next: check | done`, +and `settings.max_loop_iterations: 0`, with the wait in the script or the +tool. Each visit starts with a fresh conversation (flat context), Ctrl-C is +honored at every step boundary, and `settings.timeout` bounds the whole run. + +```yaml +settings: + max_loop_iterations: 0 + timeout: 3600 +start: check +nodes: + check: + type: llm + prompt: "Run `ci_status` once and reply with exactly `ready` or `pending`." + tools: [ci_status] + state_updates: { status: "{{output}}" } + next: gate + gate: + type: script + script: scripts/ci_gate.sh # reads `status` from GRAPH_STATE; prints + # {"_next":"done"} when ready, else sleeps 30s + # and prints {"_next":"check"} + done: + type: end + output: "{{status}}" +``` + +Inside a `map` branch, `max_iterations: 0` applies **per item**: with +`max_concurrency: N` you get up to `N` concurrent unbounded loops. + ### Skills on `llm` nodes `llm` nodes are the only place [skills](Skills) attach inside a graph. Two