docs(graph): llm max_iterations: 0 disables the turn cap; add Turn cap subsection

Dark-Alex-17 committed 2026-09-10 16:27:12 -06:00
1 parent 3ef6c0bffd
commit ae65ba3930
1 file changed
+71 -5
+71 -5
@@ -161,9 +161,8 @@ nodes:
is stopped only by `settings.timeout`, an enclosing agent node's `timeout`, is stopped only by `settings.timeout`, an enclosing agent node's `timeout`,
or Ctrl-C; if you also set `timeout: 0`, nothing bounds the run but or Ctrl-C; if you also set `timeout: 0`, nothing bounds the run but
completion. The validator warns on the explicit `0`. The `0 = no bound` completion. The validator warns on the explicit `0`. The `0 = no bound`
rule applies to `timeout` and `max_loop_iterations` only — rule applies to `timeout`, `max_loop_iterations`, and an `llm` node's
`max_iterations: 0` on an `llm` node runs zero turns and fails, and `max_iterations`; `max_concurrency: 0` is a validation error.
`max_concurrency: 0` is a validation error.
- **`timeout`:** Wall-clock cap on the entire graph run. Checked before each - **`timeout`:** Wall-clock cap on the entire graph run. Checked before each
super-step **and** enforced during super-steps: if the budget elapses while super-step **and** enforced during super-steps: if the budget elapses while
nodes are still running, the in-flight branches are aborted and the run nodes are still running, the in-flight branches are aborted and the run
@@ -574,7 +573,7 @@ ask_for_code:
## llm ## llm
A one-shot LLM call with an optional bounded tool-call loop. Unlike `agent` A one-shot LLM call with an optionally bounded tool-call loop. Unlike `agent`
nodes, this does NOT spawn a sub-agent; it runs in a fresh isolated context nodes, this does NOT spawn a sub-agent; it runs in a fresh isolated context
with a caller-supplied system prompt and user prompt. Tool access is strictly with a caller-supplied system prompt and user prompt. Tool access is strictly
opt-in: an `llm` node gets **no tools at all** unless its `tools` field opt-in: an `llm` node gets **no tools at all** unless its `tools` field
@@ -597,7 +596,7 @@ grade_research:
top_p: null top_p: null
reasoning_effort: null # optional override; model-dependent effort level (e.g. low, medium, high) reasoning_effort: null # optional override; model-dependent effort level (e.g. low, medium, high)
max_attempts: 1 # transient-error retries (default 1) max_attempts: 1 # transient-error retries (default 1)
max_iterations: 10 # tool-call-loop turn cap (default 10) max_iterations: 10 # tool-call-loop turn cap (default 10; 0 = no cap)
fallback: skip # routes here if all attempts fail fallback: skip # routes here if all attempts fail
state_updates: state_updates:
grade: "{{output}}" grade: "{{output}}"
@@ -680,6 +679,73 @@ failure message containing one of: `timed out`, `rate limit`, `429`,
other error fails immediately without consuming further attempts. The other error fails immediately without consuming further attempts. The
default is `1` (no retries). default is `1` (no retries).
### Turn cap (`max_iterations`)
`max_iterations` caps the number of model-call turns in one execution of the
node (default `10`). If the model is still calling tools on the last turn, the
node fails with `llm node hit max_iterations (N) before LLM concluded` and
routes to `fallback` if declared. `0` disables the cap; the validator warns
on the explicit `0`.
**Watcher pattern (short/medium polls).** Use `max_iterations: 0` for a node
whose job is to poll a condition through tool calls (CI status, a file
appearing, a queue draining) and conclude only when it holds — a loop that
converges on an external signal, not on a turn count. Rules:
- The *tool* does the waiting (`sleep 30 && check`), not the model.
- Keep tool output tiny (`pending` / `ready`), never a log dump.
- **Set a non-zero `timeout:` on the node.** An `llm` node's `timeout` is
unset by default, which is *no* bound.
- Leave `max_attempts` at `1`: a retry restarts from the original prompt and
replays every tool side effect.
**What still ends an unbounded node, honestly:**
- (a) The model concludes (returns without tool calls).
- (b) `timeout:` on the node or `settings.timeout` on the graph. Either fires
only between tool calls, so the effective bound is `timeout + one tool
call`.
- (c) **The context window.** Every turn appends its tool results to the
conversation and nothing compacts them between turns, so input cost grows
quadratically in the number of turns and the node eventually fails with
`Exceed max_input_tokens limit` (routes to `fallback` if declared) — an
error unrelated to anything you configured.
- (d) Ctrl-C, **only while `Generating…` is showing** (an in-flight model
call, on a TTY). During a tool's sleep it is swallowed: the tool exits with
code 130, the model sees a failed poll, and the watcher polls again. When it
does land it fails the *node* (→ `fallback`); it does not abort the graph.
**For long-lived watchers, prefer a graph-level self-loop:** an `llm` check
node with the default cap, a `script` gate that emits `_next: check | done`,
and `settings.max_loop_iterations: 0`, with the wait in the script or the
tool. Each visit starts with a fresh conversation (flat context), Ctrl-C is
honored at every step boundary, and `settings.timeout` bounds the whole run.
```yaml
settings:
max_loop_iterations: 0
timeout: 3600
start: check
nodes:
check:
type: llm
prompt: "Run `ci_status` once and reply with exactly `ready` or `pending`."
tools: [ci_status]
state_updates: { status: "{{output}}" }
next: gate
gate:
type: script
script: scripts/ci_gate.sh # reads `status` from GRAPH_STATE; prints
# {"_next":"done"} when ready, else sleeps 30s
# and prints {"_next":"check"}
done:
type: end
output: "{{status}}"
```
Inside a `map` branch, `max_iterations: 0` applies **per item**: with
`max_concurrency: N` you get up to `N` concurrent unbounded loops.
### Skills on `llm` nodes ### Skills on `llm` nodes
`llm` nodes are the only place [skills](Skills) attach inside a graph. Two `llm` nodes are the only place [skills](Skills) attach inside a graph. Two