docs(graph): llm max_iterations: 0 disables the turn cap; add Turn cap subsection
1 file changed
+71
-5
+71
-5
@@ -161,9 +161,8 @@ nodes:
|
|||||||
is stopped only by `settings.timeout`, an enclosing agent node's `timeout`,
|
is stopped only by `settings.timeout`, an enclosing agent node's `timeout`,
|
||||||
or Ctrl-C; if you also set `timeout: 0`, nothing bounds the run but
|
or Ctrl-C; if you also set `timeout: 0`, nothing bounds the run but
|
||||||
completion. The validator warns on the explicit `0`. The `0 = no bound`
|
completion. The validator warns on the explicit `0`. The `0 = no bound`
|
||||||
rule applies to `timeout` and `max_loop_iterations` only —
|
rule applies to `timeout`, `max_loop_iterations`, and an `llm` node's
|
||||||
`max_iterations: 0` on an `llm` node runs zero turns and fails, and
|
`max_iterations`; `max_concurrency: 0` is a validation error.
|
||||||
`max_concurrency: 0` is a validation error.
|
|
||||||
- **`timeout`:** Wall-clock cap on the entire graph run. Checked before each
|
- **`timeout`:** Wall-clock cap on the entire graph run. Checked before each
|
||||||
super-step **and** enforced during super-steps: if the budget elapses while
|
super-step **and** enforced during super-steps: if the budget elapses while
|
||||||
nodes are still running, the in-flight branches are aborted and the run
|
nodes are still running, the in-flight branches are aborted and the run
|
||||||
@@ -574,7 +573,7 @@ ask_for_code:
|
|||||||
|
|
||||||
## llm
|
## llm
|
||||||
|
|
||||||
A one-shot LLM call with an optional bounded tool-call loop. Unlike `agent`
|
A one-shot LLM call with an optionally bounded tool-call loop. Unlike `agent`
|
||||||
nodes, this does NOT spawn a sub-agent; it runs in a fresh isolated context
|
nodes, this does NOT spawn a sub-agent; it runs in a fresh isolated context
|
||||||
with a caller-supplied system prompt and user prompt. Tool access is strictly
|
with a caller-supplied system prompt and user prompt. Tool access is strictly
|
||||||
opt-in: an `llm` node gets **no tools at all** unless its `tools` field
|
opt-in: an `llm` node gets **no tools at all** unless its `tools` field
|
||||||
@@ -597,7 +596,7 @@ grade_research:
|
|||||||
top_p: null
|
top_p: null
|
||||||
reasoning_effort: null # optional override; model-dependent effort level (e.g. low, medium, high)
|
reasoning_effort: null # optional override; model-dependent effort level (e.g. low, medium, high)
|
||||||
max_attempts: 1 # transient-error retries (default 1)
|
max_attempts: 1 # transient-error retries (default 1)
|
||||||
max_iterations: 10 # tool-call-loop turn cap (default 10)
|
max_iterations: 10 # tool-call-loop turn cap (default 10; 0 = no cap)
|
||||||
fallback: skip # routes here if all attempts fail
|
fallback: skip # routes here if all attempts fail
|
||||||
state_updates:
|
state_updates:
|
||||||
grade: "{{output}}"
|
grade: "{{output}}"
|
||||||
@@ -680,6 +679,73 @@ failure message containing one of: `timed out`, `rate limit`, `429`,
|
|||||||
other error fails immediately without consuming further attempts. The
|
other error fails immediately without consuming further attempts. The
|
||||||
default is `1` (no retries).
|
default is `1` (no retries).
|
||||||
|
|
||||||
|
### Turn cap (`max_iterations`)
|
||||||
|
|
||||||
|
`max_iterations` caps the number of model-call turns in one execution of the
|
||||||
|
node (default `10`). If the model is still calling tools on the last turn, the
|
||||||
|
node fails with `llm node hit max_iterations (N) before LLM concluded` and
|
||||||
|
routes to `fallback` if declared. `0` disables the cap; the validator warns
|
||||||
|
on the explicit `0`.
|
||||||
|
|
||||||
|
**Watcher pattern (short/medium polls).** Use `max_iterations: 0` for a node
|
||||||
|
whose job is to poll a condition through tool calls (CI status, a file
|
||||||
|
appearing, a queue draining) and conclude only when it holds — a loop that
|
||||||
|
converges on an external signal, not on a turn count. Rules:
|
||||||
|
|
||||||
|
- The *tool* does the waiting (`sleep 30 && check`), not the model.
|
||||||
|
- Keep tool output tiny (`pending` / `ready`), never a log dump.
|
||||||
|
- **Set a non-zero `timeout:` on the node.** An `llm` node's `timeout` is
|
||||||
|
unset by default, which is *no* bound.
|
||||||
|
- Leave `max_attempts` at `1`: a retry restarts from the original prompt and
|
||||||
|
replays every tool side effect.
|
||||||
|
|
||||||
|
**What still ends an unbounded node, honestly:**
|
||||||
|
|
||||||
|
- (a) The model concludes (returns without tool calls).
|
||||||
|
- (b) `timeout:` on the node or `settings.timeout` on the graph. Either fires
|
||||||
|
only between tool calls, so the effective bound is `timeout + one tool
|
||||||
|
call`.
|
||||||
|
- (c) **The context window.** Every turn appends its tool results to the
|
||||||
|
conversation and nothing compacts them between turns, so input cost grows
|
||||||
|
quadratically in the number of turns and the node eventually fails with
|
||||||
|
`Exceed max_input_tokens limit` (routes to `fallback` if declared) — an
|
||||||
|
error unrelated to anything you configured.
|
||||||
|
- (d) Ctrl-C, **only while `Generating…` is showing** (an in-flight model
|
||||||
|
call, on a TTY). During a tool's sleep it is swallowed: the tool exits with
|
||||||
|
code 130, the model sees a failed poll, and the watcher polls again. When it
|
||||||
|
does land it fails the *node* (→ `fallback`); it does not abort the graph.
|
||||||
|
|
||||||
|
**For long-lived watchers, prefer a graph-level self-loop:** an `llm` check
|
||||||
|
node with the default cap, a `script` gate that emits `_next: check | done`,
|
||||||
|
and `settings.max_loop_iterations: 0`, with the wait in the script or the
|
||||||
|
tool. Each visit starts with a fresh conversation (flat context), Ctrl-C is
|
||||||
|
honored at every step boundary, and `settings.timeout` bounds the whole run.
|
||||||
|
|
||||||
|
```yaml
|
||||||
|
settings:
|
||||||
|
max_loop_iterations: 0
|
||||||
|
timeout: 3600
|
||||||
|
start: check
|
||||||
|
nodes:
|
||||||
|
check:
|
||||||
|
type: llm
|
||||||
|
prompt: "Run `ci_status` once and reply with exactly `ready` or `pending`."
|
||||||
|
tools: [ci_status]
|
||||||
|
state_updates: { status: "{{output}}" }
|
||||||
|
next: gate
|
||||||
|
gate:
|
||||||
|
type: script
|
||||||
|
script: scripts/ci_gate.sh # reads `status` from GRAPH_STATE; prints
|
||||||
|
# {"_next":"done"} when ready, else sleeps 30s
|
||||||
|
# and prints {"_next":"check"}
|
||||||
|
done:
|
||||||
|
type: end
|
||||||
|
output: "{{status}}"
|
||||||
|
```
|
||||||
|
|
||||||
|
Inside a `map` branch, `max_iterations: 0` applies **per item**: with
|
||||||
|
`max_concurrency: N` you get up to `N` concurrent unbounded loops.
|
||||||
|
|
||||||
### Skills on `llm` nodes
|
### Skills on `llm` nodes
|
||||||
|
|
||||||
`llm` nodes are the only place [skills](Skills) attach inside a graph. Two
|
`llm` nodes are the only place [skills](Skills) attach inside a graph. Two
|
||||||
|
|||||||
Reference in new issue
Block a user