From 8375bcc352f835b91b9b460cd44095a8dd3b1dba Mon Sep 17 00:00:00 2001 From: Alex Clarke Date: Wed, 9 Sep 2026 20:06:13 -0600 Subject: [PATCH] docs: map branches are per-item subgraph chains Rewrite the map section for subgraph branches: field reference with the templated max_concurrency, branch subgraph rules in user terms (incl. the now-honored fallback and the load error for fallback: end_*), the output contract (key removed from the fork, read-before-write outcomes, the null residual with the default key), a worked spawn -> verify -> retry-or-accept example, and what stays unsupported. Fix the routing table, dynamic fan-out and troubleshooting sentences that said a branch cannot have next, and add the chain-lifetime and accepted-then-unread notes to teammate messaging. --- Graph-Agents.md | 293 +++++++++++++++++++++++++++++++++++++++--------- 1 file changed, 240 insertions(+), 53 deletions(-) diff --git a/Graph-Agents.md b/Graph-Agents.md index 7a40346..c2d2771 100644 --- a/Graph-Agents.md +++ b/Graph-Agents.md @@ -719,8 +719,14 @@ it replaces RRF and graph search is not applied. Dynamic fan-out over a runtime-determined list. The `map` node reads a JSON array out of state (via the `over:` template), spawns one parallel -sub-branch per item invoking the same branch node, and collects each -sub-branch's output into a state array in input order. +sub-branch per item, and collects each sub-branch's result into a state +array in input order. + +Each sub-branch runs a **per-item chain** of nodes on a private fork of the +state. The chain starts at the node named by `branch:` (with the item bound +to `as:`) and follows that node's `next:`, script `_next`, and `fallback:` +edges until it reaches a node with no `next` and no `_next`. A single node +with no `next:` is still the simplest branch: a one-step chain. This is how a graph agent runs "research these N subjects in parallel" when N is determined at run-time. For a fixed-cardinality fan-out (you know the @@ -733,19 +739,19 @@ fan_out_subjects: type: map over: "{{subjects}}" # required; template must resolve to a JSON array as: subject # required; bound name inside each sub-branch - branch: research_subject # required; node id to invoke once per item - output_key: output # optional; state key the branch writes (default "output") - collect_into: research_results # required; state key to receive the array of outputs - max_concurrency: 5 # optional; overrides settings.max_concurrency + branch: research_subject # required; ENTRY node of the per-item chain + output_key: finding # optional; state key the chain writes (default "output") + collect_into: research_results # required; state key to receive the array of results + max_concurrency: 5 # optional; integer or "{{key}}" template next: rank # required; where to go after the map collects -research_subject: # the branch node (atomic — runs N times) +research_subject: # chain step 1 (runs once per item) id: research_subject type: llm prompt: "Research {{subject}} in depth." state_updates: - output: "{{output}}" # branch must write to `output_key` - # Note: no `next:` on a map branch; map branches are atomic + finding: "{{output}}" # the chain must write `output_key` + # no `next:` here, so this is also the last step: the chain ends ``` ### Field reference @@ -755,42 +761,176 @@ research_subject: # the branch node (atomic — runs N times) empty, the map runs zero sub-branches and writes `[]` to `collect_into` (which is correct semantically. I.e. mapping over nothing yields nothing). - **`as`:** The state key under which each item is bound inside its - sub-branch. The branch node then references it via `{{}}` in its - prompt / templates. -- **`branch`:** The id of the node to invoke per item. The branch's - outputs across all sub-branches are aggregated into `collect_into`. -- **`output_key`:** The state key each sub-branch must write its result - to. Default `"output"`. The map executor reads this from each - sub-branch's state, extracts the value, and inserts it into the - `collect_into` array. + sub-branch. Every node in the chain can read it via `{{}}` in its + prompt / templates (or from `GRAPH_STATE` in a script). +- **`branch`:** The id of the chain's **entry** node. The branch subgraph is + that node plus everything reachable from it through declared `next:` and + `fallback:` edges; the validator checks the whole subgraph (see + [Branch subgraph rules](#branch-subgraph-rules)). +- **`output_key`:** The state key whose value in the item's **final** fork + state becomes the item's result. Default `"output"`. The chain must + write it; see [Output contract](#output-contract). - **`collect_into`:** The state key in the **parent's** state that receives the array of all sub-branch outputs, **in input-list order regardless of which sub-branch finished first**. This is the map's user-visible determinism contract. -- **`max_concurrency`:** Per-map cap on concurrent sub-branches; falls - back to `settings.max_concurrency` (default 8) when unset. Must be - `>= 1` if set. +- **`max_concurrency`:** Cap on items in flight. One slot is held for an + item's **whole chain**, so the cap counts items, not steps. Falls back to + `settings.max_concurrency` (default 8) when unset. Accepts either an + integer (`>= 1`) or a `"{{key}}"` template that must resolve at run time + to a positive integer: a JSON number or a numeric string, so a script can + emit `{"review_budget": "3"}`. Anything else (`0`, `"high"`, a missing + key) fails the map node with an error naming the template and the value + it resolved to. A quoted string with no `{{...}}` in it (e.g. `"4"`) is + rejected at load time: write an integer or a template. + `settings.max_concurrency` itself cannot be templated. - **`next`:** Required. Where the parent super-step continues after the map collects. -### Branch node constraints +### Branch subgraph rules -Map branches are **atomic**. One node, one execution per item, no -chaining inside the branch. The validator enforces several rules at load -time: +At load time the validator computes each map's branch subgraph (the entry +node plus everything reachable through `next:` and `fallback:`) and +enforces these rules. Errors are reported on the map node and name the +offending node. -- Branch type must be `llm`, `agent`, `rag`, or `script`. Approval, input, - end, or another map node as the branch is a load-time error. -- Branch must **not** have `next:` declared. Anything chained off the - branch belongs after the map (downstream of the `collect_into` write). -- Branch's `state_updates` keys must be a subset of `output_key`. The - branch can write to its own `output_key` and nowhere else. -- Branch must **not** declare `output_schema`. Top-level schema-properties - would auto-merge into state, polluting it across N parallel branches. - Use explicit `state_updates: { output: "{{output}}" }` instead. +- **Node types.** Every node in the subgraph must be `llm`, `agent`, `rag`, + or `script`. An `approval`, `input`, `end`, or `map` node anywhere in the + subgraph is a load error. A chain ends by *omission*, never with an `end` + node, so the main-flow idiom `fallback: end_failure` is an error on a + branch node and gets a targeted hint: *a branch's `fallback` must stay + inside the branch subgraph*. **Before this release `fallback:` on a map + branch was accepted but never honored.** It is honored now, per item, so + either point it at a branch-local recovery node or remove it. +- **No fan-out inside a branch.** `next: [a, b]` on any subgraph node is a + load error. Restructure as one flattened map, or a second map after the + join. +- **Disjoint from the main flow.** No subgraph node may also be reachable + from `start` without entering a map branch (`next`, `fallback`, and + approval routes all count as main-flow edges). Such a node would run + standalone, with no `as` binding, and could route into the middle of a + chain. The error lists the shared node(s). A branch `next:` that escapes + to the main flow trips this rule, and if the escape path also reaches an + `end` (or approval/input/map) node the node-type rule fires for that node + too. Both errors are correct; fix the escaping edge. The check is static: + a main-flow *script* `_next` into a branch node is not detected, like any + other undeclared `_next`. +- **Writes are fork-local.** Chain nodes may use `state_updates` with any + keys, and `output_schema`. Everything they write lives in the item's fork + and is discarded at collect, except `output_key`. +- **Declared writer for `output_key` (warning).** If no node in the + subgraph declares a write to `output_key` (a `state_updates` key or an + `output_schema` property) the validator warns, because the map will error + at run time if the chain never writes it. Any `script` node in the + subgraph suppresses the warning: scripts can write any key. +- **No declared cycles.** `next:`/`fallback:` back to an earlier chain node + is a cycle and is rejected like any other declared cycle. Per-item loops + are script `_next` loops; see + [Retrying inside a branch](#retrying-inside-a-branch-with-_next). +- Two maps **may** share a branch subgraph. Forks are private, so a + per-item pipeline can be reused by several maps. -If a sub-branch fails to write `output_key`, the map node errors loudly -(no silent missing-output behavior). +### Output contract + +- The item's result is the value of `output_key` in its **final** fork + state. Any chain node may write it; the last write wins. +- **The chain must write it.** The map removes `output_key` from each + item's fork before the chain starts, so a value the parent held is never + collected as if the item produced it. An item that finishes without + writing the key fails the map: + + ``` + map node 'fan_out_subjects': sub-branch [1] did not write output_key 'finding'; + write it via state_updates, output_schema, or a script's JSON output + (the parent's value is not inherited inside a map branch) + ``` + + Writing a value equal to the parent's is fine: removal, not diffing, + decides whether the chain wrote. +- **Reading `output_key` before writing it** sees the key absent, and + where the read happens decides what that means: in a + `prompt`/`instructions` template it is a strict-interpolation error; in a + `state_updates` template it silently renders as `""`; in a script the key + is simply missing from `GRAPH_STATE`. The accumulator idiom + `output_key: summary` + `state_updates: { summary: "{{summary}}\n{{output}}" }` + therefore no longer prepends the parent's value. A per-item accumulator + seeded from the parent is cross-item coupling that maps don't offer; seed + per-item input through `as:` instead. +- **Prefer a named key.** With the default `output_key: output`, an + llm/agent/rag chain step whose `state_updates` don't name `output` leaves + `output: null` behind (the transient completion binding, see + [the implicit `output` key](#the-implicit-output-key-gotcha)) and `null` + is what gets collected, with no error. `output_key: finding` + + `state_updates: { finding: "{{output}}" }` never collides with the + completion binding and fails loudly when forgotten. +- Every other fork write (`draft`, `attempts`, `feedback`, ...) is + branch-local scratch: visible to later chain steps, gone at collect. + +### Retrying inside a branch with `_next` + +A chain loops only through a script's `_next`. Together with fork-local +scratch that gives each item its own retry loop, spawn, verify, then +retry or accept, with no coupling between items: + +```yaml +review_each_file: + type: map + over: "{{files}}" + as: file + branch: review_file + output_key: verdict + collect_into: file_verdicts + max_concurrency: "{{review_budget}}" + next: aggregate + +review_file: # step 1: do the work + type: agent + agent: file-reviewer + teammates: true + prompt: "Review {{file}}. Prior feedback (if any): {{retry_feedback}}" + state_updates: + draft: "{{output}}" + next: check_review + +check_review: # step 2: verify it + type: llm + prompt: "Does this review of {{file}} meet the bar?\n\n{{draft}}" + output_schema: + type: object + properties: + pass: { type: boolean } + feedback: { type: string } + next: check_gate + +check_gate: # step 3: route (deterministic, no LLM) + type: script + script: scripts/check_gate.py +``` + +`check_gate.py` reads `pass`, `feedback`, `draft`, and a `retry_count` it +keeps in fork state, then prints either +`{"_next": "review_file", "retry_feedback": "...", "retry_count": 2}` to +loop or `{"verdict": ...}` with no `_next` to end the chain. Seed +`retry_feedback` in `initial_state` (or with an upstream node): the first +`review_file` prompt reads it, and prompts fail strict interpolation on a +missing key. + +Rules for the loop: + +- `_next` must target a node inside the branch subgraph. Routing outside it + fails the item: `... routed to 'aggregate' which is outside the branch + subgraph rooted at 'review_file' (branch nodes: check_gate, check_review, + review_file). Script `_next` targets inside a map branch must stay within + the branch.` +- Each node's per-item visit count is bounded by + `settings.max_loop_iterations` on the fork; exceeding it fails the item + and names the node and the count. Keep your own budget (`retry_count`) + under that hard cap. +- Item errors are fail-fast and carry the chain position: + `map node 'review_each_file': sub-branch [2] failed at node 'check_review' (step 2): ...`. +- A `script` step that fails consults its `fallback:` (a branch-local node) + before failing the item, exactly like a main-flow script. An `llm` step's + `max_attempts` and `fallback:` work per item the same way. ### Empty `over` @@ -800,11 +940,20 @@ an error. It's the correct semantic for "map over nothing yields nothing". If you want to reject empty `over` as an error, gate it with a script node upstream. -### Nested map nodes are not supported in v1 +### Not supported in v1 -A map branch cannot itself be another map node. The validator rejects this -at load time. If you need M * N parallel work, restructure to a single map -over a flattened list. +- **Fan-out or a nested `map` inside a branch subgraph.** The validator + rejects both at load time. If you need M * N parallel work, restructure + to a single map over a flattened list, or a second map after the join. +- **`end` nodes inside a branch.** Chains end by omission; `end` means + "the graph returns". +- **Per-item error policy** (`on_item_error`, skipping or `null`-filling a + failed item). One failing item fails the map. +- **Per-item timeouts.** `settings.timeout` applies to the whole + super-step; node-level `timeout`s (llm/agent/script) apply per step. +- **Shared state between items mid-run.** Items communicate via + [teammate messaging](#teammate-messaging-in-fan-outs); results merge at + collect. --- @@ -914,6 +1063,11 @@ Nodes route via three mechanisms in priority order: | `map` | **Yes** - `next` is where the parent super-step continues after the map collects. | Yes | | `end` | No - terminal. | n/a | +Inside a map's branch subgraph the "needs `next`" column relaxes: a chain +node with no `next` (and no script `_next`) is the chain's last step, not +an error. Fan-out is never allowed inside a branch. See +[Branch subgraph rules](#branch-subgraph-rules). + ### Failure-handling contract | Node type | Success | Failure with `fallback` | Failure without `fallback` | @@ -1007,15 +1161,26 @@ fan_out_subjects: type: map over: "{{subjects}}" # list determined at run-time as: subject - branch: research_subject + branch: research_subject # entry node of the per-item chain + output_key: finding collect_into: research_results # array, in input order next: rank ``` -The map executor spawns one sub-branch per `subjects` item, each -invoking `research_subject` with `{{subject}}` bound to the item. -Outputs are collected into `research_results` **in input order** -regardless of which sub-branch finished first. +The map executor spawns one sub-branch per `subjects` item. Each +sub-branch runs a per-item chain on its own fork of the state: it starts at +`research_subject` with `{{subject}}` bound to the item and follows that +node's `next:` / script `_next` / `fallback:` edges until a node with no +onward route ends the chain. The chain writes its result to `output_key` +(`finding`); those values are collected into `research_results` **in input +order** regardless of which sub-branch finished first, and every other +write the chain made is discarded with the fork. + +A branch node may therefore have `next:`. A single node with no `next:` is +just the shortest chain. What a branch chain may *not* do is fan out, +contain another `map`, or end in an `end` node; see +[Branch subgraph rules](#branch-subgraph-rules) and +[Retrying inside a branch](#retrying-inside-a-branch-with-_next). ## Teammate messaging in fan-outs @@ -1040,6 +1205,24 @@ Mechanics: requires at least two members. Each fan-out gets its *own* registry: a flagged map fan-out and a flagged agent node in the same super-step form two separate teams and cannot message each other. +- **In a map, the identity is the item, not the node.** A map provisions + one identity and one inbox per item when *any* agent node in its branch + subgraph has `teammates: true` (and there are at least two items). That + identity lives for the item's **whole chain**: it is handed to each + `teammates: true` agent step in the chain (a second flagged agent step + speaks as the same teammate and drains the same inbox), it is never + handed to unflagged agent steps or to `llm`/`rag`/`script` steps, and it + is retired only when the chain ends -- on success, on error, and when an + agent step times out. Messages sent to an item while it is between agent + steps (running a gate script, say) wait in its inbox until its next + flagged agent step checks it. +- **Accepted-then-unread window.** Because the identity outlives the last + agent step, a message delivered after an item's *last* `teammates: true` + agent step but before its chain ends is accepted ("Message delivered to + teammate") and never read. With a single-node branch that window is + microseconds; with a chain it is the whole tail of the chain (gate + scripts, llm steps). Design teams so cross-talk happens during the agent + steps, not after them. - **Pre-provisioned inboxes.** Every member's `(id, inbox)` pair is created before any branch starts, so a fast sibling can message one that has not started yet -- the message simply waits in its inbox. @@ -1162,9 +1345,10 @@ map's `output_key`, see [`map` nodes](#map)). Two consequences of the restore step that bite in parallel super-steps: - A node with **no `state_updates`** writes nothing at all. A map branch - that omits `state_updates` produces whatever `output` the fork inherited - (usually `null`), not the completion. Always declare what the branch - writes. + chain that never declares a write to the map's `output_key` therefore + fails the map at run time (the key is removed from each item's fork + before the chain starts, so nothing is inherited), and the validator + warns about it at load time. Always declare what the branch writes. - When `output` was **absent** from state before the super-step, every LLM/agent/rag branch that has `state_updates` leaves `output: null` behind (the restore falls back to `null`). Two or more such branches @@ -1326,7 +1510,7 @@ for both branches to feed into a shared end node). ## Approval and input nodes are forbidden in fan-out The validator rejects any approval or input node that's an immediate -fan-out target (`next: [approve, ...]` or as a map's `branch:`). The +fan-out target (`next: [approve, ...]` or anywhere in a map's branch subgraph). The reason is UX: ten concurrent "type your answer" prompts would fundamentally not work in a CLI. Put approval / input nodes **after** the join, downstream of the merge. @@ -1531,11 +1715,14 @@ startup. - **Approval or input as a fan-out target**: the immediate target of a `next: [...]` is an `approval` or `input` node. These would queue N concurrent user prompts, which is a UX disaster. So they're rejected at load time -- **Map branch violates strict mode**: the validator enforces that a - `map` node's `branch:` target (1) is `llm`, `agent`, `rag`, or `script` - (not approval/input/end/another map); (2) has no `next:` declared; - (3) writes only to its `output_key` via `state_updates` (if any); (4) - has no `output_schema` declared +- **Map branch subgraph violation**: the validator checks every node + reachable from a `map` node's `branch:` entry through `next:`/`fallback:` + edges: (1) all must be `llm`, `agent`, `rag`, or `script` (no + approval/input/end/another map -- `fallback: end_*` on a branch node is + the usual culprit); (2) none may fan out with `next: [...]`; (3) none may + also be reachable from `start` on the main flow. A missing declared writer + for the map's `output_key` is a warning, not an error. See + [Branch subgraph rules](#branch-subgraph-rules) - **Script in a parallel branch with no `state_updates`**: a script node whose immediate predecessor is a fan-out has unknown writes from the validator's perspective (scripts can emit arbitrary JSON to state).