feat: Deep research upgrade so sisyphus and the architect now have 3 possible research types: librarian standard, librarian deep, and true deep research with reflexion

assets/agents/deep-research/README.md

@@ -13,6 +13,13 @@ this agent runs a fixed graph: every request goes through the same
 `plan -> parallel research -> vet -> critique -> synthesize -> verify -> approve`
 pipeline.
 
+Two variables tune a run (pass via `--variables`): `reflexion_budget`
+(default 2) caps the automated critique -> re-research passes — raise it
+to let the run churn until the critique passes — and
+`autonomy: autonomous` skips the final human approval gate so other
+agents can spawn this one autonomously. Each sub-question is researched
+by a spawned `librarian` running at deep budgets.
+
 This agent is also the **canonical reference for the Coyote graph
 system**: it exercises every node type (`script`, `llm`, `rag`, `map`,
 `agent`, `input`, `approval`, `end`) and both static fan-out and

assets/agents/deep-research/graph.yaml

@@ -7,7 +7,12 @@ description: |
   delegates the final write-up to a focused sub-agent, checks that the
   cited sources are reachable, and gates the result behind human
   approval. A reviewer's free-form feedback at the approval step feeds
-  back into another research pass.
+  back into another research pass. Each sub-question is researched by a
+  spawned librarian running at deep budgets. Tunable via --variables:
+  reflexion_budget (default 2) caps the automated critique->re-research
+  passes - raise it to let the run churn until the critique passes;
+  autonomy=autonomous skips the human approval gate (REQUIRED when
+  another agent spawns this one).
 
   This is the canonical Coyote graph-agent reference: it exercises every
   node type (script, llm, rag, map, agent, input, approval, end) and
@@ -27,8 +32,16 @@ conversation_starters:
   - 'How does HTTP/3 differ from HTTP/2?'
   - 'Summarize recent advances in solid-state battery chemistry'
 
+variables:
+  - name: reflexion_budget
+    description: "Max automated critique->re-research passes before synthesis proceeds anyway. Default 2; raise it (4-8) for exhaustive runs that should churn until the critique passes."
+    default: '2'
+  - name: autonomy
+    description: "supervised (default): the finished report stops at the human approval gate. autonomous: skip the gate and return the report directly - REQUIRED when another agent spawns this one (a human gate would block the run)."
+    default: 'supervised'
+
 settings:
-  max_loop_iterations: 40
+  max_loop_iterations: 120
   log_state_snapshots: false
   validate_before_run: true
   max_concurrency: 4
@@ -142,23 +155,11 @@ nodes:
 
   research_one_question:
     id: research_one_question
-    type: llm
-    instructions: |
-      You are a web research assistant. Investigate the SINGLE question
-      given to you using your tools: search the web, fetch and read
-      pages, and search arXiv for academic sources.
-
-      Rules:
-        - Every factual claim must be backed by a real source you
-          actually retrieved. Never fabricate URLs, page titles,
-          authors, or DOIs.
-        - Prefer primary and authoritative sources over aggregators.
-        - Where sources disagree, report the disagreement rather than
-          papering over it.
-        - Put the URL (or DOI) inline next to each claim it supports.
-
-      Return organized findings in plain text. Do not include
-      meta-commentary about the process.
+    type: agent
+    agent: librarian
+    # The leaf research primitive: triage -> doc+OSS search -> cited
+    # synthesis. `inputs:` overlays the librarian graph's initial state,
+    # pinning research_depth to deep for every sub-question lane.
     prompt: |
       Research question: {{question}}
 
@@ -166,16 +167,13 @@ nodes:
       {{local_context}}
 
       {{research_feedback}}
-    tools:
-      - web_search_coyote
-      - fetch_url_via_curl
-      - search_arxiv
-      - mcp:ddg-search
-    max_iterations: 10
+    inputs:
+      research_depth: deep
+    timeout: 1900
     max_attempts: 2
     state_updates:
       finding: '{{output}}'
-    # On failure the engine writes "LLM node ...failed: <chain>" into `finding`
+    # On failure the engine writes "Agent node failed: <chain>" into `finding`
     # via the state_updates above BEFORE routing to the fallback. The marker
     # is branch-local: a map branch's fallback must stay inside the branch.
     fallback: question_fault
@@ -312,6 +310,13 @@ nodes:
     id: verify_sources
     type: script
     script: scripts/verify_sources.py
+    next: approval_gate
+
+  approval_gate:
+    id: approval_gate
+    type: script
+    description: Autonomy router - autonomous runs (spawned by another agent) skip the human approval gate and end accepted with the source check + pipeline notes attached; supervised (default) proceeds to the approval node. A crash fails forward via `next` to approve, the safe human-gated direction.
+    script: scripts/approval_gate.py
     next: approve
 
   approve:
@@ -348,6 +353,15 @@ nodes:
     type: end
     output: '{{report}}{{pipeline_notes}}'
 
+  end_autonomous:
+    id: end_autonomous
+    type: end
+    output: |
+      {{report}}
+
+      ----
+      {{source_check}}{{pipeline_notes}}
+
   end_rejected:
     id: end_rejected
     type: end

assets/agents/deep-research/scripts/approval_gate.py

@@ -0,0 +1,38 @@
+#!/usr/bin/env python3
+"""Autonomy router for deep-research's approval gate.
+
+Runs after verify_sources. When the `autonomy` variable is
+'autonomous' (the run was spawned by another agent — architect or
+sisyphus research tiers — with no human at the terminal), the report
+skips the human approval node and ends accepted: the reflexion loop
+already served as the quality gate, and end_autonomous carries the
+source check + pipeline notes with the report so the caller sees them.
+Any other value (default 'supervised') proceeds to the human approval
+gate unchanged.
+
+Routing (`_next`):
+  - autonomy == 'autonomous' -> end_autonomous
+  - anything else            -> approve
+
+Fail-safe: a crash here fails FORWARD via the node's `next` to
+`approve` — the safe, human-gated direction.
+"""
+import json
+import os
+
+
+def load_state():
+    if path := os.environ.get("GRAPH_STATE_FILE"):
+        with open(path) as f:
+            return json.load(f)
+    return json.loads(os.environ.get("GRAPH_STATE", "{}"))
+
+
+def main():
+    state = load_state()
+    autonomy = str(state.get("autonomy") or "supervised").strip().lower()
+    target = "end_autonomous" if autonomy == "autonomous" else "approve"
+    print(json.dumps({"_next": target}))
+
+
+main()

assets/agents/deep-research/scripts/question_fault.py

@@ -4,10 +4,11 @@
 Reached ONLY via research_one_question's `fallback:` route inside the
 `research_each_question` map branch (branch-local — a map branch's fallback
 must stay inside the branch subgraph). Grounded in engine behavior: on
-llm-node failure the engine writes "LLM node …failed: <chain>" into
-`finding` via the node's state_updates BEFORE routing to the fallback (on
-success the same key holds the researched findings, which never start with
-the "LLM node" prefix). This node normalizes the failure text into a
+branch failure the engine writes its failure text into `finding` via the
+node's state_updates BEFORE routing to the fallback — "Agent node failed:
+<chain>" now that the branch spawns a librarian agent, "LLM node …failed:
+<chain>" for an llm branch (on success the same key holds the researched
+findings, which never start with either prefix). This node normalizes the failure text into a
 PIPELINE-FAULT finding and the chain ends here — the map collects the fault
 as the lane's `finding`, so one dead research lane never sinks the whole
 map. combine_findings later lifts PIPELINE-FAULT findings into
@@ -53,7 +54,7 @@ def main():
     state = load_state()
     raw = state.get("finding")
     detail = raw.strip().replace("\n", " ") if isinstance(raw, str) else ""
-    if not detail.startswith("LLM node"):
+    if not detail.startswith(("LLM node", "Agent node")):
         detail = "the research lane died without recording a failure detail"
     if len(detail) > MAX_DETAIL_CHARS:
         detail = detail[: MAX_DETAIL_CHARS - 1] + "…"

assets/agents/deep-research/scripts/reflexion_gate.py

@@ -2,7 +2,8 @@
 """Reflexion gate for deep-research.
 
 Runs after `critique` has reviewed the current research findings. If the
-critique's verdict is REVISE and the reflexion budget is not spent,
+critique's verdict is REVISE and the reflexion budget (the graph's
+`reflexion_budget` variable, default 2) is not spent,
 loops back to `research` with the critique attached as
 `research_feedback`, so the retry is informed rather than a blind
 re-run. Otherwise it proceeds to `synthesize`.
@@ -26,9 +27,11 @@ import json
 import os
 import re
 
-# Automated revision passes allowed. `research` runs at most
-# MAX_REFLEXION_REVISIONS + 1 times per user pass. Bump to allow more.
-MAX_REFLEXION_REVISIONS = 2
+# Default automated revision passes when the `reflexion_budget` variable
+# is absent or malformed. `research` runs at most budget + 1 times per
+# user pass. Callers raise it via --variables {"reflexion_budget": "6"}
+# to let the run churn until the critique passes.
+DEFAULT_REFLEXION_REVISIONS = 2
 
 
 def load_state():
@@ -69,8 +72,9 @@ def main():
     critique = state.get("critique") or ""
     verdict = parse_verdict(critique)
     attempts = as_int(state.get("research_attempts"))
+    budget = as_int(state.get("reflexion_budget"), DEFAULT_REFLEXION_REVISIONS)
 
-    if verdict == "REVISE" and attempts < MAX_REFLEXION_REVISIONS:
+    if verdict == "REVISE" and attempts < budget:
         feedback = (
             "A reviewer judged the previous research pass incomplete. "
             "Address every point in the critique below:\n\n" + critique