Commit Graph
10 Commits
Author SHA1 Message Date
Dark-Alex-17 00c3b60c2f feat(graph): stream llm-node completions so node timeout governs long generations
Graph llm nodes (and the structured-output extractor they share with agent
nodes) now obtain the model reply through a quiet streaming transport,
`call_chat_completions_streaming_quiet`, instead of the non-streaming call.
Because chunks arrive as they are generated, reqwest's client-level
`read_timeout` (default 300s) bounds only a genuine stall between chunks and
the node's own `timeout` becomes the effective bound on total generation.
Previously zero bytes arrived until the whole reply finished, so a 300s
read_timeout silently capped every llm node regardless of its `timeout`
(seen live: code-reviewer's `synthesize` node dying on a 61-file review).

The quiet transport renders nothing (silent SseHandler, no spinner), keeps
tool-call handling identical to the non-streaming path (call-loop detection
off), falls back to the non-streaming request for catalog `no_stream` models
(raced against the abort), and never returns partial output as success.

Supporting changes:
- `sse_stream`/`json_stream` keep the typed `reqwest::Error` in the anyhow
  chain so `is_transient_error` still classifies read_timeout stalls as
  transient and `max_attempts` retries them (previously stringified).
- `structured::extract` takes the graph/session AbortSignal and refuses to
  issue an extraction request for an already-aborted graph.
- `run_chat_loop` gains an injectable CompletionRunner seam (owning client
  construction + the before-hook) so the timeout/retry/abort behaviour is
  unit-testable over a paused clock; a fake `impl Client` drives the real
  trait-default streaming for 600s of virtual time; real-socket tests pin
  that read_timeout fires on a stall but not on a slow continuous stream.
2026-09-14 10:50:34 -06:00
Dark-Alex-17 a8fb32b6bd fix: strip reasoning blocks for structured LLM output in graph agents 2026-08-05 16:10:03 -06:00
Dark-Alex-17 320dbf2479 fix: Correctly inherit graph-global model for extractor model if none is defined 2026-07-17 11:42:28 -06:00
Dark-Alex-17 73a4499c68 fix: don't silently fail on skill role composition extraction in llm nodes 2026-06-04 09:09:55 -06:00
Dark-Alex-17 260bf4e5bc fmt: applied formatting to refactored mcp_servers and tools lists 2026-06-03 14:02:06 -06:00
Dark-Alex-17 ece66448e0 refactor: support both CSV and list formats for enabled_tools 2026-06-03 13:58:24 -06:00
Dark-Alex-17 a254d60876 refactor: Support both CSV and list formats for enabled_mcp_servers 2026-06-03 13:23:13 -06:00
Dark-Alex-17 81c037515e feat: improved UX for parallel graph execution 2026-05-20 18:54:20 -06:00
Dark-Alex-17 5bd0766a60 style: Cleaned up all graph agent code 2026-05-18 13:46:52 -06:00
Dark-Alex-17 48c52b5829 feat: added structured-output extraction for llm and agent nodes 2026-05-14 15:36:10 -06:00