test(suite): replace plan-label references with self-contained wording

Rust test comments/assert messages and deep-research script docstrings
cited numbered requirement labels (R3/R5) and task labels (TASK-016,
TASK-021) that are defined only in an external plan document. Reword
each to say what it means on its own: fail-safe crash guards, the
validator's fallback-capture warning, task-queue anchors.
This commit is contained in:
Dark-Alex-17 committed 2026-09-14 10:50:34 -06:00
1 parent 48753eaa46
commit 081fa2c9d4
8 files changed
+24 -23

No files matched your search

+1 -1
View File
@@ -263,7 +263,7 @@ pass — every fault is visible to the reviewer or the caller — and some
end node always renders its sentinel, so the graph degrades instead of end node always renders its sentinel, so the graph degrades instead of
dying. dying.
All 11 scripts also carry a top-level crash guard (R3): a script that All 11 scripts also carry a top-level crash guard: a script that
throws records a fault and fails forward where the flow allows it (a throws records a fault and fails forward where the flow allows it (a
crashed `reflexion_gate` proceeds to `synthesize`; a crashed crashed `reflexion_gate` proceeds to `synthesize`; a crashed
`verify_sources` reports that sources were NOT checked) instead of `verify_sources` reports that sources were NOT checked) instead of
@@ -7,7 +7,7 @@ the RAG knowledge lookup) as a single super-step. The validator requires
declared parallel-branch script outputs, so we emit an empty JSON object declared parallel-branch script outputs, so we emit an empty JSON object
explicitly here. explicitly here.
Fail-safe (R3): the node reads no state and computes nothing, so its Fail-safe: the node reads no state and computes nothing, so its
happy-path output IS the sane degraded output — the guard emits the same happy-path output IS the sane degraded output — the guard emits the same
empty object on any crash. No PIPELINE-FAULT is recorded: appending one empty object on any crash. No PIPELINE-FAULT is recorded: appending one
would require reading state (the very thing that crashed) or clobbering would require reading state (the very thing that crashed) or clobbering
@@ -14,7 +14,7 @@ surface their collected output_key). This script lifts such findings into
`pipeline_faults` (deduplicated, since a reflexion or feedback loop can `pipeline_faults` (deduplicated, since a reflexion or feedback loop can
re-run the map) so dead lanes surface in the report's Pipeline notes. re-run the map) so dead lanes surface in the report's Pipeline notes.
Fail-safe (R3): on a crash (e.g. malformed GRAPH_STATE), `findings` Fail-safe: on a crash (e.g. malformed GRAPH_STATE), `findings`
becomes the fault text itself and a PIPELINE-FAULT entry is recorded — becomes the fault text itself and a PIPELINE-FAULT entry is recorded —
the pipeline proceeds visibly degraded, and the human approval gate is the pipeline proceeds visibly degraded, and the human approval gate is
the backstop. Existing faults are preserved best-effort. the backstop. Existing faults are preserved best-effort.
@@ -11,7 +11,7 @@ budget.
Routing (`_next`): always research_each_question. Routing (`_next`): always research_each_question.
Fail-safe (R3): a crash here means the user's feedback text cannot be Fail-safe: a crash here means the user's feedback text cannot be
recovered, but their INTENT — another research pass — is unambiguous recovered, but their INTENT — another research pass — is unambiguous
(they chose neither accept nor reject). The degraded route therefore (they chose neither accept nor reject). The degraded route therefore
still loops to `research_each_question`, with a generic feedback note still loops to `research_each_question`, with a generic feedback note
@@ -10,7 +10,7 @@ Routing (`_next`):
- prompt present -> (no _next; static next: plan) - prompt present -> (no _next; static next: plan)
- prompt empty -> ask_topic - prompt empty -> ask_topic
Fail-safe (R3): a crash here (e.g. malformed GRAPH_STATE) means the Fail-safe: a crash here (e.g. malformed GRAPH_STATE) means the
caller's prompt cannot be read at all, so proceeding silently is caller's prompt cannot be read at all, so proceeding silently is
meaningless — the degraded route is `ask_topic`, which asks the user for meaningless — the degraded route is `ask_topic`, which asks the user for
the topic directly, plus a PIPELINE-FAULT entry so the loss is visible in the topic directly, plus a PIPELINE-FAULT entry so the loss is visible in
@@ -16,7 +16,7 @@ Reflexion is a best-effort quality booster, not a hard gate: once the
budget is spent the workflow proceeds anyway, and the human approval budget is spent the workflow proceeds anyway, and the human approval
step is the final backstop. step is the final backstop.
Fail-safe (R3): a crash here fails FORWARD to `synthesize` — the same Fail-safe: a crash here fails FORWARD to `synthesize` — the same
direction as the existing malformed-critique PASS default — with a direction as the existing malformed-critique PASS default — with a
PIPELINE-FAULT entry recorded (best-effort preserving existing faults), PIPELINE-FAULT entry recorded (best-effort preserving existing faults),
so a broken gate costs the run its automated critique routing, never the so a broken gate costs the run its automated critique routing, never the
@@ -15,7 +15,7 @@ rendered section when `pipeline_faults` is non-empty and the empty
string otherwise, keeping the happy-path `{{report}}{{pipeline_notes}}` string otherwise, keeping the happy-path `{{report}}{{pipeline_notes}}`
output byte-identical to the pre-hardening `{{report}}`. output byte-identical to the pre-hardening `{{report}}`.
Fail-safe (R3): per-URL probes were already guarded; the top level now Fail-safe: per-URL probes were already guarded; the top level now
is too. On a crash the summary says sources were NOT checked, a is too. On a crash the summary says sources were NOT checked, a
PIPELINE-FAULT is recorded (best-effort preserving existing faults), and PIPELINE-FAULT is recorded (best-effort preserving existing faults), and
pipeline_notes still renders — the approval gate remains the backstop. pipeline_notes still renders — the approval gate remains the backstop.
+17 -16
View File
@@ -8181,7 +8181,7 @@ mod tests {
] { ] {
assert!( assert!(
architect.contains(anchor), architect.contains(anchor),
"architect config lost TASK-021 anchor: {anchor:?}" "architect config lost task-queue mirroring anchor: {anchor:?}"
); );
} }
@@ -8199,7 +8199,7 @@ mod tests {
] { ] {
assert!( assert!(
sisyphus.contains(anchor), sisyphus.contains(anchor),
"sisyphus config lost TASK-021 anchor: {anchor:?}" "sisyphus config lost task-queue chain anchor: {anchor:?}"
); );
} }
@@ -8214,7 +8214,7 @@ mod tests {
] { ] {
assert!( assert!(
spawn.contains(anchor), spawn.contains(anchor),
"DEFAULT_SPAWN_INSTRUCTIONS lost TASK-021 anchor: {anchor:?}" "DEFAULT_SPAWN_INSTRUCTIONS lost task-queue semantics anchor: {anchor:?}"
); );
} }
assert!( assert!(
@@ -8492,7 +8492,7 @@ mod tests {
); );
} }
// ---- adversary suite-script regression tests (R5) ---- // ---- adversary suite-script regression tests ----
// //
// Fault paths of the adversary's verdict/gate scripts, exercised by // Fault paths of the adversary's verdict/gate scripts, exercised by
// invoking `python3 <script>` with a synthetic GRAPH_STATE env — the same // invoking `python3 <script>` with a synthetic GRAPH_STATE env — the same
@@ -9427,7 +9427,7 @@ mod tests {
assert_eq!(holistic.max_iterations, 20); assert_eq!(holistic.max_iterations, 20);
} }
// ---- review-gauntlet suite-script regression tests (R5) ---- // ---- review-gauntlet suite-script regression tests ----
// //
// Degradation paths of the gauntlet's lane/builder/gate scripts, // Degradation paths of the gauntlet's lane/builder/gate scripts,
// exercised the same way as the adversary suite above: `python3 <script>` // exercised the same way as the adversary suite above: `python3 <script>`
@@ -9914,7 +9914,7 @@ mod tests {
); );
} }
// ---- code-reviewer suite-script regression tests (R5) ---- // ---- code-reviewer suite-script regression tests ----
// //
// Fail-closed fault paths of the code-reviewer's verdict/fault/render // Fail-closed fault paths of the code-reviewer's verdict/fault/render
// scripts, exercised the same way as the suites above: `python3 <script>` // scripts, exercised the same way as the suites above: `python3 <script>`
@@ -10503,7 +10503,7 @@ mod tests {
); );
} }
// ---- finding-verifier suite-script regression tests (R5) ---- // ---- finding-verifier suite-script regression tests ----
// //
// Fail-closed fault paths of the finding-verifier's marker scripts, // Fail-closed fault paths of the finding-verifier's marker scripts,
// exercised the same way as the suites above: `python3 <script>` with a // exercised the same way as the suites above: `python3 <script>` with a
@@ -10738,7 +10738,7 @@ mod tests {
); );
} }
// ---- step-runner fault-wiring regression tests (R5) ---- // ---- step-runner fault-wiring regression tests ----
// //
// route_review.sh is a bash script node; the graph's script executor runs // route_review.sh is a bash script node; the graph's script executor runs
// `.sh` scripts through bash with the same GRAPH_STATE env contract, so // `.sh` scripts through bash with the same GRAPH_STATE env contract, so
@@ -10899,7 +10899,7 @@ mod tests {
if skip_step_runner_bash_harness() { if skip_step_runner_bash_harness() {
return; return;
} }
// R3: a fault-noting script must never itself kill the pipeline — // A fault-noting script must never itself kill the pipeline —
// the harness asserts exit 0 + JSON stdout even for a bare state. // the harness asserts exit 0 + JSON stdout even for a bare state.
let out = run_step_runner_script("note_llm_fault.sh", &json!({})); let out = run_step_runner_script("note_llm_fault.sh", &json!({}));
assert_eq!(out, json!({"fault_note": ""})); assert_eq!(out, json!({"fault_note": ""}));
@@ -10949,8 +10949,9 @@ mod tests {
assert_eq!(review.fallback.as_deref(), Some("write_handoff")); assert_eq!(review.fallback.as_deref(), Some("write_handoff"));
// The llm nodes with fallbacks capture their failure text so the // The llm nodes with fallbacks capture their failure text so the
// fallback path can see why it was reached (the TASK-016 warning — // fallback path can see why it was reached (the validator's
// this is what shrank the step-runner warning baseline to zero). // fallback-capture warning — this is what shrank the step-runner
// warning baseline to zero).
// orient and write_handoff route their fallbacks through // orient and write_handoff route their fallbacks through
// note_llm_fault, which distills the capture into fault_note; // note_llm_fault, which distills the capture into fault_note;
// edge_case_sweep still falls back to write_handoff, which renders // edge_case_sweep still falls back to write_handoff, which renders
@@ -11037,9 +11038,9 @@ mod tests {
} }
} }
// ---- deep-research suite-script regression tests (R5) ---- // ---- deep-research suite-script regression tests ----
// //
// The deep-research graph's R3 guards and fault markers, exercised by // The deep-research graph's crash guards and fault markers, exercised by
// invoking `python3 <script>` with a synthetic GRAPH_STATE env. The raw // invoking `python3 <script>` with a synthetic GRAPH_STATE env. The raw
// runner feeds deliberately malformed JSON to prove every guarded script // runner feeds deliberately malformed JSON to prove every guarded script
// emits a sane degraded output instead of crashing the node. // emits a sane degraded output instead of crashing the node.
@@ -11077,7 +11078,7 @@ mod tests {
eprintln!("skipping: python3 not available"); eprintln!("skipping: python3 not available");
return; return;
} }
// R3: every guarded script and fault marker must exit 0 with JSON // Fail-safe: every guarded script and fault marker must exit 0 with JSON
// stdout even when GRAPH_STATE is not JSON at all (the runner // stdout even when GRAPH_STATE is not JSON at all (the runner
// asserts both); per-script degraded shapes are pinned below. // asserts both); per-script degraded shapes are pinned below.
for script in [ for script in [
@@ -11516,8 +11517,8 @@ mod tests {
.unwrap(); .unwrap();
// plan: fail-closed. Its output_schema + fallback combination requires // plan: fail-closed. Its output_schema + fallback combination requires
// a state_updates capture (the TASK-016 warning) so plan_fault can see // a state_updates capture (the validator's fallback-capture warning) so
// why it was reached. // plan_fault can see why it was reached.
let NodeType::Llm(plan) = &graph.get_node("plan").unwrap().node_type else { let NodeType::Llm(plan) = &graph.get_node("plan").unwrap().node_type else {
panic!("plan must be an llm node") panic!("plan must be an llm node")
}; };