testing

docs/QA-CHECKLIST.md

@@ -0,0 +1,232 @@
+# Loki QA Checklist
+
+Behavioral verification checklist for the REST API refactor.
+Run after each step or phase to confirm existing functionality
+is preserved.
+
+## How to use
+
+- [ ] = not yet verified for current step
+- [x] = verified working
+- SKIP = not applicable to current step
+
+Check each item manually in the REPL and/or CLI. If a check
+fails, stop and investigate before proceeding.
+
+---
+
+## 1. Build & Test Baseline
+
+- [ ] `cargo check` — zero warnings, zero errors
+- [ ] `cargo clippy` — zero warnings
+- [ ] `cargo test` — all tests pass (63 as of Step 8g)
+
+## 2. CLI — Basic Operations
+
+- [ ] `loki "hello"` — single-shot chat works, response printed
+- [ ] `loki --role <name> "hello"` — role applied, response uses role context
+- [ ] `loki --session <name> "hello"` — session created/resumed, response saved
+- [ ] `loki --model <model_id> "hello"` — specified model used
+- [ ] `loki --prompt "you are a pirate" "hello"` — temp role applied
+- [ ] `loki --info` — system info printed, exits cleanly
+- [ ] `loki --list-models` — model list printed
+- [ ] `loki --list-roles` — role list printed (no hidden files)
+- [ ] `loki --list-sessions` — session list printed
+- [ ] `loki --list-agents` — agent list printed (no `.shared` directory)
+- [ ] `loki --dry-run "hello"` — no API call, input echoed
+- [ ] `loki --no-stream "hello"` — non-streaming response
+
+## 3. CLI — File Input
+
+- [ ] `loki --file /tmp/test.txt "summarize"` — file content included
+- [ ] `loki --file /tmp/test.txt` — file content sent without extra text
+
+## 4. CLI — Agent (non-interactive)
+
+- [ ] `loki --agent <name> "do something"` — agent starts, tools available, response returned
+- [ ] Agent MCP servers start (if configured)
+- [ ] Agent tool calls execute correctly (e.g., execute_command)
+
+## 5. CLI — Shell Execute
+
+- [ ] `loki -e "list files in /tmp"` — shell command generated
+- [ ] Shell command explanation shown (describe mode)
+- [ ] Shell command execution works when confirmed
+
+## 6. CLI — Macro
+
+- [ ] `loki --macro <name> "input"` — macro executes
+
+## 7. REPL — Startup & Exit
+
+- [ ] `loki` — REPL starts, welcome message shown
+- [ ] `.exit` — REPL exits cleanly
+- [ ] Ctrl+D — REPL exits cleanly
+- [ ] Ctrl+C — prints exit hint, does not exit
+
+## 8. REPL — Chat
+
+- [ ] Type a message — response printed
+- [ ] `.continue` — continues previous response
+- [ ] `.regenerate` — regenerates last response
+- [ ] `.copy` — copies last response to clipboard
+
+## 9. REPL — Roles
+
+- [ ] `.role <name>` — switches to role, prompt changes
+- [ ] `.role <name> <text>` — one-shot role message
+- [ ] `.info role` — shows role info
+- [ ] `.edit role` — opens editor for current role
+- [ ] `.save role <name>` — saves current role
+- [ ] `.exit role` — exits role, prompt resets
+- [ ] Role with MCP servers — servers start on `.role <name>`
+- [ ] Role with MCP servers — MCP tools available in chat
+- [ ] `.exit role` with MCP — servers stop, MCP tools removed
+
+## 10. REPL — Sessions
+
+- [ ] `.session` — starts temp session
+- [ ] `.session <name>` — starts/resumes named session
+- [ ] `.info session` — shows session info
+- [ ] `.edit session` — opens editor
+- [ ] `.save session <name>` — saves session
+- [ ] `.empty session` — clears messages
+- [ ] `.compress session` — compresses session
+- [ ] `.exit session` — exits session
+- [ ] Session with MCP servers — servers start
+- [ ] Session carry-over prompt — "incorporate last Q&A?" appears when applicable
+
+## 11. REPL — Agents
+
+- [ ] `.agent <name>` — agent starts, tools compiled, prompt changes
+- [ ] `.agent <name> <session>` — agent starts with specific session
+- [ ] `.agent <name> key=value` — agent starts with variables
+- [ ] `.info agent` — shows agent info
+- [ ] `.starter` — shows conversation starters
+- [ ] `.starter <n>` — executes starter
+- [ ] `.edit agent-config` — opens agent config editor
+- [ ] `.exit agent` — exits agent cleanly
+- [ ] Agent with MCP servers — servers start
+- [ ] Agent tool calls work (execute_command, fs_read, etc.)
+- [ ] Agent global tools work (tools listed in `global_tools`)
+- [ ] Agent tool file changes picked up on restart (delete .ts, .sh used instead)
+- [ ] Auto-continuation works (todo list drives continuation)
+- [ ] `.clear todo` — clears todo list
+
+## 12. REPL — Sub-Agent Escalation
+
+- [ ] Parent agent spawns sub-agent via tool call
+- [ ] Sub-agent runs at depth > 0
+- [ ] Sub-agent escalation: sub-agent calls user__ask → parent gets notification
+- [ ] Parent calls agent__reply_escalation → sub-agent unblocked, resumes
+- [ ] Multiple pending escalations shown in notification
+- [ ] Max depth enforcement — sub-agent spawn rejected beyond max_agent_depth
+
+## 13. REPL — RAG
+
+- [ ] `.rag <name>` — initializes/loads RAG
+- [ ] `.info rag` — shows RAG info
+- [ ] `.sources rag` — shows citation sources
+- [ ] `.edit rag-docs` — modify RAG documents
+- [ ] `.rebuild rag` — rebuilds RAG index
+- [ ] `.exit rag` — exits RAG
+- [ ] RAG embeddings used in chat (search results included)
+
+## 14. REPL — MCP Servers
+
+- [ ] MCP servers start at REPL init (if globally enabled)
+- [ ] `.set enabled_mcp_servers <name>` — changes active servers
+- [ ] `.set mcp_server_support true/false` — toggles support
+- [ ] MCP tool invocation works (mcp__invoke_<server>)
+- [ ] MCP tool search works (mcp__search_<server>)
+- [ ] MCP tool describe works (mcp__describe_<server>)
+
+## 15. REPL — Settings
+
+- [ ] `.set temperature 0.5` — changes temperature
+- [ ] `.set top_p 0.9` — changes top_p
+- [ ] `.set model <name>` — changes model
+- [ ] `.set dry_run true` — enables dry run
+- [ ] `.set stream false` — disables streaming
+- [ ] `.set save true/false` — toggles save
+- [ ] `.set highlight true/false` — toggles highlighting
+- [ ] `.set save_session true/false/null` — changes session save behavior
+- [ ] `.set compression_threshold <n>` — changes threshold
+
+## 16. REPL — Tab Completion
+
+- [ ] `.role<TAB>` — shows role names (no hidden files)
+- [ ] `.agent<TAB>` — shows agent names (no `.shared` directory)
+- [ ] `.session<TAB>` — shows session names
+- [ ] `.rag<TAB>` — shows RAG names
+- [ ] `.macro<TAB>` — shows macro names
+- [ ] `.model<TAB>` — shows model names with descriptions
+- [ ] `.set <TAB>` — shows setting names
+- [ ] `.set temperature <TAB>` — shows current value
+- [ ] `.set enabled_tools <TAB>` — shows tool names
+- [ ] `.set enabled_mcp_servers <TAB>` — shows server names
+
+## 17. REPL — Delete
+
+- [ ] `.delete role <name>` — deletes role
+- [ ] `.delete session <name>` — deletes session
+- [ ] `.delete rag <name>` — deletes RAG
+- [ ] `.delete macro <name>` — deletes macro
+- [ ] `.delete agent-data <name>` — deletes agent data
+
+## 18. REPL — Vault
+
+- [ ] `.vault list` — lists secrets
+- [ ] `.vault add <name>` — adds secret
+- [ ] `.vault get <name>` — retrieves secret
+- [ ] `.vault update <name>` — updates secret
+- [ ] `.vault delete <name>` — deletes secret
+
+## 19. REPL — Prelude
+
+- [ ] `repl_prelude: "role:coder"` — auto-loads role on REPL start
+- [ ] `repl_prelude: "session:mysession"` — auto-loads session
+- [ ] `repl_prelude: "mysession:coder"` — auto-loads session with role
+
+## 20. REPL — Miscellaneous
+
+- [ ] `.help` — shows help text
+- [ ] `.info` — shows system info
+- [ ] `.authenticate` — OAuth flow (if configured)
+- [ ] `.file <path>` — includes file in next message
+- [ ] `.file <url>` — fetches URL content
+- [ ] Unknown command — shows error message
+- [ ] Multi-line input (:::) — works correctly
+- [ ] Ctrl+O — opens editor for input buffer
+
+## 21. Session Compression & Autoname
+
+- [ ] Session auto-compression triggers when threshold exceeded
+- [ ] Compression message shown ("Compressing the session.")
+- [ ] Session auto-naming triggers for new sessions
+- [ ] Auto-continuation after compression works (agent resumes)
+
+## 22. Error Handling
+
+- [ ] Invalid role name — error shown, REPL continues
+- [ ] Invalid model name — error shown, REPL continues
+- [ ] Network error during chat — error shown, REPL continues
+- [ ] MCP server crash — error shown, REPL continues
+- [ ] Tool execution failure — error returned to LLM as tool result
+
+---
+
+## Phase-specific notes
+
+### Phase 1 (Steps 3-10): Config split into AppState + RequestContext
+
+Known bridge-window limitations (acceptable until Steps 9-10):
+- `ReplCompleter`/`ReplPrompt` still hold `GlobalConfig`
+- `Input` still holds `GlobalConfig` internally
+- `eval_tool_calls` still takes `&GlobalConfig`
+- Dual sync (`sync_ctx_to_config`/`sync_config_to_ctx`) required
+
+### Post-Phase 1 verification focus:
+- All items above should work identically to pre-refactor behavior
+- No new warnings or errors in build
+- Performance should be equivalent (no observable slowdown)