refactor(rag): extract RagProvider trait and add YamlProvider

Introduce a narrow `RagProvider` trait covering vector search and content
retrieval, and make `Rag` delegate to a boxed provider instead of owning an
HNSW index directly. `YamlProvider` is the sole implementation for now.

The trait deliberately stays narrow: embeddings, chunking, BM25 keyword
search, graph RAG, entity extraction, RRF merging and persistence all remain
on `Rag`/`RagData`, so a new storage backend does not have to reimplement
Coyote's indexing logic.

Notable points:

- `fetch_content`'s ordering contract is part of the trait, not an accident.
  Implementations must return results in input-`ids` order; `hybrid_search`
  passes an RRF-ranked list straight to the prompt builder, so a provider
  returning storage order would silently discard the ranking.
- The content store is keyed on `data.files`, never `data.vectors`. Both the
  content map and BM25 now route through the new `RagData::iter_documents()`
  so the two key spaces match by construction. `RagData::add` zips document
  ids with embeddings and truncates silently, so ids in `files \ vectors` are
  genuinely reachable.
- A provider keyword-search failure degrades to an empty ranker with a
  warning rather than failing the whole query; it is one of three RRF inputs.
  It deliberately does not fall back to the local BM25, which would be a
  silent ranking-algorithm swap once a provider with native FTS exists.
- The rerank path builds its text and id vectors from a single `fetch_content`
  result in one pass, so the reranker's positional indices cannot desync.

This is not a bit-for-bit no-op. `vector_search` now dedups by best score and
sorts globally instead of concatenating per-chunk hit lists. Single-chunk
queries (the common case) are unaffected. Multi-chunk queries get corrected
rank assignment and no longer let a document that matched several query
chunks accumulate multiple RRF contributions. There is no overall cap on the
merged pool — truncation remains `reciprocal_rank_fusion`'s job.

`RagData::get()` is removed: its only two callers were the content lookups
replaced here, and an unused private-module method fails the build under
`--deny warnings`. Its three tests were rewritten against `iter_documents()`,
one of which now guards the files-vs-vectors keying directly.

Implements Phase 2 of the RAG driver abstraction design (§6).

src/rag/provider.rs

@@ -0,0 +1,87 @@
+use super::{DocumentId, RagData};
+use anyhow::Result;
+use async_trait::async_trait;
+
+/// Abstracts where RAG vector data is stored and queried.
+///
+/// Implementors:
+///   - YamlProvider: HNSW in-memory, state derived from RagData.vectors/files
+///   - DuckDbProvider: DuckDB on-disk vector index + document store
+///   - QdrantProvider: remote Qdrant collection
+///
+/// The Rag orchestrator owns: embeddings, chunking, BM25 keyword search, graph RAG,
+/// entity extraction, RRF merging. Providers own: vector storage and content retrieval.
+#[async_trait]
+pub trait RagProvider: Send + Sync {
+    /// Vector similarity search. Returns (DocumentId, score) sorted by score desc.
+    /// `embedding` is a single query vector from Coyote's embedding model.
+    async fn vector_search(
+        &self,
+        embedding: &[f32],
+        top_k: usize,
+        min_score: f32,
+    ) -> Result<Vec<(DocumentId, f32)>>;
+
+    /// Resolve document IDs to their page content.
+    ///
+    /// **Ordering contract:** implementations MUST return results in the same
+    /// relative order as the input `ids` slice. `hybrid_search` passes an
+    /// RRF-ranked list and feeds the result straight to the LLM — a provider
+    /// that returns rows in storage order (e.g. Qdrant `get_points`, DuckDB
+    /// `WHERE id IN (...)`) would silently discard the ranking. Implementations
+    /// that query an unordered backend must re-sort by input position before
+    /// returning.
+    ///
+    /// Returns only IDs that were found; callers must handle partial returns
+    /// (a missing ID is skipped, not an error).
+    /// YamlProvider: reads from an in-memory content map built from data.files.
+    /// DuckDbProvider: queries the documents table by id.
+    /// QdrantProvider: fetches payload from the remote collection.
+    async fn fetch_content(&self, ids: &[DocumentId]) -> Result<Vec<(DocumentId, String)>>;
+
+    /// Rebuild internal indexes from freshly updated RagData.
+    /// Called once at the end of every sync_documents pass.
+    ///
+    /// `full_rebuild` mirrors `sync_documents`' `refresh` parameter:
+    ///   - `true`  — a full re-index (`.rebuild rag`, `--rebuild-rag`, initial build).
+    ///     Destructive strategies (wipe-then-reindex) are permitted.
+    ///   - `false` — an incremental change (`.edit rag-docs` adding/removing a file).
+    ///     Implementations MUST NOT wipe existing state; upsert only.
+    ///
+    /// The parameter is part of the signature from the outset so it is fixed
+    /// while there is exactly one implementor. Yaml/DuckDb ignore it —
+    /// rebuilding their local state wholesale is fast and always correct.
+    /// Only a remote provider is destructive enough to care.
+    ///
+    /// YamlProvider: rebuilds HNSW + content map from data.vectors/files.
+    /// DuckDbProvider: writes new rows to DuckDB, deletes removed rows.
+    /// QdrantProvider: no-op while attach-only — remote data is unchanged.
+    async fn rebuild_indexes(&mut self, data: &RagData, full_rebuild: bool) -> Result<()>;
+
+    /// Keyword / full-text search. Returns (DocumentId, BM25-style score) sorted desc.
+    ///
+    /// Default impl returns `Ok(vec![])` — callers fall back to `Rag.bm25` (local in-memory
+    /// BM25 built from `data.files`). DuckDbProvider overrides this with a native FTS query
+    /// (DuckDB's `fts` extension, installed once at schema-creation time).
+    ///
+    /// Callers check `has_native_keyword_search()` before deciding which path to take:
+    ///   - true  → call this method; skip `Rag.bm25`
+    ///   - false → call `Rag.keyword_search()` which uses `Rag.bm25` (sync, infallible)
+    ///
+    /// YamlProvider and QdrantProvider do NOT override this (return empty).
+    async fn keyword_search(&self, query: &str, top_k: usize) -> Result<Vec<(DocumentId, f32)>> {
+        let _ = (query, top_k);
+        Ok(vec![])
+    }
+
+    /// Returns true if this provider implements a native keyword-search index.
+    /// When false, `Rag.hybrid_search` uses the local `Rag.bm25` field instead.
+    fn has_native_keyword_search(&self) -> bool {
+        false
+    }
+
+    /// Deep-clone the provider with fresh indexes derived from `data`.
+    /// Required because Box<dyn RagProvider> is not Clone.
+    /// Called by Rag's Clone impl (which clones before mutating in rebuild_rag/edit_rag_docs).
+    fn duplicate(&self, data: &RagData) -> Box<dyn RagProvider>;
+}