refactor(rag): extract RagProvider trait and add YamlProvider
Introduce a narrow `RagProvider` trait covering vector search and content retrieval, and make `Rag` delegate to a boxed provider instead of owning an HNSW index directly. `YamlProvider` is the sole implementation for now. The trait deliberately stays narrow: embeddings, chunking, BM25 keyword search, graph RAG, entity extraction, RRF merging and persistence all remain on `Rag`/`RagData`, so a new storage backend does not have to reimplement Coyote's indexing logic. Notable points: - `fetch_content`'s ordering contract is part of the trait, not an accident. Implementations must return results in input-`ids` order; `hybrid_search` passes an RRF-ranked list straight to the prompt builder, so a provider returning storage order would silently discard the ranking. - The content store is keyed on `data.files`, never `data.vectors`. Both the content map and BM25 now route through the new `RagData::iter_documents()` so the two key spaces match by construction. `RagData::add` zips document ids with embeddings and truncates silently, so ids in `files \ vectors` are genuinely reachable. - A provider keyword-search failure degrades to an empty ranker with a warning rather than failing the whole query; it is one of three RRF inputs. It deliberately does not fall back to the local BM25, which would be a silent ranking-algorithm swap once a provider with native FTS exists. - The rerank path builds its text and id vectors from a single `fetch_content` result in one pass, so the reranker's positional indices cannot desync. This is not a bit-for-bit no-op. `vector_search` now dedups by best score and sorts globally instead of concatenating per-chunk hit lists. Single-chunk queries (the common case) are unaffected. Multi-chunk queries get corrected rank assignment and no longer let a document that matched several query chunks accumulate multiple RRF contributions. There is no overall cap on the merged pool — truncation remains `reciprocal_rank_fusion`'s job. `RagData::get()` is removed: its only two callers were the content lookups replaced here, and an unused private-module method fails the build under `--deny warnings`. Its three tests were rewritten against `iter_documents()`, one of which now guards the files-vs-vectors keying directly. Implements Phase 2 of the RAG driver abstraction design (§6).
This commit is contained in:
@@ -0,0 +1,6 @@
|
||||
mod yaml;
|
||||
// Use `self::` on every re-export in this file. Once a `mod duckdb;` sits here
|
||||
// alongside a dependency on the `duckdb` CRATE, a bare `pub use duckdb::...`
|
||||
// is ambiguous (E0659) — `use` paths resolve against both this module's items
|
||||
// and the extern prelude, and `use` declarations may not shadow.
|
||||
pub use self::yaml::YamlProvider;
|
||||
Reference in New Issue
Block a user