Slice 031: Session search (FTS5) - #38
Merged
Merged
Conversation
…slice opens ROADMAP row 031 to in_progress. NOTES.md records that exqlite 0.40.0's SQLite 3.53.4 carries FTS5 and that porter stems running to runs and not to ran, then the six-line build plan, the two-item manual queue and three deviations stated before any code. plan_check: PASS. Signed-off-by: Ayla Croft <aylacroft@proton.me>
…ty.Memory.Search, the reindex task Migration branching on the adapter: messages_fts with porter unicode61 and three triggers plus a backfill on SQLite, a generated content_tsv column with a GIN index on Postgres. Search.messages/2 with bound parameters, FTS5 terms quoted, filters, limit capped; reindex/0. Tests: AC1 on insert, update and delete; AC2 as measured (runs, not ran); filters and operator-shaped queries; AC3 over 10,000 rows with the counts equal and the time printed (32 ms here). Signed-off-by: Ayla Croft <aylacroft@proton.me>
… Memory Trinity.Tools.SessionSearch (risk read, effect none, query and limit capped at 50), registered in every environment; hits as text the model reads, wrapped untrusted. The Tools boundary gains Memory (docs/01 row as built). Tests: registered as a core read outside the catalog; at most limit hits with snippets through the runner in force, with the decision and query receipts; the no-hit text; a limit past the schema refused. Signed-off-by: Ayla Croft <aylacroft@proton.me>
…s built Signed-off-by: Ayla Croft <aylacroft@proton.me>
…ts it Signed-off-by: Ayla Croft <aylacroft@proton.me>
PROOF.md with every criterion's evidence by test name against run 35545182679 (gate, postgres and fips green; the postgres job's generated column doing AC1 and AC3's work). The manual halves in proof/: the GIF of nvidia:nemotron calling session_search and answering from last week's session, the search page with marked hits, and the deep link into the older session. NOTES.md: seven findings, three follow-ups. ROADMAP row 031 to done; coverage.tsv row 76.96 %. Signed-off-by: Ayla Croft <aylacroft@proton.me>
Signed-off-by: Ayla Croft <aylacroft@proton.me>
Merged
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Full-text search over every message: an FTS5 table with porter stemming kept by triggers on SQLite, a generated
tsvectorcolumn with a GIN index on Postgres,Trinity.Memory.Search.messages/2over both (the query bound as a parameter, FTS5 terms quoted so operators are text),mix trinity.search.reindex, thesession_searchcore read tool, and the/searchpage whose hits deep-link into the session at the message.Measured before code: exqlite 0.40.0's SQLite 3.53.4 carries FTS5; porter stems "running" to "runs" and not to "ran" (no stemmer maps an irregular past tense), so AC2's "ran" half is asserted as not found, with the reason. Reindex of 10,000 rows: 32 to 46 ms here; nothing to rebuild on Postgres.
Manual halves in
proof/: the GIF ofnvidia:nemotroncallingsession_searchand answering "October 21st" from a session seeded a week earlier; the search page with marked hits; the deep link into the older session. PROOF.md atslices/031-session-search-fts/PROOF.md.Signed-off-by: Ayla Croft aylacroft@proton.me