Make embedding model and task prefixes configurable via env#118
Closed
taehyeonglim wants to merge 1 commit into
Closed
Make embedding model and task prefixes configurable via env#118taehyeonglim wants to merge 1 commit into
taehyeonglim wants to merge 1 commit into
Conversation
The default embedding model (Xenova/bge-small-en-v1.5) is English-only. On non-English corpora retrieval quality collapses: in an A/B on a Korean-language corpus (same conversations, same pre-registered queries), the default model placed the right conversation in the top-3 for 1/3 queries, while Xenova/multilingual-e5-small (also 384-dim) scored 3/3. Adds three env overrides, all defaulting to current behavior: - EPISODIC_MEMORY_EMBEDDING_MODEL - EPISODIC_MEMORY_EMBEDDING_QUERY_PREFIX - EPISODIC_MEMORY_EMBEDDING_PASSAGE_PREFIX Docs cover the 384-dim constraint and the reindex-after-switching caveat. Adds test/embedding-env.test.ts. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
There was a problem hiding this comment.
Pull request overview
This PR makes the embeddings layer configurable via environment variables so non-English corpora can use a multilingual encoder (and appropriate task prefixes) without changing code, while keeping defaults identical to today.
Changes:
- Add env-driven overrides for embedding model id, query prefix, and passage prefix in
src/embeddings.ts. - Add a new Vitest suite covering query-prefix env override behavior and module reload semantics.
- Document the new configuration and caveats in
README.md, and record the change inCHANGELOG.md.
Reviewed changes
Copilot reviewed 4 out of 4 changed files in this pull request and generated 3 comments.
| File | Description |
|---|---|
src/embeddings.ts |
Reads embedding model/prefix configuration from env and applies passage prefix during exchange embedding generation. |
test/embedding-env.test.ts |
Adds tests for env-driven query prefix overrides and idempotency. |
README.md |
Documents how to configure multilingual embedding models and prefixes, plus reindex caveats. |
CHANGELOG.md |
Adds an Unreleased entry describing the new env configuration knobs. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
Comment on lines
+31
to
+35
| const MODEL_ID = process.env.EPISODIC_MEMORY_EMBEDDING_MODEL || 'Xenova/bge-small-en-v1.5'; | ||
| const MODEL_DTYPE = 'q8'; | ||
| export const BGE_QUERY_PREFIX = 'Represent this sentence for searching relevant passages: '; | ||
| export const BGE_QUERY_PREFIX = | ||
| process.env.EPISODIC_MEMORY_EMBEDDING_QUERY_PREFIX ?? | ||
| 'Represent this sentence for searching relevant passages: '; |
| } | ||
|
|
||
| return generateEmbedding(combined); | ||
| return generateEmbedding(PASSAGE_PREFIX + combined); |
Comment on lines
99
to
+101
| } | ||
|
|
||
| return generateEmbedding(combined); | ||
| return generateEmbedding(PASSAGE_PREFIX + combined); |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Why
The default embedding model (
Xenova/bge-small-en-v1.5) is English-only. On non-English corpora, retrieval quality collapses. A/B on a Korean-language corpus (3,862 real Claude Code conversations, same pre-registered queries, hit@3 against known-answer sessions):What
Three env overrides, all defaulting to current behavior — no behavior change unless set:
EPISODIC_MEMORY_EMBEDDING_MODEL(defaultXenova/bge-small-en-v1.5; must stay 384-dim)EPISODIC_MEMORY_EMBEDDING_QUERY_PREFIX(default: BGE sentence prefix; e5-family wantsquery:)EPISODIC_MEMORY_EMBEDDING_PASSAGE_PREFIX(default""; e5-family wantspassage:, applied before the 2000-char truncation)Docs cover the 384-dim constraint and the reindex-after-switching caveat (
EMBEDDING_VERSIONis not bumped by env changes). Tests:test/embedding-env.test.ts— override,??empty-string semantics, idempotency under an overridden prefix. Subset run (embedding-env + query-prefix): 7/7.Happily running this in production against a mixed Korean/English corpus. Thanks for the tool — the conversation-archive design saved us from Claude Code's ~30-day transcript cleanup.
🤖 Generated with Claude Code