Declare the embedding width beside the model instead of assuming it

[sroussey]

Closes #354. Closes #359.

The problem

SEC_EMBEDDING_DIMENSIONS was a compile-time 768 used both to create the vector column and to check it, so stored.dimensions === dimensions was 768 === 768 on every path. The model id, meanwhile, is free-form env input.

A genuinely narrower model therefore opened the knowledge base without complaint and failed on the first chunk, from inside @workglow/knowledge-base, with a message naming neither the variable nor the model — after the weights had downloaded and the run had started.

The fix

The width is declared next to the model it belongs to:

const DEFAULT_EMBEDDING_MODEL = "onnx:Xenova/bge-base-en-v1.5:q8";
const DEFAULT_EMBEDDING_DIMENSIONS = 768;

bge-base-en-v1.5 is a BERT-base encoder with a hidden size of 768, and the record registers it with pooling: "mean" — so the vector handed back is the mean of the last hidden states and has exactly that many components. It is a property of the pinned model, and the two now move together: a reader who changes one sees the other on the next line.

Resolution is SEC_EMBEDDING_DIMENSIONS → the pinned model's declared width → refuse.

What this replaced, and why it is smaller

An earlier version of this branch read the width from the model's published config.json on the Hub and remembered it under SEC_RAW_DATA_FOLDER/model-cache/. It worked, and it bought a network call, a cache file, and an async hop through hftModelRecord → secModelRecord → getSecKnowledgeBase — to discover a number about a model this CLI pins. The whole embeddingWidth module and its cache are gone, and every signature it made asynchronous is back to synchronous.

A hand-written table of ten model widths came before that, and is also gone. The fix is not to look the width up; it is to stop pretending a constant describes a model it was not written for.

The trade, stated plainly

A model this CLI does not pin no longer resolves by itself. SEC_EMBEDDING_MODEL=onnx:Xenova/all-MiniLM-L6-v2:q8 worked under both the table and the fetch; it now refuses until SEC_EMBEDDING_DIMENSIONS=384 is set.

The guard the issue asked for is intact and is what always mattered: the width is resolved before any DDL, and a width that cannot be established refuses at open, naming the model and the variable. Stating the width is also the only answer for a cloud embedding endpoint, which has no local weights to inspect — and .env.test no longer has to pin a width to keep the suite off the network, because nothing reaches for it.

A malformed override throws rather than falling through to the default: creating the column at a width the model does not produce is the corruption the whole guard exists to prevent.

Two tests were measuring the wrong guard, and one passed while doing it

refuses an index built by a different model asserted that the error names both models and SEC_EMBEDDING_MODEL. The width refusal beside it names all three too — so once the width guard started firing first, that test went green against an entirely different error.

Both tests now state the width they need to reach the model-id check, and that one also asserts not comparable, wording only the real guard uses. Verified by removing the width again and watching it fail, rather than assuming.

Verification

9 cases in embeddingDimensions.test.ts: the pinned model by default and named explicitly (they are one constant, so both must reach 768), the refusal naming model and variable, the override rescuing a non-pinned model, the override beating the default, and four malformed values.

format-check, tsc --noEmit, lint clean; 667 passed / 21 skipped across src/config, src/kb, src/task and src/cli. grep confirms no model-cache, huggingface.co or resolveEmbeddingWidth remain.

Merge notes

Two merges of main are in this branch; both conflicts were CHANGELOG bullets where neither side edits the other's, so both sets are kept. The first also interleaved two adjacent, independent describe blocks in secKnowledgeBase.test.ts — the width refusal and the dry-run guard — and both are kept whole. secKnowledgeBase.ts auto-merged correctly: the width is resolved before any DDL, with the dry-run guard where the setupDatabase() calls were.

.claude/CLAUDE.md's env table describes both variables in their new terms.

🤖 Generated with Claude Code

https://claude.ai/code/session_01LWp6Z6wvAPDaDCjAFcTSj6

[sroussey]

Done in a34b558c — the table is gone and the width comes from the model.

How it asks

resolveEmbeddingWidth fetches the model's published config.json from the Hub — the same file the runtime loads the architecture from — and reads hidden_size (with d_model / n_embd / hidden_dim / dim for the other architectures).

hidden_size is the width here rather than an approximation of it: these records are registered with pooling: "mean", so the vector handed back is the mean of the last hidden states and has exactly that many components. A model with a projection head would not change that, because the record does not apply one.

On the two objections in my own "Left open"

I wrote that this "needs the model loaded or a maintained table either way". Both halves were wrong:

• Loading is not needed. The config is a couple of kilobytes, and the weights are not touched. The answer is remembered per repo under SEC_RAW_DATA_FOLDER/model-cache/embedding-widths.json, so a model used once resolves with no request at all. First use does need the network — and first use downloads the weights over the same connection, so there is no case where this is the request that cannot be made.
• Measuring by embedding is the option that is actually blocked, and by this repo's own comment: the provider "checks the tensor it produced against native_dimensions, so an embedding record without it fails AFTER running the model, with the declared width reported as undefined". The width has to exist before the first embed, so it cannot come from one.

What is left, honestly

SEC_EMBEDDING_DIMENSIONS stays first, and it is now the answer for a model with no config to read rather than for one nobody listed — a cloud endpoint (gemini-embedding-001) has no org/repo shape at all, and inventing a Hub path for it would ask about a model that is not there and read the 404 as a width. A width that cannot be established refuses at open, naming the model and the variable, before any table is created.

Resolving the width is asynchronous now, which reaches hftModelRecord and secModelRecord; both were only ever called from registerModelIds, which already awaited.

Verified

32 cases in embeddingWidth.test.ts, split across the repo parser, the config reader, and the resolver. Three are the ones that matter:

• asks once and remembers — the second call is served from disk with fetch stubbed to reject, and the request count is still 1.
• opens a model no table here lists — onnx:some-org/some-unlisted-model:q8 resolves to 1024. That is the case the table refused, and refusing was the entire cost.
• takes an explicit width without asking anything — the override short-circuits the Hub entirely; asserted on the call list, not just the return value.

format-check, lint, typecheck clean; 387 passed / 6 skipped across src/config, src/kb, src/task/kb and src/cli. .env.test pins SEC_EMBEDDING_DIMENSIONS=768 so no other suite reaches the network for it.

---

Generated by Claude Code