Skip to content

feat: URLs for every file-reference column and its rendered versions; vector search per column model - #2008

Merged
pyramation merged 9 commits into
mainfrom
feat/image-version-urls
Nov 16, 2026
Merged

pyramation merged 9 commits into
mainfrom
feat/image-version-urls

Conversation

@pyramation

@pyramation pyramation commented Nov 16, 2026 •

Copy link
Copy Markdown
Contributor

Summary

Two read-side pieces that the db work (constructive-db #4369, #4371, #4372) needs before a client can use what it stores. Planning: constructive-planning#2519.

1. Every file-reference column gets URLs (graphile-presigned-url-plugin)

One mechanism for every column of a file domain and its array form, not a hook per type. The domains are one declared list, FILE_REF_DOMAINS = ['image', 'upload'] in graphile-storage-registry (overridable with the plugin option fileRefDomains); a future file type is a name added there. constructive-db marks these columns by domain only: there is no smart tag on a business-table file column (@storageFiles is on the files table), and db itself keys on the same domain names in data_file_ref and provision_table. attachment is a text domain holding a link, so it is not in the list.

A file column is a JSON domain and cannot grow fields of its own, so each one gets two sibling fields:

products {
  nodes {
    photo                      # image: the document, as before
    photoUrl(width: 640)       # narrowest rendered version >= 640px, else the source
    photoVersions { name key width height format size url }
    manualUrl                  # upload: no versions, so the source
    manualVersions             # []
  }
}
# array column `shots` -> shotsUrl(width): [String], shotsVersions: [[FileVersion!]]

FileVersion is name key width height format size url, dimensions nullable. A version without a width never satisfies width, so the source is returned.

Security rule: the document is data, never a key to sign. A client can write its own versions (and key) into the document, so:

id      = document.id                      -- nothing else is read for authorization
row     = SELECT id, key, bucket_id, is_public, status, filename FROM <files table> WHERE id = ANY($ids)
          -- on the request's own connection: reader RLS decides, no role switch
source  = url(row.key, row.filename)       -- never document.key
version = url(v.key) only if isVersionKeyOf(row.key, v.key)
          -- v.key == `${row.key}.versions/<one segment>`; no '/', '.', '..'

No id, no row the reader can see, a row that is not live, or a bucket with no physical name gives null. A forged entry is dropped, not an error.

Same bucket handling as downloadUrl, each asserted against downloadUrl or the seeded coordinates in file-ref-urls.integration.test.ts: public file -> publicUrlPrefix URL (versions too); private -> presigned in the database's own physical bucket; a file in the database's second storage module is found; unreconciled bucket -> null; the source URL carries the row's filename as content-disposition.

Cost. Documents resolved in the same tick share one read per request (RequestFiles, keyed on the request's pgSettings): one files lookup per storage module for all ids, one coordinate read per bucket. Storage queries for a 50-row list selecting photoUrl and photoVersions:

queries
before this commit 305
now 3
downloadUrl on 50 files rows, for comparison 106

The test asserts <= 4.

Tests (graphql/server-test/__tests__/file-ref-urls.integration.test.ts): for image, image[], upload and upload[]: another file's key, another file's version key, .. traversal, nested path, the source key listed as a version, a lying top-level key, a file RLS hides from the reader, a document with no id; plus the bucket cases, the query count, and that no type or field is named after one file kind.

2. A vector column is searched with its own model (graphile-search, graphile-llm)

Unchanged by the rework. constructive-db #4372 stamps each vector with the model that produced it and tags the columns: @embeddingModel <model> on the vector column when its node names one, @embeddingModelOf <column> on the companion <column>_model. This PR reads those tags:

  • VectorNearbyInput.text is embedded with the column's @embeddingModel, else the request's model (tenant, then platform), as before. EmbedderFunction takes an optional model.
  • VectorNearbyInput gains model: String, filled in server-side when text is embedded. On a column with a companion model column, only rows whose stamp equals it are compared. A client sending its own vector may name the model; without one nothing is filtered (unchanged behavior).
  • unifiedSearch embeds the text once per distinct model among the table's vector columns and scores each column with its own vector. A row stamped with another model gets a null distance, not a wrong one. It still only ranks, it does not start filtering.
  • A column with no companion model column behaves exactly as before.

Tests: graphile-llm/src/__tests__/vector-model.test.ts.

Review focus

  • RequestFiles batches with process.nextTick and caches rows for the life of the request's pgSettings object. A file deleted mid-request keeps its URL until the request ends.
  • An id found in an earlier storage module is not looked up in later ones, as resolveStorageModuleByFileId does.
  • VECTOR_COLUMNS_EXT is the same string constant in graphile-search/src/plugin.ts and graphile-llm/src/embed-inputs.ts (graphile-search does not depend on graphile-llm).
  • In pgvector.ts, the model condition narrows whereClause only when a distance already made one; otherwise it rides on scorableClause and a CASE around the score, so unifiedSearch semantics do not change.

After merge (Dan)

  1. Publish from main: graphile-storage-registry, graphile-presigned-url-plugin, graphile-search, graphile-llm and their dependents (graphile-settings, @constructive-io/graphql-server).
  2. In constructive-db, bump the constructive pin to the published sha/versions and run regenerate-branch.yml so the SDK/schema snapshots pick up <column>Url, <column>Versions, FileVersion and VectorNearbyInput.model.
  3. In constructive-db-integration-tests, bump pins.json (constructive, constructive-db) to those shas, then merge integration-tests remove travis #18; until the pin moves its hops only pass when the run is dispatched with constructive_ref set to this branch's sha.
  4. No tenant action: the new fields are additive and a document without versions resolves to the source URL.

Link to Devin session: https://app.devin.ai/sessions/d7ca5693b59842008312a29e09ddacb2
Open in Devin Desktop: https://app.devin.ai/desktop/session/d7ca5693b59842008312a29e09ddacb2?variant=devin
Requested by: @pyramation


🟢 Devin Review found nothing to fix · View findings · Follow Devin's work

Devin Review

@devin-ai-integration

Copy link
Copy Markdown
Contributor

I'll fix CI failures and address comments from users with write access. I'll skip comments containing "(aside)".

  • Disable automatic comment, CI, and merge conflict monitoring

@luxor-code-review

luxor-code-review Bot commented Nov 16, 2026 •

Copy link
Copy Markdown

Review complete. 🟠 1 high

💬 Inline comments (1)

  • 🟠 Failed per-model embed falls back to wrong-model vector — plugin.ts:1374

The change introduces per-model embedding across the search stack: a new @embeddingModel smart tag drives per-column model selection, embed inputs and the embedder are reworked to accept and report models, and the pgvector adapter guards scoring against model-mismatched vectors. The presigned-URL plugin gains imageUrlVersions/related fields that resolve version keys for image and image[] columns against resolved files, plus a preset wiring update. Test support adds an albums table to the upload-columns seed fixture and a new integration test covering version URLs and RLS behavior.

Files Change
graphile/graphile-llm/src/embed-inputs.ts, embedder.ts, types.ts Rework embedding input collection and the embedder to carry per-column models and report the model actually used.
graphile/graphile-search/src/adapters/pgvector.ts, plugin.ts Add the @embeddingModel smart tag, per-column embedding in unifiedSearch, and a same-model guard in the pgvector score expression.
graphile/graphile-presigned-url-plugin/src/image-version-urls.ts, index.ts, preset.ts New version-URL resolution fields for image and image[] columns, exported via the plugin preset.
graphql/server-test/__fixtures__/seed/upload-columns/schema.sql Add albums table with image column and grants to the seed fixture.
graphql/server-test/__tests__/image-version-urls.integration.test.ts, graphile/graphile-llm/src/__tests__/vector-model.test.ts New integration test for image-version URLs and RLS, plus unit tests for per-model embedding.

Reviewed commit: 9c3f82e

@luxor-code-review luxor-code-review Bot left a comment •

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This PR makes pgvector search embed each vector column with its own declared embedding model, adds image-version URL resolution fields for image[] columns in the presigned-URL plugin, and extends the test fixtures and integration tests accordingly.

Key findings

  • 🟠 Failed per-model embed falls back to wrong-model vector — plugin.ts:1374

Comment thread graphile/graphile-search/src/plugin.ts Resolved

@devin-ai-integration devin-ai-integration Bot left a comment •

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

✅ Devin Review: No Issues Found

Devin Review analyzed this PR and found no bugs or issues to report.

Devin Review

@devin-ai-integration

Copy link
Copy Markdown
Contributor

Integration hops for this PR: constructive-io/constructive-db-integration-tests#18 (merge only after this is published and pins.json is bumped). Gateway-from-source runs on c4f034be008210d53774587b53b4f2d3db6637cb: kitchen-sink-images and embedding-lanes, both green.

Written by Devin

<column>Url / <column>Versions for each column of a declared file domain
(FILE_REF_DOMAINS: image, upload) and its array form; ImageVersion becomes
FileVersion. The files rows of a request are read in one lookup per storage
module, and the source URL carries the uploaded filename as downloadUrl does.
@devin-ai-integration devin-ai-integration Bot changed the title feat: URLs for an image column's own rendered versions; vector search per column model feat: URLs for every file-reference column and its rendered versions; vector search per column model Nov 16, 2026
@pyramation
pyramation merged commit e77d9e2 into main Nov 16, 2026
20 checks passed
@pyramation
pyramation deleted the feat/image-version-urls branch November 16, 2026 21:41
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant