Skip to content

fix: flex image cache identity - #1735

Closed
hachall wants to merge 22 commits into
mainfrom
fix/flex-image-cache-identity
Closed

hachall wants to merge 22 commits into
mainfrom
fix/flex-image-cache-identity

Conversation

@hachall

@hachall hachall commented Sep 9, 2026

Copy link
Copy Markdown
Contributor

No description provided.

hachall and others added 2 commits September 8, 2026 22:58
…aching survives images

Flex bodies are stored with content-addressed dw-img:// tokens, but the
daemon's DispatchProcessor swapped them for fresh per-attempt signed URLs
BEFORE the dispatch looped back through the edge - so the prompt-cache
classifier hashed a different URL every call. Every image-bearing prefix
re-keyed on every dispatch: reads froze at the last pre-image boundary
while post-image entries were written (and billed) every call and never
read back. Byte-identical images made no difference. Realtime was
unaffected because its normaliser runs below the cache layer. Reported
and reproduced by graspit (2026-09).

Fix: the image-normaliser middleware now runs Mode::AllAndTokens and signs
dw-img:// tokens itself, below the prompt-cache layer, with the dispatch
TTL; the DispatchProcessor keeps only the ZDR decrypt. The cache now
hashes the stable token, which is exactly the identity we want (bytes,
not URLs). A token is only signed for the principal whose user or org
submitted the image (image_access), so an unowned or unattributable
token is refused with 403 - daemon legs pass because the hidden batch key
resolves to the enqueueing principal. Retries still get a fresh URL per
attempt (each loops back through the edge).

Tests: walker AllAndTokens coverage; middleware signs an owned token,
refuses another principal's token, refuses an unattributed caller; the
error-status contract gains Forbidden -> 403.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
…ownership

Resolves the lib.rs conflict (main moved the middleware pool to DynPools;
kept alongside the new token_ttl).

Also tightens the token-signing contract from the previous commit:

- dw-img:// tokens are an internal representation (flex enqueue / file
  ingest store them, and customers can read them back from stored inputs),
  so the edge now signs them ONLY on a daemon dispatch. The dispatch
  processor stamps every dispatch (batch_metadata "dispatch" -> header
  x-fusillade-batch-dispatch: 1, same trust perimeter as the stream marker);
  a client presenting a token directly - even its own - gets 403, so no
  unreviewed token-reuse surface is opened by accident.
- Ownership is by billing principal: an image submitted under an org key by
  one member is accessible to a dispatch under any key of that org (the
  hidden batch key's shape: user_id = org, created_by = whichever member),
  and personal keys match the submitting user. Covered by a new cross-member
  org test; plus tests for the no-marker and unattributed refusals.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Copilot AI lite review requested due to automatic review settings September 9, 2026 18:41
@cloudflare-workers-and-pages

cloudflare-workers-and-pages Bot commented Sep 9, 2026

Copy link
Copy Markdown

Deploying control-layer with  Cloudflare Pages  Cloudflare Pages

Latest commit: ebaf627
Status: ✅  Deploy successful!
Preview URL: https://e59706e2.control-layer.pages.dev
Branch Preview URL: https://fix-flex-image-cache-identit.control-layer.pages.dev

View logs

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

Dispatch-time token authorization can race flex enqueue’s fire-and-forget image_access insert, causing false 403s on immediately-dispatched queued requests.

Once you've addressed the issues Copilot identified, you can request another Copilot review.

Pull request overview

This PR fixes flex/batch prompt-cache identity for image-bearing requests by ensuring the cache hashes stable dw-img://… tokens (not per-dispatch signed URLs). It moves token signing to the edge image-normalizer middleware (below the prompt-cache layer), and introduces an explicit “daemon dispatch” marker so token signing is only permitted on trusted loopback dispatches.

Changes:

  • Add a dispatch marker (x-fusillade-batch-dispatch: 1) on every daemon dispatch, and use it to gate dw-img:// token signing.
  • Extend the image-normalizer walker/middleware to handle both URL/data-uri normalization and token signing (Mode::AllAndTokens), with an ownership check via image_access.
  • Remove the old dispatch-processor JIT token signing so prompt-cache keys remain stable across dispatch attempts.
File summaries
File Description
dwctl/src/lib.rs Wires token_ttl into the edge image-normalizer middleware state; removes dispatch-time JIT signing wiring.
dwctl/src/inference/outbound_request.rs Defines dispatch marker key/header constants for loopback detection.
dwctl/src/inference/middleware.rs Updates flex enqueue path commentary to reflect edge-loopback signing behavior.
dwctl/src/inference/image_normalizer_middleware.rs Adds token signing + authorization gated by dispatch marker; adds Forbidden mapping and SQLx-backed tests.
dwctl/src/inference/engine/dispatch_processor.rs Removes token signing from dispatch preparation; stamps dispatch marker into batch_metadata.
dwctl/src/image_normalizer/walker.rs Adds Mode::AllAndTokens to traverse/replace URLs, data URIs, and tokens in order.
dwctl/src/image_normalizer/mod.rs Introduces NormalizeError::Forbidden and updates module docs to reflect new signing placement.
dwctl/src/api/handlers/images.rs Adds is_token_accessible helper for token ownership/authorization checks.
dwctl/src/api/handlers/files.rs Handles the new Forbidden normalizer variant defensively during ingest-time normalization.
Review details
  • Files reviewed: 9/9 changed files
  • Comments generated: 1
  • Review effort level: Lite

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

Comment thread dwctl/src/inference/image_normalizer_middleware.rs Outdated
hachall and others added 3 commits September 10, 2026 08:28
…writes required

Review follow-ups on #1735:

- Drop the daemon-dispatch marker gate. A client re-sending a request it
  downloaded (whose body carries dw-img:// tokens) now works, provided the
  caller's user or organization submitted the image; today such a request
  fails upstream (the provider cannot fetch a token), so nothing working is
  taken away. Ownership remains the single gate: a non-owner gets 403 whether
  or not the bytes exist, so there is no existence oracle, and an owner can
  only mint URLs to bytes they already hold. Removes the marker constants,
  the processor stamping, the middleware check and its test.

- Close the race Copilot flagged: flex enqueue recorded image_access
  fire-and-forget, but the claim loop runs every 100ms, so a request could
  loop back before its own ownership row existed and fail terminally with a
  403 on its own image. The write is now awaited and REQUIRED on both queued
  paths (flex enqueue, file ingest) via try_record_image_access; a failed
  write fails the submission as a retryable 503 rather than accepting a
  request that could not later be served. Realtime keeps its fire-and-forget
  write (it signs immediately; no later authorisation depends on the row).

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

Database failures, TTL selection, and single-organization grants can incorrectly reject requests or weaken URL-expiry policy.

Once you've addressed the issues Copilot identified, you can request another Copilot review.

Review details

Suppressed comments (1)

dwctl/src/api/handlers/images.rs:200

  • Making image_access a dispatch authorization gate exposes the table's single-org limitation: its primary key is (user_id, sha256), and this upsert replaces organization_id when the same member submits identical bytes under another organization. Members of the first organization then lose access to a token they previously submitted, contradicting the new organization-wide ownership behavior. Store independent grants per organization (including personal scope) rather than one mutable organization per user/hash.
    sqlx::query!(
        r#"
        INSERT INTO image_access (user_id, organization_id, sha256, mime, bytes_len, first_seen_at, last_seen_at)
        VALUES ($1, $2, $3, $4, $5, NOW(), NOW())
        ON CONFLICT (user_id, sha256) DO UPDATE
  • Files reviewed: 8/8 changed files
  • Comments generated: 3
  • Review effort level: Balanced

Comment thread dwctl/src/inference/image_normalizer_middleware.rs Outdated
Comment thread dwctl/src/inference/image_normalizer_middleware.rs Outdated
Comment thread dwctl/src/inference/image_normalizer_middleware.rs Outdated
hachall and others added 3 commits September 11, 2026 11:08
… client re-sends

Address review on the flex image-cache fix. Authorising EVERY token signing
against `image_access` made the daemon's own dispatch depend on a
best-effort bookkeeping row and a live DB lookup, which turned three
non-problems into terminal 403s on the customer's own image: a lookup error
collapsed to "no owner", an org key that upserted over a personal grant,
and a dispatch that raced the row insert.

Two tiers now, split by a `dispatch` batch_metadata marker the daemon sets
on every dispatch (same trust argument as the stream marker: the ingress
strips `x-fusillade-*` from external requests):

- Marked daemon dispatch: sign on trust with the dispatch TTL, no lookup.
  The stored body was written by our own ingest under that request's
  principal, so it IS the authorisation record.
- Client-originated token: authorise per token against `image_access`
  (submitting user, or any member of the submitting org) and sign with the
  realtime TTL, since this is an ordinary request rather than a dispatch.
  Lookup failure is a retryable 503, not a 403; unknown key / no row is 403.

Flex enqueue keeps the awaited, required `image_access` write (it now only
backs the client re-send path), and fails the submission retryably when the
caller cannot be attributed instead of persisting tokens nobody may re-send.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

The dispatch marker permits an authorization bypass, and organization grants are overwritten for cross-org submissions.

Once you've addressed the issues Copilot identified, you can request another Copilot review.

Review details
  • Files reviewed: 9/9 changed files
  • Comments generated: 3
  • Review effort level: Balanced

Comment thread dwctl/src/inference/image_normalizer_middleware.rs Outdated
Comment thread dwctl/src/api/handlers/images.rs
Comment thread dwctl/src/inference/engine/dispatch_processor.rs Outdated
Review: a `x-fusillade-batch-dispatch` header is a forgeable authorisation
bypass anywhere dwctl is reachable without the stripping proxy (the default
compose exposes it directly). It was also redundant: the bearer on every
dispatch is the principal's hidden `batch`-purpose key — flex enqueue and
batch creation both store it on the request, and hidden keys are never
exposed to clients — so the daemon identity is already application-
verifiable at the image layer from the key lookup that was happening anyway.

`try_resolve_caller` now returns the attribution plus `is_daemon_dispatch`
(`purpose = 'batch' AND hidden`). The middleware trusts and signs tokens with
the dispatch TTL on that, and authorises every other caller against
`image_access` with the realtime TTL. The dispatch marker constants and the
processor-side stamping are gone; the daemon test dispatches with a real
hidden batch key.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

Queued requests can bypass token ownership checks, and organization grants are not represented reliably.

Once you've addressed the issues Copilot identified, you can request another Copilot review.

Review details

Files not reviewed (2)

  • .sqlx/query-6fe8629904a81b14af375d1ecf8d15a550019b8c02b07a6156fffeadf1637c75.json: Generated file
  • .sqlx/query-ee3562dcb489ec1525802e1acf4a9f498587db7a9ac16ced1aefa64e1e11db11.json: Generated file

Suppressed comments (1)

dwctl/src/api/handlers/images.rs:259

  • This check relies on image_access retaining every organization grant, but that table is keyed by (user_id, sha256) and its upsert stores only one organization_id. If one member submits the same image under org A and later org B, the second write replaces the first grant, so other members of org A now receive 403 when re-sending a token that was submitted under their org. Represent grants per organization (or in a separate grant table) before using this lookup for token authorization.
pub async fn is_token_accessible(
    pool: &sqlx::PgPool,
    attribution: &ImageAttribution,
    token: ImageToken,
) -> std::result::Result<bool, sqlx::Error> {
    let mut conn = pool.acquire().await?;
    is_authorized_to_view(&mut conn, &token.0, attribution.user_id, attribution.organization_id).await
  • Files reviewed: 9/11 changed files
  • Comments generated: 1
  • Review effort level: Balanced

Comment thread dwctl/src/inference/image_normalizer_middleware.rs Outdated
hachall and others added 2 commits September 11, 2026 14:48
… upload

Review: the hidden batch key proves a dispatch is the daemon's, but not that
every token in the stored body came from our own ingest — both submission
paths walked with `Mode::All`, which left a pre-existing `dw-img://` value
in a client body untouched. A client could persist another tenant's token
and have the trusted dispatch branch sign it unchecked.

Both paths now walk with `Mode::AllAndTokens`: a token already in the body
is authorised against the submitter's `image_access` (submitting user or
any member of the submitting org) and kept as is, refused otherwise (flex
403, file upload validation error on that line; a lookup failure is
retryable). With no way to authorise (no pool or attribution) the token is
refused rather than persisted on trust. That closes the invariant the
dispatch branch relies on: nothing reaches the store unless our ingest
produced it under this principal or this check passed.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟢 Approval recommended

The implementation and coverage are coherent; only minor inline documentation corrections remain.

Review details

Files not reviewed (2)

  • .sqlx/query-6fe8629904a81b14af375d1ecf8d15a550019b8c02b07a6156fffeadf1637c75.json: Generated file
  • .sqlx/query-ee3562dcb489ec1525802e1acf4a9f498587db7a9ac16ced1aefa64e1e11db11.json: Generated file
  • Files reviewed: 9/11 changed files
  • Comments generated: 3
  • Review effort level: Balanced

Comment thread dwctl/src/api/handlers/files.rs Outdated
Comment thread dwctl/src/api/handlers/images.rs Outdated
Comment thread dwctl/src/lib.rs Outdated
Three comments still described the round-2 model, where the daemon
consulted `image_access` at dispatch. The hidden batch key now takes the
trust branch without that lookup; the required write on the queued paths
exists so the submitting principal can re-submit the tokens it downloads.
Also: tokens can arrive on ordinary client requests (realtime TTL), not
only on the loopback.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

The final unterminated record in synced JSONL files bypasses image-token authorization.

Once you've addressed the issues Copilot identified, you can request another Copilot review.

Review details

Files not reviewed (2)

  • .sqlx/query-6fe8629904a81b14af375d1ecf8d15a550019b8c02b07a6156fffeadf1637c75.json: Generated file
  • .sqlx/query-ee3562dcb489ec1525802e1acf4a9f498587db7a9ac16ced1aefa64e1e11db11.json: Generated file
  • Files reviewed: 10/12 changed files
  • Comments generated: 1
  • Review effort level: Balanced

Comment thread dwctl/src/connections/sync.rs Outdated

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

Retryable sync failures become fatal, and organization image grants are not preserved across multiple organizations.

Once you've addressed the issues Copilot identified, you can request another Copilot review.

Review details

Files not reviewed (2)

  • .sqlx/query-6fe8629904a81b14af375d1ecf8d15a550019b8c02b07a6156fffeadf1637c75.json: Generated file
  • .sqlx/query-ee3562dcb489ec1525802e1acf4a9f498587db7a9ac16ced1aefa64e1e11db11.json: Generated file

Suppressed comments (1)

dwctl/src/api/handlers/images.rs:260

  • The new client-token authorization assumes image_access can retain every organization grant, but its primary key is (user_id, sha256) and the upsert replaces organization_id whenever the same person submits identical bytes under another organization (migrations/102_image_normalization.sql:17-25, images.rs:229-235). After that, another member of the first organization is incorrectly denied here even though that organization submitted the image. Model access as separate per-principal/per-organization grants (with a migration) and cover submissions by one user under two organizations.
pub async fn is_token_accessible(
    pool: &sqlx::PgPool,
    attribution: &ImageAttribution,
    token: ImageToken,
) -> std::result::Result<bool, sqlx::Error> {
    let mut conn = pool.acquire().await?;
    is_authorized_to_view(&mut conn, &token.0, attribution.user_id, attribution.organization_id).await
  • Files reviewed: 10/12 changed files
  • Comments generated: 1
  • Review effort level: Balanced

Comment thread dwctl/src/connections/sync.rs

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔵 Needs a closer look

The access schema retains only one organization grant per user and image, causing valid members of later organizations to receive 403 responses.

Review details

Files not reviewed (2)

  • .sqlx/query-6fe8629904a81b14af375d1ecf8d15a550019b8c02b07a6156fffeadf1637c75.json: Generated file
  • .sqlx/query-ee3562dcb489ec1525802e1acf4a9f498587db7a9ac16ced1aefa64e1e11db11.json: Generated file

Suppressed comments (1)

dwctl/src/api/handlers/images.rs:260

  • This authorization can wrongly reject a valid organization member when the same acting user has submitted the same image in more than one organization. image_access is keyed by (user_id, sha256) (dwctl/migrations/102_image_normalization.sql:17-26), and the upsert at images.rs:231-235 preserves only one organization_id; after a later submission in Org B, the row can still name Org A, so another Org B member gets false and a 403. The storage model/upsert needs to retain a distinct grant per organization before this lookup can enforce the documented user-or-organization rule.
pub async fn is_token_accessible(
    pool: &sqlx::PgPool,
    attribution: &ImageAttribution,
    token: ImageToken,
) -> std::result::Result<bool, sqlx::Error> {
    let mut conn = pool.acquire().await?;
    is_authorized_to_view(&mut conn, &token.0, attribution.user_id, attribution.organization_id).await
  • Files reviewed: 10/12 changed files
  • Comments generated: 0 new
  • Review effort level: Balanced

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

Transient sync failures become fatal, SQLx metadata is incomplete, and the migration and token-trust rollout have blocking risks.

Get a fresh assessment by requesting another Copilot review.

Review details

Files not reviewed (2)

  • .sqlx/query-6fe8629904a81b14af375d1ecf8d15a550019b8c02b07a6156fffeadf1637c75.json: Generated file
  • .sqlx/query-ee3562dcb489ec1525802e1acf4a9f498587db7a9ac16ced1aefa64e1e11db11.json: Generated file

Suppressed comments (2)

dwctl/src/connections/sync.rs:447

  • A transient image_access lookup is converted to Abort, but run_ingest_file maps FileStreamResult::Aborted to an anyhow error and build_ingest_file_job returns TaskError::Fatal (lines 879-881 and 110-132). The file is therefore permanently marked failed instead of retried. Propagate a typed retryable ingest error through this path and return TaskError::Retryable without updating the failed counters.
            Err(e) if e.is_retryable() => {
                tracing::error!(
                    line_num = line_number,
                    error = e.message(),
                    "Could not authorise image token in synced record; aborting ingest"

dwctl/src/inference/image_normalizer_middleware.rs:208

  • A hidden batch key does not prove every token in the stored body passed the new authorization checks. Flex and file uploads skip all token validation while image normalization is disabled; those queued bodies—and pre-deployment pending rows—can later be dispatched after the feature is enabled, at which point this branch signs an arbitrary client-supplied token on trust. Either authorize daemon tokens against the stored principal here or ensure tokens are rejected/authorized on every submission path regardless of the feature flag, including existing queued data.
                if caller.is_daemon_dispatch {
                    // The daemon's own dispatch of a body our ingest stored
                    // under this principal: the stored body is the
                    // authorisation record, so sign on trust with the dispatch
                    // TTL. No `image_access` dependency, so neither a
  • Files reviewed: 11/13 changed files
  • Comments generated: 3
  • Review effort level: Balanced

Comment thread dwctl/migrations/140_image_access_grant_per_organization.sql Outdated
Comment thread dwctl/migrations/140_image_access_grant_per_organization.sql Outdated
Comment thread dwctl/src/api/handlers/images.rs Outdated
hachall and others added 2 commits September 17, 2026 12:54
…ive table

Review round on #1735:

- No trusted dispatch branch. A hidden batch key proved the caller was the
  daemon, not that every token in the stored body had passed the
  submission-time checks (bodies queued while normalisation was off, or
  before those checks shipped, had not). Every token is now authorised
  against the image grants for the bearer's principal; the daemon's key
  resolves to the submitting principal, so a dispatch is checked exactly
  like a client re-send. The bearer only picks the TTL.

- Organisation grants move to a new `image_access_org_grants` table
  (migration 145, additive) instead of re-keying `image_access`: dropping
  its primary key would have broken ON CONFLICT for instances still on the
  previous release mid-rollout, and adding a stored generated column would
  have rewritten the table under an exclusive lock. The existing upsert is
  unchanged (its offline sqlx entry stays valid); the view/authorisation
  lookup checks the submitter's row, the new grants, and the legacy
  organisation column for rows written before the table existed.

- Connection sync: a transient lookup failure while authorising a record
  is a `RetryableIngest`, mapped to `TaskError::Retryable` without marking
  the entry failed or bumping counters, instead of a fatal abort.

- Offline sqlx metadata regenerated for the two changed/added queries.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

Legacy grants need backfilling, and raw synced images still receive the shorter realtime TTL.

Get a fresh assessment by requesting another Copilot review.

Review details

Files not reviewed (5)

  • .sqlx/query-25802e2f35ed9c7e1d1b488330e5a9777fe6a6dc2e17f973d1080d74ae990eaf.json: Generated file
  • .sqlx/query-6fe8629904a81b14af375d1ecf8d15a550019b8c02b07a6156fffeadf1637c75.json: Generated file
  • .sqlx/query-b19ed863200bff1dce8e037f820bcb4759acfd49d6886a6edc0b8d8acf985170.json: Generated file
  • .sqlx/query-b5c559a08d5cc7d10a64b02fdaf339f34e2f05672e3cced88e04f58d662cfd9f.json: Generated file
  • .sqlx/query-ee3562dcb489ec1525802e1acf4a9f498587db7a9ac16ced1aefa64e1e11db11.json: Generated file
  • Files reviewed: 11/16 changed files
  • Comments generated: 2
  • Review effort level: Balanced

Comment thread dwctl/migrations/145_image_access_org_grants.sql
Comment thread dwctl/src/inference/image_normalizer_middleware.rs Outdated
hachall and others added 2 commits September 17, 2026 13:49
…a loopback

Review round on #1735:

- Migration 145 copies every existing organization grant from the legacy
  `image_access.organization_id` column into `image_access_org_grants`.
  Without it, the first post-upgrade submission of an image under a second
  organization would overwrite the legacy column and leave the first
  organization with no grant anywhere — the regression the table exists
  to prevent. Share lock on the old table only.

- The daemon/realtime TTL choice now covers freshly ingested images, not
  only signed tokens. Synced records keep their raw image URLs until
  dispatch, so a daemon loopback reaches the ingest path too; its signed
  URL must outlive a full processing attempt.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

Daemon TTL selection can regress during database failures, and rolling deployment can permanently miss organization grants.

Get a fresh assessment by requesting another Copilot review.

Review details

Files not reviewed (5)

  • .sqlx/query-25802e2f35ed9c7e1d1b488330e5a9777fe6a6dc2e17f973d1080d74ae990eaf.json: Generated file
  • .sqlx/query-6fe8629904a81b14af375d1ecf8d15a550019b8c02b07a6156fffeadf1637c75.json: Generated file
  • .sqlx/query-b19ed863200bff1dce8e037f820bcb4759acfd49d6886a6edc0b8d8acf985170.json: Generated file
  • .sqlx/query-b5c559a08d5cc7d10a64b02fdaf339f34e2f05672e3cced88e04f58d662cfd9f.json: Generated file
  • .sqlx/query-ee3562dcb489ec1525802e1acf4a9f498587db7a9ac16ced1aefa64e1e11db11.json: Generated file
  • Files reviewed: 11/16 changed files
  • Comments generated: 4
  • Review effort level: Balanced

Comment thread dwctl/migrations/145_image_access_org_grants.sql
Comment thread dwctl/src/inference/image_normalizer_middleware.rs Outdated
Comment thread dwctl/src/api/handlers/files.rs Outdated
Comment thread dwctl/src/connections/sync.rs Outdated
… memoise token verdicts

Review round on #1735:

- Migration 145 installs an AFTER INSERT/UPDATE trigger on `image_access`
  that mirrors every organization write into `image_access_org_grants`.
  The snapshot backfill alone left a window: instances still on the
  previous release keep writing only the legacy column until they drain,
  and a second-organization write there would have been lost. The trigger
  and the legacy column are for a later contract migration to drop.

- The dispatch-vs-realtime TTL choice no longer depends solely on the key
  lookup. The daemon's `x-fusillade-batch-created-at` metadata header is on
  every dispatch (batch and flex; `created_at` is a default metadata field)
  and selects the dispatch TTL on its own, so a raw image ingested on a
  loopback during a lookup outage cannot get a URL that expires
  mid-attempt. The header decides TTL only, never authorisation.

- File upload and connection sync memoise token verdicts per submission
  (`TokenAuthCache`), so a file repeating a token across thousands of
  records costs one lookup, not one per record.

- The sync retry path records through `background_error!` (warning tier)
  so retries appear on the background-error dashboards without paging.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

The migration can miss organization grants written during its backfill-trigger installation window.

Get a fresh assessment by requesting another Copilot review.

Review details

Files not reviewed (5)

  • .sqlx/query-25802e2f35ed9c7e1d1b488330e5a9777fe6a6dc2e17f973d1080d74ae990eaf.json: Generated file
  • .sqlx/query-6fe8629904a81b14af375d1ecf8d15a550019b8c02b07a6156fffeadf1637c75.json: Generated file
  • .sqlx/query-b19ed863200bff1dce8e037f820bcb4759acfd49d6886a6edc0b8d8acf985170.json: Generated file
  • .sqlx/query-b5c559a08d5cc7d10a64b02fdaf339f34e2f05672e3cced88e04f58d662cfd9f.json: Generated file
  • .sqlx/query-ee3562dcb489ec1525802e1acf4a9f498587db7a9ac16ced1aefa64e1e11db11.json: Generated file

Suppressed comments (4)

Previously missed (4) — in code that hasn't changed since the last review.

dwctl/src/api/handlers/files.rs:385

  • This “on trust” statement conflicts with the new dispatch implementation, which authorizes every token through is_token_accessible. Please describe both checks accurately; this is a security invariant, not merely an implementation detail.
    dwctl/src/api/handlers/images.rs:219
  • This documents the opposite of the new security behavior: image_normalizer_middleware calls is_token_accessible for daemon tokens too (lines 219-236), and the new tests require unrecorded daemon tokens to be rejected. Describing dispatch as trusted could lead callers to treat this write as unnecessary; document that both dispatch and client re-submission depend on the grant.
    dwctl/src/connections/sync.rs:460
  • The dispatch no longer signs stored tokens “on trust”; image_normalizer_middleware rechecks the grant before signing. This comment should state that ingest-time authorization rejects bad records early while dispatch provides the final authorization check.
    dwctl/src/inference/image_normalizer_middleware.rs:335
  • The dispatch is not trusted by the implementation: the middleware re-resolves the hidden key and checks is_token_accessible before signing. Update this contract so future changes do not remove the required grant under the mistaken assumption that submission-time validation is the sole authorization boundary.
  • Files reviewed: 11/16 changed files
  • Comments generated: 2
  • Review effort level: Balanced

Comment on lines +29 to +33
INSERT INTO image_access_org_grants (organization_id, sha256, granted_by, first_seen_at, last_seen_at)
SELECT organization_id, sha256, user_id, first_seen_at, last_seen_at
FROM image_access
WHERE organization_id IS NOT NULL
ON CONFLICT (organization_id, sha256) DO NOTHING;
Comment on lines +461 to +465
if line_error.is_none()
&& body.contains("dw-img://")
&& let Ok(mut body_val) = serde_json::from_str::<serde_json::Value>(&body)
{
match crate::api::handlers::files::authorize_submitted_tokens(&mut body_val, token_pool, token_attribution, token_auth).await {
@hachall hachall closed this Sep 17, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants