Skip to content

feat(images): add LQIP placeholders (dominant_color + blur_data_url) - #34

Merged
JexPY merged 2 commits into
masterfrom
feat/lqip-placeholders
Aug 24, 2026
Merged

JexPY merged 2 commits into
masterfrom
feat/lqip-placeholders

Conversation

@JexPY

@JexPY JexPY commented Aug 24, 2026

Copy link
Copy Markdown
Owner

Every ready image record now carries two tiny, self-contained placeholder values so a consumer can paint something meaningful before the real bytes arrive:

  • dominant_color — average colour as #rrggbb, for a solid background behind a loading image.
  • blur_data_url — a 16px WebP as a data:image/webp;base64,... URI (135–167 bytes in practice), for a blurred preview (Next.js placeholder="blur" and equivalents).

Migration 0008 adds both columns, nullable, no backfill. alembic downgrade -1 rolls back and re-applies cleanly — verified against the real Postgres service.

Exposed regardless of visibility

This deliberately breaks the "extras are public-only" symmetry of storage_key / renditions / accelerator URLs. These two fields carry no storage key, no URL and nothing cross-tenant, and a private record needs a placeholder exactly as much as a public one. A test asserts placeholders are present on a private upload while storage_key, renditions and thumbnail_url are absent.

IMAGE_PIPELINE_VERSION is deliberately NOT bumped

This adds metadata; it does not change the bytes of any stored object. Bumping would force a pointless re-encode of every re-uploaded image. Pre-existing rows carry NULL. Flagging it here because a reviewer would reasonably expect the opposite.

Where the tile comes from, and why it matters

The placeholder is derived from the encoded output buffer, after the main write_to_buffer. That placement is load-bearing twice over — both found by measuring on a real 24.5MP iPhone photo rather than a fixture:

  • pyvips recomputes a pipeline for every independent sink. Extracting from the full-resolution source forced a second full decode + ICC transform: 591ms from the source vs 15.8ms reading the output back. thumbnail_buffer on the source is not the fix either — shrink-on-load does not exist for PNG, where it measured 62.7ms, slower than on HEIC.
  • It must sit below the MAX_IMAGE_PIXELS guard. Extraction is the first call that forces evaluation, so above the guard a decompression bomb decodes fully before being rejected: a 120KB, 36-megapixel PNG took 256ms to reject instead of 0.3ms, synchronously, on an endpoint rate-limited at only 2 r/s per IP.

test_oversized_image_rejected_before_any_placeholder_work pins that ordering by monkeypatching the extractor to raise, so it fails deterministically rather than on a timing threshold.

The performance test was also rewritten: the original called .copy_memory() first, which pre-materialized the pixels and hid the decode — it reported 1.6ms for what is a 591ms operation on a real photo.

The trade-off, measured

Deriving from the lossy, downscaled output rather than the original source could in principle shift the placeholder. Measured across three real photos and all three optimization profiles:

Photo vs source across profiles
IMG_6034.HEIC (4284×5712) ≤1 / 255 per channel 1
IMG_6342.HEIC (3024×4032) 0 0
prof-pic.jpg (1024×1024) 0 0

At most one unit out of 255. Blur payload sizes are identical across profiles.

Verification

487 passed, 14 skipped · ruff check · ruff format --check · mypy app — all clean via docker compose run --rm --build test.

End to end through the running stack on real iPhone photos:

Photo Total Notes
4284×5712 HEIC 763 ms vs a 745 ms pre-placeholder baseline → ~18 ms of placeholder cost
3024×4032 HEIC 502 ms
1024×1024 JPEG 110 ms
36MP PNG bomb 0.3 ms rejected before any decode

Blur payloads 135–167 bytes against a 1200-byte tripwire test.

Notes for the reviewer

  • The INSERT in PostgresMetadataStore.create went from 15 to 17 placeholders — column list, $N list and positional args all verified to line up.
  • _image_response was 11 parameters after refactoring custom_* into the shared CustomImageParams dataclass, down from 12 and under the ≤13 bound.
  • docs/plans/media-engine/session-2-placeholders.md carries a "Corrected after implementation" note: the plan's original guidance (extract from the normalized source, before the max-dim resize) was wrong on both counts, and the note records the measurements so the mistake isn't repeated.

JexPY added 2 commits August 25, 2026 00:06
Every ready image record now carries two tiny, self-contained placeholder
values so a consumer can paint something meaningful before the real bytes
arrive: `dominant_color` (average colour as `#rrggbb`) and `blur_data_url`
(a 16px WebP as a `data:image/webp;base64,...` URI, ~135-170 bytes in
practice). Migration `0008` adds both columns, nullable, no backfill.

Both are exposed regardless of `visibility`, deliberately breaking the
"extras are public-only" symmetry of `storage_key`/`renditions`/accelerator
URLs: they carry no storage key, no URL and nothing cross-tenant, and a
private record needs a placeholder exactly as much as a public one.

`IMAGE_PIPELINE_VERSION` is deliberately NOT bumped. This adds metadata; it
does not change the bytes of any stored object, so a bump would force a
pointless re-encode of every re-uploaded image. Pre-existing rows carry NULL.

The placeholder tile is derived from the **encoded output buffer**, after the
main `write_to_buffer`, and that placement is load-bearing twice over:

- pyvips recomputes a pipeline for every independent sink, so extracting from
  the full-resolution source forced a *second* full decode + ICC transform.
  Measured on a real 24.5MP iPhone photo: 591ms from the source vs 15.8ms
  reading the output back. `thumbnail_buffer` on the source is not the fix
  either -- shrink-on-load does not exist for PNG, where it measured 62.7ms,
  slower than on HEIC.
- It must sit *below* the `MAX_IMAGE_PIXELS` guard. Extraction is the first
  call that forces evaluation, so above the guard a decompression bomb is
  fully decoded before being rejected: a 120KB, 36-megapixel PNG took 256ms
  to reject instead of 0.3ms, on the synchronous request path.

A new test pins that ordering by monkeypatching the extractor to raise, so it
fails deterministically rather than on a timing threshold. The performance
test now measures real encoded bytes -- an earlier version called
`.copy_memory()` first, which pre-materialized the pixels and reported 1.6ms
for what is a 591ms operation on a real photo.

Verified: 487 passed / 14 skipped; ruff check, ruff format --check and mypy
clean. Migration applies, `downgrade -1` rolls back, and re-applies. Measured
end to end through the running stack on real iPhone photos -- a 4284x5712 HEIC
is 763ms total (vs a 745ms pre-placeholder baseline, so ~18ms of placeholder
cost), 3024x4032 is 502ms, and a 1024x1024 JPEG is 110ms; blur payloads are
135-167 bytes against a 1200-byte tripwire.
@sonarqubecloud

Copy link
Copy Markdown

@JexPY
JexPY merged commit 3cf61ba into master Aug 24, 2026
12 checks passed
@JexPY
JexPY deleted the feat/lqip-placeholders branch August 24, 2026 20:30
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant