Read PORT.md first; this file is the map of what already exists, how to add
to it, and where the port deliberately differs from upstream.
server/scripts/dev.sh # start the dev DB (127.0.0.1:55421), migrate, serve on 127.0.0.1:8421
server/scripts/dev.sh migrate # only migrate + seed
cd server
cargo test # unit + integration tests (no database needed)
cargo clippy --all-targets -- -D warnings
cargo fmt --check # rustfmt.toml: max_width 120dev.sh generates server/docker/.env.devdb (database password, gitignored)
on first use and never prints it. The file lives in the MAIN checkout
(/path/to/modlight/server/docker/.env.devdb, found through
the shared git directory): every worktree uses the same database volume, so
they must share one password. Extra variables for the server can go into
server/.env (gitignored, loaded like dotenv/config upstream; variables
already in the shell win). Logs go to stderr, filtered by RUST_LOG.
Database tests (tests/db_compat.rs) run only when a server they may create
databases on is given; each run creates and drops its own database:
set -a; source "$(dirname "$(git rev-parse --path-format=absolute --git-common-dir)")/server/docker/.env.devdb"; set +a
MODLIGHT_TEST_DATABASE_URL="postgresql://modlight:${POSTGRES_PASSWORD}@127.0.0.1:55421/modlight" \
cargo test --test db_compatCLI: modlight-server [serve] (migrate, then serve; the upstream container ran
migrate.js before server.js) and modlight-server migrate (migrate.ts:
raw DATABASE_URL, PG* variables when unset, no production checks).
Environment: everything lib/config.ts reads (see src/config.rs, ENV_VARS),
plus BIND_ADDR (default 127.0.0.1; the Docker image sets 0.0.0.0 inside
the container), MODLIGHT_ROOT (where migrations/, messages/, public/
and web/dist/ live; default: the working directory if it contains
migrations/, else the crate directory), TZ (the timeZone handed to the
SPA, default UTC like the upstream container) and RUST_LOG. NODE_ENV=production
still means production (refusals in config.rs, production CSP, sanitised
action errors), so upstream .env files keep their meaning.
| Rust | Upstream | Status |
|---|---|---|
main.rs |
server.ts, migrate.ts |
done: CLI, runtime dirs, migrate-then-serve, jobs, graceful shutdown |
http.rs |
server.ts request handling |
done: raw upload bypass, path normalisation, router assembly, hyper loop without timeouts |
http/proxy.rs |
proxy.ts |
done |
http/headers.rs |
next.config.ts headers() |
done (CSP computed at startup) |
http/cookies.rs, http/set_cookie.rs |
Next / better-call / better-auth cookie parsing and serialisation | done |
http/static_files.rs |
public/, app/icon.svg, app/apple-icon.png, SPA assets/ |
done |
config.rs |
lib/config.ts |
done |
paths.rs |
(working directory conventions) | done |
error.rs |
(typed error at the edge) | done |
api_json.rs |
lib/api-json.ts + the per-route json() helpers |
done |
db/mod.rs, db/migrate.rs |
lib/db/index.ts, migrate.ts + drizzle 0.45.2 migrator |
done |
db/models.rs, db/types.rs |
lib/db/schema/*.ts + drizzle/*.sql |
done (every table and column, checked against information_schema) |
rpc/ |
Next server actions | done (framework) |
actions/tokens.rs |
app/actions/tokens.ts |
done |
actions/{uploads,folders,duplicates,shortlinks,shortlink_unlock,pastes}.rs |
app/actions/ same names |
done (parity: tests/parity/library_actions.py) |
actions/admin_library.rs |
app/actions/admin.ts (loadOrphans, deleteOrphan, purgeAllOrphans, loadDeletionImpact, deleteUserAccount, recalculateEveryQuota, runStorageScan), components/admin/confirm.ts |
done; registered as module admin beside the accounts port's own actions/admin.rs |
js_value.rs |
(JavaScript operators on raw arguments) | done |
actions/avatar.rs |
app/actions/avatar.ts |
done (myAvatar is in CONTRACT 7.3 but unreachable on the reference build: no client component imports it) |
actions/convert.rs |
app/actions/convert.ts |
done (parity: tests/parity_media/convert_actions.py) |
actions/unlock.rs |
app/actions/unlock.ts |
done (verifyMediaPassword; parity media_serve.py --only gates) |
pages/ |
page data, SPA shell, metadata | done (framework + table of every page and route handler) |
pages/root_layout.rs |
app/layout.tsx |
done |
pages/layout_app.rs, pages/layout_auth.rs, account pages |
app/(app)/layout.tsx, app/(auth)/layout.tsx, /, /welcome, /download, auth pages, /dashboard/{profile,tokens,admin} |
done (accounts port; the (app) layout reads the library's data functions) |
pages/dashboard*.rs, pages/shortlink_unlock.rs |
/dashboard, /dashboard/{folders,trash,duplicates,links,paste,upload}, /dashboard/admin/users/[id]/files, /s/[code]/unlock, /s/[code]/[name]/unlock |
done (parity: tests/parity/library_pages.py) |
pages/media.rs, pages/public_profile.rs, pages/public_album.rs |
app/[userCode]/[mediaCode]/page.tsx, app/[userCode]/page.tsx, app/u/[userCode]/page.tsx, app/a/[userCode]/[slug]/page.tsx |
done: props, generateMetadata (Open Graph, Twitter, oEmbed and link-info discovery), the owner's skin and wallpaper on <html>, the tagless action redirect (parity media_serve.py) |
i18n/ |
i18n/request.ts, next-intl formatting |
done (locale, messages, ICU subset used by the dictionaries) |
themes.rs, theme_cookies.rs |
lib/themes.ts, lib/theme-cookies.ts |
done |
auth/cookies.rs, auth/session.rs |
better-auth 1.6.13 cookies + getSession |
done |
auth_helpers.rs |
lib/auth-helpers.ts |
done (functions + axum extractors) |
auth/api.rs, auth/password.rs, auth/two_factor.rs, auth/admin.rs, auth/email.rs |
/api/auth/*, better-auth plugins |
layout + docs only (auth agent) |
api_scopes.rs, api_token.rs, api_caller.rs |
lib/api-scopes.ts, lib/api-token.ts, lib/api-caller.ts |
done |
password.rs |
lib/password.ts |
done: client_ip, rate limiter, file-password hashing (accounts port); unlock tokens and cookies, owner-scoped consent (library port) |
ids.rs, audit.rs |
lib/ids.ts, lib/audit.ts |
done |
storage/ |
lib/storage/* |
done: local driver; S3 driver on aws-sdk-s3 with ports of lib-storage Upload (s3/lib_storage.rs) and s3-request-presigner (s3/presigner.rs, URLs byte-identical to the JS SDK's); is_not_found |
upload/tus_server/s3_store.rs |
@tus/s3-store 2.0.4 |
done (S3Store::from_env = makeDatastore() for S3); keys, .info JSON, metadata and tags as the TS store, uploads resume across the two |
routes/api/health.rs |
app/api/health/route.ts |
done |
jobs.rs |
timers in server.ts |
done: tus sweep (upload/tus.rs), trash purge (maintenance/trash.rs), mail settings (accounts port) |
upload/sharex.rs, upload/busboy.rs |
lib/upload/sharex.ts + busboy 1.6's part rules |
done (security fix 2; parity library.py --only sharex) |
upload/tus.rs, upload/tus_server/{server,store,file_store}.rs |
lib/upload/tus.ts, @tus/server 2.4.1, @tus/utils 0.7.0, @tus/file-store 2.1.0 |
done (records interchangeable with the TS store; tus-js-client 4.3.1 resumes across both; tests/parity/tus_client.cjs) |
upload/{finalize,disk,folder_target,duplicate_lookup,duplicate_check}.rs |
lib/upload/* |
done |
media/{sniff,mime,urls,paths,paste_type,actions,derived_cache,convert_targets,av_targets}.rs |
lib/media/* same names |
done (derived_cache: keys, purge, mayCacheDerived; reading and writing cached conversions belongs to the media serving port) |
media/serve.rs |
lib/media/serve.ts |
done: getMedia (+ lookup_media, which tells a spent budget apart), viewerRelation, consumeView, cacheHeadersFor, parseRange, streamMedia; security fix 1 (the view grant, ViewBudget) |
media/{bots,render_ticket,embed,link_formats}.rs |
lib/media/ same names |
done (media serving port) |
media/{convert,convert_plan,convert_size,convert_verdict,convert_notes,convert_in_place,download_menu,av_convert,av_action,motion,motion_preview,timeline,timeline_preview,poster,frame_step,source_preview,card,card_art,card_render,player_i18n,playback_health}.rs, media/pdf_lib.rs |
lib/media/ same names, the part of pdf-lib 1.17.1 convert.ts uses |
done (media processing port: section 12) |
media/part_a.rs |
the processing half of lib/media as the serving half calls it |
the serving port's adapter, wired to the processing modules (section 12) |
crypto/envelope.rs |
lib/crypto/envelope.ts |
server part: constants, isEncryptedUpload (the rest runs in the browser) |
routes/media_action.rs |
app/[userCode]/[mediaCode]/[action]/route.ts |
done: raw, raw.<ext>, download, render, timeline.vtt/jpg, convert (the engines through part_a) |
routes/{media_delete,album_zip,embed_card,oembed}.rs, archiver.rs |
app/[userCode]/[mediaCode]/delete/[token], app/a/[userCode]/[slug]/zip, app/embed/[version]/[card], app/oembed; archiver 8 / zip-stream 7 / compress-commons 7 |
done (security fixes 5 and 6; the ZIP writer is archiver's zip layout for stream entries) |
http/web_response.rs |
Fetch new Response(text) defaults |
done (text/plain;charset=UTF-8, headers object semantics) |
media/vips.rs + media/vips/*, media/{image,probe,avif_brand,blurhash,sharp_runtime,transform_cost,transform_queue,avatar,abort}.rs |
sharp 0.34.5 / libvips 8.17, lib/media/{image,probe,avif-brand,sharp-runtime,transform-cost,transform-queue,avatar}.ts, blurhash 2.0.5 |
done (section 11) |
media/{ffmpeg,av_probe,av_cost,av_workspace,av_meta}.rs |
lib/media/* same names |
done |
media/{docx,docx_support,zip_guard}.rs, media/mammoth/ |
lib/media/{docx,docx-support,zip-guard}.ts, mammoth 1.12.1 (+ JSZip 3.10.1, @xmldom/xmldom 0.8.13, lop 0.4.2, dingbat-to-unicode 1.0.1) |
done (section 13) |
naming.rs, naming_words.rs, folders.rs, uploads.rs, shortlinks.rs, quota.rs, format.rs, avatar.rs, avatar_store.rs, background.rs, paste_languages.rs, js_date.rs, link_info/parse.rs, maintenance/trash.rs |
lib/ same names (js_date.rs: V8's new Date(string)) |
done |
data/{uploads,upload_query,folders,duplicates,admin_files}.rs |
lib/data/ same names |
done |
routes/api/{uploads,uploads_check,folders,duplicates,shortlinks,link_info,avatar,background,admin_disk}.rs, routes/{avatar,s_code,s_code_name}.rs |
app/api/{uploads,uploads/check,folders,duplicates,shortlinks,link-info,avatar,background,admin/disk}, app/avatar/[userCode], app/bg/[userCode], app/s/[code], app/s/[code]/[name] |
done (parity: library.py, library_routes.py) |
runtime/limits.rs |
lib/runtime/limits.ts |
done |
turnstile.rs, actions/turnstile.rs |
none (not upstream) | optional Cloudflare Turnstile on the public entry points and its admin card (section 9, item 43) |
downloads.rs, routes/api/downloads.rs, pages/dashboard_downloads.rs |
none (not upstream) | Downloads: the desktop app's builds for macOS and Windows, published with RELEASE_PUBLISH_TOKEN (section 9, item 45) |
media/strip.rs, media/strip/{jpeg,png,webp,gif,heif,raster,av,pdf,office}.rs, data/strip_metadata.rs, routes/api/strip_metadata.rs, actions/strip_metadata.rs |
none (not upstream) | removing metadata from files on upload, one switch per account (section 9, item 47) |
Reference sources (read-only, never executed) of the exact library versions
upstream pins are unpacked in _scratch/srv-skeleton/src/: better-auth 1.6.13,
@better-auth/core 1.6.13, better-call 1.3.5, @better-auth/utils 0.4.1,
drizzle-orm 0.45.2 (integrity checked against pnpm-lock.yaml).
- Models (
db/models.rs): one struct per table, field names = column names, serde camelCase like drizzle's$inferSelect. Each implementsColumns(TABLE,COLUMNSin database order), so queries readformat!("SELECT {} FROM {} WHERE ...", User::COLUMNS, User::TABLE). - Timestamps are
JsDateeverywhere: decodestimestampandtimestamptzas UTC (drizzle stores UTC wall time in the Better Authtimestampcolumns), binds astimestamptz(every connection runs withTimeZone=UTC), serialises astoISOString()(...000Z) in JSON and as{"$date"}in RPC/page payloads.JsDate::now()has JavaScript's millisecond precision. - Tagged JSON (
rpc/tagged.rs): build payloads with typed structs andrpc::ok(&value)/PageData::new(&props, meta), or withmodlight_server::tagged_json!({...}). Plainserde_json::json!serialises its interpolated values immediately, so aJsDateinside it becomes a plain string instead of{"$date"}. - Errors: route handlers return
AppResult<Response>;AppError::messagegives{"message": ...}with a status,AppError::Internal(anyanyhow/sqlxerror via?) logs and answers 500{"message":"Internal server error"}likeserver.tsfail(). - Sessions:
RequestSession(one lookup per request) with the refresh policy of the context the upstream code ran in; see section 8.
- Create
src/routes/<path>.rsnamed after the upstream path (app/api/uploads/route.ts->src/routes/api/uploads.rs,app/[userCode]/[mediaCode]/[action]/route.ts->src/routes/media_action.rs). - Expose
pub fn routes() -> Router<AppState>registering the upstream path in axum syntax with the upstream folder names as parameter names:/{userCode}/{mediaCode}/{action},/api/auth/{*all}(axum keeps one name per position, and the page table already uses these names). - Add one line to
http::router()insrc/http.rs(the central list).
Until step 3, the URL answers 501 naming the upstream file (the placeholder in
pages::fallback), so unported handlers are visible. What the edge already
does for every handler: the next.config.ts headers (config values win over
the same header set by a handler, exactly like Next), OPTIONS -> 204 with
Allow, an unsupported method -> 405 without a body, HEAD from GET, panics ->
500, cookies pushed into ResponseCookies written as Set-Cookie.
Tools for the handler body:
use crate::api_json::{json_private, read_json_object, supplied, JSON_BODY_MAX};
use crate::api_caller::resolve_api_owner;
use crate::auth_helpers::{MaybeUser, RequireUser, Sessions};
async fn patch(State(state): State<AppState>, Sessions(session): Sessions, req: Request) -> AppResult<Response> {
let (parts, body) = req.into_parts();
let (owner, scope) = resolve_api_owner(&state, &session, &parts.headers, TokenCapability::Write)
.await?
.or_respond(Some(PRIVATE_NO_STORE))?; // 401/403 refusals, upstream wording
let input = read_json_object(&parts.headers, body, JSON_BODY_MAX).await?; // 415/413/400 like upstream
...
Ok(json_private(StatusCode::OK, &result))
}Session extractors (auth_helpers.rs) use the route-handler policy (refresh
the row AND re-issue the cookie): Sessions (the lazy RequestSession),
MaybeUser (await getSession()), RequireUser / RequireAdmin (307 to
/login / /dashboard, as redirect() in a route handler). Every surface that
accepts an API token names its capability (api_token::authenticate_api_token,
api_caller::resolve_api_caller/resolve_api_owner); keep the inventory of
tests/unit/token-scope-coverage.test.ts in mind.
The raw upload paths of server.ts (/api/files*, POST /upload,
POST /api/upload) bypass all of the above: replace the placeholder in
http::raw_upload_router with the tus and ShareX routers. They see the request
before normalisation, with no body limit and no timeouts.
- Create
src/actions/<module>.rs(shortlink-unlock.ts->shortlink_unlock.rs). - Write each export as
async fn name(ctx: ActionCtx) -> ActionResultand register it under the upstream names:reg.register("shortlink-unlock", "unlockShortlink", unlock_shortlink); - Add
<module>::register(reg);toactions::register_all.
ActionCtx: args.get::<T>(i) (missing/undefined read as null, so
Option<T> is None; args.raw(i) and args.is_undefined(i) for exact
checks; args.num(i) for NaN/Infinity), form (for X-Modlight-Rpc: form
calls), require_user() / require_admin() / get_session(),
set_cookie(SetCookie) / delete_cookie(name) / request_cookies(),
revalidate_path(..), client_ip(), locale(), state.
Return rpc::ok(&value), rpc::void() (resolves to undefined),
Err(ActionError::Redirect(url)), Err(ActionError::NotFound) or
Err(ActionError::thrown("message")); ? on anyhow/sqlx errors is a
throw. Envelopes, origin check, body limit and cookies are the framework's
job (rpc/mod.rs); tests/rpc_framework.rs has a complete example module.
The upstream unit tests of an action port best against a small *Deps trait
the way actions/tokens.rs does it.
- Create
src/pages/<name>.rs(app/(app)/dashboard/folders/page.tsx->src/pages/dashboard_folders.rs). - Write
pub async fn load(ctx: PageCtx) -> PageResultreturningPageData::new(&props, meta):propsis exactly the object the upstream server component passed to its client component(s),metawhat itsmetadata/generateMetadatareturned (pages/meta.rs:PageMeta::titled,Title::Absolute,Field::Clearfornull, Open Graph, Twitter, alternates,.noindex(), rawextratags). - Register it in
pages::register_allby its app path frompages/table.rs:reg.page("/dashboard/folders", dashboard_folders::load);Layout loaders the same way:reg.layout(Layout::App, layout_app::load);
PageCtx: param("userCode"), search.get("next") / search.to_value()
(Next's searchParams object), require_user() / require_admin() /
get_session(), cookies(), locale(), t("namespace").t("key") /
t_with("key", &[("count", 3.into())]) (ICU like next-intl), client_ip(),
mode (Document or Navigation), state. End with
Err(PageError::redirect(url)), Err(PageError::NotFound) or ? on any error.
Layouts run first, outside in (root, then (app) or (auth)); a layout that
redirects stops the page. A page or layout without a loader answers 501 with
the upstream file name. The response shapes are in pages/mod.rs (module
docs). A page URL loaded directly returns the SPA shell (web/dist/index.html,
or a placeholder until the SPA is built) with the payload inlined; the SPA
fetches GET /_page?path=<encodeURIComponent(location.pathname)>&<the page's own query> on client-side navigation and may send
X-Modlight-Messages: <locale>:<messagesHash> to skip the dictionaries.
Write the work in the module that owns it (upstream path), then add one entry
to jobs::jobs() with its first-run delay and interval. The three upstream
timers are already scheduled: tus sweep (boot, then hourly), trash purge (60 s,
then every 6 hours), mail settings warm-up (once at boot); their work
functions are stubs to fill in: upload::tus::sweep_tus_uploads,
maintenance::trash::purge_expired_trash, mail_settings::warm_mail_settings.
auth/session.rs is Better Auth 1.6.13's getSession with lib/auth.ts's
options; verified against the reference server (section 10). Cookie
better-auth.session_token (__Secure- prefix when APP_URL is https), value
encodeURIComponent(token + "." + base64(HMAC-SHA256(AUTH_SECRET, token))),
7-day expiry, refreshed when older than a day. Where a refresh may happen
depends on the Next context upstream ran in, so each caller picks a
RefreshPolicy:
| Context | Policy | Upstream reason |
|---|---|---|
route handler, server action, /api/auth/get-session |
DbAndCookie |
nextCookies() copies the cookie into the response |
| page loaded as a document | DbOnly |
a Server Component cannot set cookies (the write is swallowed) |
/_page (client navigation) |
Skip |
RSC: 1 requests skip the refresh |
| raw upload paths | DbOnly |
no request scope for cookies() |
A session for a banned account is still returned (Better Auth deletes the
sessions when banning and refuses to create new ones:
check_ban_before_session_create); resolve_api_owner refuses banned owners
itself, as upstream. AuthSession::output() serialises exactly like
auth.api.getSession() (same fields, same order). Accounts created through
Better Auth get theme = "modern" written explicitly (its additionalFields
default), not the column default.
Deliberate, each for a reason:
-
Malformed percent-encoding in a path answers 400 (upstream: 500
text/plain), per "malformed requests answered 400". -
Thrown action errors: in production the envelope carries React's generic production message plus a
digest, because that is what the upstream client saw (Next strips the message); withNODE_ENVunset the real message is sent. PORT.md's{"error":{"message":"<message>"}}shape is kept. -
/_pageenvelopes added beyond PORT.md:{"reload": true}(the URL is a route handler or a static file: load it as a document), 501{"notImplemented": true, "route", "file"}, 500{"error": ...}; anotFoundenvelope also carrieslayouts.rootandmeta; the root layout includesmessagesHash, andmessagesisnullwhen the SPA already has them (X-Modlight-Messages). -
Shortener host pass-through also covers
/_page,/_rpcand/assets/(the SPA's surface, like/_nextupstream). A short code named exactlyassets/...on the shortener host is the one thing shadowed. -
RPC origin check compares the
Originhost withHost(X-Forwarded-Hostonly when TRUST_PROXY is on), like Next's server-action check, which survives TLS termination;X-Modlight-Rpcmust be1orform; unknown action 404; bodies over 1 MB (Next's default server-action limit, never raised upstream) 413. -
Session reads are memoised per request (upstream re-queried; same answer, fewer queries).
-
Cookie serialisation: cookies set in route handlers and actions use Next's format (
Path,Expires,Max-Age, ...,SameSite=lax); Better Auth's own endpoints should useSetCookie::to_better_call_string. Same semantics; attribute order differs between the two exactly as upstream. -
Security fix 12.7 is in
password::client_ip:X-Real-IPonly with TRUST_PROXY on, else the socket peer address. -
Head tags: icons are linked without Next's
?icon.<hash>cache-buster; the not-foundrobotstag comes after the title; Next's framework tags (next-size-adjust, preload links) are gone. Link-preview relevant tags (title, description, Open Graph, Twitter, canonical, oEmbed alternates) are rendered in Next's order; diff them against the reference once the media pages are ported. -
Static files are served without
ETag(withLast-Modified), with the upstreamCache-Controlvalues. -
HTTP: HTTP/1.1 only (as Node), no
X-Powered-By, no NextVary: rsc, noKeep-Alive: timeout=5header. Response compression is Next's (item 42). -
Jobs never overlap a slow previous run (upstream
setIntervalcould). -
servevalidates the configuration before migrating (the upstream container migrated first and refused to start afterwards). -
S3 transport: the Rust SDK gives up connecting after 3.1 s and fails a transfer the server stalls (no bytes for 5 s while it is being read; a reader that pauses is not a stall), where Node would wait.
statstill swallows every error like upstream, so/api/healthreportsstorage: okon an S3 install whose endpoint is down, as upstream did. The runtime image needs a CA bundle for HTTPS endpoints (rustls reads the system store). -
pastes.createPastelanguages:LANG_EXT[lang]upstream also finds what every JavaScript object inherits, so alanguageofconstructoror__proto__(the lower-cased names among them) took a function or an object as the extension, wrote the bytes under<code>.function Object() { [native code] }/<code>.[object Object]and then threw (verified on a private reference). Here only the 28 listed languages map to an extension; any other name istxt, and the paste is created. -
uploads.emptyTrashwith a storage driver that cannot be built (an S3 install missing a variable) throws even for an empty trash; upstream'sgetStorage()only ran once a batch had rows. Either way the install is broken; the answer differs only for an empty trash. -
ShareX (security fix 2): the header token is authenticated before a byte of the body is read; a request that names no header token may still carry a legacy
tokenfield, but its body is capped at 8 MiB (LEGACY_UPLOAD_MAX_SIZEoverrides; 413 with the upstream wording); quota and free space are re-checked every 8 MiB while a chunked body streams (507 "User disk quota exceeded." / the low-disk message), instead of after the whole file landed on disk. -
tus wire details: the status line carries a reason phrase, empty bodies go out with
Content-Length: 0(Node chunked them), and there is noConnection: keep-aliveheader. Kept from upstream: an upload finished at creation (zero length, or creation-with-upload of the whole body) is finalized, and its URL answers 404 afterwards. -
DELETE /api/uploadsreads at most 1 MiB of body (upstream:req.json()without a limit); 500 ids need about 20 KB. -
libvips: only the loaders sharp-libvips 1.2.4 ships may run (Homebrew's ImageMagick loader would accept BMP and the
avisAVIF production refuses); HEVC pixels are refused (HEVC_DECODE = false) as production cannot decode them. Built against libvips 8.18, image metadata, thumbnails and avatars are byte-identical to production's 8.17.3 except CMYK input (mean 3/255 colour shift); on Debian 13's 8.16.1, 2 of 20 blurhashes differ by a digit. The runtime image should carry libvips 8.17 or newer. -
ffprobe: durations come from whatever ffprobe the host has. The reference image (Debian 12, ffmpeg 5.1) counts an MP3's encoder padding (1.541224 s) where ffmpeg 9 does not (1.5 s); ship the reference's ffmpeg major for identical numbers.
-
GET /api/link-infosends the handler's ownVary: Authorization, Cookie(Next replaces it with itsrsc, ...list on every route handler; see 11). -
Kept on purpose (upstream behaviour worth knowing):
listFoldersalways reportscount: 0(the subquery comparesuploadswith itself). (This item also listed the two behaviours items 38 and 39 now fix.) -
A thrown error in a route handler answers like Next:
500, empty body, noContent-Type(error::route_handler_error_response; seen on the reference on/api/duplicateswith an out-of-range count and on/s/%00). Onlyserver.ts' raw upload paths answer itsfail()JSON{"message":"Internal server error"}(also for a panic there). Decided at integration (2026-10-20); before, every 500 was the JSON one. -
read_json_objectparses withjs_json::parse(V8'sJSON.parse: no nesting limit, lone surrogate escapes,1e400= Infinity), as upstreamreadJsonObject(decided at integration with the reference's answers:maxViews: 1e400-> 400 "View limit must be a number",expiresInDays: 1e400-> a link without expiry, a field nested 3000 levels deep ignored like any unknown field). The routes still read serde_json values (api_json::js_object_to_map): a non-finite number becomes the RPC's tagged{"$num":"Infinity"}(api_json::is_js_number,rpc::tagged::raw_numread it), so a client that sends that literal object as a field value is read as Infinity; containers deeper than 64 levels are kept empty (no route reads that deep; serde_json's drop is recursive). Lone surrogates become U+FFFD as before. -
Security fix 1, view limits (
media::serve::ViewBudget, the exact rule is in its doc comment). For a file withmax_views, and anybody but its owner or an administrator: a request upstream counted is counted (unchanged); a request upstream exempted (aRangenot starting at 0,Sec-Fetch-Dest: image|video,w/h/motion, a bot) is served without consuming only with the VIEW GRANT of a counted view, else it consumes one. The grant is a signed cookieml_view_<uploadId>=<exp>.<HMAC(AUTH_SECRET, "view|<id>|<exp>")>(6 hours, httpOnly,SameSite=Lax,Securein production), minted by every counted view of a capped file: the preview page's (document and/_page), and/raw//download's. It also lets its holder through when that view was the LAST one (views == max_views), which repairs two upstream failures: the page's own<img>/<video>on the last permitted view, and an encrypted one-time file whose second ranged read 404ed (verified on the reference). It applies to everything that carries the file's content: the stored bytes (whole or ranged) and pictures made of them (resized stills, moving tiles, timeline sheets); refusals and the cue track spend nothing. Also: a preview bot on the page of a capped file is not counted (as upstream) and gets no content (no inlined text, no render ticket), the card of a capped text file carries no excerpt, and a capped item travels in an album ZIP only by spending a view (as a download). Costs: a cross-host page (owner's preferred domain different from the host the page was opened on) cannot carry the cookie, so its<img>pays a view; public gallery tiles of a capped file pay a view each; resuming a capped download withRangepays a view. -
Security fix 5: the delete-by-link token is compared in constant time.
-
Security fix 6:
/oembeddescribes a published file with a view cap as{version, title, provider_name, provider_url, cache_age, type: "link"}: no raw URL, no dimensions, no player HTML (asembed.tswithholdsog:image/og:video). -
Album ZIP: the reference answers every
/a/<tag>/<slug>/zipwith 500 (import archiver from "archiver"against archiver 8, which has only named exports:TypeError: (0, a.default) is not a function). The port serves the archive the route was written for, in archiver's zip layout for stream entries (src/archiver.rs); deflate bytes come fromflate2, not Node's zlib, so archives are equivalent (same entries, order, names, flags, attributes, CRCs and contents), not byte-identical. -
Metadata placement: Next 16 streams a slow
generateMetadatainto the BODY for browsers (it blocks the head only for the bots it recognises); the port always puts it in<head>. Same tags, same order. -
Plain-object lookups:
RENDERABLE,EXT_CATEGORYandLONG_TYPESmatch only their listed keys (upstream foundconstructorand friends on the prototype, so a stored typeconstructorwas offered the render toggle and an extensionconstructora card URL spelled from a function). -
Derived responses (thumbnails, motion tiles) carry
Content-Length; Next sent them chunked. Same bytes. -
ffmpeg version (media processing): the argument vectors are upstream's and identical to the reference's (verified,
tests/parity_media), but they were written for ffmpeg 5.1 (Debian 12, the production image). On ffmpeg 7+ (Homebrew's 9 on the Mac):-fflags +shortestis no longer a format flag, so audio -> MP4 fails; the mjpeg encoder refuses the limited-range YUV the timeline sheet hands it, sotimeline.jpgfails; a clip offset past the end (?t=9999) fails writing the GIF trailer where 5.1 wrote a one-frame GIF; ffprobe measures VFR files, AVIF sequences and MP3 padding differently, so a few plans differ (item 21). Homebrew's build also has no libvorbis and no libwebp (ogg, and ffmpeg's animated WebP of an AVIF sequence, fail). Run the server with the reference's ffmpeg major (the Docker image on Debian 12) for identical behaviour; with it, every status, header, argument vector and deterministic output matched. -
Text on the embed card: libvips' Pango is put on its fontconfig backend (
PANGOCAIRO_BACKEND=fc, set once at libvips start unless the operator set it). It is the only backend Linux has; on macOS Pango would otherwise pick CoreText, which never sees thefontfileand draws a system face. With it every card is pixel-identical to the reference. -
PDF output (
media/pdf_lib.rs): pdf-lib's file structure object for object; the Flate streams are compressed by miniz rather than pako (they inflate to the same bytes). Like pdf-lib, the info dates and the XObject name (Image-+ random) change on every call, so no two PDFs are equal byte for byte on either server. -
HEVC_DECODE = false, decided exactly as upstream decides it: by the libheif sharp-libvips 1.2.4 ships (1.20.2 with aom only, no libde265, checked in the reference container:metadata()reads a HEIC's header ascompression: hevc,toBuffer()fails with "Support for this compression format has not been built in"). So HEIC gets no blurhash, no thumbnail of its own, no conversion target (convert-targets offers none) and no motion tile (not-animated), on both servers. -
Lookups by a caller's string in
media/source_preview.rs(paste language -> extension) use exact keys, as item 31 does for the card table. -
Security fix: the tus URL of a finished upload (integration, 2026-10-20). On S3 a finished tus upload IS the stored object (finalize adopts the key in place) and its
<id>.inforecord stayed, so upstream'sGET /api/files/<id>streamed the file to anybody holding the tus URL whatever its later privacy, password or view limit, andDELETE(tus termination, no authentication upstream) deleted the object from under its media row. NowGETnever serves a finished upload (404, as an unknown id; the optionserve_finished_uploads),DELETEof a finished upload answers 400 "Cannot terminate an already completed upload" (@tus/server's owndisableTerminationForFinishedUploads), and once the upload is finalized on S3 its tus record (.info,.part) is deleted, so the URL answers 404 to HEAD/PATCH/DELETE exactly as on the local driver. Unfinished uploads resume as before. Tests:upload::tus_server::serverfinished_uploads_are_not_served_or_terminated,tests/library_s3_e2e.rs(MinIO). -
Replacing or removing an avatar or a wallpaper deletes the previous object (integration, 2026-10-20). Upstream's
UPDATE ... RETURNINGread the NEW key, sodiscard(old, key)never deleted anything and every replacement left an object behind for good. The previous key is now read under the row lock of the same statement (update ... from (select ... for update) prev returning prev.<column>), and its object is deleted after the update; only this account's own keys (avatars/<userId>-...,backgrounds/<userId>-...) are ever deleted, becauseuser.imagecan also be written through Better Auth'supdate-user. Test:tests/library_routes_db.rsreplacing_avatar_or_wallpaper_deletes_the_previous_object. -
PORT.md section 12, fixes 1 to 7, re-checked in the integrated build (2026-10-20), each with a test that fails on upstream's behaviour: 1 view limits: items 26,
tests/media_serving_db.rsfix1_every_bypass_of_the_audit_is_closed,fix1_the_legitimate_flows_ride_on_the_view_grant,fix1_preview_bots_get_no_content_of_a_capped_file(+ paritymedia_serve.py --only fix1); 2 ShareX: item 17,tests/security_fixes_db.rsfix2_sharex_authenticates_before_the_body_and_caps_what_streams(an unknown header token is refused with the body unread; no token: 413 after 8 MiB; a chunked body over the quota: 507 while streaming, nothing kept); 3 first administrator:auth::adapter(pg_advisory_xact_lock),tests/security_fixes_db.rsfix3_concurrent_first_sign_ups_make_exactly_one_admin(12 concurrent sign-ups on an empty install, one admin); 4POST /api/releases/app:tests/releases.rspublishing_authenticates_before_it_reads_the_bodyand the 415 / 413 checks inrelease_channel_against_a_database; 5 constant-time delete token: item 27,tests/media_serving_db.rsfix5_delete_by_token; 6/oembedof a capped file: item 28,tests/media_serving_db.rsfix6_oembed_describes_a_capped_file_without_its_bytes; 7X-Real-IPonly behindTRUST_PROXY: item 8,password::tests::honours_x_real_ip_only_behind_a_trusted_proxy,auth::ip::tests::real_ip_only_behind_a_trusted_proxy. -
Document redirects and the dashboard boundary (integration, 2026-10-20, decided with the reference's answers, CONTRACT 12.4). A page's or layout's
redirect()answers307+Locationwith a whole HTML document as the body, as Next does (Next sends its__next_error__shell; here the SPA shell with the redirect payload inlined, which the SPA follows if a client ever shows it). Underapp/(app)/dashboard/, whoseloading.tsxmakes Next flush the page before it runs, a page's ownredirect()ornotFound()answers200like the reference: the page's own title (Admin - Modlight,User files - Modlight, else the app name), anotFound()withrobots: noindex, a redirect with Next's<meta id="__next-page-redirect" http-equiv="refresh" content="1;url=...">inside<noscript>(the SPA follows the inlined redirect payload itself; a live refresh would load the target a second time a second later). Layout redirects (the(app)layout'srequireUser()) stay307, as on the reference. The proxy gate and the trailing-slash308keep their text bodies (already as upstream). Malformed percent-escapes keep item 1 (400). -
Response compression (integration, 2026-10-20): Next's
compress: true(compression 1.7.4), checked response by response on the reference: documents,/_pageand/_rpc(the RSC payloads' place) and static files (/assets/*,public/, from 1 KiB) are gzip- or deflate-encoded for a client that accepts it, withVary: Accept-Encoding; route handlers (/api/*, media,/oembed,/embed,/icon.svg), the raw upload paths and every206never are (http::compression). Bytes differ from Node's zlib (same content). -
Intended difference: optional Cloudflare Turnstile (2026-10-21, a new feature the owner asked for on top of the 1:1 port;
src/turnstile.rs,src/actions/turnstile.rs, the SPA'ssrc/turnstile/). With no complete configuration nothing changes: no check, no outbound call, the CSP and every payload as before exceptlayouts.root.config.turnstileSiteKey: null, theturnstiledictionary inlayouts.root.messagesand the admin page'sturnstileprop.- Configuration: the admin panel's "Cloudflare Turnstile" card (Admin ->
Settings: switch, site key, write-only secret key, "Test") stores
turnstile_enabled("on"/"off"),turnstile_site_keyandturnstile_secret_enc(secret box, purposeturnstile). Env fallbackTURNSTILE_SITE_KEY+TURNSTILE_SECRET_KEY(both, trimmed; one alone warns once and stays off). Block precedence as for SMTP: once the switch row holds a value the panel decides everything (an environment secret never fills a panel block), "use the environment" sets the three rows to NULL. On = switch on + site key + a secret that decrypts. A 30 s snapshot per process (AppState::turnstile), re-read by the root layout and the admin page, warmed at boot; a database failure keeps the last known configuration (or the environment's) for one period. - Where:
POST /api/auth/sign-in/email,/sign-up/email,/request-password-reset(any POST whose path below/api/authcontains one of them, better-auth's captcha plugin'sdefaultEndpointsandincludesmatch), inauth::api::handleafter the rate limiter and before the body is read; and the actionslogin-identifier.signInWithIdentifier(after its own throttle) andinvite.acceptInvitation(after its throttle, before the invitation is read). Server-side calls (auth::server::call_endpoint) are not checked, so the sign-in those actions perform does not spend a second siteverify on a single-use token. Not: 2FA, reset-password itself, verification links, API tokens. - Token and refusals: header
x-captcha-response(the plugin's), so no action signature or body changes. Better Auth routes answer the plugin's codes as better-call APIErrors: missing400 MISSING_RESPONSE, refused403 VERIFICATION_FAILED; Cloudflare unreachable503 SERVICE_UNAVAILABLE"CAPTCHA service unavailable" (the plugin answers 500UNKNOWN_ERROR"Something went wrong" and logs that code). Actions answer in their own shape:{status: "captcha" | "captcha-unavailable"},{ok: false, reason: "captcha" | "captcha-unavailable"}. The SPA showsturnstile.widget.*texts (en, pl) for each. - Verification:
POST https://challenges.cloudflare.com/turnstile/v0/siteverify, form-encodedsecret,response,remoteip(the plugin posts JSON; Cloudflare takes both).remoteip=password::client_ip(the action throttles' rule:X-Real-IPonly behind TRUST_PROXY, else the socket peer) when it is an IP, sent unnormalised (Better Auth's limiter key cuts IPv6 to /64, which is no address). hyper-util client with hyper-rustls (ring, Mozilla roots like the mail transport), 10 s for the whole exchange, answer capped at 64 KiB; only HTTP 200 with"success": truepasses; anything else unanswered is "unavailable" and refuses (fails closed). A token over 2048 characters is refused without a call, an empty one is "missing" without a call. Logged: Cloudflare's error codes and transport errors, never the secret or the token. - Order: the plugin runs before the rate limiter; here after it, so a flood of bad tokens is throttled before it reaches Cloudflare, and a refusal is recorded against the bucket like any other answer.
- Saving: a save that turns the check on first sends the secret with
Cloudflare's dummy token (
XXXX.DUMMY.TOKEN.XXXX):invalid-input-secret/missing-input-secretrefuses the save (secret-rejected), no answer refuses it (unreachable), because a wrong secret would refuse every sign-in, the administrator's included. A changed site key forgets the stored secret. The "Test" button does the same probe (also reporting a test secret that accepts everyone); both share a throttle of 10 per 10 minutes per administrator. Audit entriesturnstile.settings_changed(row names,secretChanged) andturnstile.settings_cleared. - CSP: while on,
appCspand the/authorizepolicy gethttps://challenges.cloudflare.comappended toscript-srcandframe-src(http/headers.rs, both variants built at startup, the snapshot picks one per response), on every page, because the SPA reaches/loginfrom any page without a new document.connect-srcis not needed (the widget talks to Cloudflare from its own frame; checked in Chrome with the test keys: no violation). - SPA and i18n: the four auth forms became ports in
web/src/overrides/components/auth/(CONTRACT.md 11); the strings aremessages/<locale>/turnstile.json, an 18th client namespace (i18n::NAMESPACES), so upstream's files stay verbatim. - Locked out (a widget whose host names do not match, a secret revoked
at Cloudflare):
UPDATE settings SET value = 'off' WHERE key = 'turnstile_enabled';takes effect within 30 s (or restart); setting the threeturnstile_*rows to NULL hands control back to the environment. - Tests:
turnstile::tests(precedence, codes, CSP inhttp::headers),tests/turnstile.rs(a siteverify stub on 127.0.0.1: pass, refused, missing, oversized, network error, timeout, off; every entry point),tests/turnstile_db.rs(the card, the secret box, the audit, block precedence, real sign-ups and sign-ins),web/tests/turnstile.test.tsx.
- Configuration: the admin panel's "Cloudflare Turnstile" card (Admin ->
Settings: switch, site key, write-only secret key, "Test") stores
-
Intended addition: desktop sign-in pickup (2026-10-24). The loopback handoff of RFC 8252 depends on the BROWSER posting the token from an https page to
http://127.0.0.1:<port>/, and browsers refuse: Safari blocks it (mixed content / loopback), Chrome's Local Network Access drops it once its permission was dismissed. The app's listener is fine; the bytes never come. So the app also collects the outcome from the server (PKCE, RFC 7636, cut down), and whichever answer arrives first wins.- Flow: the app makes
verifier(32 random bytes, base64url) and sendschallenge = base64url(SHA-256(verifier))(43 characters) in/authorize?port=&name=&state=&challenge=.parse_authorize_request_with_challengevalidates it (malformed ->invalid-challenge, an invalid request),authorize_pathcarries it through/login?next=. The page loader registers{state -> challenge, user id}as pending (the only thing it writes;register_render, rate-limited per account and per address);approveDesktopClientwith a challenge stores the token, the newdenyDesktopClientstoresaccess_denied(actions/authorize.rs), both ONLY on a live entry that the same account registered with the same challenge (security review 2026-10-24, P1: the challenge is in the URL, so it proves nothing about who decides). Otherwise nothing is stored and the approval answerspickup: false: the page then uses the loopback POST and the manual key, which reach only the approver's own machine. The app pollsPOST /api/authorize/pickup {state, verifier}every 2 s. - Confirmation code (review 2026-10-24, P2):
confirmation_code(challenge)= first 6 base32 characters ofSHA-256("modlight-confirm:" || challenge), shown asXXX-XXXby the app while it waits and by the prompt next to Approve ("Approve only if this code matches the one in your Modlight app"); the page props carry it asconfirmCode. A phishing link carries somebody else's challenge, so its code is one no app of the victim's shows. The app computes the same (modlight-apisignin::confirmation_code, same test vector554-LIO). - Store (
authorize/pickup.rs): in memory,state -> {challenge, user id, outcome, expires}, 5 minutes, at most 4096 entries (swept every 30 s and when full). The first account and challenge seen for a state bind it; another account or challenge never replaces a live entry. One account holds at most 5 live entries (its own oldest pending one makes room); when the map is full the oldest PENDING entry goes, and a decided entry (a token waiting for its app) is never evicted before it expires (review P3). Renders: 30 per account, 120 per address per 10 minutes. Handed out once, then deleted. A restart loses pending pickups (loopback and manual key remain). - Route (
routes/api/authorize_pickup.rs):read_json_object(1 KiB), constant-time compare of the hashed verifier;200 {"token"}or200 {"error":"access_denied"}once,202 {"status":"pending"},404 {"error":"not_found"}for unknown, expired and mismatched alike,400 {"error":"invalid_request"},429 {"error":"rate_limited"}+Retry-After(120 per minute per client IP).Cache-Control: no-store. No session or API key: the verifier is the credential. Tokens and verifiers are never logged. - SPA:
components/authorize/authorize-prompt.tsxis a port (web/src/overrides/), callingweb/src/rpc/authorize-pickup.ts; with a challenge the page shows the confirmation code next to Approve, says the key will be handed to the Modlight app that shows that code (wherever it runs), after approving says the app picks the key up by itself, still tries the loopback POST, and keeps the key under "App still not signed in?" to copy; withpickup: falseit falls back to upstream's flow. Strings:messages/<locale>/authorizePickup.json, a 19th client namespace. Without a challenge (older apps) everything is as before. - Tests:
authorize::pickup::tests,routes::api::authorize_pickup::tests,actions::authorize::tests(challenge, deny),authorize::loopback(challenge parsing and the login round trip),web/tests/authorize-pickup.test.tsx, apptests/signin_pickup_fake.rsand the opt-in real-servertests/signin_pickup.rs(MODLIGHT_PICKUP_E2E);tests/secreview_pickup.rs(the review's attacks, asserting that each fails).
- Flow: the app makes
-
Intended addition: Downloads (PL "Do pobrania", 2026-10-27). The server keeps the desktop app's builds for macOS and Windows; signed-in accounts download them, build machines publish them with a token, the desktop app reads the list for updates. Nothing upstream changes: new routes, a new table, a new page.
- Table
releases(migration 0015, drizzle journal and snapshot like the others):idtext pk (32 hex),platform(macos|windows, CHECK),arch(arm64|x64, CHECK),version,build(may be empty),built,published(timestamptz),sizebigint,sha256,filename,uploader,note; index (platform,arch,published). Modeldb::models::Release. - Files: through the storage layer at
releases/<id>/<filename>(local and S3 alike:ingest_local_file,read_streamwith ranges,delete_prefix). The storage scan of the admin page skipsreleases/likederived/,avatars/andbackgrounds/. Chunks of an unfinished upload are local files underUPLOAD_TMP_PATH/releases/<upload>/(upload.json,chunk-<n>);finishjoins them in order while hashing, checks the declared size and SHA-256 (else 400 and the folder is removed), hands the joined file to the driver, inserts the row and drops every build of the same platform and architecture beyond the newest 5 in one transaction (advisory lock per platform+arch), then deletes their files. A finished upload keeps adonemarker for a day so a repeatedfinishanswers the same build; an upload untouched for a day is swept at the next start. Onefinishper upload at a time (409busy).publishedis monotonic within the process. - Routes (
routes/api/downloads.rs; errors JSON{"error","code"},Cache-Control: private, no-store):GET /api/downloadsandGET /downloads/{id}/{filename}for any signed-in account, by session or by an API token of any scope (capabilityidentify: the list reveals nothing about an account, and the desktop app must find its updates whatever its token's scope); the file withContent-Length,Content-Disposition: attachment,ETag= the quoted SHA-256,If-None-Match(304),Range/If-Range(206).POST /api/downloads/upload(read_json_objectrules, 16 KiB),PUT .../upload/{upload}/{n}(chunks of at most 50 MiB, 413 beyond, or beyond the declared size),POST .../upload/{upload}/finishfor a publisher:RELEASE_PUBLISH_TOKENas a bearer token (x-api-keytoo; SHA-256 of both sides compared in constant time,config::PublishTokenkeeps only the digest), an administrator's session, or an administrator'sfullAPI token (capabilityadmin, as/api/releases/app).DELETE /api/downloads/{id}for administrators only. The publish token opens nothing else (401 elsewhere). The credential is checked before any body is read; a token decides alone when one is offered (a wrong one is never rescued by a cookie). 401unauthorized/account_disabled, 403forbidden/insufficient_scope/invalid_scope. - Shadowing:
/downloads/{id}/{filename}has the shape of upstream's/{userCode}/{mediaCode}/{action}; a path that names no build (wrong id or file name) is handed to the media action unchanged, so an account taggeddownloadsfrom before keeps its links. The tagsdownloadsandreleasesare added toRESERVED_TAGS(a new account cannot take them;releasesbecause user files live under<tag>/in storage). - Audit (
audit::record_audit_by, a NULLuser_idfor the token):downloads.published,downloads.pruned,downloads.deleted,downloads.rejected, withby/byEmail/viaand the build. - Page
/dashboard/downloads(pages/dashboard_downloads.rs, a 28th page inpages/table.rs; CONTRACT.md 6.28): props are theGET /api/downloadsanswer. The SPA (web/src/downloads/) shows a card per platform, newest build on top, older folded; administrators also delete and see how to publish (withwindow.location.origin). Strings:messages/<locale>/downloads.json, a 20th client namespace, andnav.downloads. - Configuration:
RELEASE_PUBLISH_TOKEN(optional;setup-local.shgenerates it; a value under 24 characters warns at start). - Tests:
downloads::tests,routes::api::downloads::tests,config::tests(publish_token_*),tests/downloads.rs(auth rules, chunked publish with a full 50 MiB chunk, wrong SHA-256, wrong size, missing chunk, keep-5, delete, ranges, the page, the media-action fallback),web/tests/downloads.test.tsx.
- Table
-
Intended addition: per-account file encryption (2026-10-27, the owner's request; the design, the threat model, the API for bearer clients and the limits are in
docs/ENCRYPTION.md). An account without a vault sees nothing new: no query beyond a subselect inlookup_media, no cookie, no header, the same payloads (the new props are absent), except the Settings page'sencryptionprop and the emptyuploads.encryptioncolumn.- Code:
src/vault/(cipher, keys, store, access, view, write, gate, unlock, status, pass),src/actions/encryption.rs(RPC moduleencryption),src/routes/api/encryption.rs(/api/encryption,/unlock,/lock), migration 0016 (uploads.encryption,file_vaults,vault_grants,vault_cleanup), the SPA'ssrc/encryption/. - Where it hooks in: the writers (
upload::finalize,pastes.createPaste,media::convert_in_place) encrypt with the public key; the readers get a decrypting storage view only behind the gate (vault::access::open_filesinroutes/media_action.rsand itsrender,pages/media.rs,routes/album_zip.rs,actions/avatar.rs,convert_in_place); the galleries (pages/public_{profile,album}.rs);lookup_mediamarks every file of a gated account (Upload::vault_gated), which makescache_headers_forprivate,may_cache_derivedfalse, and the embed plan,/oembedand/api/link-infotreat it as password-protected;http.rslets the unlock form keep its own CSP on a raw URL (vault::gate::GateDocumentCsp);jobs::startrunsvault::pass::worker. - Answers: locked content is
401for a browser (an HTML form for a document, text otherwise) and423 LockedJSON{"error": "files_locked"}for a bearer client (401 would sign the desktop app out). - Tests:
vault::*::tests(crypto round trips, Range across chunk borders, tampering, wrong passwords, grants, gate escaping, rate-limit buckets),tests/vault_db.rs(bearer uploads, every public surface, grants, the pass with an interruption and a restart, rate limits),web/tests/encryption.test.tsx.
- Code:
-
Intended addition: removing metadata on upload (2026-11-07, the owner's request). One switch per account, "Remove metadata from files on upload", off by default: an account that never turns it on sees no change (its uploads are stored byte for byte, and no new query runs: the switch is a column of the owner row finalize already holds).
- Columns (migration 0017, drizzle journal and snapshot like the others):
user.strip_metadata boolean NOT NULL DEFAULT false(the switch) anduploads.metadata_kept varchar(16)(NULL, or why a file uploaded with the switch on kept its metadata). ModelsUser::strip_metadataandUpload::metadata_kept, both#[sqlx(default)]and never serialised. The TypeScript server ignores both. - Where it runs:
upload::finalize, right after the quota reservation and before anything describes the file: the type sniffing, the dimensions and blurhash, the ffprobe metadata, the checksum, the vault's encryption and the row all see the cleaned copy (media::strip::strip_upload, which works on a copy in a scratch folder ofUPLOAD_TMP_PATH, removed with it, and never changes the upload's own file). The reservation is moved to the cleaned size (a larger cleaned copy that does not fit the quota is not used). The cleaned copy replaces the temp file (removed after the commit, as the ingest would have moved it); an S3 tus upload's object is copied out, cleaned, stored under a new key and deleted after the commit, as the vault does with its plaintext. The answers of every upload path keep their shape. - Writers: every upload goes through finalize, so the switch covers the
panel (tus
/api/files, session), the desktop app (tus with its token, and its plainPOST /upload), ShareX and scripts (POST /upload,POST /api/upload). It does not apply to the other writers, on purpose: pastes (pastes.createPaste) store text typed in the panel, with no file metadata to remove;convertin place or as a copy stores what libvips (keep: none) or ffmpeg (-map_metadata -1) produced, which carries none of the source's metadata (in place,metadata_keptis cleared); avatars and wallpapers are re-encoded by libvips; Downloads' builds are not user files; the vault's pass rewrites ciphertext of files already stored. Files already uploaded are never touched. - Duplicates: a ShareX answer's
duplicate_ofis looked up after the insert with the cleaned checksum, leaving the new row out (duplicate_lookup::first_match_except); a tus upload, which upstream never hashes, gets the cleaned file's checksum on its row. Cleaning is deterministic (no time stamps,-fflags +bitexact), so the same photo uploaded twice is two copies of the same bytes. - What each format loses and keeps (told by the bytes, never by the
name; the picture is never decoded where the format allows it):
- JPEG (segments): EXIF and XMP (APP1), IPTC/Photoshop (APP13), MPF and every other APPn, comments, a JFIF thumbnail, and whatever follows the end of the image (MPF's second pictures, a motion photo's video) go; JFIF APP0, Adobe APP14, the ICC profile chunks, the tables and the entropy-coded data stay byte for byte. A rotated photo keeps an EXIF holding only Orientation.
- PNG and APNG (chunks):
tEXt,zTXt,iTXt,tIME,eXIf, every unknown ancillary chunk and anything afterIENDgo;IHDR PLTE IDAT IEND tRNS gAMA cHRM sRGB iCCP sBIT pHYs bKGD cICP mDCV cLLI acTL fcTL fdATstay with their CRCs. An unknown critical chunk makes the file unknown (kept). - WebP (RIFF):
EXIF,XMPand unknown chunks go, the VP8X flags and the RIFF size are put right;VP8 VP8L VP8X ALPH ANIM ANMF ICCPstay. - GIF (blocks): comment, unknown and application extensions other than NETSCAPE2.0/ANIMEXTS1.0 go, and anything after the trailer; frames, delays, colour tables and the loop stay.
- HEIC and AVIF (HEIF): the EXIF and XMP items are overwritten in place
at the same length (EXIF by a minimal block holding only Orientation,
or none; XMP by an empty packet; then zeros). No box moves, so the coded
picture,
irot/imir,colr, depth and gain maps are the same bytes, and no codec is needed (the image's libvips cannot decode HEVC, so a re-encode could never clean an iPhone photo). An item may only be overwritten insidemdatoridatand away from every other item's bytes. An image sequence also loses its movie times and itsudtaandmeta(renamedfree, zeroed). An AVIF this cannot follow is saved again by libvips (quality 90, 4:4:4,keep=icc, orientation applied): the only lossy step of the feature, for unusual files. - TIFF: libvips, losslessly (the same pixels in their own depth;
keep=icc; the Orientation tag carried; the compression kept, or deflate for a lossy one). Camera raws (TIFF outside) are kept as they came (format). - Audio and video (MP4, MOV, M4V, 3GP, M4A, MKV, WebM, AVI, MP3, FLAC, Ogg,
Opus, WAV): ffmpeg 5.1 in the image,
-map 0 -map -0:d? -map_metadata -1 -map_chapters -1 -c copy -fflags +bitexact -strict unofficial, the demuxer forced from the allowlist,+faststartand the major brand kept on the mov family. Every global and stream tag (GPS, dates, device, title, artist) and every chapter goes, and so do data streams (phone telemetry, a GoPro GPS track, an MP4's chapter track); every audio and picture stream is copied packet for packet, cover art included, and the rotation of a phone video (display matrix side data) survives. The result is checked with ffprobe before it is used: the same audio and picture streams, the same rotation, no tag naming a place, a time or a person; otherwise it is thrown away. A bare MPEG frame sync that ffprobe does not read as media is left alone. Other containers (ASF, FLV, MPEG-TS, AAC, APE, WavPack) are left alone. - PDF (lopdf 0.47): the trailer's Info dictionary and every
Metadata(XMP) stream go, and so do older revisions of an incrementally saved file (unreferenced objects are pruned); pages, fonts and pictures are the same objects. Kept as they came: password-protected files (protected), signed ones (signed: a rewrite would break the signature), files whose cross-reference table had to be rebuilt (damaged). A PDF/A file loses its conformance mark with its XMP. Pictures inside a PDF keep any EXIF they carry. - Office Open XML (docx, xlsx, pptx and relatives with a main part) and
OpenDocument (odt, ods, odp...):
docProps/core.xmlanddocProps/app.xmlbecome empty property sets,docProps/custom.xmlgoes with its content type and relationship,meta.xmlbecomes an emptyoffice:meta; every other entry is copied byte for byte, local header included, so ODF's first storedmimetypestays first. Signed packages and password-protected ODF are kept as they came. Pictures inside keep any EXIF they carry; comments and tracked changes inside the document body are content, not metadata, and stay. - Anything else (text, archives, executables, BMP, SVG, PSD...): untouched.
- Fails open: a file that cannot be cleaned is stored exactly as it came;
the upload never fails because of this step. The reason is logged (with
the code, not the name) and stored in
uploads.metadata_kept:damaged(a broken or unusual layout),format(a camera raw, an AVIF sequence libvips refuses),protected,signed,too-large(over a limit below, a deadline, or a cleaned copy that does not fit the quota),busy(the media queue was full or the disk had no room for a second copy),tool(ffmpeg or libvips missing or failing). The Settings card lists those files (live ones, newest first, 20 of them and the total). - Limits: the byte-level rewriters (JPEG, PNG, WebP, GIF, HEIF, the zip
of a document) stream file to file in constant memory, whatever the size;
ffmpeg works on files (
AV_WEIGHTon the media limiter, the 45 s copy deadline, 10 s probes, 2 GiB at most: beyond,too-largeat once); libvips (TIFF, the AVIF fallback) and lopdf (64 MiB at most, decompressed object streams bounded at 64 MiB) hold a conversion's slot with a 30 s deadline; an S3 object is staged within 45 s; a zip is only read when the end of its directory says it may be a document. The parsers are fuzzed (media::strip::fuzz: every truncation and 3000 mutations per format must end inOkorErr, read nothing past the end, not loop, and be final: cleaning a result again changes nothing). - Switch: the Settings page (
stripMetadataprop, the card in "Appearance & security"), the RPC modulestrip-metadata(loadStripMetadata,setStripMetadata), andGET/PUT /api/strip-metadatafor the desktop app and scripts (CONTRACT.md section 11; capabilitiesreadandwrite). Strings:messages/<locale>/stripMetadata.json, a 21st client namespace. - Crate:
lopdf0.47 (MIT), with no default features; it bringsaes/cbc/ecb/cipher0.5 (MIT or Apache-2.0, for encrypted PDFs it only detects here),rand0.10,rangemap,brotli-decompressor(BSD-3-Clause or MIT),weezl,miniz_oxide0.9 (flate2 still uses zlib-rs:any_zlibwins over its default backend). All compatible with AGPL-3.0. No new system package: libvips (tiffsave,heifsavewith AV1,autorot) and ffmpeg/ffprobe 5.1 are already in the image. - Tests:
media::strip::*::tests(each rewriter, the HEIF item confinement,tiff_look, the ffmpeg arguments and checks),media::strip::fuzz,tests/media_strip.rs(real files: JPEG with a real ICC profile and rotated EXIF, PNG, still and animated WebP, animated GIF, AVIF and HEIC from libvips, TIFF from ImageMagick, twelve audio and video containers made with ffmpeg and compared packet for packet, cover art, PDF; skipped when a tool is missing),tests/strip_db.rs(the switch and token scopes, ShareX, tus from the panel and from the app, checksums and duplicates of the cleaned file, a broken file kept and listed, an encrypted account, a docx, the switch off storing byte for byte),routes::api::strip_metadata::tests,web/tests/strip-metadata.test.tsx.
- Columns (migration 0017, drizzle journal and snapshot like the others):
- Fresh dev database ->
modlight-server migrate->pg_dump --schema-onlyidentical to the reference database (only pg_dump's random\restrictkey differs);drizzle.__drizzle_migrationsrows (id, hash, created_at) and thesettingsseed identical. - Reference dump restored into the dev server -> the migrator applies nothing.
- A session created by signing in on the reference server (8420) is accepted
by the Rust reader on a copy of that database (same AUTH_SECRET), its
getSessionoutput is identical field by field and in order, the cookie re-issued on refresh is byte-identical to the one the reference issued, and the three refresh policies write what the table in section 8 says. A cookie with a tampered signature is refused. tests/db_compat.rschecks every model againstinformation_schemacolumn by column.
To repeat the session check: sign in on 8420 (QA credentials in
_wzorzec/qa-accounts.env), pg_dump -Fc the reference database into a new
database on 55421, run the server against it with the reference AUTH_SECRET,
and send the cookie to /_page?path=/login (the root layout's theme comes
from the session user) or to POST /_rpc/tokens/myTokens.
Upstream does all image work with sharp 0.34.5, a thin C++ layer over libvips
(8.17.3 in production). Calling libvips with the same operations and
arguments gives the same pixels, so the port binds libvips directly:
hand-written extern "C" declarations in vips/ffi.rs (no bindgen, no extra
crates); build.rs links what pkg-config --libs vips names (libvips 8.15 or
newer; brew install vips on macOS, libvips-dev in the image).
Layers: ffi.rs (raw declarations, GValue, VipsProgress) ->
operation.rs (Operation: any libvips operation by nickname, like vips8's
VImage::call) -> image.rs (Image: one owned reference, Blob, typed
wrappers, the timeout) -> sharp.rs (Sharp: sharp's pipeline.cc and
metadata() for the options the server uses).
Rules:
init()runsvips_initonce with sharp-runtime's settings (threads fromeffective_cpu_count(), cache 32 MB / 10 files / 50 operations).- Everything is blocking: use the async methods or
run_blocking(tokio's blocking pool, thenvips_thread_shutdown). Expensive work goes throughtransform_queue::image_transforms()like upstream'simageTransforms. Imageholds one GObject reference,Sendbut notSync; metadata is written only throughImage::mutateon an uncached private copy.- Inputs are copied into
vips_tracked_mallocmemory so the 32 MB cache ceiling bounds what cached operations can pin. - Errors take and clear the process-wide error buffer (concurrent failures can mix messages); VIPS warnings go to tracing at debug level.
- Loaders: JPEG, PNG, WebP, GIF, TIFF, HEIF/AVIF and SVG only (section 9, item 20); loads are sequential, as in sharp 0.34.5.
Image::set_timeoutis sharp's.timeout()(anevalhandler that kills the evaluation); encoders that never report progress run to the end.- Argument names and types are checked against the operation's own declarations before anything reaches GObject.
Adding an operation needs no new FFI: Operation::new("flatten")?.set("in", self).set("background", background).call()?.image("out"). Scalars are
i32/f64/bool/Arg::Str, enums and flags Arg::Nick; arguments only
newer libvips releases have go through set_if_known with a comment naming the
release; encoders are *save_buffer operations passed to Image::save_buffer.
Copy the order and arguments from sharp's pipeline.cc (sources in
_scratch/srv-library/ref-src/sharp@0.34.5). The media processing port
added flatten, join: { animated }, composite and text input (section 12).
What the serving routes (/[userCode]/[mediaCode]/[action]: convert,
raw?motion=1, raw?w= for a video, timeline.vtt, timeline.jpg; the
render of a .docx; /embed/[version]/[card]) and the media page call, by
upstream name. MediaRow is db::models::Upload (with rowMimeType
applied, as get_media returns it). Engine errors that upstream let escape
(BusyError, TimeoutError, AbortedError from the limiter;
AvTimeoutError, AvAbortedError from ffmpeg/ffprobe) come back as
anyhow::Errors to downcast_ref - the route's transformRefusal.
| Upstream | Rust | Notes |
|---|---|---|
avConvert(req) (av-action.ts) |
media::av_action::av_convert(&AppState, AvConvertRequest) -> anyhow::Result<Option<Response>> |
None = not audio/video, take the image branch. AvConvertRequest { media, privileged, to: Option<String>, offset_seconds (?t=), clip_seconds (?d=), signal, admit: Option<AdmitHook> (consumeView, download: true), dry_run (?check=1) }. Builds the whole response (refusals with X-Convert-Refused/limit, 204 with X-Convert-Note, streamed download); runs inside the limiter itself. |
convertOptionFor, conversionBudget, probeImage, convertImageBuffer (convert.ts) |
media::convert::{convert_option_for, conversion_budget_for(&ImageProbe, ConvertTarget), conversion_budget(animated, frame_pixels, total_pixels, to), probe_image(Bytes), convert_image_buffer(Bytes, ConvertTarget, ConvertOptions { animated, timeout_ms, signal }) -> anyhow::Result<Converted { buffer, mime, ext }>} |
the image branch of /convert; the ?check=1 row shortcut is conversion_budget(frames > 1, w*h, w*h*frames, to). |
derivedKey, contentHash, mayCacheDerived, readDerived, writeDerived, readDerivedStream, writeDerivedFile |
media::derived_cache::* (read_derived(&Storage, key) -> Option<Bytes>, write_derived(&Storage, key, Bytes), ...) |
best effort, as upstream. |
motionPreview(media, opts, signal) (motion.ts) |
media::motion::motion_preview(&AppState, &Upload, MotionOptions { width, height, fit: MotionFit }, AbortSignal) -> anyhow::Result<Result<(Bytes, u32 frames), MotionRefusal>> |
run it inside image_transforms().run(.., MOTION_WEIGHT, MOTION_TIMEOUT_MS); X-Motion-Frames is the u32; refusal tokens MotionRefusal::as_str(). Query parsing: motion_preview::{wants_motion, MOTION_DEFAULT_SIZE, clamp_motion_edge}. |
timelineTrack, timelineSheet (timeline.ts) |
media::timeline::{timeline_track, timeline_sheet}(&AppState, &Upload, AbortSignal) -> anyhow::Result<Result<String / Bytes, TimelineRefusal>> |
inside the limiter with TIMELINE_TRACK_* / TIMELINE_*; actions timeline_preview::{TIMELINE_TRACK_ACTION, TIMELINE_SHEET_ACTION}. |
videoPoster(media, signal) (poster.ts) |
media::poster::video_poster(&AppState, &Upload, AbortSignal) -> Option<Bytes> |
None = no poster (the route answers 415, never the video). open_video_source is shared with the timeline. |
docxToHtml(buf), MAX_DOCX_BYTES, docxSupport (docx.ts) |
media::docx::docx_to_html(Bytes) -> DocxResult, media::docx_support::{MAX_DOCX_BYTES, docx_support} |
section 13; run it inside the limiter at thumbnail weight with DOCX_TIMEOUT_MS, as the route does. |
renderCard(choice), parseCardFile, cardChoiceFor, cardUrl, EMBED_CARD_VERSION (card-render.ts, card.ts) |
media::card_render::render_card(&CardChoice) -> Result<Bytes, String>, media::card::{parse_card_file, card_choice_for, card_path, card_url, EMBED_CARD_VERSION, CARD_WIDTH, CARD_HEIGHT} |
render_card caches every pair it drew for the life of the process (shared futures, failures not cached). |
diagnosePlayback(media) (playback-health.ts) |
media::playback_health::diagnose_playback(&PlaybackSource) -> Option<PlaybackDiagnosis> (PlaybackSource::from(&Upload)) |
serialises like upstream's object (playback prop of the media page). |
showsSource, inlineTextPlan, sourceReadings, codeLanguageLabel, MAX_INLINE_TEXT (source-preview.ts) |
media::source_preview::* |
|
downloadOptionsFor, downloadHintKey, downloadLabelKey, sectionHeadingKey (download-menu.ts) |
media::download_menu::* (DownloadOption serialises as upstream's literals) |
the media page's menu (canEncode = owner or admin). |
estimateTargetSize, formatEstimate, ... (convert-size.ts) |
media::convert_size::* |
|
playerLang, buildPlayerDictionary (player-i18n.ts) |
media::player_i18n::* |
|
readVerdict/rememberVerdict/verdictKey (convert-verdict.ts), noteKey/REFUSAL_KEYS (convert-notes.ts) |
media::convert_verdict::*, media::convert_notes::* |
process-wide memo, as upstream. |
REASON_TEXT, refusal, verdictResponse (av-action.ts) |
media::av_action::{REASON_TEXT, refusal, verdict_response, base_name, source_tag, source_kind_of} |
base_name is the download name rule (filename.replace(/\.[^.]+$/, "") || code). |
Also ported and used inside: av_convert (planAv, runAvPlan, avVariant,
frameOffset, animationFilters, webScaleFilter, audioCoverPath),
convert_plan and convert_in_place (convertUploadInPlace behind the
convert RPC module), frame_step (the player's arithmetic, server twin),
pdf_lib (section 9, item 35).
Verification (re-runnable, see each script's header):
S=_scratch/srv-media-a # scratch reference on 8446, Rust server on 8426
python3 server/tests/parity_media/corpus.py $S/corpus
python3 server/tests/parity_media/media_parity.py ref # 595 cases on the reference
python3 server/tests/parity_media/media_parity.py rust --ffmpeg ref # the reference image's ffmpeg 5.1
python3 server/tests/parity_media/media_parity.py compare --ffmpeg ref
python3 server/tests/parity_media/media_parity.py cards # all 155 embed cards
python3 server/tests/parity_media/convert_actions.py # the convert RPC module, 20 checksThrough the routes: part B's tests/parity/media_serve.py on the merged
binary, 1113/1113 (with the reference image's ffmpeg; encoder-dependent
bytes compared by decoded content).
Measured on 2026-10-20 (libvips 8.18.7 here, 8.17.3 in the reference): statuses, meaningful headers and ffmpeg/ffprobe argument vectors identical in all 595 cases; outputs byte-identical for every JPEG, WebP (still and animated, including every motion tile), GIF, MP4/MOV remux and encode, MP3, WAV, poster and timeline sheet and track; PNG and AVIF differ in their bytes only (encoder builds; same pixels), Matroska and Ogg differ by their random segment/serial ids (same streams), PDFs as item 35; CMYK input shifts colour by about 3/255 (section 11). All 155 embed cards are pixel-identical. With Homebrew's ffmpeg 9 instead, see item 33.
media::docx::docx_to_html(Bytes) -> DocxResult is docxToHtml
(Result<DocxPreview { html, images, images_dropped }, DocxRefusal>,
DocxRefusal::as_str() gives upstream's token: not-a-zip, malformed,
too-big, too-many-entries, zip64, unsupported-method, too-large,
legacy-doc, not-a-document); media::zip_guard::inspect_zip and
media::docx_support::{docx_support, MAX_DOCX_BYTES} are the two helpers.
The route (/{tag}/{code}/render, gating, status codes) is not part of it.
media/mammoth/ is mammoth 1.12.1 ported file by file (docx/body_reader.rs
= lib/docx/body-reader.js, ...), with what it runs of JSZip
(jszip.rs: central directory, local names, size check, no CRC check),
@xmldom/xmldom (xml/xmldom.rs: sax.js and the DOM rules, every
warning/error is a failure because mammoth throws on them), lop
(lop.rs, the style-map grammar's combinators) and dingbat-to-unicode.
mammoth/js.rs holds the JavaScript semantics it needs (parseInt,
ToNumber, {} as a map with Object.prototype showing through, UTF-16
strings). The conversion runs on a thread of its own (256 MB stack, the
readers recurse); pictures go through the libvips binding one at a time in
document order (mapSeries upstream), so the 3 MB budget drops the same ones.
New dependency: flate2 on zlib-rs (zlib's inflate semantics, as node:zlib
and pako).
Verified by tests/parity_media/: docx_corpus.py builds 229 documents
(hand-built OOXML per mammoth feature, embedded style maps, pictures, hostile
archives, xmldom edge cases, Object.prototype names, macOS textutil output,
mammoth's own Word/LibreOffice test documents); docx_parity.py runs each
through upstream's own docx.ts, zip-guard.ts, docx-support.ts,
sharp-runtime.ts and limits.ts inside the reference container (node 24
strips the types; imports rewritten to the container's packages) and through
docx_to_html (tests/media_docx.rs, ignored test parity_dump), and
compares the JSON. On 2026-10-20: 222 identical byte for byte, pictures
included (libvips 8.18 here and 8.17.3 in the reference wrote the same
WebP); 3 known differences (items 3 and 4 below); 4 that never finish
upstream (item 1). docx_route_e2e.py fetches a sample through the
reference's real /render route with minted tickets: same bytes as the
harness.
To repeat (from the main checkout; the reference app runs on 8446, see
_scratch/srv-media-a/):
python3 server/tests/parity_media/docx_corpus.py _scratch/srv-media-a/docx/corpus
python3 server/tests/parity_media/docx_parity.py --corpus _scratch/srv-media-a/docx/corpus \
--work _scratch/srv-media-a/docx/run --container modlight-srvmedia-a-ref-app-1
python3 server/tests/parity_media/docx_route_e2e.py --corpus _scratch/srv-media-a/docx/corpus \
--work _scratch/srv-media-a/docx/run f-basic.docx i-budget.docx h-bomb-lying.docxDifferences, all deliberate or unavoidable:
- Hangs. Upstream never finishes on some inputs the guard lets
through; the port answers. JSZip loops forever on ZIP64 "extensible data"
whose length walks the reader backwards (
not-a-documenthere); xmldom's error recovery is quadratic (the port stops at the first report, which is the same answer); some error-free scans are quadratic (many unterminated<?or CDATA sections, many distinct element names: memoised here, same tree). - Entries the guard never inflated (a central record beyond the declared count, or the directory JSZip finds after its prepended-data shift) are expanded with a ceiling of 40 MiB + 1 (JSZip: none) and fail past it.
- Prototype pollution not reproduced. A style with
w:type="__proto__"writes itself ontoObject.prototypein upstream (every object in the Node process gains a property, for the rest of its life: e.g. a style idw:typeremoves every<w:br/>). Here such a style is ignored. An object assigned to a__proto__key of mammoth's maps does not become the prototype of the other keys (inheritedObject.prototypemembers are reproduced:constructor,toString, ... as ids, element names, relationship types). - Nesting depth. Upstream runs out of V8 stack at a depth that depends
on what nests and on JIT warm-up (fresh process: 1898 nested elements in a
paragraph, 909 nested
w:sdt, 306 nested tables; warm: 2499, 1249, 312). The port fails an XML part nested deeper than 1901 elements (the fresh limit for plain nesting); deeperw:sdt/table nesting below that renders here and fails upstream. - Lone UTF-16 surrogates (from character references) are carried as U+10F800..U+10FFFF inside the port and resolved at the end exactly as the JavaScript string would be encoded; a document containing those private-use code points literally would see them treated as surrogates.