From b2dcd0c27133a82ac0b6a1c71cabfbab28ce1c05 Mon Sep 17 00:00:00 2001 From: Gleb Linnik Date: Thu, 12 Nov 2026 15:09:47 +0300 Subject: [PATCH 1/7] ASS subtitles: attached fonts, every-video-frame karaoke on the TV, straight colour over video (#1347) Styled ASS subtitles now load their attached fonts, animate heavy karaoke at the video's frame rate on the TV, and keep translucent colour as authored over video. - Fonts: main's 32 MiB attached-font cap is gone. A file's attachments are detached from libavformat as it opens (`detach_fonts`), the source store holds one `Arc<[u8]>` per font, and every libass renderer borrows it through the new facade call `plx_ass_add_font_borrowed` (facade ABI 5) instead of copying it. - Composite (`src/ass.c`): zero-coverage groups are skipped, the row blend is NEON with an exact div255 (the scalar path stays), and regions are published premultiplied through `Painter::tex_premultiplied`. Stats split wall time from thread CPU time; `clock_gettime` links through `-lrt` for firmware 6.4 and older. - Colour over video: the TV compositor blends the UI plane over video as straight alpha, so every partial-alpha pixel came out darkened. While the video plane is bound, a final GPU pass (`gfx/straighten.rs`, `GL_ARM_shader_framebuffer_fetch`) converts the framebuffer to straight colour over the boxes drawn that frame. - Rate: the ASS mailbox pipelines animated frames ahead of the playhead. A pool of up to 3 libass renderers runs below the frame thread's priority (`task::lower_priority`), sized by a pure memory verdict (`ass/pool.rs`: a quarter of MemAvailable for the helpers, the open primary not charged). Dense event boundaries no longer abandon the plan (`Rendered::moves_on`). An adaptive render scale (`ass/scale.rs`, down to 1/2) draws heavy animated stretches smaller, with hysteresis; stills stay full size. - `tests/fixtures/make_ass_stress.py` places per-syllable karaoke effects using libass's real advances. Measured on the TV (webOS 4.10.2, release build, stress fixture): - Subtitle updates in the heavy opening: 7-9/s before this change. With a 24p video it is now a median of 26/s, about 4% late. With the same video at 30p it is a median of 29.7/s, 2.2% late. - Composite p95 fell from 90-107 ms to 15-25 ms at full size. - Peak RSS fell from 541 MB to 287-372 MB. - Placement matches upstream libass 0.17.5 on captured frames, and the phone-recorded subtitle timing matches the test card's frame counter to within about a frame. Closes #1328 --- .agents/skills/profile-tv/SKILL.md | 4 +- ci/build-libass.py | 7 +- ci/list-caps.ini | 18 +- ci/module-layers.ini | 4 +- ci/test_ass_composite.py | 93 +- ci/test_ass_regions.py | 256 +++- docs/agent-reference.md | 21 +- docs/ass-subtitles.md | 184 ++- docs/egl-partial-update-and-damage.md | 9 +- include/ass.h | 56 +- locales/be/widgets.json | 1 - locales/ca/widgets.json | 1 - locales/de/widgets.json | 1 - locales/en/widgets.json | 4 - locales/es/widgets.json | 1 - locales/fr/widgets.json | 1 - locales/hu/widgets.json | 1 - locales/it/widgets.json | 1 - locales/nl/widgets.json | 1 - locales/pl/widgets.json | 1 - locales/pt/widgets.json | 1 - locales/ru/widgets.json | 1 - rust-modules/base/src/lib.rs | 2 + rust-modules/base/src/task.rs | 107 ++ rust-modules/base/src/testheap.rs | 148 ++ rust-modules/gfx/src/gfx.rs | 354 ++++- rust-modules/gfx/src/gfx/backdrop.rs | 1 + rust-modules/gfx/src/gfx/straighten.rs | 1249 +++++++++++++++++ rust-modules/gfx/src/rtl.rs | 6 +- .../gfx/src/shaders/fs_straighten.frag | 50 + .../gfx/src/shaders/vs_straighten.vert | 19 + rust-modules/gfx/src/text.rs | 31 +- .../media-codecs/src/subtitle_extract.rs | 106 +- .../media/src/ff_font_memory_tests.rs | 248 ++++ .../media/src/ff_image_subtitle_tests.rs | 9 +- rust-modules/media/src/ff_subside.rs | 3 +- .../media/src/ff_subtitle_charset_tests.rs | 11 +- rust-modules/media/src/player/ass.rs | 790 +++++++++-- .../media/src/player/ass/font_names.rs | 388 +++++ .../media/src/player/ass/font_names/tests.rs | 370 +++++ rust-modules/media/src/player/ass/mailbox.rs | 703 +++++++++- .../media/src/player/ass/mailbox_tests.rs | 1118 ++++++++++++++- rust-modules/media/src/player/ass/pool.rs | 121 ++ rust-modules/media/src/player/ass/scale.rs | 253 ++++ .../media/src/player/ass/scale_tests.rs | 270 ++++ .../media/src/player/ass/synth_fonts.rs | 180 +++ rust-modules/media/src/player/ass/tests.rs | 732 +++++++++- .../media/src/player/ass/tests/synth.rs | 11 +- rust-modules/media/src/player/ass_perf.rs | 493 +++++-- rust-modules/media/src/player/ass_source.rs | 107 +- .../media/src/player/demux_session.rs | 5 - rust-modules/media/src/player/mod.rs | 13 + rust-modules/media/src/player/subtitle_cue.rs | 3 +- rust-modules/play-feed/src/demux_clock.rs | 1 - rust-modules/play-feed/src/ff.rs | 19 +- rust-modules/playback-contract/src/demux.rs | 1 - .../screens/src/player/ass_subtitles.rs | 108 +- rust-modules/src/app/adapters/hostsim.rs | 11 +- .../src/app/adapters/hostsim/sim_video.rs | 17 +- rust-modules/src/app/run.rs | 91 +- rust-modules/src/shot.rs | 18 +- rust-modules/ui/src/lib.rs | 45 + src/ass.c | 230 ++- src/ass_composite.h | 51 +- src/ass_fonts.c | 39 +- src/ass_internal.h | 14 + tests/fixtures/make_ass_stress.py | 622 +++++++- tools/hud_bundle.py | 2 +- 68 files changed, 9242 insertions(+), 595 deletions(-) create mode 100644 rust-modules/base/src/testheap.rs create mode 100644 rust-modules/gfx/src/gfx/straighten.rs create mode 100644 rust-modules/gfx/src/shaders/fs_straighten.frag create mode 100644 rust-modules/gfx/src/shaders/vs_straighten.vert create mode 100644 rust-modules/media/src/ff_font_memory_tests.rs create mode 100644 rust-modules/media/src/player/ass/font_names.rs create mode 100644 rust-modules/media/src/player/ass/font_names/tests.rs create mode 100644 rust-modules/media/src/player/ass/pool.rs create mode 100644 rust-modules/media/src/player/ass/scale.rs create mode 100644 rust-modules/media/src/player/ass/scale_tests.rs create mode 100644 rust-modules/media/src/player/ass/synth_fonts.rs diff --git a/.agents/skills/profile-tv/SKILL.md b/.agents/skills/profile-tv/SKILL.md index bc68d8c752..d32669611b 100644 --- a/.agents/skills/profile-tv/SKILL.md +++ b/.agents/skills/profile-tv/SKILL.md @@ -46,8 +46,8 @@ consumed CPU. ## Styled-subtitle (libass) cost -With an ASS/SSA track rendering, the event log carries `assperf:` lines (`window_ms` the span the counts cover, render and whole-job p95/max, bytes -published and uploaded in that window, Normal+Ahead `jobs`, `ahead_hit`, `nwait`, and `worker tid=`) and `FRAMEDROP spans=` gains `assup` (the +With an ASS/SSA track rendering, the event log carries `assperf:` lines, one per worker `w` (the primary and up to two helpers a moving cue starts; `window_ms` the span the counts cover, render and whole-job p95/max, bytes +published and uploaded in that window, Normal+Ahead `jobs`, `ahead_hit`, `shown` (pipeline frames promoted: updates a second), `late`, `stale`, `nwait`, the libass/composite split `lib_p95_us` / `comp_p95_us` (wall) and `lib_cpu_p95_us` / `comp_cpu_p95_us` (the worker thread's CPU time in the same phases: wall far above CPU means it waited for the CPU, not that it computed) with the `images_p95` / `img_kpx_p95` it drew, `scale` (the smallest size a job of the window was drawn at, `1.00` / `0.67` / `0.50`: a heavy moving stretch is drawn smaller, read the figures at their own scale, and each move is one `ass: render scale A -> B` line), and `worker w= tid= nice=`) and `FRAMEDROP spans=` gains `assup` (the UI-thread GL upload; needs `/tmp/plxnative-framedrop` armed). Read these before attributing a late frame to subtitles; follow the worker with `tools/tv-sched-trace.sh --tid `. See `docs/ass-subtitles.md`. diff --git a/ci/build-libass.py b/ci/build-libass.py index db3e73b538..cbc61a65e9 100755 --- a/ci/build-libass.py +++ b/ci/build-libass.py @@ -142,7 +142,10 @@ def verify_library(path, darwin, readelf=None): else: dynamic = subprocess.check_output([readelf, '-d', str(path)], text=True) needed = re.findall(r'\(NEEDED\).*\[([^\]]+)\]', dynamic) - allowed = {'libc.so.6', 'libm.so.6', 'libgcc_s.so.1', 'libpthread.so.0', 'ld-linux.so.3'} + # librt: src/ass.c times its render phases with clock_gettime, which the build sysroot's glibc + # (2.12, the one this library links against, not the TV's) keeps there. Newer firmware (7.4 and + # later) provides the same clock_gettime@GLIBC_2.4 from libc.so.6. + allowed = {'libc.so.6', 'libm.so.6', 'libgcc_s.so.1', 'libpthread.so.0', 'librt.so.1', 'ld-linux.so.3'} symbols = subprocess.check_output([readelf, '--dyn-syms', '--wide', str(path)], text=True) names = [] for line in symbols.splitlines(): @@ -327,7 +330,7 @@ def autotools_build(name, options): exports = work / 'exports.map' exports.write_text('{ global: plx_ass_*; local: *; };\n') command += ['-shared', '-Wl,-soname,' + name, '-Wl,--no-undefined', - '-Wl,--exclude-libs,ALL', '-Wl,--version-script=' + str(exports)] + '-Wl,--exclude-libs,ALL', '-Wl,--version-script=' + str(exports), '-lrt'] if not host: # A stable identity for the shipped (stripped) library, so the `.debug` that # `make symbols-libs` extracts from the unstripped $prefix/lib copy can be paired with it. diff --git a/ci/list-caps.ini b/ci/list-caps.ini index 8865ccbeb0..4d618e2e65 100644 --- a/ci/list-caps.ini +++ b/ci/list-caps.ini @@ -492,7 +492,7 @@ reason = Bytes of a redirect body drained; an HTTP transport bound. class = format reason = Prefix check of a Content-Range header. -[media-codecs/src/subtitle_extract.rs::collect_manifest] +[media-codecs/src/subtitle_extract.rs::detach_fonts] class = format reason = Four-byte font magic check on an attached font. @@ -544,9 +544,21 @@ reason = Head/tail byte fingerprint of a side-loaded subtitle. class = format reason = MAX_EVENT_BYTES: container, codec or ABR estimator bound of the playback pipeline; not a list the user browses. -[media/src/player/ass.rs::MAX_FONT_BYTES] +[media/src/player/ass/font_names.rs::fold] +class = not-a-list +reason = Clips one font or style NAME to 1024 bytes before it is compared; a name is text, no face or font is left out by it (a longer name only matches more, never less). + +[media/src/player/ass/font_names.rs::cff_name] +class = not-a-list +reason = Clips the CFF font NAME read from a face to 1024 bytes before it is compared; text length, no face is left out by it. + +[media/src/player/ass/scale.rs::FLIP_WINDOW_MS] +class = not-a-list +reason = How long after a render-scale step up a step down counts as taking it back (a time window of the hysteresis, in playhead milliseconds); no subtitle, cue or frame is left out by it. + +[media/src/player/ass.rs::MAX_ASS_RENDERERS] class = work -reason = Memory budget for embedded fonts (FontBudget); a font past it is skipped and its text falls back to the default face, never a refused track. +reason = Styled-subtitle renderers (a worker and a libass instance each) that may run at once while a moving cue's pipeline wants more frames than one makes; bounds concurrency and memory, and every frame is still rendered, only fewer at a time. [media/src/player/ass.rs::MAX_PIXELS] class = format diff --git a/ci/module-layers.ini b/ci/module-layers.ini index b1d6e17239..e1527b5a0f 100644 --- a/ci/module-layers.ini +++ b/ci/module-layers.ini @@ -31,13 +31,13 @@ # draws through (neither may name the other), the hold-band cut (`hold`: the rows a paged store # keeps, which the store and `ui::cards::HoldBand` both apply), the details-line parts # (`detail_parts`: the set a profile keeps and the caption policy words), and the test-only lock/net -# helpers every layer's tests use. +# helpers every layer's tests use (`testheap`: the C heap's live bytes, for the tests that count copies). uses = members = eventlog paths process clock task cbuf sha256 sha1 b64 spki dynlib checkpoint storage_worker fontcov surface tile hold detail_parts window diag::zlib diag::spans diag::heartbeat devtrig - testlock testnet testscratch testrace + testlock testnet testscratch testrace testheap [machine] # EXTRACTED: this layer is the workspace crate `plx_machine` (rust-modules/machine/), the second diff --git a/ci/test_ass_composite.py b/ci/test_ass_composite.py index c93df89a3b..0b4ee84a08 100644 --- a/ci/test_ass_composite.py +++ b/ci/test_ass_composite.py @@ -1,5 +1,12 @@ #!/usr/bin/env python3 -"""Exhaustive native arithmetic check against the scalar compositing reference.""" +"""Exhaustive native arithmetic check against the scalar compositing reference. + +The scalar blend (`ass_blend_pixel`) is checked against the formula on every (coverage, source, +destination) triple. Where the host has NEON (`__ARM_NEON`: the 32-bit ARM TVs and arm64 Macs) the +eight-pixel kernel the row loop uses (`ass_blend8`) is checked the same way, lane by lane with a +different pixel in every lane, so a wrong rounding or a lane that leaks into its neighbour fails; a +host without NEON prints that the vector kernel was not exercised. +""" from pathlib import Path import os import shlex @@ -31,14 +38,88 @@ } } } - if (a) { - unsigned expected = (s*255 + a/2)/a; - if (expected > 255) expected = 255; - if (ass_straight_channel(s, a, 65536/a) != expected) return 2; + } + } + puts("PASS: 16777216 blend combinations match scalar RGBA"); +#ifdef PLX_ASS_NEON + /* The rounding primitive: every numerator a blend can form (at most 255*255) divides as the + * scalar's (n + 127) / 255, eight lanes at a time. */ + for (unsigned base = 0; base <= 65025; base += 8) { + uint16_t in[8]; + uint8_t out[8]; + for (unsigned k = 0; k < 8; ++k) in[k] = base + k <= 65025 ? base + k : 0; + vst1_u8(out, ass_div255_lanes(vld1q_u16(in))); + for (unsigned k = 0; k < 8; ++k) + if (out[k] != (in[k] + 127) / 255) { + fprintf(stderr, "div255 n=%u: %u != %u\n", in[k], out[k], (in[k] + 127) / 255); + return 1; + } + } + /* The kernel at opacity 255, where the coverage is the mask byte itself: every (a, s, d) triple + * reaches every lane, and each lane holds a different one from its neighbours (a and d step by + * lane-dependent amounts, both bijections over a0 and d0), channel by channel as above. */ + for (unsigned s = 0; s < 256; ++s) { + const unsigned src[4] = {s, 255 - s, s ^ 85, 255}; + const AssSource8 source = ass_source8(src[0] | (src[1] << 8) | (src[2] << 16) | + ((uint32_t)src[3] << 24)); + for (unsigned a0 = 0; a0 < 256; ++a0) { + for (unsigned d0 = 0; d0 < 256; ++d0) { + uint8_t mask[8], dst[32], expected[32]; + for (unsigned k = 0; k < 8; ++k) { + const unsigned a = (a0 + 37 * k) & 255, d = (d0 + 91 * k) & 255; + const unsigned dc[4] = {d, d ^ 170, 255 - d, d}; + mask[k] = (uint8_t)a; + for (unsigned c = 0; c < 4; ++c) { + dst[4 * k + c] = (uint8_t)dc[c]; + expected[4 * k + c] = (uint8_t)((src[c] * a + dc[c] * (255 - a) + 127) / 255); + } + } + ass_blend8(dst, vld1_u8(mask), vdup_n_u8(255), source); + for (unsigned i = 0; i < 32; ++i) + if (dst[i] != expected[i]) { + fprintf(stderr, "ass_blend8 s=%u a0=%u d0=%u lane %u channel %u: %u != %u\n", + s, a0, d0, i / 4, i % 4, dst[i], expected[i]); + return 1; + } + } + } + } + /* The coverage the kernel computes, (mask * opacity + 127) / 255, is the scalar code's table for + * every opacity and mask byte: here the opacity varies, with a source and destination that + * change with them. */ + for (unsigned opacity = 0; opacity < 256; ++opacity) { + for (unsigned m0 = 0; m0 < 256; m0 += 8) { + for (unsigned variant = 0; variant < 16; ++variant) { + const unsigned s = (opacity * 7 + m0 * 3 + variant * 29) & 255; + const unsigned src[4] = {s, 255 - s, s ^ 85, 255}; + const AssSource8 source = ass_source8(src[0] | (src[1] << 8) | (src[2] << 16) | + ((uint32_t)src[3] << 24)); + uint8_t mask[8], dst[32], expected[32]; + for (unsigned k = 0; k < 8; ++k) { + const unsigned m = m0 + k, a = (m * opacity + 127) / 255; + const unsigned d = (m * 5 + opacity * 11 + variant * 53 + k) & 255; + const unsigned dc[4] = {d, d ^ 170, 255 - d, 255 - (d ^ 85)}; + mask[k] = (uint8_t)m; + for (unsigned c = 0; c < 4; ++c) { + dst[4 * k + c] = (uint8_t)dc[c]; + expected[4 * k + c] = (uint8_t)((src[c] * a + dc[c] * (255 - a) + 127) / 255); + } + } + ass_blend8(dst, vld1_u8(mask), vdup_n_u8((uint8_t)opacity), source); + if (memcmp(dst, expected, sizeof dst) != 0) { + fprintf(stderr, "ass_blend8 opacity=%u mask %u..%u variant %u differs from the " + "scalar coverage table\n", opacity, m0, m0 + 7, variant); + return 1; + } } } } - puts("PASS: 16777216 blend combinations and 65280 alpha conversions match scalar RGBA"); + puts("PASS: NEON ass_blend8 matches scalar RGBA on 16777216 (coverage, source, destination) " + "triples per lane and on every (mask, opacity) pair"); +#else + puts("NOTE: this host has no NEON: the vector kernel (ass_blend8) was NOT exercised, " + "only the scalar blend"); +#endif return 0; } ''' diff --git a/ci/test_ass_regions.py b/ci/test_ass_regions.py index 8388c532e7..3ac4e1858e 100644 --- a/ci/test_ass_regions.py +++ b/ci/test_ass_regions.py @@ -4,7 +4,16 @@ The producer stub supplies immutable coverage buffers, as pinned libass does. Count real blend operations, then compare cached output with a fresh compositor across motion, content changes, clipping, region merges, gaps and arena shifts. -No native dependency build is needed for this host regression gate. +Count the pixels the blend loop steps onto one at a time although they have nothing to +draw (the compositor skips runs of them whole), and compare the compositor's pixels with +a frozen copy of the old scalar blend loop on random masks. The published pixels are +PREMULTIPLIED RGBA (each colour channel at most its alpha), drawn with a premultiplied +blend: the compositor does no straight-alpha conversion pass. No native dependency build +is needed for this host regression gate. + +The whole program is built and run twice: as the host's compiler builds the compositor (where +`__ARM_NEON` is defined, the eight-pixel NEON row path of src/ass_composite.h) and again with +`-DPLX_ASS_NO_NEON`, the scalar row path every other target runs, so an arm64 host covers both. """ from pathlib import Path import os @@ -52,19 +61,64 @@ static int ass_track_set_feature(ASS_Track *t, int f, int e) { return 0; } static void ass_set_line_spacing(ASS_Renderer *r, double s) {} static size_t plxi_library_fonts(ASS_Library *l) { return 0; } +static int plxi_library_borrow_font(ASS_Library *l, const char *n, const uint8_t *d, size_t z, size_t *i) { *i = 0; return 0; } +static void plxi_library_forget_borrowed(ASS_Library *l, const size_t *i, size_t c) {} static void ass_set_storage_size(ASS_Renderer *r, int w, int h) {} ''' SOURCE = r''' #include #include #include +#include #include "ass_composite.h" -static unsigned blends; + +/* FROZEN: the scalar compositor's blend loop as it was before the zero-run skipping (src/ass.c at + * 8c8020ae0, compose_frame's first loop), run over a whole canvas instead of per region. Every pixel + * is visited one at a time; this is the definition of the pixels the optimised code must produce. + * Its output is PREMULTIPLIED RGBA, which is what the compositor publishes: the straight-alpha pass + * that followed the loop (and that 8c8020ae0 still ran) is gone, so the frozen copy of it is too. + * Do not "improve" it, and keep it above the ass_blend_pixel counter below so it is not counted. */ +static void reference_compose(const ASS_Image *list, int width, int height, uint8_t *canvas) +{ + for (const ASS_Image *p = list; p; p = p->next) { + if (p->w <= 0 || p->h <= 0) continue; + int l = p->dst_x < 0 ? 0 : p->dst_x, t = p->dst_y < 0 ? 0 : p->dst_y; + long long right = (long long)p->dst_x + p->w, bottom = (long long)p->dst_y + p->h; + int r = right > width ? width : (int)right, b = bottom > height ? height : (int)bottom; + if (r <= l || b <= t) continue; + uint32_t color = (p->color >> 24) | ((p->color >> 8) & 0xff00u) | + ((p->color << 8) & 0xff0000u) | 0xff000000u; + unsigned opacity = 255 - (p->color & 255); + uint8_t coverage[256]; + for (unsigned i = 0; i < 256; ++i) + coverage[i] = (uint8_t)((i * opacity + 127) / 255); + for (int y = t; y < b; ++y) { + const uint8_t *mask = p->bitmap + (size_t)(y - p->dst_y) * p->stride; + uint8_t *dst = canvas + ((size_t)y * width + l) * 4; + for (int x = l; x < r; ++x, dst += 4) { + unsigned a = coverage[mask[x - p->dst_x]]; + if (!a) continue; + if (a == 255) { + ass_store_rgba(dst, color); + continue; + } + ass_store_rgba(dst, ass_blend_pixel(ass_load_rgba(dst), color, a)); + } + } + } +} + +/* `blends` counts every pixel the compositor blends: one per scalar ass_blend_pixel call and eight + * per NEON vector operation (the vector operation does not pass through ass_blend_pixel, and draws + * its zero and opaque lanes too). `vectors` counts the vector operations alone. */ +static unsigned blends, zero_blend, vectors; static uint32_t counted_blend(uint32_t dst, uint32_t color, unsigned a) { ++blends; return ass_blend_pixel(dst, color, a); } #define ass_blend_pixel counted_blend +#define PLX_ASS_VISIT_ZERO(kind) (++zero_##kind) +#define PLX_ASS_VISIT_VECTOR(pixels) (blends += (pixels), ++vectors) #include "ass.c" static ASS_Image *input; @@ -80,7 +134,6 @@ ctx->track = &track; ctx->renderer = &renderer; ctx->first = 1; - for (unsigned a = 1; a < 256; ++a) ctx->reciprocal[a] = 65536u / a; return ctx; } enum { WIDTH = 96, HEIGHT = 48, CANVAS = WIDTH * HEIGHT * 4 }; @@ -96,10 +149,22 @@ b->rgba + y * b->width * 4, b->width * 4); } } +/* Published pixels are premultiplied: no colour channel can exceed its own alpha. (Straight RGBA of + * a half-covered sign has full-strength colour at alpha 128, which is the shape this refuses.) */ +static void assert_premultiplied(const uint8_t *canvas, const char *what) { + for (size_t i = 0; i < CANVAS; i += 4) + for (int c = 0; c < 3; ++c) + if (canvas[i + c] > canvas[i + 3]) { + fprintf(stderr, "%s: pixel %zu is not premultiplied: rgba %u %u %u %u (channel %d above alpha)\n", + what, i / 4, canvas[i], canvas[i + 1], canvas[i + 2], canvas[i + 3], c); + exit(1); + } +} static unsigned check(PlxAss *ctx, int width) { uint8_t actual[CANVAS], expected[CANVAS]; blends = 0; render(ctx, width, actual); + assert_premultiplied(actual, "composed frame"); unsigned cached = blends; PlxAss *fresh = context(); blends = 0; @@ -108,7 +173,173 @@ plx_ass_destroy(fresh); return cached; } +static uint32_t rng = 0x13280001u; +static uint32_t rnd32(void) { + rng ^= rng << 13; rng ^= rng >> 17; rng ^= rng << 5; + return rng; +} +static unsigned below(unsigned n) { return rnd32() % n; } +/* A mask of `h` rows of `stride` bytes (the last row only `w` long, as libass allocates it): runs of + * zeros of every length 0..40 between runs of coverage 255, 1, 2..4 or random, so a zero run starts + * at every alignment. The padding between rows is nonzero garbage no image may read. */ +static void fill_mask(uint8_t *mask, int w, int h, int stride) { + for (int y = 0; y < h; ++y) { + uint8_t *row = mask + (size_t)y * stride; + int x = 0, zeros = (int)below(2); + while (x < w) { + int len = zeros ? (int)below(41) : 1 + (int)below(12), kind = (int)below(4); + for (int i = 0; i < len && x + i < w; ++i) + row[x + i] = zeros ? 0 : kind == 0 ? 255 : kind == 1 ? 1 : kind == 2 ? 2 + below(3) : 1 + below(255); + x += len; + zeros ^= 1; + } + if (y < h - 1) + for (int i = w; i < stride; ++i) row[i] = 1 + below(255); + } +} +/* Random images over a small canvas: clipped on every side, overlapping, opacities from fully + * opaque to fully transparent (opacity 1 maps every coverage below 128 to 0: a nonzero mask byte + * that draws nothing). Each frame is composed by the real compositor and by the frozen scalar loops. */ +static void random_frames(int frames) { + static const unsigned alphas[] = {0, 0, 1, 2, 127, 254, 255}; + for (int f = 0; f < frames; ++f) { + ASS_Image images[5]; + uint8_t *masks[5]; + int count = 1 + (int)below(5); + for (int i = 0; i < count; ++i) { + int w = 1 + (int)below(70), h = 1 + (int)below(5), stride = w + (int)below(10); + masks[i] = malloc((size_t)stride * (h - 1) + w); /* exact: a read past the row end is out of bounds */ + assert(masks[i]); + fill_mask(masks[i], w, h, stride); + unsigned alpha = below(8) == 7 ? below(256) : alphas[below(7)]; + images[i] = (ASS_Image){w, h, stride, masks[i], (rnd32() & ~255u) | alpha, + (int)below(WIDTH + 41) - 30, (int)below(HEIGHT + 4) - 3, + i + 1 < count ? &images[i + 1] : NULL}; + } + input = &images[0]; + PlxAss *ctx = context(); + uint8_t actual[CANVAS], expected[CANVAS]; + memset(expected, 0, CANVAS); + render(ctx, WIDTH, actual); + assert_premultiplied(actual, "random frame"); + reference_compose(input, WIDTH, HEIGHT, expected); + if (memcmp(actual, expected, CANVAS) != 0) { + fprintf(stderr, "random frame %d differs from the scalar reference\n", f); + exit(1); + } + plx_ass_destroy(ctx); + for (int i = 0; i < count; ++i) free(masks[i]); + } +} +/* Pixels the blend loop stepped onto singly although it had nothing to draw, for one image. */ +static unsigned zero_visits(ASS_Image *image) { + input = image; + PlxAss *ctx = context(); + uint8_t canvas[CANVAS]; + zero_blend = 0; + render(ctx, WIDTH, canvas); + plx_ass_destroy(ctx); + return zero_blend; +} + +/* One pixel of the published canvas. */ +static void pixel_at(const uint8_t *canvas, int x, int y, uint8_t out[4]) { + memcpy(out, canvas + ((size_t)y * WIDTH + x) * 4, 4); +} + int main(void) { + /* The published bytes are PREMULTIPLIED RGBA, exactly. A sign of colour (255, 128, 64) at coverage + * 128 over nothing is (255*128+127)/255, (128*128+127)/255, (64*128+127)/255 = 128, 64, 32 at + * alpha 128 (the straight form of it is 255, 128, 64, 128). Coverage 255 is the colour itself at + * alpha 255, and the one-pixel border the region keeps around a sign stays 0, 0, 0, 0. */ + { + uint8_t mask[4 * 3]; + memset(mask, 128, sizeof mask); + for (int x = 0; x < 4; ++x) mask[2 * 4 + x] = 255; + ASS_Image sign = {4, 3, 4, mask, 0xff804000, 10, 20, NULL}; + input = &sign; + PlxAss *ctx = context(); + uint8_t canvas[CANVAS], px[4]; + render(ctx, WIDTH, canvas); + static const uint8_t half[4] = {128, 64, 32, 128}, solid[4] = {255, 128, 64, 255}, none[4] = {0, 0, 0, 0}; + pixel_at(canvas, 11, 20, px); + if (memcmp(px, half, 4) != 0) { + fprintf(stderr, "half-coverage pixel is rgba %u %u %u %u, want the premultiplied %u %u %u %u\n", + px[0], px[1], px[2], px[3], half[0], half[1], half[2], half[3]); + return 1; + } + pixel_at(canvas, 12, 22, px); + if (memcmp(px, solid, 4) != 0) { + fprintf(stderr, "opaque pixel is rgba %u %u %u %u, want %u %u %u %u\n", + px[0], px[1], px[2], px[3], solid[0], solid[1], solid[2], solid[3]); + return 1; + } + pixel_at(canvas, 9, 19, px); + if (memcmp(px, none, 4) != 0) { + fprintf(stderr, "border pixel is rgba %u %u %u %u, want 0 0 0 0\n", px[0], px[1], px[2], px[3]); + return 1; + } + plx_ass_destroy(ctx); + } + /* Zero runs are skipped whole. A sign's coverage is mostly zero (44% of the iterations of the old + * blend loop on the #1328 fixture). Eight rows of 24 zeros, 8 coverage-200 bytes, 32 zeros: no + * zero byte shares a group of four with coverage, so no zero byte may be visited singly. */ + { + uint8_t mask[64 * 8]; + for (int y = 0; y < 8; ++y) + for (int x = 0; x < 64; ++x) mask[y * 64 + x] = x >= 24 && x < 32 ? 200 : 0; + ASS_Image sign = {64, 8, 64, mask, 0x11223300, 8, 6, NULL}; + unsigned blend = zero_visits(&sign); + if (blend != 0) { + fprintf(stderr, "zero runs were stepped through pixel by pixel: %u zero-coverage pixels in the " + "blend loop (want 0, the old loop: %u)\n", blend, 56u * 8); + return 1; + } + } + /* Zero runs of 37 and 24 around a 9-byte run of coverage 255, starting off every group boundary: + * each row may step onto the zeros of the two groups the run's ends fall into and onto its tail + * (at most 3 each), no more. */ + { + uint8_t mask[80 * 5 + 70]; + for (int y = 0; y < 6; ++y) + for (int x = 0; x < 70; ++x) mask[y * 80 + x] = x >= 37 && x < 46 ? 255 : 0; + ASS_Image sign = {70, 6, 80, mask, 0x44556600, 3, 20, NULL}; + unsigned blend = zero_visits(&sign), blend_bound = 6 * 9; /* sign rows */ + if (blend > blend_bound) { + fprintf(stderr, "a misaligned sign was stepped through pixel by pixel: %u zero-coverage pixels in " + "the blend loop (at most %u)\n", blend, blend_bound); + return 1; + } + } + /* The vector path fires on a group of eight pixels with coverage and on no other. Four rows of 40, + * every byte zero but one aligned group (bytes 8..15) of coverage 200: a NEON build blends each + * row with ONE vector operation (4 operations, 32 pixels), any other build with eight scalar + * blends (no vector operation, 32 pixels), and neither steps onto a zero pixel singly. */ + { + uint8_t mask[40 * 4]; + memset(mask, 0, sizeof mask); + for (int y = 0; y < 4; ++y) + for (int x = 8; x < 16; ++x) mask[y * 40 + x] = 200; + ASS_Image sign = {40, 4, 40, mask, 0x77665500, 20, 10, NULL}; + input = &sign; + PlxAss *ctx = context(); + uint8_t canvas[CANVAS]; + blends = vectors = zero_blend = 0; + render(ctx, WIDTH, canvas); + plx_ass_destroy(ctx); +#ifdef PLX_ASS_NEON + const unsigned want_vectors = 4; +#else + const unsigned want_vectors = 0; +#endif + if (blends != 32 || vectors != want_vectors || zero_blend != 0) { + fprintf(stderr, "a group of eight with coverage: %u pixels blended in %u vector operations, " + "%u zero pixels stepped onto (want 32 in %u, none)\n", + blends, vectors, zero_blend, want_vectors); + return 1; + } + } + random_frames(4000); uint8_t masks[2][256]; for (int i = 0; i < 256; ++i) { masks[0][i] = (uint8_t)(40 + i % 170); @@ -155,7 +386,11 @@ sign.bitmap = masks[0]; masks[0][0] ^= 15; check(ctx, WIDTH); plx_ass_destroy(ctx); - puts("PASS: unchanged regions skip blending; 253 frames match fresh composition"); +#ifdef PLX_ASS_NEON + puts("PASS (NEON row path): published pixels are premultiplied; unchanged regions skip blending; zero runs are skipped; 253 frames match fresh composition; 4000 random frames match the scalar loop"); +#else + puts("PASS (scalar row path): published pixels are premultiplied; unchanged regions skip blending; zero runs are skipped; 253 frames match fresh composition; 4000 random frames match the scalar loop"); +#endif return 0; } ''' @@ -166,8 +401,11 @@ (folder / 'ass' / 'ass.h').write_text(HEADER) source, binary = folder / 'test.c', folder / 'test' source.write_text(SOURCE) - subprocess.run([*shlex.split(os.environ.get('HOST_CC', 'cc')), '-std=gnu99', '-O2', - '-Wall', '-Wextra', '-Werror', '-Wno-unused-parameter', - '-I' + str(folder), '-I' + str(ROOT / 'src'), - str(source), '-o', str(binary)], check=True) - subprocess.run([str(binary)], check=True) + # The row path the host's compiler selects, then the scalar one (a host without NEON builds the + # same program twice). + for flags in ([], ['-DPLX_ASS_NO_NEON']): + subprocess.run([*shlex.split(os.environ.get('HOST_CC', 'cc')), '-std=gnu99', '-O2', + '-Wall', '-Wextra', '-Werror', '-Wno-unused-parameter', *flags, + '-I' + str(folder), '-I' + str(ROOT / 'src'), + str(source), '-o', str(binary)], check=True) + subprocess.run([str(binary)], check=True) diff --git a/docs/agent-reference.md b/docs/agent-reference.md index b3167b61b7..8febbcc620 100644 --- a/docs/agent-reference.md +++ b/docs/agent-reference.md @@ -1160,7 +1160,8 @@ which the linking section explains is load-bearing rather than tidy. **Subtitles (summary — `docs/ass-subtitles.md` is the deep-dive; `rust-modules/media/src/player/CLAUDE.md` has the producers).** The client draws subtitles itself; the TV's subtitle engine is unreachable in -buffer-feed. Every TEXT subtitle renders through the one bundled libass worker (`player::ass`), +buffer-feed. Every TEXT subtitle renders through the one bundled libass renderer (`player::ass`: a +primary worker, plus up to two helpers while a moving cue is pipelined, each with its own libass instance), whether it is authored ASS/SSA, an embedded SubRip/WebVTT/mov_text/SAMI track or a sidecar `.srt`/`.vtt`: a text track gets a header `text_ass::header` synthesizes from the viewer's size, position and colour picks, and its cues become ASS events (`text_ass::convert`, `ass_source::push_text`). Such a source has @@ -1617,6 +1618,20 @@ every thin back-edge with `file:line` (the work list for breaking it up); `--dot Background notifications revoke the borrowed Wayland handles and block presentation, including idle keepalives and queued uploads. DID foreground reacquires the handles and invalidates the UI before rendering resumes. A failed or non-Wayland query leaves both handles null. +- **The compositor multiplies our surface by its alpha, so the frame over the bound video plane is + straightened** (`rust-modules/gfx/src/gfx/straighten.rs`, the module doc has the measurement, the + buffer-age argument and the cost). The framebuffer holds PREMULTIPLIED colour over the player's + transparent clear; the platform composites it as `ui.rgb * ui.a + video * (1 - ui.a)`. So the + frame's last draw on a frame that drew anything while `Player::video_plane_bound` is `rgb / a` + (alpha unchanged, blending off) over the boxes of the surface the frame drew into and nowhere else + (every draw call in gfx declares its extent, `Extent`; a draw with none is the whole surface; at + most 16 disjoint boxes, because a pixel two quads cover would be divided twice), in place through `GL_ARM_shader_framebuffer_fetch` / + `GL_EXT_shader_framebuffer_fetch` and through a copy of the frame when the driver has neither (the + simulator): `present_and_swap` calls it after the dev capture's read and before the swap. A bound + frame must therefore stay a COMPLETE redraw from a clear: nothing may blend premultiplied colour + over pixels a buffer kept. The simulator stands in for the compositor (`draw_under`, straight + alpha, over the decoded picture or black) on those frames. The boot log's `straighten:` line says + which program the device took. - **Deploy uses a tmp+mv dance** (`plxnative.new` → `mv`) so scp succeeds while the old binary is still executing (avoids `ETXTBSY`). The TV drops to standby after a few idle minutes, so a deploy can die mid-scp — when scripting around `make deploy`, md5-compare local vs on-TV binary after @@ -2355,7 +2370,9 @@ path. Never run only this one before a release. `tests/README.md` has the tier t prove. - **Screen capture:** `tools/capture-screen.sh [out.png] [DISPLAY|VIDEO|GRAPHIC]` grabs the panel output. `DISPLAY` = video plane + UI composited (use this); `VIDEO`-only failing with "no - signal state" is itself a diagnostic that nothing is decoded on the video plane. + signal state" is itself a diagnostic that nothing is decoded on the video plane. `GRAPHIC` of a + frame over the bound video plane holds STRAIGHT colour (the straighten pass divided it by its alpha, + so a translucent pixel reads brighter than what the app drew); only `DISPLAY` is the picture. - **Perf gates:** `./tests/run.py --fps` runs the UI-tier FPS regression scenes (gates per scene in `tests/manifest.json`; `--fps-player` adds the player tier), asserting the app's once/sec heartbeat. **Six assertions in three families. Three RATE gates — and picking the wrong one is how a frozen diff --git a/docs/ass-subtitles.md b/docs/ass-subtitles.md index c333a9c37d..ddae1c4724 100644 --- a/docs/ass-subtitles.md +++ b/docs/ass-subtitles.md @@ -50,14 +50,17 @@ libass structures through the same headers used to compile that library. No firm new Starfish API is required. Pins, checksums, license notices and corresponding source archives are part of the normal package/source-bundle workflow. -The facade's ABI is 4 (`plx_ass_abi_version()`; `include/ass.h`). Two callers check it and refuse any +The facade's ABI is 5 (`plx_ass_abi_version()`; `include/ass.h`). Two callers check it and refuse any other number: `player/ass.rs` for the subtitles, and `plx_gfx::rtl` for the UI's Arabic and Hebrew -text through the `plx_ass_text_*` calls ABI 4 added (`src/ass_text.c`). Besides the load/render calls +text through the `plx_ass_text_*` calls ABI 4 added (`src/ass_text.c`). ABI 5 let `plx_ass_add_font` +run after `plx_ass_load` and lifted its 64 MiB limit (issue #1328, "Attached fonts" below), and +`plx_ass_add_font_borrowed` hands libass a font by reference instead of copying it. Besides the load/render calls it sets track features (`plx_ass_set_features`, Unicode line breaking), the libass cache bounds -(`plx_ass_set_cache_limits`) and extra line spacing (`plx_ass_set_line_spacing`), reports counters +(`plx_ass_set_cache_limits`) and extra line spacing (`plx_ass_set_line_spacing`), reports counters and the last render's phase timing (`plx_ass_stats`), and replaces the loaded track in place (`plx_ass_replace_track`). The renderer scans its fonts (`ass_set_fonts`) only on its first load; a replaced track keeps every opened face -and does not extract its own `[Fonts]` section again. The engine uses the replace for two +and does not extract its own `[Fonts]` section again, and a font added after the load is opened +alone at the next frame (libass's `ass_update_embedded_fonts`), so `font_scans` stays 1. The engine uses the replace for two cases: sliding the window of a long sidecar script, and a restyle (a Text-origin source whose synthesized header or script changed with the viewer's text size or position, same fonts): the track is replaced, the held events are appended again and the line spacing of the new header's size @@ -81,6 +84,42 @@ are uncovered because Noto Sans ships them in Regular only: Yi (`ii`), Avestan ( traditional Mongolian (`mn`, whose Cyrillic form is Inter's). U+2060 is not carried by any face: libass skips the default-ignorables (U+200B-200F, U+2060-206F) before it looks a glyph up. +Attached fonts. A file may attach far more font bytes than a renderer needs (the report in #1328: 17 +fonts, 83 MB; the opening's five karaoke faces sat after ~32 MB of faces no style names and before +small faces that still fit, so the old 32 MiB cap dropped exactly the five the opening needed and +kept twelve it did not), and no byte cap +decides what a track can draw: the demuxer takes every font attachment out of the format as it opens (`detach_fonts`, which also frees +libavformat's `codecpar` copy) and hands them over with the headers (`collect_manifest`), the +source store keeps the file's list whole (a same-file reopen keeps the `Arc` it holds, so the +renderer's same-fonts check is a pointer compare), and the renderer hands libass the faces the track +NAMES. With no font provider libass selects a face by the requested family, then "Inter", then the +CJK file, so an attached face that no requested name matches is never drawn and its bytes would be +dead weight. `player/ass/font_names.rs` (pure) reads each attachment's sfnt `name` table (family 1, +typographic family 16, WWS family 21, full 4, PostScript 6, on every platform, plus the CFF font name, +TrueType Collections included) and selects a face when a requested name equals one of them under ASCII +case folding, the rule being a superset of libass's own matching so it cannot hide text. The requested names are the Fontname of every `Style:` line of the +header (a sidecar script: its whole text, windowed or not), the always-requested "Inter" and "Arial", +and every `\fn` override, which `Native::append` reads in each event packet before `plx_ass_chunk` and +adds the face for then. A face shares its family with others: all of them are added together, in +container order, so libass's weight scoring and add-order tie break are what loading every font would +give. A font that cannot be read is added (unknown means needed); a byte-identical second copy is not. +A restyle of a text track reads the new header the same way. The event log says how many the renderer +holds, numbers only: `ass: embedded ... fonts=17 font_bytes=...` when the file's inventory is read, +and `ass: renderer fonts=5/17 font_bytes=...` for each renderer (again when a later event adds a face). +What is resident is the file's attachments as libavformat holds them (the Matroska demuxer's own +element buffer, one copy per attachment for as long as the file is open, and with the side reader, +which does not run `avformat_find_stream_info`, the stream's internal codec context's too; both are +private to libavformat) and the store's copy of them, one `Arc<[u8]>` per font. libass is handed no +copy: `Native::font_shared` gives it the store's buffer by reference (`plx_ass_add_font_borrowed`, which +the renderer keeps alive until after `plx_ass_destroy`), so a face a track names is resident once +however many renderers draw with it. The demuxer +frees its own `codecpar->extradata` copy of every attachment the moment `avformat_open_input` returns +(`detach_fonts`). What libass still copies per renderer are the bundled faces (`fonts::bundled`) and a +font the script's own `[Fonts]` section carries (libass extracts those itself). Measured on the host with +a 6 x 8 MiB fixture in a fresh process (`make check-ffmpeg`, `ff_font_memory_tests.rs`; `make check-ass`, +`player/ass/tests.rs`), the main demuxer holds the fonts 1x beside the store's 1x (3x and 1x before) and +a renderer over them holds 9 MiB (57 MiB before). + The demuxer copies ASS headers, complete timed packets and font attachments out of FFmpeg. The source store removes trailing zero terminators from headers and packets. A zero byte inside a header makes the track invalid. The raw byte limits apply before trimming. @@ -114,11 +153,53 @@ retains the entire script. It also incorporates the video's font attachments, in those arrive after the external file. Source reuse includes server identity, stream identity, delivery key and codec. Backward seeks and Off→On do not re-download a loaded script. -`player::ass` owns the native objects on one worker. A published frame carries the window of +`player::ass` owns the native objects on its workers: one primary, started by the first request, and +up to two helpers. A published frame carries the window of playheads it stays right for, so a UI tick inside the window enqueues nothing; the one pending slot holds either a Normal job (the screen needs a frame) or an Ahead job (the next boundary, rendered early and promoted by the first tick at or after it). A completed frame is accepted only -for the current source/selection epoch. The UI never +for the current source/selection epoch. + +A MOVING cue (karaoke, `\t`, `\move`, `\fad`, an Effect) is right for one playhead only, so a frame +asked for when the screen needs it is shown one render late (50-150 ms on the television, issue +#1328). While the player is running the mailbox instead PLANS the frames (`player::ass::mailbox`): +it hands out `Pipe` jobs for future playheads on a grid of one `step` (what a render costs over the +workers, scaled by the playback speed, never under one video frame or 33 ms, never over 250 ms), the +first a `lead` ahead of the playhead (a render, a quarter more, and a UI tick), holds each frame +that lands, and a tick shows the held frame with the greatest playhead not after its own: never +early, at most a step old. An event boundary inside a moving stretch (the stress fixture's opening has +one every ~19 ms) does not end a frame or take the plan off its grid; the plan resolves it at its own +step, and only the boundary that ends the stretch is rendered at its own playhead (a frame the worker +reports as `moves_on` stands until a newer one lands or its age runs out). Nothing is planned past the horizon (the lead and a step per worker), so +a worker with nothing to do parks. The pace (`running`, the applied speed, the video frame period) +and the tick interval reach the mailbox as the values of `ass::request`'s `Pace` +(`player::subtitle_pace`), never as a clock or a registry read inside it. A revision that only +APPENDED events keeps every planned frame for a playhead before the first event it adds +(`earliest_change_ms`, built on `append_from`) and replans from there; a revision that reaches the +playhead, a seek, a new source or size, a jump of the clock, a pause and a worker so slow that +nothing lands in time (a hole) each throw the plan away and ask for one Normal job, as a still cue does. +The cost the plan paces by is the worker-reported time of the moving frames (a job that opened or +re-synced the renderer is no sample), a running mean capped at 500 ms a sample, kept as the cost at +FULL size (a frame drawn smaller is taken back to it, see "A heavy moving stretch is drawn smaller"). + +One worker cannot make thirty frames a second from an 80 ms render, so the plan asks for the workers +that can (`workers_wanted`: the cost, at the size the stretch is drawn at, over the 33 ms floor) and the runtime starts helpers up to what the +device allows: the primary and a helper per other core, at most `MAX_ASS_RENDERERS` (3), and no more +helpers than a quarter of the memory available holds (the primary is already open when the verdict is +asked, so it is not charged again; `pool::decide`: each renderer is the fonts it copies, the +bundled faces (the attached ones are the store's, shared), plus a bitmap cache; `/proc/meminfo`'s `MemAvailable` is read +once for a selection, the first time a pipeline drawing at full size would want a helper (`pool_wanted`), and the verdict is logged as +`ass: pool renderers=N of M font_bytes=.. per_renderer=.. mem_available=.. budget=..`; a heavy file on +a short television runs the primary alone, and the primary always runs). Every worker +owns a whole libass instance (`Engine`: its own `ASS_Library`, renderer and track, which libass does not +let two threads share), fed the same `Source` snapshot, so a helper costs the bundled faces, the caches +and the events again, but not the attached fonts, which it references; it is started from the same `task::spawn` site as the primary, only while a moving cue is +planned, counted in the plan's pacing only once it has rendered a steady frame (its first job opens the +source and lands late), and gives its renderer back after 10 s with nothing to do. Each job carries the +newest frame that had landed (`Job::reference`) and the renderer shares the allocation of every region +that draws identically to one of that frame's (before looking at its own last frame), so a static sign +rendered by another engine is not uploaded again by the screen: the texture cache keys on the +allocation. The UI never waits for font parsing or rasterization. Native change detection avoids work for unchanged output. Overlapping libass images are composed in order into disjoint regions, with a maximum of 64 regions and one bounded pixel arena. Consecutive libass image identities also retain native region pixels, so a moving or karaoke @@ -129,6 +210,56 @@ player screen’s `ass_subtitles` cache reuses their GL uploads. Changed regions remaining textures; gaps, Off and unmount release them. The cache reports all retained texture bytes to the shared render accounting. +The pixels of a region are PREMULTIPLIED RGBA, published as composed (#1328): the facade blends +every layer in premultiplied space and does no straight-alpha conversion pass, which was about 45% of +the composite's time on the host and walked mostly transparent pixels. The screen draws them through +`Painter::tex_premultiplied` (`gfx::draw_tex_premultiplied`: `ONE, ONE_MINUS_SRC_ALPHA`, the tint's +colour scaled by its alpha after the painter's cascade), so the ink rung and a fade dim them as they +dim a straight texture. Image subtitles (PGS, VobSub) and glyph textures are uploaded straight and +stay on `Painter::tex`. + +## A heavy moving stretch is drawn smaller + +Helpers are limited by the set's cores and memory, so a stretch that one renderer cannot make thirty +frames a second of (the opening of the issue's file is ~85 ms a frame at 1080p on the television) is +drawn at 2/3 (720p) or 1/2 (540p) of the output size instead (#1328, `player::ass::scale`). libass is told +the smaller frame size and the SAME storage size (the facade's `ass_set_frame_size` takes the render +size, `ass_set_storage_size` the unchanged one, and the script's PlayRes is untouched), so the script +lays out identically and only the number of pixels +it draws changes; the regions come back in the smaller canvas and the screen draws each up to the +rectangle it covers on the output with the linear filter its upload sets (`Frame::output_edges`, +`ass_subtitles::region_rect`: both edges of a region are mapped, so regions that abut still abut). + +What it buys, measured on the host with the storage size fixed: libass takes 0.64x the time at 720p +and 0.49x at 540p, the composite 0.49x and 0.30x, and the bytes published 0.54x and 0.37x. Weighted by +the television's split of an 85 ms frame (65 ms in libass, 20 ms in the composite) a frame costs about +0.60 and 0.44 of one at full size, the constants `COST_RATIO_TWO_THIRDS` and `COST_RATIO_HALF`, rounded +down so a cost taken back to full size is never overstated. + +The choice is a pure `Governor` the mailbox owns, never a clock or a registry read. The mailbox keeps +the cost at FULL size (each sample is divided by its rung's ratio) and, on each tick of the running +pipeline, asks which rung the stretch should be drawn at: the largest whose projected step (cost x +ratio x playback speed / renderers) meets the target, 33 ms or one video frame, whichever is longer. +The renderers are the pool's verdict for the selection (`Runtime::renderers_decided`): nothing is +decided before the pool has said, and a stretch that full size would not want a helper for never +consults it and stays at full size. The first steady frame of a stretch decides at once (what the last +stretch cost says nothing of this one); inside one, a rung steps down when its projection misses the +target for 600 ms, and back up one rung only when the larger rung projects under 75% of the target for +3 s (both holds scale with the playback speed, and count only after six steady frames of the rung). A +step down within 10 s of a step up bars the next step up for 30 s, so a model that is wrong costs one +flap per half minute at most. + +Only the pipeline's `Pipe` jobs are drawn smaller, and the Normal job that fills a hole inside a reduced +stretch (a frame of another size would empty libass's caches, and the next one would pay for them +again). Any other Normal or Ahead job, the first frame of a stretch, a still cue and a paused player are +drawn at full size, and the still picture a reduced stretch hands over to is drawn again at full size by +one Normal job (the reduced frame stands until it lands, and is not withdrawn as "covered"). The rung +belongs to the selection it was decided for: a new epoch, source or size starts at full size again. The scale is not part of the `Key`, so a move never blanks the screen: the frame +on screen stays until a frame at the new scale lands, and each `Frame` says which scale it was drawn at. +libass empties its caches when the frame size changes, so a render at a new size is cold and is no cost +sample. Each move is logged once (`ass: render scale 1.00 -> 0.50 cost=.. renderers=.. rate=.. target=..`) +and every `assperf` line carries `scale=`. + The subtitle clock interpolates between the pipeline's sparse position callbacks, with at most 250 ms of extrapolation. Pause, seek and discontinuities re-anchor it. Output dimensions and the original coded video dimensions are passed separately to libass. The output canvas fits the @@ -190,26 +321,47 @@ Boot the fixture with `tools/tv-session.sh up --guest --mock --screen player=990 it never resolves a real Plex account. Configure and build the developer app against that mock first. Libass cost on the TV is read from the event log, not guessed. While a styled source renders, -`player::ass_perf` writes an `assperf: window_ms=.. calls=.. changed=.. p95_us=.. max_us=.. job_p95_us=.. -job_max_us=.. pub_bytes=.. up_bytes=.. jobs=N+A ahead_hit=.. nwait_p95_us=.. nwait_max_us=..` line when a job -arrives at least a second after the previous line (`window_ms` is the span its counts cover, +`player::ass_perf` writes an `assperf: w=.. window_ms=.. calls=.. changed=.. p95_us=.. max_us=.. job_p95_us=.. +job_max_us=.. pub_bytes=.. up_bytes=.. jobs=N+A ahead_hit=.. shown=.. late=.. stale=.. nwait_p95_us=.. +nwait_max_us=.. lib_p95_us=.. comp_p95_us=.. lib_cpu_p95_us=.. comp_cpu_p95_us=.. images_p95=.. +img_kpx_p95=.. scale=..` line per worker (`w` is the worker, 0 the primary) when a job +arrives at least a second after that worker's previous line (`window_ms` is the span its counts cover, including any idle stretch before it; read the counts as rates over it) (FFI render time, whole-job time including source load, append, flush-and-reload and window slide, `changed` and `pub_bytes` count frames published or held ahead whose pixels differ from the one on screen, bytes uploaded; `jobs` counts the worker's Normal and Ahead jobs -separately, `ahead_hit` the ahead frames the UI thread promoted, `nwait` how long a Normal job +separately (a frame the pipeline planned is an Ahead one), `ahead_hit` the boundary frames of still cues the UI thread +promoted, `shown` the frames of the moving cue's pipeline it promoted (updates a second) and `late` how many of those came more than a tick after +their playhead, `stale` the frames this worker rendered that were thrown away when they landed, `nwait` how long a Normal job waited behind an Ahead job already rendering; the partial window is written when the source is -deselected) and -one `assperf: worker tid=` line when the worker starts, which `tools/tv-sched-trace.sh --tid ` -follows. The UI-thread texture upload is the `assup` span on `FRAMEDROP spans=` (spans print only on +deselected). `up_bytes`, `ahead_hit`, `shown` and `late` are counted by the UI thread and read by the +primary's line alone (a helper's line has zeros there). The last six fields split `p95_us` between the two things `plx_ass_render` does, timed by +the facade itself with `clock_gettime` (`src/ass.c`, read back through `PlxAssStats.last_*`): `lib_p95_us` +is libass's own `ass_render_frame`, `comp_p95_us` is everything after it (bounds, regions and our +per-pixel composite (premultiplied RGBA, published as composed; eight pixels at a time with NEON, `src/ass_composite.h`, scalar on a target without it), 0 for a call that found the frame unchanged), both wall time over +the same calls as `p95_us`; `lib_cpu_p95_us` and `comp_cpu_p95_us` are those two phases on the worker +thread's CPU clock (`CLOCK_THREAD_CPUTIME_ID`), so a phase whose wall figure is far above its CPU one was +waiting for the CPU (descheduled, or blocked), not computing; `images_p95` and `img_kpx_p95` are the `ASS_Image`s libass produced for a composed frame and +the sum of their width x height in thousands of pixels, over the composed frames only; `scale` is the +smallest size a job of the window was drawn at (`1.00`, `0.67`, `0.50`: the side over the output's), so a +line is read at its own scale. Attribute a slow +render with these before optimising either half. The worker also writes +one `assperf: worker w= tid= nice=` line when it starts, which `tools/tv-sched-trace.sh --tid ` +follows. Workers run below the frame thread: the first at niceness +5, a helper at +10 +(`plx_base::task::lower_priority`, a per-thread `setpriority`, so only the worker is touched; the +line has no `nice=` where the platform has none or the kernel refused it). The UI-thread texture upload is the `assup` span on `FRAMEDROP spans=` (spans print only on `FRAMEDROP` lines, with `/tmp/plxnative-framedrop` armed); each changed region uploads with a full `glTexImage2D` (a bucketed `glTexSubImage2D` variant that reused the texture was measured on the TV and gave no gain, #735). Nothing is -logged while no styled source is selected, and a render costs one clock pair. The worker renders +logged while no styled source is selected, and a render costs one clock pair in Rust and six clock reads in the facade (three per clock). The worker renders only when the screen needs a frame (a new key, a playhead outside the published frame's validity -window, an animated cue) and renders the next boundary ahead of its time, so a steady cue costs no +window) or plans one (an animated cue, while playing: one frame per step, ahead of its playhead) and +renders the next boundary of a still cue ahead of its time, so a steady cue costs no job at all between boundaries (no line is written then), not one per UI frame. `python3 tests/fixtures/make_ass_stress.py ` builds the stress MKVs (Latin, CJK, Thai, Devanagari, Arabic, rapid, karaoke, long, 4K60; host `ffmpeg`/`ffprobe` only) to serve with `mock_pms.py --extra-media`, and the text-codec ones: `srt-*` (the scripts, rapid and long as SubRip in MKV, -plus `srt-markup` and `srt-overlap`), `mov_text-mixed.mp4` and `webvtt-mixed.mkv`. A text cue logs +plus `srt-markup` and `srt-overlap`), `mov_text-mixed.mp4` and `webvtt-mixed.mkv`. `--fonts-over-budget` builds +only `fonts-over-budget.mkv` (~410 MB, issue #1328): a dense 90 s karaoke opening with 17 attached fonts +(83 MB) whose opening faces all come after the first 32 MiB of font bytes (the cap the renderer +used to apply, removed by that issue), on 22 Mbit/s video. A text cue logs `sub cue [a..bms] len=N` (the length, never the words) for the selected track. Device checks must hold the TV lease and keep both the panel and sound off. Release the lease diff --git a/docs/egl-partial-update-and-damage.md b/docs/egl-partial-update-and-damage.md index f2bed968c6..33ef3026c7 100644 --- a/docs/egl-partial-update-and-damage.md +++ b/docs/egl-partial-update-and-damage.md @@ -382,7 +382,14 @@ shippable: hand-maintained list the motion capability exists to remove. Nothing measured here changes that. 7. **There is no pixel gate in this project.** A missed rect is a smear only a television can show you. The two captures in §3b are the entire visual verification apparatus that exists. -8. **The player route must stay out**, for the reason `system.rs:36-38` gives. +8. **The player route must stay out**, for the reason `system.rs:36-38` gives. That is also the + straighten pass's rule (`rust-modules/gfx/src/gfx/straighten.rs`, "Buffer age"): while the video + plane is bound, the frame's last draw rewrites every pixel the frame drew into (the boxes its draw + calls declared; the rest is the clear) from premultiplied to straight colour for the compositor, + so a bound frame must be a COMPLETE redraw from the clear. + Any scheme that restricts a bound frame's draws to a damage rect would also have to clear the + damaged region first and straighten exactly that region — a pixel carried over from an earlier + frame is already straight and the pass would divide it by its alpha a second time. 9. **ARM's own caveat applies and should be quoted at whoever proposes this**: *"Do not use either `EGL_KHR_partial_update` or `EGL_KHR_swap_buffers_with_damage` for applications that always re-render the whole frame, as there is an extra cost in doing so."* §2b says roughly half of a diff --git a/include/ass.h b/include/ass.h index 334ce3086e..986fd921a4 100644 --- a/include/ass.h +++ b/include/ass.h @@ -10,7 +10,11 @@ typedef struct PlxAss PlxAss; typedef struct { int x, y, width, height; size_t bytes; - const uint8_t *rgba; /* straight RGBA; valid until the next render or destroy */ + /* PREMULTIPLIED RGBA (each colour channel at most its alpha; a pixel of coverage 50% of + * colour (255, 128, 64) is 128, 64, 32, 128), composed in libass image-list order; valid until + * the next render or destroy. The caller draws it with a premultiplied blend (ONE, + * ONE_MINUS_SRC_ALPHA); drawn with a straight one it would be dark at every soft edge. */ + const uint8_t *rgba; } PlxAssBitmap; #define PLX_ASS_MAX_REGIONS 64 @@ -22,24 +26,62 @@ typedef struct { /* ABI 3: plx_ass_set_features, plx_ass_set_cache_limits, plx_ass_set_line_spacing, * plx_ass_replace_track and plx_ass_stats join the 2 surface. plx_ass_load now scans the fonts * (ass_set_fonts) only on a context's first load. - * ABI 4: the UI text surface, plx_ass_text_* (src/ass_text.c). */ -#define PLX_ASS_ABI_VERSION 4 + * ABI 4: the UI text surface, plx_ass_text_* (src/ass_text.c). + * ABI 5: plx_ass_add_font works after plx_ass_load too (the face is opened alone at the next + * frame) and has no byte limit but `int`; before, it refused once a track was loaded and past + * 64 MiB in all. plx_ass_add_font_borrowed adds a font the caller keeps, without copying it. + * PlxAssStats also carries the last plx_ass_render call's timing (wall and thread + * CPU) and image counts (last_*), the split a styled-subtitle frame's cost is attributed with. + * plx_ass_render publishes PREMULTIPLIED RGBA, where it published straight RGBA before: the straight + * conversion was a pass over every pixel of every changed region (mostly transparent ones) and the + * application's blend takes premultiplied colour as it is. The version stays 5 because ABI 5 has not + * shipped (main is on 4) and the one consumer, plx_media's `player::ass`, builds with this library + * from the same tree. */ +#define PLX_ASS_ABI_VERSION 5 +/* The last_* fields describe the most recent plx_ass_render call, in microseconds saturating at + * UINT32_MAX. last_libass_us is the wall time (CLOCK_MONOTONIC) of ass_render_frame alone; + * last_composite_us is the wall time of everything after it until the frame is returned (bounds, + * region placement and the per-pixel composite), 0 for a call that found + * the frame unchanged (it returns 0). last_libass_cpu_us and last_composite_cpu_us are the same + * two phases measured on CLOCK_THREAD_CPUTIME_ID: the CPU time the calling thread spent in them, + * so wall minus CPU is the time it was descheduled or blocked (the host's other work, a page + * fault) rather than computing. A clock that cannot be read counts 0. last_images and + * last_image_pixels (the sum of w*h) count the ASS_Images of the last frame that was composed, so + * an unchanged call leaves them as they were. All of them are 0 before the first render. Eight + * 32-bit fields come before the 64-bit one, so last_image_pixels sits on its natural 8-byte + * boundary (offset 32, size 40) on the ARM target and the host alike. */ typedef struct { - uint32_t library_fonts; /* fonts the library holds: added through plx_ass_add_font or extracted */ + uint32_t library_fonts; /* fonts the library holds: added through plx_ass_add_font or plx_ass_add_font_borrowed, or extracted from [Fonts] */ uint32_t font_scans; /* times the renderer opened every font (ass_set_fonts) */ uint32_t tracks; /* tracks loaded or replaced */ + uint32_t last_libass_us; + uint32_t last_composite_us; + uint32_t last_libass_cpu_us; + uint32_t last_composite_cpu_us; + uint32_t last_images; + uint64_t last_image_pixels; } PlxAssStats; unsigned plx_ass_abi_version(void); PlxAss *plx_ass_create(const char *default_font); void plx_ass_destroy(PlxAss *ctx); +/* Copy one font file into the library under `name`. -1 for a NULL or empty argument or a `len` + * past INT_MAX (libass takes an int), else 0. Valid before and after plx_ass_load: a font added + * after it is opened by libass alone, at the start of the next plx_ass_render that has events, and + * the faces already open are not scanned again (PlxAssStats.font_scans stays 1). A face that + * FreeType cannot open is skipped silently by libass; the copy is still held. */ int plx_ass_add_font(PlxAss *ctx, const char *name, const uint8_t *data, size_t len); +/* plx_ass_add_font without the copy: the library holds `data` itself, not a copy of it, so one + * buffer can back the same font in every context that is handed it. Same arguments, same -1s, same + * timing of the face scan. `data` must stay valid and unchanged until plx_ass_destroy of THIS context + * has returned (the renderer reads it for as long as it lives); destroy never frees it. */ +int plx_ass_add_font_borrowed(PlxAss *ctx, const char *name, const uint8_t *data, size_t len); int plx_ass_load(PlxAss *ctx, const uint8_t *data, size_t len, int script); /* Replace the loaded track on the same library and renderer, keeping every font face the - * renderer has opened. Fonts cannot change here: the new track's own [Fonts] section is NOT - * extracted, so it must repeat the first track's (a synthesized header has none; a window of the - * same script repeats the same section). Needs a first plx_ass_load. The next render returns a + * renderer has opened. The new track's own [Fonts] section is NOT extracted, so it must repeat the + * first track's (a synthesized header has none; a window of the same script repeats the same + * section); fonts the caller adds with plx_ass_add_font are unaffected. Needs a first plx_ass_load. The next render returns a * full frame, never 0 "unchanged". -1 leaves the context without a track; such a context takes only * another plx_ass_replace_track (plx_ass_load refuses once the fonts are scanned). */ int plx_ass_replace_track(PlxAss *ctx, const uint8_t *data, size_t len, int script); diff --git a/locales/be/widgets.json b/locales/be/widgets.json index f8d190d457..752f83e922 100644 --- a/locales/be/widgets.json +++ b/locales/be/widgets.json @@ -200,7 +200,6 @@ "widgets.ass.unavailable": "Стылізаваныя субцітры недаступныя ў гэтай усталёўцы", "widgets.ass.fonts_unavailable": "Не ўдалося загрузіць шрыфты субцітраў", "widgets.ass.init_failed": "Не ўдалося ініцыялізаваць стылізаваныя субцітры", - "widgets.ass.font_too_large": "Усталяваны шрыфт субцітраў завялікі", "widgets.ass.unreadable": "Не ўдалося прачытаць гэты файл стылізаваных субцітраў", "widgets.ass.font_name": "Няправільная назва шрыфту субцітраў", "widgets.ass.track_fonts": "Не ўдалося загрузіць шрыфты гэтых субцітраў", diff --git a/locales/ca/widgets.json b/locales/ca/widgets.json index 1b388bec87..1026a05765 100644 --- a/locales/ca/widgets.json +++ b/locales/ca/widgets.json @@ -196,7 +196,6 @@ "widgets.ass.unavailable": "Els subtítols amb estil no estan disponibles en aquesta instal·lació", "widgets.ass.fonts_unavailable": "No s'han pogut carregar les fonts dels subtítols", "widgets.ass.init_failed": "No s'han pogut inicialitzar els subtítols amb estil", - "widgets.ass.font_too_large": "La font de subtítols instal·lada és massa gran", "widgets.ass.unreadable": "No s'ha pogut llegir aquest fitxer de subtítols amb estil", "widgets.ass.font_name": "El nom d'aquesta font de subtítols no és vàlid", "widgets.ass.track_fonts": "No s'han pogut carregar les fonts d'aquest subtítol", diff --git a/locales/de/widgets.json b/locales/de/widgets.json index e23c36e1dc..b7984b85a8 100644 --- a/locales/de/widgets.json +++ b/locales/de/widgets.json @@ -192,7 +192,6 @@ "widgets.ass.unavailable": "Formatierte Untertitel werden in dieser Installation nicht unterstützt", "widgets.ass.fonts_unavailable": "Untertitelschriften konnten nicht geladen werden", "widgets.ass.init_failed": "Formatierte Untertitel konnten nicht initialisiert werden", - "widgets.ass.font_too_large": "Die installierte Untertitelschrift ist zu groß", "widgets.ass.unreadable": "Diese formatierte Untertiteldatei konnte nicht gelesen werden", "widgets.ass.font_name": "Dieser Name der Untertitelschrift ist ungültig", "widgets.ass.track_fonts": "Die Schriften dieses Untertitels konnten nicht geladen werden", diff --git a/locales/en/widgets.json b/locales/en/widgets.json index 59207c82df..f623750168 100644 --- a/locales/en/widgets.json +++ b/locales/en/widgets.json @@ -880,10 +880,6 @@ "value": "Couldn’t initialize styled subtitles", "description": "Message drawn in place of styled (ASS/SSA) subtitles when they cannot be shown." }, - "widgets.ass.font_too_large": { - "value": "The installed subtitle font is too large", - "description": "Message drawn in place of styled (ASS/SSA) subtitles when they cannot be shown." - }, "widgets.ass.unreadable": { "value": "This styled subtitle file could not be read", "description": "Message drawn in place of styled (ASS/SSA) subtitles when they cannot be shown." diff --git a/locales/es/widgets.json b/locales/es/widgets.json index 1316355c9c..823b7d722a 100644 --- a/locales/es/widgets.json +++ b/locales/es/widgets.json @@ -196,7 +196,6 @@ "widgets.ass.unavailable": "Los subtítulos con estilo no están disponibles en esta instalación", "widgets.ass.fonts_unavailable": "No se pudieron cargar las fuentes de los subtítulos", "widgets.ass.init_failed": "No se pudieron inicializar los subtítulos con estilo", - "widgets.ass.font_too_large": "La fuente de subtítulos instalada es demasiado grande", "widgets.ass.unreadable": "No se pudo leer este archivo de subtítulos con estilo", "widgets.ass.font_name": "El nombre de esta fuente de subtítulos no es válido", "widgets.ass.track_fonts": "No se pudieron cargar las fuentes de este subtítulo", diff --git a/locales/fr/widgets.json b/locales/fr/widgets.json index 7bd75a79d6..d06d0fb66f 100644 --- a/locales/fr/widgets.json +++ b/locales/fr/widgets.json @@ -196,7 +196,6 @@ "widgets.ass.unavailable": "Les sous-titres stylisés ne sont pas pris en charge dans cette installation", "widgets.ass.fonts_unavailable": "Impossible de charger les polices des sous-titres", "widgets.ass.init_failed": "Impossible d’initialiser les sous-titres stylisés", - "widgets.ass.font_too_large": "La police de sous-titres installée est trop volumineuse", "widgets.ass.unreadable": "Ce fichier de sous-titres stylisés n’a pas pu être lu", "widgets.ass.font_name": "Le nom de cette police de sous-titres est invalide", "widgets.ass.track_fonts": "Impossible de charger les polices de ce sous-titre", diff --git a/locales/hu/widgets.json b/locales/hu/widgets.json index eaf6a12ec7..bea4510800 100644 --- a/locales/hu/widgets.json +++ b/locales/hu/widgets.json @@ -192,7 +192,6 @@ "widgets.ass.unavailable": "Ebben a telepítésben a formázott feliratok nem támogatottak", "widgets.ass.fonts_unavailable": "A feliratbetűtípusok nem tölthetők be", "widgets.ass.init_failed": "A formázott feliratok nem inicializálhatók", - "widgets.ass.font_too_large": "A telepített feliratbetűtípus túl nagy", "widgets.ass.unreadable": "Ez a formázott feliratfájl nem olvasható", "widgets.ass.font_name": "Ennek a feliratbetűtípusnak a neve érvénytelen", "widgets.ass.track_fonts": "A felirat betűtípusai nem tölthetők be", diff --git a/locales/it/widgets.json b/locales/it/widgets.json index afa42ad214..e880fd6f76 100644 --- a/locales/it/widgets.json +++ b/locales/it/widgets.json @@ -196,7 +196,6 @@ "widgets.ass.unavailable": "Sottotitoli con stile non supportati in questa installazione", "widgets.ass.fonts_unavailable": "Impossibile caricare i font dei sottotitoli", "widgets.ass.init_failed": "Impossibile inizializzare i sottotitoli stilizzati", - "widgets.ass.font_too_large": "Il font dei sottotitoli installato è troppo grande", "widgets.ass.unreadable": "Impossibile leggere questo file di sottotitoli stilizzati", "widgets.ass.font_name": "Il nome di questo font dei sottotitoli non è valido", "widgets.ass.track_fonts": "Impossibile caricare i font di questo sottotitolo", diff --git a/locales/nl/widgets.json b/locales/nl/widgets.json index d5ee577578..ff6081fe0d 100644 --- a/locales/nl/widgets.json +++ b/locales/nl/widgets.json @@ -192,7 +192,6 @@ "widgets.ass.unavailable": "Opgemaakte ondertitels worden in deze installatie niet ondersteund", "widgets.ass.fonts_unavailable": "Kan lettertypen voor ondertitels niet laden", "widgets.ass.init_failed": "Kan opgemaakte ondertitels niet initialiseren", - "widgets.ass.font_too_large": "Het geïnstalleerde ondertitellettertype is te groot", "widgets.ass.unreadable": "Dit opgemaakte ondertitelbestand kon niet worden gelezen", "widgets.ass.font_name": "Deze naam van het ondertitellettertype is ongeldig", "widgets.ass.track_fonts": "Kan de lettertypen van deze ondertitel niet laden", diff --git a/locales/pl/widgets.json b/locales/pl/widgets.json index a18ec76486..d39e3126d2 100644 --- a/locales/pl/widgets.json +++ b/locales/pl/widgets.json @@ -200,7 +200,6 @@ "widgets.ass.unavailable": "Obsługa napisów z formatowaniem jest niedostępna w tej instalacji", "widgets.ass.fonts_unavailable": "Nie można wczytać czcionek napisów", "widgets.ass.init_failed": "Nie można zainicjować napisów z formatowaniem", - "widgets.ass.font_too_large": "Zainstalowana czcionka napisów jest za duża", "widgets.ass.unreadable": "Nie można odczytać pliku napisów z formatowaniem", "widgets.ass.font_name": "Nazwa czcionki napisów jest nieprawidłowa", "widgets.ass.track_fonts": "Nie można wczytać czcionek tych napisów", diff --git a/locales/pt/widgets.json b/locales/pt/widgets.json index b9867a1379..363803ce4e 100644 --- a/locales/pt/widgets.json +++ b/locales/pt/widgets.json @@ -196,7 +196,6 @@ "widgets.ass.unavailable": "O suporte para legendas com estilo não está disponível nesta instalação", "widgets.ass.fonts_unavailable": "Não foi possível carregar os tipos de letra das legendas", "widgets.ass.init_failed": "Não foi possível inicializar as legendas com estilo", - "widgets.ass.font_too_large": "O tipo de letra de legendas instalado é demasiado grande", "widgets.ass.unreadable": "Não foi possível ler este ficheiro de legendas com estilo", "widgets.ass.font_name": "O nome do tipo de letra destas legendas é inválido", "widgets.ass.track_fonts": "Não foi possível carregar os tipos de letra destas legendas", diff --git a/locales/ru/widgets.json b/locales/ru/widgets.json index 534b88fc52..36426fb148 100644 --- a/locales/ru/widgets.json +++ b/locales/ru/widgets.json @@ -200,7 +200,6 @@ "widgets.ass.unavailable": "В этой установке недоступны субтитры с оформлением", "widgets.ass.fonts_unavailable": "Не удалось загрузить шрифты субтитров", "widgets.ass.init_failed": "Не удалось инициализировать субтитры с оформлением", - "widgets.ass.font_too_large": "Установленный шрифт субтитров слишком велик", "widgets.ass.unreadable": "Не удалось прочитать файл субтитров с оформлением", "widgets.ass.font_name": "Некорректное имя шрифта субтитров", "widgets.ass.track_fonts": "Не удалось загрузить шрифты этих субтитров", diff --git a/rust-modules/base/src/lib.rs b/rust-modules/base/src/lib.rs index 232b6154d2..12ca51c1e9 100644 --- a/rust-modules/base/src/lib.rs +++ b/rust-modules/base/src/lib.rs @@ -33,6 +33,8 @@ pub mod hold; // `HOLD_ROWS`, `clamp_hold`: the rows a paged store keeps, which pub mod devtrig; // the /tmp trigger PRIMITIVES (`flag`, `read`, `latched_flag!`, `no_wan`...) pub mod diag; // the frame instruments (`heartbeat`, `spans`) and the zlib helper #[cfg(any(test, feature = "test-support"))] +pub mod testheap; // the C heap's live bytes and a fresh process to measure in: the tests that pin how many copies of a buffer a path keeps +#[cfg(any(test, feature = "test-support"))] pub mod testlock; // one lock for every test that touches a process-global #[cfg(any(test, feature = "test-support"))] pub mod testnet; // the accepts for a loopback test server: the one that fixes the socket's mode, and the bounded ones diff --git a/rust-modules/base/src/task.rs b/rust-modules/base/src/task.rs index 3bfc448241..6ff1eb1394 100644 --- a/rust-modules/base/src/task.rs +++ b/rust-modules/base/src/task.rs @@ -457,6 +457,54 @@ impl Beside<'_, T> { } } +/// This thread's kernel thread id: what `ps -L`, `/proc//task/` and `tools/tv-sched-trace.sh +/// --tid` name. `None` off Linux and Android, where there is none to name. +/// +/// The raw system call, which is what `gettid(2)` is: glibc grew the wrapper in 2.30, and the +/// televisions' libc has it only from webOS 7 (`tools/fwcompat.py --lib libc.so.6 --grep gettid`), +/// so naming `gettid` would stop the binary starting on older sets. `syscall` itself is in every +/// release's libc. +#[cfg(any(target_os = "linux", target_os = "android"))] +pub fn thread_id() -> Option { + // SAFETY: `SYS_gettid` takes no arguments and cannot fail. + u32::try_from(unsafe { libc::syscall(libc::SYS_gettid) }).ok() +} + +/// See the Linux definition. +#[cfg(not(any(target_os = "linux", target_os = "android")))] +pub fn thread_id() -> Option { + None +} + +/// Run the calling thread at niceness `nice` (0 is the default a thread starts at, 19 the lowest +/// priority; a level past either end is cut to it), so the threads that keep the frame loop on +/// time win a core from it. Returns whether it took effect. +/// +/// Linux keeps a niceness per kernel thread (NPTL threads are not "processes" in this one respect), +/// so `setpriority(PRIO_PROCESS, tid, nice)` lowers this thread and no other, which is what a +/// worker needs: the ASS render workers run at `+5` (the first) and `+10` (a helper), below the +/// frame thread and the playback threads that stay at 0, on a four-core set where a moving cue's +/// workers would otherwise take the cores the frame loop needs. Lowering one's own priority is +/// always permitted; it is the other way (a worker that asks for less than the niceness it has) +/// that the kernel refuses without `CAP_SYS_NICE`, which is why the result is a `bool` and not a +/// panic: a refusal only leaves the thread where it was. +/// +/// Off Linux and Android there is no per-thread niceness and the call does nothing (`false`): this +/// is a platform difference in the OS interface, not a test fork. +#[cfg(any(target_os = "linux", target_os = "android"))] +pub fn lower_priority(nice: i32) -> bool { + let Some(tid) = thread_id() else { return false }; + // SAFETY: plain integers; `PRIO_PROCESS` is `c_int` in the binding and `c_uint` (glibc's + // `__priority_which_t`) in the signature, which `as _` settles for either libc. + unsafe { libc::setpriority(libc::PRIO_PROCESS as _, tid as libc::id_t, nice.clamp(0, 19)) == 0 } +} + +/// See the Linux definition. +#[cfg(not(any(target_os = "linux", target_os = "android")))] +pub fn lower_priority(_nice: i32) -> bool { + false +} + /// A join this long parked the SDL loop for ~15 frames — the shortest stall a person reads as a /// freeze rather than a stutter. const STALL_MS: u64 = 250; @@ -571,6 +619,65 @@ mod tests { assert_eq!(rx.recv().unwrap().trim_end(), "comm-probe"); } + /// The thread's own niceness, which is what a scheduler trace reads: Linux keeps a niceness + /// per kernel thread, and `setpriority(PRIO_PROCESS, tid)` reaches that thread alone. + #[cfg(target_os = "linux")] + fn nice_of(tid: u32) -> i32 { + // getpriority's -1 is a legitimate niceness, so a failure is told by errno; a test thread + // that has just read its own tid cannot fail it. + unsafe { *libc::__errno_location() = 0 }; + let n = unsafe { libc::getpriority(libc::PRIO_PROCESS as _, tid as libc::id_t) }; + assert_eq!(unsafe { *libc::__errno_location() }, 0, "getpriority of a live thread"); + n + } + + /// A worker that lowers its priority is niced and the thread that started it is not: the + /// niceness is per kernel thread, so the ASS workers' `+5`/`+10` never reach the frame thread. + #[cfg(target_os = "linux")] + #[test] + fn lowering_priority_nices_the_calling_thread_alone() { + let parent = super::thread_id().expect("a Linux thread has an id"); + let parent_before = nice_of(parent); + let (tx, rx) = mpsc::channel(); + let h = super::spawn("nice-probe", move || { + let me = super::thread_id().expect("a Linux thread has an id"); + let before = nice_of(me); + let applied = super::lower_priority(before + 7); + tx.send((before, applied, nice_of(me))).unwrap(); + }) + .expect("spawn"); + let (before, applied, after) = rx.recv().unwrap(); + h.join().unwrap(); + assert!(applied, "lowering a thread's own priority is never refused"); + assert_eq!(after, (before + 7).min(19)); + assert_eq!(nice_of(parent), parent_before, "the spawning thread keeps its niceness"); + } + + /// The level is a niceness: past the ends it is cut to them, not refused. + #[cfg(target_os = "linux")] + #[test] + fn a_level_past_the_lowest_priority_is_cut_to_it() { + let (tx, rx) = mpsc::channel(); + let h = super::spawn("nice-clamp", move || { + let me = super::thread_id().expect("a Linux thread has an id"); + tx.send((super::lower_priority(1_000), nice_of(me))).unwrap(); + }) + .expect("spawn"); + let (applied, nice) = rx.recv().unwrap(); + h.join().unwrap(); + assert!(applied); + assert_eq!(nice, 19); + } + + /// Off Linux there is no per-thread niceness to set (the host simulator is the one place the + /// ASS workers run there): the call says so and changes nothing. + #[cfg(not(any(target_os = "linux", target_os = "android")))] + #[test] + fn lowering_priority_reports_that_it_did_nothing_off_linux() { + assert_eq!(super::thread_id(), None); + assert!(!super::lower_priority(5)); + } + /// [`super::MainThread`] earns its keep entirely by NOT being `Send` — that absence is what /// makes a closure which captured one impossible to hand to `spawn`. An absent impl is /// invisible to ordinary code, so detect it: the inherent const applies only when `T: Send`, diff --git a/rust-modules/base/src/testheap.rs b/rust-modules/base/src/testheap.rs new file mode 100644 index 0000000000..b9c9144882 --- /dev/null +++ b/rust-modules/base/src/testheap.rs @@ -0,0 +1,148 @@ +//! The process's live heap bytes, read from the C allocator, for tests that pin how many copies of +//! a buffer a path keeps (issue #1328: a file's font attachments, copied by FFmpeg, by the store +//! and by libass). +//! +//! A counting `#[global_allocator]` sees Rust's allocations only, and the copies that matter here +//! are made by FFmpeg and libass, which call `malloc` themselves. So the meter asks the platform's +//! malloc what it holds: `malloc_zone_statistics` on macOS (every zone, `size_in_use`) and +//! `mallinfo2` on glibc (`uordblks` + `hblkhd`: the arenas and the mmapped chunks). It is the bytes +//! the program has asked for and not freed, not the resident set: the same number on every host +//! whatever the pager does, which is what a copy count needs. A platform with neither reports +//! `None`, and a test that needs the meter says it could not run rather than passing. +//! +//! A test measuring a path must not share its process with other tests (a parallel test's +//! allocations are in the number): [`in_fresh_process`] runs the test binary again with only that +//! test selected, and the child reports its numbers with [`report`]. The pattern, in the test: +//! +//! ```ignore +//! let numbers = if testheap::is_child() { measure_and_report(); return; } +//! else { testheap::in_fresh_process("module::the_test") }; +//! assert!(numbers["peak"] < budget); +//! ``` + +use std::collections::HashMap; +use std::sync::atomic::{AtomicBool, AtomicUsize, Ordering}; +use std::sync::Arc; +use std::time::Duration; + +/// The environment variable a fresh process is started with, naming the test it runs. +const CHILD_ENV: &str = "PLX_TESTHEAP_CHILD"; +/// The prefix of a line [`report`] prints and [`in_fresh_process`] reads. +const LINE_PREFIX: &str = "TESTHEAP "; + +/// Bytes the C allocator holds live across the whole process, or `None` on a platform whose +/// allocator cannot be asked. +pub fn live_bytes() -> Option { + #[cfg(target_os = "macos")] + { + // SAFETY: a zeroed statistics struct is valid, and a null zone means "every zone". + let mut stats: libc::malloc_statistics_t = unsafe { std::mem::zeroed() }; + unsafe { libc::malloc_zone_statistics(std::ptr::null_mut(), &mut stats) }; + Some(stats.size_in_use) + } + #[cfg(all(target_os = "linux", target_env = "gnu"))] + { + // SAFETY: `mallinfo2` takes no argument and returns a plain struct. + let info = unsafe { libc::mallinfo2() }; + Some(info.uordblks + info.hblkhd) + } + #[cfg(not(any(target_os = "macos", all(target_os = "linux", target_env = "gnu"))))] + { + None + } +} + +/// The most live bytes seen while `work` ran, sampled every 100 microseconds from a thread of its +/// own, with the work's result. A buffer that lives for less than a sample can be missed, so the +/// value is a floor on the true peak; the copies this exists to count live for seconds. +pub fn peak_during(work: impl FnOnce() -> R) -> (R, usize) { + let stop = Arc::new(AtomicBool::new(false)); + let peak = Arc::new(AtomicUsize::new(live_bytes().unwrap_or(0))); + let sampler = { + let (stop, peak) = (Arc::clone(&stop), Arc::clone(&peak)); + crate::task::spawn("testheap-sampler", move || { + while !stop.load(Ordering::Acquire) { + peak.fetch_max(live_bytes().unwrap_or(0), Ordering::Relaxed); + std::thread::sleep(Duration::from_micros(100)); + } + peak.fetch_max(live_bytes().unwrap_or(0), Ordering::Relaxed); + }) + .expect("the heap sampler thread starts") + }; + let result = work(); + stop.store(true, Ordering::Release); + sampler.join().expect("the heap sampler does not panic"); + (result, peak.load(Ordering::Relaxed)) +} + +/// Is this process the fresh one [`in_fresh_process`] started? +pub fn is_child() -> bool { + std::env::var_os(CHILD_ENV).is_some() +} + +/// Print one number for the parent test to read. Only meaningful in a child. +pub fn report(key: &str, value: u64) { + println!("{LINE_PREFIX}{key}={value}"); +} + +/// Run `test` (the full path libtest lists, e.g. `player::ass::tests::name`) alone in a fresh +/// process of this same test binary, `--ignored` included so an ignored test can use it, and +/// return the numbers it [`report`]ed. Panics, with the child's output, when the child fails. +/// The child inherits this process's environment (the libraries a host test finds by +/// `PLXNATIVE_APP_DIR` or `PLX_FFMPEG_DIR`), plus `PLX_TESTHEAP_CHILD`. +pub fn in_fresh_process(test: &str) -> HashMap { + in_fresh_process_with_env(test, &[]) +} + +/// [`in_fresh_process`] with more environment variables for the child. A test that measures what a +/// path allocates hands its input over this way, made by the parent: whatever the child allocated to +/// MAKE the input would otherwise be in its heap, and on macOS a freed large block that malloc keeps +/// cached is handed back, rounded up, to the next large allocation, so the number moves with it. +pub fn in_fresh_process_with_env(test: &str, env: &[(&str, &std::ffi::OsStr)]) -> HashMap { + let exe = std::env::current_exe().expect("the test binary has a path"); + let out = std::process::Command::new(exe) + .args(["--exact", test, "--include-ignored", "--test-threads=1", "--nocapture"]) + .env(CHILD_ENV, test) + .envs(env.iter().copied()) + .output() + .expect("the test binary starts again"); + let text = String::from_utf8_lossy(&out.stdout); + assert!( + out.status.success(), + "the fresh process for {test} failed ({:?})\n--- stdout\n{text}\n--- stderr\n{}", + out.status, + String::from_utf8_lossy(&out.stderr) + ); + let numbers: HashMap = text + .lines() + // `--nocapture` puts libtest's own "test name ... " in front of the first report. + .filter_map(|line| line.find(LINE_PREFIX).map(|at| &line[at + LINE_PREFIX.len()..])) + .filter_map(|pair| { + let (key, value) = pair.split_once('=')?; + Some((key.to_owned(), value.trim().parse().ok()?)) + }) + .collect(); + assert!(!numbers.is_empty(), "the fresh process for {test} reported nothing\n--- stdout\n{text}"); + numbers +} + +#[cfg(test)] +mod tests { + use super::*; + + /// The meter sees a C allocation and a Rust one, and sees them go. + #[test] + fn the_meter_counts_what_is_held_and_forgets_what_is_freed() { + let Some(before) = live_bytes() else { return }; + let held = vec![1u8; 64 << 20]; + let with = live_bytes().unwrap(); + assert!(with >= before + (60 << 20), "a 64 MiB vector is held: {before} -> {with}"); + drop(held); + let (_, peak) = peak_during(|| { + let transient = vec![2u8; 32 << 20]; + std::thread::sleep(Duration::from_millis(20)); + std::hint::black_box(&transient); + }); + assert!(peak >= before + (30 << 20), "the sampler saw a 32 MiB transient: {before} -> {peak}"); + } +} diff --git a/rust-modules/gfx/src/gfx.rs b/rust-modules/gfx/src/gfx.rs index a76b613e4e..bc12222d36 100644 --- a/rust-modules/gfx/src/gfx.rs +++ b/rust-modules/gfx/src/gfx.rs @@ -19,6 +19,10 @@ use crate::overdraw::{ pub mod backdrop; mod geom; pub mod scrim; +/// The straight-alpha compositor's half of the frame: the pass that un-premultiplies the surface over +/// the bound video plane, and the ledger of what every draw wrote that decides whether a frame needs +/// it and where. +pub mod straighten; /// Draw-phase instrumentation (`phase`, the GPU timer / HWCNT / CPU profilers) and the LOAD DIAL's /// published state. pub mod profile; @@ -193,6 +197,13 @@ const FS_RAMP: &CStr = glsl!("shaders/fs_ramp.frag"); const FS_TEX: &CStr = glsl!("shaders/fs_tex.frag"); const VS_ART_SCRIM: &CStr = glsl!("shaders/vs_art_scrim.vert"); const FS_ART_SCRIM: &CStr = glsl!("shaders/fs_art_scrim.frag"); +/// The straighten pass (`gfx/straighten.rs`): one vertex shader that stretches the unit quad over a box +/// of the viewport and one fragment source in three variants, picked by a `#define` prefix. The fetch variants read the frame in place; the +/// copy variant samples a copy of it. +const VS_STRAIGHTEN: &CStr = glsl!("shaders/vs_straighten.vert"); +const FS_STRAIGHTEN_ARM: &CStr = glsl!("shaders/fs_straighten.frag", "#define PLX_STRAIGHTEN_ARM\n"); +const FS_STRAIGHTEN_EXT: &CStr = glsl!("shaders/fs_straighten.frag", "#define PLX_STRAIGHTEN_EXT\n"); +const FS_STRAIGHTEN_COPY: &CStr = glsl!("shaders/fs_straighten.frag"); const GL_VERTEX_SHADER: c_uint = 0x8B31; const GL_FRAGMENT_SHADER: c_uint = 0x8B30; const GL_COMPILE_STATUS: c_uint = 0x8B81; @@ -210,8 +221,6 @@ const GL_DITHER: c_uint = 0x0BD0; const GL_ONE: c_uint = 0x0001; const GL_SRC_ALPHA: c_uint = 0x0302; const GL_ONE_MINUS_SRC_ALPHA: c_uint = 0x0303; -#[cfg(feature = "hostsim")] -const GL_ONE_MINUS_DST_ALPHA: c_uint = 0x0305; const GL_TEXTURE_2D: c_uint = 0x0DE1; const GL_TEXTURE0: c_uint = 0x84C0; const GL_TEXTURE1: c_uint = 0x84C1; @@ -257,7 +266,10 @@ extern "C" { stride: c_int, pointer: *const c_void, ); - fn glDrawArrays(mode: c_uint, first: c_int, count: c_int); + /// The raw import. Nothing but [`glDrawArrays`] below (and the straighten pass's own quads over the + /// boxes the frame drew into, which are not content) calls it: see that wrapper. + #[link_name = "glDrawArrays"] + fn glDrawArrays_raw(mode: c_uint, first: c_int, count: c_int); fn glBindTexture(target: c_uint, texture: c_uint); fn glActiveTexture(texture: c_uint); fn glEnable(cap: c_uint); @@ -265,7 +277,9 @@ extern "C" { fn glScissor(x: c_int, y: c_int, w: c_int, h: c_int); fn glBlendFuncSeparate(src_rgb: c_uint, dst_rgb: c_uint, src_a: c_uint, dst_a: c_uint); fn glClearColor(r: f32, g: f32, b: f32, a: f32); - fn glClear(mask: c_uint); + /// The raw import: only [`glClear`] below calls it. + #[link_name = "glClear"] + fn glClear_raw(mask: c_uint); #[cfg(feature = "devtriggers")] fn glFinish(); #[cfg(feature = "devtriggers")] @@ -293,7 +307,9 @@ extern "C" { // UI self-capture (the "cap_*" section at the bottom of this file) fn glGenFramebuffers(n: c_int, ids: *mut c_uint); fn glDeleteFramebuffers(n: c_int, ids: *const c_uint); - fn glBindFramebuffer(target: c_uint, framebuffer: c_uint); + /// The raw import: only [`glBindFramebuffer`] below calls it. + #[link_name = "glBindFramebuffer"] + fn glBindFramebuffer_raw(target: c_uint, framebuffer: c_uint); fn glFramebufferTexture2D( target: c_uint, attachment: c_uint, @@ -325,6 +341,49 @@ extern "C" { fn glViewport(x: c_int, y: c_int, w: c_int, h: c_int); } +/// **Every draw call this module issues, with the extent it writes.** `glDrawArrays` is shadowed by this +/// wrapper so that every call site keeps its spelling, adds one argument and books itself in +/// [`straighten::note_draw`]: the straighten pass draws over the boxes the frame drew into and +/// nowhere else, so a draw that did not say where it lands would be a translucent pixel the pass +/// skipped. The `extent` is [`set_rect`]'s answer for every quad placed by a rect uniform (the numbers +/// the vertex shader places the quad with, not the logical rect it was asked for) and +/// [`straighten::Extent::Target`], the whole target, for a draw with none. A test pins that the raw +/// import is called nowhere else in this file. +#[allow(non_snake_case)] +#[inline(always)] +unsafe fn glDrawArrays(mode: c_uint, first: c_int, count: c_int, extent: straighten::Extent) { + straighten::note_draw(extent); + glDrawArrays_raw(mode, first, count) +} + +/// Set a program's rect uniform (`u_rect` / `u_trect`: the authored-pixel quad the vertex shader +/// places the unit quad over) and return the extent that quad writes, for the draw that follows. The +/// one function that does both, so the extent a draw books cannot differ from the rect the shader +/// draws. +#[inline(always)] +pub(crate) unsafe fn set_rect(loc: c_int, x: f32, y: f32, w: f32, h: f32) -> straighten::Extent { + glUniform4f(loc, x, y, w, h); + straighten::Extent::quad(x, y, w, h) +} + +/// `glClear`, shadowed so a clear of the surface to a colour the straighten pass does not leave alone +/// is booked ([`straighten::note_clear`]); the colour is the one [`set_clear_color`] left. +#[allow(non_snake_case)] +#[inline(always)] +unsafe fn glClear(mask: c_uint) { + straighten::note_clear(CLEAR_COLOR[3]); + glClear_raw(mask) +} + +/// `glBindFramebuffer`, shadowed so the straighten pass's ledger knows whether gfx is drawing into +/// the surface or into a target of its own ([`straighten::note_bind`]). +#[allow(non_snake_case)] +#[inline(always)] +unsafe fn glBindFramebuffer(target: c_uint, framebuffer: c_uint) { + straighten::note_bind(framebuffer); + glBindFramebuffer_raw(target, framebuffer) +} + const GL_RGBA: c_uint = 0x1908; const GL_UNSIGNED_BYTE: c_uint = 0x1401; const GL_UNPACK_ALIGNMENT: c_uint = 0x0CF5; @@ -1338,6 +1397,13 @@ pub fn init_gl() { // // The player's transparent clear is unaffected in the right direction: over `dst.a = 0`, // accumulating alpha gives `src.a`, which is what a HUD drawn over the video plane means. + // + // **What that leaves in the framebuffer is PREMULTIPLIED colour** — `(rgb * a, a)` over the + // clear — and the television's compositor takes it as STRAIGHT (`panel = ui.rgb * ui.a + + // video * (1 - ui.a)`, measured on the dev set: the 43 -> 33 above is that multiply, 43 x 0.76). + // The two read the same pixel differently, so over the BOUND video plane the frame's last draw + // is the straighten pass (`straighten.rs`), which divides the colour by its alpha for the + // compositor to multiply back. Over an opaque ground `a` is 1 and the two agree. glBlendFuncSeparate( GL_SRC_ALPHA, GL_ONE_MINUS_SRC_ALPHA, @@ -1348,6 +1414,8 @@ pub fn init_gl() { // penumbra) into a regular ordered-dither dot pattern visible along tile edges. The panel is // 888 and SURFACE_APP is snapped to exact 8-bit codes, so dithering buys nothing here — off. glDisable(GL_DITHER); + // Last: it reads GL's extension string and links against the context just built. + straighten::init(); } } @@ -1446,7 +1514,7 @@ fn rect_fill(radius: f32, focus: f32, one_colour: bool) -> RectFill { unsafe fn draw_flat(x: f32, y: f32, w: f32, h: f32, col: *const f32) { if FPROG == 0 { use_prog(PROG); - glUniform4f(LOC_RECT, x, y, w, h); + let ext = set_rect(LOC_RECT, x, y, w, h); glUniform2f(LOC_SIZE, w, h); glUniform1f(LOC_PAD, 0.0); glUniform1f(LOC_RADIUS, 0.0); @@ -1458,13 +1526,13 @@ unsafe fn draw_flat(x: f32, y: f32, w: f32, h: f32, col: *const f32) { glUniform4f(LOC_RIMCOL, 0.0, 0.0, 0.0, 0.0); glUniform1f(LOC_RIMTOP, 0.0); pill_off(); - glDrawArrays(GL_TRIANGLE_STRIP, 0, 4); + glDrawArrays(GL_TRIANGLE_STRIP, 0, 4, ext); return; } use_prog(FPROG); // u_screen is set once at init (uniforms are per-program state) - glUniform4f(FL_RECT, x, y, w, h); + let ext = set_rect(FL_RECT, x, y, w, h); glUniform4fv(FL_COL, 1, col); - glDrawArrays(GL_TRIANGLE_STRIP, 0, 4); + glDrawArrays(GL_TRIANGLE_STRIP, 0, 4, ext); } /// A top-to-bottom two-stop gradient over an axis-aligned quad through `shaders/fs_ramp.frag` — the @@ -1474,7 +1542,7 @@ unsafe fn draw_flat(x: f32, y: f32, w: f32, h: f32, col: *const f32) { unsafe fn draw_ramp(x: f32, y: f32, w: f32, h: f32, top: *const f32, bot: *const f32) { if RPROG == 0 { use_prog(PROG); - glUniform4f(LOC_RECT, x, y, w, h); + let ext = set_rect(LOC_RECT, x, y, w, h); glUniform2f(LOC_SIZE, w, h); glUniform1f(LOC_PAD, 0.0); glUniform1f(LOC_RADIUS, 0.0); @@ -1486,14 +1554,14 @@ unsafe fn draw_ramp(x: f32, y: f32, w: f32, h: f32, top: *const f32, bot: *const glUniform4f(LOC_RIMCOL, 0.0, 0.0, 0.0, 0.0); glUniform1f(LOC_RIMTOP, 0.0); pill_off(); - glDrawArrays(GL_TRIANGLE_STRIP, 0, 4); + glDrawArrays(GL_TRIANGLE_STRIP, 0, 4, ext); return; } use_prog(RPROG); // u_screen is set once at init (uniforms are per-program state) - glUniform4f(RP_RECT, x, y, w, h); + let ext = set_rect(RP_RECT, x, y, w, h); glUniform4fv(RP_TOP, 1, top); glUniform4fv(RP_BOT, 1, bot); - glDrawArrays(GL_TRIANGLE_STRIP, 0, 4); + glDrawArrays(GL_TRIANGLE_STRIP, 0, 4, ext); } /// Only the bottom band and its AA fringe are rasterized; the shader clips to the full card. @@ -1542,11 +1610,11 @@ pub fn draw_art_scrim( use_prog(AS_PROG); let radius = radius.max(0.0).min(w * 0.5).min(h * 0.5); let inner = art_scrim_inner(w, h, radius); - glUniform4f(AS_RECT, qx, qy, qw, qh); + let ext = set_rect(AS_RECT, qx, qy, qw, qh); glUniform4f(AS_SIZE, w, h, w * 0.5 - radius, h * 0.5 - radius); glUniform4f(AS_BAND, band.min(h), radius, inner[0], inner[1]); glUniform4fv(AS_COL, 1, col.as_ptr()); - glDrawArrays(GL_TRIANGLE_STRIP, 0, 4); + glDrawArrays(GL_TRIANGLE_STRIP, 0, 4, ext); } true } @@ -1600,7 +1668,7 @@ pub fn draw_rect( } else { 0.0 }; - glUniform4f(LOC_RECT, x - aa, y - aa, w + 2.0 * aa, h + 2.0 * aa); + let ext = set_rect(LOC_RECT, x - aa, y - aa, w + 2.0 * aa, h + 2.0 * aa); glUniform2f(LOC_SIZE, w + 2.0 * aa, h + 2.0 * aa); glUniform1f(LOC_PAD, pad + aa); glUniform1f(LOC_RADIUS, radius); @@ -1616,7 +1684,7 @@ pub fn draw_rect( glUniform4f(LOC_RIMCOL, 0.0, 0.0, 0.0, 0.0); // no edge-sheen (default) glUniform1f(LOC_RIMTOP, 0.0); pill_off(); - glDrawArrays(GL_TRIANGLE_STRIP, 0, 4); + glDrawArrays(GL_TRIANGLE_STRIP, 0, 4, ext); } } @@ -1668,7 +1736,7 @@ pub fn draw_rect_shaped( unsafe { use_prog(PROG); let aa = AA_BLEED; // a sheened tile is always rounded → always SDF path - glUniform4f(LOC_RECT, x - aa, y - aa, w + 2.0 * aa, h + 2.0 * aa); + let ext = set_rect(LOC_RECT, x - aa, y - aa, w + 2.0 * aa, h + 2.0 * aa); glUniform2f(LOC_SIZE, w + 2.0 * aa, h + 2.0 * aa); glUniform1f(LOC_PAD, aa); glUniform1f(LOC_RADIUS, radius); @@ -1681,7 +1749,7 @@ pub fn draw_rect_shaped( glUniform1f(LOC_RIMTOP, rimtop); glUniform1f(LOC_RGB_CAP, rgb_cap); pill_on(pill, glow); - glDrawArrays(GL_TRIANGLE_STRIP, 0, 4); + glDrawArrays(GL_TRIANGLE_STRIP, 0, 4, ext); } } @@ -1747,7 +1815,7 @@ fn draw_ambient_impl( let amp = DITHER_LSB; let (prog, l_rect, l_tl, l_tr, l_br, l_bl) = ambient_program(amp); use_prog(prog); // u_screen is set once per program at init - glUniform4f(l_rect, x, y, w, h); + let ext = set_rect(l_rect, x, y, w, h); glUniform4f(l_tl, c3(tl, 0) * dim, c3(tl, 1) * dim, c3(tl, 2) * dim, 1.0); glUniform4f(l_tr, c3(tr, 0) * dim, c3(tr, 1) * dim, c3(tr, 2) * dim, 1.0); glUniform4f(l_br, c3(br, 0) * dim, c3(br, 1) * dim, c3(br, 2) * dim, 1.0); @@ -1759,7 +1827,7 @@ fn draw_ambient_impl( glUniform2f(AL_INKA, inka[0], inka[1]); send_scrim(0, AL_SCRIM, scrim); } - draw_field_mesh(); + draw_field_mesh(ext); } } @@ -1950,26 +2018,45 @@ pub fn draw_scrim_layer(x: f32, y: f32, w: f32, h: f32, ink: [f32; 3], scrim: &s return; } use_prog(SCRIM_PROG); - glUniform4f(SCL_RECT, x, y, w, h); + let ext = set_rect(SCL_RECT, x, y, w, h); glUniform3f(SCL_INK, ink[0], ink[1], ink[2]); send_scrim(2, SCL_SCRIM, scrim); - draw_field_mesh_rows(rows); + draw_field_mesh_rows(rows, ext); } } /// Issue the field mesh (see [`field_mesh`]) — every four-corner draw goes through this. #[inline] -unsafe fn draw_field_mesh() { - draw_field_mesh_rows(0..FIELD_N); +unsafe fn draw_field_mesh(ext: straighten::Extent) { + draw_field_mesh_rows(0..FIELD_N, ext); } /// Issue rows `rows` of the field mesh (`0..FIELD_N`, top to bottom). [`field_mesh`] stores it row /// by row, `FIELD_N` squares of 6 vertices to a row, after the unit quad's 4 vertices, so a run of /// rows is one contiguous `glDrawArrays` and draws exactly the vertices the whole mesh holds there. +/// `ext` is the rect the program places the WHOLE mesh over ([`set_rect`]'s answer); the draw books +/// the part of it the rows reach ([`rows_extent`]). #[inline] -unsafe fn draw_field_mesh_rows(rows: std::ops::Range) { +unsafe fn draw_field_mesh_rows(rows: std::ops::Range, ext: straighten::Extent) { const ROW_VERTS: usize = FIELD_N * 6; - glDrawArrays(GL_TRIANGLES, (4 + rows.start * ROW_VERTS) as c_int, (rows.len() * ROW_VERTS) as c_int); + glDrawArrays( + GL_TRIANGLES, + (4 + rows.start * ROW_VERTS) as c_int, + (rows.len() * ROW_VERTS) as c_int, + rows_extent(ext, &rows), + ); +} + +/// The part of the mesh's rect `ext` that rows `rows` cover: row `j` spans `j / FIELD_N .. (j + 1) / +/// FIELD_N` of the rect's height ([`field_mesh`]). A whole-target extent has no rect to cut. +fn rows_extent(ext: straighten::Extent, rows: &std::ops::Range) -> straighten::Extent { + match ext { + straighten::Extent::Quad([x, y, w, h]) => { + let n = FIELD_N as f32; + straighten::Extent::quad(x, y + h * rows.start as f32 / n, w, h * rows.len() as f32 / n) + } + whole => whole, + } } /// Which of the two ambient programs draws a field, with its uniform locations: the dithered one @@ -2017,7 +2104,7 @@ pub fn draw_grad4( // plain twin when it linked. let (prog, l_rect, l_tl, l_tr, l_br, l_bl) = ambient_program(0.0); use_prog(prog); // u_screen is set once per program at init - glUniform4f(l_rect, x, y, w, h); + let ext = set_rect(l_rect, x, y, w, h); glUniform4fv(l_tl, 1, tl); glUniform4fv(l_tr, 1, tr); glUniform4fv(l_br, 1, br); @@ -2025,7 +2112,7 @@ pub fn draw_grad4( if prog == APROG { glUniform1f(AL_DITHER, 0.0); } - draw_field_mesh(); + draw_field_mesh(ext); } } @@ -2045,7 +2132,7 @@ pub fn draw_rrect(x: f32, y: f32, w: f32, h: f32, rad_l: f32, rad_r: f32, col: * } else { 0.0 }; - glUniform4f(LOC_RECT, x - aa, y - aa, w + 2.0 * aa, h + 2.0 * aa); + let ext = set_rect(LOC_RECT, x - aa, y - aa, w + 2.0 * aa, h + 2.0 * aa); glUniform2f(LOC_SIZE, w + 2.0 * aa, h + 2.0 * aa); glUniform1f(LOC_PAD, aa); glUniform1f(LOC_RADIUS, rad_l); @@ -2059,7 +2146,7 @@ pub fn draw_rrect(x: f32, y: f32, w: f32, h: f32, rad_l: f32, rad_r: f32, col: * glUniform1f(LOC_RIMTOP, 0.0); glUniform1f(LOC_RGB_CAP, 1.0); // no rim, glow or ring to hold pill_off(); - glDrawArrays(GL_TRIANGLE_STRIP, 0, 4); + glDrawArrays(GL_TRIANGLE_STRIP, 0, 4, ext); } } @@ -2083,7 +2170,7 @@ pub fn draw_rrect_sheened( unsafe { use_prog(PROG); let aa = AA_BLEED; - glUniform4f(LOC_RECT, x - aa, y - aa, w + 2.0 * aa, h + 2.0 * aa); + let ext = set_rect(LOC_RECT, x - aa, y - aa, w + 2.0 * aa, h + 2.0 * aa); glUniform2f(LOC_SIZE, w + 2.0 * aa, h + 2.0 * aa); glUniform1f(LOC_PAD, aa); glUniform1f(LOC_RADIUS, rad_l); @@ -2097,7 +2184,7 @@ pub fn draw_rrect_sheened( glUniform1f(LOC_RIMTOP, rimtop); glUniform1f(LOC_RGB_CAP, rgb_cap); pill_off(); - glDrawArrays(GL_TRIANGLE_STRIP, 0, 4); + glDrawArrays(GL_TRIANGLE_STRIP, 0, 4, ext); } } @@ -2159,9 +2246,9 @@ pub fn draw_shadow( let (bands, n) = shadow_bands([qx, qy, qw, qh], inner, target_map().2); let (cx, cy) = (qx + qw * 0.5, qy + qh * 0.5); for r in &bands[..n] { - glUniform4f(SL_RECT, r[0], r[1], r[2], r[3]); + let ext = set_rect(SL_RECT, r[0], r[1], r[2], r[3]); glUniform4f(SL_MAP, r[0] - cx, r[1] - cy, r[2], r[3]); - glDrawArrays(GL_TRIANGLE_STRIP, 0, 4); + glDrawArrays(GL_TRIANGLE_STRIP, 0, 4, ext); } } } @@ -2506,7 +2593,7 @@ pub fn draw_art_wash( let inv = |d: f32| if d.abs() > 0.001 { 1.0 / d } else { 0.0 }; unsafe { use_prog(g.prog); - glUniform4f(g.rect, x, y, w, h); + let ext = set_rect(g.rect, x, y, w, h); for (loc, c) in g.corners.iter().zip(corners) { glUniform4f(*loc, c[0], c[1], c[2], 1.0); } @@ -2518,7 +2605,7 @@ pub fn draw_art_wash( glUniform2f(g.inka, inka[0], inka[1]); send_scrim(1, g.scrim, scrim); glBindTexture(GL_TEXTURE_2D, tex); - draw_field_mesh(); + draw_field_mesh(ext); } } @@ -2543,9 +2630,11 @@ pub fn snap(v: f32) -> f32 { v.round() } -/// Upload a straight-alpha RGBA8 bitmap (`w`×`h`, tightly packed) into a GL texture. Reuses -/// `prev` if non-zero (re-specs it), else allocates a new id. Returns the texture id. Used for -/// image-subtitle (PGS/VobSub) overlays and the text glyph-cache textures. Main-thread only. +/// Upload an RGBA8 bitmap (`w`×`h`, tightly packed) into a GL texture. Reuses `prev` if non-zero +/// (re-specs it), else allocates a new id. Returns the texture id. The bytes are neither converted +/// nor interpreted: image-subtitle (PGS/VobSub) overlays and the text glyph-cache textures are +/// straight alpha and drawn with [`draw_tex`]; the libass regions are premultiplied and drawn with +/// [`draw_tex_premultiplied`]. Main-thread only. pub fn upload_rgba(prev: c_uint, w: c_int, h: c_int, pixels: *const u8) -> c_uint { unsafe { let mut tex = prev; @@ -3058,7 +3147,7 @@ fn draw_tex_core_in( }, }; let Some(boxes) = boxes else { - glDrawArrays(GL_TRIANGLE_STRIP, 0, 4); + glDrawArrays(GL_TRIANGLE_STRIP, 0, 4, straighten::Extent::Quad(quad)); return; }; let plan = card_plan(&boxes, LIVE_SCISSOR, masked_part); @@ -3083,7 +3172,7 @@ fn draw_tex_core_in( } let b = piece.scissor; glScissor(b[0], b[1], b[2] - b[0], b[3] - b[1]); - glDrawArrays(GL_TRIANGLE_STRIP, 0, 4); + glDrawArrays(GL_TRIANGLE_STRIP, 0, 4, straighten::Extent::Quad(quad)); } // The clip the caller had (a scrolling list's, a source pass's box) is theirs: put it back, // through the two functions that keep `LIVE_SCISSOR` honest. @@ -3360,6 +3449,34 @@ pub fn draw_tex_uv( ); } +/// The tint [`draw_tex_premultiplied`] gives the shader for a STRAIGHT-alpha `tint` `[r, g, b, a]`: +/// `[r * a, g * a, b * a, a]`. The texel is `(rgb * A, A)`, so texel x tint is `(rgb * A * r * a, A * a)`: +/// the colour a straight draw of the straight texel `(rgb, A)` blends in `r * a * A`, and the alpha it +/// blends with, `A * a`. A fade (`a` below 1) and an ink (`r`, `g`, `b` below 1) therefore dim a +/// premultiplied picture exactly as they dim a straight one; scaling the colour alone would leave the +/// picture as opaque as before. +pub fn premultiplied_tint(tint: [f32; 4]) -> [f32; 4] { + [tint[0] * tint[3], tint[1] * tint[3], tint[2] * tint[3], tint[3]] +} + +/// **Draw a texture that holds PREMULTIPLIED RGBA** (`(rgb * a, a)` per texel, no channel above its +/// alpha) as one square-cornered quad: `(x, y, w, h)` in logical px, `tint` the STRAIGHT ink and fade +/// `[r, g, b, a]` every other primitive takes ([`premultiplied_tint`] turns it into the shader's). +/// It blends `(ONE, ONE_MINUS_SRC_ALPHA)` and restores the ordinary blend, so the framebuffer comes out +/// as a straight-alpha draw of the same picture would have left it (the unit test +/// `a_premultiplied_draw_leaves_the_framebuffer_a_straight_draw_would` pins that to one code value) +/// without the texture ever being converted to straight alpha, and bilinear filtering of it +/// averages colour weighted by coverage, so a soft edge is not darkened by its transparent neighbours. +/// +/// This is the draw for a producer that composes in premultiplied space (the libass facade's regions, +/// `src/ass.c`); an image or glyph texture uploaded straight (PGS and VobSub bitmaps, the glyph cache) +/// stays on [`draw_tex`]. A premultiplied texture drawn there is too dark at every soft edge. +pub fn draw_tex_premultiplied(tex: c_uint, x: f32, y: f32, w: f32, h: f32, tint: [f32; 4]) { + let tint = premultiplied_tint(tint); + let _blend = PremultipliedBlend::enter(); + draw_tex_uv(tex, UV_FULL, x, y, w, h, 0.0, tint.as_ptr()); +} + /// Two equally sized images through the ordinary card compositor. One coverage/opacity /// calculation follows the image mix. A failed optional shader leaves the caller's static image. #[allow(clippy::too_many_arguments)] @@ -3406,21 +3523,42 @@ pub fn draw_tex_carded_blend(tex: c_uint, next: c_uint, mix: f32, crop: [f32; 4] } } -/// **Simulator only: composite `tex` UNDER everything drawn so far this frame** — the television -/// compositor's arithmetic for its hardware video plane beneath our UI plane. The UI plane holds -/// premultiplied colour over a transparent clear (see the blend note in the GL setup above), so -/// the panel shows `ui.rgb + video.rgb * (1 - ui.a)`; drawing the picture with -/// `(ONE_MINUS_DST_ALPHA, ONE)` computes exactly that, and leaves the surface opaque. Used by -/// `app::adapters::hostsim::sim_video` for the screenshot pipeline's player figure; restores the app's blend. +/// Run `draw` with the simulator's compositor blend ([`straighten::UNDER`]) set and the app's own put +/// back after. #[cfg(feature = "hostsim")] -pub fn draw_under(tex: c_uint, x: f32, y: f32, w: f32, h: f32) { +fn under_blend(draw: impl FnOnce()) { + let [src_rgb, dst_rgb, src_a, dst_a] = straighten::UNDER.gl(); unsafe { - glBlendFuncSeparate(GL_ONE_MINUS_DST_ALPHA, GL_ONE, GL_ONE_MINUS_DST_ALPHA, GL_ONE); - draw_tex(tex, x, y, w, h, 0.0, [1.0f32; 4].as_ptr()); + glBlendFuncSeparate(src_rgb, dst_rgb, src_a, dst_a); + draw(); glBlendFuncSeparate(GL_SRC_ALPHA, GL_ONE_MINUS_SRC_ALPHA, GL_ONE, GL_ONE_MINUS_SRC_ALPHA); } } +/// **Simulator only: composite `tex` UNDER everything drawn so far this frame** — the television +/// compositor's arithmetic for its hardware video plane beneath our UI plane, which is STRAIGHT alpha: +/// `panel = ui.rgb * ui.a + video * (1 - ui.a)`, measured on the dev set. The factors are +/// [`straighten::UNDER`], `(ONE_MINUS_DST_ALPHA, DST_ALPHA)` for colour, and they leave the surface +/// opaque. This is the compositor, not the intended picture: a premultiplied frame handed to it comes +/// out darker by its own alpha, which is the television's fault and exactly what the straighten pass +/// ([`straighten::straighten_surface`], run first) removes, so the simulator shows both. Used by +/// `app::adapters::hostsim::sim_video` for the screenshot pipeline's player figure and, with no +/// picture, by [`draw_under_black`]; restores the app's blend. +#[cfg(feature = "hostsim")] +pub fn draw_under(tex: c_uint, x: f32, y: f32, w: f32, h: f32) { + under_blend(|| draw_tex(tex, x, y, w, h, 0.0, [1.0f32; 4].as_ptr())); +} + +/// **Simulator only: the same compositor over an empty plane** — black. A bound-plane frame is +/// straightened (the pass has no simulator switch), so it has to go through the compositor model +/// whether or not the simulator holds a decoded picture, or a screenshot would read the straight +/// buffer itself. +#[cfg(feature = "hostsim")] +pub fn draw_under_black() { + let black = [0.0f32, 0.0, 0.0, 1.0]; + under_blend(|| draw_rect(0.0, 0.0, SCR_W, SCR_H, 0.0, 0.0, black.as_ptr(), black.as_ptr(), 0.0, 1.0)); +} + /// **A texture the renderer fills once and serves as one quad**, in one of two modes. They share the /// texture, its FBO and the draw path, and differ in WHAT the texture holds and so in how it is drawn: /// @@ -3728,7 +3866,7 @@ impl FrameCache { /// Put the region on the frame as ONE textured quad with its top-left at `at`, `tint` being the /// PREMULTIPLIED group tint `[g, g, g, a]` (`g` = the colour gain times the alpha): the texture - /// holds `(rgb * a, a)`, so the blend is `(ONE, ONE_MINUS_SRC_ALPHA)` ([`StillBlend`]) and the + /// holds `(rgb * a, a)`, so the blend is `(ONE, ONE_MINUS_SRC_ALPHA)` ([`PremultipliedBlend`]) and the /// tint scales colour AND alpha. Unlike [`draw_alpha`](Self::draw_alpha) this is page CONTENT, not /// the page itself: it does not lift [`page_frozen`], so a frozen host refuses it as it refuses /// every other primitive. @@ -3737,7 +3875,7 @@ impl FrameCache { return false; } let (w, h) = (self.region[2] as f32, self.region[3] as f32); - let _blend = StillBlend::enter(); + let _blend = PremultipliedBlend::enter(); draw_tex_core( Class::Image, self.tex, at[0], at[1], w, h, frame_cache_uv(), 0.0, tint.as_ptr(), 0.0, NO_RIM.as_ptr(), w * 0.5, h * 0.5, 0.0, NO_RIM.as_ptr(), 0.0, 0.0, None, @@ -4161,17 +4299,19 @@ fn still_fusion_eligible(tex: c_uint, tint: [f32; 4], available: bool) -> bool { tex != 0 && tint == [1.0; 4] && available } -/// The still program emits premultiplied RGB, and so does a region cache's quad -/// ([`FrameCache::draw_region`]): its texture holds `(rgb * a, a)`. Restore the ordinary blend even -/// during unwinding; no caller and no following text/overlay draw inherits this primitive's private state. -struct StillBlend; -impl StillBlend { +/// The blend of a draw whose fragment colour is already PREMULTIPLIED: `(ONE, ONE_MINUS_SRC_ALPHA)`. +/// The still program emits premultiplied RGB, a region cache's quad ([`FrameCache::draw_region`]) +/// samples a texture that holds `(rgb * a, a)`, and so does any texture [`draw_tex_premultiplied`] is +/// handed. Restore the ordinary blend even during unwinding; no caller and no following text/overlay +/// draw inherits this primitive's private state. +struct PremultipliedBlend; +impl PremultipliedBlend { fn enter() -> Self { unsafe { glBlendFuncSeparate(GL_ONE, GL_ONE_MINUS_SRC_ALPHA, GL_ONE, GL_ONE_MINUS_SRC_ALPHA); } Self } } -impl Drop for StillBlend { +impl Drop for PremultipliedBlend { fn drop(&mut self) { unsafe { glBlendFuncSeparate(GL_SRC_ALPHA, GL_ONE_MINUS_SRC_ALPHA, GL_ONE, GL_ONE_MINUS_SRC_ALPHA); } } @@ -4204,7 +4344,7 @@ pub fn draw_tex_carded_still( glUniform4fv(loc_col, 1, scrim.as_ptr()); } note_card(x, y, w, h, pad, dy); - let _blend = StillBlend::enter(); + let _blend = PremultipliedBlend::enter(); draw_tex_impl(tex, crop, x, y, w, h, radius, tint.as_ptr(), rimw, rimcol.as_ptr(), pad, shblur, shcol.as_ptr(), f, dy, Some((program, uniforms))); true @@ -5636,7 +5776,7 @@ fn blur_snapshot_with_taps(reg: [f32; 4], taps: &[f32]) { let tap = tap * plx_base::surface::render_scale() as f32; glUniform2f(BL_TEXEL, tap / c.sw as f32, tap / c.sh as f32); glBindTexture(GL_TEXTURE_2D, src); - glDrawArrays(GL_TRIANGLE_STRIP, 0, 4); + glDrawArrays(GL_TRIANGLE_STRIP, 0, 4, straighten::Extent::Target); }); } @@ -5660,7 +5800,7 @@ fn blur_snapshot_with_taps(reg: [f32; 4], taps: &[f32]) { // The Settings kernel adds an even pair of extra passes, so both the ordinary and // modal chains finish in `a`. The assertion at entry keeps that property structural. glBindTexture(GL_TEXTURE_2D, c.a); - glDrawArrays(GL_TRIANGLE_STRIP, 0, 4); + glDrawArrays(GL_TRIANGLE_STRIP, 0, 4, straighten::Extent::Target); }); glBindFramebuffer(GL_FRAMEBUFFER, plx_base::surface::default_fb()); @@ -6752,20 +6892,36 @@ pub fn video_plane_frame() -> bool { /// cache is therefore a photograph of transparent black, served back over the video for as long as /// the cache lives. /// +/// **The doors also stay shut past the straighten pass** (`straighten.rs`), which is the frame's +/// last draw while the plane is bound: from then until the next `frame_begin` framebuffer 0 holds +/// STRAIGHT colour for the television's compositor, not the premultiplied picture every consumer of +/// a snapshot composites, so a read of it is just as wrong, in another way. +/// /// **A debug build PANICS**, because reaching one of these is a structural mistake — a screen that /// declared `RenderStrategy::VideoPlane` and then asked for a snapshot — and the picture it /// produces on a television is a black rectangle nobody can explain from a log. A release build /// logs ONCE and refuses; the once is deliberate, since a door reached at all is reached every /// frame and a per-frame line would bury the event log during playback. pub fn video_plane_refuses(what: &str) -> bool { - if !video_plane_frame() { + // **Past the straighten pass the doors are shut too** (`straighten.rs`): the frame's last draw + // turned framebuffer 0 into STRAIGHT colour for the compositor, so a read of it is not the + // premultiplied picture every consumer composites. The pass is the one deliberate read of the + // framebuffer on a video-plane frame, and it caches nothing: it copies into its own texture and + // draws straight back. + let straight = straighten::straightened(); + if !video_plane_frame() && !straight { return false; } // Under `cargo test` these doors are driven ON PURPOSE — the point of // `a_video_plane_screen_replaces_its_host_and_takes_no_snapshot` is to watch each one refuse — // so the assertion is the SHIPPING debug build's, not the harness's. #[cfg(all(debug_assertions, not(any(test, feature = "test-support"))))] - panic!("{what} on a VIDEO PLANE frame: the plane is not in our framebuffer to sample"); + { + if straight { + panic!("{what} after the straighten pass: framebuffer 0 holds straight colour for the compositor"); + } + panic!("{what} on a VIDEO PLANE frame: the plane is not in our framebuffer to sample"); + } #[allow(unreachable_code)] { if !VIDEO_PLANE_TOLD.swap(true, std::sync::atomic::Ordering::Relaxed) { @@ -7002,7 +7158,7 @@ pub fn blur_snapshot_direct(reg: [f32; 4], draw_scene: &mut dyn FnMut()) -> bool note_px(Class::Blur, (tw as f64) * (th as f64)); glUniform2f(BL_TEXEL, off / c.sw as f32, off / c.sh as f32); glBindTexture(GL_TEXTURE_2D, src); - glDrawArrays(GL_TRIANGLE_STRIP, 0, 4); + glDrawArrays(GL_TRIANGLE_STRIP, 0, 4, straighten::Extent::Target); }); } @@ -7024,7 +7180,7 @@ pub fn blur_snapshot_direct(reg: [f32; 4], draw_scene: &mut dyn FnMut()) -> bool BLUR_UP_TAP / c.mh as f32, ); glBindTexture(GL_TEXTURE_2D, c.a); - glDrawArrays(GL_TRIANGLE_STRIP, 0, 4); + glDrawArrays(GL_TRIANGLE_STRIP, 0, 4, straighten::Extent::Target); }); glBindFramebuffer(GL_FRAMEBUFFER, plx_base::surface::default_fb()); @@ -7394,9 +7550,9 @@ pub fn draw_blur_backdrop( glUniform4f(GL_UVRECT, uv[0], uv[1], uv[2], uv[3]); glUniform1f(GL_RADIUS, radius); glUniform2f(GL_CH, w * 0.5, h * 0.5); - glUniform4f(GL_RECT, x, y, w, h); + let ext = set_rect(GL_RECT, x, y, w, h); profile::phase("glass.composite", || { - glDrawArrays(GL_TRIANGLE_STRIP, 0, 4); + glDrawArrays(GL_TRIANGLE_STRIP, 0, 4, ext); }); true } @@ -7514,9 +7670,9 @@ pub fn draw_glass_frozen( glUniform4f(GL_UVRECT, x / SCR_W, y / SCR_H, w / SCR_W, h / SCR_H); glUniform1f(GL_RADIUS, radius); glUniform2f(GL_CH, w * 0.5, h * 0.5); - glUniform4f(GL_RECT, x, y, w, h); + let ext = set_rect(GL_RECT, x, y, w, h); profile::phase("glass.frozen", || { - glDrawArrays(GL_TRIANGLE_STRIP, 0, 4); + glDrawArrays(GL_TRIANGLE_STRIP, 0, 4, ext); }); } true @@ -8194,11 +8350,11 @@ pub fn draw_field(x: f32, y: f32, w: f32, h: f32, tex: c_uint, tint: *const f32) } unsafe { use_prog(UPROG); // u_screen and the sampler unit are set once at init - glUniform4f(UL_RECT, x, y, w, h); + let ext = set_rect(UL_RECT, x, y, w, h); glUniform4fv(UL_TINT, 1, tint); glUniform1f(UL_DITHER, dither_for_field(w, h)); glBindTexture(GL_TEXTURE_2D, tex); - glDrawArrays(GL_TRIANGLE_STRIP, 0, 4); + glDrawArrays(GL_TRIANGLE_STRIP, 0, 4, ext); } } @@ -8230,14 +8386,14 @@ pub fn draw_field_panel( } unsafe { use_prog(PPROG); // u_screen and the sampler unit are set once at init - glUniform4f(FP_RECT, x, y, w, h); + let ext = set_rect(FP_RECT, x, y, w, h); glUniform4fv(FP_TINT, 1, tint); glUniform4f(FP_UVRECT, uv[0], uv[1], uv[2], uv[3]); glUniform2f(FP_SIZE, w, h); glUniform1f(FP_RADIUS, radius.min(w.min(h) * 0.5)); glUniform1f(FP_DITHER, dither_for_field(w, h)); glBindTexture(GL_TEXTURE_2D, tex); - glDrawArrays(GL_TRIANGLE_STRIP, 0, 4); + glDrawArrays(GL_TRIANGLE_STRIP, 0, 4, ext); } true } @@ -8892,7 +9048,7 @@ mod tests { &rest[..rest.find("\n}\n").expect("the end of the function")] }; let ramp_draw = body("unsafe fn draw_ramp("); - assert!(ramp_draw.contains("glUniform4f(RP_RECT, x, y, w, h);"), "exactly the rect's bounds"); + assert!(ramp_draw.contains("set_rect(RP_RECT, x, y, w, h);"), "exactly the rect's bounds"); assert!(ramp_draw.contains("if RPROG == 0 {") && ramp_draw.contains("use_prog(PROG);"), "the fallback"); let rect_draw = body("pub fn draw_rect("); let (gated, routed) = ( @@ -10369,6 +10525,52 @@ mod tests { ); } + /// **The premultiplied draw is the straight draw's picture, to one code value.** A CPU model of the + /// two GL blends (`SRC_ALPHA, ONE_MINUS_SRC_ALPHA` of a straight texel, `ONE, ONE_MINUS_SRC_ALPHA` of + /// a premultiplied one through [`premultiplied_tint`]) over every premultiplied pixel the libass + /// facade can publish (a channel `p` at alpha `a`, `p <= a`), against the straight texel the facade + /// USED to publish (its conversion, frozen here: `(p * 255 + a / 2) / a`). The two disagree by the + /// rounding of that conversion alone: under half a code value before the framebuffer's own rounding, + /// and so at most one code value after it. The premultiplied side is the exact one. + #[test] + fn a_premultiplied_draw_leaves_the_framebuffer_a_straight_draw_would() { + fn old_straight(p: u32, a: u32) -> f32 { + if a == 0 { 0.0 } else { ((p * 255 + a / 2) / a).min(255) as f32 / 255.0 } + } + let code = |v: f32| (v * 255.0 + 0.5).floor(); + let tints = [[1.0; 4], [0.85, 0.85, 0.85, 1.0], [0.55, 0.55, 0.55, 1.0], [1.0, 1.0, 1.0, 0.5], [0.7, 0.4, 0.2, 0.3]]; + let (mut worst, mut worst_code) = (0.0_f32, 0.0_f32); + for a in 0..=255u32 { + for p in 0..=a { + for tint in tints { + let pre = premultiplied_tint(tint); + for dst in [0.0, 40.0 / 255.0, 0.5, 1.0] { + // straight: colour * tint, blended by texel alpha * tint alpha + let alpha = a as f32 / 255.0 * tint[3]; + let old = old_straight(p, a) * tint[0] * alpha + dst * (1.0 - alpha); + // premultiplied: texel * premultiplied tint, added; same alpha + let new = p as f32 / 255.0 * pre[0] + dst * (1.0 - a as f32 / 255.0 * pre[3]); + worst = worst.max((old - new).abs() * 255.0); + worst_code = worst_code.max((code(old) - code(new)).abs()); + } + } + } + } + assert!(worst <= 0.5 + 1e-3, "the two draws differ by {worst} codes before rounding"); + assert!(worst_code <= 1.0, "and by {worst_code} codes after it"); + } + + /// The tint of a premultiplied draw: colour scaled by the fade as well, alpha as it is (a fade that + /// scaled only the colour would leave the picture opaque over what it fades into). + #[test] + fn a_premultiplied_tint_scales_the_colour_by_its_alpha() { + assert_eq!(premultiplied_tint([1.0; 4]), [1.0; 4], "white and opaque is the texel as it is"); + assert_eq!(premultiplied_tint([0.85, 0.85, 0.85, 1.0]), [0.85, 0.85, 0.85, 1.0], "an ink rung scales colour only"); + assert_eq!(premultiplied_tint([1.0, 1.0, 1.0, 0.5]), [0.5, 0.5, 0.5, 0.5], "a fade scales both"); + assert_eq!(premultiplied_tint([0.8, 0.5, 0.25, 0.5]), [0.4, 0.25, 0.125, 0.5]); + assert_eq!(premultiplied_tint([1.0, 0.5, 0.2, 0.0]), [0.0, 0.0, 0.0, 0.0], "a fully faded draw adds nothing"); + } + /// **`glClearColor` has one caller: [`set_clear_color`].** A flat ground is a clear in a colour of /// its own and puts back the colour the context held, which it knows only if every set went /// through the record. A second call anywhere in this file (the app's own, in another crate, @@ -11232,7 +11434,7 @@ mod card_split_tests { let (booked, refused, drawn) = ( draw[whole..].find("note_card_path(path").expect("booked"), draw[whole..].find("card_whole_refused(path, masked_part)").expect("refused through the real mask"), - draw.find("glDrawArrays(GL_TRIANGLE_STRIP, 0, 4)").expect("drawn"), + draw.find("glDrawArrays(GL_TRIANGLE_STRIP, 0, 4,").expect("drawn"), ); assert!(booked < refused && whole + refused < drawn, "booked {booked}, refused {refused}, drawn {drawn}"); let split = draw.find("CardCut::Pieces(rect) =>").expect("the split arm"); diff --git a/rust-modules/gfx/src/gfx/backdrop.rs b/rust-modules/gfx/src/gfx/backdrop.rs index dda2264884..00d7336b06 100644 --- a/rust-modules/gfx/src/gfx/backdrop.rs +++ b/rust-modules/gfx/src/gfx/backdrop.rs @@ -581,6 +581,7 @@ fn kind_name(tag: u64) -> String { 12 => "tex_uv".into(), 13 => "tex_stroked".into(), 14 => "tex_carded".into(), + 15 => "tex_premultiplied".into(), 16 => "ambient".into(), 17 => "grad4".into(), 18 => "field".into(), diff --git a/rust-modules/gfx/src/gfx/straighten.rs b/rust-modules/gfx/src/gfx/straighten.rs new file mode 100644 index 0000000000..8bb97d8dc8 --- /dev/null +++ b/rust-modules/gfx/src/gfx/straighten.rs @@ -0,0 +1,1249 @@ +//! **The straighten pass: the frame's premultiplied colour, made straight for a compositor that +//! multiplies it by alpha** (#1328's follow-up). +//! +//! # The fact +//! +//! Over the hardware video plane the television composites our GLES surface as STRAIGHT alpha: +//! `panel = ui.rgb * ui.a + video * (1 - ui.a)`. Measured on the dev set (webOS 4.5, Mali-T820) from +//! the three plane captures of one translucent subtitle pixel: GRAPHIC `(57, 64, 70)` at `a` 0.275, +//! VIDEO `(226, 213, 51)`, DISPLAY `(182, 172, 55)`. Straight alpha predicts `(180, 172, 56)`; +//! premultiplied would be `(221, 218, 107)`. The multiply is in the SoC mixer, not in a surface +//! setting: the only compositor-side knob the firmware exports is `QWaylandQuickSurface:: +//! setUseTextureAlpha`, and it does not reach the mixer. +//! +//! Our framebuffer, over the player's transparent clear, holds PREMULTIPLIED colour: the blend note in +//! `gfx.rs` (`glBlendFuncSeparate(SRC_ALPHA, ONE_MINUS_SRC_ALPHA, ONE, ONE_MINUS_SRC_ALPHA)`) leaves +//! `(rgb * a, a)` over `(0, 0, 0, 0)`. So every translucent pixel was multiplied by its own alpha a +//! second time on the panel: a pale subtitle glow, a soft glyph edge or a HUD scrim came out darker +//! than drawn, and a pale translucent kanji read as a dark shadow. +//! +//! # The pass +//! +//! A last draw before the swap rewrites the frame `rgb' = a > 0 ? rgb / a : 0`, alpha unchanged, +//! blending off (`fs_straighten.frag`), over the pixels the frame drew into and no others (below). The +//! compositor's multiply then returns exactly the colour the frame drew: `(rgb / a) * a + video * (1 - a) +//! = rgb + video * (1 - a)`, the premultiplied picture the frame was built as. An opaque pixel (`a = 1`) +//! and a hole (`a = 0`) are fixed points, which is what makes "the pixels the frame drew into" enough: +//! every other pixel is the transparent clear. +//! +//! **Where it runs.** From the application loop (`app/run.rs`'s `present_and_swap`), after the frame's +//! draw and its one framebuffer readback and before anything else touches the surface, and only on a +//! frame whose surface is over the BOUND video plane (`Player::video_plane_bound`) that DREW something. +//! The gate presents every frame while the plane is bound, and about 99 % of those frames are a +//! transparent clear with no draw call at all. Two reads of the frame's own state decide it: the plane +//! bit (a function argument, not gfx state: [`super::video_plane_frame`] is restored to `false` before +//! the draw returns, and it excludes the bound trailer preview, a `RenderStrategy::Page` screen whose +//! chrome is just as translucent) and [`frame_drew`], whether [`note_draw`] booked a draw onto the +//! surface since [`frame_begin`]. +//! +//! **Where it draws.** One quad over each box of the surface the frame drew into, and nowhere else. A +//! pass over the whole viewport (2.07 M pixels, about 40 fps where the frame ran at about 50 on the dev +//! set, the same fixture and the release build) touched every tile of a frame that is, for the length of +//! a film, a subtitle line and a clear. Every draw call in this crate says what it writes +//! ([`Extent`]): the authored-pixel quad its program places the unit quad over ([`super::set_rect`] sets +//! that uniform and returns the extent, so the two cannot differ), or the whole target for a draw with no +//! rect. [`note_draw`] turns it into a window-pixel box, rounded outward with a pixel of slack for the +//! vertex stage's rounding, and books it in a [`Ledger`]: at most 16 boxes, kept PAIRWISE DISJOINT by +//! folding two that share a pixel into their bounding box (and, past 16, the pair that wastes the fewest +//! pixels). Disjoint is not tidiness. The fetch program rewrites the pixel it finds, so a pixel two quads +//! cover would be divided by its alpha twice; the copy program reads an untouched copy and would not +//! show it, which is why the simulator cannot catch an overlap and a host model test pins it +//! (`the_ledger_covers_every_drawn_pixel_exactly_once_and_no_other`). Boxes that only touch stay two: +//! the rasterizer's fill rule gives their shared edge to exactly one of them. +//! +//! **What books nothing, and what books everything.** A draw into a target of gfx's own (a blur pass, a +//! field reduction, a region fill; [`note_bind`] knows from the binds, and a host-page snapshot by +//! `default_fb`) is not the surface's and books nothing; the draw that puts its pixels on the surface +//! books its own quad. A draw with no known extent ([`Extent::Target`], or a quad that is not a +//! number) books the whole surface and the pass covers the viewport, as it did before it knew where the +//! frame drew. A `glClear` is not a draw ([`note_clear`]): the two alphas the app clears the surface to, +//! 0 and 1, are fixed points, and any other alpha books the whole surface. The scissor is ignored: it +//! only narrows what a quad writes, so a box that ignores it is a superset. +//! +//! **How it reads the frame.** Preferably in place: `GL_ARM_shader_framebuffer_fetch` (the dev set has +//! it, `docs/egl-partial-update-and-damage.md` §1b) or `GL_EXT_shader_framebuffer_fetch` read the +//! pixel the tile already holds, so the pass is one fragment program over tile memory: no copy, no +//! render-pass split. Without either, the frame is copied (`glCopyTexSubImage2D`, like +//! [`FrameCache::capture_prev`](super::FrameCache::capture_prev)) into one private viewport-sized +//! texture and drawn back 1:1 with blending off; that texture exists only on that path. It does NOT go +//! through [`FrameCache::capture`](super::FrameCache::capture): a capture arms the snapshot fence that +//! DEFERS presents until the copy leaves the GPU, which is right for a page snapshot and wrong for a +//! pass that runs on every frame with content for a whole playback. No new GL entry point is bound: +//! `glGetString`, `glCopyTexSubImage2D` and `glDrawArrays` were already here; the fetch is a GLSL +//! `#extension`, chosen from the extension string as a whole token and linked as an optional program. +//! +//! # Buffer age +//! +//! The surface is `EGL_BUFFER_DESTROYED` and the config has no `SWAP_BEHAVIOR_PRESERVED_BIT` +//! (`docs/egl-partial-update-and-damage.md` §1b), so a back buffer's contents are undefined after a +//! swap; and a frame on this plane starts from a clear (the player route's `glClear`, a preview page's +//! `frame_clear_through`) and redraws the whole viewport, because a presented frame has always been a +//! whole frame ("not dirty-RECTANGLE tracking", `app/run.rs`'s present gate). So the pass runs once, +//! over a buffer composed this frame from a clear, and nothing in a later frame is ever blended over +//! straight pixels. **A bound frame therefore has to stay a complete redraw.** A damage scheme +//! (`EGL_KHR_partial_update`, today only the `plxnative-egldamage` experiment) that redrew part of +//! it and blended over the pixels the buffer kept would composite premultiplied colour onto straight +//! colour: it must clear every damaged region first and run this pass over exactly that region (the +//! ledger already holds the boxes a frame drew; it would be reset to the damage instead of to nothing). +//! The same fact is why untouched pixels may be left alone: a frame starts from a clear, so a pixel no +//! draw booked is `(0, 0, 0, 0)`. +//! +//! # Readbacks +//! +//! Every door that samples framebuffer 0 (`video_plane_refuses`) refuses once the pass has run +//! ([`straightened`]): what it would read is no longer a premultiplied picture. The simulator's own +//! readers (`shot`, the frame dump) run after its compositor model ([`super::draw_under`], which +//! composites the picture over a plane the way the television does, `UNDER`), never between the pass +//! and that model, so a screenshot is the panel and not the straight buffer. +//! +//! # Cost +//! +//! A fragment program with a reciprocal and a multiply per pixel (about 2 arithmetic words a quad on +//! Midgard) over the covered pixels, and a draw call per box (at most 16). What the covered pixels are +//! on a frame follows the frame. MODELLED, not measured, by running `Ledger::add` over assumed 1080p +//! frames: two dialogue lines about 0.15 M pixels, a heavy opening whose regions sit in the lower 40% of +//! the screen about 0.9 M, a HUD with full-width top and bottom scrims about 1.2 M; 64 regions scattered +//! over the whole frame fold to one bounding box of nearly all of it, and a draw with no extent is all +//! 2.07 M (the pass then costs what it did before). The pass over the whole viewport was measured on the +//! set as the loss of about 10 fps (about 50 to about 40 on the UI heartbeat); the covered pixels are +//! the figure that scales it, and the pass's cost over a given set of boxes is UNMEASURED on the set. +//! The copy fallback adds the frame's copy and a pass reload (about 17 MB of traffic at 1080p), whatever +//! the boxes. The boxes are the drawn extents, not the logical rects (`AA_BLEED`, shadow and glow pads +//! are inside them), and a bounding box of a few draws can cover pixels none of them touched: those are +//! the clear, a fixed point. + +use std::ffi::CStr; +use std::os::raw::{c_char, c_int, c_uint}; +use std::sync::atomic::{AtomicI32, AtomicU32, Ordering}; + +use super::{ + cap_tex, delete_tex, glActiveTexture, glBindFramebuffer, glBindTexture, glCopyTexSubImage2D, glDisable, + glDrawArrays_raw, glEnable, glGetString, glGetUniformLocation, glTexParameteri, glUniform4f, glViewport, + link_optional_program, scissor_off, use_prog, FS_STRAIGHTEN_ARM, FS_STRAIGHTEN_COPY, FS_STRAIGHTEN_EXT, + GL_BLEND, GL_FRAMEBUFFER, GL_NEAREST, GL_TEXTURE0, GL_TEXTURE_2D, GL_TEXTURE_MAG_FILTER, + GL_TEXTURE_MIN_FILTER, GL_TRIANGLE_STRIP, VS_STRAIGHTEN, +}; +use plx_base::eventlog::log; + +// ---- the ledger of draws: "what did this frame put on the surface, and where?" ------------------ + +thread_local! { + /// The boxes of the presented surface this frame drew into ([`Ledger`]), reset by [`frame_begin`]. + /// + /// THREAD-LOCAL like [`STRAIGHTENED`] below and `VIDEO_PLANE_FRAME` beside the doors that read + /// it: it is draw state, the draw is one thread's, and a process-wide ledger would let one + /// test's frame decide another's pass. + static LEDGER: std::cell::RefCell = const { std::cell::RefCell::new(Ledger::new()) }; + /// Is gfx drawing into a target of its own (a blur pass, a field reduction, a region fill) and not + /// into the surface? Set by [`note_bind`], which every `glBindFramebuffer` in this crate calls. + static OFFSCREEN: std::cell::Cell = const { std::cell::Cell::new(false) }; +} + +/// Does a draw issued now write the surface the frame is presented from? Two reads of state the +/// crate already keeps, no GL query: gfx's own last bind named the "back to the screen" framebuffer +/// ([`note_bind`]), and no host-page snapshot is being drawn ([`plx_base::surface::PageTarget`] +/// redirects `default_fb` off `frame_fb`, and binds on its own, below this crate's wrapper). +fn presented() -> bool { + !OFFSCREEN.with(|o| o.get()) && plx_base::surface::default_fb() == plx_base::surface::frame_fb() +} + +/// Book one draw call: the extent it writes, if it writes the presented surface. Every `glDrawArrays` +/// in this crate goes through the wrapper that calls this (`gfx.rs` and `text.rs` each shadow the +/// extern with one that REQUIRES an [`Extent`], and a test pins that the raw import is called nowhere +/// else), so a primitive can neither draw nor be built without saying where it draws. +/// +/// A draw into another target books nothing: its pixels are not the surface's, and the draw that puts +/// them there (a glass composite, a region quad) books its own quad. A quad with no area books nothing +/// either. The render thread is the only caller. +#[inline] +pub(crate) fn note_draw(extent: Extent) { + if !presented() { + return; + } + let b = match extent { + Extent::Target => Some(EVERYTHING), + Extent::Quad(q) => { + let (vx, vy, _, _) = plx_base::surface::viewport(); + quad_box(q, (vx, vy), plx_base::surface::scale()) + } + }; + if let Some(b) = b { + LEDGER.with(|l| l.borrow_mut().add(b)); + } +} + +/// Note that gfx bound framebuffer `fb`: every `glBindFramebuffer` in this crate calls this first. A +/// bind of the "back to the screen" framebuffer ([`plx_base::surface::default_fb`], which every pass +/// that leaves the surface ends with) is the surface again; any other bind is a target of gfx's own. +#[inline] +pub(crate) fn note_bind(fb: c_uint) { + OFFSCREEN.with(|o| o.set(fb != plx_base::surface::default_fb())); +} + +/// Book a `glClear` of the colour with this `alpha`. A clear is not a draw, and it is sound to leave +/// it out only because the two alphas the app clears the surface to are fixed points of the pass: 0 +/// (the transparent hole every frame over the plane starts from) and 1 (an opaque ground). Any other +/// alpha would leave the whole surface translucent and the pass has to cover all of it. +#[inline] +pub(crate) fn note_clear(alpha: f32) { + if presented() && !(alpha <= 0.0 || alpha >= 1.0) { + LEDGER.with(|l| l.borrow_mut().add(EVERYTHING)); + } +} + +/// Open a drawn frame: nothing has been drawn yet and the surface is not straightened. Called beside +/// `text::begin_frame`, once per presented frame, before the draw. +pub fn frame_begin() { + LEDGER.with(|l| *l.borrow_mut() = Ledger::new()); + OFFSCREEN.with(|o| o.set(false)); + STRAIGHTENED.with(|s| s.set(false)); +} + +/// Has this frame drawn anything onto the presented surface since [`frame_begin`]? +pub fn frame_drew() -> bool { + LEDGER.with(|l| !l.borrow().is_empty()) +} + +thread_local! { + /// Set by the pass, cleared by [`frame_begin`]: the surface now holds STRAIGHT colour. + /// + /// THREAD-LOCAL like `VIDEO_PLANE_FRAME` beside the doors that read it, and for the same reason: + /// it is draw state, the draw is one thread's, and a process-wide flag would let one test's + /// straightened surface refuse another test's perfectly ordinary snapshot. + static STRAIGHTENED: std::cell::Cell = const { std::cell::Cell::new(false) }; +} + +/// Has the pass run on this frame? Past it, framebuffer 0 holds straight colour, so a read of it (a +/// snapshot, a glass source, a field kick) would cache a picture the premultiplied blend cannot +/// composite: `video_plane_refuses` answers `true` for every door once this is. +pub fn straightened() -> bool { + STRAIGHTENED.with(|s| s.get()) +} + +// ---- what to do: a pure decision ------------------------------------------------------------------ + +/// The framebuffer-fetch extension a driver offers, and the program variant that reads it. +#[derive(Clone, Copy, Debug, PartialEq, Eq)] +pub enum Fetch { + /// `GL_ARM_shader_framebuffer_fetch`: `gl_LastFragColorARM`. The Mali blob on the dev set. + Arm, + /// `GL_EXT_shader_framebuffer_fetch`: `gl_LastFragData[0]`. PowerVR and others. + Ext, +} + +/// Why the pass did not run. +#[derive(Clone, Copy, Debug, PartialEq, Eq)] +pub enum Skip { + /// The surface is not over the bound video plane: it is opaque, and a pass would be an identity. + NoPlane, + /// Nothing reached the surface since the clear: the frame is transparent black, a fixed point. + NothingDrawn, + /// Neither program linked (logged once at init); the frame goes out premultiplied, as before. + NoProgram, +} + +/// What the pass does this frame. +#[derive(Clone, Copy, Debug, PartialEq, Eq)] +pub enum Plan { + /// Not at all. + Skip(Skip), + /// In place, reading the tile through this extension. + Fetch(Fetch), + /// Through a copy of the frame. + Copy, +} + +/// **The pass's schedule**: it runs on a frame over the BOUND plane that DREW something, in place when +/// the driver can fetch the framebuffer and through a copy when it cannot. `fetch` is the extension the +/// fetch program was linked for (`None` when it was not, for any reason) and `copy` says the copy +/// program linked. +pub fn plan(plane_bound: bool, drew: bool, fetch: Option, copy: bool) -> Plan { + if !plane_bound { + return Plan::Skip(Skip::NoPlane); + } + if !drew { + return Plan::Skip(Skip::NothingDrawn); + } + match (fetch, copy) { + (Some(f), _) => Plan::Fetch(f), + (None, true) => Plan::Copy, + (None, false) => Plan::Skip(Skip::NoProgram), + } +} + +/// Is `name` one of the space-separated entries of a `GL_EXTENSIONS` string? A WHOLE token, never a +/// prefix: `GL_ARM_shader_framebuffer_fetch` is the first half of `GL_ARM_shader_framebuffer_fetch_ +/// depth_stencil`, which does not declare `gl_LastFragColorARM`. +fn extension_listed(list: &str, name: &str) -> bool { + list.split_ascii_whitespace().any(|e| e == name) +} + +/// The fetch extension a `GL_EXTENSIONS` string offers: ARM's when both are listed. +fn pick_fetch(list: &str) -> Option { + if extension_listed(list, "GL_ARM_shader_framebuffer_fetch") { + Some(Fetch::Arm) + } else if extension_listed(list, "GL_EXT_shader_framebuffer_fetch") { + Some(Fetch::Ext) + } else { + None + } +} + +// ---- the programs and the copy texture ---------------------------------------------------------- + +/// The fetch program, 0 when none linked, and the extension it reads (0 none, 1 ARM, 2 EXT). +static FETCH_PROG: AtomicU32 = AtomicU32::new(0); +static FETCH_KIND: AtomicU32 = AtomicU32::new(0); +/// The copy program, 0 when not linked (it is linked only when there is no fetch program). +static COPY_PROG: AtomicU32 = AtomicU32::new(0); +/// Where the linked program keeps `u_box`, the box a quad covers (`vs_straighten.vert`); -1 with none. +static BOX_LOC: AtomicI32 = AtomicI32::new(-1); +/// The copy fallback's texture and the viewport size it holds; 0 until the first copy. +static COPY_TEX: AtomicU32 = AtomicU32::new(0); +static COPY_W: AtomicU32 = AtomicU32::new(0); +static COPY_H: AtomicU32 = AtomicU32::new(0); + +const GL_EXTENSIONS: c_uint = 0x1F03; + +fn fetch_kind() -> Option { + match FETCH_KIND.load(Ordering::Relaxed) { + 1 => Some(Fetch::Arm), + 2 => Some(Fetch::Ext), + _ => None, + } +} + +/// The driver's extension string, empty when it cannot be read. A desktop core profile (the +/// simulator) answers `glGetString(GL_EXTENSIONS)` with `GL_INVALID_ENUM`, an error that would sit in +/// the queue for the first `glGetError` a cache probes, so a context that is not ES is not asked. +fn gl_extensions() -> String { + if !super::gl_is_es() { + return String::new(); + } + // SAFETY: render thread, current context; the string is NUL-terminated and driver-owned. + unsafe { + let p = glGetString(GL_EXTENSIONS); + if p.is_null() { + return String::new(); + } + CStr::from_ptr(p as *const c_char).to_string_lossy().into_owned() + } +} + +/// Link the pass's vertex shader with the fragment source `fs` and find the `u_box` it draws each box +/// through: `None` when either fails, so a program the pass could not place a quad with is never kept. +fn link_pass(fs: &CStr) -> Option<(c_uint, c_int)> { + let p = link_optional_program(VS_STRAIGHTEN.as_ptr(), fs.as_ptr())?; + // SAFETY: the render thread, in the context that linked `p`; a name lookup. + let loc = unsafe { glGetUniformLocation(p, c"u_box".as_ptr()) }; + (loc >= 0).then_some((p, loc)) +} + +/// Link the pass's program for this driver, once per GL context. Called by `init_gl`; a failure is +/// logged and leaves the pass off (or on its copy program), never a boot failure: a frame that goes +/// out premultiplied is the picture this app has always shown. +pub fn init() { + COPY_TEX.store(0, Ordering::Relaxed); + COPY_W.store(0, Ordering::Relaxed); + COPY_H.store(0, Ordering::Relaxed); + let (mut fetch_prog, mut kind, mut box_loc) = (0, 0, -1); + if let Some(f) = pick_fetch(&gl_extensions()) { + let fs = match f { + Fetch::Arm => FS_STRAIGHTEN_ARM, + Fetch::Ext => FS_STRAIGHTEN_EXT, + }; + match link_pass(fs) { + Some((p, loc)) => { + (fetch_prog, box_loc) = (p, loc); + kind = if f == Fetch::Arm { 1 } else { 2 }; + } + None => log(&format!("straighten: the {f:?} framebuffer-fetch program failed to build - copying instead")), + } + } + let copy_prog = if fetch_prog != 0 { + 0 + } else { + link_pass(FS_STRAIGHTEN_COPY).map_or(0, |(p, loc)| { + box_loc = loc; + p + }) + }; + FETCH_PROG.store(fetch_prog, Ordering::Relaxed); + FETCH_KIND.store(kind, Ordering::Relaxed); + COPY_PROG.store(copy_prog, Ordering::Relaxed); + BOX_LOC.store(box_loc, Ordering::Relaxed); + log(&format!( + "straighten: {}", + match (fetch_kind(), copy_prog != 0) { + (Some(Fetch::Arm), _) => "GL_ARM_shader_framebuffer_fetch - the pass reads the tile in place", + (Some(Fetch::Ext), _) => "GL_EXT_shader_framebuffer_fetch - the pass reads the tile in place", + (None, true) => "no framebuffer fetch - the pass copies the frame", + (None, false) => "no program - frames over the video plane stay premultiplied", + } + )); +} + +/// **The pass.** Turn this frame's premultiplied colour straight if the frame is over the bound plane +/// and drew something ([`plan`]); returns whether it ran. Call it LAST, after every draw and every +/// readback of the frame, before the swap; the surface is `straightened` from then on. +/// +/// It draws one quad over each box the frame drew into ([`Ledger`]) and over nothing else: a pixel no +/// draw touched is the transparent clear, a fixed point, and the boxes are disjoint, so every pixel the +/// pass visits it visits once. A box is a quad of its own and not a scissor: on this GPU a scissor +/// bounds what a pass may WRITE and the tiler still walks every tile under the geometry +/// (`gfx::culled`'s measurement), so only the quad's size removes the work. +/// +/// Restores nothing it does not own: the viewport and the presented framebuffer are rebound as the +/// frame's draw left them (the surface's own), the scissor is off (so is `LIVE_SCISSOR`) and the +/// blend is back on with the func the frame used. +pub fn straighten_surface(plane_bound: bool) -> bool { + let plan = plan(plane_bound, frame_drew(), fetch_kind(), COPY_PROG.load(Ordering::Relaxed) != 0); + let (vx, vy, vw, vh) = plx_base::surface::viewport(); + if matches!(plan, Plan::Skip(_)) || vw <= 0 || vh <= 0 { + return false; + } + let ledger = LEDGER.with(|l| *l.borrow()); + let mut boxes = quads(&ledger, [vx, vy, vx + vw, vy + vh]); + // Every box may lie off the viewport (a draw wholly outside it): nothing to straighten. + let Some(first) = boxes.next() else { return false }; + let loc = BOX_LOC.load(Ordering::Relaxed); + // SAFETY: the render thread, in the frame's own context, after the draw; plain GL state. + unsafe { + glBindFramebuffer(GL_FRAMEBUFFER, plx_base::surface::default_fb()); + glViewport(vx, vy, vw, vh); + scissor_off(); + glDisable(GL_BLEND); + match plan { + Plan::Fetch(_) => use_prog(FETCH_PROG.load(Ordering::Relaxed)), + Plan::Copy => { + let tex = copy_target(vw, vh); + glActiveTexture(GL_TEXTURE0); + glBindTexture(GL_TEXTURE_2D, tex); + glCopyTexSubImage2D(GL_TEXTURE_2D, 0, 0, 0, vx, vy, vw, vh); + use_prog(COPY_PROG.load(Ordering::Relaxed)); + } + Plan::Skip(_) => unreachable!("a skipped plan returned above"), + } + for q in std::iter::once(first).chain(boxes) { + glUniform4f(loc, q[0], q[1], q[2], q[3]); + glDrawArrays_raw(GL_TRIANGLE_STRIP, 0, 4); + } + if plan == Plan::Copy { + glBindTexture(GL_TEXTURE_2D, 0); + } + glEnable(GL_BLEND); + } + STRAIGHTENED.with(|s| s.set(true)); + true +} + +/// The copy fallback's texture, sized to the viewport (made again when the viewport changed) and +/// sampled NEAREST: the pass reads it 1:1 and a filter must not touch a pixel's colour. +unsafe fn copy_target(vw: c_int, vh: c_int) -> c_uint { + let (w, h) = (vw as u32, vh as u32); + let mut tex = COPY_TEX.load(Ordering::Relaxed); + if tex == 0 || COPY_W.load(Ordering::Relaxed) != w || COPY_H.load(Ordering::Relaxed) != h { + delete_tex(tex); + tex = cap_tex(vw, vh); + glBindTexture(GL_TEXTURE_2D, tex); + glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_MIN_FILTER, GL_NEAREST); + glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_MAG_FILTER, GL_NEAREST); + COPY_TEX.store(tex, Ordering::Relaxed); + COPY_W.store(w, Ordering::Relaxed); + COPY_H.store(h, Ordering::Relaxed); + } + tex +} + +// ---- the television's compositor, as arithmetic -------------------------------------------------- + +/// One GL blend factor the simulator's compositor model uses: the data the GL call +/// ([`UnderBlend::gl`]) and the host test's model ([`UnderBlend::apply`]) both read, so the test +/// checks the factors the simulator actually sets. +#[cfg(any(test, feature = "hostsim"))] +#[derive(Clone, Copy, Debug, PartialEq, Eq)] +pub enum BlendFactor { + One, + DstAlpha, + OneMinusDstAlpha, +} + +#[cfg(any(test, feature = "hostsim"))] +impl BlendFactor { + /// The `GLenum`. + pub const fn gl(self) -> c_uint { + match self { + BlendFactor::One => 0x0001, + BlendFactor::DstAlpha => 0x0304, + BlendFactor::OneMinusDstAlpha => 0x0305, + } + } + + /// The factor's value for a destination of alpha `dst_a`. + fn weight(self, dst_a: f32) -> f32 { + match self { + BlendFactor::One => 1.0, + BlendFactor::DstAlpha => dst_a, + BlendFactor::OneMinusDstAlpha => 1.0 - dst_a, + } + } +} + +/// A `glBlendFuncSeparate` as `(source, destination)` factors for colour and for alpha. +#[cfg(any(test, feature = "hostsim"))] +#[derive(Clone, Copy, Debug, PartialEq, Eq)] +pub struct UnderBlend { + pub rgb: (BlendFactor, BlendFactor), + pub alpha: (BlendFactor, BlendFactor), +} + +#[cfg(any(test, feature = "hostsim"))] +impl UnderBlend { + /// The four `glBlendFuncSeparate` arguments. + pub const fn gl(self) -> [c_uint; 4] { + [self.rgb.0.gl(), self.rgb.1.gl(), self.alpha.0.gl(), self.alpha.1.gl()] + } + + /// What the blend writes for a source `src` over a destination `dst` (`[r, g, b, a]`, 0..1): + /// `src * f_src + dst * f_dst`, clamped, which is what the GL blend equation `FUNC_ADD` does. + pub fn apply(self, src: [f32; 4], dst: [f32; 4]) -> [f32; 4] { + let rgb = (self.rgb.0.weight(dst[3]), self.rgb.1.weight(dst[3])); + let a = (self.alpha.0.weight(dst[3]), self.alpha.1.weight(dst[3])); + let mut out = [0.0; 4]; + for i in 0..3 { + out[i] = (src[i] * rgb.0 + dst[i] * rgb.1).clamp(0.0, 1.0); + } + out[3] = (src[3] * a.0 + dst[3] * a.1).clamp(0.0, 1.0); + out + } +} + +/// **The simulator's model of the television's compositor**: the decoded picture drawn UNDER the +/// frame, `out.rgb = video * (1 - ui.a) + ui.rgb * ui.a`, the straight-alpha mix the mixer performs on +/// whatever the surface holds, and `out.a = 1` (the panel is opaque). It is the measured arithmetic, +/// not what the frame was meant to show: a premultiplied pixel it is given comes out multiplied by its +/// own alpha, which is exactly the fault the pass exists to remove, so the simulator shows the bug +/// without the pass and the fix with it. +#[cfg(any(test, feature = "hostsim"))] +pub const UNDER: UnderBlend = UnderBlend { + rgb: (BlendFactor::OneMinusDstAlpha, BlendFactor::DstAlpha), + alpha: (BlendFactor::OneMinusDstAlpha, BlendFactor::One), +}; + +// ---- where the frame drew: the boxes the pass covers ----------------------------------------------- + +/// What a draw call writes to the surface, as the draw site declares it (see [`Ledger`]). +#[derive(Clone, Copy, Debug, PartialEq)] +pub enum Extent { + /// The authored-pixel quad `(x, y, w, h)` the vertex shader places the unit quad over: the very + /// numbers the draw's `*_RECT` uniform holds. + Quad([f32; 4]), + /// Whatever target is bound, edge to edge: a draw with no rect to place it, and the fallback for + /// every extent that is not known. The pass then covers the whole viewport. + Target, +} + +impl Extent { + /// The quad `(x, y, w, h)`. + pub const fn quad(x: f32, y: f32, w: f32, h: f32) -> Self { + Extent::Quad([x, y, w, h]) + } +} + +/// Edges past this (in either direction) are clamped to it, so a box that means "everything" and a +/// union of two boxes can neither overflow an `i32` nor an `i64` area. +const FAR: i32 = 1 << 28; +/// The box that stands for the whole surface (an unknown extent): every viewport lies inside it. +const EVERYTHING: PxBox = [-FAR, -FAR, FAR, FAR]; +/// Pixels of slack on every side of a quad's box, for the rounding between the CPU's idea of the +/// quad and the pixels the rasterizer lights: the vertex stage is fp32 and the rasterizer snaps to +/// sub-pixel positions, neither of which a whole-pixel `floor`/`ceil` accounts for. +const MARGIN: i32 = 1; +/// How many boxes the pass draws at most. A bound on work, never on content: past it the two boxes +/// that waste the fewest pixels together become their bounding box, so every drawn pixel stays covered. +const SLOTS: usize = 16; + +use super::PxBox; + +/// The window-pixel box (GL's bottom-left origin, far edges exclusive) that holds every pixel the +/// authored quad `q` can light, for a viewport at `(vx, vy)` and `s` window pixels per authored pixel. +/// `None` for a quad with no area, which draws nothing; [`EVERYTHING`] for one that is not finite. +fn quad_box(q: [f32; 4], (vx, vy): (i32, i32), s: f32) -> Option { + let [x, y, w, h] = q; + if !(x.is_finite() && y.is_finite() && w.is_finite() && h.is_finite() && s > 0.0 && s.is_finite()) { + return Some(EVERYTHING); + } + if w == 0.0 || h == 0.0 { + return None; + } + let (ax0, ax1) = (x.min(x + w), x.max(x + w)); + let (ay0, ay1) = (y.min(y + h), y.max(y + h)); + let lo = |v: f32| v.floor().clamp(-(FAR as f32), FAR as f32) as i32; + let hi = |v: f32| v.ceil().clamp(-(FAR as f32), FAR as f32) as i32; + // Rows count upward from the bottom of the target, authored y downward from the top. + let (wy0, wy1) = (vy as f32 + (plx_base::surface::LOGICAL_H - ay1) * s, vy as f32 + (plx_base::surface::LOGICAL_H - ay0) * s); + Some([ + lo(vx as f32 + ax0 * s) - MARGIN, + lo(wy0) - MARGIN, + hi(vx as f32 + ax1 * s) + MARGIN, + hi(wy1) + MARGIN, + ]) +} + +/// **The boxes of the surface a frame drew into**, at most [`SLOTS`] of them and PAIRWISE DISJOINT. +/// +/// Disjoint is the contract the pass leans on: its fetch program rewrites the pixel it finds, so a +/// pixel two quads cover is divided by its alpha twice. [`Ledger::add`] keeps it as an invariant (two +/// boxes that share a pixel become their bounding box, which may now share one with a third, and so +/// on); boxes that only touch share nothing and stay two. +#[derive(Clone, Copy)] +struct Ledger { + boxes: [PxBox; SLOTS], + n: usize, +} + +fn overlaps(a: PxBox, b: PxBox) -> bool { + a[0] < b[2] && b[0] < a[2] && a[1] < b[3] && b[1] < a[3] +} + +fn contains(a: PxBox, b: PxBox) -> bool { + a[0] <= b[0] && a[1] <= b[1] && b[2] <= a[2] && b[3] <= a[3] +} + +fn union(a: PxBox, b: PxBox) -> PxBox { + [a[0].min(b[0]), a[1].min(b[1]), a[2].max(b[2]), a[3].max(b[3])] +} + +fn area(b: PxBox) -> i64 { + (b[2] - b[0]) as i64 * (b[3] - b[1]) as i64 +} + +impl Ledger { + const fn new() -> Self { + Ledger { boxes: [[0; 4]; SLOTS], n: 0 } + } + + fn is_empty(&self) -> bool { + self.n == 0 + } + + fn boxes(&self) -> &[PxBox] { + &self.boxes[..self.n] + } + + fn remove(&mut self, i: usize) { + self.n -= 1; + self.boxes[i] = self.boxes[self.n]; + } + + /// Book a box the frame drew into. Every call either returns having covered `b` or removes a box, + /// so it ends; every pixel of every box ever booked stays inside one of the boxes held. + fn add(&mut self, mut b: PxBox) { + loop { + if self.boxes().iter().any(|&e| contains(e, b)) { + return; + } + // A box that shares a pixel with `b` is folded into it, and `b` looks again: the union + // may now share one with another. + if let Some(i) = self.boxes().iter().position(|&e| overlaps(e, b)) { + b = union(self.boxes[i], b); + self.remove(i); + continue; + } + if self.n < SLOTS { + self.boxes[self.n] = b; + self.n += 1; + return; + } + // Full, and `b` is apart from all of them: fold it into the box it widens the least, the + // pair whose bounding box wastes the fewest pixels (what is covered beyond the two). + let waste = |&e: &PxBox| area(union(e, b)) - area(e) - area(b); + let i = (0..self.n).min_by_key(|&i| waste(&self.boxes[i])).expect("SLOTS is not zero"); + b = union(self.boxes[i], b); + self.remove(i); + } + } + + /// The boxes the pass draws over: the ledger's, cut to the viewport `view`, a box with nothing + /// left of it dropped. Cutting keeps them disjoint. + fn covering(&self, view: PxBox) -> impl Iterator + '_ { + self.boxes().iter().filter_map(move |&b| super::clip_px(b, Some(view))) + } +} + +/// The pass's quads for `ledger` over the viewport `view`: each box as the `(x0, y0, x1, y1)` +/// fractions of the viewport (`0, 0` its bottom-left) the vertex shader's `u_box` takes. Edges are whole +/// pixels over the viewport's size, so two boxes that touch compute the same fraction for the edge +/// they share and the rasterizer's fill rule gives it to exactly one of them. +fn quads(ledger: &Ledger, view: PxBox) -> impl Iterator + '_ { + let (w, h) = ((view[2] - view[0]) as f32, (view[3] - view[1]) as f32); + ledger.covering(view).map(move |b| { + [(b[0] - view[0]) as f32 / w, (b[1] - view[1]) as f32 / h, (b[2] - view[0]) as f32 / w, (b[3] - view[1]) as f32 / h] + }) +} + +#[cfg(test)] +mod tests { + use super::*; + + /// The television's compositor in closed form: the measured relation, on 8-bit channels, as + /// floats in codes (no rounding, so a test sees the model's own error and not a second one). + fn tv_panel(ui: [u8; 4], video: [u8; 3]) -> [f32; 3] { + let a = ui[3] as f32 / 255.0; + [0, 1, 2].map(|i| ui[i] as f32 * a + video[i] as f32 * (1.0 - a)) + } + + /// What the frame MEANT the panel to show: its premultiplied colour over the picture. + fn intended_panel(premult: [u8; 4], video: [u8; 3]) -> [f32; 3] { + let a = premult[3] as f32 / 255.0; + [0, 1, 2].map(|i| premult[i] as f32 + video[i] as f32 * (1.0 - a)) + } + + /// The pass in integer codes: `fs_straighten.frag`'s arithmetic, `1 / a` and one multiply in + /// floats and the 8-bit write's round-to-nearest. + fn straighten_px(px: [u8; 4]) -> [u8; 4] { + let a = px[3] as f32 / 255.0; + let k = if a > 0.0 { 1.0 / a } else { 0.0 }; + let c = |v: u8| ((v as f32 / 255.0 * k).clamp(0.0, 1.0) * 255.0 + 0.5).floor() as u8; + [c(px[0]), c(px[1]), c(px[2]), px[3]] + } + + fn unit(p: [u8; 4]) -> [f32; 4] { + p.map(|v| v as f32 / 255.0) + } + + /// The pixel the dev set's three plane captures were taken of: a pale translucent subtitle pixel, + /// premultiplied, alpha 70/255, over the video `(226, 213, 51)`. + const SUBTITLE: [u8; 4] = [57, 64, 70, 70]; + const VIDEO: [u8; 3] = [226, 213, 51]; + /// What the panel showed (DISPLAY capture). + const DISPLAY: [f32; 3] = [182.0, 172.0, 55.0]; + + fn within(got: [f32; 3], want: [f32; 3], codes: f32) -> bool { + (0..3).all(|i| (got[i] - want[i]).abs() <= codes) + } + + /// **The simulator's compositor does what the television's does.** Drawing the picture under a + /// frame with `UNDER`'s factors, over the unconverted premultiplied pixel of the dev set's capture, + /// gives the DISPLAY the television showed - not the premultiplied picture `(221, 218, 107)` the + /// sim used to show for it (its old `(ONE_MINUS_DST_ALPHA, ONE)` factors). + #[test] + fn the_simulator_composites_a_frame_as_the_television_does() { + let video = [VIDEO[0], VIDEO[1], VIDEO[2], 255].map(|v| v as f32 / 255.0); + let panel = UNDER.apply(video, unit(SUBTITLE)); + let codes = [panel[0] * 255.0, panel[1] * 255.0, panel[2] * 255.0]; + assert!(within(codes, DISPLAY, 3.0), "the sim's panel {codes:?} against the captured DISPLAY {DISPLAY:?}"); + assert!( + !within(codes, intended_panel(SUBTITLE, VIDEO), 25.0), + "…which is far from the premultiplied picture the frame meant ({:?})", + intended_panel(SUBTITLE, VIDEO) + ); + assert_eq!(panel[3], 1.0, "the composited panel is opaque"); + } + + /// The factors are the closed form, not a look-alike: for every alpha and a spread of colours the + /// blend `UNDER` names equals `ui.rgb * ui.a + video * (1 - ui.a)`. + #[test] + fn the_under_blend_is_the_measured_straight_alpha_mix() { + for a in 0..=255u8 { + for &c in &[0u8, 1, 57, 128, 200, 255] { + for &v in &[0u8, 51, 226, 255] { + let ui = [c, c / 2, 255 - c, a]; + let video = [v, 255 - v, v / 3]; + let blend = UNDER.apply([video[0], video[1], video[2], 255].map(|x| x as f32 / 255.0), unit(ui)); + let want = tv_panel(ui, video); + let got = [blend[0] * 255.0, blend[1] * 255.0, blend[2] * 255.0]; + assert!(within(got, want, 0.01), "ui {ui:?} over {video:?}: {got:?} vs {want:?}"); + } + } + } + } + + /// The `GLenum`s are the ones GL names; a typo in a constant would blend with another factor. + #[test] + fn the_under_blend_hands_gl_its_factors() { + assert_eq!(UNDER.gl(), [0x0305, 0x0304, 0x0305, 0x0001], "ONE_MINUS_DST_ALPHA, DST_ALPHA, ONE_MINUS_DST_ALPHA, ONE"); + } + + /// **The pass fixes the capture.** Straightened first, the dev set's pixel composites to the + /// premultiplied picture `(221, 218, 107)` the frame was drawn as, through the same compositor + /// model that gives DISPLAY without it. + #[test] + fn the_pass_makes_the_captured_pixel_show_what_was_drawn() { + let straight = straighten_px(SUBTITLE); + assert_eq!(straight[3], SUBTITLE[3], "alpha is the compositor's and is not touched"); + let video = [VIDEO[0], VIDEO[1], VIDEO[2], 255].map(|v| v as f32 / 255.0); + let panel = UNDER.apply(video, unit(straight)); + let codes = [panel[0] * 255.0, panel[1] * 255.0, panel[2] * 255.0]; + assert!(within(codes, [221.0, 218.0, 107.0], 1.5), "the panel with the pass: {codes:?}"); + assert!(within(codes, intended_panel(SUBTITLE, VIDEO), 1.0), "within a code of the picture the frame drew"); + } + + /// **For every premultiplied pixel the pass lands within a code of the intended panel**, over a + /// bright, a dark and a mid picture: the round trip `rgb / a`, rounded to 8 bits, then multiplied + /// by `a` by the compositor loses at most half a code (`0.5 * a`). + #[test] + fn the_pass_is_within_a_code_of_the_intended_panel_for_every_premultiplied_pixel() { + let mut worst = 0.0f32; + for a in 0..=255u8 { + for c in 0..=a { + for &v in &[0u8, 100, 255] { + let premult = [c, a - c, c / 2, a]; + let video = [v, v, v]; + let shown = tv_panel(straighten_px(premult), video); + let want = intended_panel(premult, video); + for i in 0..3 { + worst = worst.max((shown[i] - want[i]).abs()); + } + } + } + } + assert!(worst <= 0.5 + 1e-3, "the worst error is {worst} codes"); + } + + /// An opaque pixel and a hole are fixed points, which is what lets the pass skip a frame with + /// nothing on it and what leaves an opaque ground (the trailer preview's shelves) as it was. + #[test] + fn opaque_pixels_and_holes_are_fixed_points() { + for c in 0..=255u8 { + assert_eq!(straighten_px([c, 255 - c, c / 3, 255]), [c, 255 - c, c / 3, 255]); + } + assert_eq!(straighten_px([0, 0, 0, 0]), [0, 0, 0, 0]); + assert_eq!(straighten_px([200, 10, 90, 0]), [0, 0, 0, 0], "no colour survives a zero alpha: it would be a/0"); + } + + /// A colour above its alpha (an additive rim: the premultiplied invariant broken on purpose) + /// saturates at white-in-that-channel; it never wraps. + #[test] + fn a_colour_above_its_alpha_saturates() { + assert_eq!(straighten_px([200, 50, 100, 100]), [255, 128, 255, 100]); + } + + /// The simulator's compositor model and the pass are a pair: without the pass the sim shows the + /// darkening for a translucent pixel, and with it the same pixel is the one the frame drew. + #[test] + fn without_the_pass_a_translucent_pixel_is_darker_by_its_alpha() { + let a = SUBTITLE[3] as f32 / 255.0; + let shown = tv_panel(SUBTITLE, VIDEO); + let meant = intended_panel(SUBTITLE, VIDEO); + assert!(shown[0] < meant[0] - 30.0 && shown[1] < meant[1] - 30.0, "darkened: {shown:?} vs {meant:?}"); + // The darkening is exactly the colour the multiply removed: `ui.rgb * (1 - a)`. + for i in 0..3 { + assert!((meant[i] - shown[i] - SUBTITLE[i] as f32 * (1.0 - a)).abs() < 1e-3); + } + } + + #[test] + fn the_pass_runs_over_the_bound_plane_when_the_frame_drew() { + assert_eq!(plan(true, true, Some(Fetch::Arm), false), Plan::Fetch(Fetch::Arm)); + assert_eq!(plan(true, true, Some(Fetch::Ext), true), Plan::Fetch(Fetch::Ext), "a fetch program wins over a copy one"); + assert_eq!(plan(true, true, None, true), Plan::Copy); + } + + #[test] + fn the_pass_does_not_run_off_the_plane_or_on_an_empty_frame() { + assert_eq!(plan(false, true, Some(Fetch::Arm), true), Plan::Skip(Skip::NoPlane), "an opaque frame needs none"); + assert_eq!(plan(false, false, None, false), Plan::Skip(Skip::NoPlane)); + assert_eq!( + plan(true, false, Some(Fetch::Arm), true), + Plan::Skip(Skip::NothingDrawn), + "the plane presents every frame, nearly all of them a bare clear: no full-viewport pass for those" + ); + assert_eq!(plan(true, true, None, false), Plan::Skip(Skip::NoProgram)); + } + + /// An extension is matched as a whole token: `..._fetch` must not match the `..._depth_stencil` + /// extension that merely starts with it, and the string's edges and an empty string are handled. + #[test] + fn the_fetch_extension_is_matched_as_a_whole_token() { + let arm = "GL_EXT_foo GL_ARM_shader_framebuffer_fetch GL_OES_bar"; + assert_eq!(pick_fetch(arm), Some(Fetch::Arm)); + assert_eq!(pick_fetch("GL_ARM_shader_framebuffer_fetch"), Some(Fetch::Arm), "the only entry"); + assert_eq!(pick_fetch("GL_ARM_shader_framebuffer_fetch GL_EXT_shader_framebuffer_fetch"), Some(Fetch::Arm), "ARM first"); + assert_eq!(pick_fetch("GL_EXT_shader_framebuffer_fetch"), Some(Fetch::Ext)); + assert_eq!(pick_fetch("GL_ARM_shader_framebuffer_fetch_depth_stencil"), None, "a prefix is not the extension"); + assert_eq!(pick_fetch("GL_EXT_shader_framebuffer_fetch_non_coherent"), None); + assert_eq!(pick_fetch(""), None); + assert_eq!(pick_fetch("GL_ARM_shader_framebuffer_fetch\nGL_OES_x"), Some(Fetch::Arm), "any ASCII whitespace separates"); + } + + /// The boxes the frame's ledger holds on the real (thread-local) ledger, on [`VIEW`]'s own scale: + /// the surface's viewport is not touched by this crate's tests, so a draw books at the identity + /// mapping. Cleared by [`frame_begin`]. + fn booked() -> Vec { + let view = { + let (vx, vy, vw, vh) = plx_base::surface::viewport(); + [vx, vy, vx + vw, vy + vh] + }; + LEDGER.with(|l| l.borrow().covering(view).collect()) + } + + /// **The draw ledger.** A frame that drew onto the surface is told so, [`frame_begin`] starts the + /// next one clean and lifts the straightened latch, and the latch is what the + /// framebuffer-sampling doors refuse on. + #[test] + fn the_ledger_books_draws_and_the_frame_start_clears_it() { + let _g = plx_base::testlock::serial(); + frame_begin(); + assert!(!frame_drew() && !straightened()); + note_draw(Extent::quad(100.0, 100.0, 200.0, 50.0)); + assert!(frame_drew()); + assert_eq!(booked().len(), 1); + STRAIGHTENED.with(|s| s.set(true)); + assert!(straightened()); + assert!(super::super::video_plane_refuses("test door"), "a door refuses once the surface is straight"); + frame_begin(); + assert!(!frame_drew() && !straightened() && booked().is_empty()); + assert!(!super::super::video_plane_refuses("test door"), "and is open again on the next frame"); + } + + /// **Only a draw onto the presented surface is booked.** A blur pass, a field reduction or a region + /// fill draws into a target of gfx's own, between a bind of it and the bind back to + /// `default_fb` (every pass ends that way, and `note_bind` reads it), and none of its pixels is the + /// surface's. A frame that did nothing else has nothing to straighten. + #[test] + fn a_draw_into_another_target_is_not_booked() { + let _g = plx_base::testlock::serial(); + frame_begin(); + note_bind(7); + note_draw(Extent::quad(0.0, 0.0, 1920.0, 1080.0)); + note_draw(Extent::Target); + note_clear(0.5); + assert!(!frame_drew(), "draws and a clear into a target that is not the surface book nothing"); + note_bind(plx_base::surface::default_fb()); + note_draw(Extent::quad(10.0, 10.0, 20.0, 20.0)); + assert!(frame_drew(), "and the bind back to the surface books again"); + frame_begin(); + } + + /// **A draw with no rect is the whole surface**, and so is a quad that is not a number: the pass + /// then covers the viewport, which is what it did before it knew where the frame drew. + #[test] + fn a_draw_without_an_extent_books_the_whole_surface() { + let _g = plx_base::testlock::serial(); + frame_begin(); + note_draw(Extent::quad(100.0, 100.0, 10.0, 10.0)); + note_draw(Extent::Target); + let (vx, vy, vw, vh) = plx_base::surface::viewport(); + assert_eq!(booked(), vec![[vx, vy, vx + vw, vy + vh]], "the viewport, as one box"); + frame_begin(); + note_draw(Extent::quad(f32::NAN, 0.0, 10.0, 10.0)); + assert_eq!(booked(), vec![[vx, vy, vx + vw, vy + vh]]); + frame_begin(); + } + + /// **A clear is booked only when the pass would not leave its pixels alone.** The two alphas the + /// app clears the surface to (0, the hole; 1, an opaque ground) are fixed points. Anything between + /// would leave every pixel translucent, and any other value of the colour channel is not asked. + #[test] + fn a_translucent_clear_books_the_whole_surface_and_the_two_the_app_makes_book_nothing() { + let _g = plx_base::testlock::serial(); + frame_begin(); + note_clear(0.0); + note_clear(1.0); + assert!(!frame_drew(), "a transparent hole and an opaque ground need no straightening"); + note_clear(0.5); + assert!(frame_drew()); + let (vx, vy, vw, vh) = plx_base::surface::viewport(); + assert_eq!(booked(), vec![[vx, vy, vx + vw, vy + vh]]); + frame_begin(); + note_clear(f32::NAN); + assert!(frame_drew(), "an alpha that is not a number is not known to be a fixed point"); + frame_begin(); + } + + /// **Every draw says where it lands**: the raw `glDrawArrays` import is called in exactly one place + /// per file that draws (its wrapper, which requires an [`Extent`]) and by the pass itself, so no + /// primitive can put pixels on the surface without the ledger knowing where. A box the ledger + /// misses is a translucent pixel the pass skips and the fault returns without a trace. The two + /// other calls that decide what the pass covers (`glBindFramebuffer`: which target; `glClear`: a + /// translucent ground) are shadowed the same way, each in `gfx.rs` alone. + #[test] + fn the_raw_calls_are_reached_only_through_the_wrappers_that_book_them() { + let calls = |src: &str, raw: &str| { + let prod = &src[..src.find("\n#[cfg(test)]\nmod tests").unwrap_or(src.len())]; + prod.lines() + .filter(|l| !l.trim_start().starts_with("//") && l.contains(&format!("{raw}("))) + .filter(|l| !l.contains(&format!("fn {raw}"))) + .count() + }; + let (gfx, text, pass) = (include_str!("../gfx.rs"), include_str!("../text.rs"), include_str!("straighten.rs")); + assert_eq!(calls(gfx, "glDrawArrays_raw"), 1, "gfx.rs: only the wrapper calls the raw import"); + assert_eq!(calls(text, "glDrawArrays_raw"), 1, "text.rs: only the wrapper calls the raw import"); + assert_eq!(calls(pass, "glDrawArrays_raw"), 1, "the pass's own draw is the one more"); + assert_eq!(calls(gfx, "glClear_raw"), 1, "gfx.rs: only the wrapper clears"); + assert_eq!(calls(gfx, "glBindFramebuffer_raw"), 1, "gfx.rs: only the wrapper binds"); + assert_eq!(calls(text, "glClear_raw") + calls(text, "glBindFramebuffer_raw"), 0, "text.rs binds and clears nothing"); + for (name, src, raw, note) in [ + ("gfx.rs", gfx, "glDrawArrays", "note_draw(extent)"), + ("text.rs", text, "glDrawArrays", "note_draw(extent)"), + ("gfx.rs", gfx, "glClear", "note_clear("), + ("gfx.rs", gfx, "glBindFramebuffer", "note_bind("), + ] { + let prod = &src[..src.find("\n#[cfg(test)]\nmod tests").unwrap_or(src.len())]; + let head = format!("unsafe fn {raw}("); + let wrapper = prod.find(&head).unwrap_or_else(|| panic!("{name}: the {raw} wrapper is gone")); + let body = &prod[wrapper..wrapper + prod[wrapper..].find("\n}\n").expect("its end")]; + assert!(body.contains(note), "{name}: the {raw} wrapper books the call"); + assert!( + body.find(note) < body.find(&format!("{raw}_raw(")), + "{name}: …and books it first, so a call that panics in the driver is still counted" + ); + } + assert!(include_str!("../gfx.rs").contains("extent: straighten::Extent) {"), "the draw wrapper REQUIRES the extent"); + } + + /// **The pass draws only over the ledger's boxes**: its loop is over [`quads`], one `u_box` and one + /// draw each, and the full-viewport draw it had is gone. + #[test] + fn the_pass_loops_over_the_ledgers_quads() { + let src = include_str!("straighten.rs"); + let prod = &src[..src.find("\n#[cfg(test)]\nmod tests").expect("the unit tests")]; + let from = prod.find("pub fn straighten_surface(").expect("the pass"); + let body = &prod[from..from + prod[from..].find("\n}\n").expect("its end")]; + assert!(body.contains("quads(&ledger,"), "the quads come from the ledger"); + assert!(body.contains("for q in std::iter::once(first).chain(boxes) {"), "one draw per quad"); + let draw = body.find("glDrawArrays_raw(").expect("the draw"); + assert!(body[..draw].rfind("glUniform4f(loc, q[0], q[1], q[2], q[3]);").is_some(), "each draw sets its own box first"); + assert_eq!(body.matches("glDrawArrays_raw(").count(), 1); + let vs = include_str!("../shaders/vs_straighten.vert"); + assert!(vs.contains("uniform highp vec4 u_box;") && vs.contains("v_uv = mix(u_box.xy, u_box.zw, a_pos);")); + assert!(vs.contains("gl_Position = vec4(v_uv * 2.0 - 1.0, 0.0, 1.0);"), "the box is placed as the fractions of the viewport"); + } + + /// The rows of the field mesh cover the part of the rect they span, and a whole-target extent has + /// no rect to cut. + #[test] + fn the_rows_of_the_field_mesh_book_the_part_of_the_rect_they_span() { + use super::super::{rows_extent, FIELD_N}; + let whole = Extent::quad(10.0, 100.0, 200.0, 160.0); + assert_eq!(rows_extent(whole, &(0..FIELD_N)), whole, "every row is the rect itself"); + assert_eq!(rows_extent(whole, &(0..FIELD_N / 2)), Extent::quad(10.0, 100.0, 200.0, 80.0), "the top half"); + assert_eq!(rows_extent(whole, &(FIELD_N / 2..FIELD_N)), Extent::quad(10.0, 180.0, 200.0, 80.0), "the bottom half"); + assert_eq!(rows_extent(Extent::Target, &(2..3)), Extent::Target); + } + + /// The shader is the CPU model: the divide is guarded at zero alpha, colour is multiplied by the + /// reciprocal, alpha passes through, and the two fetch variants name their extension's built-in. + #[test] + fn the_shader_is_the_arithmetic_the_model_checks() { + let fs = include_str!("../shaders/fs_straighten.frag"); + assert!(fs.contains("float k = c.a > 0.0 ? 1.0 / c.a : 0.0;")); + assert!(fs.contains("gl_FragColor = vec4(c.rgb * k, c.a);")); + assert!(fs.contains("#define PLX_BEHIND gl_LastFragColorARM")); + assert!(fs.contains("#define PLX_BEHIND gl_LastFragData[0]")); + assert!(fs.contains("#extension GL_ARM_shader_framebuffer_fetch : require")); + assert!(fs.contains("#extension GL_EXT_shader_framebuffer_fetch : require")); + assert!(fs.contains("texture2D(u_tex, v_uv)"), "the copy variant samples its copy"); + } + + // ---- the extent ledger --------------------------------------------------------------------------- + + /// A small viewport the model tests count pixels on: 192 x 108, with authored space at one tenth. + const VIEW: PxBox = [0, 0, 192, 108]; + const TENTH: f32 = 0.1; + + /// The box of an authored quad on [`VIEW`]. + fn at_tenth(q: [f32; 4]) -> Option { + quad_box(q, (0, 0), TENTH) + } + + /// The boxes the pass draws over for `ledger` on [`VIEW`]. + fn over_view(ledger: &Ledger) -> Vec { + ledger.covering(VIEW).collect() + } + + /// How many of `boxes` cover each pixel of [`VIEW`], row-major. + fn coverage(boxes: &[PxBox]) -> Vec { + let (w, h) = ((VIEW[2] - VIEW[0]) as usize, (VIEW[3] - VIEW[1]) as usize); + let mut count = vec![0u8; w * h]; + for b in boxes { + for y in b[1].max(VIEW[1])..b[3].min(VIEW[3]) { + for x in b[0].max(VIEW[0])..b[2].min(VIEW[2]) { + count[y as usize * w + x as usize] += 1; + } + } + } + count + } + + /// A deterministic spread of authored quads: small and large, near and far, some on the edges. + fn spread(n: usize) -> Vec<[f32; 4]> { + let mut s = 0x9E37_79B9u32; + let mut next = move |m: u32| { + s = s.wrapping_mul(1_664_525).wrapping_add(1_013_904_223); + (s >> 8) % m + }; + (0..n) + .map(|_| [next(2100) as f32 - 100.0, next(1200) as f32 - 60.0, 1.0 + next(700) as f32, 1.0 + next(300) as f32]) + .collect() + } + + /// **The pass covers every pixel any draw touched, once, and no other.** A few hundred quads of + /// every size are booked; each pixel of the viewport is then covered at most once (the fetch + /// program divides what it finds, so a pixel two quads cover would be divided twice: wrong colour, + /// and on the television only, because the simulator's copy program reads an untouched copy), and + /// exactly once wherever a draw's own box reaches. Without this the quads could not be trusted + /// to touch a translucent pixel at all. + #[test] + fn the_ledger_covers_every_drawn_pixel_exactly_once_and_no_other() { + let quads: Vec<[f32; 4]> = spread(300); + let mut ledger = Ledger::new(); + let mut drawn = Vec::new(); + for q in &quads { + if let Some(b) = at_tenth(*q) { + drawn.push(b); + ledger.add(b); + } + } + let covering = over_view(&ledger); + assert!(covering.len() <= SLOTS, "{} boxes", covering.len()); + let (got, want) = (coverage(&covering), coverage(&drawn)); + for (i, (g, w)) in got.iter().zip(&want).enumerate() { + assert!(*g <= 1, "pixel {i} is covered by {g} quads"); + assert!(*g >= 1 || *w == 0, "pixel {i} was drawn into and is not covered"); + } + } + + /// The pass is not the screen: a dozen small quads leave most of the viewport alone. (The property + /// above holds for the whole viewport too; this is the half that says the work went down.) + #[test] + fn a_frame_of_small_quads_touches_a_small_part_of_the_surface() { + let mut ledger = Ledger::new(); + for q in spread(12).iter().map(|q| [q[0], q[1], q[2] / 8.0, q[3] / 8.0]) { + if let Some(b) = at_tenth(q) { + ledger.add(b); + } + } + let touched = coverage(&over_view(&ledger)).iter().filter(|c| **c > 0).count(); + assert!(touched * 4 < 192 * 108, "{touched} of {} pixels", 192 * 108); + } + + /// With room to spare the pass covers exactly the drawn boxes: a frame with two small regions on + /// it is not widened to the screen, nor to their bounding box. + #[test] + fn a_few_separate_quads_are_covered_exactly() { + let a = at_tenth([100.0, 100.0, 300.0, 60.0]).unwrap(); + let b = at_tenth([1500.0, 800.0, 200.0, 100.0]).unwrap(); + let mut ledger = Ledger::new(); + ledger.add(a); + ledger.add(b); + let mut covering = over_view(&ledger); + covering.sort(); + let mut want = vec![a, b]; + want.sort(); + assert_eq!(covering, want); + let area = |boxes: &[PxBox]| coverage(boxes).iter().filter(|c| **c > 0).count(); + assert!(area(&covering) < 192 * 108 / 8, "two small regions are a small part of the surface"); + } + + /// A box inside a box already booked adds nothing, and two that share a pixel become one: the + /// shape every glyph over its scrim takes. + #[test] + fn a_contained_box_adds_nothing_and_overlapping_boxes_merge() { + let mut ledger = Ledger::new(); + ledger.add([10, 10, 100, 60]); + ledger.add([20, 20, 30, 30]); + assert_eq!(over_view(&ledger), vec![[10, 10, 100, 60]]); + ledger.add([90, 50, 150, 100]); + assert_eq!(over_view(&ledger), vec![[10, 10, 150, 100]], "the overlap folds both into their bounding box"); + } + + /// Boxes that only TOUCH share no pixel and stay two: the pass draws each once, and nothing is + /// widened over the corner they leave out. + #[test] + fn boxes_that_touch_stay_two() { + let mut ledger = Ledger::new(); + ledger.add([0, 0, 50, 50]); + ledger.add([50, 0, 100, 50]); + ledger.add([0, 50, 50, 60]); + let mut covering = over_view(&ledger); + covering.sort(); + assert_eq!(covering, vec![[0, 0, 50, 50], [0, 50, 50, 60], [50, 0, 100, 50]]); + } + + /// Past [`SLOTS`] separate boxes the two that waste the fewest pixels together are folded, and + /// nothing that was drawn is ever left out. + #[test] + fn past_the_slots_the_cheapest_pair_folds() { + let mut ledger = Ledger::new(); + let far: Vec = (0..SLOTS as i32).map(|i| [i * 11, 100 - i, i * 11 + 4, 104 - i]).collect(); + for b in &far { + ledger.add(*b); + } + assert_eq!(over_view(&ledger).len(), SLOTS); + // A seventeenth box two pixels from the first one: they are the cheapest pair. + ledger.add([0, 96, 4, 100]); + let covering = over_view(&ledger); + assert_eq!(covering.len(), SLOTS, "still at most {SLOTS} boxes"); + for b in &far { + assert!(covering.iter().any(|c| c[0] <= b[0] && c[1] <= b[1] && c[2] >= b[2] && c[3] >= b[3]), "{b:?} is not covered"); + } + assert!(covering.contains(&[0, 96, 4, 104]) || covering.iter().any(|c| c[0] == 0 && c[1] <= 96), "the new box folded into its neighbour"); + assert!(coverage(&covering).iter().all(|c| *c <= 1), "and no two boxes share a pixel"); + } + + /// **An extent that is not known is the whole surface.** A draw with no rect to place it, or one + /// whose rect is not a number, books the box that holds every viewport, and every later box folds + /// into it. + #[test] + fn an_unknown_extent_is_the_whole_surface() { + let mut ledger = Ledger::new(); + ledger.add([10, 10, 20, 20]); + ledger.add(EVERYTHING); + ledger.add([30, 30, 40, 40]); + assert_eq!(over_view(&ledger), vec![VIEW]); + assert_eq!(quad_box([f32::NAN, 0.0, 10.0, 10.0], (0, 0), 1.0), Some(EVERYTHING)); + assert_eq!(quad_box([0.0, 0.0, f32::INFINITY, 10.0], (0, 0), 1.0), Some(EVERYTHING)); + assert_eq!(quad_box([0.0, 0.0, 10.0, 10.0], (0, 0), 0.0), Some(EVERYTHING), "no scale is no mapping"); + } + + /// A quad becomes the window box it can reach: y is flipped (GL's rows count upward), the box is + /// rounded OUTWARD and given a pixel of slack, and the viewport's origin and scale are applied. + #[test] + fn a_quad_is_the_window_box_it_can_reach() { + assert_eq!(quad_box([100.0, 200.0, 300.0, 50.0], (0, 0), 1.0), Some([99, 829, 401, 881])); + assert_eq!(quad_box([100.5, 200.25, 10.0, 10.0], (0, 0), 1.0), Some([99, 868, 112, 881]), "outward on both axes"); + assert_eq!(quad_box([100.0, 200.0, 300.0, 50.0], (10, 20), 0.5), Some([59, 434, 211, 461]), "origin and scale"); + assert_eq!(quad_box([100.0, 200.0, -300.0, 50.0], (0, 0), 1.0), quad_box([-200.0, 200.0, 300.0, 50.0], (0, 0), 1.0), "a flipped quad is the same box"); + assert_eq!(quad_box([5.0, 5.0, 0.0, 10.0], (0, 0), 1.0), None, "no width, no pixels"); + assert_eq!(quad_box([5.0, 5.0, 10.0, 0.0], (0, 0), 1.0), None, "no height, no pixels"); + } + + /// **The pass's quads are the boxes and only them**: each quad's `u_box` maps back to a box of + /// the ledger cut to the viewport, a box wholly outside the viewport draws nothing, and the + /// pixels the quads cover are the pixels the ledger holds, no more. + #[test] + fn the_pass_draws_quads_only_over_the_ledger() { + let mut ledger = Ledger::new(); + ledger.add([20, 30, 80, 50]); + ledger.add([150, 90, 260, 400]); // runs off the right and top edges + ledger.add([-90, -90, -10, -10]); // wholly outside + let q: Vec<[f32; 4]> = quads(&ledger, VIEW).collect(); + assert_eq!(q.len(), 2, "the box outside the viewport is not drawn"); + let back = |q: &[f32; 4]| [(q[0] * 192.0).round() as i32, (q[1] * 108.0).round() as i32, (q[2] * 192.0).round() as i32, (q[3] * 108.0).round() as i32]; + let mut boxes: Vec = q.iter().map(back).collect(); + boxes.sort(); + assert_eq!(boxes, vec![[20, 30, 80, 50], [150, 90, 192, 108]]); + let covered = coverage(&boxes); + assert_eq!(covered.iter().filter(|c| **c > 0).count(), 60 * 20 + 42 * 18); + assert!(covered.iter().all(|c| *c <= 1)); + } + + /// An empty ledger has nothing to cover: the pass is not run for it. + #[test] + fn an_empty_ledger_covers_nothing() { + let ledger = Ledger::new(); + assert!(ledger.is_empty()); + assert!(quads(&ledger, VIEW).next().is_none()); + } + + /// The wiring no host test can drive: `init_gl` links the pass's programs. + #[test] + fn init_gl_links_the_pass() { + let src = include_str!("../gfx.rs"); + let from = src.find("pub fn init_gl()").expect("init_gl"); + let body = &src[from..from + src[from..].find("\n}\n").expect("its end")]; + assert!(body.contains("straighten::init()"), "init_gl must link the straighten programs"); + } +} diff --git a/rust-modules/gfx/src/rtl.rs b/rust-modules/gfx/src/rtl.rs index aefdcf0560..726264dc3f 100644 --- a/rust-modules/gfx/src/rtl.rs +++ b/rust-modules/gfx/src/rtl.rs @@ -12,7 +12,9 @@ //! //! **The engine is the one the subtitles already use.** `libass-plx` (`ci/build-libass.py`) links a //! pinned FreeType, HarfBuzz and FriBidi statically and exports only its `plx_ass_*` facade; ABI 4 -//! adds `plx_ass_text_*` (`src/ass_text.c`). This module binds that surface: +//! added `plx_ass_text_*` (`src/ass_text.c`), and this module binds that surface (the facade is at +//! ABI 5 now; its changes, `plx_ass_add_font`, `PlxAssStats` and the premultiplied output of +//! `plx_ass_render`, are all outside the `plx_ass_text_*` calls bound here): //! //! * [`bidi`]: FriBidi's resolved levels (after rule L1), the visual order (rule L2) and the //! mirrored brackets of one line. `text.rs` groups the line into runs from these. @@ -45,7 +47,7 @@ struct NativeRun { } /// The facade ABI this file is written against (`PLX_ASS_ABI_VERSION`); any other is not used. -const NATIVE_ABI: u32 = 4; +const NATIVE_ABI: u32 = 5; /// `plx_ass_text_bidi`'s own bound, mirrored so an over-long line is refused here without a call. const BIDI_MAX_CHARS: usize = 16_384; diff --git a/rust-modules/gfx/src/shaders/fs_straighten.frag b/rust-modules/gfx/src/shaders/fs_straighten.frag new file mode 100644 index 0000000000..55ec6c8174 --- /dev/null +++ b/rust-modules/gfx/src/shaders/fs_straighten.frag @@ -0,0 +1,50 @@ +// The STRAIGHTEN pass (`gfx/straighten.rs`): the frame's premultiplied colour turned into the straight +// colour the television's compositor multiplies by alpha. `rgb' = a > 0 ? rgb / a : 0`, alpha as it is. +// +// WHY. Over the hardware video plane the platform composites our surface as `ui.rgb * ui.a + video * +// (1 - ui.a)` (measured on the dev set: GRAPHIC (57, 64, 70, a 0.275), VIDEO (226, 213, 51), DISPLAY +// (182, 172, 55); straight alpha predicts (180, 172, 56), premultiplied (221, 218, 107)). The +// framebuffer holds `(rgb * a, a)` (`gfx.rs`'s blend note), so every translucent pixel would be +// multiplied by its own alpha a second time. Dividing it out here makes the compositor's multiply +// return the colour the frame drew: `(rgb / a) * a + video * (1 - a)` is `rgb + video * (1 - a)`. +// +// ONE SOURCE, THREE PROGRAMS, picked by the prefix `gfx::straighten` puts in front of it: +// PLX_STRAIGHTEN_ARM GL_ARM_shader_framebuffer_fetch - the colour already in the tile, read in place +// PLX_STRAIGHTEN_EXT GL_EXT_shader_framebuffer_fetch - the same read under the Khronos-era name +// (neither) a copy of the frame, sampled 1:1 - for a GPU with no fetch (the simulator) +// The two fetch programs read the pixel from tile memory: no copy, no second render pass, no memory +// traffic beyond what the frame already does. The copy program is the fallback and costs a copy of the +// frame plus a pass reload. +// +// `a == 0` writes zero colour and not `rgb / 0`: the compositor ignores such a pixel, but a capture of +// the graphics plane should read it as the nothing it is. A colour above its alpha (an additive rim) +// saturates at the 8-bit write, which is all a straight picture can say for it. +// +// PRECISION: the divide runs in highp (the fragment stage's highp is established on this device: +// `fs_field_panel.frag` declares highp varyings and uniforms). An fp16 `1 / a` followed by an fp16 +// product can land half a code off at the top of the range, and the divide is this program's whole job. +// What this file cannot choose is the precision of the value the fetch returns (`gl_LastFragColorARM` +// and `gl_LastFragData` are mediump in their extensions: an 8-bit value rounded to fp16, within +// ~0.13 code at 1.0) or of `gl_FragColor`; the f32 round-trip bound in `gfx/straighten.rs`'s tests is +// the model's, and the pass's real error is a code-level quantity only a TV capture can state. +#if defined(PLX_STRAIGHTEN_ARM) +#extension GL_ARM_shader_framebuffer_fetch : require +#define PLX_BEHIND gl_LastFragColorARM +#elif defined(PLX_STRAIGHTEN_EXT) +#extension GL_EXT_shader_framebuffer_fetch : require +#define PLX_BEHIND gl_LastFragData[0] +#endif +precision mediump float; +varying highp vec2 v_uv; +#if !defined(PLX_STRAIGHTEN_ARM) && !defined(PLX_STRAIGHTEN_EXT) +uniform sampler2D u_tex; +#endif +void main(){ +#if defined(PLX_STRAIGHTEN_ARM) || defined(PLX_STRAIGHTEN_EXT) + highp vec4 c = PLX_BEHIND; +#else + highp vec4 c = texture2D(u_tex, v_uv); +#endif + highp float k = c.a > 0.0 ? 1.0 / c.a : 0.0; + gl_FragColor = vec4(c.rgb * k, c.a); +} diff --git a/rust-modules/gfx/src/shaders/vs_straighten.vert b/rust-modules/gfx/src/shaders/vs_straighten.vert new file mode 100644 index 0000000000..2241a4713e --- /dev/null +++ b/rust-modules/gfx/src/shaders/vs_straighten.vert @@ -0,0 +1,19 @@ +// The STRAIGHTEN pass's vertex shader (`gfx/straighten.rs`): the shared unit quad, stretched over ONE +// box of the viewport. The pass draws a quad over each box the frame drew into and over nothing else +// (a pixel no draw touched is the transparent clear, which the pass leaves as it is), so the quad is +// not the viewport: `u_box` is the box as the fractions `(x0, y0, x1, y1)` of the viewport, `0, 0` its +// bottom-left, and `v_uv` is the pixel's place in the WHOLE viewport, which is all the COPY variant +// (`fs_straighten.frag` without a fetch extension) needs to read its own copy back 1:1. +// +// `mix` lands exactly on the box's edges at the unit quad's corners (0 and 1), so two boxes that touch +// compute the same position for the edge they share, and the rasterizer's fill rule gives each pixel +// of it to exactly one of them: the fetch program must not visit a pixel twice. Vertex shaders are +// highp on every GPU this runs on; the edges are whole pixels over the viewport's size, half a pixel +// from the nearest pixel centre. +attribute vec2 a_pos; +uniform highp vec4 u_box; +varying highp vec2 v_uv; +void main(){ + v_uv = mix(u_box.xy, u_box.zw, a_pos); + gl_Position = vec4(v_uv * 2.0 - 1.0, 0.0, 1.0); +} diff --git a/rust-modules/gfx/src/text.rs b/rust-modules/gfx/src/text.rs index 19f044ee57..ef2a430934 100644 --- a/rust-modules/gfx/src/text.rs +++ b/rust-modules/gfx/src/text.rs @@ -226,7 +226,20 @@ extern "C" { fn glUniform4fv(loc: c_int, count: c_int, value: *const f32); fn glUniform1i(loc: c_int, x: c_int); fn glUniform4f(loc: c_int, x: f32, y: f32, z: f32, w: f32); - fn glDrawArrays(mode: c_uint, first: c_int, count: c_int); + /// The raw import: only [`glDrawArrays`] below calls it. + #[link_name = "glDrawArrays"] + fn glDrawArrays_raw(mode: c_uint, first: c_int, count: c_int); +} + +/// A draw call, booked in the straighten pass's ledger first with the extent it writes +/// (`gfx::straighten::note_draw`): the pass draws over the boxes the frame drew into, and a glyph is a +/// draw like any other. Shadows the raw import so the call sites below keep their spelling and add the +/// extent ([`crate::gfx::set_rect`]'s answer for the quad's rect uniform); the same wrapper `gfx.rs` has. +#[allow(non_snake_case)] +#[inline(always)] +unsafe fn glDrawArrays(mode: c_uint, first: c_int, count: c_int, extent: crate::gfx::straighten::Extent) { + crate::gfx::straighten::note_draw(extent); + glDrawArrays_raw(mode, first, count) } static mut TPROG: c_uint = 0; @@ -341,8 +354,8 @@ pub fn draw_mask_inked(tex: c_uint, x: f32, y: f32, w: f32, h: f32, tint: *const } glUniform4fv(TLI_COL, 1, tint); glBindTexture(GL_TEXTURE_2D, tex); - glUniform4f(TLI_RECT, x, y, w, h); - glDrawArrays(GL_TRIANGLE_STRIP, 0, 4); + let ext = crate::gfx::set_rect(TLI_RECT, x, y, w, h); + glDrawArrays(GL_TRIANGLE_STRIP, 0, 4, ext); } true } @@ -2863,16 +2876,16 @@ pub fn draw_text( if bind_ink(&clip) { glUniform4fv(TLI_COL, 1, col); glBindTexture(GL_TEXTURE_2D, p.tex); - glUniform4f(TLI_RECT, ox, oy, p.quad.w, p.quad.h); - glDrawArrays(GL_TRIANGLE_STRIP, 0, 4); + let ext = crate::gfx::set_rect(TLI_RECT, ox, oy, p.quad.w, p.quad.h); + glDrawArrays(GL_TRIANGLE_STRIP, 0, 4, ext); return p.w; } } crate::gfx::use_prog(TPROG); // TL_SCREEN / TL_TEX / texture unit 0 set once at init glUniform4fv(TL_COL, 1, col); glBindTexture(GL_TEXTURE_2D, p.tex); - glUniform4f(TL_RECT, ox, oy, p.quad.w, p.quad.h); - glDrawArrays(GL_TRIANGLE_STRIP, 0, 4); + let ext = crate::gfx::set_rect(TL_RECT, ox, oy, p.quad.w, p.quad.h); + glDrawArrays(GL_TRIANGLE_STRIP, 0, 4, ext); p.w } } @@ -3089,8 +3102,8 @@ pub fn draw_text_fade( glUniform2f(TLF_VBOT, vb0, vb1); glBindTexture(GL_TEXTURE_2D, p.tex); let (ox, oy) = p.origin(zoom); - glUniform4f(TLF_RECT, ox, oy, p.quad.w, p.quad.h); - glDrawArrays(GL_TRIANGLE_STRIP, 0, 4); + let ext = crate::gfx::set_rect(TLF_RECT, ox, oy, p.quad.w, p.quad.h); + glDrawArrays(GL_TRIANGLE_STRIP, 0, 4, ext); p.w } } diff --git a/rust-modules/media-codecs/src/subtitle_extract.rs b/rust-modules/media-codecs/src/subtitle_extract.rs index ced39f7197..099870cc98 100644 --- a/rust-modules/media-codecs/src/subtitle_extract.rs +++ b/rust-modules/media-codecs/src/subtitle_extract.rs @@ -29,18 +29,81 @@ pub struct AssTrack { pub authored: bool, } -/// Read ASS headers and font attachments from the container, with admission BEFORE any payload -/// copy. The caller supplies the presentation layer's exact font budget policy. +/// Take the font attachments out of the format `fmt`, right after `avformat_open_input` returns: +/// each font is copied once into the shared buffer the source store, and through it every libass +/// renderer, will hold, and the stream's `codecpar->extradata` it was read from is freed. An +/// attachment that is not a font is freed without a copy. Nothing else reads an attachment's +/// extradata, so the format keeps no copy of its own that the app owns (issue #1328: 17 fonts, 83 MB, +/// read three times over by libavformat and again by the store). +/// +/// libavformat still holds the Matroska demuxer's own parsed element of every attachment until the +/// context closes; no API reaches it without the demuxer's private structures, so it stays. The +/// stream's internal codec context also holds a copy (`avformat_open_input` fills it from the codec +/// parameters before this can run): on the main demuxer `avformat_find_stream_info` rebuilds that +/// context from the now-empty parameters and the copy goes, while a reader that never calls it +/// (the side reader) keeps it until close. +/// +/// Every font attachment is taken, whole: which of them a renderer is handed is its choice, by the +/// names a track uses, and no byte budget decides here what a file's own attachments may carry. /// /// # Safety -/// `fmt` and `streams` must name a live FFmpeg format and its full stream table. Every track -/// index must be a valid index into that table. +/// `fmt` must name a format context `avformat_open_input` has opened and nobody else is using. +pub unsafe fn detach_fonts(fmt: *mut AVFormatContext) -> Vec { + let mut fonts = Vec::new(); + let streams = (*fmt).streams; + for i in 0..(*fmt).nb_streams { + let st = *streams.add(i as usize); + let cp = ffi::stream_codecpar(st); + if (*cp).codec_type != ffi::AVMEDIA_TYPE_ATTACHMENT { continue; } + let len = (*cp).extradata_size.max(0) as usize; + if len >= 4 && !(*cp).extradata.is_null() { + let data = std::slice::from_raw_parts((*cp).extradata, len); + if matches!(&data[..4], b"\x00\x01\x00\x00" | b"OTTO" | b"ttcf" | b"true") { + let dict = *((st as *const u8).add(ffi::OFF_STREAM_METADATA) as *const *const AVDictionary); + let entry = if dict.is_null() { std::ptr::null_mut() } + else { ffi::av_dict_get(dict, c"filename".as_ptr(), std::ptr::null(), 0) }; + let name = if !entry.is_null() && !(*entry).value.is_null() { + std::ffi::CStr::from_ptr((*entry).value).to_string_lossy().into_owned() + } else { format!("attachment-{i}.ttf") }; + fonts.push(EmbeddedFont { name, data: data.into() }); + } + } + free_extradata(cp); + } + fonts +} + +/// [`detach_fonts`] for a reader that has no use for the fonts: the attachments are freed, not +/// copied. +/// +/// # Safety +/// As [`detach_fonts`]. +pub unsafe fn release_attachments(fmt: *mut AVFormatContext) { + let streams = (*fmt).streams; + for i in 0..(*fmt).nb_streams { + let cp = ffi::stream_codecpar(*streams.add(i as usize)); + if (*cp).codec_type == ffi::AVMEDIA_TYPE_ATTACHMENT { free_extradata(cp); } + } +} + +/// Free a stream's `codecpar->extradata` and say so in the size, as `avcodec_parameters_free` leaves +/// them. `av_freep` nulls the pointer. +unsafe fn free_extradata(cp: *mut ffi::AVCodecParameters) { + ffi::av_freep(std::ptr::addr_of_mut!((*cp).extradata) as *mut c_void); + (*cp).extradata_size = 0; +} + +/// Read ASS headers from the container and bring along the `fonts` [`detach_fonts`] took out of it. +/// A header is refused past `max_header_bytes` BEFORE its payload is copied. +/// +/// # Safety +/// `streams` must name a live FFmpeg format's full stream table. Every track index must be a valid +/// index into that table. pub unsafe fn collect_manifest( - fmt: *mut AVFormatContext, streams: *mut *mut AVStream, tracks: &[AssTrack], max_header_bytes: usize, - mut admit_font: impl FnMut(usize) -> bool, + fonts: Vec, ) -> Manifest { let mut headers = Vec::with_capacity(tracks.len()); for track in tracks { @@ -58,25 +121,7 @@ pub unsafe fn collect_manifest( headers.push(Some(std::slice::from_raw_parts((*cp).extradata, len).to_vec())); } } - let mut fonts = Vec::new(); - let mut font_bytes = 0usize; - for i in 0..(*fmt).nb_streams { - let st = *streams.add(i as usize); - let cp = ffi::stream_codecpar(st); - let len = (*cp).extradata_size.max(0) as usize; - if (*cp).codec_type != ffi::AVMEDIA_TYPE_ATTACHMENT || len < 4 || (*cp).extradata.is_null() { continue; } - let data = std::slice::from_raw_parts((*cp).extradata, len); - if !matches!(&data[..4], b"\x00\x01\x00\x00" | b"OTTO" | b"ttcf" | b"true") { continue; } - let dict = *((st as *const u8).add(ffi::OFF_STREAM_METADATA) as *const *const AVDictionary); - let entry = if dict.is_null() { std::ptr::null_mut() } - else { ffi::av_dict_get(dict, c"filename".as_ptr(), std::ptr::null(), 0) }; - let name = if !entry.is_null() && !(*entry).value.is_null() { - std::ffi::CStr::from_ptr((*entry).value).to_string_lossy().into_owned() - } else { format!("attachment-{i}.ttf") }; - if !admit_font(len) { continue; } - font_bytes += len; - fonts.push(EmbeddedFont { name, data: data.into() }); - } + let font_bytes = fonts.iter().map(|font| font.data.len()).sum(); Manifest { headers, fonts, font_bytes } } @@ -312,6 +357,9 @@ pub struct Job { tracks: Vec, kinds: Vec, decoders: ImageDecoders, + /// The file's font attachments, taken out of the format at open ([`detach_fonts`]) and handed + /// over by [`Job::manifest`]. Empty for a job that inspects no subtitles. + fonts: Vec, } impl Job { @@ -320,6 +368,7 @@ impl Job { let mut job = Self { fmt: std::ptr::null_mut(), pkt: std::ptr::null_mut(), input: Some(input), video_index: -1, tracks: Vec::new(), kinds: Vec::new(), decoders: ImageDecoders::empty(), + fonts: Vec::new(), }; // SAFETY: the source and AVIO state outlive this format context; Drop reverses every // allocation and the format pointer is updated after open_input even on failure. @@ -331,6 +380,7 @@ impl Job { let r = ffi::avformat_open_input(&mut fmt, std::ptr::null(), std::ptr::null_mut(), std::ptr::null_mut()); job.fmt = fmt; if r < 0 || job.fmt.is_null() { return Err(JobOpenError::Failed); } + if inspect_subtitles { job.fonts = detach_fonts(job.fmt); } else { release_attachments(job.fmt); } job.pkt = ffi::av_packet_alloc(); if job.pkt.is_null() { return Err(JobOpenError::Failed); } let streams = (*job.fmt).streams; @@ -368,11 +418,13 @@ impl Job { self.kinds = kinds; } - pub fn manifest(&self, max_header_bytes: usize, admit_font: impl FnMut(usize) -> bool) -> Manifest { + /// The ASS headers and the fonts. The fonts are handed over, not copied: a second call has none. + pub fn manifest(&mut self, max_header_bytes: usize) -> Manifest { let tracks: Vec<_> = self.tracks.iter().zip(&self.kinds).map(|(track, kind)| AssTrack { stream_index: track.stream_index, authored: matches!(kind, DecodeKind::Ass { .. }), }).collect(); - unsafe { collect_manifest(self.fmt, (*self.fmt).streams, &tracks, max_header_bytes, admit_font) } + let fonts = std::mem::take(&mut self.fonts); + unsafe { collect_manifest((*self.fmt).streams, &tracks, max_header_bytes, fonts) } } /// Seek the video clock to a nonnegative time. The caller distinguishes cancellation from diff --git a/rust-modules/media/src/ff_font_memory_tests.rs b/rust-modules/media/src/ff_font_memory_tests.rs new file mode 100644 index 0000000000..c208cf5b42 --- /dev/null +++ b/rust-modules/media/src/ff_font_memory_tests.rs @@ -0,0 +1,248 @@ +//! How many copies of a file's font attachments the process holds once the demuxer has handed them +//! over (issue #1328: 17 attached fonts, 83 MB, ~490 MB resident on a television with 1.6 GB). +//! +//! A Matroska font attachment is read by libavformat once and then kept in three places for as long +//! as the format context lives: the demuxer's own parsed element (`matroskadec`'s `EbmlBin`, freed at +//! `read_close`), the stream's `codecpar->extradata`, and the stream's internal codec context +//! (`avformat_open_input` ends in `update_stream_avctx`, which `avcodec_parameters_to_context`-copies +//! the extradata of EVERY stream, an attachment's included). The app then copied the fonts once more +//! into the source store. Only the `codecpar` copy is the app's to free (`detach_fonts`); the element +//! buffer is private to the demuxer, and the codec context's is private to libavformat until +//! `avformat_find_stream_info` rebuilds it from the codec parameters (the main demuxer runs it, so +//! it ends with the element buffer alone; the side reader does not, and keeps the context's copy +//! too). Those are the copies of the demux half; the renderer's own are `player/ass/tests.rs`. +//! +//! The meter is the C heap (`plx_base::testheap`): FFmpeg `malloc`s these buffers itself. Each +//! measurement runs in a fresh single-threaded process whose input the PARENT made, so neither +//! another test's allocations nor the making of the fixture is in the number, and a copy shows as +//! a multiple of the fonts' bytes: the assertions are on that multiple. +//! `PLX_ASS_FONT_FIXTURE=` measures a real file instead of the generated one, e.g. the +//! report's `tests/fixtures/make_ass_stress.py --fonts-over-budget` MKV (411 MB, never committed). +//! +//! Needs the host FFmpeg, libass is not involved, and a system `ffmpeg` to generate the fixture, so +//! it is `#[ignore]`d everywhere but `make check-ffmpeg`. + +use super::*; +use crate::player::ass_source; +use crate::player::subtitle_cue::{begin_sources, SideTracks}; +use plx_base::testheap; +use plx_media_codecs::subtitle_extract::{collect_manifest, detach_fonts, AssTrack, Job, ReadOutcome, ReadSeek}; +use plx_playback_contract::subtitle::DecodeKind; +use std::collections::HashMap; +use std::io::{Read, Seek, SeekFrom}; +use std::path::{Path, PathBuf}; + +const MIB: usize = 1024 * 1024; +const FACES: usize = 6; +const FACE_BYTES: usize = 8 * MIB; +const FIXTURE_ENV: &str = "PLX_ASS_FONT_FIXTURE"; +const SIDE_TEST: &str = "ff::subtitle_charset_tests::font_memory::the_side_reader_hands_the_fonts_over_without_keeping_a_copy"; +const MAIN_TEST: &str = "ff::subtitle_charset_tests::font_memory::the_main_demuxer_hands_the_fonts_over_without_keeping_a_copy"; + +/// A file read in place: no byte of it lives in the heap this test measures. +struct FileSource(std::fs::File, i64); + +impl ReadSeek for FileSource { + fn read(&mut self, dst: &mut [u8]) -> ReadOutcome { + match self.0.read(dst) { + Ok(0) => ReadOutcome::End, + Ok(n) => ReadOutcome::Bytes(n), + Err(_) => ReadOutcome::Failed, + } + } + fn seek(&mut self, absolute: i64) -> bool { + u64::try_from(absolute).is_ok_and(|to| self.0.seek(SeekFrom::Start(to)).is_ok()) + } + fn size(&self) -> i64 { self.1 } + fn cancelled(&self) -> bool { false } +} + +/// A scratch directory removed when the test is done with it, however it ends. +struct Scratch(PathBuf); + +impl Drop for Scratch { + fn drop(&mut self) { + let _ = std::fs::remove_dir_all(&self.0); + } +} + +/// Six 8 MiB faces attached to a 2 s clip with one ASS track, made in `dir` with the system `ffmpeg` +/// (`None`: there is none, and the test says so and stops). +fn make_fixture(dir: &Path) -> Option { + std::fs::create_dir_all(dir).ok()?; + let inter = std::fs::read(Path::new(env!("CARGO_MANIFEST_DIR")).join("../../pkg/appfont.ttf")).ok()?; + let mut args: Vec = ["-y", "-v", "error", "-f", "lavfi", "-i", "color=c=black:s=64x64:r=10:d=2", "-i", "sub.ass"] + .iter().map(|s| (*s).to_owned()).collect(); + args.extend(["-map", "0:v", "-map", "1:0", "-c:v", "mpeg4", "-c:s", "copy"].map(str::to_owned)); + for i in 0..FACES { + let face = crate::player::ass::synth_fonts::rename(&inter, &format!("Plx Heap {i}"), "Regular", Some(FACE_BYTES)); + std::fs::write(dir.join(format!("heap-{i}.ttf")), face).ok()?; + args.extend(["-attach".to_owned(), format!("heap-{i}.ttf"), format!("-metadata:s:t:{i}"), "mimetype=font/ttf".to_owned()]); + } + args.push("out.mkv".to_owned()); + std::fs::write( + dir.join("sub.ass"), + "[Script Info]\nScriptType: v4.00+\nPlayResX: 320\nPlayResY: 180\n\n[V4+ Styles]\n\ + Format: Name, Fontname, Fontsize, PrimaryColour, SecondaryColour, OutlineColour, BackColour, Bold, Italic, Underline, StrikeOut, ScaleX, ScaleY, Spacing, Angle, BorderStyle, Outline, Shadow, Alignment, MarginL, MarginR, MarginV, Encoding\n\ + Style: Default,Plx Heap 0,24,&H00FFFFFF,&H000000FF,&H00000000,&H00000000,0,0,0,0,100,100,0,0,1,0,0,2,0,0,0,1\n\n\ + [Events]\nFormat: Layer, Start, End, Style, Name, MarginL, MarginR, MarginV, Effect, Text\n\ + Dialogue: 0,0:00:00.00,0:00:01.00,Default,,0,0,0,,hello\n", + ).ok()?; + let made = std::process::Command::new("ffmpeg").current_dir(dir).args(&args).status().ok()?; + made.success().then(|| dir.join("out.mkv")) +} + +/// The heap a measurement added, over `base`, as the numbers the parent test asserts on. +fn report_over(base: usize, what: &str, now: usize, peak: usize) { + testheap::report(&format!("{what}_held"), now.saturating_sub(base) as u64); + testheap::report(&format!("{what}_peak"), peak.saturating_sub(base) as u64); +} + +/// The MKV the parent made, in the child. +fn fixture_in_child() -> PathBuf { + std::env::var_os(FIXTURE_ENV).map(PathBuf::from).expect("the parent test hands the fixture over") +} + +/// `Job::open` and `manifest`, as the side reader does (`ff_subside::open_side`), then the hand-over +/// to the store; and again after the job is closed, so the store's own copy is told apart from +/// libavformat's. +fn measure_side_reader() { + let _g = plx_base::testlock::serial(); + let _ffmpeg = bind_host_ffmpeg(); + ass_source::reset(); + crate::player::request_subtitle(0); + let file = std::fs::File::open(fixture_in_child()).expect("the fixture opens"); + let size = file.metadata().expect("the fixture has a size").len() as i64; + let base = testheap::live_bytes().expect("a host whose malloc can be asked"); + let ((held, closed, font_bytes), peak) = testheap::peak_during(|| { + let mut job = Job::open(Box::new(FileSource(file, size)), true).ok().expect("the side reader opens the fixture"); + let tracks = SideTracks::new(job.tracks()); + job.configure(tracks.decode_plan()); + let manifest = job.manifest(ass_source::MAX_HEADER_BYTES); + let font_bytes = manifest.font_bytes; + begin_sources("fixture:font-memory", tracks.kinds(), manifest); + let held = testheap::live_bytes().unwrap(); + drop(job); + (held, testheap::live_bytes().unwrap(), font_bytes) + }); + testheap::report("font_bytes", font_bytes as u64); + report_over(base, "open", held, peak); + testheap::report("store_only", closed.saturating_sub(base) as u64); +} + +/// `avformat_open_input`, `avformat_find_stream_info`, then the manifest: the order `ff.rs` opens the +/// film it plays in. +fn measure_main_demuxer() { + let _g = plx_base::testlock::serial(); + let _ffmpeg = bind_host_ffmpeg(); + ass_source::reset(); + crate::player::request_subtitle(0); + let path = CString::new(fixture_in_child().to_str().unwrap()).unwrap(); + let base = testheap::live_bytes().expect("a host whose malloc can be asked"); + let ((held, font_bytes), peak) = testheap::peak_during(|| unsafe { + let mut fmt = avformat_alloc_context(); + assert!(avformat_open_input(&mut fmt, path.as_ptr(), std::ptr::null_mut(), std::ptr::null_mut()) >= 0); + let fonts = detach_fonts(fmt); + assert!(avformat_find_stream_info(fmt, std::ptr::null_mut()) >= 0); + let streams = (*fmt).streams; + let infos = SubTracks::inspect(fmt, streams); + let presentation = SideTracks::new(&infos); + let tracks: Vec<_> = infos.iter().zip(presentation.decode_plan()).map(|(info, kind)| AssTrack { + stream_index: info.stream_index, + authored: matches!(kind, DecodeKind::Ass { .. }), + }).collect(); + let manifest = collect_manifest(streams, &tracks, ass_source::MAX_HEADER_BYTES, fonts); + let font_bytes = manifest.font_bytes; + begin_sources("fixture:font-memory", presentation.kinds(), manifest); + let held = testheap::live_bytes().unwrap(); + avformat_close_input(&mut fmt); + (held, font_bytes) + }); + testheap::report("font_bytes", font_bytes as u64); + report_over(base, "open", held, peak); + testheap::report("store_only", testheap::live_bytes().unwrap().saturating_sub(base) as u64); +} + +/// `bytes` as a multiple of the font bytes, for a failure message. +fn copies(bytes: u64, fonts: u64) -> f64 { + bytes as f64 / fonts as f64 +} + +/// The fresh process's numbers for `test`; `None` in the child itself (which measures and reports) +/// and when there is no `ffmpeg` to generate the fixture (said, like the other generated-MKV tests +/// do, and not a pass that measured nothing). +fn numbers(test: &str, tag: &str, child: fn()) -> Option> { + if testheap::is_child() { + child(); + return None; + } + let scratch = Scratch(std::env::temp_dir().join(format!("plx-fontmem-{}-{tag}", std::process::id()))); + let mkv = match std::env::var_os(FIXTURE_ENV) { + Some(given) => PathBuf::from(given), + None => match make_fixture(&scratch.0) { + Some(made) => made, + None => { + eprintln!("SKIPPED: no system ffmpeg to generate the fixture"); + return None; + } + }, + }; + Some(testheap::in_fresh_process_with_env(test, &[(FIXTURE_ENV, mkv.as_os_str())])) +} + +/// What libavformat holds once the manifest is taken, as a multiple of the fonts: everything live +/// but the store's own copy. +fn reader_copies(n: &HashMap) -> f64 { + copies(n["open_held"].saturating_sub(n["store_only"]), n["font_bytes"]) +} + +fn assert_fonts_handed_over(n: &HashMap, what: &str, libavformat_keeps: f64) { + let fonts = n["font_bytes"]; + eprintln!( + "{what}: {:.0} MiB of fonts; libavformat holds {:.2}x, the open peaks at {:.2}x, the store alone holds {:.2}x", + fonts as f64 / MIB as f64, + reader_copies(n), + copies(n["open_peak"], fonts), + copies(n["store_only"], fonts) + ); + assert!(fonts >= (FACES * FACE_BYTES) as u64 || std::env::var_os(FIXTURE_ENV).is_some(), "the fixture's fonts: {fonts}"); + // What libavformat keeps for itself (`libavformat_keeps` copies, with half a copy of room) is out + // of the app's reach; `codecpar->extradata`, the copy over that, is the app's and the hand-over + // frees it. + assert!( + reader_copies(n) < libavformat_keeps + 0.5, + "{what} holds the {:.0} MiB of fonts {:.2} more times after the hand-over, not {libavformat_keeps}: `codecpar->extradata` is still there", + fonts as f64 / MIB as f64, + reader_copies(n) + ); + assert!( + copies(n["open_peak"], fonts) < 3.5, + "{what} peaked at {:.2} times the fonts: the hand-over holds a copy beside the three libavformat made", + copies(n["open_peak"], fonts) + ); + assert!( + copies(n["store_only"], fonts) < 1.25, + "after {what} closes the store alone holds them, {:.2} times", + copies(n["store_only"], fonts) + ); +} + +/// The side reader (`Job`: `avformat_open_input` alone) used to hold the fonts three times once it had +/// handed them to the store (and a fourth time in the store); now libavformat's element buffer and +/// codec context are the two. +#[test] +#[ignore = "needs the bundled host FFmpeg and a system ffmpeg; run make check-ffmpeg"] +fn the_side_reader_hands_the_fonts_over_without_keeping_a_copy() { + let Some(n) = numbers(SIDE_TEST, "side", measure_side_reader) else { return }; + assert_fonts_handed_over(&n, "the open side reader", 2.0); +} + +/// The main demuxer takes the fonts right after `avformat_open_input`, then runs +/// `avformat_find_stream_info`, which rebuilds the stream's codec context from the now empty codec +/// parameters: libavformat is left with the element buffer alone. +#[test] +#[ignore = "needs the bundled host FFmpeg and a system ffmpeg; run make check-ffmpeg"] +fn the_main_demuxer_hands_the_fonts_over_without_keeping_a_copy() { + let Some(n) = numbers(MAIN_TEST, "main", measure_main_demuxer) else { return }; + assert_fonts_handed_over(&n, "the open demuxer", 1.0); +} diff --git a/rust-modules/media/src/ff_image_subtitle_tests.rs b/rust-modules/media/src/ff_image_subtitle_tests.rs index c081dcc083..3b1ae33253 100644 --- a/rust-modules/media/src/ff_image_subtitle_tests.rs +++ b/rust-modules/media/src/ff_image_subtitle_tests.rs @@ -15,7 +15,7 @@ use super::*; use crate::player::subtitle_cue::{SideTracks, SubCue}; use crate::player::SHARED; use crate::player::{ass, ass_source}; -use plx_media_codecs::subtitle_extract::{collect_manifest, AssTrack}; +use plx_media_codecs::subtitle_extract::{collect_manifest, detach_fonts, AssTrack}; use plx_playback_contract::subtitle::{CaptionRetention, DecodeKind}; use std::sync::Arc; @@ -62,9 +62,8 @@ impl AssFixture { stream_index: info.stream_index, authored: matches!(kind, DecodeKind::Ass { .. }), }).collect(); - let mut budget = ass::FontBudget::default(); - let manifest = collect_manifest(self.fmt, (*self.fmt).streams, &tracks, - ass_source::MAX_HEADER_BYTES, |len| budget.admit(len)); + let manifest = collect_manifest((*self.fmt).streams, &tracks, + ass_source::MAX_HEADER_BYTES, detach_fonts(self.fmt)); crate::player::subtitle_cue::begin_sources("fixture:ass-header", self.presentation.kinds(), manifest) } @@ -110,7 +109,7 @@ impl Drop for AssReset { fn ass_frame(source: Arc, now_ms: i64, visible: bool) -> Arc { let deadline = std::time::Instant::now() + std::time::Duration::from_secs(5); loop { - if let Some(frame) = ass::request(source.clone(), now_ms, 320, 180, 320, 180) { + if let Some(frame) = ass::request(source.clone(), now_ms, 320, 180, 320, 180, ass::Pace::PAUSED) { if frame.error.is_some() || !frame.rects.is_empty() == visible { return frame; } diff --git a/rust-modules/media/src/ff_subside.rs b/rust-modules/media/src/ff_subside.rs index ba54f46335..8c578f6c4d 100644 --- a/rust-modules/media/src/ff_subside.rs +++ b/rust-modules/media/src/ff_subside.rs @@ -237,8 +237,7 @@ fn open_side(cfg: &ReadCfg, stop: &Arc) -> Result<(Job, Option= count { return Err(RunEnd::Unusable); } diff --git a/rust-modules/media/src/ff_subtitle_charset_tests.rs b/rust-modules/media/src/ff_subtitle_charset_tests.rs index de407e75e5..333d37b4a1 100644 --- a/rust-modules/media/src/ff_subtitle_charset_tests.rs +++ b/rust-modules/media/src/ff_subtitle_charset_tests.rs @@ -16,7 +16,7 @@ use super::*; use crate::player::ass::Content; use crate::player::ass_source; use crate::player::subtitle_cue::{SideTracks, SubCue}; -use plx_media_codecs::subtitle_extract::{collect_manifest, AssTrack}; +use plx_media_codecs::subtitle_extract::{collect_manifest, detach_fonts, AssTrack}; use plx_playback_contract::subtitle::{CaptionRetention, DecodeKind}; fn fixture_retention() -> CaptionRetention { @@ -32,9 +32,7 @@ unsafe fn fixture_subs(fmt: *mut AVFormatContext, key: &str) -> (SubTracks, Side stream_index: info.stream_index, authored: matches!(kind, DecodeKind::Ass { .. }), }).collect(); - let mut budget = crate::player::ass::FontBudget::default(); - let manifest = collect_manifest(fmt, streams, &tracks, ass_source::MAX_HEADER_BYTES, - |len| budget.admit(len)); + let manifest = collect_manifest(streams, &tracks, ass_source::MAX_HEADER_BYTES, detach_fonts(fmt)); let generation = crate::player::subtitle_cue::begin_sources(key, presentation.kinds(), manifest); (SubTracks::open(streams, infos, plan), presentation, generation) } @@ -140,3 +138,8 @@ fn a_packet_without_a_duration_is_clipped_by_the_cue_after_it() { assert_eq!(events[0].duration_ms, starts[1] - starts[0], "the first cue ends where the second begins"); assert_eq!(events[1].duration_ms, 4000, "the last cue keeps the default"); } + +/// The memory the demuxer half of the font attachments costs (issue #1328): a module of its own, in the +/// `ff::subtitle_charset_tests` filter `make check-ffmpeg` already runs. +#[path = "ff_font_memory_tests.rs"] +mod font_memory; diff --git a/rust-modules/media/src/player/ass.rs b/rust-modules/media/src/player/ass.rs index 4ccbeda8e2..f8385dd5a4 100644 --- a/rust-modules/media/src/player/ass.rs +++ b/rust-modules/media/src/player/ass.rs @@ -1,13 +1,17 @@ -//! Styled subtitles, rendered by our pinned libass on one worker. The SDL thread +//! Styled subtitles, rendered by our pinned libass on a worker, and on up to two helpers while a +//! moving cue needs more frames than one worker can make (issue #1328); a moving stretch that the +//! renderers still cannot make thirty frames a second of is drawn at two thirds or half of the +//! output size (`scale`, `docs/ass-subtitles.md`). The SDL thread //! only replaces a bounded request slot and reads an immutable RGBA publication; //! it never parses fonts/events, rasterizes, or waits for the native renderer. use std::ffi::CString; use std::os::raw::{c_char, c_void}; -use std::sync::atomic::{AtomicU64, Ordering}; +use std::sync::atomic::{AtomicU64, AtomicUsize, Ordering}; use std::sync::{Arc, Mutex, OnceLock}; +use std::time::Duration; mod mailbox; -use mailbox::{Job, Kind, Mailbox, Rendered}; +use mailbox::{Job, Mailbox, Rendered}; use super::ass_timing::{self, Validity}; const MAX_SCRIPT_BYTES: usize = super::ass_script::WINDOW_BYTES; @@ -15,15 +19,20 @@ const MAX_EVENT_BYTES: usize = 256 * 1024; // No event-count bound here: `ass_source` holds an embedded track, and `ass_script` a long sidecar // script, to a window far under the native library's own guard (`plx_ass_chunk` refuses past // 20 000), so a second count would only be a way to hide a track that is otherwise showable. -pub const MAX_FONT_BYTES: usize = 32 * 1024 * 1024; -/// Bookkeeping charged per font on top of its bytes, so a file of countless tiny attachments is -/// bounded by the same byte budget without a font-count constant. -const FONT_OVERHEAD_BYTES: usize = 1024; +// No font-byte bound either (issue #1328): the file's attachments are held by the demuxer (the +// copies libavformat keeps for itself) and once by the store, and every renderer REFERENCES the +// store's buffer for the fonts the track names (`font_names`; `plx_ass_add_font_borrowed`) instead +// of copying it, so a font is resident once however many renderers draw with it, and only the fonts +// the track can draw are even opened. const MAX_PIXELS: usize = 3840 * 2160; const MAX_REGIONS: usize = 64; mod aligns; +mod font_names; +mod pool; +mod scale; pub use aligns::AlignSet; +pub use scale::RenderScale; #[derive(Clone, Debug, PartialEq, Eq, Hash)] pub struct Event { @@ -39,26 +48,6 @@ pub struct Font { pub data: Arc<[u8]>, } -/// The memory bound on embedded fonts: fonts are admitted in container order until their bytes -/// (plus a small per-font overhead) fill `MAX_FONT_BYTES`. A font that does not fit is skipped -/// and a smaller later one may still fit; text naming a skipped font renders in the app's -/// default face instead of the track being refused or its text dropped. -#[derive(Default)] -pub struct FontBudget { - used: usize, -} - -impl FontBudget { - pub fn admit(&mut self, len: usize) -> bool { - let cost = len.saturating_add(FONT_OVERHEAD_BYTES); - if len == 0 || self.used.saturating_add(cost) > MAX_FONT_BYTES { - return false; - } - self.used += cost; - true - } -} - #[derive(Clone, Debug)] pub enum Content { Embedded { @@ -120,7 +109,9 @@ pub struct Rect { pub y: i32, pub width: i32, pub height: i32, - /// Straight (not premultiplied) RGBA, composited in libass image-list order. + /// PREMULTIPLIED RGBA (no colour channel above its alpha), composited in libass image-list order + /// and published by the facade as composed: it is drawn with a premultiplied blend + /// (`Painter::tex_premultiplied`), never the straight-alpha one the image subtitles use. pub rgba: Arc<[u8]>, } @@ -137,7 +128,6 @@ pub enum Fault { Unavailable, FontsUnavailable, InitFailed, - FontTooLarge, Unreadable, FontName, TrackFonts, @@ -160,7 +150,6 @@ impl Fault { Fault::Unavailable => msg::widgets_ass_unavailable(), Fault::FontsUnavailable => msg::widgets_ass_fonts_unavailable(), Fault::InitFailed => msg::widgets_ass_init_failed(), - Fault::FontTooLarge => msg::widgets_ass_font_too_large(), Fault::Unreadable => msg::widgets_ass_unreadable(), Fault::FontName => msg::widgets_ass_font_name(), Fault::TrackFonts => msg::widgets_ass_track_fonts(), @@ -177,13 +166,40 @@ pub struct Frame { pub source_id: u64, /// Changes only when pixels/placement, canvas or the error state changes. pub serial: u64, + /// The output canvas the frame answers for: the size it was requested at, whatever size it was + /// drawn at (`scale`). pub width: i32, pub height: i32, - /// Disjoint regions; an empty list clears output between dialogue events. + /// Disjoint regions; an empty list clears output between dialogue events. Their coordinates are + /// in the canvas libass drew, which is the output canvas at `scale` ([`Frame::render_size`]): + /// [`Frame::output_edges`] takes one back to the output. pub rects: Vec, + /// The size the regions were drawn at, as a fraction of the output (issue #1328). + pub scale: RenderScale, pub error: Option, } +impl Frame { + /// The canvas the regions' coordinates are in: the output canvas at the frame's scale. + pub fn render_size(&self) -> (i32, i32) { + self.scale.render_size(self.width, self.height) + } + + /// Where `rect`, a region of this frame, lies on the output canvas: its left, top, right and + /// bottom edges. Both edges of each axis are scaled, never the origin and the size, so regions + /// that abut in the frame abut on the output, and a full-size frame maps to itself exactly. + pub fn output_edges(&self, rect: &Rect) -> (f32, f32, f32, f32) { + let (render_w, render_h) = self.render_size(); + let (sx, sy) = (self.width as f32 / render_w as f32, self.height as f32 / render_h as f32); + ( + rect.x as f32 * sx, + rect.y as f32 * sy, + (rect.x + rect.width) as f32 * sx, + (rect.y + rect.height) as f32 * sy, + ) + } +} + pub fn next_source_id() -> u64 { static NEXT: AtomicU64 = AtomicU64::new(1); NEXT.fetch_add(1, Ordering::Relaxed) @@ -202,11 +218,93 @@ struct Key { storage_height: i32, } +impl Key { + /// Both keys belong to one timeline: the same epoch, source and size, whatever the revision. + /// A revision alone moves when events are appended, which starts them ahead of the playhead. + fn same_timeline(&self, other: &Key) -> bool { + Key { revision: other.revision, ..*self } == *other + } +} + +/// How the playhead is moving, as values the caller measured: the mailbox reads no clock and no +/// registry, so everything it needs to pace a moving cue's frames arrives here (issue #1328). +#[derive(Clone, Copy, Debug, PartialEq)] +pub struct Pace { + /// The playhead is advancing (playing). Paused, loading or stepping: a moving cue is one frame + /// for one clock, as for a still cue. + pub running: bool, + /// Playhead milliseconds per wall-clock millisecond: 1.0 at normal speed. + pub rate: f64, + /// One video frame period in playhead milliseconds: no moving frame is asked for closer + /// together than this. + pub frame_ms: i64, + /// Wall-clock milliseconds between this UI tick and the one before it: a frame rendered for a + /// playhead is shown by the first tick at or after it, up to this much late. + pub tick_ms: i64, +} + +impl Pace { + /// A playhead that is not advancing: the pace of a paused or loading player. + pub const PAUSED: Pace = Pace { running: false, rate: 1.0, frame_ms: Pace::FILM_FRAME_MS, tick_ms: 16 }; + /// A source whose frame rate is unknown is treated as film (24 fps), as `speeds_within` does. + const FILM_FRAME_MS: i64 = 42; + + /// The pace from what the player reports: `rate_permille` is the applied speed in thousandths + /// (1000 = normal), `fps` the source video's frame rate (0 = unknown or a transcode) and + /// `tick_ms` the wall-clock time since the previous tick. Out-of-range values are clamped: a + /// pace can only be slower or faster than real time by a sane factor, and a tick is not zero. + pub fn of(running: bool, rate_permille: i32, fps: f64, tick_ms: i64) -> Pace { + let frame_ms = if fps >= 1.0 && fps.is_finite() { + ((1000.0 / fps).round() as i64).clamp(4, 100) + } else { + Self::FILM_FRAME_MS + }; + Pace { + running, + rate: (f64::from(rate_permille) / 1000.0).clamp(0.25, 4.0), + frame_ms, + tick_ms: tick_ms.clamp(4, 100), + } + } +} + +/// The most renderers (a worker thread and its own libass instance) that may run at once: the +/// primary, started with the first request, and helpers started while a moving cue's pipeline can +/// use them (issue #1328). A renderer is its own copy of the fonts and the events, so each costs +/// memory as well as a core. +const MAX_ASS_RENDERERS: usize = 3; +/// A helper keeps its renderer only while the pipeline keeps it busy: one that found nothing to do +/// for this long gives the renderer back (and goes out of the mailbox's pacing) and opens it afresh +/// when it is needed again. +const HELPER_IDLE: Duration = Duration::from_secs(10); +/// The niceness a worker runs at (`plx_base::task::lower_priority`): the first below the frame +/// thread and the playback threads, a helper below it, so a moving cue's extra renderers take the +/// cores the frame loop does not need and a late frame costs a subtitle update, not a video frame. +const PRIMARY_NICE: i32 = 5; +const HELPER_NICE: i32 = 10; + pub struct Runtime { source_id: AtomicU64, epoch: AtomicU64, + /// Whether the primary worker (slot 0) started: decided once, by the first request. started: OnceLock, - worker: OnceLock, + /// Whether the helper of slot `i + 1` started: decided once, when the pipeline first wants it. + helpers: [OnceLock; MAX_ASS_RENDERERS - 1], + /// The running workers' threads by slot, to unpark. + threads: [OnceLock; MAX_ASS_RENDERERS], + /// Helpers whose renderer is open and has rendered a frame: the mailbox paces a moving cue over + /// one more worker for each. A cold helper's first jobs open the source and land late, so it is + /// not counted until it has rendered a steady frame. + warm_helpers: AtomicUsize, + /// The bytes of fonts one renderer holds that a helper would copy again (the bundled faces; the + /// attached ones the track names are the store's, referenced by every renderer), as the primary + /// reports them. + font_bytes: AtomicUsize, + /// How many renderers (the primary included) the current selection may hold: 1 from the + /// moment a selection begins, until the pool has decided it (`renderers_for`). + renderers: AtomicUsize, + /// The selection `renderers` was decided for. + decided_epoch: AtomicU64, mailbox: Mutex, } @@ -216,19 +314,108 @@ impl Runtime { source_id: AtomicU64::new(0), epoch: AtomicU64::new(1), started: OnceLock::new(), - worker: OnceLock::new(), + helpers: [const { OnceLock::new() }; MAX_ASS_RENDERERS - 1], + threads: [const { OnceLock::new() }; MAX_ASS_RENDERERS], + warm_helpers: AtomicUsize::new(0), + font_bytes: AtomicUsize::new(0), + renderers: AtomicUsize::new(1), + decided_epoch: AtomicU64::new(0), mailbox: Mutex::new(Mailbox::new()), } } + /// MAIN THREAD: a different source is selected: a new epoch, and the helpers withdrawn until the + /// pool has decided how many renderers this one may hold. + fn begin_selection(&self, source_id: u64) { + self.epoch.fetch_add(1, Ordering::AcqRel); + self.source_id.store(source_id, Ordering::Release); + self.renderers.store(1, Ordering::Release); + } + + /// MAIN THREAD: how many renderers the selection of `epoch` may hold. Decided once per selection, + /// the first time a pipeline wants a helper, from the fonts a renderer holds and the memory + /// available (`mem_available` is the read, supplied by the caller: the verdict is + /// `pool::decide`, which touches nothing), and logged. A verdict above 1 wakes the helpers + /// that were parked while it was awaited. + fn renderers_for(&self, epoch: u64, mem_available: impl FnOnce() -> Option) -> usize { + if self.decided_epoch.load(Ordering::Acquire) == epoch { + return self.renderers.load(Ordering::Acquire); + } + let font_bytes = self.font_bytes.load(Ordering::Relaxed) as u64; + let (mem, slots) = (mem_available(), renderer_slots()); + let verdict = pool::decide(font_bytes, mem, slots); + let number = |n: Option| n.map_or_else(|| String::from("na"), |n| n.to_string()); + plx_base::eventlog::log(&format!( + "ass: pool renderers={} of {slots} font_bytes={font_bytes} per_renderer={} mem_available={} budget={}", + verdict.renderers, + verdict.per_renderer, + number(mem), + number(verdict.budget), + )); + self.renderers.store(verdict.renderers, Ordering::Release); + self.decided_epoch.store(epoch, Ordering::Release); + if verdict.renderers > 1 { + self.wake(); + } + verdict.renderers + } + + /// How many renderers the selection of `epoch` may hold, once the pool has said (`None` before): + /// what the mailbox decides the render scale against. Atomics only, so a request reads it every tick. + fn renderers_decided(&self, epoch: u64) -> Option { + (self.decided_epoch.load(Ordering::Acquire) == epoch).then(|| self.renderers.load(Ordering::Acquire)) + } + + /// A helper of `slot` may not take jobs: the selection's verdict does not reach it. + fn helper_barred(&self, slot: usize) -> bool { + slot > 0 && slot >= self.renderers.load(Ordering::Acquire) + } + fn wake(&self) { // Unlike a condition-variable notification, this permit survives the // gap between checking the mailbox and parking. Before registration, - // the worker has not checked the startup mailbox yet. - if let Some(worker) = self.worker.get() { - worker.unpark(); + // the worker has not checked the startup mailbox yet. Every worker is + // woken: each takes what is there for it (a frame of the plan, or the + // cleanup of a source that was cleared) and parks again. + for thread in self.threads.iter().filter_map(OnceLock::get) { + thread.unpark(); } } + + /// Start the helpers that a pipeline asking for `wanted` workers can use on this device and in + /// this much memory. Off the mailbox lock: starting a thread is not work for the lock the + /// workers take their jobs under. The pool is consulted when drawing at full size would want a + /// helper (`pool_wanted`), even if the scale the plan draws at needs none: its verdict is what + /// the scale is decided against. + fn start_helpers(&self, wanted: usize, pool_wanted: usize, epoch: u64) { + if pool_wanted < 2 { + return; + } + let renderers = self.renderers_for(epoch, read_mem_available); + for slot in 1..wanted.min(renderers) { + self.helpers[slot - 1].get_or_init(|| spawn_worker(slot)); + } + } +} + +/// The memory the device has available, read where the pool asks for it: once for a selection. +fn read_mem_available() -> Option { + std::fs::read_to_string("/proc/meminfo").ok().and_then(|text| pool::mem_available_bytes(&text)) +} + +/// How many renderers this device can run at once: the primary and a helper for every other core, +/// at most [`MAX_ASS_RENDERERS`]. Read once; a request never asks the OS. +fn renderer_slots() -> usize { + static SLOTS: OnceLock = OnceLock::new(); + *SLOTS.get_or_init(|| { + let cores = std::thread::available_parallelism().map_or(1, usize::from); + 1 + cores.saturating_sub(1).min(MAX_ASS_RENDERERS - 1) + }) +} + +/// The one place a styled-subtitle worker is started, the primary and every helper alike. +fn spawn_worker(slot: usize) -> bool { + plx_base::task::spawn("ass-render", move || worker(slot)).is_some() } #[inline] @@ -237,7 +424,8 @@ fn runtime() -> &'static Runtime { } /// MAIN THREAD: ask for the frame right for `now_ms` and return only this selection's last -/// completed frame. Contention drops this tick's request rather than waiting. +/// completed frame. Contention drops this tick's request rather than waiting. `pace` is how the +/// playhead is moving ([`Pace`]). pub fn request( source: Arc, now_ms: i64, @@ -245,14 +433,14 @@ pub fn request( height: i32, storage_width: i32, storage_height: i32, + pace: Pace, ) -> Option> { if source.id == 0 { clear(); return None; } if runtime().source_id.load(Ordering::Acquire) != source.id { - runtime().epoch.fetch_add(1, Ordering::AcqRel); - runtime().source_id.store(source.id, Ordering::Release); + runtime().begin_selection(source.id); } let key = Key { epoch: runtime().epoch.load(Ordering::Acquire), @@ -271,21 +459,46 @@ pub fn request( height }, }; - let started = *runtime() - .started - .get_or_init(|| plx_base::task::spawn("ass-render", worker).is_some()); + let started = *runtime().started.get_or_init(|| spawn_worker(0)); let mut mailbox = runtime().mailbox.try_lock().ok()?; if !started { return Some(mailbox.publish_error(key, error_frame(key, Fault::StartFailed))); } - let requested = mailbox.request(key, now_ms, &source); + // what the pool has said about this selection, if it has: the render scale is decided against it + mailbox.set_capacity(runtime().renderers_decided(key.epoch)); + let requested = mailbox.request(key, now_ms, &source, pace); if requested.promoted { - super::ass_perf::note_ahead_hit(); + if requested.piped { + super::ass_perf::note_shown(requested.late); + } else { + super::ass_perf::note_ahead_hit(); + } } if requested.wake { runtime().wake(); } - mailbox.view(&key) + let frame = mailbox.view(&key); + drop(mailbox); + if let Some(change) = requested.scale_change { + log_scale_change(&change); + } + runtime().start_helpers(requested.workers_wanted, requested.pool_wanted, key.epoch); + frame +} + +/// Each move of the render scale is logged once, with the numbers it was decided from (`assperf` +/// carries the scale in force on every line besides). +fn log_scale_change(change: &scale::Change) { + let load = &change.load; + plx_base::eventlog::log(&format!( + "ass: render scale {} -> {} cost={:.1}ms renderers={} rate={:.2} target={}ms", + change.from.label(), + change.to.label(), + load.cost_full_ms, + load.renderers, + load.rate, + load.target_ms, + )); } /// MAIN THREAD: the alignment classes (bottom, middle, top) of the cues active in the frame @@ -323,7 +536,7 @@ fn retire_off( if runtime.source_id.load(Ordering::Acquire) != 0 || (engine.is_empty() && mailbox.is_empty()) { return None; } - Some((std::mem::take(mailbox), std::mem::take(engine))) + Some((mailbox.retire(), std::mem::take(engine))) } fn next_serial() -> u64 { @@ -338,22 +551,54 @@ fn error_frame(key: Key, message: Fault) -> Arc { width: key.width, height: key.height, rects: Vec::new(), + scale: RenderScale::Full, error: Some(message), }) } -fn worker() { +/// A helper's standing in the mailbox's pacing: counted from its first steady render until it +/// gives its renderer back, so the plan's frame step is over the workers that deliver. +#[derive(Default)] +struct Pacing { + counted: bool, +} + +impl Pacing { + /// Under the mailbox lock, so a count and the worker number it sets are never read apart. + fn set(&mut self, runtime: &Runtime, mailbox: &mut Mailbox, counted: bool) { + if self.counted == counted { + return; + } + self.counted = counted; + let helpers = if counted { + runtime.warm_helpers.fetch_add(1, Ordering::Relaxed) + 1 + } else { + runtime.warm_helpers.fetch_sub(1, Ordering::Relaxed) - 1 + }; + mailbox.set_workers(1 + helpers); + } +} + +/// One styled-subtitle worker: slot 0 is the primary, the others are helpers. Each owns its own +/// libass instance (`Engine`), so nothing native is shared between workers; they share only the +/// mailbox, under its lock. +fn worker(slot: usize) { runtime() - .worker + .threads[slot] .set(std::thread::current()) - .expect("one styled subtitle worker"); - super::ass_perf::log_worker_tid(); + .expect("one styled subtitle worker per slot"); + let nice = if slot == 0 { PRIMARY_NICE } else { HELPER_NICE }; + let niced = plx_base::task::lower_priority(nice); + super::ass_perf::log_worker_tid(slot, niced.then_some(nice)); let mut engine = Engine::default(); + let mut pacing = Pacing::default(); + let mut idle_since = std::time::Instant::now(); loop { let job = { let mut mailbox = runtime().mailbox.lock().unwrap_or_else(|e| e.into_inner()); loop { if let Some(retired) = retire_off(runtime(), &mut mailbox, &mut engine) { + pacing.set(runtime(), &mut mailbox, false); // Native destruction can be slow too: release the mailbox first. drop(mailbox); super::ass_perf::flush(); @@ -361,14 +606,36 @@ fn worker() { mailbox = runtime().mailbox.lock().unwrap_or_else(|e| e.into_inner()); continue; } - if let Some(job) = mailbox.take() { + // A helper the selection's memory verdict does not reach takes nothing. + let barred = runtime().helper_barred(slot); + if barred { + pacing.set(runtime(), &mut mailbox, false); + } else if let Some(job) = mailbox.take() { break job; } + let helper_holds_renderer = slot > 0 && !engine.is_empty(); + if helper_holds_renderer && (barred || idle_since.elapsed() >= HELPER_IDLE) { + // Nothing for a helper to do for a while, or no memory for it: give the renderer + // back, and stay parked until a plan wants it again. Decided under the lock, so a + // job handed out meanwhile is not missed. + let retired = std::mem::take(&mut engine); + pacing.set(runtime(), &mut mailbox, false); + drop(mailbox); + super::ass_perf::flush(); + drop(retired); + mailbox = runtime().mailbox.lock().unwrap_or_else(|e| e.into_inner()); + continue; + } drop(mailbox); // A queue/cancel between the check above and this park leaves a // permit, so it cannot strand native resources until another key. - // A truly idle worker needs no periodic wakeup. - std::thread::park(); + // A truly idle worker needs no periodic wakeup; a helper that holds a + // renderer wakes once to give it back. + if helper_holds_renderer { + std::thread::park_timeout(HELPER_IDLE.saturating_sub(idle_since.elapsed())); + } else { + std::thread::park(); + } mailbox = runtime().mailbox.lock().unwrap_or_else(|e| e.into_inner()); } }; @@ -379,20 +646,33 @@ fn worker() { let live = || runtime().epoch.load(Ordering::Acquire) == job.key.epoch; let rendered = live().then(|| { let job_started = std::time::Instant::now(); - let rendered = engine.render(&job); - super::ass_perf::record_job(job_started.elapsed(), job.kind != Kind::Normal); + let mut rendered = engine.render(&job); + let elapsed = job_started.elapsed(); + super::ass_perf::record_job(elapsed, job.kind.is_ahead(), job.scale); + // What a steady frame costs paces the animated pipeline; a job that re-read the fonts or + // the events is no sample of it. + rendered.cost_us = + (!engine.resynced).then(|| u32::try_from(elapsed.as_micros()).unwrap_or(u32::MAX)); rendered }); + if slot == 0 { + // What a helper would copy: the pool decides how many may run from it. + runtime().font_bytes.store(engine.font_bytes(), Ordering::Relaxed); + } let finished = { let mut mailbox = runtime().mailbox.lock().unwrap_or_else(|e| e.into_inner()); // Every exit finishes the job, so the mailbox never believes one is still in flight. let outcome = rendered.filter(|_| live()); let ran = outcome.is_some(); + if slot > 0 && ran && !engine.resynced { + pacing.set(runtime(), &mut mailbox, true); + } let finished = mailbox.finish(&job, outcome); ran.then_some(finished) }; + idle_since = std::time::Instant::now(); let Some(finished) = finished else { continue }; - super::ass_perf::finish_call(finished.changed, finished.pub_bytes); + super::ass_perf::finish_call(finished.changed, finished.pub_bytes, finished.stale); if finished.wake { plx_machine::present::wake_from_worker(); } @@ -444,6 +724,20 @@ pub(super) fn validate(source: &Source) -> Result<(), Fault> { Ok(()) } +/// The earliest playhead whose picture the move from `old` to `new` can change: the first start +/// among the events `new` adds, `i64::MAX` when it adds none (expiry alone changes no picture +/// ahead of the playhead), and `i64::MIN` for a move the renderer cannot take as an append, which +/// can change any picture. An appended event is drawn from its own start on, so every frame for an +/// earlier playhead stays right. +fn earliest_change_ms(old: &Source, new: &Source) -> i64 { + match (append_from(old, new), &new.content) { + (Some(first), Content::Embedded { events, .. }) => { + events.get(first..).and_then(|added| added.iter().map(|e| e.start_ms).min()).unwrap_or(i64::MAX) + } + _ => i64::MIN, + } +} + /// Pruning expired events and appending preserves libass's parsed events, font /// caches and ReadOrder set, including a long event spanning many short ones. fn append_from(old: &Source, new: &Source) -> Option { @@ -536,9 +830,20 @@ struct Engine { aligns_index: Option, frame: Option>, error: Option, + /// The last [`Engine::render`] opened, restyled or re-fed the renderer: that job re-read the + /// fonts or the events, and what it cost is no sample of what a frame costs. + resynced: bool, + /// The frame size and the storage size the renderer last drew at: a render at another pair is + /// cold (libass drops its caches on a resize), and is no sample either. + rendered_size: Option<((i32, i32), (i32, i32))>, } impl Engine { + /// The bytes of fonts the renderer holds, 0 with none open. + fn font_bytes(&self) -> usize { + self.native.as_ref().map_or(0, Native::font_bytes) + } + fn is_empty(&self) -> bool { self.source.is_none() && self.native.is_none() @@ -556,7 +861,7 @@ impl Engine { if let Some(index) = super::ass_script::ScriptIndex::parse(bytes) { if index.windowed() { let window = index.window(now_ms); - windowed = Some(Native::open_data(fonts, &window.text, 1)?); + windowed = Some(Native::open_named(fonts, bytes, &window.text, 1)?); self.script = Some(Scripted { index, window: Some((window.lo_ms, window.hi_ms)) }); } else { self.script = Some(Scripted { index, window: None }); @@ -600,6 +905,8 @@ impl Engine { let Some(native) = self.native.as_mut() else { return self.open(source, now_ms); }; + // The new header may name a face the held renderer was not handed; the ones it holds stay. + native.seed(source_fonts(source), track_text(source)); match &source.content { Content::Embedded { header, .. } => { native.replace_track(header, 0)?; @@ -632,7 +939,7 @@ impl Engine { /// way, or the end of the window). The old track is released first, so two windows are never /// held at once. fn slide(&mut self, source: &Source, now_ms: i64) -> Result<(), Fault> { - let (Some(Scripted { index, window: Some(held) }), Content::Script { fonts, .. }) = + let (Some(Scripted { index, window: Some(held) }), Content::Script { bytes, fonts }) = (self.script.as_mut(), &source.content) else { return Ok(()); @@ -642,12 +949,13 @@ impl Engine { } let window = index.window(now_ms); *held = (window.lo_ms, window.hi_ms); + self.resynced = true; // Every window repeats the script's header, `[Fonts]` included, so the held renderer // already has the fonts: swap the track in place rather than reopening every face. match self.native.as_mut() { Some(native) => native.replace_track(&window.text, 1), None => { - self.native = Some(Native::open_data(fonts, &window.text, 1)?); + self.native = Some(Native::open_named(fonts, bytes, &window.text, 1)?); Ok(()) } } @@ -662,7 +970,17 @@ impl Engine { self.validity(&job.source, job.now_ms) }; let aligns = self.aligns(&job.source, job.now_ms); - Rendered { frame, validity, aligns } + let moves_on = self.moves_on(&job.source, &validity); + Rendered { frame, validity, aligns, cost_us: None, moves_on } + } + + /// Whether the picture at the end of `validity`'s window moves as well: the boundary is inside a + /// moving stretch (a karaoke syllable ends while others go on) and not its end. The pipeline + /// resolves the former at its own step instead of ending the frame there (`mailbox`). + fn moves_on(&self, source: &Source, validity: &Validity) -> bool { + validity.animated + && validity.until_ms != i64::MAX + && self.validity(source, validity.until_ms).animated } /// The window at `now_ms` from every event's timing, clamped to the script window held. @@ -692,6 +1010,7 @@ impl Engine { fn render_frame(&mut self, request: &Job) -> Arc { let key = request.key; + self.resynced = false; let source_changed = self.source.as_ref().is_none_or(|old| { old.id != request.source.id || old.revision != request.source.revision }); @@ -706,7 +1025,7 @@ impl Engine { let Content::Embedded { events, .. } = &request.source.content else { unreachable!() }; - unsafe { plx_ass_prune_before(native.0, prune_deadline(events)) }; + unsafe { plx_ass_prune_before(native.ptr, prune_deadline(events)) }; native.append(&request.source, from) } else if self .source @@ -718,11 +1037,14 @@ impl Engine { // as expiry + append, and a seek or a track switch brings a different // timeline. Reset events only, retaining parsed header, fonts and caches. let native = self.native.as_mut().unwrap(); - unsafe { plx_ass_flush_events(native.0) }; + unsafe { plx_ass_flush_events(native.ptr) }; + self.resynced = true; native.append(&request.source, 0) } else if self.source.as_ref().is_some_and(|old| restyles(old, &request.source)) { + self.resynced = true; self.restyle(&request.source, request.now_ms) } else { + self.resynced = true; self.open(&request.source, request.now_ms) }; self.error = result.err(); @@ -754,28 +1076,47 @@ impl Engine { self.frame = Some(frame.clone()); return frame; } - let previous = self - .frame - .as_ref() - .filter(|frame| { - frame.source_id == key.source_id - && frame.width == key.width - && frame.height == key.height - }) - .map_or(&[][..], |frame| frame.rects.as_slice()); + // The frame this one follows (another worker's, when several take turns) is looked at before + // this engine's own last one: a region that draws identically to one of its regions shares + // its allocation, so the screen keeps the texture it already uploaded for it (issue #1328). + // ... and only at the size this job draws at: regions of another size share nothing with it. + let for_key = |frame: &&Arc| { + frame.source_id == key.source_id + && frame.width == key.width + && frame.height == key.height + && frame.scale == request.scale + }; + let size = (request.scale.render_size(key.width, key.height), (key.storage_width, key.storage_height)); + if self.rendered_size.replace(size).is_some_and(|last| last != size) { + // libass empties its outline, bitmap and composite caches when the frame size changes: this + // render is cold, and what it costs is no sample of what a frame costs + self.resynced = true; + } + let reference = request.reference.as_ref().filter(for_key); + let previous = [ + reference.map_or(&[][..], |frame| frame.rects.as_slice()), + self.frame.as_ref().filter(for_key).map_or(&[][..], |frame| frame.rects.as_slice()), + ]; let result = self .native .as_mut() .expect("loaded ASS source") - .render(key, request.now_ms, previous); + .render(key, request.scale, request.now_ms, &previous); let frame = match result { - Ok(None) => match &self.frame { + Ok(None) => match self.frame.clone() { Some(frame) if frame.error.is_none() => { if frame.source_id == key.source_id && frame.width == key.width && frame.height == key.height { - return frame.clone(); + // libass found nothing changed since THIS engine's last frame; the picture + // may still be the reference's, which the screen shows. + let shared = match reference { + Some(reference) => interned(&frame, reference), + None => frame, + }; + self.frame = Some(shared.clone()); + return shared; } // libass reports "unchanged" against the track the renderer showed before a // reuse (a seek, a track switch with the same header): the pixels stand, but @@ -786,22 +1127,24 @@ impl Engine { width: key.width, height: key.height, rects: frame.rects.clone(), + scale: frame.scale, error: None, }) } - Some(frame) => return frame.clone(), + Some(frame) => return frame, None => error_frame(key, Fault::RenderFailed), }, Ok(Some(rects)) => { - if let Some(old) = &self.frame { - if old.source_id == key.source_id - && old.width == key.width - && old.height == key.height - && old.error.is_none() - && old.rects == rects - { - return old.clone(); - } + // A picture that is one already known, the reference's or this engine's own last, is + // that very frame: the screen keeps its serial and its textures. + let identical = reference + .into_iter() + .chain(self.frame.as_ref().filter(for_key)) + .find(|old| old.error.is_none() && old.rects == rects) + .cloned(); + if let Some(old) = identical { + self.frame = Some(old.clone()); + return old; } Arc::new(Frame { source_id: key.source_id, @@ -809,6 +1152,7 @@ impl Engine { width: key.width, height: key.height, rects, + scale: request.scale, error: None, }) } @@ -826,6 +1170,59 @@ impl Engine { } } +/// The region among `candidates` that holds exactly these pixels: the one at the same place if there +/// is one, else any (a translation). Its allocation is what a region drawing the same pixels shares. +fn shared_region<'a>( + mut candidates: impl Iterator + Clone, + x: i32, + y: i32, + width: i32, + height: i32, + pixels: &[u8], +) -> Option<&'a Rect> { + let same = |old: &&Rect| old.width == width && old.height == height && old.rgba.as_ref() == pixels; + candidates.clone().filter(|r| r.x == x && r.y == y).find(same).or_else(|| candidates.find(same)) +} + +/// `frame` with every region that draws what a region of `reference` draws sharing that region's +/// allocation; `reference` itself when the two pictures are the same, and `frame` itself when it +/// shares nothing with it. +fn interned(frame: &Arc, reference: &Arc) -> Arc { + if Arc::ptr_eq(frame, reference) { + return frame.clone(); + } + let mut shares = false; + let rects: Vec = frame + .rects + .iter() + .map(|rect| { + let old = shared_region(reference.rects.iter(), rect.x, rect.y, rect.width, rect.height, &rect.rgba); + match old { + Some(old) if !Arc::ptr_eq(&old.rgba, &rect.rgba) => { + shares = true; + Rect { rgba: old.rgba.clone(), ..rect.clone() } + } + _ => rect.clone(), + } + }) + .collect(); + if rects == reference.rects { + return reference.clone(); + } + if !shares { + return frame.clone(); + } + Arc::new(Frame { + source_id: frame.source_id, + serial: next_serial(), + width: frame.width, + height: frame.height, + rects, + scale: frame.scale, + error: None, + }) +} + #[repr(C)] #[derive(Default)] pub struct Bitmap { @@ -844,19 +1241,28 @@ pub struct NativeFrame { regions: *const Bitmap, } -/// `PlxAssStats` in `include/ass.h`. +/// `PlxAssStats` in `include/ass.h`. The `last_*` fields are the facade's own account of the last +/// `plx_ass_render` call (wall and thread-CPU microseconds per phase, the image counts); eight +/// `u32`s come first, so the `u64` sits on its 8-byte boundary here as in C. #[repr(C)] #[derive(Default)] pub struct NativeStats { library_fonts: u32, font_scans: u32, tracks: u32, + last_libass_us: u32, + last_composite_us: u32, + last_libass_cpu_us: u32, + last_composite_cpu_us: u32, + last_images: u32, + last_image_pixels: u64, } /// The facade ABI this file is written against (`PLX_ASS_ABI_VERSION` in `include/ass.h`); a -/// library reporting any other number is not loaded. ABI 4 added only the UI text surface -/// (`plx_ass_text_*`, bound by `plx_gfx::rtl`); every function this file calls is unchanged. -const NATIVE_ABI: u32 = 4; +/// library reporting any other number is not loaded. ABI 5 lets `plx_ass_add_font` run after +/// `plx_ass_load` and lifts its byte limit (`Native::append` adds a font when an event first names +/// it), so an older library, which would refuse every late font, is refused here instead. +const NATIVE_ABI: u32 = 5; /// libass's glyph-count and bitmap-MiB cache bounds for every renderer. const CACHE_GLYPHS: i32 = 1000; @@ -868,6 +1274,7 @@ plx_base::dynlib! { fn plx_ass_create(default_font: *const c_char) -> *mut c_void; fn plx_ass_destroy(ctx: *mut c_void); fn plx_ass_add_font(ctx: *mut c_void, name: *const c_char, data: *const u8, len: usize) -> i32; + fn plx_ass_add_font_borrowed(ctx: *mut c_void, name: *const c_char, data: *const u8, len: usize) -> i32; fn plx_ass_load(ctx: *mut c_void, data: *const u8, len: usize, script: i32) -> i32; fn plx_ass_chunk(ctx: *mut c_void, data: *const u8, len: usize, start_ms: i64, duration_ms: i64) -> i32; fn plx_ass_prune_before(ctx: *mut c_void, deadline_ms: i64); @@ -881,7 +1288,25 @@ plx_base::dynlib! { } } -struct Native(*mut c_void); +/// One libass renderer. `selection` is which of the file's attached fonts it has been handed: the +/// ones the track names (`font_names`), at open and as later events first name more. `given` and +/// `given_bytes` count what libass actually took, for the log line. +/// +/// The attached fonts are BORROWED: libass references the store's own buffer (`plx_ass_add_font_borrowed`) +/// instead of copying it, so the bytes of a font are resident once however many renderers hold it +/// (issue #1328). `shared` keeps the buffers libass points at alive, and it is declared (so dropped) +/// after the native context, which `Drop` destroys first. +struct Native { + ptr: *mut c_void, + selection: font_names::Selection, + given: usize, + given_bytes: usize, + /// The bytes of the bundled faces it was given at open (`fonts::bundled`): libass copies those. + bundled_bytes: usize, + /// The attached fonts libass was handed by reference. Never read: it holds the buffers alive + /// until after `plx_ass_destroy`. + shared: Vec>, +} #[cfg(test)] thread_local! { @@ -910,7 +1335,7 @@ fn asset_dir() -> &'static std::path::Path { impl Drop for Native { fn drop(&mut self) { - unsafe { plx_ass_destroy(self.0) }; + unsafe { plx_ass_destroy(self.ptr) }; } } @@ -927,8 +1352,15 @@ impl Native { } /// A renderer holding `fonts` and the track `data` reads as: a script (`script` 1) or a codec - /// private header (0). + /// private header (0). The track's own text names the fonts to hand libass. fn open_data(fonts: &[Font], data: &[u8], script: i32) -> Result { + Self::open_named(fonts, data, data, script) + } + + /// [`Native::open_data`] for a track `data` that is only a part of the text which names the + /// fonts: one window of a long sidecar script, whose whole text is `named`, so a face a line + /// outside the first window asks for is held before the window slides to it. + fn open_named(fonts: &[Font], named: &[u8], data: &[u8], script: i32) -> Result { #[cfg(test)] OPEN_DATA_CALLS.with(|c| c.set(c.get() + 1)); static LOADED: OnceLock = OnceLock::new(); @@ -945,7 +1377,14 @@ impl Native { if ptr.is_null() { return Err(Fault::InitFailed); } - let mut native = Self(ptr); + let mut native = Self { + ptr, + selection: font_names::Selection::new(fonts), + given: 0, + given_bytes: 0, + bundled_bytes: 0, + shared: Vec::new(), + }; native.configure()?; // Inter, Inter Bold, then the per-script Noto faces (`fonts.rs`: the order is load-bearing, // and the files are read here, per renderer, never kept). The CJK face is libass's DEFAULT @@ -954,81 +1393,142 @@ impl Native { // "Noto Sans CJK KR", while glyph fallback never goes through it. for path in fonts::bundled(asset_dir())? { let bytes = std::fs::read(&path).map_err(|_| Fault::FontsUnavailable)?; - if bytes.len() > MAX_FONT_BYTES { - return Err(Fault::FontTooLarge); - } let name = path.file_name().and_then(|n| n.to_str()).ok_or(Fault::FontsUnavailable)?; native.font(name, &bytes)?; + native.bundled_bytes += bytes.len(); } - // A font the renderer cannot take (unnamed, over the budget, rejected by the native - // side) is skipped: the text still shows, in the default face. - let mut budget = FontBudget::default(); - for font in fonts.iter() { - if font.name.is_empty() || font.name.len() > 1024 || font.name.contains('\0') - || !budget.admit(font.data.len()) - { + // The attached fonts the track names, in container order after the bundled ones. A font the + // renderer cannot take (unnamed, rejected by the native side) is skipped: the text still + // shows, in the default face. + let wanted = native.selection.seed(named); + native.admit(fonts, &wanted); + if unsafe { plx_ass_load(native.ptr, data.as_ptr(), data.len(), script) } < 0 { + return Err(Fault::Unreadable); + } + native.log_fonts(fonts.len()); + Ok(native) + } + + /// Hand libass the attached fonts at `wanted` (indices of `fonts`, container order). Works + /// before and after the track is loaded: libass opens a face added later at its next frame. + fn admit(&mut self, fonts: &[Font], wanted: &[usize]) { + for font in wanted.iter().filter_map(|&i| fonts.get(i)) { + if font.name.is_empty() || font.name.len() > 1024 || font.name.contains('\0') { continue; } - let _ = native.font(&font.name, &font.data); + if self.font_shared(&font.name, &font.data).is_ok() { + self.given += 1; + self.given_bytes += font.data.len(); + } } - if unsafe { plx_ass_load(native.0, data.as_ptr(), data.len(), script) } < 0 { - return Err(Fault::Unreadable); + } + + /// Hand libass what a header or script (`text`) names that it does not hold yet: a text track + /// restyled in place, whose new header may name another face. + fn seed(&mut self, fonts: &[Font], text: &[u8]) { + let wanted = self.selection.seed(text); + self.admit(fonts, &wanted); + if !wanted.is_empty() { + self.log_fonts(fonts.len()); } - Ok(native) + } + + /// The bytes of fonts this renderer holds that another renderer would copy again: the bundled + /// faces, which libass copies per renderer. The attached fonts it references are not here: they + /// are the store's buffers, one for every renderer. + fn font_bytes(&self) -> usize { + self.bundled_bytes + } + + /// How many of the file's `attached` fonts libass holds, numbers only: the event log is + /// sanitized and a font name is the file's own text. A file with no attachments logs nothing. + fn log_fonts(&self, attached: usize) { + if attached == 0 { + return; + } + plx_base::eventlog::log(&format!( + "ass: renderer fonts={}/{attached} font_bytes={}", + self.given, self.given_bytes + )); } /// The renderer settings every renderer starts with. Authored scripts break lines the way /// they always have (at spaces and `\N`), with no extra line spacing. fn configure(&mut self) -> Result<(), Fault> { let applied = unsafe { - plx_ass_set_cache_limits(self.0, CACHE_GLYPHS, CACHE_BITMAP_MIB) >= 0 - && plx_ass_set_features(self.0, 0) >= 0 - && plx_ass_set_line_spacing(self.0, 0.0) >= 0 + plx_ass_set_cache_limits(self.ptr, CACHE_GLYPHS, CACHE_BITMAP_MIB) >= 0 + && plx_ass_set_features(self.ptr, 0) >= 0 + && plx_ass_set_line_spacing(self.ptr, 0.0) >= 0 }; applied.then_some(()).ok_or(Fault::InitFailed) } /// Break lines by the Unicode rules (libunibreak) in the loaded track and every later one. fn set_wrap_unicode(&mut self, on: bool) -> bool { - unsafe { plx_ass_set_features(self.0, i32::from(on)) >= 0 } + unsafe { plx_ass_set_features(self.ptr, i32::from(on)) >= 0 } } /// Extra pixels between lines at the render size; negative tightens. fn set_line_spacing(&mut self, spacing: f64) -> bool { - unsafe { plx_ass_set_line_spacing(self.0, spacing) >= 0 } + unsafe { plx_ass_set_line_spacing(self.ptr, spacing) >= 0 } } /// Swap the loaded track for `data` on the same library and renderer, keeping every opened /// face. Only for a track whose `[Fonts]` section is the first one's (a window of the same /// script, or a header without one): the new track's fonts are not extracted. fn replace_track(&mut self, data: &[u8], script: i32) -> Result<(), Fault> { - if unsafe { plx_ass_replace_track(self.0, data.as_ptr(), data.len(), script) } < 0 { + if unsafe { plx_ass_replace_track(self.ptr, data.as_ptr(), data.len(), script) } < 0 { return Err(Fault::Unreadable); } Ok(()) } + fn try_stats(&self) -> Option { + let mut stats = NativeStats::default(); + (unsafe { plx_ass_stats(self.ptr, &mut stats) } >= 0).then_some(stats) + } + #[cfg(test)] fn stats(&self) -> NativeStats { - let mut stats = NativeStats::default(); - assert!(unsafe { plx_ass_stats(self.0, &mut stats) } >= 0); - stats + self.try_stats().expect("plx_ass_stats refused a live context") } + /// Hand libass a font it copies: for bytes this renderer owns, the bundled faces it reads from + /// the asset directory per renderer. fn font(&mut self, name: &str, bytes: &[u8]) -> Result<(), Fault> { let name = CString::new(name).map_err(|_| Fault::FontName)?; - if unsafe { plx_ass_add_font(self.0, name.as_ptr(), bytes.as_ptr(), bytes.len()) } < 0 { + if unsafe { plx_ass_add_font(self.ptr, name.as_ptr(), bytes.as_ptr(), bytes.len()) } < 0 { return Err(Fault::TrackFonts); } Ok(()) } + /// Hand libass a font it references: `bytes` stay alive and unchanged in `shared` until after + /// the native context is destroyed, which is what the facade requires of a borrowed font. + fn font_shared(&mut self, name: &str, bytes: &Arc<[u8]>) -> Result<(), Fault> { + let name = CString::new(name).map_err(|_| Fault::FontName)?; + if unsafe { plx_ass_add_font_borrowed(self.ptr, name.as_ptr(), bytes.as_ptr(), bytes.len()) } < 0 { + return Err(Fault::TrackFonts); + } + self.shared.push(Arc::clone(bytes)); + Ok(()) + } + + /// Append `events[from..]`. An event that overrides the font (`\fn`) with a face the renderer + /// does not hold yet gets it first, so libass never meets the name before the face and caches + /// the fallback for it. fn append(&mut self, source: &Source, from: usize) -> Result<(), Fault> { if let Content::Embedded { events, .. } = &source.content { + let fonts = source_fonts(source); for event in &events[from..] { + let wanted = self.selection.wanted(&event.payload); + if !wanted.is_empty() { + self.admit(fonts, &wanted); + self.log_fonts(fonts.len()); + } if unsafe { plx_ass_chunk( - self.0, + self.ptr, event.payload.as_ptr(), event.payload.len(), event.start_ms, @@ -1043,15 +1543,26 @@ impl Native { Ok(()) } - fn render(&mut self, key: Key, now_ms: i64, previous: &[Rect]) -> Result>, Fault> { + /// `previous` are the regions of the frames to share allocations with, in the order they are + /// looked at ([`shared_region`]). + fn render( + &mut self, + key: Key, + scale: RenderScale, + now_ms: i64, + previous: &[&[Rect]], + ) -> Result>, Fault> { + // libass draws at the frame size and lays the script out against the storage size, so a smaller + // frame is the same picture with fewer pixels (issue #1328); the regions come back in its canvas. + let (width, height) = scale.render_size(key.width, key.height); let mut frame = NativeFrame::default(); let started = std::time::Instant::now(); let result = unsafe { plx_ass_render( - self.0, + self.ptr, now_ms, - key.width, - key.height, + width, + height, key.storage_width, key.storage_height, &mut frame, @@ -1061,6 +1572,17 @@ impl Native { if result < 0 { return Err(Fault::RenderFailed); } + if let Some(stats) = self.try_stats() { + super::ass_perf::record_native(super::ass_perf::NativeCost { + lib_us: stats.last_libass_us, + comp_us: stats.last_composite_us, + lib_cpu_us: stats.last_libass_cpu_us, + comp_cpu_us: stats.last_composite_cpu_us, + images: stats.last_images, + image_pixels: stats.last_image_pixels, + composed: result == 1, + }); + } if result == 0 { return Ok(None); } @@ -1082,11 +1604,11 @@ impl Native { || bitmap .x .checked_add(bitmap.width) - .is_none_or(|x| x > key.width) + .is_none_or(|x| x > width) || bitmap .y .checked_add(bitmap.height) - .is_none_or(|y| y > key.height) + .is_none_or(|y| y > height) || pixels.is_none_or(|n| n > MAX_PIXELS || n * 4 != bitmap.bytes) || bitmap.rgba.is_null() { @@ -1104,18 +1626,16 @@ impl Native { return Err(Fault::OverlappingRegions); } let pixels = unsafe { std::slice::from_raw_parts(bitmap.rgba, bitmap.bytes) }; - let same_pixels = |old: &&Rect| { - old.width == bitmap.width - && old.height == bitmap.height - && old.rgba.as_ref() == pixels - }; // Compare before copying. Static signs retain their immutable pixel allocation // while another region animates; translations can reuse it at a new position too. - let old = previous - .iter() - .filter(|r| r.x == bitmap.x && r.y == bitmap.y) - .find(same_pixels) - .or_else(|| previous.iter().find(same_pixels)); + let old = shared_region( + previous.iter().flat_map(|rects| rects.iter()), + bitmap.x, + bitmap.y, + bitmap.width, + bitmap.height, + pixels, + ); let rgba = old.map_or_else(|| Arc::from(pixels), |r| r.rgba.clone()); rects.push(Rect { x: bitmap.x, @@ -1131,7 +1651,11 @@ impl Native { mod fonts; +#[cfg(test)] +pub(crate) mod synth_fonts; #[cfg(test)] mod tests; #[cfg(test)] mod mailbox_tests; +#[cfg(test)] +mod scale_tests; diff --git a/rust-modules/media/src/player/ass/font_names.rs b/rust-modules/media/src/player/ass/font_names.rs new file mode 100644 index 0000000000..2feef905f3 --- /dev/null +++ b/rust-modules/media/src/player/ass/font_names.rs @@ -0,0 +1,388 @@ +//! Which attached fonts a track asks for (issue #1328). Pure: bytes in, indices out. +//! +//! A file may attach far more font bytes than any renderer needs: an anime release carries a face +//! for every sign and karaoke style, 17 of them and 83 MB in the report. libass here runs with no +//! font provider (`src/ass.c`), so `ass_font_select` tries the requested family, then "Inter", then +//! the CJK file, and **an attached face that no requested name matches is never selected**: the bytes +//! it costs libass draw nothing. So the renderer hands libass the faces the track NAMES and nothing +//! else, and no byte count decides what is shown. A track names a face +//! +//! * in its header: the Fontname of every `Style:` line (a sidecar script: the whole script); +//! * or in an event: a `\fn` override, which [`Selection::wanted`] reads when the event is appended. +//! +//! **The rule is a superset of libass's matching**, so it can never hide text: a face is wanted when +//! a requested name equals (ASCII case folded, which is all `ass_strcasecmp` folds) its name table's +//! family (ID 1), full name (ID 4) or PostScript name (ID 6), on any platform and not only the +//! Microsoft one libass reads, the typographic and WWS family (ID 16, 21: libass counts only a +//! Microsoft-platform ID 1, and a face with none gets FreeType's `family_name`, which prefers these +//! two), or the CFF font name. "Inter" and "Arial" are always requested (`family_default` and libass's default Fontname). Wanting a face wants every +//! attachment that shares a family with it, added together in container order, so libass's weight +//! scoring and its add-order tie break between the members of a family are what they would be with +//! every font loaded. A face this module cannot read is WANTED: unknown means needed, because FreeType +//! may still name it. A byte-identical second copy is never added: libass would pick the first anyway. +//! Over-matching costs memory only; under-matching is what the tests pin. + +use super::Font; +use std::collections::HashMap; + +/// libass cuts a name at its own 1024-byte buffer (`get_font_info`); so does this. +const NAME_RECORD_BYTES: usize = 1024; +/// The names every renderer requests whatever the track says: libass's `family_default` and the +/// Fontname a style gets when it sets none. +const ALWAYS: [&[u8]; 2] = [b"inter", b"arial"]; + +const NAME_FAMILY: u16 = 1; +const NAME_FULL: u16 = 4; +const NAME_POSTSCRIPT: u16 = 6; +const NAME_TYPOGRAPHIC_FAMILY: u16 = 16; +const NAME_WWS_FAMILY: u16 = 21; + +/// What one face answers to, normalized by [`fold`]. +#[derive(Debug, Default, PartialEq, Eq)] +pub(super) struct Face { + /// Family (ID 1, 16, 21), full and PostScript names. + pub names: Vec>, + /// The family names alone: what the family closure joins faces by. + pub families: Vec>, +} + +/// A name as it is compared: ASCII-lowercased, whitespace trimmed, one leading `@` (libass's +/// vertical-text marker) dropped. `None` for what can never name a face: nothing at all, or "0", +/// which `\fn0` reads as "the style's own font". +pub(super) fn fold(raw: &[u8]) -> Option> { + let raw = trim(raw); + let raw = trim(raw.strip_prefix(b"@").unwrap_or(raw)); + if raw.is_empty() || raw == b"0" { + return None; + } + let raw = &raw[..raw.len().min(NAME_RECORD_BYTES)]; + Some(raw.to_ascii_lowercase()) +} + +fn trim(mut bytes: &[u8]) -> &[u8] { + while let [first, rest @ ..] = bytes { + if !matches!(first, b' ' | b'\t' | b'\r' | b'\n') { + break; + } + bytes = rest; + } + while let [rest @ .., last] = bytes { + if !matches!(last, b' ' | b'\t' | b'\r' | b'\n') { + break; + } + bytes = rest; + } + bytes +} + +fn be16(bytes: &[u8], at: usize) -> Option { + let end = at.checked_add(2)?; + Some(u16::from_be_bytes(bytes.get(at..end)?.try_into().ok()?) as usize) +} + +fn be32(bytes: &[u8], at: usize) -> Option { + let end = at.checked_add(4)?; + Some(u32::from_be_bytes(bytes.get(at..end)?.try_into().ok()?) as usize) +} + +/// The bytes of table `tag` of the sfnt whose offset table starts at `start`. +fn table<'a>(bytes: &'a [u8], start: usize, tag: &[u8; 4]) -> Option<&'a [u8]> { + let count = be16(bytes, start.checked_add(4)?)?; + let directory = start.checked_add(12)?; + for i in 0..count { + let record = directory.checked_add(i.checked_mul(16)?)?; + if bytes.get(record..record.checked_add(4)?)? == tag { + let offset = be32(bytes, record + 8)?; + let length = be32(bytes, record + 12)?; + return bytes.get(offset..offset.checked_add(length)?); + } + } + None +} + +/// A name record's string as UTF-8: UTF-16BE for the Unicode and Windows platforms, Latin-1 for +/// Mac Roman (its bytes above 0x7F are not Latin-1, but no ASCII fold maps them to anything a style +/// would spell, and libass reads no Mac record at all). +fn decode(platform: usize, raw: &[u8]) -> Option { + match platform { + 0 | 3 => { + let units = raw.chunks_exact(2).map(|u| u16::from_be_bytes([u[0], u[1]])); + Some(char::decode_utf16(units).map(|c| c.unwrap_or('\u{fffd}')).collect()) + } + 1 => Some(raw.iter().map(|&b| char::from(b)).collect()), + _ => None, + } +} + +/// The names of the face whose offset table starts at `start`, or `None` when it names nothing +/// this module can read: no `name` table, one that is truncated, or one with no usable record. +fn face(bytes: &[u8], start: usize) -> Option { + let names = table(bytes, start, b"name")?; + let count = be16(names, 2)?; + let strings = be16(names, 4)?; + let mut found: Vec<(u16, Vec)> = Vec::new(); + for i in 0..count { + let record = 6usize.checked_add(i.checked_mul(12)?)?; + let (platform, id) = (be16(names, record)?, be16(names, record + 6)? as u16); + if !matches!(id, NAME_FAMILY | NAME_FULL | NAME_POSTSCRIPT | NAME_TYPOGRAPHIC_FAMILY | NAME_WWS_FAMILY) { + continue; + } + let (length, offset) = (be16(names, record + 8)?, be16(names, record + 10)?); + let from = strings.checked_add(offset)?; + let Some(raw) = names.get(from..from.checked_add(length)?) else { continue }; + let Some(text) = decode(platform, raw) else { continue }; + if let Some(name) = fold(text.as_bytes()) { + found.push((id, name)); + } + } + // libass counts only a Microsoft-platform ID 1 as a family and otherwise takes FreeType's + // `family_name`, which prefers ID 16 and 21; which of the two a face gets is decided by records + // this reader does not keep apart, so ID 16 and 21 are families always (a superset). + let mut out = Face::default(); + for (id, name) in found { + let family = matches!(id, NAME_FAMILY | NAME_TYPOGRAPHIC_FAMILY | NAME_WWS_FAMILY); + if family && !out.families.contains(&name) { + out.families.push(name.clone()); + } + if (family || matches!(id, NAME_FULL | NAME_POSTSCRIPT)) && !out.names.contains(&name) { + out.names.push(name); + } + } + // An OpenType/CFF face's PostScript name, as FreeType reports it, is the CFF font name. + if let Some(name) = cff_name(bytes, start).and_then(|n| fold(&n)) { + if !out.names.contains(&name) { + out.names.push(name); + } + } + (!out.names.is_empty()).then_some(out) +} + +/// The first entry of a `CFF ` table's Name INDEX. +fn cff_name(bytes: &[u8], start: usize) -> Option> { + let cff = table(bytes, start, b"CFF ")?; + let index = *cff.get(2)? as usize; + let count = be16(cff, index)?; + let offset_size = *cff.get(index.checked_add(2)?)? as usize; + if count == 0 || !(1..=4).contains(&offset_size) { + return None; + } + let offsets = index.checked_add(3)?; + let read = |n: usize| -> Option { + let at = offsets.checked_add(n.checked_mul(offset_size)?)?; + let raw = cff.get(at..at.checked_add(offset_size)?)?; + Some(raw.iter().fold(0usize, |sum, &b| (sum << 8) | b as usize)) + }; + // Offsets count from the byte before the data, so the first entry is `[off0 - 1, off1 - 1)`. + let data = offsets.checked_add(count.checked_add(1)?.checked_mul(offset_size)?)?.checked_sub(1)?; + let (from, to) = (read(0)?, read(1)?); + let name = cff.get(data.checked_add(from)?..data.checked_add(to)?)?; + Some(name[..name.len().min(NAME_RECORD_BYTES)].to_vec()) +} + +/// Every face of the font file `bytes` (one, or a TrueType Collection's several), or `None` when +/// the file is not one this module can name: the caller treats that as "wanted". +pub(super) fn faces(bytes: &[u8]) -> Option> { + if bytes.get(..4)? == b"ttcf" { + let count = be32(bytes, 8)?.min(bytes.len() / 4); + let mut seen = std::collections::HashSet::new(); + let mut starts: Vec = Vec::new(); + for i in 0..count { + let start = be32(bytes, 12 + i * 4)?; + if seen.insert(start) { + starts.push(start); + } + } + // One unreadable face makes the whole file unknown: it may be the one a style names. + starts.iter().map(|&start| face(bytes, start)).collect::>>().filter(|f| !f.is_empty()) + } else { + Some(vec![face(bytes, 0)?]) + } +} + +/// `text` is the bytes of an ASS header or script, or of one event's packet: every name it asks +/// for, folded. `script` is true for a header or a whole script, whose `Style:` lines name a +/// Fontname each, and whose `[Fonts]` and `[Graphics]` sections (uuencoded, so they hold backslashes +/// that are not tags) are not read; an event's packet holds no style lines and is read for `\fn` alone. +pub(super) fn requested(text: &[u8], script: bool) -> Vec> { + let mut out = Vec::new(); + if !script { + overrides(text, &mut out); + return out; + } + let (mut styles, mut skip, mut column) = (false, false, None::); + for line in text.split(|&b| b == b'\n') { + let line = trim(line); + if line.first() == Some(&b'[') { + let section = line.to_ascii_lowercase(); + styles = section.windows(6).any(|w| w == b"styles"); + skip = section.starts_with(b"[fonts") || section.starts_with(b"[graphics"); + continue; + } + if skip { + continue; + } + if styles { + if let Some(rest) = strip_key(line, b"format:") { + column = rest.split(|&b| b == b',').position(|f| trim(f).eq_ignore_ascii_case(b"fontname")); + } else if let Some(rest) = strip_key(line, b"style:") { + let field = rest.split(|&b| b == b',').nth(column.unwrap_or(1)); + out.extend(field.and_then(fold)); + } + } + overrides(line, &mut out); + } + out +} + +fn strip_key<'a>(line: &'a [u8], key: &[u8]) -> Option<&'a [u8]> { + (line.len() >= key.len() && line[..key.len()].eq_ignore_ascii_case(key)).then(|| &line[key.len()..]) +} + +/// Every `\fn` argument in `text`. libass reads a tag after a backslash and any spaces, ends a bare +/// argument at the next `\`, and ends the tag name at a `(`; the argument is also read to the `}` +/// of its block, and both readings are kept (a `(` is part of the name in some files and the start +/// of a parenthesized argument in others; the extra name costs a face at most). An argument inside +/// `\t(..)` ends at that tag's `)` (`ass_parse_tags` is handed the end of the `\t` argument), so +/// `{\t(0,500,\fnFoo)}` asks for "Foo" and the name is also cut at the first `)` that precedes any +/// `(`. +fn overrides(text: &[u8], out: &mut Vec>) { + let mut at = 0; + while let Some(found) = text[at..].iter().position(|&b| b == b'\\') { + at += found + 1; + let mut tag = at; + while text.get(tag) == Some(&b' ') { + tag += 1; + } + if text.get(tag..tag + 2) != Some(b"fn".as_slice()) { + continue; + } + let start = tag + 2; + let end = start + text[start..].iter().position(|&b| matches!(b, b'\\' | b'}' | b'\n' | b'\r')).unwrap_or(text.len() - start); + let argument = &text[start..end]; + out.extend(fold(argument)); + let open = argument.iter().position(|&b| b == b'('); + if let Some(paren) = open { + out.extend(fold(&argument[..paren])); + let inner = &argument[paren + 1..]; + let close = inner.iter().position(|&b| b == b')').unwrap_or(inner.len()); + out.extend(fold(&inner[..close])); + } + // Inside `\t(..)` the name ends at the tag's own `)`; a `)` after a `(` closes that `(`. + if let Some(close) = argument.iter().position(|&b| b == b')').filter(|&c| open.is_none_or(|o| c < o)) { + out.extend(fold(&argument[..close])); + } + at = end; + } +} + +/// One attachment, as the selection knows it. +struct Entry { + /// No name could be read: wanted at the first seed. + unknown: bool, + /// A byte-identical copy of an earlier attachment: never added. + duplicate: bool, + families: Vec>, +} + +/// The attached fonts of one file and which of them the renderer has been given. Owned by the +/// renderer's `Native`: the font list is fixed for the renderer's life, so indices stay valid. +pub(super) struct Selection { + entries: Vec, + by_name: HashMap, Vec>, + by_family: HashMap, Vec>, + added: Vec, +} + +impl Selection { + /// Read the names of every font once. + pub(super) fn new(fonts: &[Font]) -> Self { + let mut entries = Vec::with_capacity(fonts.len()); + let mut by_name: HashMap, Vec> = HashMap::new(); + let mut by_family: HashMap, Vec> = HashMap::new(); + // Same-length, equal-bytes copies: a length match is rare and a mismatch ends at the first + // differing byte, so this reads a font in full only when it is a real duplicate. + let mut firsts: HashMap> = HashMap::new(); + for (index, font) in fonts.iter().enumerate() { + let kept = firsts.entry(font.data.len()).or_default(); + let duplicate = kept.iter().any(|&i| fonts[i].data == font.data); + if !duplicate { + kept.push(index); + } + let faces = if duplicate { None } else { faces(&font.data) }; + let mut families = Vec::new(); + if let Some(faces) = &faces { + for face in faces { + for name in &face.names { + let holders = by_name.entry(name.clone()).or_default(); + if !holders.contains(&index) { + holders.push(index); + } + } + for family in &face.families { + let members = by_family.entry(family.clone()).or_default(); + if !members.contains(&index) { + members.push(index); + families.push(family.clone()); + } + } + } + } + entries.push(Entry { unknown: !duplicate && faces.is_none(), duplicate, families }); + } + let added = vec![false; fonts.len()]; + Self { entries, by_name, by_family, added } + } + + /// The fonts a header (or a whole sidecar script) asks for that the renderer does not hold + /// yet, in container order: its styles' Fontnames, every `\fn`, the always-requested names, and + /// every font this module cannot read. Marks them held. + pub(super) fn seed(&mut self, header: &[u8]) -> Vec { + let mut names = requested(header, true); + names.extend(ALWAYS.iter().map(|n| n.to_vec())); + let mut wanted = self.take(&names); + for (index, entry) in self.entries.iter().enumerate() { + if entry.unknown && !self.added[index] { + wanted.push(index); + } + } + self.mark(wanted) + } + + /// The fonts the `\fn` overrides of one event's packet ask for that the renderer does not hold + /// yet, in container order. Marks them held: the caller adds them before the event. + pub(super) fn wanted(&mut self, event: &[u8]) -> Vec { + let names = requested(event, false); + if names.is_empty() { + return Vec::new(); + } + let wanted = self.take(&names); + self.mark(wanted) + } + + /// The indices holding any of `names`, joined by family, not yet added. + fn take(&self, names: &[Vec]) -> Vec { + let mut wanted = Vec::new(); + for name in names { + for &index in self.by_name.get(name).into_iter().flatten() { + wanted.push(index); + for family in &self.entries[index].families { + wanted.extend(self.by_family[family].iter().copied()); + } + } + } + wanted + } + + fn mark(&mut self, mut wanted: Vec) -> Vec { + wanted.sort_unstable(); + wanted.dedup(); + wanted.retain(|&i| !self.added[i] && !self.entries[i].duplicate); + for &index in &wanted { + self.added[index] = true; + } + wanted + } +} + +#[cfg(test)] +mod tests; diff --git a/rust-modules/media/src/player/ass/font_names/tests.rs b/rust-modules/media/src/player/ass/font_names/tests.rs new file mode 100644 index 0000000000..d1fd10d37a --- /dev/null +++ b/rust-modules/media/src/player/ass/font_names/tests.rs @@ -0,0 +1,370 @@ +use super::super::synth_fonts::{self as synth, NameRecord}; +use super::*; +use std::sync::Arc; + +fn shipped(file: &str) -> Vec { + std::fs::read(super::super::asset_dir().join(file)).expect("a shipped face in pkg") +} + +/// The folded names of one face, sorted, for an order-free comparison. +fn names(face: &Face) -> Vec<&str> { + let mut all: Vec<&str> = face.names.iter().map(|n| std::str::from_utf8(n).unwrap()).collect(); + all.sort_unstable(); + all +} + +fn font(file: &str, family: &str, style: &str) -> Font { + // A tiny CFF face: nothing here is drawn, only named. + let ps = format!("{}-{style}", family.replace(' ', "")); + Font { name: file.into(), data: Arc::from(synth::cff_font(&synth::windows_names(family, style), &ps)) } +} + +fn garbage(file: &str, fill: u8) -> Font { + Font { name: file.into(), data: Arc::from(vec![fill; 64]) } +} + +const STYLES: &str = "[V4+ Styles]\nFormat: Name, Fontname, Fontsize\nStyle: Default,Zed Sans,24\n\n[Events]\n"; + +fn header_naming(fontname: &str) -> Vec { + format!("[V4+ Styles]\nFormat: Name, Fontname, Fontsize\nStyle: Default,{fontname},24\n\n[Events]\n").into_bytes() +} + +// ---- the name-table reader -------------------------------------------------------------------- + +#[test] +fn a_shipped_face_answers_to_its_family_full_and_postscript_names() { + let faces = faces(&shipped("appfont.ttf")).expect("Inter parses"); + assert_eq!(faces.len(), 1); + assert_eq!(names(&faces[0]), ["inter", "inter regular", "inter-regular"]); + assert_eq!(faces[0].families, [b"inter".to_vec()]); +} + +#[test] +fn a_face_with_two_families_answers_to_both() { + // Inter Bold also answers to "PlxSubs" (a Windows record under another language id). + let faces = faces(&shipped("appfont-bold.ttf")).unwrap(); + let mut families = faces[0].families.clone(); + families.sort(); + assert_eq!(families, [b"inter".to_vec(), b"plxsubs".to_vec()]); + assert!(names(&faces[0]).contains(&"inter bold")); +} + +#[test] +fn a_renamed_real_face_is_named_by_its_new_family_and_nothing_old() { + let renamed = synth::rename(&shipped("appfont.ttf"), "Plx Karaoke", "Bold", None); + let faces = faces(&renamed).unwrap(); + assert_eq!(names(&faces[0]), ["plx karaoke", "plx karaoke bold", "plxkaraoke-bold"]); +} + +#[test] +fn a_collection_names_every_face() { + let one = synth::rename(&shipped("appfont.ttf"), "Plx One", "Regular", None); + let two = synth::rename(&shipped("appfont.ttf"), "Plx Two", "Regular", None); + let faces = faces(&synth::collection(&[one, two])).expect("a TTC parses"); + assert_eq!(faces.len(), 2); + assert_eq!(faces[0].families, [b"plx one".to_vec()]); + assert_eq!(faces[1].families, [b"plx two".to_vec()]); +} + +#[test] +fn a_collection_with_one_unreadable_face_is_unknown() { + let one = synth::rename(&shipped("appfont.ttf"), "Plx One", "Regular", None); + let mut broken = synth::collection(&[one.clone(), one]); + broken[16..20].copy_from_slice(&0xFFFF_FF00u32.to_be_bytes()); + assert_eq!(faces(&broken), None, "a face whose offset table is off the file cannot be named"); +} + +#[test] +fn a_mac_only_name_table_is_read_as_latin_1() { + let records: Vec = vec![ + (1, 0, 1, b"Mac Family".to_vec()), + (1, 0, 4, b"Mac Family Bold".to_vec()), + (1, 0, 6, b"MacFamily-Bold".to_vec()), + ]; + let face = &faces(&synth::cff_font(&records, "Zzz")).unwrap()[0]; + assert_eq!(names(face), ["mac family", "mac family bold", "macfamily-bold", "zzz"]); +} + +#[test] +fn a_face_with_no_family_record_answers_to_its_typographic_family() { + // FreeType's family_name fallback reads ID 16 and 21. + let bare: Vec = vec![ + (3, 1, 4, synth::utf16("Only Full")), + (3, 1, 16, synth::utf16("Typo Family")), + (3, 1, 21, synth::utf16("Wws Family")), + ]; + let face = &faces(&synth::cff_font(&bare, "Cff Name")).unwrap()[0]; + assert_eq!(names(face), ["cff name", "only full", "typo family", "wws family"]); + assert_eq!(face.families, [b"typo family".to_vec(), b"wws family".to_vec()]); +} + +#[test] +fn the_typographic_family_is_a_name_even_beside_a_family_record() { + // libass counts only a Microsoft-platform ID 1, so a face whose ID 1 sits on another platform + // is named by FreeType's family_name, which prefers ID 16: ID 16 and 21 are always names. + let with_family: Vec = vec![(3, 1, 1, synth::utf16("Fam")), (3, 1, 16, synth::utf16("Typo"))]; + let face = &faces(&synth::cff_font(&with_family, "Cff")).unwrap()[0]; + assert_eq!(names(face), ["cff", "fam", "typo"]); + assert_eq!(face.families, [b"fam".to_vec(), b"typo".to_vec()]); +} + +#[test] +fn a_style_naming_the_typographic_family_selects_a_face_whose_only_id_1_is_a_mac_record() { + // The Mac-platform ID 1 "Foo" is invisible to libass, which then falls back to FreeType's + // family_name: the preferred family, ID 16 "Foo Pro". A style naming "Foo Pro" must select it. + let records: Vec = vec![(1, 0, 1, b"Foo".to_vec()), (1, 0, 16, b"Foo Pro".to_vec())]; + let fonts = [ + Font { name: "foo.otf".into(), data: Arc::from(synth::cff_font(&records, "Zzz")) }, + font("b.otf", "Unrelated", "Regular"), + ]; + let mut selection = Selection::new(&fonts); + assert_eq!(selection.seed(&header_naming("Foo Pro")), [0]); + let mut by_mac_family = Selection::new(&fonts); + assert_eq!(by_mac_family.seed(&header_naming("Foo")), [0], "the Mac family stays a name too"); +} + +#[test] +fn the_cff_font_name_is_a_postscript_name() { + let records: Vec = vec![(3, 1, 1, synth::utf16("Fam"))]; + let face = &faces(&synth::cff_font(&records, "Fam-Heavy")).unwrap()[0]; + assert_eq!(names(face), ["fam", "fam-heavy"]); +} + +#[test] +fn names_are_ascii_folded_only_and_trimmed() { + assert_eq!(fold(b" @Arial Black \t").unwrap(), b"arial black"); + assert_eq!(fold("ÜNÏ".as_bytes()).unwrap(), "ÜnÏ".as_bytes(), "only ASCII letters fold, as ass_strcasecmp does"); + assert_eq!(fold(b"@"), None); + assert_eq!(fold(b" "), None); + assert_eq!(fold(b"0"), None, "\\fn0 is the style's own font"); + assert_eq!(fold(&vec![b'a'; 5000]).unwrap().len(), NAME_RECORD_BYTES); +} + +#[test] +fn utf16_names_with_non_bmp_characters_are_decoded() { + let records: Vec = vec![(3, 10, 1, synth::utf16("Font \u{1F600} Name"))]; + let face = &faces(&synth::cff_font(&records, "N")).unwrap()[0]; + assert!(names(face).contains(&"font \u{1F600} name")); +} + +#[test] +fn bytes_that_are_not_a_font_or_are_cut_short_never_panic_and_name_nothing() { + assert_eq!(faces(b""), None); + assert_eq!(faces(b"abc"), None); + assert_eq!(faces(&[0u8; 64]), None); + assert_eq!(faces(&vec![0xFFu8; 4096]), None); + let real = shipped("appfont.ttf"); + for cut in (0..200).chain([1000, 4000]) { + let _ = faces(&real[..cut]); + } + // Cut inside the name table: its records run past the end of the file. + let tiny = synth::cff_font(&synth::windows_names("Fam", "Regular"), "Fam-Regular"); + for cut in 0..tiny.len() { + let _ = faces(&tiny[..cut]); + } + assert!(faces(&tiny).is_some()); + assert_eq!(faces(&tiny[..tiny.len() - 30]), None, "a truncated name table is unknown, not partly read"); +} + +#[test] +fn a_font_with_a_corrupted_byte_anywhere_never_panics() { + let tiny = synth::cff_font(&synth::windows_names("Fam", "Regular"), "Fam-Regular"); + for at in 0..tiny.len() { + for value in [0x00, 0x01, 0x7F, 0x80, 0xFF] { + let mut bytes = tiny.clone(); + bytes[at] = value; + let _ = faces(&bytes); + } + } + let ttc = synth::collection(&[tiny.clone(), tiny]); + for at in 0..64 { + for value in [0x00, 0xFF, 0x80] { + let mut bytes = ttc.clone(); + bytes[at] = value; + let _ = faces(&bytes); + } + } + // A collection claiming four billion faces reads only the offsets that exist. + let mut huge = ttc; + huge[8..12].copy_from_slice(&u32::MAX.to_be_bytes()); + let _ = faces(&huge); +} + +// ---- the names a track asks for ----------------------------------------------------------------- + +fn texts(found: Vec>) -> Vec { + found.into_iter().map(|n| String::from_utf8(n).unwrap()).collect() +} + +#[test] +fn a_style_line_names_the_fontname_column_the_format_line_gives() { + let header = "[V4+ Styles]\nFormat: Name, Fontsize, Fontname, Bold\nStyle: A,20,Gothic One,0\nStyle: B,20,@Vertical Mincho,0\n"; + assert_eq!(texts(requested(header.as_bytes(), true)), ["gothic one", "vertical mincho"]); + let sloppy = "[V4 Styles]\nStyle: Default,Old Style,16,&H00FFFFFF\r\n"; + assert_eq!(texts(requested(sloppy.as_bytes(), true)), ["old style"], "no Format line: column 1"); + let other = "[V4+ Styles]\nFormat: Fontname, Name\nStyle: Zed Font,Default\n"; + assert_eq!(texts(requested(other.as_bytes(), true)), ["zed font"], "Fontname in the first column"); +} + +#[test] +fn only_a_styles_section_has_style_lines() { + let script = "[Events]\nFormat: Layer, Start\nStyle: Nope,Not A Font\n[V4+ Styles]\nFormat: Name, Fontname\nStyle: S,A Font\n"; + assert_eq!(texts(requested(script.as_bytes(), true)), ["a font"]); +} + +#[test] +fn fn_overrides_are_read_in_a_header_a_script_and_an_event() { + let event = br"0,0,Default,,0,0,0,,{\an8\fn Karaoke Sans \b1\fn@Tall One}text\Nmore{\fn(Paren Font)}"; + assert_eq!(texts(requested(event, false)), ["karaoke sans", "tall one", "(paren font)", "paren font"]); + let dialogue = b"[Events]\nDialogue: 0,0:00:00.00,0:00:01.00,Default,,0,0,0,,{\\fnScript Face}x\n"; + assert_eq!(texts(requested(dialogue, true)), ["script face"]); +} + +#[test] +fn a_fn_with_a_parenthesis_keeps_both_readings() { + // libass ends the name at a `(`; a block that spells it into the name keeps it. Both are asked. + assert_eq!(texts(requested(br"{\fnNoto Sans (Bold)}", false)), ["noto sans (bold)", "noto sans", "bold"]); +} + +#[test] +fn a_fn_inside_a_t_tag_ends_at_the_tags_closing_parenthesis() { + // libass hands the nested parse the end of the `\t` argument, so the name stops at its `)`. + let asked = texts(requested(br"{\t(0,500,\fnFoo)}text", false)); + assert!(asked.contains(&"foo".to_string()), "{asked:?}"); + let asked = texts(requested(br"0,0,Default,,0,0,0,,{\t(\fn Bar Sans )\b1}x", false)); + assert!(asked.contains(&"bar sans".to_string()), "{asked:?}"); + // A tail after the `)`, another tag inside, and a second `\t` in the same block. + let asked = texts(requested(br"{\t(0,500,1,\fnFoo\fs20)\t(0,9,\fnBaz)\bord2}", false)); + assert!(asked.contains(&"foo".to_string()) && asked.contains(&"baz".to_string()), "{asked:?}"); + // An unclosed `\t(` reads to the end of its block, as before. + let asked = texts(requested(br"{\t(0,500,\fnQux}", false)); + assert!(asked.contains(&"qux".to_string()), "{asked:?}"); + // Selection: the face the nested `\fn` names is added when its event arrives. + let fonts = [font("a.otf", "Foo", "Regular"), font("b.otf", "Other", "Regular")]; + let mut selection = Selection::new(&fonts); + assert_eq!(selection.wanted(br"1,0,Default,,0,0,0,,{\t(0,500,\fnFoo)}x"), [0]); +} + +#[test] +fn fn0_an_empty_fn_and_other_tags_ask_for_nothing() { + assert_eq!(texts(requested(br"{\fscx90\fsp2\frz4\fad(1,2)}{\f}{\fr10}", false)), Vec::::new()); + assert_eq!(texts(requested(br"{\fn0}{\fn}{\fs20\fscx90}", false)), Vec::::new()); + // A backslash that ends the text, and a tag cut off at the end. + assert_eq!(texts(requested(b"{\\fn", false)), Vec::::new()); + assert_eq!(texts(requested(b"\\", false)), Vec::::new()); + assert_eq!(texts(requested(b"{\\ fnSpaced}", false)), ["spaced"], "libass skips spaces after the backslash"); +} + +#[test] +fn the_fonts_and_graphics_sections_are_not_read_for_tags() { + // uuencoded data holds backslashes; a `\fn` in it is not a tag. + let script = "[Fonts]\nfontname: a.ttf\n!!\\fnNot A Tag!!\n\n[Graphics]\n\\fnAlso Not\n\n[V4+ Styles]\nFormat: Name, Fontname\nStyle: S,Real\n"; + assert_eq!(texts(requested(script.as_bytes(), true)), ["real"]); +} + +// ---- the selection ------------------------------------------------------------------------------- + +#[test] +fn a_style_selects_the_face_it_names_and_no_other() { + let fonts = [font("a.otf", "Zed Sans", "Regular"), font("b.otf", "Other One", "Regular"), font("c.otf", "Third", "Bold")]; + let mut selection = Selection::new(&fonts); + assert_eq!(selection.seed(STYLES.as_bytes()), [0]); + assert_eq!(selection.seed(STYLES.as_bytes()), Vec::::new(), "once held, never added twice"); +} + +#[test] +fn inter_and_arial_are_always_requested() { + let fonts = [font("a.otf", "Other", "Regular"), font("i.otf", "Inter", "Regular"), font("r.otf", "ARIAL", "Bold")]; + let mut selection = Selection::new(&fonts); + assert_eq!(selection.seed(&header_naming("Nothing Matches")), [1, 2]); +} + +#[test] +fn a_full_name_or_postscript_name_selects_its_face() { + let fonts = [font("a.otf", "Fam", "Heavy"), font("b.otf", "Unrelated", "Regular")]; + let mut by_full = Selection::new(&fonts); + assert_eq!(by_full.seed(&header_naming("fam heavy")), [0]); + let mut by_postscript = Selection::new(&fonts); + assert_eq!(by_postscript.seed(&header_naming("Fam-Heavy")), [0]); +} + +#[test] +fn a_matched_face_brings_every_member_of_its_family_in_container_order() { + let fonts = [ + font("regular.otf", "Fam", "Regular"), + font("other.otf", "Other", "Regular"), + font("bold.otf", "Fam", "Bold"), + font("italic.otf", "Fam", "Italic"), + ]; + let mut by_family = Selection::new(&fonts); + assert_eq!(by_family.seed(&header_naming("FAM")), [0, 2, 3]); + // Naming the Bold face by its full name still brings the whole family: libass scores its + // members by weight and slant, and an add-order tie break that needs them all present. + let mut by_full = Selection::new(&fonts); + assert_eq!(by_full.seed(&header_naming("Fam Bold")), [0, 2, 3]); +} + +#[test] +fn a_late_fn_adds_only_what_is_new_and_a_repeat_adds_nothing() { + let fonts = [font("a.otf", "Alpha", "Regular"), font("b.otf", "Beta", "Regular"), font("c.otf", "Gamma", "Regular")]; + let mut selection = Selection::new(&fonts); + assert_eq!(selection.seed(&header_naming("Alpha")), [0]); + assert_eq!(selection.wanted(br"1,0,Default,,0,0,0,,{\fn@BETA}x"), [1]); + assert_eq!(selection.wanted(br"2,0,Default,,0,0,0,,{\fnBeta\fnAlpha}y"), Vec::::new()); + assert_eq!(selection.wanted(br"3,0,Default,,0,0,0,,{\fnGamma\fnBeta}z"), [2]); + assert_eq!(selection.wanted(b"4,0,Default,,0,0,0,,no override"), Vec::::new()); + assert_eq!(selection.wanted(br"5,0,Default,,0,0,0,,{\fnNot Attached}"), Vec::::new()); +} + +#[test] +fn a_byte_identical_copy_is_added_once_and_the_first_is_kept() { + let first = font("first.otf", "Twin", "Regular"); + let copy = Font { name: "copy.otf".into(), data: first.data.clone() }; + let again = Font { name: "again.otf".into(), data: Arc::from(first.data.to_vec()) }; + let fonts = [first, copy, again, font("x.otf", "Twin", "Bold")]; + let mut selection = Selection::new(&fonts); + assert_eq!(selection.seed(&header_naming("Twin")), [0, 3], "the copies never reach libass"); +} + +#[test] +fn a_face_that_cannot_be_read_is_wanted_once_at_the_first_seed() { + let fonts = [garbage("a.pfb", 7), font("b.otf", "Named", "Regular"), garbage("c.pfb", 7), garbage("d.pfb", 9)]; + let mut selection = Selection::new(&fonts); + assert_eq!(selection.seed(&header_naming("Elsewhere")), [0, 3], "unknown means wanted; the copy of a.pfb is not"); + assert_eq!(selection.seed(&header_naming("Named")), [1]); + assert_eq!(selection.seed(&header_naming("Named")), Vec::::new()); +} + +#[test] +fn an_empty_list_selects_nothing_and_a_script_with_no_fonts_does_not_fail() { + let mut selection = Selection::new(&[]); + assert_eq!(selection.seed(b"[Script Info]\n"), Vec::::new()); + assert_eq!(selection.wanted(br"{\fnX}"), Vec::::new()); +} + +#[test] +fn a_collection_is_wanted_when_any_of_its_faces_is_named() { + let one = synth::rename(&shipped("appfont.ttf"), "Plx One", "Regular", None); + let two = synth::rename(&shipped("appfont.ttf"), "Plx Two", "Regular", None); + let fonts = [ + Font { name: "pair.ttc".into(), data: Arc::from(synth::collection(&[one, two])) }, + font("x.otf", "Unrelated", "Regular"), + ]; + let mut selection = Selection::new(&fonts); + assert_eq!(selection.seed(&header_naming("Plx Two")), [0]); +} + +/// Issue #1328 in the report's own shape: seventeen attachments whose first twelve have no style +/// naming them, and five, last in the container, that the opening's karaoke styles do. The names +/// are the report's file names; the families are what the styles ask for. +#[test] +fn the_five_faces_a_karaoke_opening_names_are_selected_from_seventeen() { + let families = [ + "Filler 0", "Filler 1", "Filler 2", "Filler 3", "Filler 4", "Filler 5", "Filler 6", "Filler 7", + "Filler 8", "Filler 9", "Filler 10", "Filler 11", "Kiza Kinryou", "Mastro Sans", "Roboto", "Stylus", "Lehn", + ]; + let fonts: Vec = families.iter().enumerate().map(|(i, f)| font(&format!("{i}.otf"), f, "Regular")).collect(); + let header = "[V4+ Styles]\nFormat: Name, Fontname\nStyle: Sign,Kiza Kinryou\nStyle: Op1,Mastro Sans\nStyle: Op2,Roboto\nStyle: Op3,Stylus\nStyle: Op4,Lehn\nStyle: Default,Arial\n"; + let mut selection = Selection::new(&fonts); + assert_eq!(selection.seed(header.as_bytes()), [12, 13, 14, 15, 16]); +} diff --git a/rust-modules/media/src/player/ass/mailbox.rs b/rust-modules/media/src/player/ass/mailbox.rs index eb3f0f9a71..1c97f377ab 100644 --- a/rust-modules/media/src/player/ass/mailbox.rs +++ b/rust-modules/media/src/player/ass/mailbox.rs @@ -1,22 +1,81 @@ -//! The request/publish mailbox between the SDL thread and the render worker (plan section 4.2). +//! The request/publish mailbox between the SDL thread and the render workers (plan section 4.2). //! //! A frame is published with the window of playheads it stays right for ([`Validity`]), so a UI //! tick inside the window costs no job. The worker also renders the next boundary ahead of its //! time (an `Ahead` job), and the next tick at or after that boundary promotes the held frame. //! +//! A Normal result is published unless the timeline it was rendered for (epoch, source, size) was +//! replaced meanwhile or the job was withdrawn, not only when it answers the latest request: a +//! revision that moved only appended events, which start ahead of the playhead, so the frame is +//! still right for its time. A render that takes several packet intervals would otherwise never land. +//! +//! **A moving cue is a pipeline** (issue #1328). A frame of a moving cue is right for one playhead +//! only, so asking for it when the screen needs it shows it one render late (50-150 ms on the +//! television). While the player is running ([`Pace`]) the mailbox instead plans frames for FUTURE +//! playheads on a grid of one `step` (the render cost over the workers, never under one video +//! frame), a `lead` ahead of the playhead (a render, a quarter more, and a tick), hands them out as +//! `Pipe` jobs, holds the frames that land, and shows the one with the greatest playhead not after +//! now: never early, and at most a step old. Frames are planned up to a horizon only (`lead` plus a +//! step per worker), so a spare worker parks rather than rendering what the screen will not reach. +//! An event boundary inside a moving stretch (a karaoke script has one every ~19 ms) does not end a +//! frame or move the plan off its grid: the plan resolves it at its own step, and only the boundary +//! that ends the stretch is rendered at its own playhead ([`Published::ends_at`]). +//! Appended events leave the plan alone from the first event they add onward only ([`adopt_revision`]). +//! Anything the plan cannot absorb (a jump of the clock, a seek, a size, a source, a stalled worker) +//! throws it away and asks for one Normal job, exactly as a still cue does. +//! +//! **A moving stretch the renderers cannot keep up with is drawn smaller** (issue #1328, [`Governor`]). +//! The mailbox keeps the cost of a frame at full size (a frame drawn smaller is taken back to it by +//! its rung's cost ratio), asks the governor on every tick of the running pipeline which rung the +//! stretch should be drawn at given the renderers the pool allows ([`Mailbox::set_capacity`]), and +//! hands the `Pipe` jobs that rung. Only those are drawn smaller (and the Normal job that fills a +//! hole inside a reduced stretch, so libass's caches are not emptied by a frame of another size and +//! again by the next): a Normal or Ahead job otherwise, the first frame of a stretch, a still picture +//! and a paused one are drawn at full size, and a still picture the pipeline handed over drawn +//! smaller is drawn again at full size by one Normal job. The scale is not part of the [`Key`], so a +//! move never replaces the frame on screen: it stands until a frame of the new scale lands, and each +//! frame says which scale it was drawn at. The rung belongs to the selection it was decided for: a +//! new epoch, source or size starts at full size again and waits for the pool. +//! //! Pure bookkeeping: no thread, no native call, no clock but the one stamp behind [`Job::queued`]. -//! `Runtime` owns the lock and the wake; the worker loop owns the rendering. -use super::{AlignSet, Frame, Key, Source}; +//! The pace and the playhead arrive as values in [`Mailbox::request`]; the worker takes frames +//! against the latest of them between ticks. `Runtime` owns the lock and the wake; the worker loop +//! owns the rendering. +//! +//! [`adopt_revision`]: Mailbox::adopt_revision +use super::scale::{Change, Governor, Load}; +use super::{earliest_change_ms, AlignSet, Frame, Key, Pace, RenderScale, Source}; use crate::player::ass_timing::Validity; use std::sync::Arc; use std::time::Instant; +/// A moving cue is never asked for closer together than this (about thirty a second), whatever +/// the render cost over the workers allows: past it the pictures are not told apart. +const STEP_FLOOR_MS: i64 = 33; +/// The step is not stretched past this however slow the render, so a pathological sample cannot +/// freeze the plan. +const STEP_CEIL_MS: i64 = 250; +/// One render's cost in the mean is capped here: a stall is not what a frame costs. +const COST_CEIL_US: u32 = 500_000; +/// The render cost assumed before the first sample (a television; a host only does better). +const COST_DEFAULT_MS: i64 = 80; + #[derive(Clone, Copy, Debug, PartialEq, Eq)] pub(super) enum Kind { /// What the screen needs now: always beats an Ahead job. Normal, /// The boundary after the published frame, rendered early. `after` is that frame's `pub_id`. Ahead { after: u64 }, + /// A frame of a moving cue for a future playhead, planned by the animated pipeline; `run` is + /// the pipeline generation it was planned under, and a result of another one is discarded. + Pipe { run: u64 }, +} + +impl Kind { + /// Rendered ahead of the playhead rather than for what the screen needs now. + pub fn is_ahead(self) -> bool { + !matches!(self, Kind::Normal) + } } pub(super) struct Job { @@ -27,18 +86,34 @@ pub(super) struct Job { pub source: Arc, /// A Normal job enqueued while an Ahead job was rendering: when it began waiting (R6). pub queued: Option, + /// The newest frame that had landed when the job was taken: the renderer shares the regions it + /// draws identically with that frame's, so a static sign keeps the allocation the screen has + /// already uploaded whichever worker renders the next frame. Set when a worker takes the job. + pub reference: Option>, + /// The size libass draws the frame at, as a fraction of the output (issue #1328). + pub scale: RenderScale, } +/// A job a worker holds. The mailbox serves several workers, so these are a set keyed by `seq`. #[derive(Clone, Copy)] struct InFlight { kind: Kind, seq: u64, + /// The playhead the job renders. + target_ms: i64, + /// Results for a target at or past this are discarded: events were appended from there on + /// since the job began, and the frame cannot show them. + stale_from: i64, } pub(super) struct Published { /// Identity of this publication, for `Ahead { after }`. pub pub_id: u64, pub epoch: u64, + /// The key it was rendered for. It may trail the latest request's by revision alone (the newer + /// job is still out, or the pipeline adopted the revision), and then no rule that asks + /// `key == requested` takes the frame as covering the request: `request` answers R2.5 until + /// the newer revision's own frame lands. pub key: Key, pub frame: Arc, pub validity: Validity, @@ -46,6 +121,30 @@ pub(super) struct Published { /// a Text-origin source has them (plan §2.4). Held beside the frame, so a promoted ahead frame /// carries its own and the two are never read from different frames. pub aligns: Option, + /// The playhead the frame was rendered for: what a moving frame is right for. + pub target_ms: i64, + /// The picture past `validity.until_ms` moves as well ([`Rendered::moves_on`]). + pub moves_on: bool, +} + +impl Published { + /// The playhead from which the frame no longer stands for the clock, besides its age. + /// + /// A moving frame's window ends at the next event boundary, but a boundary INSIDE a moving + /// stretch (an event starts or ends while others still move) is one the plan resolves at its + /// own step, as it resolves everything else about the picture: a karaoke opening has an event + /// boundary every ~19 ms, far inside one render, and a frame that died at each of them would + /// never be shown or covered the screen (issue #1328). So such a frame stands until a newer + /// one replaces it or its age runs out ([`Mailbox::max_age_ms`]). Only the end of the stretch, + /// where the picture stops moving, is a boundary the frame ends at, and the plan renders that + /// one at its own playhead. + fn ends_at(&self) -> i64 { + if self.validity.animated && self.moves_on { + i64::MAX + } else { + self.validity.until_ms + } + } } /// What the worker made of a job. @@ -53,25 +152,69 @@ pub(super) struct Rendered { pub frame: Arc, pub validity: Validity, pub aligns: Option, + /// What the job cost in wall-clock microseconds, for the pipeline's pacing; `None` when it is + /// no sample of a steady render (the worker was not timing it, or the job opened or re-synced a + /// renderer, which re-reads every font and says nothing of what a frame costs). + pub cost_us: Option, + /// The picture at the end of the frame's window moves too: the boundary is inside a moving + /// stretch, not its end ([`Published::ends_at`]). + pub moves_on: bool, +} + +/// What the pipeline plans against: the pace and the playhead of the latest request. The worker +/// takes its next frame between ticks, so this is a value the mailbox was handed, not a clock it reads. +#[derive(Clone, Copy)] +struct Plan { + pace: Pace, + now_ms: i64, +} + +/// The frame the next one is planned after. +enum Frontier<'a> { + /// Rendered and known: its validity says where a boundary falls. + Landed(&'a Published), + /// Still rendering: only its playhead is known. + Flying(i64), } pub(super) struct Mailbox { - /// ONE slot, either kind. + /// ONE slot, a Normal job or the Ahead job of a still cue's next boundary. pending: Option, - /// Set when the worker takes `pending`, cleared on completion. - in_flight: Option, + /// Set when a worker takes a job, cleared on completion: one per busy worker. + in_flight: Vec, /// Seq of an in-flight NORMAL job a covered request made moot; its result is discarded. withdrawn: u64, - /// The latest NORMAL job: key, now, seq. Ahead jobs never write it. + /// The latest NORMAL job: key, now, seq. Ahead jobs never write it. A job's result is + /// published when its key shares this key's timeline, whatever its seq (`finish`). requested: Option<(Key, i64, u64)>, + /// The source of the latest request: what a frame planned now is rendered from, and what the + /// next revision is compared with to find where it can change a picture. + requested_source: Option>, /// Seq of the latest NORMAL job the worker completed (published or discarded) or that was /// withdrawn while pending. answered: u64, pub published: Option, - /// At most ONE ahead frame. + /// At most ONE ahead frame of a still cue. next: Option, + /// The frames of a moving cue that landed ahead of their playhead, by playhead. + held: Vec, + /// The pipeline generation: a `Pipe` job of another one is discarded when it lands. + run: u64, + /// Set while the pipeline plans; `None` while paused, still or waiting for a Normal job. + plan: Option, pub_ids: u64, seqs: u64, + /// How many workers serve this mailbox: a moving cue's frame step is the render cost over this. + workers: usize, + /// What a frame costs at FULL size, the mean the pipeline paces by, wall-clock microseconds (0 = + /// no sample yet). A frame drawn smaller is taken back to full size by its rung's cost ratio + /// before it joins the mean, so the cost does not change with the scale it is measured at. + cost_us: u32, + /// How many renderers the pool allows the selection, once it has said: what the scale is + /// decided against, and nothing is decided before it has. + capacity: Option, + /// The scale a moving stretch's frames are drawn at. + governor: Governor, } impl Default for Mailbox { @@ -81,12 +224,26 @@ impl Default for Mailbox { } /// What a `request` did, for the caller to act on outside the lock logic. -#[derive(Default, Clone, Copy, Debug, PartialEq, Eq)] +#[derive(Default, Clone, Copy, Debug, PartialEq)] pub(super) struct Requested { /// A job was queued: wake the worker. pub wake: bool, - /// The held ahead frame was promoted (`ahead_hit`). + /// A held frame was promoted: the still cue's boundary frame (`ahead_hit`), or a frame of the + /// pipeline (`piped`). pub promoted: bool, + /// The promoted frame was one the pipeline planned (`shown`). + pub piped: bool, + /// ... and it was promoted more than a tick after its playhead: it landed late (`late`). + pub late: bool, + /// How many workers the pipeline could use to reach about thirty frames a second; 0 when it + /// plans nothing. The runtime starts helpers up to what the device allows. + pub workers_wanted: usize, + /// How many renderers the plan would want if it drew at full size: the pool is consulted about + /// the selection when this is more than one (`Runtime::start_helpers`), and the scale is not + /// decided before it has answered. + pub pool_wanted: usize, + /// The scale moving frames are drawn at moved with this request (to be logged once). + pub scale_change: Option, } /// What finishing a job published, for the worker's accounting. @@ -99,30 +256,73 @@ pub(super) struct Finished { /// for the `assperf` account. pub changed: bool, pub pub_bytes: u64, + /// An ahead frame was thrown away: its timeline, run or target no longer stands (`assperf`'s `stale=`). + pub stale: bool, +} + +/// `ceil` for a millisecond count computed in floating point: a product that is exact on paper +/// must not round up for the last bit of a representation error. +fn ceil_ms(x: f64) -> i64 { + (x - 1e-6).ceil() as i64 } impl Mailbox { pub const fn new() -> Self { Self { pending: None, - in_flight: None, + in_flight: Vec::new(), withdrawn: 0, requested: None, + requested_source: None, answered: 0, published: None, next: None, + held: Vec::new(), + run: 0, + plan: None, pub_ids: 0, seqs: 0, + workers: 1, + cost_us: 0, + capacity: None, + governor: Governor::new(), } } + /// How many workers take jobs from this mailbox (at least one). + pub fn set_workers(&mut self, workers: usize) { + self.workers = workers.max(1); + } + + /// How many renderers the pool allows the selection, once it has said (`None` before): what the + /// scale is decided against. Nothing is decided while it has not. + pub fn set_capacity(&mut self, renderers: Option) { + self.capacity = renderers; + } + + + /// Everything the mailbox holds, handed out whole so it can be destroyed outside the lock, and + /// a fresh mailbox left in its place that keeps what is not a thing to retire: the worker + /// count, the cost it measured and the counters (a job another worker still holds finishes + /// against the new mailbox, and its `seq` must not be one the new mailbox hands out). + pub fn retire(&mut self) -> Mailbox { + let mut fresh = Mailbox::new(); + fresh.workers = self.workers; + fresh.cost_us = self.cost_us; + fresh.pub_ids = self.pub_ids; + fresh.seqs = self.seqs; + fresh.run = self.run + 1; + std::mem::replace(self, fresh) + } + /// Nothing held: a worker with no source may park without cleanup. pub fn is_empty(&self) -> bool { self.pending.is_none() - && self.in_flight.is_none() + && self.in_flight.is_empty() && self.requested.is_none() && self.published.is_none() && self.next.is_none() + && self.held.is_empty() } /// A Normal job is pending or in flight (or was, and its result is still awaited). @@ -134,13 +334,20 @@ impl Mailbox { self.requested.map(|(key, _, _)| key) } + /// Queue a Normal job: what the screen needs now, drawn at full size. (A hole inside a reduced + /// stretch is recovered at the stretch's own size: [`Mailbox::enqueue_normal_at`].) fn enqueue_normal(&mut self, key: Key, now_ms: i64, source: &Arc) { + self.enqueue_normal_at(key, now_ms, source, RenderScale::Full); + } + + fn enqueue_normal_at(&mut self, key: Key, now_ms: i64, source: &Arc, scale: RenderScale) { self.seqs += 1; self.requested = Some((key, now_ms, self.seqs)); + self.requested_source = Some(source.clone()); // A Normal job this one replaces keeps its earlier stamp: the wait began then (R6). let queued = match self.pending.as_ref() { Some(Job { kind: Kind::Normal, queued: Some(since), .. }) => Some(*since), - _ => matches!(self.in_flight, Some(InFlight { kind: Kind::Ahead { .. }, .. })) + _ => (self.in_flight.len() >= self.workers && self.in_flight.iter().any(|f| f.kind.is_ahead())) .then(Instant::now), }; self.pending = Some(Job { @@ -150,6 +357,8 @@ impl Mailbox { seq: self.seqs, source: source.clone(), queued, + reference: None, + scale, }); } @@ -162,13 +371,12 @@ impl Mailbox { self.pending = None; withdrew = true; } - if let (Some(flight), Some((_, _, latest))) = (self.in_flight, self.requested) { - if flight.kind == Kind::Normal - && flight.seq == latest - && flight.seq > self.answered - && flight.seq != self.withdrawn + if let Some((_, _, latest)) = self.requested { + if self.in_flight.iter().any(|f| f.kind == Kind::Normal && f.seq == latest) + && latest > self.answered + && latest != self.withdrawn { - self.withdrawn = flight.seq; + self.withdrawn = latest; withdrew = true; } } @@ -184,7 +392,7 @@ impl Mailbox { } let after = published.pub_id; let same = |kind: Kind| kind == Kind::Ahead { after }; - if self.in_flight.is_some_and(|f| same(f.kind)) { + if self.in_flight.iter().any(|f| same(f.kind)) { return false; } match &self.pending { @@ -201,24 +409,258 @@ impl Mailbox { seq: self.seqs, source: source.clone(), queued: None, + reference: None, + scale: RenderScale::Full, }); true } - /// R2: MAIN THREAD, under the lock. `now_ms` is the playhead this tick asks about. - pub fn request(&mut self, key: Key, now_ms: i64, source: &Arc) -> Requested { - // R2.1: a new key (late push, seek, restyle, size change). - if self.requested_key() != Some(key) { + /// The frame step of the pipeline, in playhead milliseconds: what one render costs over the + /// workers sharing it, scaled to the playhead's speed, never under a video frame or about + /// thirty a second, and never past [`STEP_CEIL_MS`]. + pub fn step_ms(&self, pace: Pace) -> i64 { + let per_frame = self.cost_ms() * pace.rate / self.workers as f64; + ceil_ms(per_frame).max(pace.frame_ms.max(STEP_FLOOR_MS)).min(STEP_CEIL_MS) + } + + /// How far ahead of the playhead a moving frame is planned, in playhead milliseconds: a + /// render and a quarter more for the variation of one, and a tick for the wait for the tick + /// that shows it. Workers do not change it: one render takes as long whoever does it. + pub fn lead_ms(&self, pace: Pace) -> i64 { + ceil_ms((self.cost_ms() * 1.25 + pace.tick_ms as f64) * pace.rate) + } + + /// How many workers the plan could use: enough that the frame step reaches the thirty-a-second + /// floor at the cost measured, drawing at the scale it does (more would render frames the step + /// never asks for). + pub fn workers_wanted(&self, pace: Pace) -> usize { + self.workers_at(pace, self.governor.scale()) + } + + /// How many workers the plan would want drawing at full size: what the pool is asked about, since + /// the scale is what it can make of the answer. + pub fn pool_wanted(&self, pace: Pace) -> usize { + self.workers_at(pace, RenderScale::Full) + } + + fn workers_at(&self, pace: Pace, scale: RenderScale) -> usize { + let floor = pace.frame_ms.max(STEP_FLOOR_MS) as f64; + ((self.cost_full_ms() * scale.cost_ratio() * pace.rate / floor) - 1e-6).ceil().max(1.0) as usize + } + + /// The oldest a held or published moving frame may be and still stand for the playhead: a + /// frame later than this is a hole, and the plan is restarted. + fn max_age_ms(&self, pace: Pace) -> i64 { + self.lead_ms(pace) + 2 * self.step_ms(pace) + } + + /// What a frame costs at full size, in milliseconds. + pub fn cost_full_ms(&self) -> f64 { + if self.cost_us == 0 { + COST_DEFAULT_MS as f64 + } else { + f64::from(self.cost_us) / 1000.0 + } + } + + /// What a frame of the moving stretch costs now, at the scale it is drawn at. + fn cost_ms(&self) -> f64 { + self.cost_full_ms() * self.governor.scale().cost_ratio() + } + + /// A sample of what a render drawn at `scale` costs, into the mean the pipeline paces by (a + /// quarter of each new one; the first seeds it), taken back to full size first. The first + /// sample of a moving stretch replaces the mean: what the last stretch cost says nothing of + /// this one. + fn note_cost(&mut self, sample_us: u32, scale: RenderScale) { + let full = (f64::from(sample_us) / scale.cost_ratio()).round() as u32; + let sample = full.clamp(1, COST_CEIL_US); + let first = self.governor.is_fresh(); + self.cost_us = if self.cost_us == 0 || first { + sample + } else { + ((u64::from(self.cost_us) * 3 + u64::from(sample) + 2) / 4) as u32 + }; + // only a frame of the rung in use says how that rung is doing; a frame of another one (a + // Normal job's, one that was in flight over a move) feeds the mean and no more + if first || scale == self.governor.scale() { + self.governor.note_sample(); + } + } + + /// The scale for the next frames of the running pipeline, when the pool has said what it has. + fn decide_scale(&mut self, now_ms: i64, pace: Pace) -> Option { + let renderers = self.capacity?; + let load = Load { + cost_full_ms: self.cost_full_ms(), + renderers, + rate: pace.rate, + target_ms: pace.frame_ms.max(STEP_FLOOR_MS), + }; + self.governor.observe(now_ms, load) + } + + /// A still picture is on screen: the next moving frame begins a stretch (issue #1328). + fn picture_shown(&mut self) { + if self.published.as_ref().is_some_and(|p| !p.validity.animated) { + self.governor.begin_stretch(); + } + } + + /// The pipeline is the way to show the moving frame on screen: the player is running and the + /// frame published for this timeline moves with the clock. + fn piping(&self, key: &Key, pace: Pace) -> bool { + pace.running + && self.published.as_ref().is_some_and(|p| p.validity.animated && p.key.same_timeline(key)) + } + + /// Throw the plan away: the frames held, the jobs in flight (they are discarded when they + /// land, for the generation moved on) and the plan itself. + fn stop_pipe(&mut self) { + self.held.clear(); + self.run += 1; + self.plan = None; + } + + /// The first frame the plan may have jobs for after the ones it has: the greatest playhead + /// among the frames held and the frames in flight, else the one on screen. + fn frontier(&self) -> Option> { + let flying = self + .in_flight + .iter() + .filter(|f| matches!(f.kind, Kind::Pipe { run } if run == self.run) && f.target_ms < f.stale_from) + .map(|f| f.target_ms) + .max(); + match (self.held.last(), flying) { + (Some(held), Some(flying)) if flying > held.target_ms => Some(Frontier::Flying(flying)), + (Some(held), _) => Some(Frontier::Landed(held)), + (None, Some(flying)) => Some(Frontier::Flying(flying)), + (None, None) => self.published.as_ref().filter(|p| p.validity.animated).map(Frontier::Landed), + } + } + + /// The playhead of the next frame to plan, `None` when there is nothing more to plan yet. + /// + /// One step after the frontier, but never closer than a `lead` to the playhead: a render + /// started now lands by then. A frame that landed knows where its picture ends, and a + /// frontier whose next step would pass that boundary plans the boundary itself instead, so + /// the end of a moving stretch is rendered at its own playhead (a boundary inside the stretch + /// is not one: [`Published::ends_at`]). A still picture after a moving one needs no more + /// frames, and nothing is planned beyond the horizon. + fn next_target(&self) -> Option { + let Plan { pace, now_ms } = self.plan?; + let (step, lead) = (self.step_ms(pace), self.lead_ms(pace)); + let floor = now_ms + lead; + let horizon = floor + self.workers as i64 * step; + if self.held.iter().any(|f| !f.validity.animated && f.validity.until_ms == i64::MAX) { + // the still picture after the cue has landed + return None; + } + let target = match self.frontier() { + None => floor, + Some(Frontier::Flying(at)) => (at + step).max(floor), + Some(Frontier::Landed(frame)) if frame.validity.animated => { + let target = (frame.target_ms + step).max(floor); + if frame.ends_at() < target { + frame.ends_at().max(now_ms) + } else { + target + } + } + Some(Frontier::Landed(frame)) => match frame.validity.until_ms { + i64::MAX => return None, + until => until.max(now_ms), + }, + }; + (target <= horizon).then_some(target) + } + + /// A worker is idle and could be given a job: the wake to send. + fn wants_job(&self) -> bool { + self.in_flight.len() < self.workers && (self.pending.is_some() || self.next_target().is_some()) + } + + /// A revision of the timeline the pipeline is running appended events: keep the plan for every + /// playhead before the first of them (appended events start ahead of the playhead, so a frame + /// of an earlier playhead is right whatever was added) and replan from there. Not for a + /// revision that reaches the playhead or that the renderer cannot take as an append. + fn adopt_revision(&mut self, key: Key, now_ms: i64, source: &Arc, pace: Pace) -> bool { + let Some(old) = self.requested_key() else { return false }; + if !pace.running || !key.same_timeline(&old) || key.revision <= old.revision { + return false; + } + // With a Normal job out the plan is stopped and its frame will start it anew; with none, the + // frame on screen has to be one the appended events leave right. + let moving = self.published.as_ref().is_some_and(|p| p.validity.animated && p.key.same_timeline(&key)); + if !self.normal_outstanding() && !moving { + return false; + } + let Some(earliest) = self.requested_source.as_ref().map(|old| earliest_change_ms(old, source)) else { + return false; + }; + if earliest <= now_ms { + return false; + } + self.requested = self.requested.map(|(_, now, seq)| (key, now, seq)); + self.requested_source = Some(source.clone()); + self.held.retain(|f| f.target_ms < earliest); + for flight in self.in_flight.iter_mut().filter(|f| f.kind != Kind::Normal) { + flight.stale_from = flight.stale_from.min(earliest); + } + true + } + + /// R2: MAIN THREAD, under the lock. `now_ms` is the playhead this tick asks about; `pace` is + /// how it is moving. + pub fn request(&mut self, key: Key, now_ms: i64, source: &Arc, pace: Pace) -> Requested { + // R2.1: a new key (late push, seek, restyle, size change), but for a revision the pipeline + // takes in. + if self.requested_key() != Some(key) && !self.adopt_revision(key, now_ms, source, pace) { + if self.requested_key().is_none_or(|old| !old.same_timeline(&key)) { + // another selection (or size): the rung was decided for the last one, and the pool has + // not said what this one may have + self.governor = Governor::new(); + } self.enqueue_normal(key, now_ms, source); self.next = None; - return Requested { wake: true, promoted: false }; + self.stop_pipe(); + return Requested { wake: true, ..Requested::default() }; + } + if self.piping(&key, pace) { + return self.request_piping(key, now_ms, source, pace); + } + if self.plan.is_some() { + // the player stopped running, or the moving picture stopped moving + self.stop_pipe(); + } + let lagging = self.published.as_ref().is_some_and(|p| p.key != key && p.key.same_timeline(&key)); + if lagging && !self.normal_outstanding() { + // The pipeline took revisions in and left no Normal job behind: one frame under this + // revision hands over to the rules below, which wait for the key's own first frame. + self.enqueue_normal(key, now_ms, source); + return Requested { wake: true, ..Requested::default() }; } + self.request_static(key, now_ms, source) + } + + /// R2.2-R2.5: a still cue, or a moving one for a clock that is not running. + fn request_static(&mut self, key: Key, now_ms: i64, source: &Arc) -> Requested { let Some(published) = self.published.as_ref().filter(|p| p.key == key) else { // R2.5: the key's first frame is still in flight. return Requested::default(); }; let animated = published.validity.animated; let mut out = Requested::default(); + if !animated && published.validity.contains(now_ms) && published.frame.scale != RenderScale::Full { + // The still picture after a reduced stretch is drawn smaller than the output: it is drawn + // again at full size, and the reduced frame stands until that lands. It is not "covered" + // (below), which would withdraw that job. + if !self.normal_outstanding() { + self.enqueue_normal(key, now_ms, source); + out.wake = true; + } + return out; + } if !animated && published.validity.contains(now_ms) { // R2.2: covered. if self.withdraw() { @@ -233,6 +675,7 @@ impl Mailbox { { // R2.3: promote the frame rendered ahead, keeping its pub_id. self.published = self.next.take(); + self.picture_shown(); self.withdraw(); out.promoted = true; out.wake = self.arm_ahead(source); @@ -247,6 +690,91 @@ impl Mailbox { out } + /// Whether a held frame can be shown at `now_ms`: a moving one from its own playhead on, for + /// as long as a plan's lateness allows and before the boundary that ends its stretch + /// ([`Published::ends_at`]); a still one inside its window. + fn showable(frame: &Published, now_ms: i64, max_age: i64) -> bool { + if frame.validity.animated { + (frame.target_ms..frame.ends_at()).contains(&now_ms) && now_ms - frame.target_ms <= max_age + } else { + frame.validity.contains(now_ms) + } + } + + /// R2 for a moving cue while the player runs: promote the frame whose playhead has come, keep + /// the plan going, or give it up for one Normal job if the screen would otherwise be left + /// without a picture for the playhead. + fn request_piping(&mut self, key: Key, now_ms: i64, source: &Arc, pace: Pace) -> Requested { + let mut out = Requested::default(); + if self.normal_outstanding() { + // the pipeline starts again from the frame that job is rendering + self.plan = None; + return out; + } + self.plan = Some(Plan { pace, now_ms }); + out.scale_change = self.decide_scale(now_ms, pace); + let (lead, step, max_age) = (self.lead_ms(pace), self.step_ms(pace), self.max_age_ms(pace)); + // what can no longer be shown: past its boundary or too late + self.held.retain(|f| { + let past = f.ends_at() <= now_ms || (f.validity.animated && f.target_ms + max_age < now_ms); + !past + }); + let on_screen = self.published.as_ref().map_or(i64::MIN, |p| p.target_ms); + let pick = self.held.iter().rposition(|f| f.target_ms > on_screen && Self::showable(f, now_ms, max_age)); + if let Some(at) = pick { + let frame = self.held.remove(at); + self.held.drain(..at.min(self.held.len())); + // frames older than the one shown are of no use + self.held.retain(|f| f.target_ms > frame.target_ms); + let moving = frame.validity.animated; + // a frame is on time when the first tick at or after its playhead shows it + out.late = moving && now_ms - frame.target_ms > ceil_ms(pace.tick_ms as f64 * pace.rate); + self.published = Some(frame); + out.promoted = true; + out.piped = true; + if !moving { + // the cue ended: its still picture is shown, and the boundary logic takes over + self.picture_shown(); + self.stop_pipe(); + let lagging = self.published.as_ref().is_some_and(|p| p.key != key); + if lagging { + self.enqueue_normal(key, now_ms, source); + out.wake = true; + } else { + out.wake = self.arm_ahead(source); + } + return out; + } + } + let Some(published) = self.published.as_ref() else { return out }; + let slack = pace.frame_ms; + let covered = published.target_ms - now_ms <= slack + && now_ms - published.target_ms <= max_age + && now_ms < published.ends_at(); + if !covered { + let (lo, hi) = (now_ms - max_age, now_ms + lead + (self.workers as i64 + 1) * step); + let coming = self.held.iter().any(|f| (lo..=hi).contains(&f.target_ms)) + || self.in_flight.iter().any(|f| { + matches!(f.kind, Kind::Pipe { run } if run == self.run) + && f.target_ms < f.stale_from + && (lo..=hi).contains(&f.target_ms) + }); + if !coming { + // a hole: nothing on screen is right and nothing is on its way. The frame that fills + // it is drawn at the size the stretch is: a frame at another size makes libass empty + // its caches, and the pipeline's next frame would pay for them again. + self.enqueue_normal_at(key, now_ms, source, self.governor.scale()); + self.stop_pipe(); + out.wake = true; + return out; + } + } + out.wake = self.wants_job(); + out.workers_wanted = self.workers_wanted(pace); + out.pool_wanted = self.pool_wanted(pace); + out + } + /// The alignment classes of the published frame whose `serial` is asked for; `None` when the /// published frame is another one, or has none. pub fn aligns_of(&self, serial: u64) -> Option { @@ -278,6 +806,8 @@ impl Mailbox { frame: frame.clone(), validity: Validity::ALWAYS, aligns: None, + target_ms: 0, + moves_on: false, }); frame } @@ -292,23 +822,61 @@ impl Mailbox { (changed, bytes) } - /// WORKER, under the lock: take the pending job and mark it in flight. + /// WORKER, under the lock: take a job and mark it in flight. The pending one first (a Normal + /// job, or a still cue's Ahead job); else, while the pipeline plans, the next frame of the + /// moving cue. A worker asks only when it is idle, so jobs are never more than workers. pub fn take(&mut self) -> Option { - let job = self.pending.take()?; - self.in_flight = Some(InFlight { kind: job.kind, seq: job.seq }); + let mut job = match self.pending.take() { + Some(job) => job, + None => { + let now_ms = self.next_target()?; + let (key, source) = (self.requested_key()?, self.requested_source.clone()?); + self.seqs += 1; + let scale = self.governor.scale(); + Job { kind: Kind::Pipe { run: self.run }, key, now_ms, seq: self.seqs, source, queued: None, reference: None, scale } + } + }; + // The newest picture that has landed: the frame this one follows on screen, or close to it. + job.reference = self.held.last().or(self.published.as_ref()).map(|p| p.frame.clone()); + self.in_flight.push(InFlight { kind: job.kind, seq: job.seq, target_ms: job.now_ms, stale_from: i64::MAX }); Some(job) } /// WORKER, under the lock: R3 and R5. `outcome` is `None` when the job did not render or /// its epoch ended; the job is finished either way, so no counter can stick. + /// + /// A Normal result is published unless the job was withdrawn or its timeline (epoch, source, + /// size) is no longer the latest request's. A job whose key differs from the latest request's + /// by revision alone still publishes, under its own key: appended events start ahead of the + /// playhead, so the frame stays right for its time, and on a slow set one render outlives + /// several appends (issue #1328). Such a frame arms no Ahead job and leaves the newer pending + /// job alone; that job publishes the newest revision when it lands. + /// + /// A `Pipe` result is held for its playhead when the plan it was made under still stands: the + /// generation, the timeline, the first appended event and the frame on screen all allow it. pub fn finish(&mut self, job: &Job, outcome: Option) -> Finished { - self.in_flight = None; + // A job this mailbox does not hold was handed out before the mailbox was retired (another + // worker's, the epoch having ended): there is nothing of it to settle. + let Some(at) = self.in_flight.iter().position(|f| f.seq == job.seq) else { + return Finished::default(); + }; + let flight = self.in_flight.swap_remove(at); + // The pipeline paces by what a MOVING frame costs: a still picture, an empty one or an error + // costs next to nothing and says nothing of the frames the pipeline is for. + if let Some(cost_us) = outcome.as_ref().filter(|o| o.validity.animated).and_then(|o| o.cost_us) { + self.note_cost(cost_us, job.scale); + } match job.kind { Kind::Normal => { self.answered = self.answered.max(job.seq); - let Some(Rendered { frame, validity, aligns }) = outcome else { return Finished::default() }; - let latest = self.requested.is_some_and(|(key, _, seq)| key == job.key && seq == job.seq); - if job.seq == self.withdrawn || !latest { + let Some(Rendered { frame, validity, aligns, moves_on, .. }) = outcome else { return Finished::default() }; + // A result stands unless the playhead was covered without it (`withdrawn`) or the + // timeline it was rendered for was replaced. A revision that moved meanwhile only + // appended events, which start ahead of the playhead, so the frame is still right + // for its time; on a slow set a render outlives several appends, and holding the + // frame back until one lands in a quiet gap shows nothing for a whole opening. + let timeline = self.requested.is_some_and(|(key, _, _)| key.same_timeline(&job.key)); + if job.seq == self.withdrawn || !timeline { return Finished::default(); } let (changed, pub_bytes) = self.differs(&frame, job.key.epoch); @@ -320,18 +888,26 @@ impl Mailbox { frame, validity, aligns, + target_ms: job.now_ms, + moves_on, }); + self.picture_shown(); self.next = None; - self.arm_ahead(&job.source); - Finished { wake: changed, changed, pub_bytes } + self.held.clear(); + // Only the newest request's frame has a boundary worth rendering ahead; the + // newer job still pending will arm it when it lands. + if self.requested_key() == Some(job.key) { + self.arm_ahead(&job.source); + } + Finished { wake: changed, changed, pub_bytes, stale: false } } Kind::Ahead { after } => { - let Some(Rendered { frame, validity, aligns }) = outcome else { return Finished::default() }; + let Some(Rendered { frame, validity, aligns, moves_on, .. }) = outcome else { return Finished::default() }; let current = self.published.as_ref().is_some_and(|p| p.pub_id == after && p.key == job.key) && self.requested_key() == Some(job.key) && self.next.is_none(); if !current { - return Finished::default(); + return Finished { stale: true, ..Finished::default() }; } let (changed, pub_bytes) = self.differs(&frame, job.key.epoch); self.pub_ids += 1; @@ -342,13 +918,66 @@ impl Mailbox { frame, validity, aligns, + target_ms: job.now_ms, + moves_on, }); // Nothing to show yet: the promotion on the UI thread is what the screen sees. - Finished { wake: false, changed, pub_bytes } + Finished { wake: false, changed, pub_bytes, stale: false } + } + Kind::Pipe { run } => { + let Some(Rendered { frame, validity, aligns, moves_on, .. }) = outcome else { return Finished::default() }; + let stale = Finished { stale: true, ..Finished::default() }; + let on_screen = self.published.as_ref().filter(|p| p.validity.animated && p.key.same_timeline(&job.key)); + let standing = run == self.run + && self.requested_key().is_some_and(|key| key.same_timeline(&job.key)) + && job.now_ms < flight.stale_from + && on_screen.is_some_and(|p| p.target_ms < job.now_ms); + let at = self.held.partition_point(|f| f.target_ms < job.now_ms); + if !standing || self.held.get(at).is_some_and(|f| f.target_ms == job.now_ms) { + return stale; + } + // different from the frame before it in time, which is what is on screen when it is shown + let before = at.checked_sub(1).and_then(|i| self.held.get(i)).or(self.published.as_ref()); + let changed = before.is_none_or(|old| old.frame.serial != frame.serial); + let pub_bytes = if changed { frame.rects.iter().map(|r| r.rgba.len() as u64).sum() } else { 0 }; + self.pub_ids += 1; + self.held.insert( + at, + Published { + pub_id: self.pub_ids, + epoch: job.key.epoch, + key: job.key, + frame, + validity, + aligns, + target_ms: job.now_ms, + moves_on, + }, + ); + // Nothing to show yet: the tick at its playhead is what the screen sees. + Finished { wake: false, changed, pub_bytes, stale: false } } } } + /// The scale a moving stretch's frames are drawn at now. Everything else is drawn at full size: + /// a still picture, a paused one, the first frame of a stretch and any job that is not the + /// pipeline's (`take`). + #[cfg(test)] + pub fn scale(&self) -> RenderScale { + self.governor.scale() + } + + #[cfg(test)] + pub fn held_targets(&self) -> Vec { + self.held.iter().map(|f| f.target_ms).collect() + } + + #[cfg(test)] + pub fn in_flight_len(&self) -> usize { + self.in_flight.len() + } + #[cfg(test)] pub fn pending_queued(&self) -> Option { self.pending.as_ref().and_then(|j| j.queued) diff --git a/rust-modules/media/src/player/ass/mailbox_tests.rs b/rust-modules/media/src/player/ass/mailbox_tests.rs index 7c3c87a731..70e0f4bdff 100644 --- a/rust-modules/media/src/player/ass/mailbox_tests.rs +++ b/rust-modules/media/src/player/ass/mailbox_tests.rs @@ -1,5 +1,6 @@ //! The state machine of plan section 4.2 with a fake renderer: no native library, so these run in //! `make check`. Each cue paints one pixel holding its tag; tag 0 paints nothing. +use super::mailbox::Kind; use super::*; use crate::player::ass_timing::{self, Motion, Span}; @@ -27,6 +28,9 @@ struct Sim { /// A windowed script: how far ahead of the playhead the held window reaches. slide: Option, window: Option<(i64, i64)>, + /// How the playhead moves. A paused pace is the one every test below was written for: a moving + /// cue is then one frame per clock (the animated pipeline needs a running pace, tested apart). + pace: Pace, } fn key_of(epoch: u64, revision: u64) -> Key { @@ -63,6 +67,7 @@ impl Sim { promoted: 0, slide: None, window: None, + pace: Pace::PAUSED, } } @@ -74,7 +79,7 @@ impl Sim { fn render(&mut self, job: &Job) -> mailbox::Rendered { match job.kind { Kind::Normal => self.normal.push(job.now_ms), - Kind::Ahead { .. } => self.ahead.push(job.now_ms), + Kind::Ahead { .. } | Kind::Pipe { .. } => self.ahead.push(job.now_ms), } if let Some(span) = self.slide { // Engine::slide: a playhead outside the held window feeds the renderer again. @@ -97,6 +102,7 @@ impl Sim { width: job.key.width, height: job.key.height, rects, + scale: job.scale, error: None, }) } @@ -114,7 +120,7 @@ impl Sim { validity.from_ms = validity.from_ms.max(lo); validity.until_ms = validity.until_ms.min(hi); } - mailbox::Rendered { frame, validity, aligns: None } + mailbox::Rendered { frame, validity, aligns: None, cost_us: None, moves_on: false } } fn complete(&mut self, job: Job) { @@ -129,7 +135,7 @@ impl Sim { } fn request(&mut self, now: i64) -> Option { - let out = self.mb.request(self.key, now, &self.source); + let out = self.mb.request(self.key, now, &self.source, self.pace); self.promoted += u32::from(out.promoted); self.mb.view(&self.key).map(|f| f.rects.first().map_or(0, |r| r.rgba[0])) } @@ -167,11 +173,11 @@ fn error_frame_requests_once() { let key = key_of(1, 1); let mut jobs = 0; for tick in 0..120 { - mb.request(key, tick * 16, &source); + mb.request(key, tick * 16, &source, Pace::PAUSED); while let Some(job) = mb.take() { jobs += 1; let frame = error_frame(job.key, Fault::RenderFailed); - mb.finish(&job, Some(mailbox::Rendered { frame, validity: Validity::ALWAYS, aligns: None })); + mb.finish(&job, Some(mailbox::Rendered { frame, validity: Validity::ALWAYS, aligns: None, cost_us: None, moves_on: false })); } assert_eq!(mb.view(&key).unwrap().error, Some(Fault::RenderFailed)); } @@ -363,11 +369,15 @@ fn ahead_never_overwrites_pending_normal() { assert_eq!(sim.mb.pending_job().map(|(k, _)| k), Some(Kind::Normal)); } +/// A moving cue under a pace that is NOT running (paused, a frame step, a scrub): one frame per +/// distinct clock, nothing ahead. With a running playhead the cue is pipelined instead, and +/// `an_animated_run_is_shown_on_time_with_one_worker_through_every_append` says how. #[test] fn animated_cue_requests_every_tick() { let mut animated = cue(0, 100_000, 4); animated.animated = true; let mut sim = Sim::new(vec![animated]); + sim.pace = Pace::PAUSED; for tick in 0..120 { sim.tick(tick * 16); } @@ -469,7 +479,7 @@ fn ahead_frame_from_old_epoch_not_promoted() { sim.tick(100); assert!(sim.mb.next_frame().is_some()); sim.key = key_of(2, 1); // a seek re-ids the track: the epoch moved - let out = sim.mb.request(sim.key, 1500, &sim.source); + let out = sim.mb.request(sim.key, 1500, &sim.source, Pace::PAUSED); assert!(!out.promoted && out.wake); assert_eq!(sim.mb.pending_job().map(|(k, _)| k), Some(Kind::Normal)); assert!(sim.mb.view(&sim.key).is_none(), "the old epoch's frame is never shown"); @@ -513,7 +523,7 @@ fn a_cancelled_epoch_finishes_the_job_without_publishing() { assert!(sim.mb.pending_job().is_none()); // the Normal job counts as answered, so a later request is not blocked behind it sim.key = key_of(2, 1); - assert!(sim.mb.request(sim.key, 100, &sim.source).wake); + assert!(sim.mb.request(sim.key, 100, &sim.source, Pace::PAUSED).wake); } #[test] @@ -574,7 +584,1099 @@ fn aligns_are_read_for_the_published_frames_serial_only_and_follow_a_promotion() let next_serial = next.frame.serial; sim.mb.finish(&ahead, Some(next)); assert_eq!(sim.mb.aligns_of(next_serial), None, "held ahead, not published yet"); - assert!(sim.mb.request(sim.key, 1100, &sim.source).promoted); + assert!(sim.mb.request(sim.key, 1100, &sim.source, Pace::PAUSED).promoted); assert_eq!(sim.mb.aligns_of(next_serial), Some(AlignSet { bottom: false, middle: false, top: true })); assert_eq!(sim.mb.aligns_of(serial), None); } + +#[test] +fn a_render_slower_than_the_packet_rate_still_publishes_issue_1328() { + // A karaoke line whose picture changes every 100 ms, so every tick asks for a render. + let cues: Vec = (0..40) + .map(|i| Cue { start: i * 100, end: (i + 1) * 100, tag: (i + 1) as u8, animated: true }) + .collect(); + let mut sim = Sim::new(cues); + // Not running: the clock here moves per tick with no playhead behind it, which is the per-clock path. + // The running path's version of this regression is the on-time test with revisions appended. + sim.pace = Pace::PAUSED; + let (mut now, mut revision) = (0, 1); + sim.tick(now); + let mut published = 0; + for _ in 0..20 { + let job = sim.mb.take().unwrap_or_else(|| { + now += 16; + sim.request(now); + sim.mb.take().expect("an animated frame asks for a job on every tick") + }); + // On the television one render takes ~150 ms (9 ticks) while a demuxed packet lands every + // ~33 ms (2 ticks): the revision moves several times before the render completes. + for tick in 0..9 { + now += 16; + if tick % 2 == 0 { + revision += 1; + sim.key = key_of(1, revision); + } + sim.request(now); + } + let before = sim.published().pub_id; + sim.complete(job); + published += u32::from(sim.published().pub_id != before); + } + assert!(published >= 18, "a completed render reaches the screen ({published} of 20)"); + let shown = sim.mb.view(&sim.key).map(|f| i64::from(f.rects[0].rgba[0])); + assert!(shown.is_some_and(|tag| tag + 3 >= now / 100), "the picture moved with the clock ({shown:?} at {now} ms)"); +} + +#[test] +fn a_render_lagging_only_by_revision_publishes_and_leaves_the_newer_job_alone() { + let mut sim = Sim::new(vec![cue(0, 100_000, 1), cue(2000, 3000, 9)]); + sim.request(100); + let flying = sim.mb.take().unwrap(); + let old = flying.key; + sim.key = key_of(1, 2); // a packet was appended while the job rendered + assert!(sim.mb.request(sim.key, 116, &sim.source, Pace::PAUSED).wake, "the new revision queues its own job"); + let outcome = sim.render(&flying); + let done = sim.mb.finish(&flying, Some(outcome)); + assert!(done.wake && done.changed, "the screen is woken for the frame"); + assert_eq!(sim.published().key, old, "published under the key it was rendered for"); + assert_eq!(sim.mb.view(&sim.key).unwrap().rects[0].rgba[0], 1, "and shown at once"); + assert!(sim.mb.next_frame().is_none(), "no Ahead job is armed from a lagging frame"); + assert!(matches!(sim.mb.pending_job(), Some((Kind::Normal, 116))), "the newer job still runs"); + // The lagging key falls through to R2.5: nothing extra is queued while the newer job is out. + assert_eq!(sim.mb.request(sim.key, 132, &sim.source, Pace::PAUSED), mailbox::Requested::default()); + sim.drain(); + assert_eq!(sim.published().key, sim.key, "the newest revision lands as usual"); + let boundary = sim.published().validity.until_ms; + assert_eq!(sim.mb.next_frame().map(|n| n.validity.from_ms), Some(boundary), "the newest frame arms the next boundary"); +} + +#[test] +fn a_render_for_a_replaced_timeline_is_still_discarded() { + let base = key_of(1, 1); + let newer = [ + ("epoch", Key { epoch: 2, revision: 2, ..base }), + ("source", Key { source_id: 2, revision: 2, ..base }), + ("width", Key { width: 640, revision: 2, ..base }), + ("height", Key { height: 360, revision: 2, ..base }), + ("storage width", Key { storage_width: 640, revision: 2, ..base }), + ("storage height", Key { storage_height: 360, revision: 2, ..base }), + ]; + for (what, key) in newer { + let mut sim = Sim::new(vec![cue(0, 100_000, 1)]); + sim.request(100); + let flying = sim.mb.take().unwrap(); + sim.key = key; + sim.request(116); + let outcome = sim.render(&flying); + let done = sim.mb.finish(&flying, Some(outcome)); + assert_eq!(done, mailbox::Finished::default(), "a changed {what} discards the result"); + assert!(sim.mb.published.is_none(), "nothing was published for a changed {what}"); + assert!(matches!(sim.mb.pending_job(), Some((Kind::Normal, 116))), "the newer job still runs ({what})"); + } +} + +// ---- the animated pipeline (issue #1328) ------------------------------------------------------- +// +// A moving cue was rendered once per tick for the clock of that tick, and the result landed one +// render later, so a 80 ms render showed pictures 80-160 ms old, a dozen times a second. With a +// running playhead the mailbox now plans the frames: it renders ahead for future playheads on a +// step the measured cost sets, holds them, and shows each at its playhead, never before it. +// +// The rig is a discrete-event model: a moving cue over the whole script, `workers` workers that each +// spend `cost_ms` of wall-clock time per render, a UI tick every 16 ms and a playhead of `rate` x +// wall time (plus an `offset` the viewer can move). Each frame carries the playhead it was rendered +// for in its one pixel, so a test reads what the screen shows and how old it is. A worker looks for +// work only when it finishes a job or is woken (`Requested::wake`), as the real one parks, so a +// missing wake shows up as an idle worker. + +use std::collections::BTreeSet; + +const END: i64 = 1_000_000; + +fn playing(rate: f64, frame_ms: i64) -> Pace { + Pace { running: true, rate, frame_ms, tick_ms: 16 } +} + +fn event_at(start_ms: i64) -> Event { + Event { start_ms, duration_ms: 1000, payload: Arc::from(&b"0,0,Default,,0,0,0,,x"[..]) } +} + +fn script_of(revision: u64, events: Vec) -> Arc { + Arc::new(Source { + id: 1, + revision, + origin: Origin::Authored, + content: Content::Embedded { + header: Arc::from(&b"[Script Info]\n"[..]), + events: events.into(), + fonts: Arc::from([]), + }, + }) +} + +/// The frame for `job`'s playhead: one pixel holding it, animated until the cue ends at `cue_end` +/// and a still empty picture after. `cost_ms` is what the worker reports for the job. +fn moving(job: &Job, cost_ms: i64, cue_end: i64) -> mailbox::Rendered { + moving_between(job, cost_ms, cue_end, 0) +} + +/// As [`moving`], with an event starting or ending every `boundary_ms` playhead milliseconds (0: +/// none before `cue_end`): the frame's window of validity ends at the next of them, as a dense +/// karaoke script's does (the issue's fixture has ~53 edges a second). +fn moving_between(job: &Job, cost_ms: i64, cue_end: i64, boundary_ms: i64) -> mailbox::Rendered { + let target = job.now_ms; + let (rects, validity) = if target < cue_end { + let rgba: Arc<[u8]> = Arc::from((target as i32).to_le_bytes()); + let (from_ms, until_ms) = match boundary_ms { + 0 => (0, cue_end), + b => (target.div_euclid(b) * b, (target.div_euclid(b) * b + b).min(cue_end)), + }; + (vec![Rect { x: 0, y: 0, width: 1, height: 1, rgba }], Validity { from_ms, until_ms, animated: true }) + } else { + (Vec::new(), Validity { from_ms: cue_end, until_ms: i64::MAX, animated: false }) + }; + let frame = Arc::new(Frame { + source_id: job.key.source_id, + serial: next_serial(), + width: job.key.width, + height: job.key.height, + rects, + scale: job.scale, + error: None, + }); + // the boundary is the end of the moving stretch only when it is the cue's: before that the + // picture after it moves as well + let moves_on = validity.animated && validity.until_ms < cue_end; + mailbox::Rendered { frame, validity, aligns: None, cost_us: Some(cost_ms as u32 * 1000), moves_on } +} + +/// The playhead a `moving` frame was rendered for. +fn target_of(frame: &Frame) -> Option { + frame.rects.first().map(|r| i64::from(i32::from_le_bytes(r.rgba[..4].try_into().unwrap()))) +} + +/// A mailbox with `workers` workers and a moving frame published at 1000 whose render cost +/// `cost_ms`: the state a pipeline starts from. +fn warm(workers: usize, cost_ms: i64) -> (Mailbox, Arc, Key) { + let mut mb = Mailbox::new(); + mb.set_workers(workers); + let (source, key) = (script_of(1, Vec::new()), key_of(1, 1)); + mb.request(key, 1000, &source, playing(1.0, 33)); + let job = mb.take().expect("the first frame is a Normal job"); + mb.finish(&job, Some(moving(&job, cost_ms, END))); + (mb, source, key) +} + +struct Busy { + done_at: i64, + cost: i64, + job: Job, +} + +struct Rig { + mb: Mailbox, + source: Arc, + events: Vec, + key: Key, + pace: Pace, + cost_ms: i64, + cue_end: i64, + /// An event boundary every this many playhead milliseconds (0: only the cue's end). + boundary_ms: i64, + /// Added to the playhead: the viewer's subtitle timing offset moving. + offset: i64, + wall: i64, + busy: Vec>, + parked: Vec, + /// The costs of the next jobs, ahead of `cost_ms`: a stall. + slow: Vec, + normals: u32, + pipes: u32, + stale: u32, + /// Every job taken: its kind, the playhead it renders and the revision it was handed. + taken: Vec<(Kind, i64, u64)>, + /// `(playhead, playhead of the frame)` at each promotion. + promotions: Vec<(i64, i64)>, + /// `(playhead, playhead of the frame on screen)` after each tick. + shown: Vec<(i64, Option)>, + /// What a frame costs at each rung over what it costs at full size (full, two thirds, half): the + /// model's by default, which a test sets to something else to see what a wrong model does. + ratios: [f64; 3], + /// The scale and kind of every job taken. + scales: Vec<(Kind, RenderScale)>, + /// The scale of the frame on screen after each tick (`None`: no frame). + shown_scales: Vec>, + /// `(playhead, from, to)` for each move of the scale a request reported. + changes: Vec<(i64, RenderScale, RenderScale)>, +} + +impl Rig { + fn new(workers: usize, cost_ms: i64) -> Self { + let mut mb = Mailbox::new(); + mb.set_workers(workers); + Self { + mb, + source: script_of(1, Vec::new()), + events: Vec::new(), + key: key_of(1, 1), + pace: playing(1.0, 33), + cost_ms, + cue_end: END, + boundary_ms: 0, + offset: 0, + wall: 1000, + busy: (0..workers).map(|_| None).collect(), + parked: vec![true; workers], + slow: Vec::new(), + normals: 0, + pipes: 0, + stale: 0, + taken: Vec::new(), + promotions: Vec::new(), + shown: Vec::new(), + ratios: RenderScale::LADDER.map(RenderScale::cost_ratio), + scales: Vec::new(), + shown_scales: Vec::new(), + changes: Vec::new(), + } + } + + fn now(&self) -> i64 { + (self.wall as f64 * self.pace.rate) as i64 + self.offset + } + + fn try_take(&mut self, worker: usize) { + if self.busy[worker].is_some() { + return; + } + match self.mb.take() { + Some(job) => { + match job.kind { + Kind::Normal => self.normals += 1, + _ => self.pipes += 1, + } + self.taken.push((job.kind, job.now_ms, job.key.revision)); + self.scales.push((job.kind, job.scale)); + let rung = RenderScale::LADDER.iter().position(|&s| s == job.scale).unwrap(); + let cost = if self.slow.is_empty() { + (self.cost_ms as f64 * self.ratios[rung]).round() as i64 + } else { + self.slow.remove(0) + }; + self.busy[worker] = Some(Busy { done_at: self.wall + cost, cost, job }); + self.parked[worker] = false; + } + None => self.parked[worker] = true, + } + } + + /// Finish every job due by wall time `until`, in order, each worker taking its next job at the + /// moment it finished. + fn finish_due(&mut self, until: i64) { + while let Some((at, worker)) = self + .busy + .iter() + .enumerate() + .filter_map(|(w, b)| b.as_ref().map(|b| (b.done_at, w))) + .filter(|&(at, _)| at <= until) + .min() + { + self.wall = at; + let Busy { cost, job, .. } = self.busy[worker].take().unwrap(); + let done = self.mb.finish(&job, Some(moving_between(&job, cost, self.cue_end, self.boundary_ms))); + self.stale += u32::from(done.stale); + self.try_take(worker); + } + self.wall = until; + } + + fn tick(&mut self) { + self.finish_due(self.wall + self.pace.tick_ms); + let now = self.now(); + let out = self.mb.request(self.key, now, &self.source, self.pace); + if let Some(change) = out.scale_change { + self.changes.push((now, change.from, change.to)); + } + if out.promoted { + let target = self.mb.published.as_ref().map_or(0, |p| p.target_ms); + self.promotions.push((now, target)); + } + if out.wake { + for worker in 0..self.busy.len() { + if self.parked[worker] { + self.try_take(worker); + } + } + } + let frame = self.mb.view(&self.key); + self.shown_scales.push(frame.as_ref().map(|f| f.scale)); + self.shown.push((now, frame.and_then(|f| target_of(&f)))); + } + + fn run(&mut self, ms: i64) { + let until = self.wall + ms; + while self.wall < until { + self.tick(); + } + } + + /// A packet whose event starts at `start_ms` arrives: a new revision of the same script. + fn append(&mut self, start_ms: i64) { + self.events.push(event_at(start_ms)); + let revision = self.source.revision + 1; + self.source = script_of(revision, self.events.clone()); + self.key = Key { revision, ..self.key }; + } + + /// How old the frame on screen was at each tick from `from` on, in playhead milliseconds. + fn ages(&self, from: usize) -> Vec { + self.shown[from..].iter().map(|&(now, target)| now - target.expect("a frame is on screen")).collect() + } + + fn distinct_shown(&self, from: usize) -> usize { + self.shown[from..].iter().map(|&(_, target)| target).collect::>().len() + } + + fn late_promotions(&self) -> usize { + self.promotions.iter().filter(|&&(now, target)| now - target > 16).count() + } +} + +#[test] +fn an_animated_run_is_shown_on_time_with_one_worker_through_every_append() { + let mut rig = Rig::new(1, 80); + rig.run(1000); + let (normals, from) = (rig.normals, rig.shown.len()); + for i in 0..300 { + if i % 2 == 0 { + // a demuxed packet every other tick, for an event well ahead of the playhead + let now = rig.now(); + rig.append(now + 5000); + } + rig.tick(); + } + assert_eq!(rig.normals, normals, "an append far ahead of the playhead asks for no Normal job"); + let ages = rig.ages(from); + assert!( + ages.iter().all(|age| (0..=100).contains(age)), + "a frame is never early, and at most a step (80 ms) and a tick old: {ages:?}" + ); + assert!(rig.distinct_shown(from) >= 55, "{} different frames in 4.8 s", rig.distinct_shown(from)); + assert!( + rig.late_promotions() * 10 <= rig.promotions.len(), + "{} of {} frames were promoted more than a tick after their playhead", + rig.late_promotions(), + rig.promotions.len() + ); +} + +#[test] +fn three_workers_reach_thirty_frames_a_second_from_an_eighty_millisecond_render() { + let mut rig = Rig::new(3, 80); + rig.run(1000); + let (normals, from, pipes) = (rig.normals, rig.shown.len(), rig.pipes); + rig.run(5000); + assert_eq!(rig.normals, normals); + let distinct = rig.distinct_shown(from); + assert!(distinct >= 140, "{distinct} different frames in 5 s"); + let ages = rig.ages(from); + assert!(ages.iter().all(|age| (0..=50).contains(age)), "a 33 ms step and a tick: {ages:?}"); + assert!(rig.pipes - pipes <= 165, "{} renders for about 150 frames: little is wasted", rig.pipes - pipes); + assert!(rig.late_promotions() * 10 <= rig.promotions.len()); +} + +/// The television's heavy karaoke opening (issue #1328, `fonts-over-budget.mkv`): about 2400 events in +/// 90 s, so an event starts or ends every ~19 ms and each frame's window of validity is shorter than a +/// render. The pipeline never engaged there: for twenty seconds every job was a Normal one, nothing was +/// planned ahead and no frame was ever shown by the plan. +#[test] +fn a_moving_cue_with_an_event_boundary_every_few_milliseconds_is_still_pipelined() { + // (workers, render cost, ms between event boundaries, frames to show in 5 s) + for (workers, cost_ms, boundary_ms, frames) in + [(1, 90, 19, 50_u32), (3, 90, 19, 130), (1, 90, 7, 50), (3, 90, 7, 130), (1, 90, 40, 50), (1, 90, 250, 50)] + { + let mut rig = Rig::new(workers, cost_ms); + rig.boundary_ms = boundary_ms; + rig.run(2000); + let (normals, from, pipes) = (rig.normals, rig.shown.len(), rig.pipes); + rig.run(5000); + let name = format!("{workers} worker(s) at {cost_ms} ms, a boundary every {boundary_ms} ms"); + assert!(rig.normals - normals <= 1, "{name}: {} Normal jobs in 5 s, the plan was given up", rig.normals - normals); + assert!(rig.pipes - pipes >= frames, "{name}: {} planned renders in 5 s", rig.pipes - pipes); + assert!(rig.distinct_shown(from) >= frames as usize, "{name}: {} different frames in 5 s", rig.distinct_shown(from)); + let ages = rig.ages(from); + let step = rig.mb.step_ms(rig.pace); + assert!( + ages.iter().all(|age| (0..=step + 40).contains(age)), + "{name}: a frame is never early and at most a step and a tick old: {ages:?}" + ); + } +} + +/// Boundaries inside the moving stretch are the plan's to resolve at its own step, so the end of a dense +/// stretch is shown within a step of it (a lone cue's end is exact: `when_the_moving_cue_ends...`), +/// and the plan hands over to the still picture without giving up on the way. +#[test] +fn a_dense_stretch_hands_over_to_the_still_picture_within_a_step_of_its_end() { + let mut rig = Rig::new(1, 90); + rig.boundary_ms = 19; + rig.cue_end = 3501; + rig.run(3000); + assert!(rig.normals <= 1, "{} Normal jobs: the plan is not given up on the way", rig.normals); + let published = rig.mb.published.as_ref().expect("a frame"); + assert!(!published.validity.animated, "the still picture after the cue is what is shown"); + let handed_over = rig + .shown + .iter() + .find(|&&(now, shown)| now >= 3501 && shown.is_none()) + .map(|&(now, _)| now) + .expect("the still picture was shown"); + assert!(handed_over - 3501 <= 90 + 32, "the cue ended at 3501 and the picture changed at {handed_over}"); + let pipes = rig.pipes; + rig.run(1000); + assert_eq!(rig.pipes, pipes, "nothing is planned for a picture that does not move"); +} + +#[test] +fn the_frame_step_follows_the_render_cost_over_the_workers_and_never_drops_below_a_video_frame() { + let pace = playing(1.0, 33); + let (mut mb, ..) = warm(1, 80); + assert_eq!((mb.step_ms(pace), mb.lead_ms(pace)), (80, 116), "the cost; and the cost a quarter over, plus a tick"); + mb.set_workers(3); + assert_eq!(mb.step_ms(pace), 33, "27 ms of cost per frame is under the 30 a second ceiling"); + assert_eq!(mb.step_ms(playing(1.0, 42)), 42, "24 fps content gets no step under its frame period"); + assert_eq!(mb.lead_ms(pace), 116, "workers do not change how long one render takes"); + assert_eq!(mb.step_ms(playing(2.0, 33)), 54, "at twice the speed a render spans twice the playhead: 160 over 3"); + assert_eq!(mb.lead_ms(playing(2.0, 33)), 232); + let (slow, ..) = warm(1, 2000); + assert_eq!(slow.step_ms(pace), 250, "a pathological render does not stretch the step without bound"); + assert_eq!(slow.lead_ms(pace), 641, "and its sample is capped at half a second"); +} + +#[test] +fn a_jump_of_the_clock_inside_one_key_restarts_the_pipeline_with_one_normal_job() { + for jump in [-5000, 5000] { + let mut rig = Rig::new(3, 80); + rig.run(1500); + let normals = rig.normals; + rig.offset = jump; + rig.tick(); + assert!(rig.mb.held_targets().is_empty(), "the frames planned for the old clock are of no use ({jump})"); + rig.run(1500); + assert_eq!(rig.normals, normals + 1, "one Normal job for the new clock ({jump})"); + let from = rig.shown.len(); + rig.run(1000); + let ages = rig.ages(from); + assert!(ages.iter().all(|age| (0..=50).contains(age)), "on time again at the new clock ({jump}): {ages:?}"); + } +} + +#[test] +fn a_stalled_worker_is_recovered_from_without_showing_a_frame_early() { + let mut rig = Rig::new(1, 80); + rig.run(1000); + let normals = rig.normals; + rig.slow = vec![900]; + rig.run(3000); + let early = rig.shown.iter().filter(|&&(now, target)| target.is_some_and(|t| t > now)).count(); + assert_eq!(early, 0, "a frame is never shown before its playhead"); + assert!(rig.normals - normals <= 3, "the stall re-anchors at most a few times: {}", rig.normals - normals); + let from = rig.shown.len() - 60; + let ages = rig.ages(from); + assert!(ages.iter().all(|age| (0..=100).contains(age)), "on time again after the stall: {ages:?}"); +} + +#[test] +fn no_frame_is_planned_past_the_horizon_so_a_spare_worker_parks() { + for workers in [1usize, 3] { + let pace = playing(1.0, 33); + let (mut mb, source, key) = warm(workers, 80); + mb.request(key, 1100, &source, pace); + let (step, lead) = (mb.step_ms(pace), mb.lead_ms(pace)); + let mut targets = Vec::new(); + while let Some(job) = mb.take() { + assert!(matches!(job.kind, Kind::Pipe { .. }), "{:?}", job.kind); + targets.push(job.now_ms); + } + assert_eq!(targets.first(), Some(&(1100 + lead)), "the first lands in time: a render and a tick ahead"); + assert!(targets.windows(2).all(|w| w[1] - w[0] == step), "one step apart: {targets:?}"); + assert!(targets.last().is_some_and(|&t| t <= 1100 + lead + workers as i64 * step)); + assert_eq!(targets.len(), workers + 1, "up to the horizon and no further ({workers} workers): {targets:?}"); + } +} + +#[test] +fn a_held_frame_is_never_shown_before_its_playhead() { + let pace = playing(1.0, 33); + let (mut mb, source, key) = warm(1, 80); + mb.request(key, 1100, &source, pace); + let job = mb.take().expect("a frame is planned"); + let target = job.now_ms; + mb.finish(&job, Some(moving(&job, 80, END))); + for now in (1100..target).step_by(4) { + let out = mb.request(key, now, &source, pace); + assert!(!out.promoted, "promoted at {now}, before {target}"); + assert_eq!(mb.published.as_ref().map(|p| p.target_ms), Some(1000)); + } + let out = mb.request(key, target, &source, pace); + assert!(out.promoted, "and promoted at its own playhead"); + assert_eq!(mb.published.as_ref().map(|p| p.target_ms), Some(target)); +} + +#[test] +fn the_greatest_frame_not_after_the_playhead_is_shown_whatever_order_the_frames_landed_in() { + let pace = playing(1.0, 33); + for order in ["later first", "earlier first", "earlier after the later was shown"] { + let (mut mb, source, key) = warm(2, 80); + mb.request(key, 1100, &source, pace); + let (early, late) = (mb.take().unwrap(), mb.take().unwrap()); + assert!(early.now_ms < late.now_ms); + let land = |mb: &mut Mailbox, job: &Job| mb.finish(job, Some(moving(job, 80, END))); + let mut last = mailbox::Finished::default(); + match order { + "later first" => { + land(&mut mb, &late); + land(&mut mb, &early); + } + "earlier first" => { + land(&mut mb, &early); + land(&mut mb, &late); + } + _ => { + land(&mut mb, &late); + assert!(mb.request(key, late.now_ms, &source, pace).promoted); + last = land(&mut mb, &early); + } + } + if order != "earlier after the later was shown" { + let out = mb.request(key, late.now_ms + 5, &source, pace); + assert!(out.promoted, "{order}"); + } + assert_eq!(mb.published.as_ref().map(|p| p.target_ms), Some(late.now_ms), "{order}"); + assert!(mb.held_targets().iter().all(|&t| t > late.now_ms), "the older frame is dropped ({order})"); + if order == "earlier after the later was shown" { + assert!(last.stale, "a frame older than the one on screen is thrown away"); + } + } +} + +#[test] +fn a_seek_a_new_source_or_a_new_size_throws_the_whole_plan_away() { + let pace = playing(1.0, 33); + let key = key_of(1, 1); + let newer = [ + ("epoch", Key { epoch: 2, ..key }), + ("source", Key { source_id: 2, ..key }), + ("size", Key { width: 640, ..key }), + ("storage size", Key { storage_height: 360, ..key }), + ]; + for (what, newer) in newer { + let (mut mb, source, key) = warm(3, 80); + mb.request(key, 1100, &source, pace); + let (flying, also) = (mb.take().unwrap(), mb.take().unwrap()); + let landed = mb.take().unwrap(); + mb.finish(&landed, Some(moving(&landed, 80, END))); + assert!(!mb.held_targets().is_empty(), "{what}: a frame is held"); + assert!(mb.request(newer, 1116, &source, pace).wake, "{what}"); + assert!(mb.held_targets().is_empty(), "{what}: the held frames are dropped"); + assert_eq!(mb.pending_job().map(|(k, _)| k), Some(Kind::Normal), "{what}"); + for job in [flying, also] { + let done = mb.finish(&job, Some(moving(&job, 80, END))); + assert!(done.stale && !done.changed, "{what}: a frame of the old timeline is discarded"); + } + assert!(mb.held_targets().is_empty(), "{what}"); + } +} + +#[test] +fn an_append_keeps_the_frames_before_its_first_event_and_replans_from_there() { + let mut rig = Rig::new(3, 80); + rig.run(1500); + let normals = rig.normals; + let held = rig.mb.held_targets(); + assert!(held.len() >= 2, "three workers hold frames ahead: {held:?}"); + let cut = held[1]; + rig.append(cut); + rig.tick(); + // a worker that took a job before this tick's request read the revision before it + let taken = rig.taken.len(); + let after = rig.mb.held_targets(); + assert!(after.iter().all(|&t| t < cut), "the frames from the new event on are dropped: {after:?} (cut {cut})"); + let from = rig.shown.len(); + rig.run(1500); + assert_eq!(rig.normals, normals, "the pipeline replans; it does not ask for a Normal job"); + assert!( + rig.taken[taken..].iter().all(|&(_, _, revision)| revision == rig.key.revision), + "every render after the append reads the new revision" + ); + let ages = rig.ages(from); + assert!(ages.iter().all(|age| (0..=120).contains(age)), "no hole is left where the frames were dropped: {ages:?}"); +} + +#[test] +fn an_append_that_reaches_the_playhead_restarts_with_a_normal_job() { + let mut rig = Rig::new(3, 80); + rig.run(1500); + let normals = rig.normals; + let now = rig.now(); + rig.append(now - 20); + rig.tick(); + assert!(rig.mb.held_targets().is_empty(), "what was planned is dropped with the picture it was planned from"); + rig.run(300); + assert_eq!(rig.normals, normals + 1, "the picture on screen may be wrong: it is rendered again"); + rig.run(1200); + assert_eq!(rig.normals, normals + 1, "and the pipeline starts again from that frame"); + let from = rig.shown.len() - 30; + assert!(rig.ages(from).iter().all(|age| (0..=50).contains(age))); +} + +#[test] +fn a_normal_job_waits_only_when_every_worker_is_busy_with_a_frame_ahead() { + let pace = playing(1.0, 33); + for (workers, busy, waits) in [(2usize, 2usize, true), (2, 1, false), (1, 1, true)] { + let (mut mb, source, key) = warm(workers, 80); + mb.request(key, 1100, &source, pace); + let _rendering: Vec = (0..busy).map(|_| mb.take().unwrap()).collect(); + mb.request(Key { epoch: 2, ..key }, 1116, &source, pace); + assert_eq!(mb.pending_queued().is_some(), waits, "{workers} workers, {busy} busy"); + } +} + +#[test] +fn a_paused_player_plans_nothing_and_a_new_clock_renders_one_frame_for_it() { + let (mut mb, source, key) = warm(3, 80); + mb.request(key, 1100, &source, playing(1.0, 33)); + let flying = mb.take().expect("a frame is planned while playing"); + let paused = Pace { running: false, ..playing(1.0, 33) }; + mb.request(key, 1000, &source, paused); + assert!(mb.held_targets().is_empty()); + assert!(mb.take().is_none(), "nothing is planned while paused"); + let done = mb.finish(&flying, Some(moving(&flying, 80, END))); + assert!(done.stale && mb.held_targets().is_empty(), "a frame planned before the pause is discarded"); + // a frame step or an offset change gives a new clock: one frame for it, as for any moving cue + assert!(mb.request(key, 1500, &source, paused).wake); + assert_eq!(mb.pending_job(), Some((Kind::Normal, 1500))); + let job = mb.take().unwrap(); + mb.finish(&job, Some(moving(&job, 80, END))); + // playing again plans from the frame on screen + mb.request(key, 1520, &source, playing(1.0, 33)); + assert!(matches!(mb.take().map(|j| j.kind), Some(Kind::Pipe { .. }))); +} + +#[test] +fn a_faster_playhead_plans_further_ahead_in_playhead_time() { + for (rate, lead, step) in [(1.0, 116, 80), (2.0, 232, 160), (0.5, 58, 40)] { + let pace = playing(rate, 33); + let (mut mb, source, key) = warm(1, 80); + mb.request(key, 1100, &source, pace); + let first = mb.take().expect("a frame is planned"); + assert_eq!(first.now_ms, 1100 + lead, "rate {rate}"); + let second = mb.take().expect("and one more within the horizon"); + assert_eq!(second.now_ms - first.now_ms, step, "rate {rate}"); + } +} + +#[test] +fn a_still_cue_costs_the_same_jobs_whatever_the_pace() { + for pace in [Pace::PAUSED, playing(1.0, 33), playing(2.0, 42)] { + let mut sim = Sim::new(vec![cue(1000, 200_000, 7), cue(300_000, 400_000, 8)]); + sim.pace = pace; + for tick in 0..120 { + assert_eq!(sim.tick(2000 + tick * 16).unwrap_or(7), 7); + } + assert_eq!(sim.jobs(), (1, 1), "one Normal job and one Ahead job for a still cue ({pace:?})"); + assert_eq!(sim.ahead, [200_000], "the Ahead job renders the cue's end"); + assert_eq!(sim.tick(200_000), Some(0), "and the boundary is promoted exactly as before"); + assert_eq!(sim.promoted, 1); + } +} + +#[test] +fn with_one_worker_one_job_is_in_flight_and_only_a_few_frames_are_held() { + let mut rig = Rig::new(1, 80); + for _ in 0..300 { + rig.tick(); + assert!(rig.mb.in_flight_len() <= 1); + assert!(rig.mb.held_targets().len() <= 3, "{:?}", rig.mb.held_targets()); + } + assert!(rig.pipes > 40, "the single worker kept rendering ahead: {}", rig.pipes); +} + +#[test] +fn finishing_a_job_clears_it_and_a_job_of_a_retired_mailbox_is_ignored() { + let pace = playing(1.0, 33); + let (mut mb, source, key) = warm(2, 80); + assert_eq!(mb.in_flight_len(), 0, "the warm-up's job was finished"); + mb.request(key, 1100, &source, pace); + let (a, b) = (mb.take().unwrap(), mb.take().unwrap()); + assert_eq!(mb.in_flight_len(), 2); + mb.finish(&a, None); + assert_eq!(mb.in_flight_len(), 1, "a job that did not render is finished too"); + // the epoch ended while `b` was rendering: its mailbox was retired and is a fresh one now + drop(mb.retire()); + assert!(mb.is_empty()); + assert_eq!(mb.finish(&b, Some(moving(&b, 80, END))), mailbox::Finished::default()); + assert!(mb.is_empty(), "nothing of a job the mailbox does not hold is settled against it"); + mb.request(key, 2000, &source, pace); + let next = mb.take().unwrap(); + assert!(next.seq > b.seq, "a retired mailbox never hands out a seq an old job may still carry"); +} + +#[test] +fn when_the_moving_cue_ends_the_pipeline_stops_and_the_still_picture_takes_over() { + let mut rig = Rig::new(1, 80); + rig.cue_end = 1700; + rig.run(1800); + assert!( + rig.taken.iter().any(|&(_, target, _)| target == 1700), + "the end of the cue is rendered at its own playhead, not at the next step: {:?}", + rig.taken + ); + let pipes = rig.pipes; + rig.run(1500); + let published = rig.mb.published.as_ref().expect("a frame"); + assert!(!published.validity.animated, "the still picture after the cue is what is shown"); + assert_eq!(rig.pipes, pipes, "no frame is planned for a picture that does not move"); + assert!(rig.mb.held_targets().is_empty()); +} + +#[test] +fn the_earliest_change_of_an_append_is_the_first_event_it_adds() { + let old = script_of(1, vec![event_at(100), event_at(200)]); + let appended = script_of(2, vec![event_at(100), event_at(200), event_at(900), event_at(700)]); + assert_eq!(earliest_change_ms(&old, &appended), 700, "the earliest start among the new events"); + assert_eq!(earliest_change_ms(&old, &script_of(2, vec![event_at(100), event_at(200)])), i64::MAX, "nothing new"); + // expiry from the front and an append: only the added event counts, what expired is in the past + let slid = script_of(3, vec![event_at(200), event_at(800)]); + assert_eq!(earliest_change_ms(&old, &slid), 800); + // anything the renderer cannot express as an append can change every playhead + let mut other_id = (*appended).clone(); + other_id.id = 2; + assert_eq!(earliest_change_ms(&old, &other_id), i64::MIN); + let mut restyled = (*appended).clone(); + restyled.content = Content::Embedded { + header: Arc::from(&b"[Script Info]\nTitle: x\n"[..]), + events: Arc::from([event_at(100), event_at(200), event_at(900)]), + fonts: Arc::from([]), + }; + assert_eq!(earliest_change_ms(&old, &restyled), i64::MIN); + let whole = Source { + id: 1, + revision: 2, + origin: Origin::Authored, + content: Content::Script { bytes: Arc::from(&b"[Script Info]\n"[..]), fonts: Arc::from([]) }, + }; + assert_eq!(earliest_change_ms(&old, &whole), i64::MIN); +} + +#[test] +fn the_pace_reads_the_players_values_and_clamps_the_odd_ones() { + let film = Pace::of(true, 1000, 23.976, 16); + assert_eq!((film.running, film.rate, film.frame_ms, film.tick_ms), (true, 1.0, 42, 16)); + assert_eq!(Pace::of(true, 1000, 0.0, 16).frame_ms, 42, "an unknown rate is film"); + assert_eq!(Pace::of(true, 1000, f64::NAN, 16).frame_ms, 42); + assert_eq!(Pace::of(true, 1000, 59.94, 16).frame_ms, 17); + assert_eq!(Pace::of(true, 1000, 29.97, 16).frame_ms, 33); + assert_eq!(Pace::of(true, 1000, 1000.0, 16).frame_ms, 4, "never a zero step"); + assert_eq!(Pace::of(true, 2000, 24.0, 16).rate, 2.0); + assert_eq!(Pace::of(true, 50, 24.0, 16).rate, 0.25); + assert_eq!(Pace::of(true, 90_000, 24.0, 16).rate, 4.0); + assert_eq!(Pace::of(false, 1000, 24.0, 0).tick_ms, 4); + assert_eq!(Pace::of(false, 1000, 24.0, 4_000_000_000).tick_ms, 100, "a stall in the caller's ticks"); + assert!(!Pace::of(false, 1000, 24.0, 16).running); +} + +// The workers a pipeline can use, and what the screen and the renderers are told about its frames. + +#[test] +fn a_job_carries_the_newest_landed_frame_as_its_reference() { + let pace = playing(1.0, 33); + let (mut mb, source, key) = warm(2, 80); + mb.request(key, 1100, &source, pace); + let reference = |job: &Job| job.reference.as_ref().and_then(|frame| target_of(frame)); + let (first, second) = (mb.take().unwrap(), mb.take().unwrap()); + assert_eq!(reference(&first), Some(1000), "the frame on screen, while nothing newer has landed"); + assert_eq!(reference(&second), Some(1000)); + let first_target = first.now_ms; + mb.finish(&first, Some(moving(&first, 80, END))); + let third = mb.take().expect("a third frame is within the horizon of two workers"); + assert_eq!(reference(&third), Some(first_target), "the newest frame that landed, not the one on screen"); + // a Normal job after the plan was thrown away refers to what is on screen again + mb.request(Key { width: 640, ..key }, 1130, &source, pace); + let normal = mb.take().unwrap(); + assert_eq!(normal.kind, Kind::Normal); + assert_eq!(reference(&normal), Some(1000), "the plan's frames were dropped with it"); +} + +#[test] +fn a_still_cues_ahead_job_refers_to_the_frame_it_follows() { + let mut sim = Sim::new(vec![cue(1000, 2000, 7), cue(2000, 3000, 8)]); + sim.request(1100); + let normal = sim.mb.take().unwrap(); + assert!(normal.reference.is_none(), "nothing was shown before the first frame"); + sim.complete(normal); + let ahead = sim.mb.take().unwrap(); + assert!(matches!(ahead.kind, Kind::Ahead { .. })); + let shown = sim.mb.view(&sim.key).unwrap(); + assert!(ahead.reference.as_ref().is_some_and(|frame| Arc::ptr_eq(frame, &shown))); +} + +#[test] +fn the_pipeline_asks_for_the_workers_that_reach_thirty_frames_a_second() { + for (cost_ms, rate, wanted) in [(20, 1.0, 1), (50, 1.0, 2), (80, 1.0, 3), (80, 2.0, 5), (500, 1.0, 16)] { + let (mut mb, source, key) = warm(1, cost_ms); + let out = mb.request(key, 1100, &source, playing(rate, 33)); + assert_eq!(out.workers_wanted, wanted, "{cost_ms} ms at {rate}x"); + } + let (mut mb, source, key) = warm(1, 80); + assert_eq!(mb.request(key, 1100, &source, Pace::PAUSED).workers_wanted, 0, "nothing planned while paused"); + let mut sim = Sim::new(vec![cue(1000, 200_000, 7)]); + sim.pace = playing(1.0, 33); + assert_eq!(sim.mb.request(sim.key, 2000, &sim.source, sim.pace).workers_wanted, 0, "a still cue plans nothing"); +} + +#[test] +fn a_promoted_frame_of_the_pipeline_says_it_was_late_only_past_a_tick() { + let pace = playing(1.0, 33); + for (after, late) in [(0, false), (16, false), (17, true), (60, true)] { + let (mut mb, source, key) = warm(1, 80); + mb.request(key, 1100, &source, pace); + let job = mb.take().unwrap(); + mb.finish(&job, Some(moving(&job, 80, END))); + let out = mb.request(key, job.now_ms + after, &source, pace); + assert!(out.promoted && out.piped, "{after} ms after its playhead"); + assert_eq!(out.late, late, "{after} ms after its playhead, one tick being 16"); + } + // a still cue's frame promoted early is the boundary one, not the pipeline's + let mut sim = Sim::new(vec![cue(0, 1000, 1), cue(1000, 2000, 2)]); + sim.request(100); + sim.drain(); + let out = sim.mb.request(sim.key, 1100, &sim.source, Pace::PAUSED); + assert!(out.promoted && !out.piped && !out.late); +} + +// The render scale (issue #1328): the pipeline draws a moving stretch smaller only while it cannot +// meet the target at full size, over the renderers the pool has, and never shows a frame less for it. +// The rig's jobs cost what the scale they are given makes them cost (`Rig::ratios`). + +use RenderScale::{Full, Half, TwoThirds}; + +/// A rig whose pool allows `renderers` renderers, all of them warm. +fn pooled(renderers: usize, cost_ms: i64) -> Rig { + let mut rig = Rig::new(renderers, cost_ms); + rig.mb.set_capacity(Some(renderers)); + rig +} + +/// Hold the playhead at `frozen` for `ms` of wall time: the player is paused. +fn pause(rig: &mut Rig, frozen: i64, ms: i64) { + rig.pace = Pace { running: false, ..rig.pace }; + let until = rig.wall + ms; + while rig.wall < until { + rig.offset = frozen - (rig.wall + rig.pace.tick_ms); + rig.tick(); + } +} + +/// The opening of the issue's file on the television: ~85 ms a frame at full size, which one renderer +/// cannot make thirty of a second at any size, two can at two thirds and three at full size. +#[test] +fn a_heavy_stretch_is_drawn_smaller_so_the_renderers_the_pool_has_reach_thirty_frames_a_second() { + // (renderers, the rung the stretch settles at, different frames shown in 5 s at least) + for (renderers, expected, frames) in [(1, Half, 110), (2, TwoThirds, 140), (3, Full, 140)] { + let mut rig = pooled(renderers, 85); + rig.run(1500); + let (from, normals) = (rig.shown.len(), rig.normals); + rig.run(5000); + let name = format!("{renderers} renderer(s)"); + assert_eq!(rig.mb.scale(), expected, "{name}: {:?}", rig.changes); + assert_eq!(rig.normals, normals, "{name}: the plan is not given up on the way"); + assert!(rig.distinct_shown(from) >= frames, "{name}: {} different frames in 5 s", rig.distinct_shown(from)); + let step = rig.mb.step_ms(rig.pace); + let ages = rig.ages(from); + assert!(ages.iter().all(|age| (0..=step + 40).contains(age)), "{name}: a step is {step} ms: {ages:?}"); + assert!(rig.changes.len() <= 1, "{name}: decided once: {:?}", rig.changes); + } +} + +#[test] +fn the_first_frame_of_a_stretch_is_full_size_and_the_pipeline_then_draws_smaller() { + let mut rig = pooled(1, 85); + rig.run(3000); + assert_eq!(rig.mb.scale(), Half); + let kinds: Vec<_> = rig.scales.iter().map(|&(kind, scale)| (matches!(kind, Kind::Pipe { .. }), scale)).collect(); + assert_eq!(kinds[0], (false, Full), "the stretch's first frame is a Normal job at full size: {kinds:?}"); + assert!(kinds[1..].iter().any(|&(pipe, scale)| pipe && scale == Half), "{kinds:?}"); + assert!( + kinds.iter().take_while(|&&(pipe, _)| !pipe).all(|&(_, scale)| scale == Full), + "what the screen needs before the pipeline plans anything is drawn at full size: {kinds:?}" + ); +} + +#[test] +fn a_cheap_moving_cue_and_a_still_one_stay_at_full_size() { + let mut rig = pooled(1, 12); + rig.run(8000); + assert_eq!(rig.mb.scale(), Full); + assert!(rig.changes.is_empty() && rig.scales.iter().all(|&(_, scale)| scale == Full), "{:?}", rig.changes); + // a still cue, however small the pool would make it: every job full size, and the frame + let mut sim = Sim::new(vec![cue(1000, 200_000, 7)]); + sim.pace = playing(1.0, 33); + sim.mb.set_capacity(Some(1)); + for tick in 0..120 { + sim.tick(2000 + tick * 16); + } + assert_eq!(sim.published().frame.scale, Full); + assert_eq!(sim.mb.scale(), Full); +} + +#[test] +fn no_scale_is_decided_before_the_pool_has_said_how_many_renderers_it_has() { + let mut rig = Rig::new(1, 85); + rig.run(3000); + assert_eq!(rig.mb.scale(), Full, "the pool has not answered: there is no evidence of what it can make"); + assert!(rig.changes.is_empty()); + rig.mb.set_capacity(Some(1)); + rig.run(200); + assert_eq!(rig.mb.scale(), Half); + assert_eq!(rig.changes.len(), 1); +} + +/// A change of scale is not a change of picture: the frame on screen stays until a frame at the new +/// scale lands, the frames already planned are still shown, and the plan is not given up. +#[test] +fn a_scale_change_keeps_the_frame_on_screen_until_the_first_one_at_the_new_scale_lands() { + let mut rig = pooled(1, 25); + rig.run(3000); + assert_eq!(rig.mb.scale(), Full, "25 ms a frame meets the target at full size"); + let (normals, from) = (rig.normals, rig.shown.len()); + rig.cost_ms = 70; + rig.run(4000); + assert_eq!(rig.changes.iter().map(|c| (c.1, c.2)).collect::>(), [(Full, Half)]); + assert_eq!(rig.normals, normals, "the plan was kept: no Normal job, no blank while one renders"); + let scales = &rig.shown_scales[from..]; + assert!(scales.iter().all(Option::is_some), "a frame was on screen at every tick of the change"); + let first_half = scales.iter().position(|&s| s == Some(Half)).expect("frames at the new scale landed"); + assert!(first_half > 0 && scales[..first_half].iter().all(|&s| s == Some(Full)), "the old frames stood until then"); + assert!(scales[first_half..].iter().all(|&s| s == Some(Half)), "and the new ones took over for good"); + assert!(rig.ages(from).iter().all(|age| *age >= 0), "and none was shown early"); +} + +/// The last frame of a reduced stretch is the still picture after it: shown from the pipeline at the +/// size it was drawn, then drawn again at full size. The reduced frame stands until then, and the +/// full-size job is not withdrawn as "covered" by it. +#[test] +fn the_still_picture_after_a_reduced_stretch_is_redrawn_at_full_size() { + let mut rig = pooled(1, 85); + rig.cue_end = 4000; + rig.run(2500); + assert_eq!(rig.mb.scale(), Half); + let (normals, from) = (rig.normals, rig.shown.len()); + rig.run(4000); + let published = rig.mb.published.as_ref().expect("a frame"); + assert!(!published.validity.animated, "the cue has ended"); + assert_eq!(published.frame.scale, Full, "the still picture is full size"); + assert_eq!(rig.normals - normals, 1, "one full-size job for it, and no more"); + assert_eq!(rig.scales.last().map(|s| s.1), Some(Full)); + assert!(rig.shown_scales[from..].iter().all(Option::is_some), "a frame was on screen all the way"); + let pipes = rig.pipes; + rig.run(1000); + assert_eq!((rig.pipes, rig.normals - normals), (pipes, 1), "and nothing follows"); +} + +#[test] +fn a_paused_player_shows_a_full_size_frame_of_a_reduced_stretch() { + let mut rig = pooled(1, 85); + rig.run(3000); + assert_eq!(rig.mb.published.as_ref().map(|p| p.frame.scale), Some(Half)); + let (normals, frozen) = (rig.normals, rig.now()); + pause(&mut rig, frozen, 1000); + let published = rig.mb.published.as_ref().expect("a frame"); + assert_eq!(published.frame.scale, Full, "a paused picture is drawn at full size"); + assert_eq!(rig.normals - normals, 1, "once"); + assert_eq!(rig.mb.scale(), Half, "the stretch's rung is kept for when it runs again"); + // played again from where it stopped: the pipeline goes on at the rung it was at + rig.pace = playing(1.0, 33); + rig.offset = frozen - (rig.wall + rig.pace.tick_ms); + rig.run(2000); + let last_pipes: Vec<_> = rig.scales.iter().rev().filter(|(kind, _)| matches!(kind, Kind::Pipe { .. })).take(5).collect(); + assert!(!last_pipes.is_empty() && last_pipes.iter().all(|(_, scale)| *scale == Half), "{last_pipes:?}"); +} + +#[test] +fn a_cost_sample_is_taken_back_to_full_size_by_the_scale_it_was_drawn_at() { + let mut rig = pooled(1, 80); + rig.run(3000); + assert_eq!(rig.mb.scale(), Half); + // the frames it timed cost 35 ms each: that is 80 ms at full size, not a cheaper stretch + let cost = rig.mb.cost_full_ms(); + assert!((cost - 80.0).abs() <= 3.0, "{cost}"); + let step = rig.mb.step_ms(rig.pace); + assert!((35..=36).contains(&step), "and the step follows the cost at the rung in use: {step}"); +} + +#[test] +fn the_pool_is_asked_about_what_full_size_would_want_and_the_plan_wants_what_its_scale_needs() { + let pace = playing(1.0, 33); + for (cost_ms, pool) in [(85, 3), (20, 1)] { + let (mut mb, source, key) = warm(1, cost_ms); + let out = mb.request(key, 1100, &source, pace); + assert_eq!(out.pool_wanted, pool, "{cost_ms} ms"); + assert_eq!(out.workers_wanted, pool, "the workers the plan wants are the same while it draws at full size"); + } + // drawing smaller, it wants fewer workers than the pool is asked for + let mut rig = pooled(1, 85); + rig.run(3000); + let out = rig.mb.request(rig.key, rig.now(), &rig.source, rig.pace); + assert_eq!((out.pool_wanted, out.workers_wanted), (3, 2)); +} + +/// The model may be wrong about what two thirds costs: the move up looks comfortable and costs more +/// than projected, and it is taken back. That happens once per half minute at most. +#[test] +fn a_model_that_is_wrong_flaps_a_bounded_number_of_times() { + let mut rig = pooled(1, 40); + rig.ratios = [1.0, 0.9, 0.44]; + rig.run(60_000); + assert!(rig.changes.len() <= 6, "{} moves in a minute: {:?}", rig.changes.len(), rig.changes); + let moves = rig.changes.iter().map(|c| c.0).collect::>(); + assert!(moves.windows(2).all(|w| w[1] - w[0] >= 500), "never twice within half a second: {moves:?}"); +} + +/// The rung belongs to the selection it was decided for: a track switched to directly (a new epoch, no +/// `clear` between) starts at full size again, whatever the track before it was drawn at, and waits for +/// the pool to say what it has. +#[test] +fn a_new_selection_starts_at_full_size_and_waits_for_the_pool() { + let mut rig = pooled(1, 85); + rig.run(3000); + assert_eq!(rig.mb.scale(), Half); + // the next selection: the runtime has not heard from the pool for it, and its frames are cheap + rig.key = Key { epoch: 2, ..rig.key }; + rig.mb.set_capacity(None); + rig.cost_ms = 12; + let (from, moves) = (rig.scales.len(), rig.changes.len()); + rig.run(3000); + assert_eq!(rig.mb.scale(), Full, "the old track's rung is not the new one's"); + assert!(rig.scales.len() > from + 5, "the new selection plans frames"); + // a worker may take one more frame of the old plan before the tick that sees the new key + let first = from + rig.scales[from..].iter().position(|(kind, _)| *kind == Kind::Normal).expect("its first frame"); + assert!(rig.scales[first..].iter().all(|&(_, scale)| scale == Full), "{:?}", &rig.scales[first..]); + assert_eq!(rig.changes.len(), moves, "no move of its own: {:?}", rig.changes); +} + +/// A hole in a reduced stretch is recovered at the rung the stretch is drawn at: a frame at full size +/// would make libass empty its caches, and the next frame at the old size would pay for them again. +#[test] +fn a_hole_in_a_reduced_stretch_is_recovered_at_its_own_size() { + let mut rig = pooled(1, 85); + rig.run(3000); + assert_eq!(rig.mb.scale(), Half); + let (from, normals) = (rig.scales.len(), rig.normals); + rig.slow = vec![700]; + rig.run(4000); + assert!(rig.normals > normals, "the stall left a hole, and a Normal job recovered it"); + let recoveries: Vec<_> = rig.scales[from..].iter().filter(|(kind, _)| *kind == Kind::Normal).collect(); + assert!(recoveries.iter().all(|(_, scale)| *scale == Half), "{recoveries:?}"); + assert_eq!(rig.mb.scale(), Half); +} diff --git a/rust-modules/media/src/player/ass/pool.rs b/rust-modules/media/src/player/ass/pool.rs new file mode 100644 index 0000000000..f008161c05 --- /dev/null +++ b/rust-modules/media/src/player/ass/pool.rs @@ -0,0 +1,121 @@ +//! How many renderers the memory can hold (issue #1328). Pure: numbers in, a verdict out. +//! +//! Every renderer is a whole libass instance with its own copy of the bundled faces (`ass_add_font` +//! copies the bytes) and its own bitmap cache, so a helper costs what the primary does. The attached +//! fonts the track names are not in that cost: every renderer references the store's one buffer +//! (`plx_ass_add_font_borrowed`), so a file whose karaoke styles name five display faces puts those +//! tens of megabytes in the process once, not once per renderer. A television still has little to +//! spare; a helper that starts where the memory is not there trades a late subtitle frame for the +//! process. The runner (`Runtime::renderers_for`) reads `MemAvailable` and the primary's font bytes +//! and asks here once per selection; nothing here reads a file or a clock. +//! +//! The number is a bound on helpers only: the primary always runs, so no subtitle is ever hidden by +//! it, a pipeline merely has fewer workers to share its frames over. And the budget is spent on +//! helpers only: the verdict is asked once the primary is open, so what the primary holds is already +//! gone from `MemAvailable`. Charging it again counted it twice (TV, #1328: 164 MiB available, a +//! 41 MiB budget and 28 MiB a renderer answered one renderer where a helper fit). + +/// The helpers may take this fraction (`1 / MEMORY_SHARE_DIVISOR`) of the memory available, +/// together: the rest is the video pipeline's, the UI's and the headroom a spike needs. A share, +/// not a number of megabytes, so a television with little free gets fewer helpers and one with +/// plenty gets them all; the event log line says what each verdict was made of. +pub(super) const MEMORY_SHARE_DIVISOR: u64 = 4; + +/// What the pool may hold, and why. +#[derive(Clone, Copy, Debug, PartialEq, Eq)] +pub(super) struct Verdict { + /// How many renderers, the primary included: at least 1, at most the ceiling asked for. + pub renderers: usize, + /// What one renderer holds that another would not share: the fonts it copies and its bitmap cache. + pub per_renderer: u64, + /// What the helpers may take together; `None` when the memory available is not known. + pub budget: Option, +} + +/// What one renderer holds beyond its fonts: libass's bitmap cache, which grows to its cap +/// (`CACHE_BITMAP_MIB`) on a busy moving cue. The glyph outlines, FreeType's face state and the +/// parsed events are not counted: a floor on a renderer, not a ceiling. +fn cache_bytes() -> u64 { + u64::from(super::CACHE_BITMAP_MIB.unsigned_abs()) << 20 +} + +/// How many renderers may run: `font_bytes` is what one copies in fonts (what the primary reports), +/// `mem_available` what the device has free with the primary already open (`None` when it could not +/// be read: nothing to gate on, the cores bound it), `ceiling` the most the cores and the pool's own +/// cap allow. The primary always runs and is not charged to the budget, so the answer is 1 plus the +/// helpers the budget holds, never below 1. +pub(super) fn decide(font_bytes: u64, mem_available: Option, ceiling: usize) -> Verdict { + let per_renderer = font_bytes.saturating_add(cache_bytes()); + let ceiling = ceiling.max(1); + let budget = mem_available.map(|available| available / MEMORY_SHARE_DIVISOR); + let fits = budget.map_or(ceiling as u64, |budget| 1 + budget / per_renderer); + let renderers = usize::try_from(fits).unwrap_or(usize::MAX).clamp(1, ceiling); + Verdict { renderers, per_renderer, budget } +} + +/// `MemAvailable` in bytes from the text of `/proc/meminfo` (the kernel's own estimate of what a +/// new allocation can take without swapping; absent before Linux 3.14). +pub(super) fn mem_available_bytes(meminfo: &str) -> Option { + let line = meminfo.lines().find_map(|line| line.strip_prefix("MemAvailable:"))?; + let kib: u64 = line.trim().strip_suffix("kB")?.trim().parse().ok()?; + kib.checked_mul(1024) +} + +#[cfg(test)] +mod tests { + use super::*; + + const MIB: u64 = 1 << 20; + + #[test] + fn mem_available_is_read_from_the_meminfo_line_in_bytes() { + let meminfo = "MemTotal: 1892864 kB\nMemFree: 102400 kB\nMemAvailable: 612345 kB\nBuffers: 1 kB\n"; + assert_eq!(mem_available_bytes(meminfo), Some(612_345 * 1024)); + assert_eq!(mem_available_bytes("MemTotal: 5 kB\nMemFree: 4 kB\n"), None, "an old kernel has no such line"); + assert_eq!(mem_available_bytes("MemAvailable: lots kB\n"), None); + assert_eq!(mem_available_bytes(""), None); + } + + #[test] + fn a_renderer_costs_its_fonts_and_its_bitmap_cache() { + let verdict = decide(10 * MIB, Some(4096 * MIB), 3); + assert_eq!(verdict.per_renderer, 10 * MIB + u64::from(super::super::CACHE_BITMAP_MIB.unsigned_abs()) * MIB); + } + + #[test] + fn light_fonts_get_every_renderer_the_device_has_cores_for() { + let verdict = decide(2 * MIB, Some(800 * MIB), 3); + assert_eq!(verdict.renderers, 3, "a quarter of 800 MiB holds seven helpers of 26 MiB"); + assert_eq!(decide(2 * MIB, Some(800 * MIB), 2).renderers, 2, "and never more than the cores allow"); + } + + #[test] + fn heavy_fonts_leave_the_pipeline_one_renderer_where_memory_is_short() { + // the fixture: the opening's five display faces and the bundled ones, ~40 MiB in each renderer + let verdict = decide(40 * MIB, Some(200 * MIB), 3); + assert_eq!(verdict.renderers, 1, "a quarter of 200 MiB is 50 MiB: no helper of 64 MiB"); + assert_eq!(decide(40 * MIB, Some(800 * MIB), 3).renderers, 3, "the same fonts fit where there is memory"); + assert_eq!(decide(40 * MIB, Some(400 * MIB), 3).renderers, 2, "100 MiB holds one helper of 64"); + } + + #[test] + fn the_primary_always_runs_and_an_unknown_memory_does_not_stop_a_helper() { + assert_eq!(decide(500 * MIB, Some(64 * MIB), 3).renderers, 1, "never below the primary"); + assert_eq!(decide(500 * MIB, Some(0), 3).renderers, 1); + let unknown = decide(40 * MIB, None, 3); + assert_eq!((unknown.renderers, unknown.budget), (3, None), "no number, no gate: the cores bound it"); + assert_eq!(decide(2 * MIB, Some(800 * MIB), 0).renderers, 1, "a ceiling of nothing is still the primary"); + } + + #[test] + fn the_primary_already_open_is_not_charged_to_the_budget() { + // measured on the TV (#1328): 164 MiB available with the primary open, 28 MiB a renderer + let verdict = decide(3 * MIB, Some(157 * MIB), 3); + assert_eq!(verdict.renderers, 2, "a 39 MiB budget holds one helper of 27 MiB beside the primary"); + } + + #[test] + fn the_budget_is_a_quarter_of_what_is_available() { + assert_eq!(decide(MIB, Some(400 * MIB), 3).budget, Some(100 * MIB)); + } +} diff --git a/rust-modules/media/src/player/ass/scale.rs b/rust-modules/media/src/player/ass/scale.rs new file mode 100644 index 0000000000..16563350e9 --- /dev/null +++ b/rust-modules/media/src/player/ass/scale.rs @@ -0,0 +1,253 @@ +//! The render scale of a moving cue's frames (issue #1328): the size libass is asked to draw at, +//! as a fraction of the output size, and the pure decision that picks it. +//! +//! A moving cue's frames cost what libass draws in them, which is the number of pixels as much as +//! anything: the opening of the issue's file costs ~85 ms a frame at 1080p on the television, so +//! one renderer makes ~12 a second against the ~30 the cue deserves, and more renderers are limited +//! by the set's cores and memory. Drawn at half the size the same frames cost ~0.45 as much +//! (measured on the host with the storage size fixed: libass 0.64x at 720p and 0.49x at 540p, the +//! composite 0.49x and 0.30x, bytes published 0.54x and 0.37x). So a stretch is drawn smaller only +//! while it cannot meet the target at full size, and every other picture (a still cue, a paused +//! player, the first frame of a stretch) stays at full size. +//! +//! The choice is a [`Governor`] with hysteresis: a stretch's first measured frame decides its rung +//! at once (what the last stretch cost says nothing of this one); inside a stretch it steps down +//! when the projection at the current rung misses the target for [`DOWN_HOLD_MS`], and back up one +//! rung only when the larger rung would comfortably meet it ([`COMFORT`]) for [`UP_HOLD_MS`], with a +//! bar after a step up that had to be taken back. +//! +//! Pure: no clock, no thread and no native call. The measurements arrive as a [`Load`] and the +//! playhead as a value; [`Mailbox`](super::mailbox::Mailbox) owns the [`Governor`] and the runtime +//! logs what it reports. + +/// What a frame drawn at two thirds of each side costs over one at full size: the host's +/// measurement weighted by the television's split of an 85 ms frame (libass 65 ms, composite 20 ms) +/// is 0.605, and the constant is rounded DOWN so that a cost taken back to full size is never +/// overstated and a step up never projects better than it should. +const COST_RATIO_TWO_THIRDS: f64 = 0.60; +/// The same at half of each side: 0.445 measured, rounded down. +const COST_RATIO_HALF: f64 = 0.44; +/// A larger rung is worth going back to only when its projected step is under this share of the +/// target: the band between it and the target is where a rung stays put. +const COMFORT: f64 = 0.75; +/// How long a projection that misses the target has to last, in playhead milliseconds at normal +/// speed, before the rung steps down. +const DOWN_HOLD_MS: i64 = 600; +/// How long a comfortable projection has to last before the rung steps up one. +const UP_HOLD_MS: i64 = 3_000; +/// A hold is only counted over frames of the rung's own: this many, drawn since the last move, +/// have to have been timed first. +const SETTLE_SAMPLES: u32 = 6; +/// A step down within this long of a step up means the projection was too kind; the next step up +/// is barred for [`FLIP_BAR_MS`], so a model that is wrong costs one flap per half minute at most. +const FLIP_WINDOW_MS: i64 = 10_000; +const FLIP_BAR_MS: i64 = 30_000; + +/// A frame size as a fraction of the output size. libass is told the smaller frame size but the +/// SAME storage size, so the script lays out identically and only the number of pixels it draws +/// changes; the screen draws the regions back up to the output rectangle with linear filtering. +#[derive(Clone, Copy, Debug, Default, PartialEq, Eq, Hash)] +pub enum RenderScale { + /// The output size: the picture a still cue, a paused player and the first frame of a moving + /// cue are drawn at. + #[default] + Full, + /// Two thirds of each side (720p of a 1080p output). + TwoThirds, + /// Half of each side (540p of a 1080p output). + Half, +} + +impl RenderScale { + /// The rungs, the largest first. + pub const LADDER: [RenderScale; 3] = [RenderScale::Full, RenderScale::TwoThirds, RenderScale::Half]; + + /// The side of the frame over the side of the output. + pub fn factor(self) -> f64 { + match self { + RenderScale::Full => 1.0, + RenderScale::TwoThirds => 2.0 / 3.0, + RenderScale::Half => 0.5, + } + } + + /// The frame size libass is asked for when the output is `width` x `height`: each side scaled + /// and rounded, never under one pixel. `Full` is the output size itself. + pub fn render_size(self, width: i32, height: i32) -> (i32, i32) { + let side = |n: i32| ((f64::from(n) * self.factor()).round() as i32).max(1); + (side(width), side(height)) + } + + /// What a frame costs at this size over what it costs at full size. + pub fn cost_ratio(self) -> f64 { + match self { + RenderScale::Full => 1.0, + RenderScale::TwoThirds => COST_RATIO_TWO_THIRDS, + RenderScale::Half => COST_RATIO_HALF, + } + } + + /// The next rung up, if there is one. + fn larger(self) -> Option { + match self { + RenderScale::Full => None, + RenderScale::TwoThirds => Some(RenderScale::Full), + RenderScale::Half => Some(RenderScale::TwoThirds), + } + } + + /// The number the event log prints: the side's factor to two places. + pub fn label(self) -> &'static str { + match self { + RenderScale::Full => "1.00", + RenderScale::TwoThirds => "0.67", + RenderScale::Half => "0.50", + } + } +} + +/// What the decision is made from. +#[derive(Clone, Copy, Debug, PartialEq)] +pub struct Load { + /// What a frame costs at full size, in milliseconds: the mean over the moving frames the + /// workers made, each taken back to full size by its own rung's cost ratio. + pub cost_full_ms: f64, + /// How many renderers the pool allows the selection (the primary included). + pub renderers: usize, + /// Playhead milliseconds per wall-clock millisecond. + pub rate: f64, + /// The step the pipeline aims at, in playhead milliseconds: about thirty a second, or one video + /// frame, whichever is longer. + pub target_ms: i64, +} + +impl Load { + /// The frame step the plan settles at drawing at `scale`: the cost over the renderers, in + /// playhead milliseconds. + pub fn step_at(&self, scale: RenderScale) -> f64 { + self.cost_full_ms * scale.cost_ratio() * self.rate / self.renderers.max(1) as f64 + } + + /// The largest rung whose projection meets the target, else the smallest there is. + fn fitting(&self) -> RenderScale { + let target = self.target_ms as f64; + RenderScale::LADDER.into_iter().find(|&rung| self.step_at(rung) <= target).unwrap_or(RenderScale::Half) + } + + /// Where a stretch drawn at `current` should be: down to what fits when it misses the target, + /// one rung up when the larger one would comfortably meet it, else where it is. + fn wanted(&self, current: RenderScale) -> RenderScale { + let target = self.target_ms as f64; + if self.step_at(current) > target { + return self.fitting(); + } + match current.larger() { + Some(larger) if self.step_at(larger) <= COMFORT * target => larger, + _ => current, + } + } +} + +/// A move from one rung to another. +#[derive(Clone, Copy, Debug, PartialEq)] +pub struct Change { + pub from: RenderScale, + pub to: RenderScale, + /// The numbers it was decided from, for the log. + pub load: Load, +} + +/// The scale a moving stretch is drawn at, with the hysteresis that keeps it from flapping. +#[derive(Clone, Copy, Debug, Default)] +pub struct Governor { + scale: RenderScale, + /// Frames of the current rung timed since the last move or the start of the stretch. + samples: u32, + /// A stretch has begun and its first sample has not been judged. + fresh: bool, + /// The rung a projection has been asking for, and the playhead it first did. + pending: Option<(RenderScale, i64)>, + /// The playhead of the last step up, for [`FLIP_WINDOW_MS`]. + last_up_ms: Option, + /// No step up before this playhead. + barred_until_ms: i64, +} + +impl Governor { + /// At full size, with the first sample deciding: nothing has been drawn yet. + pub const fn new() -> Self { + Self { scale: RenderScale::Full, samples: 0, fresh: true, pending: None, last_up_ms: None, barred_until_ms: 0 } + } + + /// The rung moving frames are drawn at now. + pub fn scale(&self) -> RenderScale { + self.scale + } + + /// A moving stretch begins after a still picture (or on a new timeline): what the last one + /// cost says nothing of this one, so its first sample replaces the cost and decides at once. + pub fn begin_stretch(&mut self) { + self.fresh = true; + self.samples = 0; + self.pending = None; + } + + /// The next sample is the first of a stretch: it replaces what was known of the cost. + pub fn is_fresh(&self) -> bool { + self.fresh && self.samples == 0 + } + + /// A steady frame of the current rung was timed. + pub fn note_sample(&mut self) { + self.samples = self.samples.saturating_add(1); + } + + /// One UI tick of a running pipeline at playhead `now_ms`: the move to make, if any. Nothing is + /// decided before a sample has been timed (the cost is a default until then). + pub fn observe(&mut self, now_ms: i64, load: Load) -> Option { + if self.samples == 0 { + return None; + } + if self.fresh { + self.fresh = false; + let rung = load.fitting(); + return (rung != self.scale).then(|| self.apply(now_ms, load, rung)); + } + if self.samples < SETTLE_SAMPLES { + return None; + } + let wanted = load.wanted(self.scale); + if wanted == self.scale { + self.pending = None; + return None; + } + let up = wanted.factor() > self.scale.factor(); + if up && now_ms < self.barred_until_ms { + self.pending = None; + return None; + } + let since = match self.pending { + Some((rung, since)) if rung == wanted && now_ms >= since => since, + _ => { + self.pending = Some((wanted, now_ms)); + now_ms + } + }; + let hold = if up { UP_HOLD_MS } else { DOWN_HOLD_MS }; + let hold = (hold as f64 * load.rate).round() as i64; + (now_ms - since >= hold).then(|| self.apply(now_ms, load, wanted)) + } + + fn apply(&mut self, now_ms: i64, load: Load, to: RenderScale) -> Change { + let from = self.scale; + if to.factor() > from.factor() { + self.last_up_ms = Some(now_ms); + } else if self.last_up_ms.is_some_and(|up| now_ms - up < FLIP_WINDOW_MS) { + self.barred_until_ms = now_ms + FLIP_BAR_MS; + } + self.scale = to; + self.samples = 0; + self.pending = None; + Change { from, to, load } + } +} diff --git a/rust-modules/media/src/player/ass/scale_tests.rs b/rust-modules/media/src/player/ass/scale_tests.rs new file mode 100644 index 0000000000..b5b93afe81 --- /dev/null +++ b/rust-modules/media/src/player/ass/scale_tests.rs @@ -0,0 +1,270 @@ +//! The render-scale decision (issue #1328) with no renderer: the governor is fed the load a +//! moving stretch measures, one UI tick at a time, and the moves it makes are read back. +use super::scale::{Change, Governor, Load, RenderScale}; +use super::scale::RenderScale::{Full, Half, TwoThirds}; + +fn load(cost_full_ms: f64, renderers: usize) -> Load { + Load { cost_full_ms, renderers, rate: 1.0, target_ms: 33 } +} + +/// A governor over a running pipeline: a UI tick every 16 ms and a cost sample every other tick (a +/// frame every 32 ms, which is what a pipeline at the target rate makes). +struct Run { + gov: Governor, + now: i64, + ticks: u32, + /// `(playhead of the tick, from, to)` for every move. + moves: Vec<(i64, RenderScale, RenderScale)>, +} + +impl Run { + /// A stretch that has begun and has had its first sample judged at `first`. + fn begun(first: Load) -> Self { + let mut run = Run { gov: Governor::default(), now: 10_000, ticks: 0, moves: Vec::new() }; + run.gov.begin_stretch(); + run.gov.note_sample(); + run.record(first); + run + } + + /// A stretch past its first sample, still at full size: what the following ticks decide is + /// decided by a hold, not at once. + fn settled() -> Self { + Run::begun(load(1.0, 1)) + } + + fn record(&mut self, load: Load) { + if let Some(Change { from, to, .. }) = self.gov.observe(self.now, load) { + self.moves.push((self.now, from, to)); + } + } + + fn tick(&mut self, load: Load) { + self.now += 16; + self.ticks += 1; + if self.ticks % 2 == 0 { + self.gov.note_sample(); + } + self.record(load); + } + + fn run(&mut self, ms: i64, load: Load) { + let until = self.now + ms; + while self.now < until { + self.tick(load); + } + } + + fn scale(&self) -> RenderScale { + self.gov.scale() + } +} + +#[test] +fn the_rungs_are_fractions_of_the_output_size() { + assert_eq!(RenderScale::LADDER, [Full, TwoThirds, Half]); + assert_eq!(Full.render_size(1920, 1080), (1920, 1080)); + assert_eq!(TwoThirds.render_size(1920, 1080), (1280, 720)); + assert_eq!(Half.render_size(1920, 1080), (960, 540)); + // a letterboxed authored viewport keeps its shape to a pixel + assert_eq!(TwoThirds.render_size(1920, 800), (1280, 533)); + assert_eq!(Half.render_size(1920, 800), (960, 400)); + assert_eq!(Half.render_size(1, 1), (1, 1), "never an empty frame"); + assert_eq!((Full.factor(), TwoThirds.factor(), Half.factor()), (1.0, 2.0 / 3.0, 0.5)); + assert_eq!([Full.label(), TwoThirds.label(), Half.label()], ["1.00", "0.67", "0.50"]); + assert_eq!(RenderScale::default(), Full); +} + +/// The ratios are the measured ones: libass 0.64 and 0.49, the composite 0.49 and 0.30 (host, same +/// frames, frame size changed with the storage size fixed), weighted by the television's 65 ms and +/// 20 ms of an 85 ms frame, and rounded DOWN so a cost taken back to full size is never overstated. +#[test] +fn a_smaller_frame_costs_less_by_the_measured_ratios() { + assert_eq!(Full.cost_ratio(), 1.0); + assert_eq!(TwoThirds.cost_ratio(), 0.60); + assert_eq!(Half.cost_ratio(), 0.44); + let (lib, comp) = ((0.64, 0.49), (0.49, 0.30)); + let weighted = |l: f64, c: f64| (65.0 * l + 20.0 * c) / 85.0; + assert!(TwoThirds.cost_ratio() <= weighted(lib.0, comp.0), "{}", weighted(lib.0, comp.0)); + assert!(Half.cost_ratio() <= weighted(lib.1, comp.1), "{}", weighted(lib.1, comp.1)); +} + +#[test] +fn a_cheap_animation_stays_at_full_size() { + let mut run = Run::begun(load(12.0, 1)); + run.run(60_000, load(12.0, 1)); + assert_eq!(run.scale(), Full); + assert!(run.moves.is_empty(), "{:?}", run.moves); + // even a frame at the edge of the target (33 ms of cost for a 33 ms step) is no reason to shrink + let mut run = Run::begun(load(33.0, 1)); + run.run(30_000, load(33.0, 1)); + assert!(run.moves.is_empty(), "{:?}", run.moves); +} + +#[test] +fn the_first_sample_of_a_heavy_stretch_decides_at_once() { + // the opening of the issue's file: 85 ms a frame, one renderer + let run = Run::begun(load(85.0, 1)); + assert_eq!(run.moves, [(10_000, Full, Half)]); + assert_eq!(run.scale(), Half); + // a cost that two thirds is enough for does not go further down + let run = Run::begun(load(50.0, 1)); + assert_eq!(run.moves, [(10_000, Full, TwoThirds)]); +} + +#[test] +fn nothing_is_decided_without_a_sample() { + let mut gov = Governor::default(); + gov.begin_stretch(); + assert!(gov.is_fresh()); + assert_eq!(gov.observe(10_000, load(500.0, 1)), None, "no sample, no evidence: the cost is a default"); + assert_eq!(gov.scale(), Full); + gov.note_sample(); + assert_eq!(gov.observe(10_016, load(500.0, 1)).map(|c| c.to), Some(Half)); + assert!(!gov.is_fresh(), "the first sample has been judged"); +} + +#[test] +fn a_sustained_cost_steps_down_after_its_hold_and_not_before() { + let mut run = Run::settled(); + let t0 = run.now; + run.run(550, load(85.0, 1)); + assert_eq!(run.scale(), Full, "a cost that has lasted half a second is not yet sustained"); + run.run(300, load(85.0, 1)); + assert_eq!(run.moves.len(), 1, "{:?}", run.moves); + let (at, from, to) = run.moves[0]; + assert_eq!((from, to), (Full, Half)); + assert!((600..=850).contains(&(at - t0)), "stepped down {} ms into it", at - t0); + run.run(10_000, load(85.0, 1)); + assert_eq!(run.moves.len(), 1, "and it stays down: nothing is smaller"); +} + +#[test] +fn it_steps_to_two_thirds_and_then_to_half_as_the_cost_grows() { + let mut run = Run::settled(); + run.run(2_000, load(50.0, 1)); + assert_eq!(run.moves.iter().map(|m| (m.1, m.2)).collect::>(), [(Full, TwoThirds)]); + // the cost grows (a denser stretch): two thirds no longer meets the target either + run.run(3_000, load(80.0, 1)); + assert_eq!(run.moves.iter().map(|m| (m.1, m.2)).collect::>(), [(Full, TwoThirds), (TwoThirds, Half)]); + assert_eq!(run.scale(), Half); +} + +#[test] +fn a_spike_shorter_than_the_hold_changes_nothing_however_often_it_comes() { + let mut run = Run::settled(); + run.run(400, load(85.0, 1)); + run.run(2_000, load(12.0, 1)); + assert!(run.moves.is_empty(), "{:?}", run.moves); + // one heavy sample in a few: the verdict is reset by every sample that does not agree + for _ in 0..200 { + run.run(300, load(85.0, 1)); + run.run(300, load(20.0, 1)); + } + assert!(run.moves.is_empty(), "{:?}", run.moves); +} + +#[test] +fn it_does_not_flap_at_the_edge() { + // a cost just over the target at full size: it steps down once and stays there for a minute + let mut run = Run::settled(); + run.run(60_000, load(34.0, 1)); + assert_eq!(run.moves.len(), 1, "{:?}", run.moves); + assert_eq!((run.moves[0].1, run.moves[0].2), (Full, TwoThirds)); + // a cost that wavers across the target is judged sample by sample and never held + let mut run = Run::settled(); + for _ in 0..1000 { + run.tick(load(30.0, 1)); + run.tick(load(36.0, 1)); + } + assert!(run.moves.is_empty(), "{:?}", run.moves); +} + +#[test] +fn it_goes_back_up_only_when_the_larger_size_comfortably_meets_the_target() { + let mut run = Run::begun(load(85.0, 1)); + assert_eq!(run.scale(), Half); + // 28 ms at full size meets the 33 ms target but not comfortably (under three quarters of it); the + // rung below it, at 17 ms, does, so a cost that settles there goes up one rung and no further + run.run(60_000, load(28.0, 1)); + assert_eq!(run.moves.iter().map(|m| (m.1, m.2)).collect::>(), [(Full, Half), (Half, TwoThirds)]); + assert_eq!(run.scale(), TwoThirds); +} + +#[test] +fn it_returns_to_full_size_a_rung_at_a_time_once_the_cost_is_low() { + let mut run = Run::begun(load(85.0, 1)); + assert_eq!(run.scale(), Half); + let t0 = run.now; + run.run(2_500, load(20.0, 1)); + assert_eq!(run.scale(), Half, "the cost has not held for three seconds yet"); + run.run(1_000, load(20.0, 1)); + assert_eq!(run.scale(), TwoThirds, "one rung at a time"); + let (at, ..) = run.moves[1]; + assert!((3_000..=3_400).contains(&(at - t0)), "stepped up {} ms in", at - t0); + run.run(4_000, load(20.0, 1)); + assert_eq!(run.scale(), Full, "and the next rung after another hold"); + run.run(30_000, load(20.0, 1)); + assert_eq!(run.moves.len(), 3, "and then it stays: {:?}", run.moves); +} + +#[test] +fn a_failed_step_up_bars_the_next_one_for_half_a_minute() { + let mut run = Run::begun(load(85.0, 1)); + run.run(4_000, load(20.0, 1)); + assert_eq!(run.scale(), TwoThirds); + // two thirds turns out to cost more than the projection said: back down at once + run.run(1_000, load(60.0, 1)); + assert_eq!(run.scale(), Half); + let down = run.now; + // the cost is comfortable again: no step up for the half minute that follows + run.run(25_000, load(20.0, 1)); + assert_eq!(run.scale(), Half, "{:?}", run.moves); + run.run(10_000, load(20.0, 1)); + assert_eq!(run.scale(), TwoThirds); + assert!(run.moves.last().unwrap().0 - down >= 30_000, "{:?}", run.moves); +} + +#[test] +fn the_renderer_count_the_pool_has_decides_what_meets_the_target() { + // 85 ms a frame: one renderer cannot make 30 a second at any size, two can at two thirds, three + // at full size + for (renderers, expected) in [(1, Some(Half)), (2, Some(TwoThirds)), (3, None)] { + let run = Run::begun(load(85.0, renderers)); + assert_eq!(run.moves.first().map(|m| m.2), expected, "{renderers} renderer(s)"); + } + // a pool of none is a pool of one + let run = Run::begun(load(85.0, 0)); + assert_eq!(run.scale(), Half); +} + +#[test] +fn a_faster_clock_makes_the_same_cost_a_longer_step() { + let fast = Load { rate: 2.0, ..load(20.0, 1) }; + assert!(Run::begun(load(20.0, 1)).moves.is_empty()); + assert_eq!(Run::begun(fast).scale(), TwoThirds, "20 ms of work spans 40 ms of playhead at twice the speed"); +} + +#[test] +fn the_target_is_a_video_frame_for_slow_content() { + let film = Load { target_ms: 42, ..load(40.0, 1) }; + assert!(Run::begun(film).moves.is_empty(), "40 ms makes a 24 fps frame"); + assert_eq!(Run::begun(load(40.0, 1)).scale(), TwoThirds, "but not a 33 ms step"); +} + +#[test] +fn a_move_reports_the_numbers_it_was_made_from() { + let mut gov = Governor::default(); + gov.begin_stretch(); + gov.note_sample(); + let heavy = load(85.0, 1); + assert_eq!(gov.observe(5_000, heavy), Some(Change { from: Full, to: Half, load: heavy })); +} + +#[test] +fn a_projection_is_the_cost_at_that_size_over_the_renderers() { + let l = Load { rate: 1.5, ..load(80.0, 2) }; + assert_eq!(l.step_at(Full), 60.0); + assert!((l.step_at(TwoThirds) - 36.0).abs() < 1e-9); + assert!((l.step_at(Half) - 26.4).abs() < 1e-9); +} diff --git a/rust-modules/media/src/player/ass/synth_fonts.rs b/rust-modules/media/src/player/ass/synth_fonts.rs new file mode 100644 index 0000000000..75fe721bad --- /dev/null +++ b/rust-modules/media/src/player/ass/synth_fonts.rs @@ -0,0 +1,180 @@ +//! Test-only font synthesis: a real shipped face (`pkg/appfont.ttf`) given another name, padded to +//! any size, or a name table written record by record, so a test can prove that a face was selected +//! by its name (it draws differently from the fallback) without committing a font. A Rust port of +//! `tests/fixtures/make_ass_stress.py::rename_font` (`sfnt_build`, `renamed_name_table`), plus the +//! shapes that script never needs: a Mac-only or family-less name table, a TrueType Collection and +//! an OpenType/CFF face. + +use std::collections::BTreeMap; + +pub(super) type Tag = [u8; 4]; +pub(super) type Tables = BTreeMap>; + +/// One name record: (platform, encoding, name id, raw bytes). The language is always English. +pub(super) type NameRecord = (u16, u16, u16, Vec); + +fn u16_at(bytes: &[u8], at: usize) -> usize { + u16::from_be_bytes(bytes[at..at + 2].try_into().unwrap()) as usize +} + +fn u32_at(bytes: &[u8], at: usize) -> usize { + u32::from_be_bytes(bytes[at..at + 4].try_into().unwrap()) as usize +} + +/// The version tag and every table of a single-face font file. +pub(super) fn parse(data: &[u8]) -> (Tag, Tables) { + let tables = (0..u16_at(data, 4)) + .map(|i| { + let record = 12 + 16 * i; + let offset = u32_at(data, record + 8); + let length = u32_at(data, record + 12); + (data[record..record + 4].try_into().unwrap(), data[offset..offset + length].to_vec()) + }) + .collect(); + (data[..4].try_into().unwrap(), tables) +} + +fn sum(bytes: &[u8]) -> u32 { + bytes.chunks(4).fold(0u32, |sum, chunk| { + let mut word = [0; 4]; + word[..chunk.len()].copy_from_slice(chunk); + sum.wrapping_add(u32::from_be_bytes(word)) + }) +} + +/// A font file for `tables`, its tables placed from absolute offset `base` (non-zero inside a +/// collection): sorted directory, 4-byte aligned tables, real checksums. +pub(super) fn build_at(version: Tag, tables: &Tables, base: usize) -> Vec { + let count = tables.len(); + let entry = usize::BITS as usize - 1 - count.leading_zeros() as usize; + let search = 16 << entry; + let mut directory = Vec::new(); + let mut body: Vec = Vec::new(); + let mut head_at = None; + for (tag, table) in tables { + let mut table = table.clone(); + if tag == b"head" { + head_at = Some(base + 12 + 16 * count + body.len() + 8); + table[8..12].fill(0); // checkSumAdjustment is summed as zero + } + directory.extend_from_slice(tag); + directory.extend_from_slice(&sum(&table).to_be_bytes()); + directory.extend_from_slice(&((base + 12 + 16 * count + body.len()) as u32).to_be_bytes()); + directory.extend_from_slice(&(table.len() as u32).to_be_bytes()); + body.extend_from_slice(&table); + body.resize(body.len().next_multiple_of(4), 0); + } + let mut font = version.to_vec(); + for field in [count, search, entry, 16 * count - search] { + font.extend_from_slice(&(field as u16).to_be_bytes()); + } + font.extend_from_slice(&directory); + font.extend_from_slice(&body); + if let Some(at) = head_at { + let at = at - base; + let adjustment = 0xB1B0_AFBAu32.wrapping_sub(sum(&font)); + font[at..at + 4].copy_from_slice(&adjustment.to_be_bytes()); + } + font +} + +pub(super) fn build(version: Tag, tables: &Tables) -> Vec { + build_at(version, tables, 0) +} + +/// A format-0 `name` table of `records`. +pub(super) fn name_table(records: &[NameRecord]) -> Vec { + let mut sorted: Vec<&NameRecord> = records.iter().collect(); + sorted.sort_by_key(|(platform, encoding, id, _)| (*platform, *encoding, *id)); + let mut head = Vec::new(); + let mut store: Vec = Vec::new(); + for (platform, encoding, id, raw) in sorted { + for field in [*platform, *encoding, 0x409, *id, raw.len() as u16, store.len() as u16] { + head.extend_from_slice(&field.to_be_bytes()); + } + store.extend_from_slice(raw); + } + let mut table = Vec::new(); + for field in [0u16, (head.len() / 12) as u16, (6 + head.len()) as u16] { + table.extend_from_slice(&field.to_be_bytes()); + } + table.extend_from_slice(&head); + table.extend_from_slice(&store); + table +} + +pub(super) fn utf16(text: &str) -> Vec { + text.encode_utf16().flat_map(u16::to_be_bytes).collect() +} + +/// The Windows records of a face named `family`, `style`: family, subfamily, unique, full and +/// PostScript names (ids 1, 2, 3, 4, 6). +pub(super) fn windows_names(family: &str, style: &str) -> Vec { + let full = if style == "Regular" { family.to_string() } else { format!("{family} {style}") }; + let postscript = format!("{}-{style}", family.replace(' ', "")); + [(1, family), (2, style), (3, "plx-ass-test"), (4, &full), (6, &postscript)] + .into_iter() + .map(|(id, text)| (3, 1, id, utf16(text))) + .collect() +} + +/// `data`, a single-face font, renamed to `family`, `style`; with `pad_to` the file is exactly that +/// many bytes, the difference a private `PLXP` table of unread bytes (legal sfnt, which FreeType +/// skips). The glyph tables are untouched, so the face draws as `data` does. +pub(crate) fn rename(data: &[u8], family: &str, style: &str, pad_to: Option) -> Vec { + let (version, mut tables) = parse(data); + tables.insert(*b"name", name_table(&windows_names(family, style))); + let font = build(version, &tables); + let Some(pad_to) = pad_to else { return font }; + assert!(pad_to % 4 == 0 && pad_to > font.len() + 16, "{family}: base is larger than {pad_to}"); + let mut state = family.bytes().fold(0x9E37_79B9_7F4A_7C15u64, |s, b| (s ^ u64::from(b)).wrapping_mul(0x100_0000_01B3)); + let mut pad: Vec = Vec::with_capacity(pad_to); + while pad.len() < pad_to - font.len() - 16 { + state ^= state << 13; + state ^= state >> 7; + state ^= state << 17; + let take = (pad_to - font.len() - 16 - pad.len()).min(8); + pad.extend_from_slice(&state.to_le_bytes()[..take]); + } + tables.insert(*b"PLXP", pad); + let padded = build(version, &tables); + assert_eq!(padded.len(), pad_to); + padded +} + +/// A TrueType Collection of single-face fonts: each one's tables are placed after the shared +/// `ttcf` header, and the header lists their offset tables. +pub(super) fn collection(faces: &[Vec]) -> Vec { + let header = 12 + 4 * faces.len(); + let mut out = Vec::new(); + out.extend_from_slice(b"ttcf"); + out.extend_from_slice(&0x0001_0000u32.to_be_bytes()); + out.extend_from_slice(&(faces.len() as u32).to_be_bytes()); + let mut body: Vec = Vec::new(); + let mut offsets = Vec::new(); + for face in faces { + let (version, tables) = parse(face); + let at = header + body.len(); + offsets.push(at as u32); + body.extend_from_slice(&build_at(version, &tables, at)); + } + for offset in offsets { + out.extend_from_slice(&offset.to_be_bytes()); + } + out.extend_from_slice(&body); + out +} + +/// An OpenType/CFF face with only the two tables a name read needs: `name` from `records` and a +/// `CFF ` table whose Name INDEX holds `cff_name`. +pub(super) fn cff_font(records: &[NameRecord], cff_name: &str) -> Vec { + let mut cff = vec![1, 0, 4, 1]; // major, minor, hdrSize, offSize + cff.extend_from_slice(&1u16.to_be_bytes()); // one name + cff.push(1); // offSize + cff.extend_from_slice(&[1, 1 + cff_name.len() as u8]); + cff.extend_from_slice(cff_name.as_bytes()); + let mut tables = Tables::new(); + tables.insert(*b"name", name_table(records)); + tables.insert(*b"CFF ", cff); + build(*b"OTTO", &tables) +} diff --git a/rust-modules/media/src/player/ass/tests.rs b/rust-modules/media/src/player/ass/tests.rs index 0558d4d2a3..c346452f02 100644 --- a/rust-modules/media/src/player/ass/tests.rs +++ b/rust-modules/media/src/player/ass/tests.rs @@ -1,4 +1,5 @@ use super::*; +use super::mailbox::Kind; fn event(id: u32, start_ms: i64, duration_ms: i64) -> Event { Event { @@ -21,6 +22,79 @@ fn source(events: Vec) -> Source { } } +// The memory a helper costs (issue #1328): each renderer copies the fonts, so the pool decides once +// per selection, from the memory available and the fonts the primary holds, how many may run. + +#[test] +fn a_heavy_selection_gets_no_helper_and_the_verdict_is_taken_once() { + let runtime = Runtime::new(); + runtime.font_bytes.store(40 << 20, Ordering::Relaxed); + runtime.begin_selection(7); + assert!(runtime.helper_barred(1) && runtime.helper_barred(2), "a selection starts with the primary alone"); + let epoch = runtime.epoch.load(Ordering::Acquire); + let reads = std::cell::Cell::new(0); + let memory = || { + reads.set(reads.get() + 1); + Some(200u64 << 20) + }; + assert_eq!(runtime.renderers_decided(epoch), None, "the render scale waits for the pool's verdict"); + assert_eq!(runtime.renderers_for(epoch, memory), 1, "a quarter of 200 MiB holds no helper of 64 MiB beside the primary"); + assert!(runtime.helper_barred(1) && runtime.helper_barred(2)); + assert_eq!(runtime.renderers_for(epoch, memory), 1); + assert_eq!(reads.get(), 1, "the memory is read once for a selection, not on every tick"); + assert_eq!(runtime.renderers_decided(epoch), Some(1)); + runtime.begin_selection(8); + let next = runtime.epoch.load(Ordering::Acquire); + assert_eq!(runtime.renderers_decided(next), None, "the next selection is decided afresh"); +} + +#[test] +fn a_light_selection_lets_the_helpers_the_cores_allow_run_until_the_next_selection() { + let runtime = Runtime::new(); + runtime.font_bytes.store(2 << 20, Ordering::Relaxed); + runtime.begin_selection(7); + let epoch = runtime.epoch.load(Ordering::Acquire); + let slots = renderer_slots(); + assert_eq!(runtime.renderers_for(epoch, || Some(800u64 << 20)), slots); + for slot in 1..MAX_ASS_RENDERERS { + assert_eq!(runtime.helper_barred(slot), slot >= slots, "slot {slot} of {slots} renderers"); + } + runtime.begin_selection(8); + assert!(runtime.helper_barred(1), "the next selection is decided afresh"); + let epoch = runtime.epoch.load(Ordering::Acquire); + assert_eq!(runtime.renderers_for(epoch, || None), slots, "an unknown memory does not stop a helper"); +} + +/// A boundary inside a moving stretch is not its end (`Published::ends_at`): the engine says so from the +/// timing of the events alone, with no native call. Karaoke syllables overlap, so the frame at the end +/// of one still has another moving after it; the last one's end leaves a still picture. +#[test] +fn a_frame_says_whether_the_picture_after_its_window_moves_too() { + let karaoke = |start_ms, duration_ms| Event { + start_ms, + duration_ms, + payload: b"0,0,Default,,0,0,0,,{\\k20}la".to_vec().into(), + }; + let still = event(9, 0, 10_000); + let source = source(vec![karaoke(1000, 1000), karaoke(1500, 1000), karaoke(2000, 1000), still]); + let engine = Engine::default(); + let at = |now_ms| { + let validity = engine.validity(&source, now_ms); + (validity, engine.moves_on(&source, &validity)) + }; + let (validity, moves_on) = at(1200); + assert!(validity.animated && validity.until_ms == 1500); + assert!(moves_on, "the second syllable is moving when the first one's window ends"); + let (validity, moves_on) = at(2200); + assert!(validity.animated && validity.until_ms == 2500); + assert!(moves_on, "the third is still moving at 2500"); + let (validity, moves_on) = at(2700); + assert!(validity.animated && validity.until_ms == 3000); + assert!(!moves_on, "the stretch ends there: what follows is the still event"); + let (validity, moves_on) = at(5000); + assert!(!validity.animated && !moves_on, "a still picture does not move on"); +} + #[test] fn window_retirement_keeps_native_fonts_and_retained_overlapping_events() { let long = event(0, 0, 20_000); @@ -91,7 +165,7 @@ fn off_retires_a_racing_publication_and_pending_source_outside_the_mailbox_lock( runtime.source_id.store(source.id, Ordering::Release); { let mut mailbox = runtime.mailbox.lock().unwrap(); - mailbox.request(key, 100, &source); + mailbox.request(key, 100, &source, Pace::PAUSED); // The worker owns the lock and has already accepted this epoch. Cancel // must return while the lock is held; the old publication then races in. clear_runtime(&runtime); @@ -147,7 +221,7 @@ fn cancellation_before_worker_parks_is_remembered_and_releases_resources() { let observed_idle = observed_idle.clone(); let may_park = may_park.clone(); plx_base::task::spawn("ASS cancellation test", move || { - runtime.worker.set(std::thread::current()).unwrap(); + runtime.threads[0].set(std::thread::current()).unwrap(); let mut engine = Engine { source: Some(source), frame: Some(frame), @@ -230,6 +304,8 @@ fn job(source: &Arc, now_ms: i64, width: i32, height: i32, storage_width seq: 1, source: source.clone(), queued: None, + reference: None, + scale: RenderScale::Full, } } @@ -254,6 +330,17 @@ fn pixel(frame: &Frame, x: i32, y: i32) -> [u8; 4] { r.rgba[start..start + 4].try_into().unwrap() } +/// The colour a published PREMULTIPLIED pixel stands for, as straight RGBA: for a test that asks what +/// colour a pixel is and not which bytes the facade published (those are asserted as they are). +fn straight(px: [u8; 4]) -> [u8; 4] { + let a = u32::from(px[3]); + if a == 0 || a == 255 { + return px; + } + let un = |c: u8| ((u32::from(c) * 255 + a / 2) / a).min(255) as u8; + [un(px[0]), un(px[1]), un(px[2]), px[3]] +} + /// An actual pixel contract against the same pinned library shipped to the TV. /// Run after `make libass-host` with `--include-ignored`. #[test] @@ -267,7 +354,9 @@ fn native_pixels_preserve_position_layers_alpha_motion_karaoke_and_readorder() { let first = render(&mut engine, &source, 100); assert_eq!(pixel(&first, 25, 25), [255, 0, 0, 255]); assert_eq!(pixel(&first, 45, 35), [128, 0, 127, 255]); - assert_eq!(pixel(&first, 75, 35), [0, 0, 255, 127]); + // Published premultiplied: blue at 127/255 coverage is 127 of 255, not the straight 255. + assert_eq!(pixel(&first, 75, 35), [0, 0, 127, 127]); + assert_eq!(straight(pixel(&first, 75, 35)), [0, 0, 255, 127]); let coded = engine.render(&job(&source, 100, 320, 180, 720, 480)).frame; assert_eq!(coded.error, None); assert_eq!(pixel(&coded, 25, 25), [255, 0, 0, 255]); @@ -345,6 +434,7 @@ fn native_pixels_preserve_position_layers_alpha_motion_karaoke_and_readorder() { f.rects .iter() .flat_map(|r| r.rgba.chunks_exact(4)) + .map(|p| straight([p[0], p[1], p[2], p[3]])) .filter(|p| p[1] > 200 && p[0] < 20 && p[3] > 100) .count() }; @@ -552,7 +642,7 @@ fn a_track_is_never_refused_for_the_count_or_size_of_its_fonts() { data: Arc::from(vec![1u8; len]), }; let mut fonts: Vec = (0..200).map(|i| font(i, 2048)).collect(); - fonts.push(font(200, MAX_FONT_BYTES + 1)); + fonts.push(font(200, 40 * 1024 * 1024)); let mut s = source(vec![event(1, 0, 1000)]); let Content::Embedded { fonts: slot, .. } = &mut s.content else { panic!() }; *slot = fonts.into(); @@ -609,7 +699,7 @@ fn native_key() -> Key { /// The frame at `now_ms`; `None` is libass's "identical to the last frame". fn native_frame(native: &mut Native, now_ms: i64) -> Option> { native - .render(native_key(), now_ms, &[]) + .render(native_key(), RenderScale::Full, now_ms, &[]) .expect("render of a valid track") } @@ -664,12 +754,24 @@ fn faces_script(lines: &str) -> String { HEADER.replace("Style: Default,Inter,24", "Style: Default,PlxFaces,100") + lines } +/// `PlxAssStats` in `include/ass.h` is eight `uint32_t` and a `uint64_t`: 40 bytes with the 64-bit +/// field at offset 32, on a 32-bit ARM target as on a 64-bit host, so the mirror needs no padding. +#[test] +fn native_stats_mirror_the_c_struct_layout() { + assert_eq!(std::mem::size_of::(), 40); + assert_eq!(std::mem::align_of::(), 8); + assert_eq!(std::mem::offset_of!(NativeStats, last_libass_us), 12); + assert_eq!(std::mem::offset_of!(NativeStats, last_libass_cpu_us), 20); + assert_eq!(std::mem::offset_of!(NativeStats, last_images), 28); + assert_eq!(std::mem::offset_of!(NativeStats, last_image_pixels), 32); +} + #[test] #[ignore = "requires the pinned native host libass artifact and packaged fonts"] -fn native_the_facade_is_abi_four() { +fn native_the_facade_is_abi_five() { assert!(native_ass::load(Some(asset_dir())).ok()); - assert_eq!(unsafe { plx_ass_abi_version() }, 4); - assert_eq!(NATIVE_ABI, 4); + assert_eq!(unsafe { plx_ass_abi_version() }, 5); + assert_eq!(NATIVE_ABI, 5); } /// The bundled libass was built without libunibreak until the ABI 3 bump, so a CJK line with no @@ -722,9 +824,9 @@ fn native_line_spacing_changes_the_line_pitch_by_exactly_its_value() { fn native_cache_limits_are_validated() { let native = native_with(&[], &script_text("Dialogue: 0,0:00:00.00,0:00:02.00,Default,,0,0,0,,x\n")); unsafe { - assert_eq!(plx_ass_set_cache_limits(native.0, 400, 12), 0); - assert_eq!(plx_ass_set_cache_limits(native.0, 0, 12), -1); - assert_eq!(plx_ass_set_cache_limits(native.0, 400, 0), -1); + assert_eq!(plx_ass_set_cache_limits(native.ptr, 400, 12), 0); + assert_eq!(plx_ass_set_cache_limits(native.ptr, 0, 12), -1); + assert_eq!(plx_ass_set_cache_limits(native.ptr, 400, 0), -1); assert_eq!(plx_ass_set_cache_limits(std::ptr::null_mut(), 400, 12), -1); } } @@ -750,7 +852,29 @@ fn native_replace_track_keeps_the_fonts_and_the_next_frame_is_never_unchanged() ); let first = native_frame(&mut native, 100).expect("first frame"); assert_eq!(ink(&first).bands.len(), 1); + // The facade timed that render (`assperf:` splits libass from the composite by these), on the + // wall clock and on the thread's CPU clock, and counted the images libass produced for it. + let drawn = native.stats(); + assert!( + drawn.last_libass_us > 0 + && drawn.last_libass_cpu_us > 0 + && drawn.last_images > 0 + && drawn.last_image_pixels > 0, + "{:?}", + (drawn.last_libass_us, drawn.last_libass_cpu_us, drawn.last_images, drawn.last_image_pixels) + ); assert!(native_frame(&mut native, 100).is_none(), "the same instant is unchanged"); + let unchanged = native.stats(); + assert_eq!( + (unchanged.last_composite_us, unchanged.last_composite_cpu_us), + (0, 0), + "an unchanged call composites nothing" + ); + assert_eq!( + (unchanged.last_images, unchanged.last_image_pixels), + (drawn.last_images, drawn.last_image_pixels), + "and the image counts stay those of the frame on screen" + ); // The same text replaced in place: the frame is new, never "unchanged". native.replace_track(one.as_bytes(), 1).expect("replace"); @@ -906,6 +1030,356 @@ fn native_fourteen_faces_of_one_family_all_render() { assert_eq!(ink(&native_frame(&mut native, 500).unwrap()).bands.len(), 1); } +// ---- issue #1328: the fonts a track names reach libass, whatever the file attaches ---- + +use super::synth_fonts::rename; + +const MIB: usize = 1024 * 1024; + +/// A one-glyph test face (`face-NN.ttf`: the glyph U+E000+NN, a square `(400 + 20 NN)` thousandths +/// of an em) renamed to its own family `family`, so only a track that names it draws the square. +fn named_face(face: usize, family: &str, pad_to: Option) -> Font { + let data = std::fs::read(format!("{FACE_DIR}/face-{face:02}.ttf")).expect("committed fixture face"); + Font { name: format!("{family}.ttf"), data: rename(&data, family, "Regular", pad_to).into() } +} + +/// Inter renamed to a family no track names, padded to `bytes`: what a release attaches for a +/// sign nobody uses in this episode. +fn filler(index: usize, bytes: usize) -> Font { + let inter = std::fs::read(asset_dir().join("appfont.ttf")).expect("a shipped face"); + Font { + name: format!("filler-{index}.ttf"), + data: rename(&inter, &format!("Plx Filler {index}"), "Regular", Some(bytes)).into(), + } +} + +/// A script whose one style, `Default`, names `family` at 100 px, top-left on a 320x180 canvas. +fn named_script(family: &str, lines: &str) -> String { + HEADER.replace("Style: Default,Inter,24", &format!("Style: Default,{family},100")) + lines +} + +fn glyph_line(second: i64, tag: &str, glyph: u32) -> String { + format!( + "Dialogue: 0,0:00:{second:02}.00,0:00:{:02}.00,Default,,0,0,0,,{{\\an7\\pos(10,10){tag}}}{}\n", + second + 1, + char::from_u32(glyph).unwrap() + ) +} + +/// The report's file in miniature: seven fillers whose 42 MiB is past the old 32 MiB cap come FIRST +/// in the container, and the two 8 MiB faces the track names come after them. Before the fix libass +/// never saw the named ones and their glyphs fell back; now libass is handed exactly the two the +/// track names (one by its style, one by a `\fn`), the fillers cost it nothing, and the faces it +/// was handed draw as they do with no filler attached at all. +#[test] +#[ignore = "requires the pinned native host libass artifact and packaged fonts"] +fn native_the_faces_a_track_names_render_though_fillers_past_the_old_cap_come_first() { + let mut fonts: Vec = (0..7).map(|i| filler(i, 6 * MIB)).collect(); + let attached: usize = fonts.iter().map(|f| f.data.len()).sum(); + assert!(attached > 40 * MIB, "the fillers alone are past the old 32 MiB cap: {attached}"); + let named = fonts.len(); + // The named faces are big too (the report's are 6-13 MB), so the old cap, which still let a small + // late font in, would have dropped both. + fonts.push(named_face(3, "Plx Named A", Some(8 * MIB))); + fonts.push(named_face(5, "Plx Named B", Some(8 * MIB))); + let text = named_script( + "Plx Named A", + &(glyph_line(0, "", 0xE003) + &glyph_line(2, r"\fnPlx Named B", 0xE005)), + ); + + let mut native = native_with(&fonts, &text); + let stats = native.stats(); + assert_eq!(stats.library_fonts, bundled_faces() + 2, "libass holds the two named faces and no filler"); + assert_eq!(stats.font_scans, 1); + assert_eq!(native.given, 2); + let first = native_frame(&mut native, 500).expect("a new cue"); + let second = native_frame(&mut native, 2500).expect("a new cue"); + + let side = |face: i32| (400 + 20 * face) * 100 / 1000; + for (rects, face) in [(&first, 3), (&second, 5)] { + let drawn = ink(rects); + assert_eq!(drawn.bands.len(), 1, "face {face} drew its glyph"); + let (top, bottom) = drawn.bands[0]; + assert!( + (bottom - top + 1 - side(face)).abs() <= 2 && (drawn.right - drawn.left - side(face)).abs() <= 2, + "the square of face {face} ({} px), got {}x{}", + side(face), + drawn.right - drawn.left, + bottom - top + 1 + ); + } + + // The same pixels as a renderer that was handed only the two faces, and not as one with neither. + let mut alone = native_with(&fonts[named..], &text); + assert_eq!(native_frame(&mut alone, 500).unwrap(), first); + assert_eq!(native_frame(&mut alone, 2500).unwrap(), second); + let mut neither = native_with(&fonts[..named], &text); + assert_eq!(neither.stats().library_fonts, bundled_faces()); + assert_ne!(native_frame(&mut neither, 500).unwrap(), first, "the fallback is not the named face"); + assert_ne!(native_frame(&mut neither, 2500).unwrap(), second); +} + +/// An authored track names a font in an event only: the face is added when the event is, after the +/// renderer has loaded its track and drawn a frame, and it draws the same pixels as the same face +/// loaded up front. +#[test] +#[ignore = "requires the pinned native host libass artifact and packaged fonts"] +fn native_a_font_a_later_event_names_is_added_after_load_and_draws_as_if_loaded_up_front() { + let late = named_face(3, "Plx Late", None); + let bystander = filler(0, 2 * MIB); + let fonts: Arc<[Font]> = vec![bystander, late.clone()].into(); + let header = HEADER.replace("Style: Default,Inter,24", "Style: Default,Inter,100"); + let packet = |read_order: u32, tag: &str, glyph: char| { + format!("{read_order},0,Default,,0,0,0,,{{\\an7\\pos(10,10){tag}}}{glyph}").into_bytes() + }; + let events = |count: usize| -> Arc<[Event]> { + let all = [ + (0, 1000, packet(0, "", 'A')), + (2000, 1000, packet(1, r"\fnPlx Late", '\u{E003}')), + ]; + all[..count] + .iter() + .map(|(start_ms, duration_ms, payload)| Event { + start_ms: *start_ms, + duration_ms: *duration_ms, + payload: payload.clone().into(), + }) + .collect::>() + .into() + }; + let embedded = |count: usize| Source { + id: 1, + revision: count as u64, + origin: Origin::Authored, + content: Content::Embedded { + header: header.clone().into_bytes().into(), + events: events(count), + fonts: fonts.clone(), + }, + }; + + let mut native = Native::open(&embedded(1)).expect("native fixture requires the built host library in pkg"); + let opened = native.stats(); + assert_eq!(opened.library_fonts, bundled_faces(), "the header names Inter: no attached face yet"); + let before = native_frame(&mut native, 500).expect("the first cue draws"); + assert_eq!(native.stats().font_scans, 1); + + // The demuxer reads the next packet: its `\fn` adds the face, after the first frame was drawn. + native.append(&embedded(2), 1).unwrap(); + let late_frame = native_frame(&mut native, 2500).expect("the second cue draws"); + let stats = native.stats(); + assert_eq!(stats.library_fonts, bundled_faces() + 1, "one face, the one named"); + assert_eq!(stats.font_scans, 1, "libass did not scan the other faces again"); + assert_eq!(ink(&late_frame).bands.len(), 1); + assert!(native_frame(&mut native, 500).is_some(), "and the first cue still draws"); + assert_eq!(native.given, 1); + + // The reference: the whole track at once, so the face is held before the first load. + let mut eager = Native::open(&embedded(2)).unwrap(); + assert_eq!(eager.stats().library_fonts, bundled_faces() + 1, "an eager open scans the face with the others"); + assert_eq!(native_frame(&mut eager, 500).unwrap(), before); + assert_eq!(native_frame(&mut eager, 2500).unwrap(), late_frame); + + // Without the attachment the same event falls back and draws something else. + let mut missing_source = embedded(2); + let Content::Embedded { fonts: slot, .. } = &mut missing_source.content else { unreachable!() }; + *slot = Arc::from(Vec::::new()); + let mut missing = Native::open(&missing_source).unwrap(); + assert_ne!(native_frame(&mut missing, 2500).unwrap(), late_frame); +} + +/// No byte count stands between a named face and libass: a 70 MiB face (past the old 64 MiB limit +/// the facade enforced) is accepted and drawn, and a second 40 MiB face after it is too (80 MiB in +/// all). +#[test] +#[ignore = "requires the pinned native host libass artifact and packaged fonts"] +fn native_a_face_past_the_old_facade_limit_is_accepted_and_draws() { + let big = named_face(3, "Plx Huge", Some(70 * MIB)); + assert!(big.data.len() > 64 * MIB); + let text = named_script("Plx Huge", &glyph_line(0, "", 0xE003)); + let mut native = native_with(std::slice::from_ref(&big), &text); + assert_eq!(native.stats().library_fonts, bundled_faces() + 1); + assert_eq!(native.given, 1); + let drawn = ink(&native_frame(&mut native, 500).expect("a new cue")); + assert_eq!(drawn.bands.len(), 1, "the 70 MiB face draws its glyph"); + let more = named_face(5, "Plx Huge Too", Some(40 * MIB)); + native.font(&more.name, &more.data).expect("past the old cumulative limit too"); + assert_eq!(native.stats().library_fonts, bundled_faces() + 2); +} + +// ---- issue #1328: a renderer holds no copy of the attached font bytes ---- + +use plx_base::testheap; + +/// Inter renamed to the family `Plx Heap {index}` and padded to `bytes`: a face a track names, as big +/// as the display faces of the report (6-13 MB each). +fn heap_face(index: usize, bytes: usize) -> Font { + let inter = std::fs::read(asset_dir().join("appfont.ttf")).expect("a shipped face"); + Font { + name: format!("heap-{index}.ttf"), + data: rename(&inter, &format!("Plx Heap {index}"), "Regular", Some(bytes)).into(), + } +} + +/// What a renderer costs in heap beyond the fonts it is handed, measured on the C heap of a fresh +/// process (`testheap`: libass and FreeType call `malloc` themselves, so a Rust allocator count +/// would miss exactly the copies this is about). The fonts are the store's, shared by `Arc`; the +/// renderer opens six faces of 8 MiB, draws a cue in each, and the heap it added over its fonts +/// is read back, and again once it is dropped. +fn heap_renderer_cost() { + const FACES: usize = 6; + let fonts: Vec = (0..FACES).map(|i| heap_face(i, 8 * MIB)).collect(); + let font_bytes: usize = fonts.iter().map(|f| f.data.len()).sum(); + let lines: String = (0..FACES as i64) + .map(|i| glyph_line(i, &format!(r"\fnPlx Heap {i}"), u32::from(b'H'))) + .collect(); + let text = named_script("Plx Heap 0", &lines); + let base = testheap::live_bytes().expect("a host whose malloc can be asked"); + let ((native, shown), peak) = testheap::peak_during(|| { + let mut native = native_with(&fonts, &text); + let mut shown = 0; + for second in 0..FACES as i64 { + shown += usize::from(native_frame(&mut native, second * 1000 + 500).is_some()); + } + (native, shown) + }); + assert_eq!(shown, FACES, "every cue draws"); + assert_eq!(native.given, FACES, "the track names every face"); + let held = testheap::live_bytes().unwrap(); + drop(native); + let released = testheap::live_bytes().unwrap(); + testheap::report("font_bytes", font_bytes as u64); + testheap::report("held", held.saturating_sub(base) as u64); + testheap::report("peak", peak.saturating_sub(base) as u64); + testheap::report("after_drop", released.saturating_sub(base) as u64); + drop(fonts); +} + +/// Every renderer used to copy the attached fonts it was handed (`ass_add_font` mallocs and copies), +/// so each one cost the fonts again: a primary and two helpers on the report's file held the 53 MB +/// the track names three times over, beside the store's own copy. libass reads them in place now: +/// the renderer's own heap is its face state and caches, a fraction of the font bytes and not their +/// multiple. +#[test] +#[ignore = "requires the pinned native host libass artifact and packaged fonts"] +fn native_a_renderer_holds_no_copy_of_the_fonts_it_is_handed() { + const TEST: &str = "player::ass::tests::native_a_renderer_holds_no_copy_of_the_fonts_it_is_handed"; + if testheap::is_child() { + return heap_renderer_cost(); + } + let cost = testheap::in_fresh_process(TEST); + let fonts = cost["font_bytes"]; + assert!(fonts >= 48 * MIB as u64, "the fixture is big enough to tell a copy from noise: {fonts}"); + let mib = |bytes: u64| bytes as f64 / MIB as f64; + eprintln!( + "renderer over {:.0} MiB of faces: held {:.1} MiB, peaked at {:.1} MiB, {:.1} MiB left after drop", + mib(fonts), + mib(cost["held"]), + mib(cost["peak"]), + mib(cost["after_drop"]) + ); + assert!( + cost["held"] < fonts / 4, + "a renderer of {:.1} MiB of faces held {:.1} MiB: it copies them", + mib(fonts), + mib(cost["held"]) + ); + assert!( + cost["peak"] < fonts / 3, + "a renderer of {:.1} MiB of faces peaked at {:.1} MiB: it copies them", + mib(fonts), + mib(cost["peak"]) + ); + assert!(cost["after_drop"] < 4 * MIB as u64, "a dropped renderer gives its heap back: {:.1} MiB left", mib(cost["after_drop"])); +} + +/// A font a renderer references is the store's buffer, not the library's: three renderers over the +/// same `Arc` each hold it beside a font libass extracted from the script's own `[Fonts]` (which +/// libass owns and frees), draw both, and are dropped in an order that is not their creation's. What +/// libass frees at destroy is only what it allocated: the shared bytes are intact and the store's +/// handle is the only one left (a freed or doubly freed buffer would have crashed the test, or +/// drawn a different glyph). +#[test] +#[ignore = "requires the pinned native host libass artifact and packaged fonts"] +fn native_a_borrowed_font_beside_an_extracted_one_survives_the_renderers_that_reference_it() { + let borrowed = named_face(5, "Plx Borrowed", None); + let original = borrowed.data.to_vec(); + let extracted = std::fs::read(format!("{FACE_DIR}/face-03.ttf")).unwrap(); + let (styles, events) = named_script("Plx Borrowed", "").split_once("[Events]").map(|(a, b)| (a.to_owned(), b.to_owned())).unwrap(); + let text = format!( + "{styles}{}[Events]{events}{}{}", + ass_font_section("slide.ttf", &extracted), + glyph_line(0, "", 0xE005), + glyph_line(1, r"\fnPlxFaces", 0xE003), + ); + let fonts = [borrowed.clone()]; + let mut renderers: Vec = (0..3).map(|_| native_with(&fonts, &text)).collect(); + for native in &renderers { + assert_eq!(native.given, 1, "the track names the attached face"); + assert_eq!( + native.stats().library_fonts, + bundled_faces() + 2, + "the bundled faces, the one from [Fonts] and the one referenced" + ); + } + assert_eq!(Arc::strong_count(&borrowed.data), 1 + 1 + 3, "the test's, the array's and one per renderer"); + for native in &mut renderers { + assert_eq!(ink(&native_frame(native, 500).expect("a new cue")).bands.len(), 1, "the referenced face draws"); + assert_eq!(ink(&native_frame(native, 1500).expect("a new cue")).bands.len(), 1, "and so does the extracted one"); + } + drop(renderers.remove(1)); + for native in &mut renderers { + assert_eq!( + ink(&native_frame(native, 500).expect("a new cue")).bands.len(), + 1, + "a renderer still draws the shared face after another is gone" + ); + } + renderers.clear(); + assert_eq!(&borrowed.data[..], &original[..], "the shared bytes are untouched"); + drop(fonts); + assert_eq!(Arc::strong_count(&borrowed.data), 1, "no renderer holds the buffer after it is destroyed"); +} + +/// A long sidecar script is fed to libass a window at a time, and the first window holds only its +/// first lines: a face that a line far past it asks for must be held before the window slides there. +#[test] +#[ignore = "requires the pinned native host libass artifact and packaged fonts"] +fn native_a_windowed_script_holds_the_face_a_line_outside_its_first_window_names() { + let t = |ms: i64| format!("{}:{:02}:{:02}.{:02}", ms / 3_600_000, ms / 60_000 % 60, ms / 1000 % 60, ms % 1000 / 10); + let mut lines = String::new(); + for i in 0..10_000i64 { + let tag = if i == 9_999 { r"\fnPlx Far" } else { "" }; + let glyph = if i == 9_999 { '\u{E003}' } else { 'x' }; + lines.push_str(&format!( + "Dialogue: 0,{},{},Default,,0,0,0,,{{\\an7\\pos(10,10){tag}}}{glyph}\n", + t(i * 1000), + t(i * 1000 + 900) + )); + } + let fonts: Arc<[Font]> = vec![named_face(3, "Plx Far", None)].into(); + let source = Arc::new(Source { + id: next_source_id(), + revision: 1, + origin: Origin::Authored, + content: Content::Script { + bytes: HEADER.replace("Style: Default,Inter,24", "Style: Default,Inter,100").into_bytes().into_iter().chain(lines.into_bytes()).collect::>().into(), + fonts, + }, + }); + let mut engine = Engine::default(); + let first = render(&mut engine, &source, 100); + assert!(!first.rects.is_empty()); + let opened = engine.native.as_ref().unwrap().stats(); + assert_eq!(opened.library_fonts, bundled_faces() + 1, "held from the first open, before any window names it"); + let far = render(&mut engine, &source, 9_999_100); + assert_eq!(ink(&far.rects).bands.len(), 1); + let (top, bottom) = ink(&far.rects).bands[0]; + assert!((bottom - top + 1 - 46).abs() <= 2, "the far line draws face 3's square, not Inter's x"); + let slid = engine.native.as_ref().unwrap().stats(); + assert_eq!((slid.font_scans, slid.library_fonts), (1, opened.library_fonts)); +} + // ---- Source.origin, the restyle branch and the alignment classes (plan §2.4, §2.5, §4.6 case 3) ---- use crate::player::text_ass::{self, AlignClass}; @@ -1446,7 +1920,7 @@ fn native_the_worker_publishes_aligns_beside_a_text_frame_and_none_for_authored( let until = |source: &Arc, now: i64, expect: Option| { let deadline = std::time::Instant::now() + std::time::Duration::from_secs(20); loop { - let frame = request(source.clone(), now, 1920, 1080, 1920, 1080); + let frame = request(source.clone(), now, 1920, 1080, 1920, 1080, Pace::PAUSED); let got = frame.as_ref().and_then(|f| published_aligns(f.serial)); if frame.is_some() && got == expect { return; @@ -1638,17 +2112,245 @@ fn a_changed_offset_on_a_paused_playhead_publishes_the_frame_of_the_new_clock() // The worker drains its mailbox: the frame is `error_frame` (any frame), the validity the cue's. let drain = |mailbox: &mut Mailbox| { while let Some(job) = mailbox.take() { - let rendered = Rendered { frame: error_frame(job.key, Fault::RenderFailed), validity: validity_of(job.now_ms), aligns: None }; + let rendered = Rendered { frame: error_frame(job.key, Fault::RenderFailed), validity: validity_of(job.now_ms), aligns: None, cost_us: None, moves_on: false }; mailbox.finish(&job, Some(rendered)); } }; // Paused at 23.0 s, the offset 0, then +2.5 s, then -2.5 s, then back, each one a frame step. for clock in [23_000, 20_500, 25_500, 20_500, 25_500, 23_000] { - mailbox.request(key, clock, &source); + mailbox.request(key, clock, &source, Pace::PAUSED); drain(&mut mailbox); - mailbox.request(key, clock, &source); + mailbox.request(key, clock, &source, Pace::PAUSED); let shown = mailbox.view(&key).expect("a frame"); let published = mailbox.published.as_ref().map(|p| p.validity).expect("published"); assert!(published.contains(clock), "clock {clock}: the frame on screen covers {published:?} ({:?})", shown.serial); } } + +// Several renderers (issue #1328): each worker owns its own libass instance, so what one engine +// would have drawn alone is what any of them draws, and the screen's texture cache is not defeated +// by the frames of one cue coming from different engines. + +const SIGN: &str = r"{\an7\pos(20,20)\c&H0000FF&\p1}m 0 0 l 40 0 40 40 0 40"; +/// Moves and changes colour: a square that only moved is the same bitmap in another place, which +/// the renderer rightly shares between frames, so the test's moving region must change its pixels. +const MOVER: &str = r"{\an7\move(100,20,260,20,0,4000)\c&HFF0000&\t(0,4000,\c&H00FF00&)\p1}m 0 0 l 30 0 30 30 0 30"; + +/// A static sign and a square that moves and changes colour for four seconds. +fn sign_and_mover() -> Arc { + script(&format!( + "Dialogue: 0,0:00:00.00,0:00:04.00,Default,,0,0,0,,{SIGN}\nDialogue: 1,0:00:00.00,0:00:04.00,Default,,0,0,0,,{MOVER}\n" + )) +} + +/// A frame as one canvas of the bytes it publishes (its regions are disjoint). +fn canvas(frame: &Frame) -> Vec { + let mut out = vec![0u8; (frame.width * frame.height * 4) as usize]; + for r in &frame.rects { + for row in 0..r.height { + let to = ((r.y + row) * frame.width + r.x) as usize * 4; + let from = (row * r.width) as usize * 4; + out[to..to + r.width as usize * 4].copy_from_slice(&r.rgba[from..from + r.width as usize * 4]); + } + } + out +} + +fn same_pixels(a: &Rect, b: &Rect) -> bool { + a.width == b.width && a.height == b.height && a.rgba == b.rgba +} + +#[test] +#[ignore = "requires the pinned native host libass artifact and packaged fonts"] +fn two_engines_on_two_threads_render_what_one_engine_renders_in_sequence() { + let source = sign_and_mover(); + let times: Vec = (1..=24).map(|i| i * 150).collect(); + let mut alone = Engine::default(); + let expected: Vec> = times.iter().map(|&t| canvas(&render(&mut alone, &source, t))).collect(); + assert!(expected.windows(2).all(|w| w[0] != w[1]), "the square moves at every playhead"); + assert!(expected[0].iter().any(|&b| b != 0), "something is drawn"); + // two workers, each with an engine of its own, taking the playheads alternately + let drawn: Vec<(usize, Vec)> = std::thread::scope(|scope| { + let workers: Vec<_> = (0..2) + .map(|worker| { + let (source, times) = (&source, ×); + scope.spawn(move || { + let mut engine = Engine::default(); + times + .iter() + .enumerate() + .filter(|(i, _)| i % 2 == worker) + .map(|(i, &t)| (i, canvas(&render(&mut engine, source, t)))) + .collect::>() + }) + }) + .collect(); + workers.into_iter().flat_map(|worker| worker.join().unwrap()).collect() + }); + assert_eq!(drawn.len(), times.len()); + for (i, pixels) in drawn { + assert!(pixels == expected[i], "the playhead {} ms is drawn the same by either engine", times[i]); + } +} + +#[test] +#[ignore = "requires the pinned native host libass artifact and packaged fonts"] +fn a_reference_frame_makes_another_engine_share_the_regions_it_draws_identically() { + let source = sign_and_mover(); + let mut first_engine = Engine::default(); + let first = render(&mut first_engine, &source, 500); + assert_eq!(first.rects.len(), 2, "a sign and a square are two regions"); + let allocations: Vec<_> = first.rects.iter().map(|r| r.rgba.clone()).collect(); + let shares_one = |frame: &Frame| { + frame.rects.iter().filter(|r| allocations.iter().any(|a| Arc::ptr_eq(a, &r.rgba))).count() + }; + + // Another engine on its own draws the sign the same, in an allocation of its own: the screen + // would upload it again for every frame the engines take turns on. + let alone = render(&mut Engine::default(), &source, 1000); + assert!(alone.rects.iter().any(|r| first.rects.iter().any(|f| same_pixels(r, f))), "the sign is the same"); + assert_eq!(shares_one(&alone), 0); + + // With the frame before it as its reference, the sign is that frame's allocation and the + // square, which moved, is a new one; the picture is the same bytes. + let mut referred = job(&source, 1000, 320, 180, 320, 180); + referred.reference = Some(first.clone()); + let shared = Engine::default().render(&referred).frame; + assert_eq!(shared.error, None); + assert_eq!(shares_one(&shared), 1, "the static sign shares, the square that changed does not"); + assert_eq!(canvas(&shared), canvas(&alone), "sharing changes no pixel"); + + // A picture that is the reference's own is that very frame: the screen already knows its serial. + let mut same = job(&source, 500, 320, 180, 320, 180); + same.reference = Some(first.clone()); + let again = Engine::default().render(&same).frame; + assert!(Arc::ptr_eq(&again, &first)); + + // ... also when the engine itself found nothing changed since its own last frame: a still sign + // drawn before by this engine with no reference is handed back as the reference's. + let still = script(&format!("Dialogue: 0,0:00:00.00,0:00:04.00,Default,,0,0,0,,{SIGN}\n")); + let reference = render(&mut Engine::default(), &still, 300); + let mut engine = Engine::default(); + let own = render(&mut engine, &still, 400); + assert!(!Arc::ptr_eq(&own, &reference), "each engine draws its own copy"); + let mut later = job(&still, 900, 320, 180, 320, 180); + later.reference = Some(reference.clone()); + let handed = engine.render(&later).frame; + assert!(Arc::ptr_eq(&handed, &reference), "unchanged, and the same pixels as the reference"); +} + +// The render scale (issue #1328): a moving stretch is drawn at a fraction of the output size, with +// the storage size unchanged so the script lays out identically, and the regions come back in the +// smaller canvas. + +fn region(x: i32, y: i32, width: i32, height: i32) -> Rect { + Rect { x, y, width, height, rgba: Arc::from(vec![0u8; (width * height * 4) as usize]) } +} + +fn frame_at(scale: RenderScale, width: i32, height: i32, rects: Vec) -> Frame { + Frame { source_id: 1, serial: 1, width, height, rects, scale, error: None } +} + +#[test] +fn a_regions_edges_map_back_to_the_output_canvas_whatever_the_scale() { + // the same region of the output, at each rung: the frame's own coordinates differ, the output's do not + let output = (480.0, 270.0, 1160.0, 640.0); + let full = frame_at(RenderScale::Full, 1920, 1080, vec![]); + assert_eq!(full.output_edges(®ion(480, 270, 680, 370)), output, "full size maps to itself exactly"); + let half = frame_at(RenderScale::Half, 1920, 1080, vec![]); + assert_eq!(half.render_size(), (960, 540)); + assert_eq!(half.output_edges(®ion(240, 135, 340, 185)), output); + let two_thirds = frame_at(RenderScale::TwoThirds, 1920, 1080, vec![]); + assert_eq!(two_thirds.render_size(), (1280, 720)); + assert_eq!(two_thirds.output_edges(®ion(320, 180, 453, 247)), (480.0, 270.0, 1159.5, 640.5)); +} + +#[test] +fn regions_that_abut_in_the_frame_abut_on_the_output() { + // a letterboxed viewport, where the rounded render height does not divide the output's + let frame = frame_at(RenderScale::TwoThirds, 1920, 800, vec![]); + assert_eq!(frame.render_size(), (1280, 533)); + for edge in [1, 100, 266, 400, 532] { + let above = frame.output_edges(®ion(0, edge - 17, 10, 17)); + let below = frame.output_edges(®ion(0, edge, 10, 20)); + assert_eq!(above.3, below.1, "the bottom of one is the top of the next at {edge}"); + } + let last = frame.output_edges(®ion(0, 500, 1280, 33)); + assert!((last.2 - 1920.0).abs() < 1e-3 && (last.3 - 800.0).abs() < 1e-3, "the far edge of the frame is the far edge of the output"); +} + +/// A cue drawn at half size is the same picture as at full size: every region sits where the full-size +/// one does, within the transparent border the facade leaves round a region, and it costs a quarter of +/// the pixels. The storage size is the one the full frame has, so libass lays the script out the same. +#[test] +#[ignore = "requires the pinned native host libass artifact and packaged fonts"] +fn a_frame_drawn_at_half_size_lays_out_where_the_full_size_one_does() { + let red = r"{\an7\pos(20,20)\c&H0000FF&\p1}m 0 0 l 40 0 40 40 0 40"; + let blue = r"{\an7\pos(120,90)\c&HFF0000&\p1}m 0 0 l 60 0 60 30 0 30"; + let source = script(&format!( + "Dialogue: 0,0:00:00.00,0:00:02.00,Default,,0,0,0,,{red}\nDialogue: 1,0:00:00.00,0:00:02.00,Default,,0,0,0,,{blue}\n" + )); + let at = |scale: RenderScale, now_ms: i64| { + let mut job = job(&source, now_ms, 640, 360, 640, 360); + job.scale = scale; + let frame = Engine::default().render(&job).frame; + assert_eq!(frame.error, None, "native fixture requires the built host library in pkg"); + frame + }; + let (full, half) = (at(RenderScale::Full, 100), at(RenderScale::Half, 100)); + assert_eq!((full.scale, half.scale), (RenderScale::Full, RenderScale::Half)); + assert_eq!((full.width, full.height, half.width, half.height), (640, 360, 640, 360), "both answer for the output"); + assert_eq!(half.render_size(), (320, 180)); + assert_eq!(full.rects.len(), 2, "two squares"); + assert_eq!(half.rects.len(), full.rects.len()); + let bytes = |f: &Frame| f.rects.iter().map(|r| r.rgba.len()).sum::(); + assert!(bytes(&half) * 3 <= bytes(&full), "a quarter of the pixels: {} against {}", bytes(&half), bytes(&full)); + let edges = |f: &Frame| { + let mut all: Vec<_> = f.rects.iter().map(|r| f.output_edges(r)).collect(); + all.sort_by(|a, b| a.partial_cmp(b).unwrap()); + all + }; + for (a, b) in edges(&full).into_iter().zip(edges(&half)) { + let close = |x: f32, y: f32| (x - y).abs() <= 3.0; + assert!( + close(a.0, b.0) && close(a.1, b.1) && close(a.2, b.2) && close(a.3, b.3), + "full {a:?} against half {b:?} on the output canvas" + ); + } + // the solid square's centre is the same colour at both sizes + assert_eq!(pixel(&half, 20, 20), [255, 0, 0, 255]); + assert_eq!(pixel(&full, 40, 40), [255, 0, 0, 255]); +} + +#[test] +#[ignore = "requires the pinned native host libass artifact and packaged fonts"] +fn a_render_at_a_new_size_is_cold_and_no_sample_and_each_frame_is_labelled_with_its_own_size() { + let sign = r"{\an7\pos(20,20)\c&H0000FF&\p1}m 0 0 l 40 0 40 40 0 40"; + let source = script(&format!("Dialogue: 0,0:00:00.00,0:00:02.00,Default,,0,0,0,,{sign}\n")); + let mut engine = Engine::default(); + let mut at = |scale: RenderScale, now_ms: i64| { + let mut job = job(&source, now_ms, 640, 360, 640, 360); + job.scale = scale; + let frame = engine.render(&job).frame; + assert_eq!(frame.error, None, "native fixture requires the built host library in pkg"); + (frame, engine.resynced) + }; + let (first, opened) = at(RenderScale::Full, 100); + assert!(opened, "the first render opens the renderer"); + let (_, again) = at(RenderScale::Full, 200); + assert!(!again, "the same size again is a steady frame"); + let (half, resized) = at(RenderScale::Half, 300); + assert!(resized, "a new size drops libass's caches: that frame is cold, no sample of a steady one"); + assert_eq!(half.scale, RenderScale::Half); + let (_, steady) = at(RenderScale::Half, 400); + assert!(!steady); + let (back, resized) = at(RenderScale::Full, 500); + assert!(resized); + assert_eq!(back.scale, RenderScale::Full, "a frame is never handed back for another size's regions"); + assert_eq!(back.rects, first.rects, "and it is the picture the full size drew before"); + // a picture with no region is labelled with the size it was drawn at as well + let (empty_full, _) = at(RenderScale::Full, 5_000); + let (empty_half, _) = at(RenderScale::Half, 5_100); + assert!(empty_full.rects.is_empty() && empty_half.rects.is_empty()); + assert_eq!((empty_full.scale, empty_half.scale), (RenderScale::Full, RenderScale::Half)); +} diff --git a/rust-modules/media/src/player/ass/tests/synth.rs b/rust-modules/media/src/player/ass/tests/synth.rs index 41332a84b3..267665d9d2 100644 --- a/rust-modules/media/src/player/ass/tests/synth.rs +++ b/rust-modules/media/src/player/ass/tests/synth.rs @@ -62,23 +62,24 @@ fn style(size: SubtitleSize, position: SubtitlePosition) -> TextStyle { /// The frame of `cues` at 1 s. fn text_frame(style: &TextStyle, cues: &[Cue]) -> Vec { let mut native = text_native(style, cues); - native.render(text_key(), 1000, &[]).expect("a valid track").expect("a first frame") + native.render(text_key(), RenderScale::Full, 1000, &[]).expect("a valid track").expect("a first frame") } -/// The pixels of a frame as `(x, y, [r, g, b, a])`. +/// The pixels of a frame as `(x, y, [r, g, b, a])`, straight RGBA: the facade publishes premultiplied +/// pixels, and what these goldens measure is where the colours are ([`straight`]). fn pixels(rects: &[Rect]) -> impl Iterator + '_ { rects.iter().flat_map(|r| { (0..r.height).flat_map(move |y| { (0..r.width).map(move |x| { let at = ((y * r.width + x) * 4) as usize; - (r.x + x, r.y + y, [r.rgba[at], r.rgba[at + 1], r.rgba[at + 2], r.rgba[at + 3]]) + (r.x + x, r.y + y, straight([r.rgba[at], r.rgba[at + 1], r.rgba[at + 2], r.rgba[at + 3]])) }) }) }) } -/// The white fill (not the black outline, and not the soft edge of either): straight RGBA with -/// most of its colour and opacity. +/// The white fill (not the black outline, and not the soft edge of either): straight RGBA ([`pixels`]) +/// with most of its colour and opacity. fn is_fill(px: [u8; 4]) -> bool { px[0] >= 128 && px[3] >= 128 } diff --git a/rust-modules/media/src/player/ass_perf.rs b/rust-modules/media/src/player/ass_perf.rs index e8a336e5fa..b6e3473736 100644 --- a/rust-modules/media/src/player/ass_perf.rs +++ b/rust-modules/media/src/player/ass_perf.rs @@ -1,32 +1,63 @@ -//! `assperf:` — what libass costs on the television, one event-log line per second while a -//! styled-subtitle source is rendering. +//! `assperf:` — what libass costs on the television, one event-log line per second per worker +//! while a styled-subtitle source is rendering. //! //! ```text -//! assperf: worker tid=1234 -//! assperf: window_ms=1000 calls=3 changed=2 p95_us=1800 max_us=4100 job_p95_us=2100 job_max_us=9000 pub_bytes=123456 up_bytes=123456 jobs=2+1 ahead_hit=1 nwait_p95_us=0 nwait_max_us=0 +//! assperf: worker w=0 tid=1234 nice=5 +//! assperf: w=0 window_ms=1000 calls=3 changed=2 p95_us=1800 max_us=4100 job_p95_us=2100 job_max_us=9000 pub_bytes=123456 up_bytes=123456 jobs=2+1 ahead_hit=1 shown=0 late=0 stale=0 nwait_p95_us=0 nwait_max_us=0 lib_p95_us=900 comp_p95_us=800 lib_cpu_p95_us=870 comp_cpu_p95_us=790 images_p95=410 img_kpx_p95=620 scale=1.00 //! ``` //! +//! `w` is the worker: 0 the primary, 1 and 2 the helpers a moving cue's pipeline starts when one +//! worker cannot make thirty frames a second (issue #1328); each keeps its own window and writes its +//! own line, and `tools/tv-sched-trace.sh --tid ` follows one. The counts the UI thread keeps +//! (`up_bytes`, `ahead_hit`, `shown`, `late`) belong to the primary's line alone: it reads and +//! restarts them, so a helper's line has zeros there and nothing is counted twice. +//! //! `window_ms` is the span the counts cover: from the previous line (or the session's first job) //! to this one. Jobs are rare while a cue is steady, so a line can follow a long idle stretch and //! the counts are only comparable as rates over `window_ms`. `calls` is how many times the worker ran `plx_ass_render` (one per job: a frame is rendered -//! when the screen needs one, and the next boundary is rendered ahead of its time, not once per UI +//! when the screen needs one, a moving cue's frames are rendered ahead of their playhead by the +//! pipeline, and the next boundary of a still cue is rendered ahead of its time, not once per UI //! frame), `changed` how many of those produced a frame whose pixels differ from the one on screen -//! (published at once, or held ahead for the next boundary), `p95_us`/`max_us` the wall time +//! (published at once, or held ahead for the next boundary or playhead), `p95_us`/`max_us` the wall time //! of the FFI call alone (nearest-rank 95th percentile over a fixed ring), `job_p95_us`/`job_max_us` //! the same for the whole worker job (source load, event append, flush-and-reload, window slide //! and the draw call: the cost a cue change or seek pays), `pub_bytes` the RGBA //! bytes of the frames published or held ahead, and `up_bytes` the bytes the UI thread uploaded with //! `glTexImage2D` in the same window ([`note_upload`]). `jobs=N+A` counts the worker's Normal and -//! Ahead jobs separately, `ahead_hit` how many times the UI thread promoted an ahead frame -//! ([`note_ahead_hit`]) and `nwait_*` how long a Normal job waited behind an Ahead job that was -//! already rendering ([`note_nwait`]; native rendering cannot be cancelled). The upload's own time is the `assup` +//! Ahead jobs separately (a frame the pipeline planned is an Ahead one), `ahead_hit` how many +//! times the UI thread promoted the boundary frame of a still cue ([`note_ahead_hit`]), `shown` +//! how many times it promoted a frame of the pipeline ([`note_shown`]: the moving cue's updates a +//! second) and `late` how many of those were promoted more than a tick after their playhead (the +//! frame had not landed by then), `stale` how many frames this worker rendered that were thrown +//! away when they landed (the plan had moved on: a seek, an appended event it could not absorb, a +//! jump of the clock) and `nwait_*` how long a Normal job waited behind an Ahead job that was +//! already rendering ([`note_nwait`]; native rendering cannot be cancelled). The last six fields +//! split `p95_us`: the facade times its two phases itself (`clock_gettime` in `src/ass.c`, +//! `PlxAssStats::last_*`) and the worker reads them after every render. `lib_p95_us` is libass's +//! own `ass_render_frame`, `comp_p95_us` is everything the facade does with the image list after it +//! (bounds, regions, the composite into premultiplied RGBA, eight pixels at a time with NEON on the TV +//! and on arm64 hosts and scalar elsewhere; 0 for a call that found the frame +//! unchanged), both wall time (`CLOCK_MONOTONIC`) and nearest-rank over the same fixed ring as +//! `p95_us` (every call that returned a result, so a window of unchanged calls pulls them down). +//! `lib_cpu_p95_us` and `comp_cpu_p95_us` are the same two phases on the worker thread's CPU clock +//! (`CLOCK_THREAD_CPUTIME_ID`): a phase whose wall figure is far above its CPU one was waiting (the +//! thread descheduled behind the UI thread or another process, or blocked), not computing, which +//! is how a slow frame is told from a starved one. `images_p95` the `ASS_Image`s libass +//! produced and `img_kpx_p95` the sum of their width x height in thousands of pixels +//! (`last_image_pixels / 1000`), both over the composed frames only. They are 0 for a worker whose +//! library reports no stats. `scale` is the smallest size a job of the window was drawn at, as the +//! side of the frame over the side of the output (`1.00`, `0.67`, `0.50`): a moving stretch that +//! cannot make thirty frames a second at full size is drawn smaller (issue #1328, `docs/ass-subtitles.md`), +//! and the figures above are for the frames of that size, so a line is read at its own `scale`. Each +//! move of the scale is also logged once as `ass: render scale 1.00 -> 0.50 ...`. The upload's own time is the `assup` //! span on `FRAMEDROP spans=` (spans print only on `FRAMEDROP` lines and only while the frame //! instrument is armed, `/tmp/plxnative-framedrop`). A window closes on the first job at least a //! second after it opened (an idle stretch emits no line, hence `window_ms`), and [`flush`] emits the partial window when the source is deselected. -//! `tools/tv-sched-trace.sh --tid ` follows the worker. //! -//! Always compiled; the cost is one clock pair per render and a fixed 256-slot ring. +//! Always compiled; the cost is one clock pair per render in Rust, six clock reads in the facade +//! (three on each of the two clocks) and a fixed 256-slot ring per figure. +use super::ass::RenderScale; use std::cell::{Cell, RefCell}; use std::sync::atomic::{AtomicU64, Ordering}; use std::time::{Duration, Instant}; @@ -34,17 +65,71 @@ use std::time::{Duration, Instant}; const RING: usize = 256; const WINDOW: Duration = Duration::from_secs(1); -static UPLOADED: AtomicU64 = AtomicU64::new(0); -static AHEAD_HITS: AtomicU64 = AtomicU64::new(0); +/// What the UI thread counts between two lines. One owner reads and restarts it (the primary +/// worker, [`finish_call`]): were every worker to, a window closing on one would take the counts of +/// them all, and the rest would print zeros for a stretch that had them. +struct Counters { + uploaded: AtomicU64, + ahead_hits: AtomicU64, + shown: AtomicU64, + late: AtomicU64, +} + +impl Counters { + const fn new() -> Self { + Self { + uploaded: AtomicU64::new(0), + ahead_hits: AtomicU64::new(0), + shown: AtomicU64::new(0), + late: AtomicU64::new(0), + } + } + + fn read(&self) -> Shared { + Shared { + up_bytes: self.uploaded.load(Ordering::Relaxed), + ahead_hits: self.ahead_hits.load(Ordering::Relaxed), + shown: self.shown.load(Ordering::Relaxed), + late: self.late.load(Ordering::Relaxed), + } + } + + /// Restart them, after a window that [`Counters::read`] them. Events landing between the read + /// and the restart are dropped (a diagnostic line, not an account). + fn reset(&self) { + for counter in [&self.uploaded, &self.ahead_hits, &self.shown, &self.late] { + counter.store(0, Ordering::Relaxed); + } + } + + /// Read and restart in one step: the partial window of a deselected source. + fn take(&self) -> Shared { + Shared { + up_bytes: self.uploaded.swap(0, Ordering::Relaxed), + ahead_hits: self.ahead_hits.swap(0, Ordering::Relaxed), + shown: self.shown.swap(0, Ordering::Relaxed), + late: self.late.swap(0, Ordering::Relaxed), + } + } +} + +static COUNTERS: Counters = Counters::new(); /// UI thread: RGBA bytes handed to `glTexImage2D` for a styled-subtitle region. pub fn note_upload(bytes: usize) { - UPLOADED.fetch_add(bytes as u64, Ordering::Relaxed); + COUNTERS.uploaded.fetch_add(bytes as u64, Ordering::Relaxed); } -/// UI thread: the ahead frame was promoted without a worker job. +/// UI thread: the ahead frame of a still cue was promoted without a worker job. pub fn note_ahead_hit() { - AHEAD_HITS.fetch_add(1, Ordering::Relaxed); + COUNTERS.ahead_hits.fetch_add(1, Ordering::Relaxed); +} + +/// UI thread: a frame of the moving cue's pipeline was promoted; `late` when it came more than a +/// tick after its playhead. +pub fn note_shown(late: bool) { + COUNTERS.shown.fetch_add(1, Ordering::Relaxed); + COUNTERS.late.fetch_add(u64::from(late), Ordering::Relaxed); } /// Fixed ring of the latest samples with a nearest-rank percentile. @@ -80,17 +165,42 @@ impl Ring { } } +/// What the facade reports of the `plx_ass_render` call just made (`PlxAssStats::last_*`). +#[derive(Clone, Copy, Debug, Default, PartialEq, Eq)] +pub(super) struct NativeCost { + /// `ass_render_frame` alone. + pub lib_us: u32, + /// Everything after it until the frame was returned: 0 when the call found it unchanged. + pub comp_us: u32, + /// The worker thread's CPU time inside the same two phases (`CLOCK_THREAD_CPUTIME_ID`), the + /// wall figures above minus the time the thread was not running. + pub lib_cpu_us: u32, + pub comp_cpu_us: u32, + /// `ASS_Image`s in the frame; only meaningful when `composed`. + pub images: u32, + /// The sum of their width x height; only meaningful when `composed`. + pub image_pixels: u64, + /// The call returned a changed frame, so the two counts above are this frame's. + pub composed: bool, +} + /// One finished worker cycle. #[derive(Clone, Copy)] pub(super) struct Cycle { /// The `plx_ass_render` call, if the cycle made one. pub render_us: Option, + /// The facade's own account of that call, when it returned a result and could be asked. + pub native: Option, /// The whole `Engine::render` job. pub job_us: u32, pub changed: bool, pub pub_bytes: u64, /// An Ahead job (rendered early) rather than a Normal one. pub ahead: bool, + /// The frame it rendered was thrown away when it landed (the plan had moved on). + pub stale: bool, + /// The size it drew at. + pub scale: RenderScale, } /// Counters other threads keep, read when a window closes. @@ -98,21 +208,34 @@ pub(super) struct Cycle { pub(super) struct Shared { pub up_bytes: u64, pub ahead_hits: u64, + pub shown: u64, + pub late: u64, } pub(super) struct Aggregator { + /// Which worker this window belongs to; a window restart keeps it. + slot: usize, started: Option, render: Ring, job: Ring, nwait: Ring, + lib: Ring, + comp: Ring, + lib_cpu: Ring, + comp_cpu: Ring, + images: Ring, + image_kpx: Ring, normal_jobs: u32, ahead_jobs: u32, + stale: u32, nwait_max_us: u32, calls: u32, changed: u32, max_us: u32, job_max_us: u32, pub_bytes: u64, + /// The smallest size a job of the window was drawn at. + scale: RenderScale, /// When the counters (these and the cross-thread ones) last restarted: the previous line, or /// the first job of a session. Survives a line so `window_ms` spans the idle gap too. counting_since: Option, @@ -121,18 +244,27 @@ pub(super) struct Aggregator { impl Aggregator { pub(super) const fn new() -> Self { Self { + slot: 0, started: None, render: Ring::new(), job: Ring::new(), nwait: Ring::new(), + lib: Ring::new(), + comp: Ring::new(), + lib_cpu: Ring::new(), + comp_cpu: Ring::new(), + images: Ring::new(), + image_kpx: Ring::new(), normal_jobs: 0, ahead_jobs: 0, + stale: 0, nwait_max_us: 0, calls: 0, changed: 0, max_us: 0, job_max_us: 0, pub_bytes: 0, + scale: RenderScale::Full, counting_since: None, } } @@ -146,6 +278,17 @@ impl Aggregator { self.calls += 1; self.max_us = self.max_us.max(us); } + if let Some(cost) = cycle.native { + self.lib.push(cost.lib_us); + self.comp.push(cost.comp_us); + self.lib_cpu.push(cost.lib_cpu_us); + self.comp_cpu.push(cost.comp_cpu_us); + if cost.composed { + self.images.push(cost.images); + self.image_kpx + .push((cost.image_pixels / 1000).min(u64::from(u32::MAX)) as u32); + } + } self.job.push(cycle.job_us); if cycle.ahead { self.ahead_jobs += 1; @@ -153,8 +296,12 @@ impl Aggregator { self.normal_jobs += 1; } self.job_max_us = self.job_max_us.max(cycle.job_us); + self.stale += u32::from(cycle.stale); self.changed += u32::from(cycle.changed); self.pub_bytes += cycle.pub_bytes; + if cycle.scale.factor() < self.scale.factor() { + self.scale = cycle.scale; + } if now.saturating_duration_since(started) < WINDOW { return None; } @@ -181,7 +328,8 @@ impl Aggregator { .counting_since .map_or(0, |since| now.saturating_duration_since(since).as_millis()); let line = format!( - "assperf: window_ms={} calls={} changed={} p95_us={} max_us={} job_p95_us={} job_max_us={} pub_bytes={} up_bytes={} jobs={}+{} ahead_hit={} nwait_p95_us={} nwait_max_us={}", + "assperf: w={} window_ms={} calls={} changed={} p95_us={} max_us={} job_p95_us={} job_max_us={} pub_bytes={} up_bytes={} jobs={}+{} ahead_hit={} shown={} late={} stale={} nwait_p95_us={} nwait_max_us={} lib_p95_us={} comp_p95_us={} lib_cpu_p95_us={} comp_cpu_p95_us={} images_p95={} img_kpx_p95={} scale={}", + self.slot, window_ms, self.calls, self.changed, @@ -194,10 +342,22 @@ impl Aggregator { self.normal_jobs, self.ahead_jobs, shared.ahead_hits, + shared.shown, + shared.late, + self.stale, self.nwait.p95(), - self.nwait_max_us + self.nwait_max_us, + self.lib.p95(), + self.comp.p95(), + self.lib_cpu.p95(), + self.comp_cpu.p95(), + self.images.p95(), + self.image_kpx.p95(), + self.scale.label() ); + let slot = self.slot; *self = Self::new(); + self.slot = slot; self.counting_since = Some(now); line } @@ -206,8 +366,12 @@ impl Aggregator { thread_local! { static AGG: RefCell = const { RefCell::new(Aggregator::new()) }; static LAST_RENDER_US: Cell> = const { Cell::new(None) }; + static LAST_NATIVE: Cell> = const { Cell::new(None) }; static LAST_JOB_US: Cell> = const { Cell::new(None) }; static LAST_JOB_AHEAD: Cell = const { Cell::new(false) }; + static LAST_JOB_SCALE: Cell = const { Cell::new(RenderScale::Full) }; + /// The worker this thread is (0 = the primary, which owns the shared counters). + static SLOT: Cell = const { Cell::new(0) }; } fn micros(elapsed: Duration) -> u32 { @@ -217,6 +381,7 @@ fn micros(elapsed: Duration) -> u32 { /// WORKER: a request was taken; drop any sample a cancelled cycle left behind. pub(super) fn begin_call() { LAST_RENDER_US.set(None); + LAST_NATIVE.set(None); LAST_JOB_US.set(None); } @@ -225,10 +390,16 @@ pub(super) fn record_render(elapsed: Duration) { LAST_RENDER_US.set(Some(micros(elapsed))); } -/// WORKER: remember the duration of the whole `Engine::render` job just run. -pub(super) fn record_job(elapsed: Duration, ahead: bool) { +/// WORKER: remember the facade's phase split of the `plx_ass_render` call just made. +pub(super) fn record_native(cost: NativeCost) { + LAST_NATIVE.set(Some(cost)); +} + +/// WORKER: remember the duration of the whole `Engine::render` job just run, and the size it drew at. +pub(super) fn record_job(elapsed: Duration, ahead: bool, scale: RenderScale) { LAST_JOB_US.set(Some(micros(elapsed))); LAST_JOB_AHEAD.set(ahead); + LAST_JOB_SCALE.set(scale); } /// WORKER: a Normal job started `waited` after it was queued behind an Ahead job in flight. @@ -236,41 +407,39 @@ pub(super) fn note_nwait(waited: Duration) { AGG.with(|a| a.borrow_mut().note_nwait(micros(waited))); } -/// Push one cycle and, when its window closes, restart the cross-thread counters. Reset only on -/// emission; events landing between the load and the store are dropped (a diagnostic line, not -/// an account). -fn close_cycle( - agg: &mut Aggregator, - now: Instant, - cycle: Cycle, - uploaded: &AtomicU64, - ahead_hits: &AtomicU64, -) -> Option { - let shared = Shared { - up_bytes: uploaded.load(Ordering::Relaxed), - ahead_hits: ahead_hits.load(Ordering::Relaxed), - }; +/// Push one cycle and, when its window closes, restart the cross-thread counters: `counters` is +/// the owner's (the primary worker's); a helper passes `None` and its line reads zeros there. +fn close_cycle(agg: &mut Aggregator, now: Instant, cycle: Cycle, counters: Option<&Counters>) -> Option { + let shared = counters.map_or_else(Shared::default, Counters::read); let line = agg.push(now, cycle, shared)?; - uploaded.store(0, Ordering::Relaxed); - ahead_hits.store(0, Ordering::Relaxed); + if let Some(counters) = counters { + counters.reset(); + } Some(line) } -/// WORKER: close one cycle (if a job was timed) and log the window when due. -pub(super) fn finish_call(changed: bool, pub_bytes: u64) { +/// The counters this thread owns: the primary worker's. +fn owned_counters() -> Option<&'static Counters> { + (SLOT.get() == 0).then_some(&COUNTERS) +} + +/// WORKER: close one cycle (if a job was timed) and log the window when due. `stale`: the frame +/// the job rendered was thrown away when it landed. +pub(super) fn finish_call(changed: bool, pub_bytes: u64, stale: bool) { let Some(job_us) = LAST_JOB_US.take() else { return; }; let cycle = Cycle { render_us: LAST_RENDER_US.take(), + native: LAST_NATIVE.take(), job_us, changed, pub_bytes, ahead: LAST_JOB_AHEAD.take(), + stale, + scale: LAST_JOB_SCALE.take(), }; - let line = AGG.with(|a| { - close_cycle(&mut a.borrow_mut(), Instant::now(), cycle, &UPLOADED, &AHEAD_HITS) - }); + let line = AGG.with(|a| close_cycle(&mut a.borrow_mut(), Instant::now(), cycle, owned_counters())); if let Some(line) = line { plx_base::eventlog::log(&line); } @@ -279,41 +448,57 @@ pub(super) fn finish_call(changed: bool, pub_bytes: u64) { /// WORKER: the source was deselected; emit the partial window and start the next session clean. pub(super) fn flush() { let line = AGG.with(|a| { - a.borrow_mut().take_partial(Instant::now(), Shared { - up_bytes: UPLOADED.swap(0, Ordering::Relaxed), - ahead_hits: AHEAD_HITS.swap(0, Ordering::Relaxed), - }) + let mut agg = a.borrow_mut(); + // Only a window with something in it takes the owner's counts: an idle helper's flush must + // not leave the primary's partial window without them. + let shared = if agg.job.pushed > 0 { owned_counters().map_or_else(Shared::default, Counters::take) } else { Shared::default() }; + agg.take_partial(Instant::now(), shared) }); if let Some(line) = line { plx_base::eventlog::log(&line); } } -/// WORKER: once, at thread start. -pub(super) fn log_worker_tid() { - #[cfg(any(target_os = "linux", target_os = "android"))] - let tid = unsafe { libc::syscall(libc::SYS_gettid) }.to_string(); - #[cfg(not(any(target_os = "linux", target_os = "android")))] - let tid = String::from("na"); - plx_base::eventlog::log(&format!("assperf: worker tid={tid}")); +/// WORKER: once, at thread start: which worker this thread is (`slot`, 0 for the primary), its +/// kernel thread id, which `tools/tv-sched-trace.sh --tid` follows, and the niceness it now runs at +/// (absent when the platform has none or the kernel refused it). +pub(super) fn log_worker_tid(slot: usize, nice: Option) { + SLOT.set(slot); + AGG.with(|a| a.borrow_mut().slot = slot); + let tid = plx_base::task::thread_id().map_or_else(|| String::from("na"), |t| t.to_string()); + let nice = nice.map_or_else(String::new, |n| format!(" nice={n}")); + plx_base::eventlog::log(&format!("assperf: worker w={slot} tid={tid}{nice}")); } #[cfg(test)] mod tests { use super::*; + /// A call whose time splits evenly between libass and the composite; a changed one is a + /// composed frame of `render_us / 10` images and `render_us` thousand pixels. fn cycle(render_us: u32, changed: bool, pub_bytes: u64) -> Cycle { Cycle { render_us: Some(render_us), + native: Some(NativeCost { + lib_us: render_us / 2, + comp_us: render_us / 2, + lib_cpu_us: render_us / 4, + comp_cpu_us: render_us / 4, + images: render_us / 10, + image_pixels: u64::from(render_us) * 1000, + composed: changed, + }), job_us: render_us * 2, changed, pub_bytes, ahead: false, + stale: false, + scale: RenderScale::Full, } } fn up(up_bytes: u64) -> Shared { - Shared { up_bytes, ahead_hits: 0 } + Shared { up_bytes, ..Shared::default() } } #[test] @@ -321,17 +506,19 @@ mod tests { let t0 = Instant::now(); let mut a = Aggregator::new(); for i in 1..=200u32 { + let mut call = cycle((i % 100 + 1) * 10, i % 10 == 0, 100); + call.stale = i % 50 == 0; let line = a.push( t0 + Duration::from_micros((i as u64 - 1) * 5030), - cycle((i % 100 + 1) * 10, i % 10 == 0, 100), - Shared { up_bytes: 777, ahead_hits: 3 }, + call, + Shared { up_bytes: 777, ahead_hits: 3, shown: 140, late: 9 }, ); if i < 200 { assert!(line.is_none()); } else { assert_eq!( line.as_deref(), - Some("assperf: window_ms=1000 calls=200 changed=20 p95_us=950 max_us=1000 job_p95_us=1900 job_max_us=2000 pub_bytes=20000 up_bytes=777 jobs=200+0 ahead_hit=3 nwait_p95_us=0 nwait_max_us=0") + Some("assperf: w=0 window_ms=1000 calls=200 changed=20 p95_us=950 max_us=1000 job_p95_us=1900 job_max_us=2000 pub_bytes=20000 up_bytes=777 jobs=200+0 ahead_hit=3 shown=140 late=9 stale=4 nwait_p95_us=0 nwait_max_us=0 lib_p95_us=475 comp_p95_us=475 lib_cpu_p95_us=237 comp_cpu_p95_us=237 images_p95=91 img_kpx_p95=910 scale=1.00") ); } } @@ -344,18 +531,18 @@ mod tests { let mut a = Aggregator::new(); assert!(a.push(t0, cycle(10, true, 1), up(0)).is_none()); let first = a.push(t0 + WINDOW, cycle(10, true, 1), up(0)).unwrap(); - assert!(first.starts_with("assperf: window_ms=1000 calls=2 "), "{first}"); + assert!(first.starts_with("assperf: w=0 window_ms=1000 calls=2 "), "{first}"); // Ten idle seconds, then a job and, a second later, the job that closes the window. assert!(a.push(t0 + 11 * WINDOW, cycle(10, true, 1), up(0)).is_none()); let second = a.push(t0 + 12 * WINDOW, cycle(10, true, 1), up(0)).unwrap(); - assert!(second.starts_with("assperf: window_ms=11000 calls=2 "), "{second}"); + assert!(second.starts_with("assperf: w=0 window_ms=11000 calls=2 "), "{second}"); // A partial window at deselection ends the session: the next one starts its own clock. assert!(a.push(t0 + 20 * WINDOW, cycle(10, true, 1), up(0)).is_none()); let partial = a.take_partial(t0 + 20 * WINDOW + WINDOW / 2, up(0)).unwrap(); - assert!(partial.starts_with("assperf: window_ms=8500 "), "{partial}"); + assert!(partial.starts_with("assperf: w=0 window_ms=8500 "), "{partial}"); assert!(a.push(t0 + 30 * WINDOW, cycle(10, true, 1), up(0)).is_none()); let next = a.take_partial(t0 + 30 * WINDOW, up(0)).unwrap(); - assert!(next.starts_with("assperf: window_ms=0 "), "{next}"); + assert!(next.starts_with("assperf: w=0 window_ms=0 "), "{next}"); } #[test] @@ -378,10 +565,13 @@ mod tests { let mut a = Aggregator::new(); let load = Cycle { render_us: None, + native: None, job_us: 50_000, changed: true, pub_bytes: 8, ahead: true, + stale: true, + scale: RenderScale::Half, }; assert!(a.push(t0, load, up(0)).is_none()); a.note_nwait(300); @@ -389,10 +579,82 @@ mod tests { let line = a.take_partial(t0, up(0)).unwrap(); assert_eq!( line, - "assperf: window_ms=0 calls=0 changed=1 p95_us=0 max_us=0 job_p95_us=50000 job_max_us=50000 pub_bytes=8 up_bytes=0 jobs=0+1 ahead_hit=0 nwait_p95_us=900 nwait_max_us=900" + "assperf: w=0 window_ms=0 calls=0 changed=1 p95_us=0 max_us=0 job_p95_us=50000 job_max_us=50000 pub_bytes=8 up_bytes=0 jobs=0+1 ahead_hit=0 shown=0 late=0 stale=1 nwait_p95_us=900 nwait_max_us=900 lib_p95_us=0 comp_p95_us=0 lib_cpu_p95_us=0 comp_cpu_p95_us=0 images_p95=0 img_kpx_p95=0 scale=0.50" ); } + #[test] + fn the_native_split_prints_its_own_percentiles() { + let t0 = Instant::now(); + let mut a = Aggregator::new(); + // The thread's CPU time is half the wall time in both phases: an even split, so the CPU + // percentiles are the wall ones halved and any mix-up of the four rings shows. + let call = |lib_us, comp_us, images, image_pixels, composed| Cycle { + render_us: Some(lib_us + comp_us), + native: Some(NativeCost { + lib_us, + comp_us, + lib_cpu_us: lib_us / 2, + comp_cpu_us: comp_us / 2, + images, + image_pixels, + composed, + }), + job_us: lib_us + comp_us, + changed: composed, + pub_bytes: 0, + ahead: false, + stale: false, + scale: RenderScale::Full, + }; + // Twenty composed frames: libass 100..=2000 us, the composite 5x that, 10..=200 images of + // 1.5 Mpx..30 Mpx; then an unchanged call, which has no composite and whose stale image + // counts must not enter the image rings. + for i in 1..=20u32 { + let cost = call(i * 100, i * 500, i * 10, u64::from(i) * 1_500_000, true); + assert!(a.push(t0, cost, up(0)).is_none()); + } + assert!(a.push(t0, call(40, 0, 9999, 999_999_999, false), up(0)).is_none()); + let line = a.take_partial(t0, up(0)).unwrap(); + // Nearest rank: ceil(21 * 0.95) = 20th of the 21 libass samples (40 sorts first), the 20th + // of the 21 composite samples (the 0 sorts first); 19th of the 20 composed frames. + assert!( + line.ends_with( + " lib_p95_us=1900 comp_p95_us=9500 lib_cpu_p95_us=950 comp_cpu_p95_us=4750 images_p95=190 img_kpx_p95=28500 scale=1.00" + ), + "{line}" + ); + } + + #[test] + fn a_window_reports_the_smallest_size_a_job_in_it_was_drawn_at_and_the_next_one_starts_over() { + let t0 = Instant::now(); + let mut a = Aggregator::new(); + let at = |scale| Cycle { scale, ..cycle(10, true, 1) }; + // the first frame of a stretch at full size, then the pipeline's at two thirds and at half + for scale in [RenderScale::Full, RenderScale::TwoThirds, RenderScale::Half, RenderScale::Full] { + assert!(a.push(t0, at(scale), up(0)).is_none()); + } + let line = a.take_partial(t0, up(0)).unwrap(); + assert!(line.ends_with(" scale=0.50"), "{line}"); + assert!(a.push(t0, at(RenderScale::TwoThirds), up(0)).is_none()); + let line = a.take_partial(t0, up(0)).unwrap(); + assert!(line.ends_with(" scale=0.67"), "the window starts over at full size: {line}"); + assert!(a.push(t0, at(RenderScale::Full), up(0)).is_none()); + assert!(a.take_partial(t0, up(0)).unwrap().ends_with(" scale=1.00")); + } + + #[test] + fn a_huge_pixel_count_saturates_the_kilopixel_figure() { + let t0 = Instant::now(); + let mut a = Aggregator::new(); + let mut cost = cycle(10, true, 0); + cost.native.as_mut().unwrap().image_pixels = u64::MAX; + assert!(a.push(t0, cost, up(0)).is_none()); + let line = a.take_partial(t0, up(0)).unwrap(); + assert!(line.ends_with(" img_kpx_p95=4294967295 scale=1.00"), "{line}"); + } + #[test] fn take_partial_emits_the_open_window_once_and_resets() { let t0 = Instant::now(); @@ -407,39 +669,74 @@ mod tests { #[test] fn closing_a_window_restarts_the_upload_counter() { let t0 = Instant::now(); - let uploaded = AtomicU64::new(0); - let hits = AtomicU64::new(0); + let counters = Counters::new(); let mut a = Aggregator::new(); - uploaded.fetch_add(4096, Ordering::Relaxed); - assert!(close_cycle(&mut a, t0, cycle(10, true, 1), &uploaded, &hits).is_none()); - assert_eq!(uploaded.load(Ordering::Relaxed), 4096, "mid-window: kept"); - let line = close_cycle(&mut a, t0 + WINDOW, cycle(10, false, 0), &uploaded, &hits).unwrap(); + counters.uploaded.fetch_add(4096, Ordering::Relaxed); + assert!(close_cycle(&mut a, t0, cycle(10, true, 1), Some(&counters)).is_none()); + assert_eq!(counters.read().up_bytes, 4096, "mid-window: kept"); + let line = close_cycle(&mut a, t0 + WINDOW, cycle(10, false, 0), Some(&counters)).unwrap(); assert!(line.contains("up_bytes=4096 "), "{line}"); - assert_eq!(uploaded.load(Ordering::Relaxed), 0, "next window starts at 0"); - uploaded.fetch_add(7, Ordering::Relaxed); - assert!(close_cycle(&mut a, t0 + 3 * WINDOW, cycle(10, false, 0), &uploaded, &hits).is_none()); - let line = close_cycle(&mut a, t0 + 5 * WINDOW, cycle(10, false, 0), &uploaded, &hits).unwrap(); + assert_eq!(counters.read().up_bytes, 0, "next window starts at 0"); + counters.uploaded.fetch_add(7, Ordering::Relaxed); + assert!(close_cycle(&mut a, t0 + 3 * WINDOW, cycle(10, false, 0), Some(&counters)).is_none()); + let line = close_cycle(&mut a, t0 + 5 * WINDOW, cycle(10, false, 0), Some(&counters)).unwrap(); assert!(line.contains("up_bytes=7 "), "{line}"); } #[test] - fn closing_a_window_reports_and_restarts_the_ahead_hits() { + fn closing_a_window_reports_and_restarts_the_ahead_hits_and_the_shown_frames() { let t0 = Instant::now(); - let (uploaded, hits) = (AtomicU64::new(0), AtomicU64::new(0)); + let counters = Counters::new(); let mut a = Aggregator::new(); - hits.fetch_add(2, Ordering::Relaxed); - let line = close_cycle(&mut a, t0, cycle(10, true, 1), &uploaded, &hits); - assert!(line.is_none() && hits.load(Ordering::Relaxed) == 2, "mid-window: kept"); - let line = close_cycle(&mut a, t0 + WINDOW, cycle(10, false, 0), &uploaded, &hits).unwrap(); - assert!(line.contains("ahead_hit=2 "), "{line}"); - assert_eq!(hits.load(Ordering::Relaxed), 0); + counters.ahead_hits.fetch_add(2, Ordering::Relaxed); + counters.shown.fetch_add(30, Ordering::Relaxed); + counters.late.fetch_add(4, Ordering::Relaxed); + let line = close_cycle(&mut a, t0, cycle(10, true, 1), Some(&counters)); + assert!(line.is_none() && counters.read().ahead_hits == 2, "mid-window: kept"); + let line = close_cycle(&mut a, t0 + WINDOW, cycle(10, false, 0), Some(&counters)).unwrap(); + assert!(line.contains("ahead_hit=2 shown=30 late=4 stale=0 "), "{line}"); + let after = counters.read(); + assert_eq!((after.ahead_hits, after.shown, after.late), (0, 0, 0)); + } + + #[test] + fn a_helper_names_its_worker_and_neither_reads_nor_restarts_the_shared_counts() { + let t0 = Instant::now(); + let counters = Counters::new(); + counters.uploaded.fetch_add(5, Ordering::Relaxed); + note_shown_into(&counters, true); + let mut helper = Aggregator::new(); + helper.slot = 2; + let mut stale = cycle(10, true, 1); + stale.stale = true; + assert!(close_cycle(&mut helper, t0, stale, None).is_none()); + let line = close_cycle(&mut helper, t0 + WINDOW, cycle(10, false, 0), None).unwrap(); + assert!(line.starts_with("assperf: w=2 window_ms=1000 "), "{line}"); + assert!(line.contains(" up_bytes=0 jobs=2+0 ahead_hit=0 shown=0 late=0 stale=1 "), "{line}"); + let owner = counters.read(); + assert_eq!((owner.up_bytes, owner.shown, owner.late), (5, 1, 1), "left for the owner"); + // the next window of the helper keeps its number + assert!(close_cycle(&mut helper, t0 + 2 * WINDOW, cycle(10, true, 1), None).is_none()); + let again = close_cycle(&mut helper, t0 + 3 * WINDOW, cycle(10, true, 1), None).unwrap(); + assert!(again.starts_with("assperf: w=2 "), "{again}"); + // the owner then takes them with its own line + let mut primary = Aggregator::new(); + assert!(close_cycle(&mut primary, t0, cycle(10, true, 1), Some(&counters)).is_none()); + let line = close_cycle(&mut primary, t0 + WINDOW, cycle(10, true, 1), Some(&counters)).unwrap(); + assert!(line.starts_with("assperf: w=0 ") && line.contains(" up_bytes=5 jobs=2+0 ahead_hit=0 shown=1 late=1 "), "{line}"); + } + + /// `note_shown` against `counters` instead of the process's: the same two counts. + fn note_shown_into(counters: &Counters, late: bool) { + counters.shown.fetch_add(1, Ordering::Relaxed); + counters.late.fetch_add(u64::from(late), Ordering::Relaxed); } #[test] fn finish_without_a_job_is_a_no_op() { // Idle (nothing ran this cycle): nothing is recorded, nothing logged. begin_call(); - finish_call(true, 99); + finish_call(true, 99, false); AGG.with(|a| assert_eq!(a.borrow().job.pushed, 0)); } @@ -447,18 +744,36 @@ mod tests { fn render_then_finish_counts_one_call() { begin_call(); record_render(Duration::from_micros(1234)); - record_job(Duration::from_micros(2000), false); - finish_call(true, 64); + record_native(NativeCost { + lib_us: 400, + comp_us: 800, + lib_cpu_us: 390, + comp_cpu_us: 700, + images: 12, + image_pixels: 345_678, + composed: true, + }); + record_job(Duration::from_micros(2000), false, RenderScale::Full); + finish_call(true, 64, false); AGG.with(|a| { let a = a.borrow(); assert_eq!( (a.calls, a.changed, a.max_us, a.job_max_us, a.pub_bytes), (1, 1, 1234, 2000, 64) ); + assert_eq!( + (a.lib.p95(), a.comp.p95(), a.images.p95(), a.image_kpx.p95()), + (400, 800, 12, 345) + ); + assert_eq!( + (a.lib_cpu.p95(), a.comp_cpu.p95()), + (390, 700), + "the CPU rings are fed from their own fields" + ); }); // The slots were consumed: a second finish does not double count. - finish_call(false, 0); - AGG.with(|a| assert_eq!(a.borrow().calls, 1)); + finish_call(false, 0, false); + AGG.with(|a| assert_eq!((a.borrow().calls, a.borrow().lib.pushed), (1, 1))); } #[test] @@ -467,10 +782,14 @@ mod tests { // Cycle 1 timed a render, then the epoch changed: it never reached finish_call. begin_call(); record_render(Duration::from_micros(9)); - record_job(Duration::from_micros(9), false); + record_native(NativeCost { lib_us: 4, comp_us: 5, ..NativeCost::default() }); + record_job(Duration::from_micros(9), false, RenderScale::Full); // Cycle 2 made no native call at all. begin_call(); - finish_call(false, 0); - AGG.with(|a| assert_eq!((a.borrow().calls, a.borrow().job.pushed), (0, 0))); + finish_call(false, 0, false); + AGG.with(|a| { + let a = a.borrow(); + assert_eq!((a.calls, a.job.pushed, a.lib.pushed, a.comp.pushed), (0, 0, 0, 0)); + }); } } diff --git a/rust-modules/media/src/player/ass_source.rs b/rust-modules/media/src/player/ass_source.rs index 293f11015a..cf817f648d 100644 --- a/rust-modules/media/src/player/ass_source.rs +++ b/rust-modules/media/src/player/ass_source.rs @@ -15,10 +15,11 @@ use std::sync::{Arc, MutexGuard}; // What can be shown is never cut by a count. Every ASS track of the file keeps its header (a // copy of bytes FFmpeg already holds, so memory is bounded by the file's own stream headers) and -// every attached font is admitted until the font byte budget (`ass::FontBudget`) is spent. What -// is bounded is per-track EVENT memory and the per-push work: a forward window of -// `MAX_EVENTS` events and `MAX_EVENT_BYTES` bytes, of which the selected track may use a whole -// script's worth and the others share one pool of the same size. +// every attached font is kept whole (the same, bounded by the file's own attachments). No byte +// budget decides which font a track can use (issue #1328): the renderer hands libass the fonts the +// track names (`ass::font_names`). What is bounded is per-track EVENT memory and the per-push work: +// a forward window of `MAX_EVENTS` events and `MAX_EVENT_BYTES` bytes, of which the selected track +// may use a whole script's worth and the others share one pool of the same size. pub const MAX_HEADER_BYTES: usize = 1024 * 1024; pub(crate) const MAX_PACKET_BYTES: usize = 256 * 1024; const MAX_EVENTS: usize = 8192; @@ -30,6 +31,14 @@ const OTHER_TRACKS_EVENT_BYTES: usize = 8 * 1024 * 1024; const STALE_AHEAD_MS: i64 = 5_000; const MAX_MEDIA_KEY_BYTES: usize = 64 * 1024; +/// The font list of a reopen of the same file is the one already held: same count, and the same +/// name and size at each place. A stronger test would read every byte of fonts that are, by the +/// file's identity, the same bytes. +fn same_font_list(held: &[Font], fresh: &[Font]) -> bool { + held.len() == fresh.len() + && held.iter().zip(fresh).all(|(a, b)| a.name == b.name && a.data.len() == b.data.len()) +} + fn trim_terminators(mut bytes: &[u8]) -> &[u8] { while bytes.last() == Some(&0) { bytes = &bytes[..bytes.len() - 1]; @@ -359,14 +368,17 @@ impl Store { if !media_key.is_empty() && media_key.len() <= MAX_MEDIA_KEY_BYTES { self.media_key = Some(media_key.to_owned()); } - // A font that does not fit the budget is skipped (a later, smaller one still fits); - // text that names it falls back to the app's default face, so nothing is hidden. - let mut budget = ass::FontBudget::default(); - let fonts: Arc<[Font]> = fonts - .into_iter() - .filter(|f| budget.admit(f.data.len())) - .collect::>() - .into(); + // Every attached font is kept (issue #1328): which of them libass is handed is the + // renderer's choice, by the names the track uses (`ass::font_names`). A reopen of the same + // file keeps the font list the tracks already hold, so the renderer's same-fonts check is a + // pointer compare and not a read of every font's bytes. + let held = previous.values().find_map(|source| match &source.content { + Content::Embedded { fonts, .. } | Content::Script { fonts, .. } => Some(fonts.clone()), + }); + let fonts: Arc<[Font]> = match held { + Some(held) if same_font_list(&held, &fonts) => held, + _ => fonts.into(), + }; self.fonts = Some(fonts.clone()); for (track, registered) in headers { let (origin, header, events) = match registered { @@ -1167,21 +1179,72 @@ mod tests { assert!(held.iter().any(|f| f.name == "f199.ttf"), "a font late in the list is found"); } + /// A demux reopen of the same file hands `begin` a fresh copy of every font. The tracks keep the + /// list they already hold, so the renderer's same-fonts check (`Arc` equality, a pointer + /// compare when it is the same `Arc`) neither reads every byte nor opens a new renderer; a + /// different file, or a list that differs in a name or a size, is taken as it comes. + #[test] + fn a_reopen_of_the_same_file_keeps_the_font_list_it_already_held() { + fn held(s: &mut Store) -> Arc<[Font]> { + match &s.selected(0).unwrap().content { + Content::Embedded { fonts, .. } | Content::Script { fonts, .. } => fonts.clone(), + } + } + let font = |name: &str, len: usize| Font { name: name.into(), data: Arc::from(vec![1u8; len]) }; + let headers = || vec![(0, TrackHeader::Authored(b"header".to_vec()))]; + let mut s = Store::new(); + s.begin("fixture", headers(), vec![font("a.ttf", 4096), font("b.ttf", 8192)]); + let first = held(&mut s); + s.retain_for_reload(); + s.begin("fixture", headers(), vec![font("a.ttf", 4096), font("b.ttf", 8192)]); + assert!(Arc::ptr_eq(&first, &held(&mut s)), "same file, same fonts: the held list is kept"); + s.begin("fixture", headers(), vec![font("a.ttf", 4096), font("b.ttf", 9000)]); + let changed = held(&mut s); + assert!(!Arc::ptr_eq(&first, &changed), "a font of another size is another list"); + assert_eq!(changed[1].data.len(), 9000); + s.begin("other file", headers(), vec![font("b.ttf", 9000)]); + assert_eq!(held(&mut s).len(), 1, "another file starts from its own fonts"); + } + + /// Issue #1328: an anime MKV attaches 17 fonts (83 MB). The five the opening's karaoke styles + /// name sat after the first seven had spent a 32 MiB cap (since removed; twelve were held in + /// all), so libass never saw them and those lines fell back to the default face. A cap on + /// content the file legitimately carries is a bug, so the store must hold every attachment, in + /// order; which of them libass is given is the renderer's choice (`ass::font_names`). + /// Sizes and names are the report's (`tests/fixtures/make_ass_stress.py`, `fonts-over-budget`). #[test] - fn fonts_past_the_byte_budget_are_skipped_not_fatal() { + fn every_attached_font_reaches_the_renderer_issue_1328() { + const ATTACHED: [(&str, usize); 17] = [ + ("AP-OTF-KizaKinryouStd-Bol.otf", 6_220_620), + ("Comiccity.ttf", 32_904), + ("DFMaruGothicPro6N-W9.otf", 11_229_304), + ("FOT-Accordion.otf", 37_300), + ("FOT-DotGothic12 Std M.ttf", 43_840), + ("FOT-MatissePro-B.otf", 9_883_924), + ("FOT-PopFury Std B.otf", 4_808_652), + ("FOT-TelopMinPro-D.otf", 7_477_668), + ("FOT-TsukuAOldMinPr6N-M.otf", 13_188_132), + ("HOT-KarakusaStd-R.otf", 6_102_048), + ("KizaJJK-B.ttf", 9_705_124), + ("KizaKinryouStdJJK-B.ttf", 13_715_688), + ("LEHN084.TTF", 47_816), + ("MastroSans-SemiBold.ttf", 245_796), + ("MastroSans-SemiBoldItalic.ttf", 262_448), + ("Roboto-Regular.ttf", 146_004), + ("stylus.ttf", 32_972), + ]; + let total: usize = ATTACHED.iter().map(|&(_, len)| len).sum(); + assert!(total > 32 * 1024 * 1024, "the report's file is over the old 32 MiB cap: {total} bytes"); let mut s = Store::new(); - let big = |name: &str, mib: usize| Font { - name: name.into(), - data: Arc::from(vec![1u8; mib * 1024 * 1024]), - }; - let fonts = vec![big("a.ttf", 20), big("b.ttf", 20), big("c.ttf", 4), big("d.ttf", 1)]; + let fonts = ATTACHED + .iter() + .map(|&(name, len)| Font { name: name.into(), data: Arc::from(vec![1u8; len]) }) + .collect(); s.begin("fixture", vec![(0, TrackHeader::Authored(b"header".to_vec()))], fonts); let held = s.fonts.clone().unwrap(); let names: Vec<_> = held.iter().map(|f| f.name.as_str()).collect(); - assert_eq!(names, ["a.ttf", "c.ttf", "d.ttf"], "b does not fit; smaller later fonts still do"); - let total: usize = held.iter().map(|f| f.data.len()).sum(); - assert!(total <= ass::MAX_FONT_BYTES); - assert!(s.selected(0).is_some(), "the track itself is never refused"); + let wanted: Vec<_> = ATTACHED.iter().map(|&(name, _)| name).collect(); + assert_eq!(names, wanted, "every attached font is held, in container order"); } fn timed(n: i64) -> Event { diff --git a/rust-modules/media/src/player/demux_session.rs b/rust-modules/media/src/player/demux_session.rs index 2a1e81dbb4..39f1336d29 100644 --- a/rust-modules/media/src/player/demux_session.rs +++ b/rust-modules/media/src/player/demux_session.rs @@ -36,7 +36,6 @@ pub struct PlayerDemuxSubtitle { subtitle_tracks: Option, subtitle_indices: Vec, bitmap_generation: Option, - font_budget: super::ass::FontBudget, } impl PlayerDemuxSubtitle { @@ -45,7 +44,6 @@ impl PlayerDemuxSubtitle { observer, subtitle_tracks: None, subtitle_indices: Vec::new(), bitmap_generation: None, - font_budget: super::ass::FontBudget::default(), } } } @@ -59,14 +57,11 @@ impl DemuxSubtitle for PlayerDemuxSubtitle { let mut generation = None; self.observer.with_active(&mut || generation = Some(SHARED.bitmap_generation.load(Ordering::Acquire))); self.bitmap_generation = generation; - self.font_budget = super::ass::FontBudget::default(); plan } fn max_header_bytes(&self) -> usize { super::ass_source::MAX_HEADER_BYTES } - fn admit_font(&mut self, bytes: usize) -> bool { self.font_budget.admit(bytes) } - fn begin_sources(&mut self, media_key: &str, manifest: Manifest) -> Option { let tracks = self.subtitle_tracks.as_ref()?; let inventory = super::subtitle_cue::source_inventory(media_key, tracks.kinds(), manifest); diff --git a/rust-modules/media/src/player/mod.rs b/rust-modules/media/src/player/mod.rs index 0ef1404f66..5fe8cdabcd 100644 --- a/rust-modules/media/src/player/mod.rs +++ b/rust-modules/media/src/player/mod.rs @@ -2400,6 +2400,19 @@ pub fn subtitle_draw(ps: &crate::route::PlaybackSession) -> Option<(std::sync::A ass_source::selected(desired_sub_idx()).map(|source| (source, playhead)) } +/// **How the subtitle playhead is moving**, for the renderer's pacing of a moving cue's frames +/// ([`ass::Pace`]): whether the player is presenting, the speed it is applied at (1x with no Feed +/// Load to say), the source's frame rate (0 for a transcode or an unreported rate, which the pace +/// reads as film) and the wall-clock time since the caller's previous tick. +pub fn subtitle_pace(ps: &crate::route::PlaybackSession, tick_ms: i64) -> ass::Pace { + ass::Pace::of( + is_playing(ps), + effective_playback_rate(ps).unwrap_or(RATE_NORMAL), + crate::route::stream_fps(ps), + tick_ms, + ) +} + /// **The file-time position a SIDECAR is asked for at playhead `playhead_ns`.** /// /// A sidecar's timestamps are the Part's. On a direct play the playhead is file time. Over a remux diff --git a/rust-modules/media/src/player/subtitle_cue.rs b/rust-modules/media/src/player/subtitle_cue.rs index 5c3317097d..756aabed97 100644 --- a/rust-modules/media/src/player/subtitle_cue.rs +++ b/rust-modules/media/src/player/subtitle_cue.rs @@ -149,7 +149,8 @@ pub(crate) fn bitmap_cue(display: plx_media_codecs::subtitle::BitmapDisplay, tra /// Install a codec-extracted inventory into the common ASS store. The job has already admitted -/// header and font bytes before copying; this layer gives tracks dialects and source generation. +/// header bytes before copying and collected every font attachment; this layer gives tracks +/// dialects and source generation. pub(crate) fn begin_sources( media_key: &str, kinds: &[PresentationTrack], diff --git a/rust-modules/play-feed/src/demux_clock.rs b/rust-modules/play-feed/src/demux_clock.rs index 1e50e4a44a..afe543ef80 100644 --- a/rust-modules/play-feed/src/demux_clock.rs +++ b/rust-modules/play-feed/src/demux_clock.rs @@ -452,7 +452,6 @@ mod tests { impl DemuxSubtitle for NoCaptions { fn plan(&mut self, _: &[TrackInfo]) -> Vec { Vec::new() } fn max_header_bytes(&self) -> usize { 0 } - fn admit_font(&mut self, _: usize) -> bool { false } fn begin_sources(&mut self, _: &str, _: Manifest) -> Option { None } fn record_tracks(&self) {} fn selected_track(&self) -> i32 { -1 } diff --git a/rust-modules/play-feed/src/ff.rs b/rust-modules/play-feed/src/ff.rs index 7c482cb87a..e8b95aa2ea 100644 --- a/rust-modules/play-feed/src/ff.rs +++ b/rust-modules/play-feed/src/ff.rs @@ -289,15 +289,16 @@ unsafe fn nth_audio_stream(fmt: *mut AVFormatContext, n: i32) -> Option { pub use plx_media_codecs::SpeedDecoder; -/// Copy the embedded ASS sources before FFmpeg can release them. The codec parameter and +/// Copy the embedded ASS headers before FFmpeg can release them, and hand over the `fonts` that +/// `detach_fonts` took out of the format when it was opened (issue #1328). The codec parameter and /// dictionary layouts are already compile-asserted against our bundled FFmpeg headers; the /// attachment media-type constant is asserted beside them. No firmware layout is inferred. unsafe fn begin_ass_sources( session: &mut dyn DemuxSession, media_key: &str, - fmt: *mut AVFormatContext, streams: *mut *mut AVStream, subs: &SubTracks, + fonts: Vec, ) -> Option { use plx_media_codecs::subtitle_extract::{collect_manifest, AssTrack}; let tracks: Vec = subs.entries().iter().zip(subs.plan()).map(|(info, kind)| AssTrack { @@ -305,8 +306,7 @@ unsafe fn begin_ass_sources( authored: matches!(kind, plx_playback_contract::subtitle::DecodeKind::Ass { .. }), }).collect(); let max_header_bytes = session.subtitle().max_header_bytes(); - let manifest = collect_manifest(fmt, streams, &tracks, max_header_bytes, - |len| session.subtitle().admit_font(len)); + let manifest = collect_manifest(streams, &tracks, max_header_bytes, fonts); session.subtitle().begin_sources(media_key, manifest) } @@ -6193,6 +6193,13 @@ pub fn demux( free_avio(avio); break; } + // The file's font attachments leave the format's stream table now, into the buffers + // the source store shares with every renderer (issue #1328): left in + // `codecpar->extradata` they would sit beside the demuxer's own copy (the Matroska + // element buffer, the only one left once `find_stream_info` rebuilds the codec + // context from the emptied parameters) for the whole playback. Nothing reads an attachment's extradata between here and the + // hand-over to `begin_ass_sources` below. + let font_attachments = plx_media_codecs::subtitle_extract::detach_fonts(fmt); if avformat_find_stream_info(fmt, std::ptr::null_mut()) < 0 { session.media().record("ff: find_stream_info failed"); if !crate::aq::aq_is_aborted(aq_p) { @@ -6575,9 +6582,11 @@ pub fn demux( SubTracks::open(streams, infos, plan) }; let ass_generation = if side_owner { + // The side reader has its own copy of the fonts; this one has no reader. + drop(font_attachments); 0 } else { - begin_ass_sources(session.as_mut(), &url, fmt, streams, &sub_tracks).unwrap_or(0) + begin_ass_sources(session.as_mut(), &url, streams, &sub_tracks, font_attachments).unwrap_or(0) }; if !side_owner { session.subtitle().record_tracks(); } diff --git a/rust-modules/playback-contract/src/demux.rs b/rust-modules/playback-contract/src/demux.rs index 27c06e0461..d0eebf9ce5 100644 --- a/rust-modules/playback-contract/src/demux.rs +++ b/rust-modules/playback-contract/src/demux.rs @@ -179,7 +179,6 @@ pub trait DemuxMedia { pub trait DemuxSubtitle { fn plan(&mut self, tracks: &[TrackInfo]) -> Vec; fn max_header_bytes(&self) -> usize; - fn admit_font(&mut self, bytes: usize) -> bool; fn begin_sources(&mut self, media_key: &str, manifest: Manifest) -> Option; fn record_tracks(&self); fn selected_track(&self) -> i32; diff --git a/rust-modules/screens/src/player/ass_subtitles.rs b/rust-modules/screens/src/player/ass_subtitles.rs index 0d63175af2..26dcfb1498 100644 --- a/rust-modules/screens/src/player/ass_subtitles.rs +++ b/rust-modules/screens/src/player/ass_subtitles.rs @@ -18,6 +18,8 @@ pub struct AssSubtitles { frame: Option>, source: u64, clock: ass_source::Clock, + /// The `now` of the previous `update`: the tick the renderer paces a moving cue by. + last_tick: Option, textures: Vec, uploaded: u64, viewport: Viewport, @@ -53,6 +55,7 @@ impl AssSubtitles { self.frame = None; self.aligns = None; } + let tick_ms = self.last_tick.replace(now).map_or(16, |last| i64::from(now.wrapping_sub(last))); let clock = self.clock.sample( position_ns, now, @@ -73,7 +76,15 @@ impl AssSubtitles { } let origin = source.origin; if let Some(frame) = - ass::request(source, clock, viewport.width, viewport.height, coded_w, coded_h) + ass::request( + source, + clock, + viewport.width, + viewport.height, + coded_w, + coded_h, + plx_media::player::subtitle_pace(ps, tick_ms), + ) { self.frame = Some(frame); self.frame_origin = origin; @@ -94,6 +105,7 @@ impl AssSubtitles { width: 0, height: 0, rects: Vec::new(), + scale: ass::RenderScale::Full, error: None, })); } @@ -156,22 +168,15 @@ impl AssSubtitles { UNKNOWN_ALIGNS }; self.spans.clear(); - self.spans.extend(frame.rects.iter().map(|r| (r.y, r.height))); + self.spans.extend(frame.rects.iter().map(|r| output_span(&frame, r))); lift_dys(origin, lift, band, &self.spans, aligns, &mut self.dys); let ink = plx_appkit::player_hud::subtitle_tint(model); for (i, (texture, rect)) in self.textures.iter().zip(&frame.rects).enumerate() { - let dy = self.dys[i]; - Painter::root().tex( - texture.id, - Rect::new( - (self.viewport.x + rect.x) as f32, - (self.viewport.y + rect.y) as f32 + dy, - rect.width as f32, - rect.height as f32, - ), - 0.0, - ink, - ); + // The facade composes the regions premultiplied (`ass::Rect::rgba`), so they take the + // premultiplied draw: the same ink rung, no straight-alpha conversion on the CPU. The + // texture is sampled linearly (`upload_rgba`), so a region drawn smaller (`frame.scale`) + // is drawn up to the rectangle it covers on the output. + Painter::root().tex_premultiplied(texture.id, region_rect(self.viewport, &frame, rect, self.dys[i]), ink); } } @@ -196,6 +201,21 @@ impl AssSubtitles { } } +/// Where a region of `frame` is drawn on the screen: the rectangle it covers on the output canvas +/// ([`ass::Frame::output_edges`], so a frame drawn smaller lands exactly where the full-size one does +/// and regions that abut still abut), inside `viewport` and moved `dy` (output pixels). +fn region_rect(viewport: Viewport, frame: &ass::Frame, rect: &ass::Rect, dy: f32) -> Rect { + let (left, top, right, bottom) = frame.output_edges(rect); + Rect::new(viewport.x as f32 + left, viewport.y as f32 + top + dy, right - left, bottom - top) +} + +/// `(y, height)` of a region on the output canvas, in whole pixels: what the lift reasons about. +fn output_span(frame: &ass::Frame, rect: &ass::Rect) -> (i32, i32) { + let (_, top, _, bottom) = frame.output_edges(rect); + let top = top.round() as i32; + (top, bottom.round() as i32 - top) +} + /// The viewport a frame is rendered into and the storage raster it is sized against. An authored /// script keeps the video's; a text subtitle is placed in screen space, so it gets the whole /// logical screen whatever the video's aspect (plan §2.1). @@ -650,6 +670,66 @@ mod tests { assert_eq!(dys_of(text(style), true, &[], BOTTOM), Vec::::new()); } + fn region(x: i32, y: i32, width: i32, height: i32) -> ass::Rect { + ass::Rect { x, y, width, height, rgba: Arc::from(vec![0u8; (width * height * 4) as usize]) } + } + + fn frame_at(scale: ass::RenderScale, width: i32, height: i32, rects: Vec) -> ass::Frame { + ass::Frame { source_id: 1, serial: 1, width, height, rects, scale, error: None } + } + + /// A moving stretch drawn at half or two thirds of the output covers the same rectangle of the + /// screen as the full-size frame: positions and sizes are mapped back by the frame's scale, inside + /// the viewport, with the HUD's lift applied on the output. + #[test] + fn a_region_drawn_smaller_lands_on_the_screen_rectangle_the_full_size_one_has() { + use ass::RenderScale::{Full, Half, TwoThirds}; + let viewport = Viewport { x: 0, y: 140, width: 1920, height: 800 }; + let full = frame_at(Full, 1920, 800, vec![region(480, 270, 680, 370)]); + let want = region_rect(viewport, &full, &full.rects[0], 0.0); + assert_eq!(want, Rect::new(480.0, 410.0, 680.0, 370.0), "full size is the region itself, in the viewport"); + let half = frame_at(Half, 1920, 800, vec![region(240, 135, 340, 185)]); + assert_eq!(region_rect(viewport, &half, &half.rects[0], 0.0), want); + let two_thirds = frame_at(TwoThirds, 1920, 800, vec![region(320, 180, 453, 247)]); + let got = region_rect(viewport, &two_thirds, &two_thirds.rects[0], 0.0); + assert!((got.x - want.x).abs() <= 1.5 && (got.y - want.y).abs() <= 1.5, "{got:?} against {want:?}"); + assert!((got.w - want.w).abs() <= 1.5 && (got.h - want.h).abs() <= 1.5, "{got:?} against {want:?}"); + // the lift is in output pixels: the same at every scale + assert_eq!( + region_rect(viewport, &half, &half.rects[0], -200.0), + Rect::new(want.x, want.y - 200.0, want.w, want.h) + ); + } + + #[test] + fn regions_drawn_smaller_still_abut_on_the_screen() { + let viewport = Viewport { x: 0, y: 0, width: 1920, height: 800 }; + let frame = frame_at(ass::RenderScale::TwoThirds, 1920, 800, vec![region(0, 100, 10, 17), region(0, 117, 10, 20)]); + let (above, below) = (region_rect(viewport, &frame, &frame.rects[0], 0.0), region_rect(viewport, &frame, &frame.rects[1], 0.0)); + assert_eq!(above.y + above.h, below.y, "the bottom of one is the top of the next"); + } + + /// The lift reasons about where a region is on the screen, so its spans are in output pixels. + #[test] + fn the_lift_reads_a_regions_span_on_the_output_whatever_the_scale() { + use ass::RenderScale::{Full, Half}; + let full = frame_at(Full, 1920, 1080, vec![region(480, 900, 680, 100)]); + assert_eq!(output_span(&full, &full.rects[0]), (900, 100)); + let half = frame_at(Half, 1920, 1080, vec![region(240, 450, 340, 50)]); + assert_eq!(output_span(&half, &half.rects[0]), (900, 100)); + // the lift moves a bottom cue the same whichever size it was drawn at + let style = TextStyle::default(); + let (y, h) = line_at(style, 0); + let dys = |frame: &ass::Frame| { + let spans: Vec<_> = frame.rects.iter().map(|r| output_span(frame, r)).collect(); + dys_of(text(style), true, &spans, BOTTOM) + }; + let full = frame_at(Full, 1920, 1080, vec![region(480, y, 680, h)]); + let half = frame_at(Half, 1920, 1080, vec![region(240, y / 2, 340, h / 2)]); + assert_eq!(dys(&half), dys(&full)); + assert_eq!(dys(&full), [-200.0]); + } + #[test] fn moving_and_reordered_regions_retain_uploads_before_recycling_changed_ones() { let a: Arc<[u8]> = Arc::from([255, 0, 0, 255]); diff --git a/rust-modules/src/app/adapters/hostsim.rs b/rust-modules/src/app/adapters/hostsim.rs index 6125a38902..05412f46b1 100644 --- a/rust-modules/src/app/adapters/hostsim.rs +++ b/rust-modules/src/app/adapters/hostsim.rs @@ -14,10 +14,13 @@ use plx_play_feed::ffi_host::{HostSink, Hooks}; /// it as `Port.sink`; a hostsim test binary of this crate reads it directly. pub(crate) static HOST_SINK: HostSink = HostSink::new(Hooks::over(&sim_video::SimVideo)); -/// The render thread's half of the simulator's picture, once per presented frame, before the -/// capture and the swap: the current decoded frame, composited UNDER what the UI drew. -pub(crate) fn composite_under() { - sim_video::composite_under(); +/// The render thread's half of the simulator's picture, once per bound-plane frame, after the +/// straighten pass and before the capture and the swap: the current decoded frame, composited UNDER +/// what the UI drew through the television compositor's straight-alpha mix. `false` when there is no +/// picture to put under it (not armed, or nothing decoded yet), which is the caller's cue to +/// composite over black instead. +pub(crate) fn composite_under() -> bool { + sim_video::composite_under() } /// The simulator's clock-sink stop; the television's pipeline has no such control. diff --git a/rust-modules/src/app/adapters/hostsim/sim_video.rs b/rust-modules/src/app/adapters/hostsim/sim_video.rs index 69ea925fe5..075d74b308 100644 --- a/rust-modules/src/app/adapters/hostsim/sim_video.rs +++ b/rust-modules/src/app/adapters/hostsim/sim_video.rs @@ -1,8 +1,9 @@ //! **The simulator's picture** (`plxnative-simvideo`, with the clock sink armed): the video access //! units the clock sink already accepts are ALSO piped to a system `ffmpeg` child process, decoded //! there, and the frame whose presentation time the sink's clock has reached is composited UNDER -//! the finished UI frame — exactly the arithmetic the television's compositor applies to its -//! hardware video plane and our transparent UI plane. +//! the finished UI frame — the arithmetic the television's compositor applies to its hardware video +//! plane and our UI plane: a STRAIGHT-alpha mix of the surface the straighten pass +//! (`plx_gfx::gfx::straighten`) has already made straight. //! //! **Why it exists.** The documentation's player figure has to show a real picture under the real //! HUD, and the simulator otherwise shows the HUD over black (`play-feed/src/ffi_host.rs`: nothing decodes). It @@ -232,17 +233,20 @@ fn spawn() -> Option { } /// The render thread's half: upload the current frame if it changed and composite it UNDER what -/// the UI drew this frame. Called once per presented frame, before the capture and the swap. -pub fn composite_under() { +/// the UI drew this frame, through the television compositor's straight-alpha mix +/// ([`plx_gfx::gfx::draw_under`]) on a surface the straighten pass has already made straight. Called +/// once per bound-plane frame, before the capture and the swap. `true` when a picture was composited; +/// `false` leaves the frame untouched for the caller to composite over black. +pub fn composite_under() -> bool { thread_local! { /// `(texture, frame number it holds)`. static TEX: std::cell::Cell<(c_uint, Option)> = const { std::cell::Cell::new((0, None)) }; } if !armed() { - return; + return false; } let latest = LATEST.lock().unwrap_or_else(|e| e.into_inner()).clone(); - let Some((n, rgba)) = latest else { return }; + let Some((n, rgba)) = latest else { return false }; TEX.with(|t| { let (mut tex, held) = t.get(); if held != Some(n) { @@ -251,4 +255,5 @@ pub fn composite_under() { } plx_gfx::gfx::draw_under(tex, 0.0, 0.0, W as f32, H as f32); }); + true } diff --git a/rust-modules/src/app/run.rs b/rust-modules/src/app/run.rs index ba9449a2a6..777a20676e 100644 --- a/rust-modules/src/app/run.rs +++ b/rust-modules/src/app/run.rs @@ -764,6 +764,20 @@ pub(crate) fn rig_clear_opaque_region() { plx_platform::tv::window::clear_opaque_region(); } +/// **Simulator only: the television's compositor, over the frame the straighten pass just made.** The +/// decoded picture under the UI when `plxnative-simvideo` holds one, black otherwise +/// ([`plx_gfx::gfx::draw_under`]'s straight-alpha mix either way). The pass has no simulator switch, so +/// this is not optional on a bound frame: a shot of the straight buffer would be the pass's own +/// intermediate picture. +#[cfg(feature = "hostsim")] +fn composite_plane_under() { + #[cfg(feature = "play-feed")] + if crate::app::adapters::hostsim::composite_under() { + return; + } + plx_gfx::gfx::draw_under_black(); +} + /// One idle iteration's font-warming slice: a key pressed during it waits at most this long more /// than the idle poll already makes it wait. const FONT_WARM_SLICE_US: u64 = 2_000; @@ -784,6 +798,8 @@ unsafe fn present_and_swap( } // the glyph cache's frame serial (phase 11, text.rs's hot window): a drawn frame plx_gfx::text::begin_frame(); + // …and the straighten pass's ledger: nothing drawn yet, the surface not yet straight. + plx_gfx::gfx::straighten::frame_begin(); // A finished underlay-field reduction is read HERE, before framebuffer 0 holds anything // of this frame — a read between the page and the surfaces splits its render pass. plx_gfx::gfx::field_frame_begin(); @@ -800,11 +816,24 @@ unsafe fn present_and_swap( crate::capture::tick(fr.now); } app.instr.mark(plx_base::diag::heartbeat::Phase::Capture); // capture - // `plxnative-simvideo`: the decoded picture goes UNDER the finished UI, as the television's - // compositor puts its video plane under ours — before the capture, so shots include it. - #[cfg(all(feature = "hostsim", feature = "play-feed"))] - if fr.player { - crate::app::adapters::hostsim::composite_under(); + // **THE FRAME'S LAST DRAW over the bound video plane: premultiplied -> straight.** The + // television's compositor multiplies our surface by its alpha (`panel = ui.rgb * ui.a + video + // * (1 - ui.a)`) and the framebuffer holds premultiplied colour, so every translucent pixel + // would be darkened by its own alpha a second time; this divides it back out, in place, on a + // frame that drew anything (`gfx/straighten.rs` has the measurement, the buffer-age argument + // and the cost). Keyed on the plane's BIT, like the capture above and the opaque region: the + // bound trailer preview is as translucent as the player. It is after every read of the + // framebuffer the frame makes (the capture tick above, which skips this very frame) and before + // the swap and everything that follows it; nothing the loop does after it samples + // framebuffer 0 on the television (`video_plane_refuses` shuts every door past it). + plx_gfx::gfx::straighten::straighten_surface(app.player.video_plane_bound); + // `plxnative-simvideo`, and the simulator's stand-in for the compositor the pass feeds: a + // bound frame goes through the television's straight-alpha mix over the decoded picture, or + // over black with none, so a shot is the panel and not the straight buffer. Before the + // screenshot seam below, so shots include it. + #[cfg(feature = "hostsim")] + if app.player.video_plane_bound { + composite_plane_under(); } // Before the swap, never after: the back buffer is undefined once presented. // A dump decides here whether this frame is written (`dev::framedump`); otherwise the @@ -10115,3 +10144,55 @@ mod wake_input_tests { } } } + +#[cfg(test)] +mod straighten_wiring_tests { + //! `present_and_swap` needs a real SDL/GL context, so the straighten pass's place in the frame is + //! pinned from the source text, the idiom `video_plane_gate_tests` and `pump_wiring_tests` use. + + /// **The pass is the frame's last draw before the swap, and every readback is before it.** The + /// order in `present_and_swap`: the ledger opens (`frame_begin`), the frame draws, the dev capture + /// stream reads the framebuffer (and skips a bound frame, which is the only one the pass runs + /// on), THEN the pass, THEN the simulator's compositor stand-in, the simulator's two readers + /// (frame dump, screenshot) and the swap. A read between the pass and the compositor would see + /// the straight buffer; a pass after the swap would straighten an undefined back buffer. + /// + /// The pass takes the plane's BIT, unnested (it does its own gating on the bit and on whether the + /// frame drew), and the simulator's compositor runs on that same bit, so the two run together or + /// not at all: a straightened frame no reader may see uncomposited. + #[test] + fn the_straighten_pass_is_last_before_the_swap_and_after_every_readback() { + let whole = std::fs::read_to_string( + std::path::Path::new(env!("CARGO_MANIFEST_DIR")).join("src/app/run.rs"), + ) + .expect("read run.rs"); + let src = whole.split("\n#[cfg(test)]\nmod ").next().expect("the loop's source"); + const PASS: &str = " plx_gfx::gfx::straighten::straighten_surface(app.player.video_plane_bound);"; + assert_eq!( + src.lines().filter(|l| *l == PASS).count(), + 1, + "the pass is called exactly once, at `present_and_swap`'s `if fr.present` depth, handed the plane's bit", + ); + assert!( + !src.contains("straighten_surface(fr.player)"), + "keyed on the plane's bit, not the route: the bound trailer preview is not the player route", + ); + let at = |needle: &str| src.find(needle).unwrap_or_else(|| panic!("`{needle}` is gone from run.rs")); + let begin = at("plx_gfx::gfx::straighten::frame_begin();"); + let draw = at("let (_vx, _vy, _vw, _vh) = draw(app, fr);"); + let capture = at("crate::capture::tick(fr.now);"); + let pass = at(PASS.trim_start()); + let compositor = at("composite_plane_under();\n }"); + let dump = at("crate::dev::framedump::after_draw("); + let shot = at("crate::shot::maybe_capture("); + let swap = at("SDL_GL_SwapWindow(app.win);"); + assert!(begin < draw, "the ledger opens before the frame draws"); + assert!(draw < capture && capture < pass, "the capture stream reads before the pass"); + assert!(pass < compositor, "the simulator's compositor runs after the pass it models"); + assert!(compositor < dump && compositor < shot, "no reader sees a straight, uncomposited frame"); + assert!(shot < swap && dump < swap, "…and the pass is done before the swap"); + // The compositor stand-in sits behind the same bit the pass is handed. + let guard = " #[cfg(feature = \"hostsim\")]\n if app.player.video_plane_bound {\n composite_plane_under();"; + assert!(src.contains(guard), "the simulator composites on exactly the frames the pass straightened"); + } +} diff --git a/rust-modules/src/shot.rs b/rust-modules/src/shot.rs index 7413df9e46..6d355ec313 100644 --- a/rust-modules/src/shot.rs +++ b/rust-modules/src/shot.rs @@ -28,7 +28,9 @@ //! PLXNATIVE_SHOT_EXIT=1 end the run after an automatic capture — the headless one-shot //! mode (an orderly stop through the app's own shutdown, never an //! `exit()` from inside the frame: see [`maybe_capture`]) -//! PLXNATIVE_SHOT_ALPHA=1 write RGBA (premultiplied, as the framebuffer holds it) +//! PLXNATIVE_SHOT_ALPHA=1 write RGBA (premultiplied, as the framebuffer holds it; a frame +//! over the bound video plane is the exception: the simulator has +//! composited it over the picture, or black, so its alpha is 1) //! //! The `shot` token on the remote FIFO captures on demand instead, which is what an interactive //! agent session uses: drive the UI, then ask for the frame. Those are NUMBERED (`shot-1.png`, @@ -209,10 +211,18 @@ pub(crate) fn read_flipped(vx: c_int, vy: c_int, vw: c_int, vh: c_int, ch: usize // opaque RGB image is the FAITHFUL screenshot, and the one that compares to a device capture, // where the TV's compositor has likewise already flattened the two planes. // + // **A frame over the BOUND video plane is the one exception, and it arrives already flattened.** + // The straighten pass (`gfx/straighten.rs`) has turned its buffer into STRAIGHT colour for the + // television's compositor, which multiplies it by alpha; the simulator stands in for that + // compositor (`run.rs`'s `composite_plane_under`, the decoded picture or black) before this + // read, so the RGB here is the panel and the alpha is 1. Reading the straight buffer itself + // would give a picture brighter than any panel shows. + // // `PLXNATIVE_SHOT_ALPHA=1` keeps it anyway, for compositing a shot over a picture of your own: - // the channels are then exactly what the framebuffer holds, i.e. PREMULTIPLIED, so the - // composite is `out = shot.rgb + picture * (1 - shot.a)` — not the straight-alpha "over" a - // viewer applies to a PNG, which is why this file will look wrong opened on its own. + // the channels are then exactly what the framebuffer holds, i.e. PREMULTIPLIED (for every frame + // but the bound-plane one above), so the composite is `out = shot.rgb + picture * (1 - shot.a)` + // — not the straight-alpha "over" a viewer applies to a PNG, which is why this file will look + // wrong opened on its own. let src_stride = w * 4; let dst_stride = w * ch; let mut rgb = vec![0u8; dst_stride * h]; diff --git a/rust-modules/ui/src/lib.rs b/rust-modules/ui/src/lib.rs index 0e8423ee56..9e8cec8264 100644 --- a/rust-modules/ui/src/lib.rs +++ b/rust-modules/ui/src/lib.rs @@ -1423,6 +1423,31 @@ impl Painter { } plx_gfx::gfx::draw_tex_uv(tex, uv, m.x, m.y, m.w, m.h, self.px(rad), t.as_ptr()); } + /// A texture that holds PREMULTIPLIED RGBA (`(rgb * a, a)` per texel), drawn in `r` as one + /// square-cornered quad: what [`tex`](Self::tex) does for a straight-alpha one, through + /// [`plx_gfx::gfx::draw_tex_premultiplied`]'s blend. `tint` is the STRAIGHT ink and fade `tex` + /// takes, and the painter's own cascade (its alpha and colour gain) applies to it exactly as it does + /// to `tex`'s, so a subtitle's ink rung or a fade dims a premultiplied picture the way it dims a + /// straight one. Used for the libass regions, which the facade composes premultiplied; a texture + /// uploaded straight (an image subtitle, a glyph) stays on `tex`. Unlike `tex` it is not cut by an + /// ink clip (`widgets::paint_inked`): that path samples a texture as coverage, which a colour + /// picture is not. + pub fn tex_premultiplied(self, tex: u32, r: Rect, tint: [f32; 4]) { + #[cfg(any(test, feature = "test-support"))] + if self.records() { + draw_census::note_tex(tex, self.place(r), self.c(tint)[3]); + } + if self.declare(r, 15, |data| { + use frame::backdrop::Value; + tex.record(data); + plx_gfx::gfx::tex_ledger::revision(tex).record(data); + tint.record(data); + }) { return; } + if self.records() { return; } + let t = self.c(tint); + let m = self.place(r); + plx_gfx::gfx::draw_tex_premultiplied(tex, m.x, m.y, m.w, m.h, t); + } /// The FROSTED ground: what the frame drew behind `r`, blurred, clipped to `r`'s rounded rect. /// /// Returns whether it drew. `false` is not an error — the blur latches itself off on a driver @@ -2413,6 +2438,26 @@ mod tests { assert_eq!(p.a, 0.125); } + /// A premultiplied texture is the same recorded quad as a straight one under the painter's cascade + /// (translate and fade), and its shader tint is the cascaded tint premultiplied AFTER the cascade: + /// the cascade scales colour by the gain and alpha by the fade, so premultiplying first would leave + /// a faded picture's colour too bright. + #[test] + fn a_premultiplied_texture_takes_the_cascade_of_a_straight_one_and_is_premultiplied_after_it() { + let r = Rect::new(10.0, 20.0, 30.0, 40.0); + let ink = [0.85, 0.85, 0.85, 1.0]; + let faded = || Painter::recording().translate(5.0, 6.0).alpha(0.5); + let straight = draw_census::capture_tex(|| faded().tex(7, r, 0.0, ink)); + let premultiplied = draw_census::capture_tex(|| faded().tex_premultiplied(7, r, ink)); + assert_eq!(premultiplied, vec![(7, Rect::new(15.0, 26.0, 30.0, 40.0), 0.5)]); + assert_eq!(premultiplied, straight, "the same texture, place and effective alpha"); + let tag = |f: &dyn Fn()| draw_census::capture(|| f()).iter().map(|c| c.0).collect::>(); + assert_eq!(tag(&|| faded().tex(7, r, 0.0, ink)), [12]); + assert_eq!(tag(&|| faded().tex_premultiplied(7, r, ink)), [15], "its own command, so a glass replay can tell them apart"); + let cascaded = Painter::root().alpha(0.5).c(ink); + assert_eq!(plx_gfx::gfx::premultiplied_tint(cascaded), [0.425, 0.425, 0.425, 0.5]); + } + #[test] fn output_gain_scales_the_colour_that_reaches_the_renderer_and_never_the_alpha() { let col = [0.8, 0.6, 0.4, 0.5]; diff --git a/src/ass.c b/src/ass.c index 9339ba1b96..398741ddb8 100644 --- a/src/ass.c +++ b/src/ass.c @@ -9,9 +9,9 @@ #include #include #include +#include #define MAX_SCRIPT_BYTES (8u * 1024u * 1024u) -#define MAX_FONT_BYTES (64u * 1024u * 1024u) #define MAX_EVENT_BYTES (256u * 1024u) #define MAX_EVENTS 20000 #define MAX_PIXELS (3840u * 2160u) @@ -33,7 +33,7 @@ struct PlxAss { ASS_Renderer *renderer; ASS_Track *track; uint8_t *rgba; - size_t bytes, font_bytes; + size_t bytes; PlxAssBitmap regions[PLX_ASS_MAX_REGIONS]; struct ImageKey *images; size_t image_capacity, image_count, region_count; @@ -43,8 +43,15 @@ struct PlxAss { int fonts_set; /* ass_set_fonts has run: it opens every font, so only once per ctx */ int wrap_unicode; uint32_t font_scans, tracks; + /* The last plx_ass_render call's cost split, reported by plx_ass_stats. */ + uint32_t last_libass_us, last_composite_us, last_libass_cpu_us, last_composite_cpu_us; + uint32_t last_images; + uint64_t last_image_pixels; char *default_font; - unsigned reciprocal[256]; + /* Indices (into the library's font table) of the fonts added by plx_ass_add_font_borrowed: their + * bytes are the caller's, so plx_ass_destroy detaches them before the library frees its own. */ + size_t *borrowed; + size_t borrowed_count, borrowed_capacity; }; /* Subtitle text and font metadata are untrusted and can be private. Do not @@ -56,12 +63,48 @@ static void quiet_message(int level, const char *format, va_list args, void *dat unsigned plx_ass_abi_version(void) { return PLX_ASS_ABI_VERSION; } +/* A stopwatch on one clock: CLOCK_MONOTONIC for wall time, CLOCK_THREAD_CPUTIME_ID for the CPU time + * the calling thread has run. A clock that cannot be read stops the watch: every lap after the + * failure counts 0, never a span measured from a start that was not taken. */ +struct Lap { + clockid_t clock; + struct timespec at; + int running; +}; + +static void lap_start(struct Lap *lap, clockid_t clock) +{ + lap->clock = clock; + lap->running = clock_gettime(clock, &lap->at) == 0; +} + +/* Microseconds since the last lap (or the start), which becomes now: one clock read ends a phase + * and starts the next. */ +static uint32_t lap_us(struct Lap *lap) +{ + struct timespec now; + if (!lap->running) return 0; + if (clock_gettime(lap->clock, &now) != 0) { + lap->running = 0; + return 0; + } + int64_t ns = ((int64_t)now.tv_sec - (int64_t)lap->at.tv_sec) * 1000000000 + + ((int64_t)now.tv_nsec - (int64_t)lap->at.tv_nsec); + lap->at = now; + if (ns <= 0) return 0; + return ns / 1000 >= UINT32_MAX ? UINT32_MAX : (uint32_t)(ns / 1000); +} + void plx_ass_destroy(PlxAss *ctx) { if (!ctx) return; if (ctx->track) ass_free_track(ctx->track); if (ctx->renderer) ass_renderer_done(ctx->renderer); + /* The renderer's faces read a borrowed font until it is done; the library frees every font's + * bytes, so those that are the caller's leave its table first. */ + plxi_library_forget_borrowed(ctx->library, ctx->borrowed, ctx->borrowed_count); if (ctx->library) ass_library_done(ctx->library); + free(ctx->borrowed); free(ctx->rgba); free(ctx->images); free(ctx->default_font); @@ -73,7 +116,6 @@ PlxAss *plx_ass_create(const char *default_font) if (!default_font || !*default_font) return NULL; PlxAss *ctx = calloc(1, sizeof(*ctx)); if (!ctx) return NULL; - for (unsigned a = 1; a < 256; ++a) ctx->reciprocal[a] = 65536u / a; ctx->default_font = strdup(default_font); ctx->library = ass_library_init(); if (!ctx->default_font || !ctx->library) goto failed; @@ -89,13 +131,39 @@ PlxAss *plx_ass_create(const char *default_font) return NULL; } +/* Hand libass one font file. libass copies the bytes (ass_add_font) and opens the face at the next + * ass_set_fonts or, once a track has events, at the start of the next frame + * (ass_render.c: ass_start_frame -> ass_update_embedded_fonts, which scans only the fonts added + * since), so this works before and after plx_ass_load and costs one face scan for this font, not a + * rescan of the others. The only limit is the `int` size ass_add_font takes: what a file attaches + * is never cut by a byte count (the Rust side chooses WHICH fonts to add, by name). */ int plx_ass_add_font(PlxAss *ctx, const char *name, const uint8_t *data, size_t len) { - if (!ctx || ctx->track || ctx->fonts_set || !name || !*name || !data || !len || - len > MAX_FONT_BYTES || ctx->font_bytes > MAX_FONT_BYTES - len) + if (!ctx || !name || !*name || !data || !len || len > INT_MAX) return -1; ass_add_font(ctx->library, name, (char *)data, (int)len); - ctx->font_bytes += len; + return 0; +} + +/* plx_ass_add_font without the copy: libass's entry points at the caller's `data`, which the caller + * keeps alive and unchanged until plx_ass_destroy returns. One font held by N renderers (the + * subtitle worker pool) is then one buffer, not N (issue #1328). The face is opened the same way, + * at the next frame. The destroy path detaches the entry so libass never frees the caller's bytes. */ +int plx_ass_add_font_borrowed(PlxAss *ctx, const char *name, const uint8_t *data, size_t len) +{ + if (!ctx || !name || !*name || !data || !len || len > INT_MAX) + return -1; + if (ctx->borrowed_count == ctx->borrowed_capacity) { + size_t capacity = ctx->borrowed_capacity ? 2 * ctx->borrowed_capacity : 16; + size_t *grown = realloc(ctx->borrowed, capacity * sizeof(*grown)); + if (!grown) return -1; + ctx->borrowed = grown; + ctx->borrowed_capacity = capacity; + } + size_t index; + if (plxi_library_borrow_font(ctx->library, name, data, len, &index) != 0) + return -1; + ctx->borrowed[ctx->borrowed_count++] = index; return 0; } @@ -142,9 +210,9 @@ int plx_ass_load(PlxAss *ctx, const uint8_t *data, size_t len, int script) if (read_track(ctx, data, len, script) < 0) return -1; /* Inter is the shipped Latin family. With provider NONE libass tries the * exact authored family, this family, then default_font (our CJK face). - * ass_set_fonts opens every font the library holds, so it runs once per ctx, here: the fonts - * are fixed from now on (plx_ass_add_font and a second load refuse), and a replaced track - * reuses the faces. */ + * ass_set_fonts opens every font the library holds, so it runs once per ctx, here: a second + * load refuses, a replaced track reuses the faces, and a font added from now on (plx_ass_add_font) + * is opened alone at the next frame. */ ass_set_fonts(ctx->renderer, ctx->default_font, "Inter", ASS_FONTPROVIDER_NONE, NULL, 1); ctx->fonts_set = 1; ++ctx->font_scans; @@ -154,8 +222,10 @@ int plx_ass_load(PlxAss *ctx, const uint8_t *data, size_t len, int script) int plx_ass_replace_track(PlxAss *ctx, const uint8_t *data, size_t len, int script) { if (!ctx || !ctx->fonts_set) return -1; - /* The first load already extracted and registered the fonts. ass_set_fonts never runs again, - * so a [Fonts] section read now would only add copies nothing uses. */ + /* Extraction is off because a replacement track's [Fonts] section would be registered and + * scanned at the next frame (ass_start_frame -> ass_update_embedded_fonts), duplicating faces + * the caller already added with plx_ass_add_font. Which faces a renderer holds is the Rust + * side's decision (font_names). */ ass_set_extract_fonts(ctx->library, 0); if (ctx->track) ass_free_track(ctx->track); ctx->track = NULL; @@ -206,6 +276,12 @@ int plx_ass_stats(const PlxAss *ctx, PlxAssStats *out) out->library_fonts = (uint32_t)plxi_library_fonts(ctx->library); out->font_scans = ctx->font_scans; out->tracks = ctx->tracks; + out->last_libass_us = ctx->last_libass_us; + out->last_composite_us = ctx->last_composite_us; + out->last_libass_cpu_us = ctx->last_libass_cpu_us; + out->last_composite_cpu_us = ctx->last_composite_cpu_us; + out->last_images = ctx->last_images; + out->last_image_pixels = ctx->last_image_pixels; return 0; } @@ -376,6 +452,81 @@ static void place_regions(PlxAss *ctx, const struct Bounds *bounds, size_t count ctx->region_count = count; } +/* Test hook: ci/test_ass_regions.py defines this before including the file to count how often the + * loop below steps onto a pixel that has nothing to draw (kind: `blend` in blend_row) instead of + * skipping it with its neighbours. It is a no-op otherwise. */ +#ifndef PLX_ASS_VISIT_ZERO +#define PLX_ASS_VISIT_ZERO(kind) ((void)0) +#endif +/* The same test hook for the NEON row path (ass_composite.h): `pixels` pixels were blended by one + * vector operation, which does not pass through ass_blend_pixel. */ +#ifndef PLX_ASS_VISIT_VECTOR +#define PLX_ASS_VISIT_VECTOR(pixels) ((void)0) +#endif + +/* A pixel of one ASS_Image onto the premultiplied destination: `mask` is libass's coverage of it, + * coverage[] the image's opacity table (coverage[0] is 0, and so can a nonzero entry be). */ +static inline void blend_pixel(uint8_t *dst, unsigned mask, const uint8_t *coverage, uint32_t color) +{ + unsigned a = coverage[mask]; + if (!a) { + PLX_ASS_VISIT_ZERO(blend); + return; + } + if (a == 255) { + ass_store_rgba(dst, color); + return; + } + ass_store_rgba(dst, ass_blend_pixel(ass_load_rgba(dst), color, a)); +} + +static inline void blend_four(uint8_t *dst, const uint8_t *mask, const uint8_t *coverage, + uint32_t color) +{ + for (int i = 0; i < 4; ++i) blend_pixel(dst + 4 * i, mask[i], coverage, color); +} + +/* One row of one ASS_Image onto the premultiplied destination: `n` pixels, mask[i] the libass + * coverage of pixel i. Most of a sign's mask is zero, which draws nothing whatever the opacity, so + * the row is read a group of bytes at a time and a group of zeros is passed over whole. The groups + * come from the row's own start and never reach past its end (the stride's padding is not coverage). + * + * With NEON the group is eight pixels and one with any coverage is blended as a vector + * (ass_blend8: the coverage computed from the image's opacity, which is coverage[255], then the + * blend), zero lanes included: they come out as the destination, exactly. The pixels after the last + * whole group, and every pixel on a target without NEON, go four at a time: a word of zeros is + * passed over whole and a word with any coverage is drawn pixel by pixel, including the nonzero mask + * bytes a faint opacity maps to coverage 0. */ +static void blend_row(uint8_t *dst, const uint8_t *mask, int n, const uint8_t *coverage, + uint32_t color) +{ + int x = 0; +#ifdef PLX_ASS_NEON + if (n >= 8) { + /* The table maps 255 to the image's own opacity: (255 * o + 127) / 255 is o. */ + const uint8x8_t opacity = vdup_n_u8(coverage[255]); + const AssSource8 source = ass_source8(color); + for (; x + 8 <= n; x += 8) { + /* Looked at in core registers: a NEON-to-core move stalls the pipeline. */ + uint64_t group; + memcpy(&group, mask + x, sizeof group); + if (!group) continue; + PLX_ASS_VISIT_VECTOR(8); + ass_blend8(dst + 4 * x, vld1_u8(mask + x), opacity, source); + } + } +#endif + for (; x + 4 <= n; x += 4) { + uint32_t word; + memcpy(&word, mask + x, sizeof word); + if (word) blend_four(dst + 4 * x, mask + x, coverage, color); + } + for (; x < n; ++x) blend_pixel(dst + 4 * x, mask[x], coverage, color); +} + +static int compose_frame(PlxAss *ctx, ASS_Image *images, int width, int height, int reusable, + PlxAssFrame *frame); + int plx_ass_render(PlxAss *ctx, int64_t now_ms, int width, int height, int storage_width, int storage_height, PlxAssFrame *frame) @@ -401,20 +552,40 @@ int plx_ass_render(PlxAss *ctx, int64_t now_ms, int width, int height, /* Any failed call breaks the consecutive-frame identity proof above. */ ctx->cache_valid = 0; int changed = 0; + struct Lap wall, cpu; + lap_start(&wall, CLOCK_MONOTONIC); + lap_start(&cpu, CLOCK_THREAD_CPUTIME_ID); ASS_Image *images = ass_render_frame(ctx->renderer, ctx->track, now_ms, &changed); + ctx->last_libass_us = lap_us(&wall); + ctx->last_libass_cpu_us = lap_us(&cpu); + ctx->last_composite_us = ctx->last_composite_cpu_us = 0; if (!changed && reusable && !ctx->first) { ctx->cache_valid = 1; return 0; } ctx->first = 0; + int result = compose_frame(ctx, images, width, height, reusable, frame); + ctx->last_composite_us = lap_us(&wall); + ctx->last_composite_cpu_us = lap_us(&cpu); + return result; +} + +/* Everything plx_ass_render does with libass's image list: bounds, retained regions, the arena and + * the per-pixel composite. Its wall time is last_composite_us, its thread CPU time + * last_composite_cpu_us. */ +static int compose_frame(PlxAss *ctx, ASS_Image *images, int width, int height, int reusable, + PlxAssFrame *frame) +{ struct Bounds bounds[PLX_ASS_MAX_REGIONS]; size_t region_count = 0; int image_count = 0; + uint64_t image_pixels = 0; for (ASS_Image *p = images; p; p = p->next) { if (++image_count > MAX_IMAGES) return -1; int l, t, r, b; int valid = clipped_bounds(p, width, height, &l, &t, &r, &b); if (valid < 0) return -1; + if (p->w > 0 && p->h > 0) image_pixels += (uint64_t)p->w * (uint64_t)p->h; if (!valid) continue; /* Transparent border preserves linear filtering at each new texture edge. */ add_bounds(bounds, ®ion_count, (struct Bounds){ @@ -422,6 +593,8 @@ int plx_ass_render(PlxAss *ctx, int64_t now_ms, int width, int height, r < width ? r + 1 : width, b < height ? b + 1 : height }); } + ctx->last_images = (uint32_t)image_count; + ctx->last_image_pixels = image_pixels; memset(frame, 0, sizeof(*frame)); if (!region_count) { ctx->region_count = ctx->image_count = 0; @@ -446,9 +619,11 @@ int plx_ass_render(PlxAss *ctx, int64_t now_ms, int width, int height, ctx->bytes = bytes; } place_regions(ctx, bounds, region_count, retained); - /* ASS_Image is an ordered list (shadow, outline, glyph, next event). Compose - * every layer in that order in premultiplied space before converting to the - * straight RGBA expected by SDL's ordinary blend mode. */ + /* ASS_Image is an ordered list (shadow, outline, glyph, next event). Compose every layer in that + * order in premultiplied space and publish the result as it is: PREMULTIPLIED RGBA, drawn by the + * application with a premultiplied blend (gfx's `draw_tex_premultiplied`). A conversion to straight + * RGBA here would be a pass over every pixel of every changed region, most of them transparent, + * to undo what the blend already holds. */ size_t image_index = 0; for (ASS_Image *p = images; p; p = p->next, ++image_index) { size_t region = ctx->images[image_index].region; @@ -466,30 +641,7 @@ int plx_ass_render(PlxAss *ctx, int64_t now_ms, int width, int height, const uint8_t *mask = p->bitmap + (size_t)(y - p->dst_y) * p->stride; uint8_t *dst = (uint8_t *)bitmap->rgba + ((size_t)(y - bitmap->y) * bitmap->width + l - bitmap->x) * 4; - for (int x = l; x < r; ++x, dst += 4) { - unsigned a = coverage[mask[x - p->dst_x]]; - if (!a) continue; - if (a == 255) { - ass_store_rgba(dst, color); - continue; - } - ass_store_rgba(dst, ass_blend_pixel(ass_load_rgba(dst), color, a)); - } - } - } - for (size_t region = 0; region < region_count; ++region) { - if (retained[region]) continue; - PlxAssBitmap *bitmap = &ctx->regions[region]; - uint8_t *rgba = (uint8_t *)bitmap->rgba; - for (size_t i = 0; i < bitmap->bytes; i += 4) { - unsigned a = rgba[i + 3]; - /* Transparent pixels are already zero; opaque pixels are already - * straight RGBA. Avoid three variable divisions for every opaque pixel - * (software division on the ARMv7 target), including static vector signs. */ - if (!a || a == 255) continue; - for (int c = 0; c < 3; ++c) { - rgba[i + c] = ass_straight_channel(rgba[i + c], a, ctx->reciprocal[a]); - } + blend_row(dst, mask + (l - p->dst_x), r - l, coverage, color); } } ctx->cache_valid = 1; diff --git a/src/ass_composite.h b/src/ass_composite.h index c5b8cceff8..b9013e0371 100644 --- a/src/ass_composite.h +++ b/src/ass_composite.h @@ -44,16 +44,49 @@ static inline uint32_t ass_blend_pixel(uint32_t dst, uint32_t source, unsigned a return ass_div255_pair(rb) | (ass_div255_pair(ga) << 8); } -/* The numerator is below 65536. Multiplication by floor(65536/alpha) - * underestimates its quotient by at most one; the remainder supplies the exact - * correction. This replaces ARMv7's software divide without changing a pixel. */ -static inline uint8_t ass_straight_channel(unsigned premultiplied, unsigned alpha, - unsigned reciprocal) +/* NEON: eight pixels at once, on every target that has it (32-bit ARM TVs, and arm64 hosts, where the + * same intrinsics exist). PLX_ASS_NO_NEON keeps the scalar row path compiled on such a host, so the + * tests can run both. The arithmetic is the scalar blend's, lane by lane, so the output is + * bit-identical: `ci/test_ass_composite.py` runs every (coverage, source, destination) triple through + * it. A pixel whose coverage is 0 or 255 needs no branch here: the formula leaves the destination or + * stores the source exactly (255*d and 255*s are multiples of 255). */ +#if defined(__ARM_NEON) && !defined(PLX_ASS_NO_NEON) +#define PLX_ASS_NEON 1 +#include + +/* ass_div255_pair for eight 16-bit lanes: n -> (n + 128 + ((n + 128) >> 8)) >> 8, which is + * (n + 127) / 255 for 0 <= n <= 65025. vrsraq adds the rounded high byte (n + ((n + 128) >> 8)) and + * vrshrn adds the 128 and shifts, narrowing: both round in full width, and the largest intermediate, + * 65025 + 254 = 65279 (+128 = 65407), stays below 2^16. */ +static inline uint8x8_t ass_div255_lanes(uint16x8_t n) +{ + return vrshrn_n_u16(vrsraq_n_u16(n, n, 8), 8); +} + +/* The four bytes of an RGBA colour (byte 0 in the low bits, as ass_load_rgba reads it), one per lane. */ +typedef struct { uint8x8_t r, g, b, a; } AssSource8; + +static inline AssSource8 ass_source8(uint32_t color) { - const unsigned n = premultiplied * 255u + alpha / 2; - unsigned q = (n * reciprocal) >> 16; - q += n - q * alpha >= alpha; - return (uint8_t)(q > 255 ? 255 : q); + return (AssSource8){ vdup_n_u8((uint8_t)color), vdup_n_u8((uint8_t)(color >> 8)), + vdup_n_u8((uint8_t)(color >> 16)), vdup_n_u8((uint8_t)(color >> 24)) }; } +/* Eight pixels of one image onto the premultiplied destination `dst` (eight RGBA pixels): `mask` is + * libass's coverage of them, `opacity` the image's opacity in every lane. The coverage is what the + * scalar code reads from its table, (mask * opacity + 127) / 255, computed here instead of looked up: + * a lookup per lane is eight scalar loads and eight stores into a vector. */ +static inline void ass_blend8(uint8_t *dst, uint8x8_t mask, uint8x8_t opacity, AssSource8 source) +{ + const uint8x8_t a = ass_div255_lanes(vmull_u8(mask, opacity)); + const uint8x8_t inverse = vmvn_u8(a); /* 255 - a */ + uint8x8x4_t d = vld4_u8(dst); /* planar: val[c] is channel c of the eight pixels */ + d.val[0] = ass_div255_lanes(vmlal_u8(vmull_u8(d.val[0], inverse), source.r, a)); + d.val[1] = ass_div255_lanes(vmlal_u8(vmull_u8(d.val[1], inverse), source.g, a)); + d.val[2] = ass_div255_lanes(vmlal_u8(vmull_u8(d.val[2], inverse), source.b, a)); + d.val[3] = ass_div255_lanes(vmlal_u8(vmull_u8(d.val[3], inverse), source.a, a)); + vst4_u8(dst, d); +} +#endif + #endif diff --git a/src/ass_fonts.c b/src/ass_fonts.c index e63a456bcf..cc54c64764 100644 --- a/src/ass_fonts.c +++ b/src/ass_fonts.c @@ -1,10 +1,47 @@ /* Compiled INTO libass-plx. The one place the facade reads a private libass structure: the * pinned build installs ass_library.h next to the public headers (ci/build-libass.py) so a test - * can prove that embedded fonts are not extracted again on every replaced track. */ + * can prove that embedded fonts are not extracted again on every replaced track, and so a font the + * caller already holds in memory is not copied a second time into the library (issue #1328). */ #include "ass_internal.h" #include "ass_library.h" +#include +#include size_t plxi_library_fonts(ASS_Library *library) { return library ? library->num_fontdata : 0; } + +int plxi_library_borrow_font(ASS_Library *library, const char *name, const uint8_t *data, size_t len, + size_t *index) +{ + static const char placeholder = 0; + if (!library || !name || !data || !len || len > INT_MAX || !index) + return -1; + size_t held = library->num_fontdata; + /* ass_add_font copies `size` bytes; one placeholder byte costs nothing and leaves a complete + * entry (strdup'd name, array grown, count bumped) whose data pointer we then replace. An + * allocation failure inside it returns without counting the entry, which is the -1 here. */ + ass_add_font(library, name, &placeholder, 1); + if (library->num_fontdata != held + 1) + return -1; + ASS_Fontdata *entry = &library->fontdata[held]; + free(entry->data); + /* libass reads a font's bytes (FT_New_Memory_Face, get_data_embedded) and never writes them. */ + entry->data = (char *)data; + entry->size = (int)len; + *index = held; + return 0; +} + +void plxi_library_forget_borrowed(ASS_Library *library, const size_t *indices, size_t count) +{ + if (!library) + return; + for (size_t i = 0; i < count; i++) { + if (indices[i] >= library->num_fontdata) + continue; + library->fontdata[indices[i]].data = NULL; + library->fontdata[indices[i]].size = 0; + } +} diff --git a/src/ass_internal.h b/src/ass_internal.h index 706c0a2110..a2ae9e51a1 100644 --- a/src/ass_internal.h +++ b/src/ass_internal.h @@ -4,8 +4,22 @@ * the library's version script only lets plx_ass_* leave. */ #include #include +#include /* Fonts the library holds (added through ass_add_font, by the facade or by [Fonts] extraction). */ size_t plxi_library_fonts(ASS_Library *library); +/* Add one font to the library WITHOUT copying it: the library's entry points at `data`, which the + * caller keeps alive and unchanged until plxi_library_forget_borrowed has run and the library has + * been closed. libass's own ass_add_font mallocs and copies; this uses it for the entry (name, + * bookkeeping, array growth) and swaps its one-byte copy for the caller's bytes. Stores the entry's + * index in `*index` and returns 0, or returns -1 with nothing added. */ +int plxi_library_borrow_font(ASS_Library *library, const char *name, const uint8_t *data, size_t len, + size_t *index); + +/* Detach the borrowed entries (by the indices plxi_library_borrow_font gave) from the library, so + * that ass_library_done frees only what libass allocated. Call after the renderer is done: its + * faces read the bytes until then. */ +void plxi_library_forget_borrowed(ASS_Library *library, const size_t *indices, size_t count); + #endif diff --git a/tests/fixtures/make_ass_stress.py b/tests/fixtures/make_ass_stress.py index 6d89ddb50f..5c77b0703d 100755 --- a/tests/fixtures/make_ass_stress.py +++ b/tests/fixtures/make_ass_stress.py @@ -33,7 +33,35 @@ Matroska blocks without a duration (the "no-duration packet" case, start + 4 s) cannot be written by ffmpeg either: its muxer always writes BlockDuration for a subtitle packet. -Needs host ffmpeg and ffprobe only (no mkvmerge): the A/V is encoded once per resolution and every +fonts-over-budget.mkv (issue #1328; `--fonts-over-budget`, built alone because it is ~410 MB): +120 s of 1080p23.976 H.264 High that really carries bitrate (a moving test card under temporal +noise, ~22 Mbit/s mean, VBV-capped near the report's 30 Mbit/s peaks and ~166 KB frames) plus AAC, +ONE default ASS track (spa): ~26 s of ordinary dialogue, then a 90 s karaoke opening of ~2400 +events (up to ~57 active at once: `\\k`/`\\kf` syllables, `\\t` scale and blur, `\\move`, `\\clip`, +`\\fad`, `\\bord`, `\\blur`, `\\be`, glow and echo layers, per-syllable pops, rings, shines and star +bursts, `\\p1` drawings, a 14/s sparkle field; invented lyrics: kanji/kana, romaji, Spanish). It +attaches 17 fonts, 83.4 MB, sized and ordered like the report's: the first seven (32 MB) fill what was +the 32 MiB `ass::MAX_FONT_BYTES` cap (removed by #1328) with faces no style names, the five the +opening's styles name (Antique, Brush Mincho, Dot, Pop, Heavy Gothic: each a different face from the +app's Inter and Noto Sans CJK KR fallback) come next and were all dropped by it, and small ones after +them (the dialogue face among them) still fit. The layout stays as the report's shape: libass is now +handed only the faces a style or `\\fn` names (`ass/font_names.rs`). Fonts, chosen over a hand-drawn set because the glyph SHAPES +must differ from the fallback: the five display faces are real OFL fonts from google/fonts +(Zen Antique, Yuji Syuku, DotGothic16, Mochiy Pop One, Dela Gothic One), pinned by commit and +sha256 like the Noto zips; the fillers and the dialogue face are pkg/ fonts. Every attachment is +`rename_font`ed in pure Python (no fontTools: the `name` table is rewritten to a distinct family, +and a private `PLXP` table of seeded random bytes pads it to the report's exact size; unknown +tables are legal sfnt and FreeType skips them, `fc-scan` and libass read them), so the file is +reproducible from pinned inputs with no pip install. `verify` asserts the attachment count and +bytes (> 32 MiB), distinct families, that every family a style names ends past the budget in +container order and that exactly the opening's five would have been dropped by a mirror of the removed `FontBudget`, that +each face has the glyphs its styles draw, one packet per event, >= 40 events at once, and a video +mean of at least 15 Mbit/s. A libass host render of the opening differs frame for frame between +the embedded faces and the fallback, so a dropped font is visible. +The per-syllable effects (pops, rings, shines, star bursts) are placed from the real glyph advances +of the Japanese face (`libass_advances`), so they sit on their glyphs. + +Needs host ffmpeg and ffprobe only (no mkvmerge); curl is used for downloads when present: the A/V is encoded once per resolution and every fixture remuxes it with `-c copy`, so the subtitle track is never subject to the encoder's sparse subtitle scheduling; the packet count is checked against the script's event count anyway. """ @@ -41,6 +69,9 @@ import hashlib import json import pathlib +import random +import re +import struct import subprocess import sys import urllib.request @@ -399,13 +430,600 @@ def verify(mkv, events, attached): return float(info["format"]["duration"]) +# ---- fonts-over-budget (issue #1328): a dense opening, 17 attached fonts, 83 MB ----------------- + +OP_DURATION = 120 +OP_START = 28.0 # the dialogue runs 0..26 s; the karaoke opening 28..118 s +OP_LINE = 5.0 # eighteen lyric lines of five seconds +BUDGET = 32 * 1024 * 1024 # the removed `MAX_FONT_BYTES` (#1328): the layout's history, not a live constant +FONT_OVERHEAD = 1024 # the removed `FONT_OVERHEAD_BYTES`, likewise +VIDEO_FLOOR_MBPS = 15.0 # the report's opening peaked near 30 Mbit/s +# The opening's libass cost knobs (host: ~3.4 ms per 1080p frame at these values, p95 3.9 ms; the +# television's per-job figure for this file is not measured): background sparkles per second and +# star particles thrown off each syllable. Both scale the events active at once (~45 average). +SPARKLES_PER_S = 14 +BURST_STARS = 3 + +# Display faces from google/fonts (OFL), pinned by commit and sha256 like the Noto zips above. +GF_COMMIT = "c30b221beb0c1da59d563feb91ec8656a1fa972a" +GF_FONTS = { # key -> (path under ofl/, sha256) + "antique": ("zenantique/ZenAntique-Regular.ttf", + "8c5cf7a136837ee705d06bbc133ea18ac06b7dd284f38aace91f5de36725c315"), + "brush": ("yujisyuku/YujiSyuku-Regular.ttf", + "82728ebafc8c97391e2dab633414a806f344b8e4e2227d307179f07b548fca61"), + "dot": ("dotgothic16/DotGothic16-Regular.ttf", + "3ad9af88726d42b40f7f365f0dcac785af73cf20ea6f1d5b44e57cc21150b8f1"), + "pop": ("mochiypopone/MochiyPopOne-Regular.ttf", + "9e009430e1316c271a5f34759c6b65fc343c4e806f193042528887e7235a92c6"), + "gothic": ("delagothicone/DelaGothicOne-Regular.ttf", + "4ff87a0965f1b0505e5a2c58424bc6ad3cff27e56a82f21c2fc9d6b0e3857ee2"), +} +LOCAL_FONTS = { # key -> path in the checkout + "inter": "pkg/appfont.ttf", "inter-bold": "pkg/appfont-bold.ttf", + "hebrew": "pkg/appfont-hebrew.ttf", "hebrew-bold": "pkg/appfont-hebrew-bold.ttf", + "arabic": "pkg/appfont-arabic.ttf", "arabic-bold": "pkg/appfont-arabic-bold.ttf", +} +FAMILY = {"dialogue": "Plx Dialogue Sans", "gothic": "PlxOP Heavy Gothic", "pop": "PlxOP Pop", + "antique": "PlxOP Antique", "brush": "PlxOP Brush Mincho", "dot": "PlxOP Dot"} +# The attachments in container order, sized and ordered like the report's 17 (83 MB): the first +# seven spend a 32 MiB cap on faces no style names, the five the opening's styles name come +# next, and the small ones after them still fit (twelve were held in all, as in the report). The dialogue face is +# natural-sized (Inter Bold is larger than the report's Roboto slot). (file, family, base, style, +# padded-to bytes or None). +SLOTS = [ + ("filler-01.ttf", "Plx Filler 01", "inter", "Regular", 6_220_620), + ("filler-02.ttf", "Plx Filler 02", "hebrew", "Regular", 32_904), + ("filler-03.ttf", "Plx Filler 03", "inter", "Regular", 11_229_304), + ("filler-04.ttf", "Plx Filler 04", "hebrew", "Regular", 37_300), + ("filler-05.ttf", "Plx Filler 05", "hebrew", "Regular", 43_840), + ("filler-06.ttf", "Plx Filler 06", "inter", "Regular", 9_883_924), + ("filler-07.ttf", "Plx Filler 07", "inter", "Regular", 4_808_652), + ("op-antique.ttf", FAMILY["antique"], "antique", "Regular", 7_477_668), + ("op-brush-mincho.ttf", FAMILY["brush"], "brush", "Regular", 13_188_132), + ("op-dot.ttf", FAMILY["dot"], "dot", "Regular", 6_102_048), + ("op-pop.ttf", FAMILY["pop"], "pop", "Regular", 9_705_124), + ("op-heavy-gothic.ttf", FAMILY["gothic"], "gothic", "Regular", 13_715_688), + ("filler-13.ttf", "Plx Filler 13", "hebrew", "Regular", 47_816), + ("filler-14.ttf", "Plx Filler 14", "arabic", "Regular", 245_796), + ("filler-15.ttf", "Plx Filler 15", "arabic-bold", "Bold", 262_448), + ("dialogue.ttf", FAMILY["dialogue"], "inter-bold", "Bold", None), + ("filler-17.ttf", "Plx Filler 17", "hebrew-bold", "Bold", 32_972), +] + + +def _u16(b, o): + return struct.unpack_from(">H", b, o)[0] + + +def _u32(b, o): + return struct.unpack_from(">I", b, o)[0] + + +def sfnt_tables(data): + """(sfnt version tag, {table tag: bytes}) of a single-font TrueType/OpenType file.""" + assert data[:4] in (b"\x00\x01\x00\x00", b"OTTO", b"true"), "not a plain sfnt (a TTC?)" + out = {} + for i in range(_u16(data, 4)): + tag, _sum, off, length = struct.unpack_from(">4sIII", data, 12 + 16 * i) + out[tag] = data[off:off + length] + return data[:4], out + + +def table_sum(b): + b = b + b"\0" * (-len(b) % 4) + return sum(struct.unpack(f">{len(b) // 4}I", b)) & 0xFFFFFFFF + + +def sfnt_build(version, tables): + """Serialize `tables` into a font: sorted directory, 4-byte aligned tables, real checksums.""" + tables = dict(tables) + tags = sorted(tables) + n = len(tags) + entry = n.bit_length() - 1 + search = 16 << entry + head = bytearray(tables[b"head"]) + head[8:12] = b"\0\0\0\0" # checkSumAdjustment is summed as zero + tables[b"head"] = bytes(head) + directory, body, head_at = b"", b"", 0 + for tag in tags: + t = tables[tag] + at = 12 + 16 * n + len(body) + if tag == b"head": + head_at = at + directory += struct.pack(">4sIII", tag, table_sum(t), at, len(t)) + body += t + b"\0" * (-len(t) % 4) + font = bytearray(struct.pack(">4sHHHH", version, n, search, entry, 16 * n - search) + + directory + body) + struct.pack_into(">I", font, head_at + 8, (0xB1B0AFBA - table_sum(bytes(font))) & 0xFFFFFFFF) + return bytes(font) + + +# Name ids copied from the base font (copyright, version, trademark, designer, licence ...); the +# family-bearing ones (1 2 3 4 6, and the typographic 16 17) are rewritten. +KEEP_NAME_IDS = {0, 5, 7, 8, 9, 10, 11, 12, 13, 14} + + +def name_records(table): + count, store = _u16(table, 2), _u16(table, 4) + for i in range(count): + pid, eid, lid, nid, length, off = struct.unpack_from(">6H", table, 6 + 12 * i) + yield pid, eid, lid, nid, table[store + off:store + off + length] + + +def font_family(data): + """The Windows English family name (name id 1) of a font.""" + _, tables = sfnt_tables(data) + for pid, eid, lid, nid, raw in name_records(tables[b"name"]): + if (pid, eid, lid, nid) == (3, 1, 0x409, 1): + return raw.decode("utf-16-be") + raise ValueError("font has no Windows family name") + + +def renamed_name_table(old, family, style): + kept = [(nid, raw) for pid, eid, lid, nid, raw in name_records(old) + if (pid, eid, lid) == (3, 1, 0x409) and nid in KEEP_NAME_IDS] + full = family if style == "Regular" else f"{family} {style}" + new = {1: family, 2: style, 3: f"{full};plx-ass-stress", 4: full, + 6: f"{family.replace(' ', '')}-{style}"} + rows = sorted(kept + [(nid, s.encode("utf-16-be")) for nid, s in new.items()]) + head, store = b"", b"" + for nid, raw in rows: + head += struct.pack(">6H", 3, 1, 0x409, nid, len(raw), len(store)) + store += raw + return struct.pack(">3H", 0, len(rows), 6 + 12 * len(rows)) + head + store + + +def rename_font(data, family, style, pad_to=None): + """`data` with its family renamed, and (when `pad_to` is given) a private `PLXP` table of + seeded random bytes appended so the file is exactly `pad_to` bytes. Unknown tables are legal + sfnt and FreeType skips them; every glyph table is untouched.""" + version, tables = sfnt_tables(data) + tables[b"name"] = renamed_name_table(tables[b"name"], family, style) + font = sfnt_build(version, tables) + if pad_to is not None: + assert pad_to % 4 == 0 and pad_to > len(font) + 16, f"{family}: base is larger than {pad_to}" + pad = random.Random(family).randbytes(pad_to - len(font) - 16) + tables[b"PLXP"] = pad + font = sfnt_build(version, tables) + assert len(font) == pad_to, f"{family}: built {len(font)} bytes, wanted {pad_to}" + return font + + +def font_cmap(data): + """{Unicode code point: glyph id} a font maps to a real glyph (cmap formats 4 and 12).""" + cmap = sfnt_tables(data)[1][b"cmap"] + found = {} + for i in range(_u16(cmap, 2)): + pid, eid, off = struct.unpack_from(">HHI", cmap, 4 + 8 * i) + if (pid, eid) not in ((0, 3), (0, 4), (3, 1), (3, 10)): + continue + fmt = _u16(cmap, off) + if fmt == 4: + segs = _u16(cmap, off + 6) // 2 + ends, starts = off + 14, off + 16 + 2 * segs + deltas, ranges = starts + 2 * segs, starts + 4 * segs + for s in range(segs): + end, start = _u16(cmap, ends + 2 * s), _u16(cmap, starts + 2 * s) + delta, ro = _u16(cmap, deltas + 2 * s), _u16(cmap, ranges + 2 * s) + for c in range(start, min(end, 0xFFFE) + 1): + g = (c + delta) & 0xFFFF if ro == 0 else \ + _u16(cmap, ranges + 2 * s + ro + 2 * (c - start)) + if ro and g: + g = (g + delta) & 0xFFFF + if g: + found[c] = g + elif fmt == 12: + for g in range(_u32(cmap, off + 12)): + start, end, glyph = struct.unpack_from(">III", cmap, off + 16 + 12 * g) + found.update((c, glyph + c - start) for c in range(start, end + 1) if glyph + c - start) + return found + + +def font_codepoints(data): + """The Unicode code points a font maps to a real glyph.""" + return set(font_cmap(data)) + + +def libass_advances(data, text, size): + """The horizontal advance in px of each character of `text` set in the font `data` at font size + `size` (a style's Fontsize at 100% scale, no \\fsp). libass sizes a face with + FT_Request_Size(FT_SIZE_REQUEST_TYPE_REAL_DIM) (`ass_face_set_size`), which maps the face's + ascender - descender, NOT its em (`head.unitsPerEm` takes no part), onto `size`; so a glyph is + advance / (ascender - descender) of `size` wide. The ascender and descender are the ones libass + leaves on the face (`set_font_metrics`, mimicking GDI): OS/2 usWinAscent and usWinDescent (read + as signed) when they do not sum to zero; else FreeType's (sfnt/sfobjs.c: OS/2 sTypo* when + fsSelection bit 7, USE_TYPO_METRICS, is set, else `hhea`); else OS/2 sTypo* when those differ, + else the `head` bounding box. Dela Gothic One's are 1160 and 288 units for a 1000 unit em: a + full-width glyph is 1000 / 1448 = 0.69 of `size` wide. Not modelled, and absent from the + opening's text: kerning (a host render of every line of the opening agrees with these advances + to 2 px). The style's Outline is symmetric and does not move a glyph's centre.""" + _, tables = sfnt_tables(data) + asc, desc = struct.unpack_from(">hh", tables[b"hhea"], 4) + os2 = tables.get(b"OS/2") + typo = (0, 0) + if os2 and len(os2) >= 78: + typo = struct.unpack_from(">hh", os2, 68) + win_asc, win_desc = struct.unpack_from(">hh", os2, 74) + if win_asc + win_desc: + asc, desc = win_asc, -win_desc + elif _u16(os2, 62) & 128: + asc, desc = typo + if asc - desc == 0: + if typo[0] - typo[1]: + asc, desc = typo + else: + _, desc, _, asc = struct.unpack_from(">hhhh", tables[b"head"], 36) # xMin yMin xMax yMax + assert asc - desc > 0, "font has no height" + metrics, count = tables[b"hmtx"], _u16(tables[b"hhea"], 34) + cmap = font_cmap(data) + return [_u16(metrics, 4 * min(cmap[ord(ch)], count - 1)) * size / (asc - desc) for ch in text] + + +# ---- the opening's script ---------------------------------------------------------------------- +# Invented lyrics (so no real song is reproduced). Words are split by "/", the items of a word by +# a space, an item is `kanji-or-kana:romaji`; one `\k` syllable per item. +LYRICS = [ + "夜:yo 明:a け:ke/の:no/空:sora/へ:e/駆:ka け:ke 出:da す:su", + "星:hoshi 屑:kuzu/の:no/海:umi/を:o/越:ko え:e て:te", + "君:kimi/の:no/声:koe/が:ga/響:hibi い:i て:te/い:i る:ru", + "未:mi 来:rai/を:o/掴:tsuka も:mo う:u/今:ima/す:su ぐ:gu に:ni", + "風:kaze/に:ni/舞:ma う:u/花:hana び:bi ら:ra/た:ta ち:chi/よ:yo", + "消:ki え:e な:na い:i/光:hikari/を:o/抱:da い:i て:te", + "走:hashi れ:re/走:hashi れ:re/永:to 遠:wa/の:no/彼:kana 方:ta", + "運:un 命:mei/の:no/扉:tobira/を:o/開:a け:ke ろ:ro", + "約:yaku 束:soku/の:no/場:ba 所:sho/で:de/会:a お:o う:u", +] +SPANISH = [ + "Hacia el cielo del amanecer, salimos corriendo", + "Cruzando el mar de polvo de estrellas", + "Tu voz está resonando dentro de mí", + "Atrapemos el futuro ahora mismo", + "Pétalos que bailan con el viento", + "Abrazando la luz que no se apaga", + "Corre, corre, hasta el confín de la eternidad", + "¡Abre la puerta del destino!", + "Nos veremos en el lugar prometido", +] +DIALOGUE = [ + "Todo está listo para la partida.", "¿Seguro que quieres ir tan lejos?", + "No tengo miedo. Ya no.", "Entonces salimos al amanecer.", + "Dicen que más allá del mar hay una ciudad de luz.", "Y que allí nadie tiene que esconderse.", + "Pues vamos a comprobarlo.", "Prepara las velas. Yo me encargo del mapa.", + "¡Mira! Ya empieza a clarear.", +] +JP_SIZE = 88 # the Japanese line's font size; its glyphs are narrower (`libass_advances`) +JP_Y = 980 # its bottom edge (\an2: baseline plus descent); the glyphs' middle is ~40 px above +STAR = "m 0 -20 l 5 -6 20 -6 8 3 12 18 0 9 -12 18 -8 3 -20 -6 -5 -6" +RING = ("m -20 0 b -20 -11 -11 -20 0 -20 b 11 -20 20 -11 20 0 " + "b 20 11 11 20 0 20 b -11 20 -20 11 -20 0") +PALETTE = ["&HFFE8D0&", "&H80E0FF&", "&HC0A0FF&", "&H70FFC0&", "&HFFFFFF&", "&HA0C8FF&"] + + +def style_line(name, family, size, primary, secondary, outline, bold, border, align, margin_v): + return (f"Style: {name},{family},{size},{primary},{secondary},{outline},&H80000000,{bold},0,0,0," + f"100,100,0,0,1,{border},0,{align},60,60,{margin_v},1") + + +def opening_events(jp_font): + """[(start_s, end_s, layer, style, text)] of the dialogue stretch and the 90 s opening. + `jp_font` is the bytes of the face the OPJp style names: the per-syllable effects are placed + from its real glyph advances. + + The effects are placed for the Japanese line's SETTLED layout (every glyph at 100%): the line's + opening `\\fscx110` (animated to 100 over 0.35 s, before the first syllable) and each syllable's + `\\fscx118` pop are transient, and a pop's scale is inherited by every later glyph until that + glyph's own block, so the glyphs shift while a syllable pops (~15 px at a pop's start for the + line's first glyphs, none for its last; more at the pop's peak). A host render of all 18 lines + puts every pop within 2 px of its glyph's settled position.""" + rnd = random.Random(1328) + ev = [] + for i, line in enumerate(DIALOGUE): + ev.append((1 + i * 2.8, 1 + i * 2.8 + 2.5, 0, "Default", line)) + ev.append((118.2, 119.8, 0, "Default", "Continuará…")) + for n in range(2 * len(LYRICS)): + t0 = OP_START + n * OP_LINE + words = [[tuple(item.split(":")) for item in w.split()] for w in LYRICS[n % len(LYRICS)].split("/")] + items = [it for w in words for it in w] + count = len(items) + weights = [(10, 8, 6, 8, 12, 6, 8, 8, 10, 14)[j % 10] for j in range(count)] + total_cs = 380 + durs = [total_cs * w // sum(weights) for w in weights] + durs[-1] += total_cs - sum(durs) + starts_ms = [400 + 10 * sum(durs[:j]) for j in range(count)] + k = "kf" if n % 2 else "k" + life_ms = int(OP_LINE * 1000) + # Each item's centre at 100% scale, from the font's real advances, as libass centres a + # `\an2\pos(960,_)` line (its width is the sum of the advances, no \fsp): so the per-syllable + # effects sit on their glyph. + widths = [sum(libass_advances(jp_font, jp, JP_SIZE)) for jp, _ in items] + centres, x = [], 960 - sum(widths) / 2 + for width in widths: + centres.append(x + width / 2) + x += width + kara = "".join( + f"{{\\{k}{d}\\t({s},{s + 140},\\fscx118\\fscy118)\\t({s + 140},{s + 360},\\fscx100\\fscy100)}}{jp}" + for (jp, _), d, s in zip(items, durs, starts_ms)) + glow = "".join( + f"{{\\k{d}\\t({s},{s + 180},\\blur14)\\t({s + 180},{s + 420},\\blur9)}}{jp}" + for (jp, _), d, s in zip(items, durs, starts_ms)) + romaji = "" + j = 0 + for w in words: + for _, ro in w: + romaji += f"{{\\kf{durs[j]}}}{ro}" + j += 1 + romaji += " " + pos = f"\\an2\\pos(960,{JP_Y})" + ev += [ + (t0 + 0.2, t0 + OP_LINE, 1, "OPEs", + f"{{\\an2\\pos(960,1045)\\fad(300,300)\\bord3\\blur1\\fscx96\\t(0,500,\\fscx100)}}{SPANISH[n % 9]}"), + (t0, t0 + OP_LINE, 2, "OPRomaji", + f"{{\\an8\\pos(960,70)\\fad(200,300)\\bord4\\blur0.8\\fscx125\\fscy125\\t(0,400,\\fscx100\\fscy100)}}{romaji.rstrip()}"), + (t0, t0 + OP_LINE, 3, "OPJp", + f"{{{pos}\\fad(250,300)\\bord5\\blur0.6\\be1\\fscx110\\fscy110\\t(0,350,\\fscx100\\fscy100)}}{kara}"), + (t0, t0 + OP_LINE, 2, "OPJp", + f"{{{pos}\\fad(250,300)\\bord12\\blur9\\1a&HFF&\\3c&HFFB060&\\3a&H60&}}{glow}"), + (t0, t0 + OP_LINE, 1, "OPJp", + f"{{\\an2\\move(955,{JP_Y + 6},975,{JP_Y - 6},0,{life_ms})\\fad(300,300)\\alpha&HA0&\\bord0\\blur3" + f"\\t(0,{life_ms},\\frz3\\fscx108)}}{''.join(jp for jp, _ in items)}"), + (t0, t0 + OP_LINE, 0, "OPBrush", + f"{{\\an5\\pos(960,470)\\fad(500,500)\\blur14\\bord0\\alpha&H90&\\1c&HFFE8D0&" + f"\\t(0,{life_ms},\\fscx125\\fscy125\\frz5)}}{items[0][0]}"), + (t0, t0 + OP_LINE, 0, "OPFx", + f"{{\\an7\\pos(0,0)\\p1\\1c&H301810&\\1a&H60&\\bord0\\blur18\\clip(0,860,0,1040)" + f"\\t(0,600,\\clip(0,860,1920,1040))\\fad(0,300)}}m 0 860 l 1920 860 1920 1040 0 1040"), + ] + for (jp, _), cx, s in zip(items, centres, starts_ms): + a = t0 + s / 1000 + cy = JP_Y - JP_SIZE // 2 + c1, c2 = rnd.choice(PALETTE), rnd.choice(PALETTE) + ev += [ + (a, a + 0.6, 5, "OPFx", + f"{{\\an5\\move({cx:.0f},{cy},{cx:.0f},{cy - 80},0,600)\\bord3\\blur1.5\\1c{c1}\\3c{c2}" + f"\\t(0,600,\\fscx210\\fscy210\\alpha&HFF&\\blur6)}}{jp}"), + # \an7, not \an5: libass anchors a drawing at its coordinate origin, so under \an5 a ring + # drawn about (0,0) drifts up-left by its own radius as it grows (host render: 18 px + # at +0.15 s, 44 px at +0.45 s); \an7 keeps its centre on pos. + (a, a + 0.55, 4, "OPFx", + f"{{\\an7\\pos({cx:.0f},{cy})\\p1\\bord4\\blur2\\1a&HFF&\\3c{c2}\\fscx30\\fscy30" + f"\\t(0,550,\\fscx260\\fscy260\\3a&HFF&)}}{RING}"), + (a, a + 0.35, 6, "OPFx", + f"{{\\an5\\pos({cx:.0f},{cy})\\bord2\\blur0.5\\1c&HFFFFFF&\\3c&HFFFFFF&" + f"\\clip({cx - 50:.0f},{cy - 70},{cx - 50:.0f},{cy + 70})" + f"\\t(0,350,\\clip({cx - 50:.0f},{cy - 70},{cx + 50:.0f},{cy + 70}))}}{jp}"), + ] + for _ in range(BURST_STARS): + dx, dy = rnd.randint(-110, 110), rnd.randint(-130, -30) + ev.append((a, a + 0.8, 7, "OPFx", + f"{{\\an5\\move({cx:.0f},{cy},{cx + dx:.0f},{cy + dy},0,800)\\p1" + f"\\fscx{rnd.randint(40, 90)}\\fscy{rnd.randint(40, 90)}\\bord1\\blur1" + f"\\1c{rnd.choice(PALETTE)}\\3c&HFFFFFF&\\t(0,800,\\frz{rnd.choice((-1, 1)) * 240}" + f"\\alpha&HFF&)}}{STAR}")) + steps = int(2 * len(LYRICS) * OP_LINE * SPARKLES_PER_S) + for i in range(steps): + a = OP_START + i / SPARKLES_PER_S + life = rnd.uniform(1.6, 2.8) + x1, y1 = rnd.randint(0, 1920), rnd.randint(40, 1040) + x2, y2 = x1 + rnd.randint(-260, 260), y1 + rnd.randint(-200, 200) + s = rnd.randint(60, 200) + ev.append((a, a + life, rnd.choice((1, 3)), "OPFx", + f"{{\\an5\\move({x1},{y1},{x2},{y2},0,{int(life * 1000)})\\p1\\fscx{s}\\fscy{s}" + f"\\bord{rnd.choice((1, 2))}\\blur{rnd.choice((0.8, 1.5, 2.5))}\\1c{rnd.choice(PALETTE)}" + f"\\3c{rnd.choice(PALETTE)}\\fad(150,300)" + f"\\t(0,{int(life * 1000)},\\frz{rnd.choice((-1, 1)) * rnd.randint(120, 360)})}}{STAR}")) + return ev + + +def opening_script(events): + head = ["[Script Info]", "ScriptType: v4.00+", "PlayResX: 1920", "PlayResY: 1080", + "ScaledBorderAndShadow: yes", "WrapStyle: 2", "", "[V4+ Styles]", + "Format: Name, Fontname, Fontsize, PrimaryColour, SecondaryColour, OutlineColour, " + "BackColour, Bold, Italic, Underline, StrikeOut, ScaleX, ScaleY, Spacing, Angle, " + "BorderStyle, Outline, Shadow, Alignment, MarginL, MarginR, MarginV, Encoding", + style_line("Default", FAMILY["dialogue"], 48, "&H00FFFFFF", "&H0000FFFF", "&H26000000", -1, 2, 2, 100), + style_line("OPJp", FAMILY["gothic"], JP_SIZE, "&H00FFFFFF", "&H0070A0FF", "&H00402010", 0, 5, 2, 100), + style_line("OPRomaji", FAMILY["pop"], 52, "&H00FFFFFF", "&H00FFC0A0", "&H00402010", 0, 4, 8, 70), + style_line("OPEs", FAMILY["antique"], 54, "&H00F0FFFF", "&H0000FFFF", "&H00301020", 0, 3, 2, 40), + style_line("OPBrush", FAMILY["brush"], 380, "&H00FFFFFF", "&H00FFFFFF", "&H00000000", 0, 0, 5, 0), + style_line("OPFx", FAMILY["dot"], JP_SIZE, "&H00FFFFFF", "&H00FFFFFF", "&H00FFB060", 0, 2, 5, 0), + "", "[Events]", "Format: Layer, Start, End, Style, Name, MarginL, MarginR, MarginV, Effect, Text"] + body = [f"Dialogue: {layer},{ts(a)},{ts(b)},{style},,0,0,0,,{text}" + for a, b, layer, style, text in events] + return "\n".join(head + body) + "\n" + + +def max_simultaneous(events): + edges = sorted([(round(a * 100), 1) for a, *_ in events] + [(round(e[1] * 100), -1) for e in events], + key=lambda p: (p[0], p[1])) + live = peak = 0 + for _, d in edges: + live += d + peak = max(peak, live) + return peak + + +# ---- build and verify --------------------------------------------------------------------------- + +def download(url, target): + """curl when the host has it (it falls back from an unreachable IPv6 address at once, where + urllib waits out the connect timeout of each address, minutes per file), else urllib.""" + part = target.with_name(target.name + ".part") + try: + subprocess.run(["curl", "-fsSL", "--max-time", "600", "-o", str(part), url], check=True) + except FileNotFoundError: + urllib.request.urlretrieve(url, part) + part.rename(target) + + +def fetch_display_fonts(cache): + """The pinned google/fonts TTFs, downloaded into `cache` and checked against their sha256.""" + got = {} + for key, (path, digest) in GF_FONTS.items(): + target = cache / pathlib.Path(path).name + if not target.exists(): + download(f"https://raw.githubusercontent.com/google/fonts/{GF_COMMIT}/ofl/{path}", target) + have = hashlib.sha256(target.read_bytes()).hexdigest() + if have != digest: + target.unlink() + sys.exit(f"{path}: sha256 {have} != pinned {digest}") + got[key] = target + return got + + +def build_attachments(out): + """Write the 17 renamed, padded fonts of `SLOTS` under /attach-1328; [(path, family)].""" + cache = out / "fonts-1328" + cache.mkdir(exist_ok=True) + base = {key: ROOT / rel for key, rel in LOCAL_FONTS.items()} + base.update(fetch_display_fonts(cache)) + attach = out / "attach-1328" + attach.mkdir(exist_ok=True) + built = [] + for name, family, key, style, size in SLOTS: + path = attach / name + path.write_bytes(rename_font(base[key].read_bytes(), family, style, size)) + built.append((path, family)) + return built + + +def encode_av_hibr(out): + """1080p23.976 High-profile H.264 that cannot compress (moving test card under temporal noise) + at 22 Mbit/s capped at 30, plus stereo AAC, OP_DURATION seconds: the decoder feed has real work. + The small VBV buffer and ipratio 1.0 keep every frame, keyframes included, near the report's + (frames up to ~166 KB; a default buffer made the first IDR 775 KB).""" + path = out / "av-1080p23976-hibr.mkv" + if not path.exists(): + part = out / "av-1080p23976-hibr.part" + run(["ffmpeg", "-v", "error", "-y", + "-f", "lavfi", "-i", "testsrc2=s=1920x1080:r=24000/1001,noise=alls=28:allf=t+u", + "-f", "lavfi", "-i", "anoisesrc=color=pink:amplitude=0.05:sample_rate=48000", + "-t", str(OP_DURATION), "-c:v", "libx264", "-preset", "veryfast", "-profile:v", "high", + "-level", "4.1", "-b:v", "22M", "-maxrate", "30M", "-bufsize", "3M", "-g", "48", + "-x264-params", "ipratio=1.0", "-pix_fmt", "yuv420p", "-c:a", "aac", "-b:a", "160k", "-ac", "2", "-f", "matroska", str(part)]) + part.rename(path) + return path + + +def mux_fonts(out, av, name, ass_text, attachments): + ass = out / f"{name}.ass" + ass.write_text(ass_text, encoding="utf-8") + cmd = ["ffmpeg", "-v", "error", "-y", "-i", str(av), "-i", str(ass)] + for path, _ in attachments: + cmd += ["-attach", str(path)] + cmd += ["-map", "0:v", "-map", "0:a", "-map", "1:s", "-c", "copy", "-disposition:s:0", "default", + "-metadata:s:s:0", "language=spa", "-t", str(OP_DURATION)] + for i, (path, _) in enumerate(attachments): + cmd += [f"-metadata:s:t:{i}", "mimetype=font/ttf", f"-metadata:s:t:{i}", f"filename={path.name}"] + mkv = out / f"{name}.mkv" + run(cmd + [str(mkv)]) + return mkv + + +def admitted(sizes): + """The set of indexes the removed `ass::FontBudget::admit` kept: container order, each font charged its + bytes plus FONT_OVERHEAD, one that does not fit skipped while a later smaller one may fit.""" + used, kept = 0, set() + for i, size in enumerate(sizes): + if size and used + size + FONT_OVERHEAD <= BUDGET: + used += size + FONT_OVERHEAD + kept.add(i) + return kept + + +def verify_fonts_over_budget(mkv, events, ass_text, attachments): + info = json.loads(run(["ffprobe", "-v", "error", "-show_streams", "-show_format", "-of", "json", + str(mkv)])) + by_type = lambda t: [s for s in info["streams"] if s["codec_type"] == t] + name = mkv.name + (video,), (audio,), (sub,), atts = (by_type("video"), by_type("audio"), by_type("subtitle"), + by_type("attachment")) + assert (video["codec_name"], video["profile"], video["width"], video["height"]) == \ + ("h264", "High", 1920, 1080), f"{name}: want 1080p H.264 High" + assert video["r_frame_rate"] == "24000/1001", f"{name}: frame rate {video['r_frame_rate']}" + assert audio["codec_name"] == "aac" and audio["channels"] == 2, f"{name}: want stereo AAC" + assert sub["codec_name"] == "ass" and sub["disposition"]["default"] == 1, f"{name}: ASS not default" + assert sub["tags"].get("language") == "spa", f"{name}: subtitle language" + duration = float(info["format"]["duration"]) + assert abs(duration - OP_DURATION) < 1, f"{name}: {duration} s" + + # The attachments: count, bytes, order, and that they are the files this script wrote. + assert len(atts) == len(attachments) >= 17, f"{name}: {len(atts)} attachments" + sizes = [int(a["extradata_size"]) for a in atts] + assert [(a["tags"]["filename"], n) for a, n in zip(atts, sizes)] == \ + [(p.name, p.stat().st_size) for p, _ in attachments], f"{name}: attachments differ from the files" + assert sum(sizes) > BUDGET, f"{name}: {sum(sizes)} font bytes do not exceed {BUDGET}" + fonts = [p.read_bytes() for p, _ in attachments] + families = [font_family(f) for f in fonts] + assert families == [fam for _, fam in attachments] and len(set(families)) == len(families), \ + f"{name}: families are not distinct" + + # The styles: every family they name is in one attachment that ends past the budget in + # container order, and the opening's are ones the budget rejects today. + styled = {m[1]: m[2] for m in (re.match(r"Style: ([^,]+),([^,]+),", l) for l in ass_text.split("\n")) + if m} + kept = admitted(sizes) + ends = [sum(sizes[:i + 1]) for i in range(len(sizes))] + for style, family in styled.items(): + i = families.index(family) + assert ends[i] > BUDGET, f"{name}: {style}'s {family} ends before the budget is spent" + assert (i in kept) == (style == "Default"), \ + f"{name}: {style}'s {family}: the budget {'keeps' if i in kept else 'drops'} it" + used = "".join(re.sub(r"\{[^}]*\}", "", t).replace("\\N", "") for _, _, _, s, t in events + if s == style and "\\p1" not in t) + missing = {c for c in used if not c.isspace() and ord(c) not in font_codepoints(fonts[i])} + assert not missing, f"{name}: {family} lacks {''.join(sorted(missing))}" + dropped = [families[i] for i in range(len(sizes)) if i not in kept] + assert set(dropped) == {f for st, f in styled.items() if st != "Default"}, \ + f"{name}: the budget drops {dropped}, not exactly the opening's faces" + + packets = run(["ffprobe", "-v", "error", "-select_streams", str(sub["index"]), + "-show_entries", "packet=pts_time", "-of", "csv=p=0", str(mkv)]).split() + assert len(packets) == len(events), f"{name}: {len(packets)} packets != {len(events)} events" + peak = max_simultaneous(events) + assert peak >= 40, f"{name}: only {peak} events at once" + + frames = [tuple(map(float, p.split(","))) for p in run( + ["ffprobe", "-v", "error", "-select_streams", str(video["index"]), "-show_entries", + "packet=pts_time,size", "-of", "csv=p=0", str(mkv)]).split()] + mbps = sum(s for _, s in frames) * 8 / duration / 1e6 + second = {} + for t, s in frames: + second[int(t)] = second.get(int(t), 0) + s * 8 / 1e6 + assert mbps >= VIDEO_FLOOR_MBPS, f"{name}: video {mbps:.1f} Mbit/s < {VIDEO_FLOOR_MBPS}" + return {"duration": duration, "attachments": len(atts), "font_bytes": sum(sizes), + "dropped": dropped, "events": len(events), "peak_events": peak, "video_mbps": mbps, + "video_peak_mbps": max(second.values()), "max_frame_bytes": int(max(s for _, s in frames))} + + +def build_fonts_over_budget(out): + attachments = build_attachments(out) + av = encode_av_hibr(out) + jp_font = next(p for p, family in attachments if family == FAMILY["gothic"]).read_bytes() + events = opening_events(jp_font) + ass_text = opening_script(events) + mkv = mux_fonts(out, av, "fonts-over-budget", ass_text, attachments) + m = verify_fonts_over_budget(mkv, events, ass_text, attachments) + print(f"{mkv.name}: {m['duration']:.1f} s, {m['events']} events (up to {m['peak_events']} at once), " + f"{m['attachments']} fonts = {m['font_bytes']} bytes (old cap {BUDGET})") + print(f" video {m['video_mbps']:.1f} Mbit/s mean, {m['video_peak_mbps']:.1f} peak second, " + f"largest frame {m['max_frame_bytes']} bytes") + print(" the budget drops: " + ", ".join(m["dropped"])) + print(f"\noutput: {out}") + + def main(): ap = argparse.ArgumentParser(description=__doc__.split("\n")[0]) ap.add_argument("output", type=pathlib.Path, help="directory for fonts, scripts and MKVs") - out = ap.parse_args().output.resolve() + ap.add_argument("--fonts-over-budget", action="store_true", + help="build only fonts-over-budget.mkv (120 s, ~410 MB, 17 fonts / 83 MB; issue #1328)") + args = ap.parse_args() + out = args.output.resolve() if ROOT in out.parents or out == ROOT: sys.exit("refusing to write into the repository checkout") out.mkdir(parents=True, exist_ok=True) + if args.fonts_over_budget: + build_fonts_over_budget(out) + return fonts = fetch_fonts(out) av = encode_av(out, "av-1080p24.mkv", "1920x1080", 24) av4k = encode_av(out, "av-2160p60.mkv", "3840x2160", 60) diff --git a/tools/hud_bundle.py b/tools/hud_bundle.py index 68c4ec99f6..aa3a9ad508 100644 --- a/tools/hud_bundle.py +++ b/tools/hud_bundle.py @@ -223,7 +223,7 @@ def copy(path, target): raise ValueError("binary does not report the source being packaged") # Both app text shaping and native ASS use this dlopen-only library. Prove that the # relocated tree can load its exports instead of trusting ldd's view of the executable. - subprocess.run(["python3", "-c", "import ctypes,sys; lib=ctypes.CDLL(sys.argv[1]); assert lib.plx_ass_abi_version() == 4; assert lib.plx_ass_create; assert lib.plx_ass_text_measure", str(stage / "pkg" / native_text.name)], + subprocess.run(["python3", "-c", "import ctypes,sys; lib=ctypes.CDLL(sys.argv[1]); assert lib.plx_ass_abi_version() == 5; assert lib.plx_ass_create; assert lib.plx_ass_text_measure", str(stage / "pkg" / native_text.name)], env={**env,"LD_LIBRARY_PATH":str(stage / "lib")},check=True) prepare_authoring(stage, inputs, wrapper, env, info) metrics = subprocess.check_output([str(wrapper), "--hud-text-metrics"], env=env) From 9f43dba98d01216b57f4551f62b4487bec6f18eb Mon Sep 17 00:00:00 2001 From: Gleb Linnik Date: Thu, 12 Nov 2026 16:24:56 +0300 Subject: [PATCH 2/7] Player: Previous always plays the previous episode, and a fresh play has one to go to (#1349) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit ## What The player's Previous button now always plays the previous episode, and a fresh play has one to go to. - **Previous is the queue step only.** It used to become Restart five seconds into an episode: it sought the playing episode to 0 (rewriting its playhead and viewOffset), and under five seconds it was disabled again. The restart face and its five-second rule are gone. From beginning stays on the Info tab, in the More menu of HUD packages that fold it, and on the detail hero. - **A fresh play gets the show's queue.** Continue Watching, the detail page's Play/Resume, Up Next and autoplay used to POST `uri=&continuous=1`, which PMS answers with the episode and its successors only, so the queue had no row before the playing one. An episode whose show is known now asks for `uri=&key=` (no `continuous`), accepts it only when the answer selects that episode, and otherwise falls back to the old request. When rows precede the selection but the window opens on it, the existing window read runs once. - **Leaving an episode keeps its progress.** Previous and Next never seek; the outgoing episode's stop report carries its real position, and the handoff's window re-read waits (bounded, 1500 ms, on the worker leg) for that report, so stepping back to an episode just left resumes it where it was. - HUD plugins: the `restart` face of the `previous` command still parses (published packages keep loading) and is no longer selected. - `tests/mock_pms.py` serves the show-keyed queue. Skip and Up Next prompts are unchanged: while one occupies the row, Previous/Next are not on it (the owner asked for no redesign there). ## Verification - Regression `a_fresh_play_of_a_middle_episode_lands_the_episodes_before_it` failed against the old request and passes; hostsim rig tests: Previous mid-episode plays the prior episode, seeks nowhere and the outgoing episode reports its real position; Previous on the first episode is disabled; Next. - Crate suites: plx_media 2015, plx_appkit 953, plx_screens 2137, plx_plex 801, plx_data 1540, application hostsim 1529, plx_media hostsim 2295 passed; `make check-nodef`, `check-deps`, ledger ratchet, localization, module layers, `tests/test_harness.py` (725), mock_pms selftest green. - TV, debug build on a real server, owner-driven: the play's queue came back as the show's (`playqueue: … rows=30`), `queue transport: Previous` went to the episode before and `Next` came back; the owner checked it. Not verified: whether PMS lists season-0 specials before S1E1 (Previous from S1E1 could land on one); PMS's window size on a long show; `tests/playback_navigation_sim.py`'s Previous block (needs `make sim-macos`; CI). The frozen published schema descriptions `site/hud/schema/hud/1.0.json`/`1.1.json` still say Previous may be "a restart of this one". 🤖 Generated with [Claude Code](https://claude.com/claude-code) --- docs/hud-plugins.md | 12 +- docs/playback-navigation.md | 24 +- docs/pms-api.md | 34 +- docs/release-notes/v0.10.0.md | 1 + locales/en/widgets.json | 2 +- rust-modules/appkit/src/hud_format/compile.rs | 4 +- .../appkit/src/hud_format/compile_tests.rs | 2 +- rust-modules/appkit/src/more_menu.rs | 6 +- .../src/player_hud/census-classic.golden.tsv | 1496 +++++++---------- rust-modules/appkit/src/player_hud/command.rs | 57 +- rust-modules/appkit/src/player_hud/model.rs | 4 +- rust-modules/appkit/src/player_hud/render.rs | 10 +- .../player_hud/render/tests/conformance.rs | 2 +- rust-modules/appkit/src/player_hud/state.rs | 19 +- rust-modules/appkit/src/player_hud/tests.rs | 51 +- rust-modules/appkit/src/player_hud/travel.rs | 8 +- rust-modules/data/src/metadata.rs | 37 + rust-modules/media/src/player/mod.rs | 2 +- rust-modules/media/src/route/decision.rs | 52 +- .../route/decision_audio_enhancement_tests.rs | 1 + .../media/src/route/decision_plan_tests.rs | 2 +- .../route/decision_queue_predecessor_tests.rs | 327 ++++ .../src/route/decision_resolve_route_tests.rs | 1 + .../src/route/decision_timeline_tests.rs | 5 +- .../src/route/decision_track_memory_tests.rs | 1 + .../media/src/route/decision_version_tests.rs | 18 +- rust-modules/media/src/route/mod.rs | 2 +- rust-modules/media/src/route/plan.rs | 13 +- .../src/route/plan_audio_enhancement_tests.rs | 1 + .../media/src/route/queue_navigation.rs | 60 +- rust-modules/plex/src/plex/timeline.rs | 197 ++- rust-modules/screens/src/detail/extras.rs | 1 + rust-modules/screens/src/detail/mod.rs | 5 + .../src/player/compiled_geometry_tests.rs | 2 +- .../screens/src/player/hud_pins_tests.rs | 2 +- rust-modules/screens/src/player/mod.rs | 4 +- .../screens/src/player/overlay_tests.rs | 2 +- rust-modules/screens/src/player/seek_jump.rs | 4 +- rust-modules/screens/src/registry.rs | 4 + rust-modules/src/app/content.rs | 4 +- rust-modules/src/app/input.rs | 1 + rust-modules/src/app/playback.rs | 4 +- .../src/app/playback/queue_transport.rs | 13 +- rust-modules/src/app/run.rs | 205 ++- rust-modules/src/app/run_uri_boot_tests.rs | 2 +- rust-modules/src/dev/scenarios.rs | 2 +- rust-modules/src/focusprobe.rs | 4 +- tests/mock_pms.py | 64 +- tests/playback_navigation_sim.py | 16 +- 49 files changed, 1690 insertions(+), 1100 deletions(-) create mode 100644 rust-modules/media/src/route/decision_queue_predecessor_tests.rs diff --git a/docs/hud-plugins.md b/docs/hud-plugins.md index f1570bfa2f..f0548cadfa 100644 --- a/docs/hud-plugins.md +++ b/docs/hud-plugins.md @@ -52,7 +52,9 @@ HUD drawing and state (`:493-497`), `BTN_MORE = BTN_N - 1` (`:670`), `pub enum Action { Previous, Next, Subtitles, Audio, More }` (`:731`). `Controls::enabled` (`:758-761`): Previous and Next by their own flags, the other three always; `Controls::pair` (`:755-757`) makes Previous/Next come and go together; - `Controls::icon` swaps in `Icon::Restart` for Previous (`:772-776`). + `Controls::icon` once swapped in `Icon::Restart` for Previous from five seconds in; it no longer + does, and `Controls` has no `restart` field: Previous is the previous episode of the play queue and + nothing else, so it is enabled exactly when the playing row has a predecessor. - F7. Row identity is an INDEX: `Controls::action(i)` (`:767-771`), `ControlSlot::items()` 5/2/1 (`:804-809`), `RowPop = CtlPop<{ BTN_N as usize }>` (`:512`), element ids `ELEM_SCRUB 0` `:1384`, `ELEM_ROW_BASE 10` `:1387`, `ELEM_TAB_BASE 20` `:1389`, `ELEM_FAILURE_BASE 30` `:1397`. Up Next @@ -586,7 +588,9 @@ through `TransportButton`, `pill` through `TabPill` labelled by the command's ap through `widgets::Button`), `size` (px), `rest` (`film` today's, `bare` glyph only, `solid` the focused colours at rest; `custom` is refused because format 1.0 defines no keys for its inks), `focus` (`accent` today's, `ring`: rest face plus the white focus ring), `face` (optional: the command's classic glyph when absent). A pair that draws one face twice (`solid` with `accent`) is rejected. Faces: an asset -or `builtin:`; `toggle_play` takes `{playing, paused}`; `previous` takes `{default, restart}`; +or `builtin:`; `toggle_play` takes `{playing, paused}`; `previous` takes `{default, restart}` (the `restart` face is still required and +parsed, because published packages carry it and format 1.0 is frozen, but the app never selects it any more: +Previous is always the previous episode and wears `default`); `toggle_stats` takes `{on, off}`; a `seek` with `by: "setting"` takes `{by_interval: {...}, default}`. A glyph's visual size is its viewBox padding over the shared `DISC_ICON_RATIO` (`ui/src/widgets.rs:1394`). `TransportButton` gains the `bare` rest face and the `ring` focus face in place. @@ -627,7 +631,7 @@ Each maps to an action the app already performs; the performer is shared with th |---|---|---|---|---| | `toggle_play` | - | the `Key::Play|Pause|PlayPause` arm, extracted to one function, PlayPause path incl. `take_target` (F13) | always | keys, OK on picture | | `seek` | `by`: a nonzero integer of seconds, whose sign is the direction (then `dir` is forbidden), or `"setting"`, which requires `dir`: `back` or `forward` | `commit_jump`'s `SeekBurst` and `CommitSeek`, extracted with the signed step in ns as a parameter (today `signed_step(forward)` = `scrub_step_ns() x +-1`, `seek_jump.rs:16-18`; a literal `by` passes `by x 10^9`) and a keep-HUD flag (today `self.hud.hide(false)`, `:43`): a control press keeps the HUD up and extends its linger, the hidden-HUD tap still hides it (F14) | duration known (`seek_jump.rs:39-40`) | Left/Right, FF/REW, hold | -| `previous`, `next` | - | `queue_transport::step` (F15) | `Controls::for_item` (`player_hud.rs:749-752`): the pair is absent for a non-episode or with neither neighbour; Previous is enabled by `previous || restart` (a restart target counts), Next by `next` (F6) | none | +| `previous`, `next` | - | `queue_transport::step` (F15) | `Controls::for_item` (`player_hud.rs:749-752`): the pair is absent for a non-episode or with neither neighbour; Previous is enabled by `previous` alone (the playing row has a predecessor in the play queue; there is no restart target any more), Next by `next` (F6) | none | | `open_tracks` | `tab`: `subtitles`, `audio` | `OverlayKind::Tracks { tab: 1 | 0 }` (F15) | always (F6) | none | | `open_more` | `page`: `root` (default), `quality`, `speed` | `OverlayKind::More { page }`; `quality` is what was `More { quality: true }` (F18); `speed` is the address PR3 added by turning that bool into `page: Option` (F16) | always | none | | `toggle_section` | `section`: `chapters`, `info` | opens or closes the section (D6) | chapters: the playing item has chapters (`chapters_panel::has_chapters`) | none | @@ -1201,7 +1205,7 @@ with computed values; PR17's compile reports the chapters plus info block's top | 7-8 | "5:27" left, "-52:33" right | `playbar.clocks: ends`; `fmt::clock`, minus from `fmt_time` (F27) | `draw_clock` | | 9 | Speech bubble | `open_tracks {tab: subtitles}`, `rest: bare`; the menu reaches Audio (F15) | `TransportButton` | | 10 | Arrow "10" | `seek {by: -10}`, static face | `TransportButton`; the extracted jump | -| 11 | Previous | `previous`, `{default, restart}` faces | `TransportButton` | +| 11 | Previous | `previous`, `{default, restart}` faces (`restart` is accepted and never selected) | `TransportButton` | | 12 | Large white disc | `toggle_play`, size 64, `rest: solid`, `focus: ring`, `{playing, paused}` | `TransportButton`; the extracted toggle | | 13 | Next | `next` | `TransportButton` | | 14 | Arrow "30" | `seek {by: 30}`, static face | `TransportButton` | diff --git a/docs/playback-navigation.md b/docs/playback-navigation.md index 7a2e829974..9426354f46 100644 --- a/docs/playback-navigation.md +++ b/docs/playback-navigation.md @@ -23,10 +23,23 @@ ends the burst; a same-item seek reload preserves it. A dismissed immediate seek through the first new picture; ordinary rebuffering and failures retain their diagnostic read-out. The default control row is **Previous, Next**, a wider gap, then **Subtitles, Audio, More**. -Previous restarts at five seconds or later and shows the restart arrow. Before five seconds it -plays the preceding queue item. Unavailable buttons are dimmed, have no pointer stop and are +Previous always plays the preceding queue item, whatever the playhead: it is the previous episode +and nothing else, and it never seeks. The episode it leaves is stopped at the position it really +reached, so its resume point is kept. It is dimmed on the first episode of the queue. Starting the +current episode over is **From beginning** on the Info tab (and a `…` menu row in a HUD package +that folds it), not a second meaning of Previous. +Unavailable buttons are dimmed, have no pointer stop and are skipped by keyboard focus. Entering the row focuses Next when enabled, otherwise its first -enabled button. Marker offers and Up Next retain their precedence over this row. +enabled button. Marker offers and Up Next retain their precedence over this row: while one +occupies it the discs, Previous and Next among them, are not drawn or focusable. + +A fresh play of an episode (a Continue Watching tile, the detail page's Play or Resume) creates its +PlayQueue from the **show** with the episode as the queue's `key`, so the queue holds the episodes +before the playing one as well as after it. (Previous, Next, Up Next and autoplay hand off inside +the queue that exists and never create one.) An episode-only queue holds just the episode and its +successors, and Previous then had nowhere to go. +The show-based answer is believed only when its selected row is the episode asked for; otherwise +the play falls back to the episode-only request. ## Queue continuity and maintenance @@ -40,7 +53,8 @@ Its main files are: | `appkit/src/player_hud/seek_jump.rs` | Short-burst requested-target accumulation | | `appkit/src/player_hud/state.rs` | One default disc row and its availability | | `appkit/src/player_hud/classic.rs` | The baseline row's shared HStack geometry (width and band); `render.rs` draws the row and its icons from the compiled `builtin:classic` | -| `media/src/route/queue_navigation.rs` | Queue neighbors, continuation and five-second rule | +| `media/src/route/queue_navigation.rs` | Queue neighbors and continuation | +| `plex/src/plex/timeline.rs` | `create_play_queue`: an episode's queue is built from its show, so the episodes before it are in it | | `plex/src/plex/session/playback_options.rs` | Independent preference types and defaults | | `plex/src/plex/queue_window.rs` | Read the existing PlayQueue around a target item | | `src/app/playback/queue_transport.rs` | Queue handoff shared with Up Next | @@ -66,7 +80,7 @@ python3 tests/playback_navigation_sim.py --sim rust-modules/target-sim/debug/plx --media /tmp/navigation-media --out /tmp/navigation-evidence ``` -This exercises queue continuity, restart after five seconds, an immediate hidden-HUD FF tap, +This exercises queue continuity, Previous disabled (and never a restart) on the first episode, an immediate hidden-HUD FF tap, and visible-HUD D-pad confirmation. Add `--instant-dpad` to exercise the optional D-pad path. For a repeated-keyframe regression, make a copy with 12-second keyframes (24 fps): diff --git a/docs/pms-api.md b/docs/pms-api.md index 0a42ae68a8..4987d15ae3 100644 --- a/docs/pms-api.md +++ b/docs/pms-api.md @@ -1002,7 +1002,8 @@ GET /library/metadata/{rk}?includeMarkers=1&includeChapters=1 ### Up Next — the `continuous=1` PlayQueue already carries it (verified live 2026-07-29) There is no "what plays next" endpoint to call: the `POST /playQueues?continuous=1` the app makes -for **every** playback (§7) returns the show's remaining episodes after the selected one, each a +for a playback of an item with no show (§7; an episode whose show is not known, a movie) returns +the show's remaining episodes after the selected one, each a **full** `Metadata` row — `thumb`, `parentIndex`/`index`, `duration`, `viewOffset`, `summary`, and `Media[0].videoCodec`/`audioCodec` + `Part[0].key`. That is everything both the Up Next card and a subsequent direct-play need, so the feature costs zero extra round-trips. @@ -1014,6 +1015,37 @@ subsequent direct-play need, so the feature costs zero extra round-trips. show returns `playQueueTotalCount: 1`, i.e. no successor. A movie behaves the same way. - The POST needs an `X-Plex-Client-Identifier`; without it PMS answers with an empty body. +### A show-keyed PlayQueue — the episodes BEFORE the playing one (spec- and client-derived, NOT yet observed live) + +The episode-only request above answers the selected episode and the ones after it with +`playQueueSelectedItemOffset: 0`; there is **no row before the selected one**, so a queue created +that way can never serve Previous (the HUD's Previous is `index > 0` of the queue's rows, and an +episode played fresh from a Continue Watching tile or a detail page started at index 0). The +`GET /playQueues/{id}` window parameters (`includeBefore`, `center`) only window a queue that +already holds the rows, so they cannot add predecessors the queue lacks. + +For an episode whose show is known the app therefore asks for the queue of the SHOW with the +episode as its first item to play: + +``` +POST /playQueues?type=video&uri=server://{machine}/com.plexapp.plugins.library/library/metadata/{showRk} + &key=/library/metadata/{episodeRk}&shuffle=0&repeat=0 (no `continuous`) +``` + +`docs/plex-openapi.json` documents `key` as "the key of the first item to play" and the show URI as +a valid `uri` (`onDeck` applies to show/season queues); python-plexapi's `PlayQueue.create(show, +startItem=episode)` sends exactly `uri` + `key`. The answer is expected to hold the show's episodes +in order with the keyed one selected (`playQueueSelectedItemID`/`Offset` on it). **That response +shape has not been observed on a live server** (no PMS was probed for this change), so the app +trusts it only when the selected row IS the requested episode on this server; a server that +ignores `key` (and answers the first episode, or the show itself), refuses it, or does not answer +costs one extra round trip (and an unused show queue left on the server): the app then asks the +episode-only request above, as it always did. When the offset says +rows precede the selected one but the answer's window opens on it, the existing window read +(`GET /playQueues/{id}?center=…`, `queue_window.rs`) is spent once to fetch them. To verify live +(owner's TV or a mock-free read-only probe): play an episode from the middle of a season and read +the `playQueueSelectedItemOffset` and the first row of the log's queue line. + ### Show chain (verified against a show, ratingKey 1857) ``` diff --git a/docs/release-notes/v0.10.0.md b/docs/release-notes/v0.10.0.md index 768467db1e..45d646626a 100644 --- a/docs/release-notes/v0.10.0.md +++ b/docs/release-notes/v0.10.0.md @@ -41,6 +41,7 @@ This release lets you choose, install and replace the player's on-screen control - **A version you chose with Play Version could be re-encoded** at a quality choice it fitted, because its own bitrate was not read. - **Opening the Library straight onto Collections or Recommended** now puts focus on the first tile instead of leaving it on the top bar. A related symptom, a flash of the old layout when going Back to the grid, could not be reproduced here and has not been changed; neither has been seen on a television. - **A set with four cores and 2 GiB of memory** decodes posters with three workers instead of two. +- **Previous now always goes to the previous episode.** Five seconds into an episode the button used to restart it, which rewound the progress Plex had saved for it, and before that it was dimmed, because an episode started from Continue Watching or a page had no earlier episode in its queue. The queue is now built from the show, so the earlier episodes are in it, and Restart is no longer a button face; **From beginning** on the player's Info tab (and in the `…` menu of a HUD that lists it) starts the episode over. The show-based queue request follows Plex's published API and was not observed against a real server: a server that does not answer it as expected leaves the queue as before, with Previous dimmed on that episode. ## Changed diff --git a/locales/en/widgets.json b/locales/en/widgets.json index f623750168..b63e33d3e3 100644 --- a/locales/en/widgets.json +++ b/locales/en/widgets.json @@ -48,7 +48,7 @@ }, "widgets.menu.previous_episode": { "value": "Previous Episode", - "description": "Row in the player overflow menu, shown when the playing episode has an earlier one (or can restart): starts the previous episode, or this one over. Same noun as widgets.next.episode, in the same capitalisation." + "description": "Row in the player overflow menu, shown when the playing episode has an earlier one: starts the previous episode. Same noun as widgets.next.episode, in the same capitalisation." }, "widgets.badge.forced": { "value": "FORCED", diff --git a/rust-modules/appkit/src/hud_format/compile.rs b/rust-modules/appkit/src/hud_format/compile.rs index 7f51d4af31..1ec54e8943 100644 --- a/rust-modules/appkit/src/hud_format/compile.rs +++ b/rust-modules/appkit/src/hud_format/compile.rs @@ -330,7 +330,9 @@ pub enum FaceRef { /// A control's `face` as the package wrote it: absent (the command's classic glyph), one glyph, or one per /// state of the command (`playing`/`paused`, `default`/`restart`, `on`/`off`, and a `seek by: setting`'s -/// `default` plus its ladder keyed by seconds). +/// `default` plus its ladder keyed by seconds). `previous`'s `restart` state is still parsed, because +/// published packages carry it and the schema is frozen, but the renderer never selects it any more +/// (`render::face_state`): Previous is always the previous episode. #[derive(Clone, Debug, PartialEq, Eq, Default)] pub struct Face { pub single: Option, diff --git a/rust-modules/appkit/src/hud_format/compile_tests.rs b/rust-modules/appkit/src/hud_format/compile_tests.rs index 743210d83f..33e0a528bb 100644 --- a/rust-modules/appkit/src/hud_format/compile_tests.rs +++ b/rust-modules/appkit/src/hud_format/compile_tests.rs @@ -67,7 +67,7 @@ fn marker(kind: MarkerKind, final_seg: bool) -> Marker { Marker { kind, start_ms: 1_000, end_ms: 2_000, final_seg } } -const PAIR: Controls = Controls { previous: true, next: true, restart: false }; +const PAIR: Controls = Controls { previous: true, next: true }; /// Every field of two geometries that differ, as text; empty when they are equal. fn diff(got: &HudGeometry, want: &HudGeometry) -> Vec { diff --git a/rust-modules/appkit/src/more_menu.rs b/rust-modules/appkit/src/more_menu.rs index f57720da2a..9a29f6ee38 100644 --- a/rust-modules/appkit/src/more_menu.rs +++ b/rust-modules/appkit/src/more_menu.rs @@ -1855,7 +1855,7 @@ mod focus_tests { fold("go_to_detail", None), ]; let full = crate::player_hud::model::HudModel { - controls: crate::player_hud::Controls::for_item(true, true, true, false), + controls: crate::player_hud::Controls::for_item(true, true, true), chapters: true, detail: true, ..Default::default() @@ -2120,7 +2120,7 @@ mod folded_tests { /// A frame's model for an episode with both neighbours (or a movie with none), chapters and a detail page. fn model(episode: bool) -> HudModel { - HudModel { controls: Controls::for_item(episode, true, true, false), chapters: true, detail: true, ..HudModel::default() } + HudModel { controls: Controls::for_item(episode, true, true), chapters: true, detail: true, ..HudModel::default() } } fn rows_of(package: &Arc, model: &HudModel) -> Vec { @@ -2173,7 +2173,7 @@ mod folded_tests { assert!(!movie.contains(&HudCommand::Previous) && !movie.contains(&HudCommand::Next), "{movie:?}"); assert!(movie.contains(&HudCommand::OpenTracks(TrackTab::Subtitles)), "the rest still folds: {movie:?}"); // an episode with only a previous neighbour has no Next row either - let only_previous = HudModel { controls: Controls::for_item(true, true, false, false), ..model(true) }; + let only_previous = HudModel { controls: Controls::for_item(true, true, false), ..model(true) }; let rows = commands(&rows_of(&minimal, &only_previous)); assert!(rows.contains(&HudCommand::Previous) && !rows.contains(&HudCommand::Next), "{rows:?}"); } diff --git a/rust-modules/appkit/src/player_hud/census-classic.golden.tsv b/rust-modules/appkit/src/player_hud/census-classic.golden.tsv index 80eaec8cf8..4629fcd7d9 100644 --- a/rust-modules/appkit/src/player_hud/census-classic.golden.tsv +++ b/rust-modules/appkit/src/player_hud/census-classic.golden.tsv @@ -62,90 +62,58 @@ en 60 pair 334b1a1942fc8efa en 61 pair 334b1a1942fc8efa en 62 pair 334b1a1942fc8efa en 63 pair 83c01e9226cb9d18 -en 64 restart 7fb012b76d2cb6c7 -en 65 restart 64f39a8af802fc65 -en 66 restart 93e93408ab0b5090 -en 67 restart da1f5f97de03a16f -en 68 restart 05eede0784f7678e -en 69 restart 3824be00c631db3c -en 70 restart d473239538ec832e -en 71 restart 93e93408ab0b5090 -en 72 restart 334b1a1942fc8efa -en 73 restart 334b1a1942fc8efa -en 74 restart 334b1a1942fc8efa -en 75 restart 334b1a1942fc8efa -en 76 restart 334b1a1942fc8efa -en 77 restart 334b1a1942fc8efa -en 78 restart 334b1a1942fc8efa -en 79 restart 83c01e9226cb9d18 -en 80 restart 6b201affa7e1b2e4 -en 81 restart f9f9444a5c1cd4b6 -en 82 restart 77f7bb557c848ef9 -en 83 restart 082a546af5474618 -en 84 restart a0d518fa5e7a051f -en 85 restart a44d7319c365ca3d -en 86 restart dee71acd1d951a17 -en 87 restart 77f7bb557c848ef9 -en 88 restart 334b1a1942fc8efa -en 89 restart 334b1a1942fc8efa -en 90 restart 334b1a1942fc8efa -en 91 restart 334b1a1942fc8efa -en 92 restart 334b1a1942fc8efa -en 93 restart 334b1a1942fc8efa -en 94 restart 334b1a1942fc8efa -en 95 restart 83c01e9226cb9d18 -en 96 skip-intro 43263a8f38778f2c -en 97 skip-intro 1fb64da3c94924e8 -en 98 skip-intro ac10a2c4f2028309 -en 99 skip-intro 16435636a0b97d69 -en 100 skip-intro 334b1a1942fc8efa -en 101 skip-intro 334b1a1942fc8efa -en 102 skip-intro 334b1a1942fc8efa -en 103 skip-intro 83c01e9226cb9d18 -en 104 skip-intro 46068dd54336a861 -en 105 skip-intro 2057f26f3adc3ec9 -en 106 skip-intro 567cbb409c3d0dd2 -en 107 skip-intro 917fcd62243e0336 -en 108 skip-intro 334b1a1942fc8efa -en 109 skip-intro 334b1a1942fc8efa -en 110 skip-intro 334b1a1942fc8efa -en 111 skip-intro 83c01e9226cb9d18 -en 112 skip-credits 20d2f5570df1cde3 -en 113 skip-credits 2d9b7d950ce46224 -en 114 skip-credits fa49b9fb3e3d9bc0 -en 115 skip-credits 7825646c07ce7f2a -en 116 skip-credits 334b1a1942fc8efa -en 117 skip-credits 334b1a1942fc8efa -en 118 skip-credits 334b1a1942fc8efa -en 119 skip-credits 83c01e9226cb9d18 -en 120 skip-credits 0323c786e4c87274 -en 121 skip-credits 2b85325e5fb80539 -en 122 skip-credits 866ef7a27a589621 -en 123 skip-credits 25d6cd391ea749c3 -en 124 skip-credits 334b1a1942fc8efa -en 125 skip-credits 334b1a1942fc8efa -en 126 skip-credits 334b1a1942fc8efa -en 127 skip-credits 83c01e9226cb9d18 -en 128 up-next 37ccaa292aea3062 -en 129 up-next 798b78f2025731e3 -en 130 up-next 798b78f2025731e3 -en 131 up-next 4e7d240a4dcfc899 -en 132 up-next 798b78f2025731e3 -en 133 up-next 334b1a1942fc8efa -en 134 up-next 334b1a1942fc8efa -en 135 up-next 334b1a1942fc8efa -en 136 up-next 334b1a1942fc8efa -en 137 up-next 83c01e9226cb9d18 -en 138 up-next d91e77e32c5851ab -en 139 up-next df8091ebb6fd5e74 -en 140 up-next df8091ebb6fd5e74 -en 141 up-next 6f27e3d2b1d1b4a2 -en 142 up-next df8091ebb6fd5e74 -en 143 up-next 334b1a1942fc8efa -en 144 up-next 334b1a1942fc8efa -en 145 up-next 334b1a1942fc8efa -en 146 up-next 334b1a1942fc8efa -en 147 up-next 83c01e9226cb9d18 +en 64 skip-intro 43263a8f38778f2c +en 65 skip-intro 1fb64da3c94924e8 +en 66 skip-intro ac10a2c4f2028309 +en 67 skip-intro 16435636a0b97d69 +en 68 skip-intro 334b1a1942fc8efa +en 69 skip-intro 334b1a1942fc8efa +en 70 skip-intro 334b1a1942fc8efa +en 71 skip-intro 83c01e9226cb9d18 +en 72 skip-intro 46068dd54336a861 +en 73 skip-intro 2057f26f3adc3ec9 +en 74 skip-intro 567cbb409c3d0dd2 +en 75 skip-intro 917fcd62243e0336 +en 76 skip-intro 334b1a1942fc8efa +en 77 skip-intro 334b1a1942fc8efa +en 78 skip-intro 334b1a1942fc8efa +en 79 skip-intro 83c01e9226cb9d18 +en 80 skip-credits 20d2f5570df1cde3 +en 81 skip-credits 2d9b7d950ce46224 +en 82 skip-credits fa49b9fb3e3d9bc0 +en 83 skip-credits 7825646c07ce7f2a +en 84 skip-credits 334b1a1942fc8efa +en 85 skip-credits 334b1a1942fc8efa +en 86 skip-credits 334b1a1942fc8efa +en 87 skip-credits 83c01e9226cb9d18 +en 88 skip-credits 0323c786e4c87274 +en 89 skip-credits 2b85325e5fb80539 +en 90 skip-credits 866ef7a27a589621 +en 91 skip-credits 25d6cd391ea749c3 +en 92 skip-credits 334b1a1942fc8efa +en 93 skip-credits 334b1a1942fc8efa +en 94 skip-credits 334b1a1942fc8efa +en 95 skip-credits 83c01e9226cb9d18 +en 96 up-next 37ccaa292aea3062 +en 97 up-next 798b78f2025731e3 +en 98 up-next 798b78f2025731e3 +en 99 up-next 4e7d240a4dcfc899 +en 100 up-next 798b78f2025731e3 +en 101 up-next 334b1a1942fc8efa +en 102 up-next 334b1a1942fc8efa +en 103 up-next 334b1a1942fc8efa +en 104 up-next 334b1a1942fc8efa +en 105 up-next 83c01e9226cb9d18 +en 106 up-next d91e77e32c5851ab +en 107 up-next df8091ebb6fd5e74 +en 108 up-next df8091ebb6fd5e74 +en 109 up-next 6f27e3d2b1d1b4a2 +en 110 up-next df8091ebb6fd5e74 +en 111 up-next 334b1a1942fc8efa +en 112 up-next 334b1a1942fc8efa +en 113 up-next 334b1a1942fc8efa +en 114 up-next 334b1a1942fc8efa +en 115 up-next 83c01e9226cb9d18 es 0 movie d989c8e593a2d9f5 es 1 movie 93e10e2ca80d2cd0 es 2 movie 93e10e2ca80d2cd0 @@ -210,90 +178,58 @@ es 60 pair 0824bb107811afd4 es 61 pair 0824bb107811afd4 es 62 pair 0824bb107811afd4 es 63 pair 8a264a6f5dc4794e -es 64 restart eb867216e3e9388f -es 65 restart a40d3d0501a71d71 -es 66 restart 15e85d8f6d99ad9c -es 67 restart 033412540bae19b7 -es 68 restart 139d387ef45dfcb6 -es 69 restart f6e6f4dd20c543f0 -es 70 restart f05115044d914152 -es 71 restart 15e85d8f6d99ad9c -es 72 restart 0824bb107811afd4 -es 73 restart 0824bb107811afd4 -es 74 restart 0824bb107811afd4 -es 75 restart 0824bb107811afd4 -es 76 restart 0824bb107811afd4 -es 77 restart 0824bb107811afd4 -es 78 restart 0824bb107811afd4 -es 79 restart 8a264a6f5dc4794e -es 80 restart 0abee5e48e25ad2c -es 81 restart 93231cb27d2ddeca -es 82 restart b48f49c6f02fe6ed -es 83 restart d08e91b5720efc40 -es 84 restart b2848f0fb0969e27 -es 85 restart bbace5151f6c9879 -es 86 restart dfb2d45b8456c473 -es 87 restart b48f49c6f02fe6ed -es 88 restart 0824bb107811afd4 -es 89 restart 0824bb107811afd4 -es 90 restart 0824bb107811afd4 -es 91 restart 0824bb107811afd4 -es 92 restart 0824bb107811afd4 -es 93 restart 0824bb107811afd4 -es 94 restart 0824bb107811afd4 -es 95 restart 8a264a6f5dc4794e -es 96 skip-intro 07ffc4962572201d -es 97 skip-intro 351aede28eb29553 -es 98 skip-intro f5b14e2e7e22bfe2 -es 99 skip-intro 45b09a9469931a32 -es 100 skip-intro 0824bb107811afd4 -es 101 skip-intro 0824bb107811afd4 -es 102 skip-intro 0824bb107811afd4 -es 103 skip-intro 8a264a6f5dc4794e -es 104 skip-intro b9ef0df0f234007a -es 105 skip-intro 248872761e7210e4 -es 106 skip-intro 56a335e84a48d1df -es 107 skip-intro 1b52cba292147efb -es 108 skip-intro 0824bb107811afd4 -es 109 skip-intro 0824bb107811afd4 -es 110 skip-intro 0824bb107811afd4 -es 111 skip-intro 8a264a6f5dc4794e -es 112 skip-credits a5d2aee9fba5a2a2 -es 113 skip-credits 7c859362e31b5862 -es 114 skip-credits 68f3d1927c5ac8fb -es 115 skip-credits efabf66269915a49 -es 116 skip-credits 0824bb107811afd4 -es 117 skip-credits 0824bb107811afd4 -es 118 skip-credits 0824bb107811afd4 -es 119 skip-credits 8a264a6f5dc4794e -es 120 skip-credits 105fdce21b9ea6eb -es 121 skip-credits c6b855e8fed392d7 -es 122 skip-credits cab926fe8d01b3d0 -es 123 skip-credits 0ce272ad71efda16 -es 124 skip-credits 0824bb107811afd4 -es 125 skip-credits 0824bb107811afd4 -es 126 skip-credits 0824bb107811afd4 -es 127 skip-credits 8a264a6f5dc4794e -es 128 up-next bfa8a97e6ce48c36 -es 129 up-next fe5e443cc3a368ff -es 130 up-next fe5e443cc3a368ff -es 131 up-next 5eb5b2ea6f1dfbf5 -es 132 up-next fe5e443cc3a368ff -es 133 up-next 0824bb107811afd4 -es 134 up-next 0824bb107811afd4 -es 135 up-next 0824bb107811afd4 -es 136 up-next 0824bb107811afd4 -es 137 up-next 8a264a6f5dc4794e -es 138 up-next c2ccd57826a6ea2f -es 139 up-next 6bd80e89b3a517a0 -es 140 up-next 6bd80e89b3a517a0 -es 141 up-next 58d3d4d1375c7c6e -es 142 up-next 6bd80e89b3a517a0 -es 143 up-next 0824bb107811afd4 -es 144 up-next 0824bb107811afd4 -es 145 up-next 0824bb107811afd4 -es 146 up-next 0824bb107811afd4 -es 147 up-next 8a264a6f5dc4794e +es 64 skip-intro 07ffc4962572201d +es 65 skip-intro 351aede28eb29553 +es 66 skip-intro f5b14e2e7e22bfe2 +es 67 skip-intro 45b09a9469931a32 +es 68 skip-intro 0824bb107811afd4 +es 69 skip-intro 0824bb107811afd4 +es 70 skip-intro 0824bb107811afd4 +es 71 skip-intro 8a264a6f5dc4794e +es 72 skip-intro b9ef0df0f234007a +es 73 skip-intro 248872761e7210e4 +es 74 skip-intro 56a335e84a48d1df +es 75 skip-intro 1b52cba292147efb +es 76 skip-intro 0824bb107811afd4 +es 77 skip-intro 0824bb107811afd4 +es 78 skip-intro 0824bb107811afd4 +es 79 skip-intro 8a264a6f5dc4794e +es 80 skip-credits a5d2aee9fba5a2a2 +es 81 skip-credits 7c859362e31b5862 +es 82 skip-credits 68f3d1927c5ac8fb +es 83 skip-credits efabf66269915a49 +es 84 skip-credits 0824bb107811afd4 +es 85 skip-credits 0824bb107811afd4 +es 86 skip-credits 0824bb107811afd4 +es 87 skip-credits 8a264a6f5dc4794e +es 88 skip-credits 105fdce21b9ea6eb +es 89 skip-credits c6b855e8fed392d7 +es 90 skip-credits cab926fe8d01b3d0 +es 91 skip-credits 0ce272ad71efda16 +es 92 skip-credits 0824bb107811afd4 +es 93 skip-credits 0824bb107811afd4 +es 94 skip-credits 0824bb107811afd4 +es 95 skip-credits 8a264a6f5dc4794e +es 96 up-next bfa8a97e6ce48c36 +es 97 up-next fe5e443cc3a368ff +es 98 up-next fe5e443cc3a368ff +es 99 up-next 5eb5b2ea6f1dfbf5 +es 100 up-next fe5e443cc3a368ff +es 101 up-next 0824bb107811afd4 +es 102 up-next 0824bb107811afd4 +es 103 up-next 0824bb107811afd4 +es 104 up-next 0824bb107811afd4 +es 105 up-next 8a264a6f5dc4794e +es 106 up-next c2ccd57826a6ea2f +es 107 up-next 6bd80e89b3a517a0 +es 108 up-next 6bd80e89b3a517a0 +es 109 up-next 58d3d4d1375c7c6e +es 110 up-next 6bd80e89b3a517a0 +es 111 up-next 0824bb107811afd4 +es 112 up-next 0824bb107811afd4 +es 113 up-next 0824bb107811afd4 +es 114 up-next 0824bb107811afd4 +es 115 up-next 8a264a6f5dc4794e be 0 movie 27131d6036779694 be 1 movie 39dfb05f22fed779 be 2 movie 39dfb05f22fed779 @@ -358,90 +294,58 @@ be 60 pair 4b3c486863553b4f be 61 pair 4b3c486863553b4f be 62 pair 4b3c486863553b4f be 63 pair 0efc26b9a7fb222d -be 64 restart fe9c0316ff6e9198 -be 65 restart 16528d17a34e65ec -be 66 restart 8166ed7078d2dccb -be 67 restart f2197d6e80096566 -be 68 restart 47dbfb662c124c83 -be 69 restart 8eb3e32171f1e479 -be 70 restart ddfda4d4deca00cc -be 71 restart 8166ed7078d2dccb -be 72 restart 4b3c486863553b4f -be 73 restart 4b3c486863553b4f -be 74 restart 4b3c486863553b4f -be 75 restart 4b3c486863553b4f -be 76 restart 4b3c486863553b4f -be 77 restart 4b3c486863553b4f -be 78 restart 4b3c486863553b4f -be 79 restart 0efc26b9a7fb222d -be 80 restart 247a71f0aaab9f01 -be 81 restart 84eeb63b749933c5 -be 82 restart 885e6a6089be40f0 -be 83 restart 03ad2a858bb9390f -be 84 restart 5bcfddd562ef5864 -be 85 restart 17efd125c8077292 -be 86 restart f8df7b4150d573b5 -be 87 restart 885e6a6089be40f0 -be 88 restart 4b3c486863553b4f -be 89 restart 4b3c486863553b4f -be 90 restart 4b3c486863553b4f -be 91 restart 4b3c486863553b4f -be 92 restart 4b3c486863553b4f -be 93 restart 4b3c486863553b4f -be 94 restart 4b3c486863553b4f -be 95 restart 0efc26b9a7fb222d -be 96 skip-intro 951c7c7c38ddd0c1 -be 97 skip-intro ab312853c5a088f6 -be 98 skip-intro 59fe2e6db9241b77 -be 99 skip-intro 97741492b5faae02 -be 100 skip-intro 4b3c486863553b4f -be 101 skip-intro 4b3c486863553b4f -be 102 skip-intro 4b3c486863553b4f -be 103 skip-intro 0efc26b9a7fb222d -be 104 skip-intro 5b14e50fd7e291fe -be 105 skip-intro 4f3095a02ff73527 -be 106 skip-intro e1084bcf0dec72b0 -be 107 skip-intro 5d837ea3f6bebe4b -be 108 skip-intro 4b3c486863553b4f -be 109 skip-intro 4b3c486863553b4f -be 110 skip-intro 4b3c486863553b4f -be 111 skip-intro 0efc26b9a7fb222d -be 112 skip-credits 4dc4cf60241b4d0a -be 113 skip-credits c5ae92c6018c0191 -be 114 skip-credits 23676a959e3598e6 -be 115 skip-credits cb1a2c21ae0dc1e5 -be 116 skip-credits 4b3c486863553b4f -be 117 skip-credits 4b3c486863553b4f -be 118 skip-credits 4b3c486863553b4f -be 119 skip-credits 0efc26b9a7fb222d -be 120 skip-credits 14a1931fbd2797a3 -be 121 skip-credits 673f673b45fe584a -be 122 skip-credits 34bfece343cf3897 -be 123 skip-credits fb85d28375533fb2 -be 124 skip-credits 4b3c486863553b4f -be 125 skip-credits 4b3c486863553b4f -be 126 skip-credits 4b3c486863553b4f -be 127 skip-credits 0efc26b9a7fb222d -be 128 up-next a3c67623f4b7a4c9 -be 129 up-next 28c43c8bb1e4999c -be 130 up-next 28c43c8bb1e4999c -be 131 up-next 6b74f56353b5daa7 -be 132 up-next 28c43c8bb1e4999c -be 133 up-next 4b3c486863553b4f -be 134 up-next 4b3c486863553b4f -be 135 up-next 4b3c486863553b4f -be 136 up-next 4b3c486863553b4f -be 137 up-next 0efc26b9a7fb222d -be 138 up-next 69d3c5df7037f196 -be 139 up-next a81db112acf92b91 -be 140 up-next a81db112acf92b91 -be 141 up-next 485fde7c6ec0c4e0 -be 142 up-next a81db112acf92b91 -be 143 up-next 4b3c486863553b4f -be 144 up-next 4b3c486863553b4f -be 145 up-next 4b3c486863553b4f -be 146 up-next 4b3c486863553b4f -be 147 up-next 0efc26b9a7fb222d +be 64 skip-intro 951c7c7c38ddd0c1 +be 65 skip-intro ab312853c5a088f6 +be 66 skip-intro 59fe2e6db9241b77 +be 67 skip-intro 97741492b5faae02 +be 68 skip-intro 4b3c486863553b4f +be 69 skip-intro 4b3c486863553b4f +be 70 skip-intro 4b3c486863553b4f +be 71 skip-intro 0efc26b9a7fb222d +be 72 skip-intro 5b14e50fd7e291fe +be 73 skip-intro 4f3095a02ff73527 +be 74 skip-intro e1084bcf0dec72b0 +be 75 skip-intro 5d837ea3f6bebe4b +be 76 skip-intro 4b3c486863553b4f +be 77 skip-intro 4b3c486863553b4f +be 78 skip-intro 4b3c486863553b4f +be 79 skip-intro 0efc26b9a7fb222d +be 80 skip-credits 4dc4cf60241b4d0a +be 81 skip-credits c5ae92c6018c0191 +be 82 skip-credits 23676a959e3598e6 +be 83 skip-credits cb1a2c21ae0dc1e5 +be 84 skip-credits 4b3c486863553b4f +be 85 skip-credits 4b3c486863553b4f +be 86 skip-credits 4b3c486863553b4f +be 87 skip-credits 0efc26b9a7fb222d +be 88 skip-credits 14a1931fbd2797a3 +be 89 skip-credits 673f673b45fe584a +be 90 skip-credits 34bfece343cf3897 +be 91 skip-credits fb85d28375533fb2 +be 92 skip-credits 4b3c486863553b4f +be 93 skip-credits 4b3c486863553b4f +be 94 skip-credits 4b3c486863553b4f +be 95 skip-credits 0efc26b9a7fb222d +be 96 up-next a3c67623f4b7a4c9 +be 97 up-next 28c43c8bb1e4999c +be 98 up-next 28c43c8bb1e4999c +be 99 up-next 6b74f56353b5daa7 +be 100 up-next 28c43c8bb1e4999c +be 101 up-next 4b3c486863553b4f +be 102 up-next 4b3c486863553b4f +be 103 up-next 4b3c486863553b4f +be 104 up-next 4b3c486863553b4f +be 105 up-next 0efc26b9a7fb222d +be 106 up-next 69d3c5df7037f196 +be 107 up-next a81db112acf92b91 +be 108 up-next a81db112acf92b91 +be 109 up-next 485fde7c6ec0c4e0 +be 110 up-next a81db112acf92b91 +be 111 up-next 4b3c486863553b4f +be 112 up-next 4b3c486863553b4f +be 113 up-next 4b3c486863553b4f +be 114 up-next 4b3c486863553b4f +be 115 up-next 0efc26b9a7fb222d fr 0 movie c6b7752285436f7d fr 1 movie 0e372426097d9c9a fr 2 movie 0e372426097d9c9a @@ -506,90 +410,58 @@ fr 60 pair 6698b211c12112a6 fr 61 pair 6698b211c12112a6 fr 62 pair 6698b211c12112a6 fr 63 pair f48520a72eef590c -fr 64 restart ddab416254c80037 -fr 65 restart e85cd2f6eae3bca7 -fr 66 restart 16ab64ee30f0e2ae -fr 67 restart 64051f3bd838e679 -fr 68 restart b0e74f17607584be -fr 69 restart 700b4685bf0e6584 -fr 70 restart 0d2133bae67ba8b0 -fr 71 restart 16ab64ee30f0e2ae -fr 72 restart 6698b211c12112a6 -fr 73 restart 6698b211c12112a6 -fr 74 restart 6698b211c12112a6 -fr 75 restart 6698b211c12112a6 -fr 76 restart 6698b211c12112a6 -fr 77 restart 6698b211c12112a6 -fr 78 restart 6698b211c12112a6 -fr 79 restart f48520a72eef590c -fr 80 restart 3b080caeacf22554 -fr 81 restart 7b598d51941bf36c -fr 82 restart bd472d0a40d7db5b -fr 83 restart cb590a081c45dac6 -fr 84 restart 24df2aeec596d00f -fr 85 restart 30a7b384cbc37dc5 -fr 86 restart ad344b75d4858aad -fr 87 restart bd472d0a40d7db5b -fr 88 restart 6698b211c12112a6 -fr 89 restart 6698b211c12112a6 -fr 90 restart 6698b211c12112a6 -fr 91 restart 6698b211c12112a6 -fr 92 restart 6698b211c12112a6 -fr 93 restart 6698b211c12112a6 -fr 94 restart 6698b211c12112a6 -fr 95 restart f48520a72eef590c -fr 96 skip-intro d30932ad4059d1a4 -fr 97 skip-intro 02bab2cd8c73fb28 -fr 98 skip-intro 7c1ef05ed4602727 -fr 99 skip-intro 0b6bd3b856b9cc71 -fr 100 skip-intro 6698b211c12112a6 -fr 101 skip-intro 6698b211c12112a6 -fr 102 skip-intro 6698b211c12112a6 -fr 103 skip-intro f48520a72eef590c -fr 104 skip-intro 584f2a4325241c69 -fr 105 skip-intro 66ebce4c8a154791 -fr 106 skip-intro 682e20929741251c -fr 107 skip-intro 372e5b65ea4e18ee -fr 108 skip-intro 6698b211c12112a6 -fr 109 skip-intro 6698b211c12112a6 -fr 110 skip-intro 6698b211c12112a6 -fr 111 skip-intro f48520a72eef590c -fr 112 skip-credits 157ee0b4af8a567b -fr 113 skip-credits fd752e7ceeebd942 -fr 114 skip-credits 0af8b28fa2a650a0 -fr 115 skip-credits 58586eb696644a4e -fr 116 skip-credits 6698b211c12112a6 -fr 117 skip-credits 6698b211c12112a6 -fr 118 skip-credits 6698b211c12112a6 -fr 119 skip-credits f48520a72eef590c -fr 120 skip-credits e6a7cf8b13ed2a5c -fr 121 skip-credits 4becfc6c647f8143 -fr 122 skip-credits 0ec88c9cd9deec99 -fr 123 skip-credits 8b9c74261406cfd7 -fr 124 skip-credits 6698b211c12112a6 -fr 125 skip-credits 6698b211c12112a6 -fr 126 skip-credits 6698b211c12112a6 -fr 127 skip-credits f48520a72eef590c -fr 128 up-next 41f52ed28125b8d8 -fr 129 up-next de645b7af09764db -fr 130 up-next de645b7af09764db -fr 131 up-next 0df25cbd8e3f6e35 -fr 132 up-next de645b7af09764db -fr 133 up-next 6698b211c12112a6 -fr 134 up-next 6698b211c12112a6 -fr 135 up-next 6698b211c12112a6 -fr 136 up-next 6698b211c12112a6 -fr 137 up-next f48520a72eef590c -fr 138 up-next 313ba44d0d0b027d -fr 139 up-next f2bcf57a3bd6013c -fr 140 up-next f2bcf57a3bd6013c -fr 141 up-next 46bf340c54013c5e -fr 142 up-next f2bcf57a3bd6013c -fr 143 up-next 6698b211c12112a6 -fr 144 up-next 6698b211c12112a6 -fr 145 up-next 6698b211c12112a6 -fr 146 up-next 6698b211c12112a6 -fr 147 up-next f48520a72eef590c +fr 64 skip-intro d30932ad4059d1a4 +fr 65 skip-intro 02bab2cd8c73fb28 +fr 66 skip-intro 7c1ef05ed4602727 +fr 67 skip-intro 0b6bd3b856b9cc71 +fr 68 skip-intro 6698b211c12112a6 +fr 69 skip-intro 6698b211c12112a6 +fr 70 skip-intro 6698b211c12112a6 +fr 71 skip-intro f48520a72eef590c +fr 72 skip-intro 584f2a4325241c69 +fr 73 skip-intro 66ebce4c8a154791 +fr 74 skip-intro 682e20929741251c +fr 75 skip-intro 372e5b65ea4e18ee +fr 76 skip-intro 6698b211c12112a6 +fr 77 skip-intro 6698b211c12112a6 +fr 78 skip-intro 6698b211c12112a6 +fr 79 skip-intro f48520a72eef590c +fr 80 skip-credits 157ee0b4af8a567b +fr 81 skip-credits fd752e7ceeebd942 +fr 82 skip-credits 0af8b28fa2a650a0 +fr 83 skip-credits 58586eb696644a4e +fr 84 skip-credits 6698b211c12112a6 +fr 85 skip-credits 6698b211c12112a6 +fr 86 skip-credits 6698b211c12112a6 +fr 87 skip-credits f48520a72eef590c +fr 88 skip-credits e6a7cf8b13ed2a5c +fr 89 skip-credits 4becfc6c647f8143 +fr 90 skip-credits 0ec88c9cd9deec99 +fr 91 skip-credits 8b9c74261406cfd7 +fr 92 skip-credits 6698b211c12112a6 +fr 93 skip-credits 6698b211c12112a6 +fr 94 skip-credits 6698b211c12112a6 +fr 95 skip-credits f48520a72eef590c +fr 96 up-next 41f52ed28125b8d8 +fr 97 up-next de645b7af09764db +fr 98 up-next de645b7af09764db +fr 99 up-next 0df25cbd8e3f6e35 +fr 100 up-next de645b7af09764db +fr 101 up-next 6698b211c12112a6 +fr 102 up-next 6698b211c12112a6 +fr 103 up-next 6698b211c12112a6 +fr 104 up-next 6698b211c12112a6 +fr 105 up-next f48520a72eef590c +fr 106 up-next 313ba44d0d0b027d +fr 107 up-next f2bcf57a3bd6013c +fr 108 up-next f2bcf57a3bd6013c +fr 109 up-next 46bf340c54013c5e +fr 110 up-next f2bcf57a3bd6013c +fr 111 up-next 6698b211c12112a6 +fr 112 up-next 6698b211c12112a6 +fr 113 up-next 6698b211c12112a6 +fr 114 up-next 6698b211c12112a6 +fr 115 up-next f48520a72eef590c it 0 movie 73d83cea8d35cf0d it 1 movie 1e0679d5ceedcd8c it 2 movie 1e0679d5ceedcd8c @@ -654,90 +526,58 @@ it 60 pair 334b1a1942fc8efa it 61 pair 334b1a1942fc8efa it 62 pair 334b1a1942fc8efa it 63 pair 83c01e9226cb9d18 -it 64 restart 7fb012b76d2cb6c7 -it 65 restart 64f39a8af802fc65 -it 66 restart 93e93408ab0b5090 -it 67 restart da1f5f97de03a16f -it 68 restart 05eede0784f7678e -it 69 restart 3824be00c631db3c -it 70 restart d473239538ec832e -it 71 restart 93e93408ab0b5090 -it 72 restart 334b1a1942fc8efa -it 73 restart 334b1a1942fc8efa -it 74 restart 334b1a1942fc8efa -it 75 restart 334b1a1942fc8efa -it 76 restart 334b1a1942fc8efa -it 77 restart 334b1a1942fc8efa -it 78 restart 334b1a1942fc8efa -it 79 restart 83c01e9226cb9d18 -it 80 restart 6b201affa7e1b2e4 -it 81 restart f9f9444a5c1cd4b6 -it 82 restart 77f7bb557c848ef9 -it 83 restart 082a546af5474618 -it 84 restart a0d518fa5e7a051f -it 85 restart a44d7319c365ca3d -it 86 restart dee71acd1d951a17 -it 87 restart 77f7bb557c848ef9 -it 88 restart 334b1a1942fc8efa -it 89 restart 334b1a1942fc8efa -it 90 restart 334b1a1942fc8efa -it 91 restart 334b1a1942fc8efa -it 92 restart 334b1a1942fc8efa -it 93 restart 334b1a1942fc8efa -it 94 restart 334b1a1942fc8efa -it 95 restart 83c01e9226cb9d18 -it 96 skip-intro de32d78211d38592 -it 97 skip-intro 4920cab97f5a4cfa -it 98 skip-intro bd528a920b4216b5 -it 99 skip-intro 910b8079b4c03451 -it 100 skip-intro 334b1a1942fc8efa -it 101 skip-intro 334b1a1942fc8efa -it 102 skip-intro 334b1a1942fc8efa -it 103 skip-intro 83c01e9226cb9d18 -it 104 skip-intro 06272706fa24b55b -it 105 skip-intro 44d5cb18d39a4fd7 -it 106 skip-intro 4c76424064478b2e -it 107 skip-intro 3d513610d2528dce -it 108 skip-intro 334b1a1942fc8efa -it 109 skip-intro 334b1a1942fc8efa -it 110 skip-intro 334b1a1942fc8efa -it 111 skip-intro 83c01e9226cb9d18 -it 112 skip-credits d60b0cba7ed3f442 -it 113 skip-credits b3606e5cba5c24d4 -it 114 skip-credits 6b0a75f6647a4b87 -it 115 skip-credits 98a3030f6fdce031 -it 116 skip-credits 334b1a1942fc8efa -it 117 skip-credits 334b1a1942fc8efa -it 118 skip-credits 334b1a1942fc8efa -it 119 skip-credits 83c01e9226cb9d18 -it 120 skip-credits 42dd3632bfdc808b -it 121 skip-credits ed7d3d922e3cd739 -it 122 skip-credits 6bc0224b63ca60dc -it 123 skip-credits b44210d929865bae -it 124 skip-credits 334b1a1942fc8efa -it 125 skip-credits 334b1a1942fc8efa -it 126 skip-credits 334b1a1942fc8efa -it 127 skip-credits 83c01e9226cb9d18 -it 128 up-next 37ccaa292aea3062 -it 129 up-next 798b78f2025731e3 -it 130 up-next 798b78f2025731e3 -it 131 up-next 4e7d240a4dcfc899 -it 132 up-next 798b78f2025731e3 -it 133 up-next 334b1a1942fc8efa -it 134 up-next 334b1a1942fc8efa -it 135 up-next 334b1a1942fc8efa -it 136 up-next 334b1a1942fc8efa -it 137 up-next 83c01e9226cb9d18 -it 138 up-next d91e77e32c5851ab -it 139 up-next df8091ebb6fd5e74 -it 140 up-next df8091ebb6fd5e74 -it 141 up-next 6f27e3d2b1d1b4a2 -it 142 up-next df8091ebb6fd5e74 -it 143 up-next 334b1a1942fc8efa -it 144 up-next 334b1a1942fc8efa -it 145 up-next 334b1a1942fc8efa -it 146 up-next 334b1a1942fc8efa -it 147 up-next 83c01e9226cb9d18 +it 64 skip-intro de32d78211d38592 +it 65 skip-intro 4920cab97f5a4cfa +it 66 skip-intro bd528a920b4216b5 +it 67 skip-intro 910b8079b4c03451 +it 68 skip-intro 334b1a1942fc8efa +it 69 skip-intro 334b1a1942fc8efa +it 70 skip-intro 334b1a1942fc8efa +it 71 skip-intro 83c01e9226cb9d18 +it 72 skip-intro 06272706fa24b55b +it 73 skip-intro 44d5cb18d39a4fd7 +it 74 skip-intro 4c76424064478b2e +it 75 skip-intro 3d513610d2528dce +it 76 skip-intro 334b1a1942fc8efa +it 77 skip-intro 334b1a1942fc8efa +it 78 skip-intro 334b1a1942fc8efa +it 79 skip-intro 83c01e9226cb9d18 +it 80 skip-credits d60b0cba7ed3f442 +it 81 skip-credits b3606e5cba5c24d4 +it 82 skip-credits 6b0a75f6647a4b87 +it 83 skip-credits 98a3030f6fdce031 +it 84 skip-credits 334b1a1942fc8efa +it 85 skip-credits 334b1a1942fc8efa +it 86 skip-credits 334b1a1942fc8efa +it 87 skip-credits 83c01e9226cb9d18 +it 88 skip-credits 42dd3632bfdc808b +it 89 skip-credits ed7d3d922e3cd739 +it 90 skip-credits 6bc0224b63ca60dc +it 91 skip-credits b44210d929865bae +it 92 skip-credits 334b1a1942fc8efa +it 93 skip-credits 334b1a1942fc8efa +it 94 skip-credits 334b1a1942fc8efa +it 95 skip-credits 83c01e9226cb9d18 +it 96 up-next 37ccaa292aea3062 +it 97 up-next 798b78f2025731e3 +it 98 up-next 798b78f2025731e3 +it 99 up-next 4e7d240a4dcfc899 +it 100 up-next 798b78f2025731e3 +it 101 up-next 334b1a1942fc8efa +it 102 up-next 334b1a1942fc8efa +it 103 up-next 334b1a1942fc8efa +it 104 up-next 334b1a1942fc8efa +it 105 up-next 83c01e9226cb9d18 +it 106 up-next d91e77e32c5851ab +it 107 up-next df8091ebb6fd5e74 +it 108 up-next df8091ebb6fd5e74 +it 109 up-next 6f27e3d2b1d1b4a2 +it 110 up-next df8091ebb6fd5e74 +it 111 up-next 334b1a1942fc8efa +it 112 up-next 334b1a1942fc8efa +it 113 up-next 334b1a1942fc8efa +it 114 up-next 334b1a1942fc8efa +it 115 up-next 83c01e9226cb9d18 pt 0 movie 73d83cea8d35cf0d pt 1 movie 1e0679d5ceedcd8c pt 2 movie 1e0679d5ceedcd8c @@ -802,90 +642,58 @@ pt 60 pair 334b1a1942fc8efa pt 61 pair 334b1a1942fc8efa pt 62 pair 334b1a1942fc8efa pt 63 pair 83c01e9226cb9d18 -pt 64 restart 7fb012b76d2cb6c7 -pt 65 restart 64f39a8af802fc65 -pt 66 restart 93e93408ab0b5090 -pt 67 restart da1f5f97de03a16f -pt 68 restart 05eede0784f7678e -pt 69 restart 3824be00c631db3c -pt 70 restart d473239538ec832e -pt 71 restart 93e93408ab0b5090 -pt 72 restart 334b1a1942fc8efa -pt 73 restart 334b1a1942fc8efa -pt 74 restart 334b1a1942fc8efa -pt 75 restart 334b1a1942fc8efa -pt 76 restart 334b1a1942fc8efa -pt 77 restart 334b1a1942fc8efa -pt 78 restart 334b1a1942fc8efa -pt 79 restart 83c01e9226cb9d18 -pt 80 restart 6b201affa7e1b2e4 -pt 81 restart f9f9444a5c1cd4b6 -pt 82 restart 77f7bb557c848ef9 -pt 83 restart 082a546af5474618 -pt 84 restart a0d518fa5e7a051f -pt 85 restart a44d7319c365ca3d -pt 86 restart dee71acd1d951a17 -pt 87 restart 77f7bb557c848ef9 -pt 88 restart 334b1a1942fc8efa -pt 89 restart 334b1a1942fc8efa -pt 90 restart 334b1a1942fc8efa -pt 91 restart 334b1a1942fc8efa -pt 92 restart 334b1a1942fc8efa -pt 93 restart 334b1a1942fc8efa -pt 94 restart 334b1a1942fc8efa -pt 95 restart 83c01e9226cb9d18 -pt 96 skip-intro 9d5f32aeaa6e430e -pt 97 skip-intro cd16a09156a08a49 -pt 98 skip-intro e1783da882e664e9 -pt 99 skip-intro 2ab5bd480149c70b -pt 100 skip-intro 334b1a1942fc8efa -pt 101 skip-intro 334b1a1942fc8efa -pt 102 skip-intro 334b1a1942fc8efa -pt 103 skip-intro 83c01e9226cb9d18 -pt 104 skip-intro 1d8246d10e3ec297 -pt 105 skip-intro 0af9515641faaa4a -pt 106 skip-intro 366fd2f80666e0b2 -pt 107 skip-intro 131ce623207985ec -pt 108 skip-intro 334b1a1942fc8efa -pt 109 skip-intro 334b1a1942fc8efa -pt 110 skip-intro 334b1a1942fc8efa -pt 111 skip-intro 83c01e9226cb9d18 -pt 112 skip-credits a1f154b25fb5c547 -pt 113 skip-credits 6a5edd426df3f0bf -pt 114 skip-credits 7d0f2e9388feef78 -pt 115 skip-credits ab18304790693638 -pt 116 skip-credits 334b1a1942fc8efa -pt 117 skip-credits 334b1a1942fc8efa -pt 118 skip-credits 334b1a1942fc8efa -pt 119 skip-credits 83c01e9226cb9d18 -pt 120 skip-credits a248d3c100ca3f98 -pt 121 skip-credits 72cb1ba246a295e4 -pt 122 skip-credits 337e7ea918a22169 -pt 123 skip-credits 9f820a11908b085d -pt 124 skip-credits 334b1a1942fc8efa -pt 125 skip-credits 334b1a1942fc8efa -pt 126 skip-credits 334b1a1942fc8efa -pt 127 skip-credits 83c01e9226cb9d18 -pt 128 up-next 37ccaa292aea3062 -pt 129 up-next 798b78f2025731e3 -pt 130 up-next 798b78f2025731e3 -pt 131 up-next 4e7d240a4dcfc899 -pt 132 up-next 798b78f2025731e3 -pt 133 up-next 334b1a1942fc8efa -pt 134 up-next 334b1a1942fc8efa -pt 135 up-next 334b1a1942fc8efa -pt 136 up-next 334b1a1942fc8efa -pt 137 up-next 83c01e9226cb9d18 -pt 138 up-next d91e77e32c5851ab -pt 139 up-next df8091ebb6fd5e74 -pt 140 up-next df8091ebb6fd5e74 -pt 141 up-next 6f27e3d2b1d1b4a2 -pt 142 up-next df8091ebb6fd5e74 -pt 143 up-next 334b1a1942fc8efa -pt 144 up-next 334b1a1942fc8efa -pt 145 up-next 334b1a1942fc8efa -pt 146 up-next 334b1a1942fc8efa -pt 147 up-next 83c01e9226cb9d18 +pt 64 skip-intro 9d5f32aeaa6e430e +pt 65 skip-intro cd16a09156a08a49 +pt 66 skip-intro e1783da882e664e9 +pt 67 skip-intro 2ab5bd480149c70b +pt 68 skip-intro 334b1a1942fc8efa +pt 69 skip-intro 334b1a1942fc8efa +pt 70 skip-intro 334b1a1942fc8efa +pt 71 skip-intro 83c01e9226cb9d18 +pt 72 skip-intro 1d8246d10e3ec297 +pt 73 skip-intro 0af9515641faaa4a +pt 74 skip-intro 366fd2f80666e0b2 +pt 75 skip-intro 131ce623207985ec +pt 76 skip-intro 334b1a1942fc8efa +pt 77 skip-intro 334b1a1942fc8efa +pt 78 skip-intro 334b1a1942fc8efa +pt 79 skip-intro 83c01e9226cb9d18 +pt 80 skip-credits a1f154b25fb5c547 +pt 81 skip-credits 6a5edd426df3f0bf +pt 82 skip-credits 7d0f2e9388feef78 +pt 83 skip-credits ab18304790693638 +pt 84 skip-credits 334b1a1942fc8efa +pt 85 skip-credits 334b1a1942fc8efa +pt 86 skip-credits 334b1a1942fc8efa +pt 87 skip-credits 83c01e9226cb9d18 +pt 88 skip-credits a248d3c100ca3f98 +pt 89 skip-credits 72cb1ba246a295e4 +pt 90 skip-credits 337e7ea918a22169 +pt 91 skip-credits 9f820a11908b085d +pt 92 skip-credits 334b1a1942fc8efa +pt 93 skip-credits 334b1a1942fc8efa +pt 94 skip-credits 334b1a1942fc8efa +pt 95 skip-credits 83c01e9226cb9d18 +pt 96 up-next 37ccaa292aea3062 +pt 97 up-next 798b78f2025731e3 +pt 98 up-next 798b78f2025731e3 +pt 99 up-next 4e7d240a4dcfc899 +pt 100 up-next 798b78f2025731e3 +pt 101 up-next 334b1a1942fc8efa +pt 102 up-next 334b1a1942fc8efa +pt 103 up-next 334b1a1942fc8efa +pt 104 up-next 334b1a1942fc8efa +pt 105 up-next 83c01e9226cb9d18 +pt 106 up-next d91e77e32c5851ab +pt 107 up-next df8091ebb6fd5e74 +pt 108 up-next df8091ebb6fd5e74 +pt 109 up-next 6f27e3d2b1d1b4a2 +pt 110 up-next df8091ebb6fd5e74 +pt 111 up-next 334b1a1942fc8efa +pt 112 up-next 334b1a1942fc8efa +pt 113 up-next 334b1a1942fc8efa +pt 114 up-next 334b1a1942fc8efa +pt 115 up-next 83c01e9226cb9d18 de 0 movie 73d83cea8d35cf0d de 1 movie 1e0679d5ceedcd8c de 2 movie 1e0679d5ceedcd8c @@ -950,90 +758,58 @@ de 60 pair 334b1a1942fc8efa de 61 pair 334b1a1942fc8efa de 62 pair 334b1a1942fc8efa de 63 pair 83c01e9226cb9d18 -de 64 restart 7fb012b76d2cb6c7 -de 65 restart 64f39a8af802fc65 -de 66 restart 93e93408ab0b5090 -de 67 restart da1f5f97de03a16f -de 68 restart 05eede0784f7678e -de 69 restart 3824be00c631db3c -de 70 restart d473239538ec832e -de 71 restart 93e93408ab0b5090 -de 72 restart 334b1a1942fc8efa -de 73 restart 334b1a1942fc8efa -de 74 restart 334b1a1942fc8efa -de 75 restart 334b1a1942fc8efa -de 76 restart 334b1a1942fc8efa -de 77 restart 334b1a1942fc8efa -de 78 restart 334b1a1942fc8efa -de 79 restart 83c01e9226cb9d18 -de 80 restart 6b201affa7e1b2e4 -de 81 restart f9f9444a5c1cd4b6 -de 82 restart 77f7bb557c848ef9 -de 83 restart 082a546af5474618 -de 84 restart a0d518fa5e7a051f -de 85 restart a44d7319c365ca3d -de 86 restart dee71acd1d951a17 -de 87 restart 77f7bb557c848ef9 -de 88 restart 334b1a1942fc8efa -de 89 restart 334b1a1942fc8efa -de 90 restart 334b1a1942fc8efa -de 91 restart 334b1a1942fc8efa -de 92 restart 334b1a1942fc8efa -de 93 restart 334b1a1942fc8efa -de 94 restart 334b1a1942fc8efa -de 95 restart 83c01e9226cb9d18 -de 96 skip-intro 8bde2eb39dce358f -de 97 skip-intro 9c87bcd400bb6772 -de 98 skip-intro ab7eec7b8cd71a2e -de 99 skip-intro 9f7f4d80c9d6314a -de 100 skip-intro 334b1a1942fc8efa -de 101 skip-intro 334b1a1942fc8efa -de 102 skip-intro 334b1a1942fc8efa -de 103 skip-intro 83c01e9226cb9d18 -de 104 skip-intro 3a9e36feee002270 -de 105 skip-intro ff9c3a4157797f2f -de 106 skip-intro 636def457d060b9b -de 107 skip-intro 8dbe08bd98a0d3e3 -de 108 skip-intro 334b1a1942fc8efa -de 109 skip-intro 334b1a1942fc8efa -de 110 skip-intro 334b1a1942fc8efa -de 111 skip-intro 83c01e9226cb9d18 -de 112 skip-credits c7338fa2dab72d80 -de 113 skip-credits 2619df0b9509403f -de 114 skip-credits ae33954fab2277e7 -de 115 skip-credits b930ed4855c80fe7 -de 116 skip-credits 334b1a1942fc8efa -de 117 skip-credits 334b1a1942fc8efa -de 118 skip-credits 334b1a1942fc8efa -de 119 skip-credits 83c01e9226cb9d18 -de 120 skip-credits 0e6181848aaa6515 -de 121 skip-credits a90fb5dcde2c37d8 -de 122 skip-credits 391e20431645fc3c -de 123 skip-credits ba6361beeb4d60b8 -de 124 skip-credits 334b1a1942fc8efa -de 125 skip-credits 334b1a1942fc8efa -de 126 skip-credits 334b1a1942fc8efa -de 127 skip-credits 83c01e9226cb9d18 -de 128 up-next 37ccaa292aea3062 -de 129 up-next 798b78f2025731e3 -de 130 up-next 798b78f2025731e3 -de 131 up-next 4e7d240a4dcfc899 -de 132 up-next 798b78f2025731e3 -de 133 up-next 334b1a1942fc8efa -de 134 up-next 334b1a1942fc8efa -de 135 up-next 334b1a1942fc8efa -de 136 up-next 334b1a1942fc8efa -de 137 up-next 83c01e9226cb9d18 -de 138 up-next d91e77e32c5851ab -de 139 up-next df8091ebb6fd5e74 -de 140 up-next df8091ebb6fd5e74 -de 141 up-next 6f27e3d2b1d1b4a2 -de 142 up-next df8091ebb6fd5e74 -de 143 up-next 334b1a1942fc8efa -de 144 up-next 334b1a1942fc8efa -de 145 up-next 334b1a1942fc8efa -de 146 up-next 334b1a1942fc8efa -de 147 up-next 83c01e9226cb9d18 +de 64 skip-intro 8bde2eb39dce358f +de 65 skip-intro 9c87bcd400bb6772 +de 66 skip-intro ab7eec7b8cd71a2e +de 67 skip-intro 9f7f4d80c9d6314a +de 68 skip-intro 334b1a1942fc8efa +de 69 skip-intro 334b1a1942fc8efa +de 70 skip-intro 334b1a1942fc8efa +de 71 skip-intro 83c01e9226cb9d18 +de 72 skip-intro 3a9e36feee002270 +de 73 skip-intro ff9c3a4157797f2f +de 74 skip-intro 636def457d060b9b +de 75 skip-intro 8dbe08bd98a0d3e3 +de 76 skip-intro 334b1a1942fc8efa +de 77 skip-intro 334b1a1942fc8efa +de 78 skip-intro 334b1a1942fc8efa +de 79 skip-intro 83c01e9226cb9d18 +de 80 skip-credits c7338fa2dab72d80 +de 81 skip-credits 2619df0b9509403f +de 82 skip-credits ae33954fab2277e7 +de 83 skip-credits b930ed4855c80fe7 +de 84 skip-credits 334b1a1942fc8efa +de 85 skip-credits 334b1a1942fc8efa +de 86 skip-credits 334b1a1942fc8efa +de 87 skip-credits 83c01e9226cb9d18 +de 88 skip-credits 0e6181848aaa6515 +de 89 skip-credits a90fb5dcde2c37d8 +de 90 skip-credits 391e20431645fc3c +de 91 skip-credits ba6361beeb4d60b8 +de 92 skip-credits 334b1a1942fc8efa +de 93 skip-credits 334b1a1942fc8efa +de 94 skip-credits 334b1a1942fc8efa +de 95 skip-credits 83c01e9226cb9d18 +de 96 up-next 37ccaa292aea3062 +de 97 up-next 798b78f2025731e3 +de 98 up-next 798b78f2025731e3 +de 99 up-next 4e7d240a4dcfc899 +de 100 up-next 798b78f2025731e3 +de 101 up-next 334b1a1942fc8efa +de 102 up-next 334b1a1942fc8efa +de 103 up-next 334b1a1942fc8efa +de 104 up-next 334b1a1942fc8efa +de 105 up-next 83c01e9226cb9d18 +de 106 up-next d91e77e32c5851ab +de 107 up-next df8091ebb6fd5e74 +de 108 up-next df8091ebb6fd5e74 +de 109 up-next 6f27e3d2b1d1b4a2 +de 110 up-next df8091ebb6fd5e74 +de 111 up-next 334b1a1942fc8efa +de 112 up-next 334b1a1942fc8efa +de 113 up-next 334b1a1942fc8efa +de 114 up-next 334b1a1942fc8efa +de 115 up-next 83c01e9226cb9d18 hu 0 movie 8764d4596fa81f11 hu 1 movie 599414dbb681321e hu 2 movie 599414dbb681321e @@ -1098,90 +874,58 @@ hu 60 pair 8b628858572e680a hu 61 pair 8b628858572e680a hu 62 pair 8b628858572e680a hu 63 pair af2a6ec9263af448 -hu 64 restart a973263fa5354863 -hu 65 restart c92a641aecec69b3 -hu 66 restart 1099f20ebcc0c98a -hu 67 restart 66ec25710f4b7d3d -hu 68 restart 389821b0682a8892 -hu 69 restart f4236e118b0cebc0 -hu 70 restart 3363fdabaf79165c -hu 71 restart 1099f20ebcc0c98a -hu 72 restart 8b628858572e680a -hu 73 restart 8b628858572e680a -hu 74 restart 8b628858572e680a -hu 75 restart 8b628858572e680a -hu 76 restart 8b628858572e680a -hu 77 restart 8b628858572e680a -hu 78 restart 8b628858572e680a -hu 79 restart af2a6ec9263af448 -hu 80 restart 9c7b9b43fe4c58e8 -hu 81 restart 190e5a4ab9683e30 -hu 82 restart 2f16716bd7ebddaf -hu 83 restart 0842a67443c2c132 -hu 84 restart 6bb6916655b7c4cb -hu 85 restart f33325f6c6dc2549 -hu 86 restart c5a23dc1e31c48b1 -hu 87 restart 2f16716bd7ebddaf -hu 88 restart 8b628858572e680a -hu 89 restart 8b628858572e680a -hu 90 restart 8b628858572e680a -hu 91 restart 8b628858572e680a -hu 92 restart 8b628858572e680a -hu 93 restart 8b628858572e680a -hu 94 restart 8b628858572e680a -hu 95 restart af2a6ec9263af448 -hu 96 skip-intro 44d14df77ea0d2fb -hu 97 skip-intro 8f092155df8f0733 -hu 98 skip-intro ab73ad91fc08d376 -hu 99 skip-intro 784af0a9a7748c44 -hu 100 skip-intro 8b628858572e680a -hu 101 skip-intro 8b628858572e680a -hu 102 skip-intro 8b628858572e680a -hu 103 skip-intro af2a6ec9263af448 -hu 104 skip-intro d8298612a8d037dc -hu 105 skip-intro 2870f3f07041f2a8 -hu 106 skip-intro abd769717dff3fdb -hu 107 skip-intro add382a7975c5f09 -hu 108 skip-intro 8b628858572e680a -hu 109 skip-intro 8b628858572e680a -hu 110 skip-intro 8b628858572e680a -hu 111 skip-intro af2a6ec9263af448 -hu 112 skip-credits cde70cd1a8adf03e -hu 113 skip-credits 7c5fddcc948912c5 -hu 114 skip-credits cf9d08791dbd14af -hu 115 skip-credits f7d8dfaf8a485b47 -hu 116 skip-credits 8b628858572e680a -hu 117 skip-credits 8b628858572e680a -hu 118 skip-credits 8b628858572e680a -hu 119 skip-credits af2a6ec9263af448 -hu 120 skip-credits 3260dc1af3f5a047 -hu 121 skip-credits 9e0f1cee4a16936a -hu 122 skip-credits a4ffeef1dfcf6f74 -hu 123 skip-credits 60929945ee412198 -hu 124 skip-credits 8b628858572e680a -hu 125 skip-credits 8b628858572e680a -hu 126 skip-credits 8b628858572e680a -hu 127 skip-credits af2a6ec9263af448 -hu 128 up-next 601fd39eb2527174 -hu 129 up-next d72f9844e8a31def -hu 130 up-next d72f9844e8a31def -hu 131 up-next 0377d32bb9d66a81 -hu 132 up-next d72f9844e8a31def -hu 133 up-next 8b628858572e680a -hu 134 up-next 8b628858572e680a -hu 135 up-next 8b628858572e680a -hu 136 up-next 8b628858572e680a -hu 137 up-next af2a6ec9263af448 -hu 138 up-next 4d5f7054a1dce079 -hu 139 up-next 71d0deafda8ee750 -hu 140 up-next 71d0deafda8ee750 -hu 141 up-next 128056e175d2a1ca -hu 142 up-next 71d0deafda8ee750 -hu 143 up-next 8b628858572e680a -hu 144 up-next 8b628858572e680a -hu 145 up-next 8b628858572e680a -hu 146 up-next 8b628858572e680a -hu 147 up-next af2a6ec9263af448 +hu 64 skip-intro 44d14df77ea0d2fb +hu 65 skip-intro 8f092155df8f0733 +hu 66 skip-intro ab73ad91fc08d376 +hu 67 skip-intro 784af0a9a7748c44 +hu 68 skip-intro 8b628858572e680a +hu 69 skip-intro 8b628858572e680a +hu 70 skip-intro 8b628858572e680a +hu 71 skip-intro af2a6ec9263af448 +hu 72 skip-intro d8298612a8d037dc +hu 73 skip-intro 2870f3f07041f2a8 +hu 74 skip-intro abd769717dff3fdb +hu 75 skip-intro add382a7975c5f09 +hu 76 skip-intro 8b628858572e680a +hu 77 skip-intro 8b628858572e680a +hu 78 skip-intro 8b628858572e680a +hu 79 skip-intro af2a6ec9263af448 +hu 80 skip-credits cde70cd1a8adf03e +hu 81 skip-credits 7c5fddcc948912c5 +hu 82 skip-credits cf9d08791dbd14af +hu 83 skip-credits f7d8dfaf8a485b47 +hu 84 skip-credits 8b628858572e680a +hu 85 skip-credits 8b628858572e680a +hu 86 skip-credits 8b628858572e680a +hu 87 skip-credits af2a6ec9263af448 +hu 88 skip-credits 3260dc1af3f5a047 +hu 89 skip-credits 9e0f1cee4a16936a +hu 90 skip-credits a4ffeef1dfcf6f74 +hu 91 skip-credits 60929945ee412198 +hu 92 skip-credits 8b628858572e680a +hu 93 skip-credits 8b628858572e680a +hu 94 skip-credits 8b628858572e680a +hu 95 skip-credits af2a6ec9263af448 +hu 96 up-next 601fd39eb2527174 +hu 97 up-next d72f9844e8a31def +hu 98 up-next d72f9844e8a31def +hu 99 up-next 0377d32bb9d66a81 +hu 100 up-next d72f9844e8a31def +hu 101 up-next 8b628858572e680a +hu 102 up-next 8b628858572e680a +hu 103 up-next 8b628858572e680a +hu 104 up-next 8b628858572e680a +hu 105 up-next af2a6ec9263af448 +hu 106 up-next 4d5f7054a1dce079 +hu 107 up-next 71d0deafda8ee750 +hu 108 up-next 71d0deafda8ee750 +hu 109 up-next 128056e175d2a1ca +hu 110 up-next 71d0deafda8ee750 +hu 111 up-next 8b628858572e680a +hu 112 up-next 8b628858572e680a +hu 113 up-next 8b628858572e680a +hu 114 up-next 8b628858572e680a +hu 115 up-next af2a6ec9263af448 ru 0 movie 3a02e93fef51ca98 ru 1 movie 895a99445a45cbb1 ru 2 movie 895a99445a45cbb1 @@ -1246,90 +990,58 @@ ru 60 pair 06804beca3a7a067 ru 61 pair 06804beca3a7a067 ru 62 pair 06804beca3a7a067 ru 63 pair 5b0f789189e0f8d1 -ru 64 restart 75101568aeaabd44 -ru 65 restart 830b0c1dab246aa4 -ru 66 restart dcc7acfc103eb703 -ru 67 restart 7b9d94dba4c37d0e -ru 68 restart a23bcd66f5935c2f -ru 69 restart 582177fea0bd2245 -ru 70 restart 0f7727fb04919316 -ru 71 restart dcc7acfc103eb703 -ru 72 restart 06804beca3a7a067 -ru 73 restart 06804beca3a7a067 -ru 74 restart 06804beca3a7a067 -ru 75 restart 06804beca3a7a067 -ru 76 restart 06804beca3a7a067 -ru 77 restart 06804beca3a7a067 -ru 78 restart 06804beca3a7a067 -ru 79 restart 5b0f789189e0f8d1 -ru 80 restart 16be7b244ae52075 -ru 81 restart 1b9e98ca0e1908bd -ru 82 restart c217f258f40b0188 -ru 83 restart ebd33147d46d3157 -ru 84 restart 2a3c3ca02e22ed68 -ru 85 restart 00a62f144a622846 -ru 86 restart ebf3b9a68f2a83df -ru 87 restart c217f258f40b0188 -ru 88 restart 06804beca3a7a067 -ru 89 restart 06804beca3a7a067 -ru 90 restart 06804beca3a7a067 -ru 91 restart 06804beca3a7a067 -ru 92 restart 06804beca3a7a067 -ru 93 restart 06804beca3a7a067 -ru 94 restart 06804beca3a7a067 -ru 95 restart 5b0f789189e0f8d1 -ru 96 skip-intro a3c0ac3b5a81d03f -ru 97 skip-intro d9c0ee39b3eeb748 -ru 98 skip-intro 7b82c6ecdefa1123 -ru 99 skip-intro 002de7d4958221c8 -ru 100 skip-intro 06804beca3a7a067 -ru 101 skip-intro 06804beca3a7a067 -ru 102 skip-intro 06804beca3a7a067 -ru 103 skip-intro 5b0f789189e0f8d1 -ru 104 skip-intro cc8d47873cf399e0 -ru 105 skip-intro d8d8402b85cf7cd5 -ru 106 skip-intro 38b30d95d7e3c840 -ru 107 skip-intro b14cf1024c3bac2d -ru 108 skip-intro 06804beca3a7a067 -ru 109 skip-intro 06804beca3a7a067 -ru 110 skip-intro 06804beca3a7a067 -ru 111 skip-intro 5b0f789189e0f8d1 -ru 112 skip-credits cb2be221196f7846 -ru 113 skip-credits d4b0b84af61e85bd -ru 114 skip-credits 99538b4c216ea3ac -ru 115 skip-credits 6bd5da05383830b1 -ru 116 skip-credits 06804beca3a7a067 -ru 117 skip-credits 06804beca3a7a067 -ru 118 skip-credits 06804beca3a7a067 -ru 119 skip-credits 5b0f789189e0f8d1 -ru 120 skip-credits bc806b623db84b7f -ru 121 skip-credits c0fa550675c4e796 -ru 122 skip-credits 1d1407a5b2e918a5 -ru 123 skip-credits 0a046fcfbf99a52e -ru 124 skip-credits 06804beca3a7a067 -ru 125 skip-credits 06804beca3a7a067 -ru 126 skip-credits 06804beca3a7a067 -ru 127 skip-credits 5b0f789189e0f8d1 -ru 128 up-next f14c3f637f78fad1 -ru 129 up-next 1f4f2a0c99d70a60 -ru 130 up-next 1f4f2a0c99d70a60 -ru 131 up-next 7d6422770046fbfd -ru 132 up-next 1f4f2a0c99d70a60 -ru 133 up-next 06804beca3a7a067 -ru 134 up-next 06804beca3a7a067 -ru 135 up-next 06804beca3a7a067 -ru 136 up-next 06804beca3a7a067 -ru 137 up-next 5b0f789189e0f8d1 -ru 138 up-next 14c1e88a3637594e -ru 139 up-next 73679a97c6baecf5 -ru 140 up-next 73679a97c6baecf5 -ru 141 up-next 58b8c46b031162de -ru 142 up-next 73679a97c6baecf5 -ru 143 up-next 06804beca3a7a067 -ru 144 up-next 06804beca3a7a067 -ru 145 up-next 06804beca3a7a067 -ru 146 up-next 06804beca3a7a067 -ru 147 up-next 5b0f789189e0f8d1 +ru 64 skip-intro a3c0ac3b5a81d03f +ru 65 skip-intro d9c0ee39b3eeb748 +ru 66 skip-intro 7b82c6ecdefa1123 +ru 67 skip-intro 002de7d4958221c8 +ru 68 skip-intro 06804beca3a7a067 +ru 69 skip-intro 06804beca3a7a067 +ru 70 skip-intro 06804beca3a7a067 +ru 71 skip-intro 5b0f789189e0f8d1 +ru 72 skip-intro cc8d47873cf399e0 +ru 73 skip-intro d8d8402b85cf7cd5 +ru 74 skip-intro 38b30d95d7e3c840 +ru 75 skip-intro b14cf1024c3bac2d +ru 76 skip-intro 06804beca3a7a067 +ru 77 skip-intro 06804beca3a7a067 +ru 78 skip-intro 06804beca3a7a067 +ru 79 skip-intro 5b0f789189e0f8d1 +ru 80 skip-credits cb2be221196f7846 +ru 81 skip-credits d4b0b84af61e85bd +ru 82 skip-credits 99538b4c216ea3ac +ru 83 skip-credits 6bd5da05383830b1 +ru 84 skip-credits 06804beca3a7a067 +ru 85 skip-credits 06804beca3a7a067 +ru 86 skip-credits 06804beca3a7a067 +ru 87 skip-credits 5b0f789189e0f8d1 +ru 88 skip-credits bc806b623db84b7f +ru 89 skip-credits c0fa550675c4e796 +ru 90 skip-credits 1d1407a5b2e918a5 +ru 91 skip-credits 0a046fcfbf99a52e +ru 92 skip-credits 06804beca3a7a067 +ru 93 skip-credits 06804beca3a7a067 +ru 94 skip-credits 06804beca3a7a067 +ru 95 skip-credits 5b0f789189e0f8d1 +ru 96 up-next f14c3f637f78fad1 +ru 97 up-next 1f4f2a0c99d70a60 +ru 98 up-next 1f4f2a0c99d70a60 +ru 99 up-next 7d6422770046fbfd +ru 100 up-next 1f4f2a0c99d70a60 +ru 101 up-next 06804beca3a7a067 +ru 102 up-next 06804beca3a7a067 +ru 103 up-next 06804beca3a7a067 +ru 104 up-next 06804beca3a7a067 +ru 105 up-next 5b0f789189e0f8d1 +ru 106 up-next 14c1e88a3637594e +ru 107 up-next 73679a97c6baecf5 +ru 108 up-next 73679a97c6baecf5 +ru 109 up-next 58b8c46b031162de +ru 110 up-next 73679a97c6baecf5 +ru 111 up-next 06804beca3a7a067 +ru 112 up-next 06804beca3a7a067 +ru 113 up-next 06804beca3a7a067 +ru 114 up-next 06804beca3a7a067 +ru 115 up-next 5b0f789189e0f8d1 pl 0 movie 560a96ef3b5df991 pl 1 movie 9e307e0b700c6238 pl 2 movie 9e307e0b700c6238 @@ -1394,90 +1106,58 @@ pl 60 pair aa3022867a7f0f00 pl 61 pair aa3022867a7f0f00 pl 62 pair aa3022867a7f0f00 pl 63 pair 7120f63c44b1de1a -pl 64 restart d864c7cc14bc6443 -pl 65 restart 488bdbb2d2509b09 -pl 66 restart 0ee908cb927a4e84 -pl 67 restart 934ff88fb0f759b7 -pl 68 restart 37dc5b3c39373dc2 -pl 69 restart 01121c6bdb06290c -pl 70 restart 7bae2d794b227205 -pl 71 restart 0ee908cb927a4e84 -pl 72 restart aa3022867a7f0f00 -pl 73 restart aa3022867a7f0f00 -pl 74 restart aa3022867a7f0f00 -pl 75 restart aa3022867a7f0f00 -pl 76 restart aa3022867a7f0f00 -pl 77 restart aa3022867a7f0f00 -pl 78 restart aa3022867a7f0f00 -pl 79 restart 7120f63c44b1de1a -pl 80 restart c435459c0b60f7c8 -pl 81 restart 9ce332eeeddde262 -pl 82 restart ba2ab038914d2fb5 -pl 83 restart 49446669053cbc40 -pl 84 restart 70f7374611010b3b -pl 85 restart 95d8fa85c1aa9bcd -pl 86 restart aa52bd3556e1264a -pl 87 restart ba2ab038914d2fb5 -pl 88 restart aa3022867a7f0f00 -pl 89 restart aa3022867a7f0f00 -pl 90 restart aa3022867a7f0f00 -pl 91 restart aa3022867a7f0f00 -pl 92 restart aa3022867a7f0f00 -pl 93 restart aa3022867a7f0f00 -pl 94 restart aa3022867a7f0f00 -pl 95 restart 7120f63c44b1de1a -pl 96 skip-intro 7801624f2268985a -pl 97 skip-intro e960ff37e1edf0ac -pl 98 skip-intro ff89936129febb1a -pl 99 skip-intro be0e566f6479a6c3 -pl 100 skip-intro aa3022867a7f0f00 -pl 101 skip-intro aa3022867a7f0f00 -pl 102 skip-intro aa3022867a7f0f00 -pl 103 skip-intro 7120f63c44b1de1a -pl 104 skip-intro 84e54d5022955533 -pl 105 skip-intro 08d769bd4e875315 -pl 106 skip-intro 1e1ef2969c8909cf -pl 107 skip-intro 5db2e73d8d00abd4 -pl 108 skip-intro aa3022867a7f0f00 -pl 109 skip-intro aa3022867a7f0f00 -pl 110 skip-intro aa3022867a7f0f00 -pl 111 skip-intro 7120f63c44b1de1a -pl 112 skip-credits 856eaadf3dfe4211 -pl 113 skip-credits f30bb775ce9cfcfe -pl 114 skip-credits 1c09cd95a02e862b -pl 115 skip-credits e333dfcb1e5b6620 -pl 116 skip-credits aa3022867a7f0f00 -pl 117 skip-credits aa3022867a7f0f00 -pl 118 skip-credits aa3022867a7f0f00 -pl 119 skip-credits 7120f63c44b1de1a -pl 120 skip-credits 0ae3b11a79a0b18e -pl 121 skip-credits d6d31d8568480877 -pl 122 skip-credits eab1d9ae42723d58 -pl 123 skip-credits 8125819609c3bbb5 -pl 124 skip-credits aa3022867a7f0f00 -pl 125 skip-credits aa3022867a7f0f00 -pl 126 skip-credits aa3022867a7f0f00 -pl 127 skip-credits 7120f63c44b1de1a -pl 128 up-next a4cc96cdd39508b6 -pl 129 up-next 8a13f2ac7dc29a53 -pl 130 up-next 8a13f2ac7dc29a53 -pl 131 up-next bf5b3eb63854d316 -pl 132 up-next 8a13f2ac7dc29a53 -pl 133 up-next aa3022867a7f0f00 -pl 134 up-next aa3022867a7f0f00 -pl 135 up-next aa3022867a7f0f00 -pl 136 up-next aa3022867a7f0f00 -pl 137 up-next 7120f63c44b1de1a -pl 138 up-next dd00f8ea8437f7af -pl 139 up-next f40892a94470ca64 -pl 140 up-next f40892a94470ca64 -pl 141 up-next 41d881840641fe6b -pl 142 up-next f40892a94470ca64 -pl 143 up-next aa3022867a7f0f00 -pl 144 up-next aa3022867a7f0f00 -pl 145 up-next aa3022867a7f0f00 -pl 146 up-next aa3022867a7f0f00 -pl 147 up-next 7120f63c44b1de1a +pl 64 skip-intro 7801624f2268985a +pl 65 skip-intro e960ff37e1edf0ac +pl 66 skip-intro ff89936129febb1a +pl 67 skip-intro be0e566f6479a6c3 +pl 68 skip-intro aa3022867a7f0f00 +pl 69 skip-intro aa3022867a7f0f00 +pl 70 skip-intro aa3022867a7f0f00 +pl 71 skip-intro 7120f63c44b1de1a +pl 72 skip-intro 84e54d5022955533 +pl 73 skip-intro 08d769bd4e875315 +pl 74 skip-intro 1e1ef2969c8909cf +pl 75 skip-intro 5db2e73d8d00abd4 +pl 76 skip-intro aa3022867a7f0f00 +pl 77 skip-intro aa3022867a7f0f00 +pl 78 skip-intro aa3022867a7f0f00 +pl 79 skip-intro 7120f63c44b1de1a +pl 80 skip-credits 856eaadf3dfe4211 +pl 81 skip-credits f30bb775ce9cfcfe +pl 82 skip-credits 1c09cd95a02e862b +pl 83 skip-credits e333dfcb1e5b6620 +pl 84 skip-credits aa3022867a7f0f00 +pl 85 skip-credits aa3022867a7f0f00 +pl 86 skip-credits aa3022867a7f0f00 +pl 87 skip-credits 7120f63c44b1de1a +pl 88 skip-credits 0ae3b11a79a0b18e +pl 89 skip-credits d6d31d8568480877 +pl 90 skip-credits eab1d9ae42723d58 +pl 91 skip-credits 8125819609c3bbb5 +pl 92 skip-credits aa3022867a7f0f00 +pl 93 skip-credits aa3022867a7f0f00 +pl 94 skip-credits aa3022867a7f0f00 +pl 95 skip-credits 7120f63c44b1de1a +pl 96 up-next a4cc96cdd39508b6 +pl 97 up-next 8a13f2ac7dc29a53 +pl 98 up-next 8a13f2ac7dc29a53 +pl 99 up-next bf5b3eb63854d316 +pl 100 up-next 8a13f2ac7dc29a53 +pl 101 up-next aa3022867a7f0f00 +pl 102 up-next aa3022867a7f0f00 +pl 103 up-next aa3022867a7f0f00 +pl 104 up-next aa3022867a7f0f00 +pl 105 up-next 7120f63c44b1de1a +pl 106 up-next dd00f8ea8437f7af +pl 107 up-next f40892a94470ca64 +pl 108 up-next f40892a94470ca64 +pl 109 up-next 41d881840641fe6b +pl 110 up-next f40892a94470ca64 +pl 111 up-next aa3022867a7f0f00 +pl 112 up-next aa3022867a7f0f00 +pl 113 up-next aa3022867a7f0f00 +pl 114 up-next aa3022867a7f0f00 +pl 115 up-next 7120f63c44b1de1a nl 0 movie 73d83cea8d35cf0d nl 1 movie 1e0679d5ceedcd8c nl 2 movie 1e0679d5ceedcd8c @@ -1542,87 +1222,55 @@ nl 60 pair 334b1a1942fc8efa nl 61 pair 334b1a1942fc8efa nl 62 pair 334b1a1942fc8efa nl 63 pair 83c01e9226cb9d18 -nl 64 restart 7fb012b76d2cb6c7 -nl 65 restart 64f39a8af802fc65 -nl 66 restart 93e93408ab0b5090 -nl 67 restart da1f5f97de03a16f -nl 68 restart 05eede0784f7678e -nl 69 restart 3824be00c631db3c -nl 70 restart d473239538ec832e -nl 71 restart 93e93408ab0b5090 -nl 72 restart 334b1a1942fc8efa -nl 73 restart 334b1a1942fc8efa -nl 74 restart 334b1a1942fc8efa -nl 75 restart 334b1a1942fc8efa -nl 76 restart 334b1a1942fc8efa -nl 77 restart 334b1a1942fc8efa -nl 78 restart 334b1a1942fc8efa -nl 79 restart 83c01e9226cb9d18 -nl 80 restart 6b201affa7e1b2e4 -nl 81 restart f9f9444a5c1cd4b6 -nl 82 restart 77f7bb557c848ef9 -nl 83 restart 082a546af5474618 -nl 84 restart a0d518fa5e7a051f -nl 85 restart a44d7319c365ca3d -nl 86 restart dee71acd1d951a17 -nl 87 restart 77f7bb557c848ef9 -nl 88 restart 334b1a1942fc8efa -nl 89 restart 334b1a1942fc8efa -nl 90 restart 334b1a1942fc8efa -nl 91 restart 334b1a1942fc8efa -nl 92 restart 334b1a1942fc8efa -nl 93 restart 334b1a1942fc8efa -nl 94 restart 334b1a1942fc8efa -nl 95 restart 83c01e9226cb9d18 -nl 96 skip-intro 87f22ab5c16031f5 -nl 97 skip-intro 387d79fceefcd3c3 -nl 98 skip-intro 0ffd0f81566ed72c -nl 99 skip-intro 397f4154f6ba7170 -nl 100 skip-intro 334b1a1942fc8efa -nl 101 skip-intro 334b1a1942fc8efa -nl 102 skip-intro 334b1a1942fc8efa -nl 103 skip-intro 83c01e9226cb9d18 -nl 104 skip-intro ba6453abd142dfc2 -nl 105 skip-intro 88aa9b67e36283e4 -nl 106 skip-intro 6eabd769837416bd -nl 107 skip-intro 1ab64f9c05b87c45 -nl 108 skip-intro 334b1a1942fc8efa -nl 109 skip-intro 334b1a1942fc8efa -nl 110 skip-intro 334b1a1942fc8efa -nl 111 skip-intro 83c01e9226cb9d18 -nl 112 skip-credits 4e223fb02918269e -nl 113 skip-credits e355d58fcc4c5c20 -nl 114 skip-credits 8510bd632a301377 -nl 115 skip-credits c256b0921cb76139 -nl 116 skip-credits 334b1a1942fc8efa -nl 117 skip-credits 334b1a1942fc8efa -nl 118 skip-credits 334b1a1942fc8efa -nl 119 skip-credits 83c01e9226cb9d18 -nl 120 skip-credits 51995a0ca205ce27 -nl 121 skip-credits 22bdf3771fd79e35 -nl 122 skip-credits f24862444fa506cc -nl 123 skip-credits 2ced00ee38a0e0c6 -nl 124 skip-credits 334b1a1942fc8efa -nl 125 skip-credits 334b1a1942fc8efa -nl 126 skip-credits 334b1a1942fc8efa -nl 127 skip-credits 83c01e9226cb9d18 -nl 128 up-next 37ccaa292aea3062 -nl 129 up-next 798b78f2025731e3 -nl 130 up-next 798b78f2025731e3 -nl 131 up-next 4e7d240a4dcfc899 -nl 132 up-next 798b78f2025731e3 -nl 133 up-next 334b1a1942fc8efa -nl 134 up-next 334b1a1942fc8efa -nl 135 up-next 334b1a1942fc8efa -nl 136 up-next 334b1a1942fc8efa -nl 137 up-next 83c01e9226cb9d18 -nl 138 up-next d91e77e32c5851ab -nl 139 up-next df8091ebb6fd5e74 -nl 140 up-next df8091ebb6fd5e74 -nl 141 up-next 6f27e3d2b1d1b4a2 -nl 142 up-next df8091ebb6fd5e74 -nl 143 up-next 334b1a1942fc8efa -nl 144 up-next 334b1a1942fc8efa -nl 145 up-next 334b1a1942fc8efa -nl 146 up-next 334b1a1942fc8efa -nl 147 up-next 83c01e9226cb9d18 +nl 64 skip-intro 87f22ab5c16031f5 +nl 65 skip-intro 387d79fceefcd3c3 +nl 66 skip-intro 0ffd0f81566ed72c +nl 67 skip-intro 397f4154f6ba7170 +nl 68 skip-intro 334b1a1942fc8efa +nl 69 skip-intro 334b1a1942fc8efa +nl 70 skip-intro 334b1a1942fc8efa +nl 71 skip-intro 83c01e9226cb9d18 +nl 72 skip-intro ba6453abd142dfc2 +nl 73 skip-intro 88aa9b67e36283e4 +nl 74 skip-intro 6eabd769837416bd +nl 75 skip-intro 1ab64f9c05b87c45 +nl 76 skip-intro 334b1a1942fc8efa +nl 77 skip-intro 334b1a1942fc8efa +nl 78 skip-intro 334b1a1942fc8efa +nl 79 skip-intro 83c01e9226cb9d18 +nl 80 skip-credits 4e223fb02918269e +nl 81 skip-credits e355d58fcc4c5c20 +nl 82 skip-credits 8510bd632a301377 +nl 83 skip-credits c256b0921cb76139 +nl 84 skip-credits 334b1a1942fc8efa +nl 85 skip-credits 334b1a1942fc8efa +nl 86 skip-credits 334b1a1942fc8efa +nl 87 skip-credits 83c01e9226cb9d18 +nl 88 skip-credits 51995a0ca205ce27 +nl 89 skip-credits 22bdf3771fd79e35 +nl 90 skip-credits f24862444fa506cc +nl 91 skip-credits 2ced00ee38a0e0c6 +nl 92 skip-credits 334b1a1942fc8efa +nl 93 skip-credits 334b1a1942fc8efa +nl 94 skip-credits 334b1a1942fc8efa +nl 95 skip-credits 83c01e9226cb9d18 +nl 96 up-next 37ccaa292aea3062 +nl 97 up-next 798b78f2025731e3 +nl 98 up-next 798b78f2025731e3 +nl 99 up-next 4e7d240a4dcfc899 +nl 100 up-next 798b78f2025731e3 +nl 101 up-next 334b1a1942fc8efa +nl 102 up-next 334b1a1942fc8efa +nl 103 up-next 334b1a1942fc8efa +nl 104 up-next 334b1a1942fc8efa +nl 105 up-next 83c01e9226cb9d18 +nl 106 up-next d91e77e32c5851ab +nl 107 up-next df8091ebb6fd5e74 +nl 108 up-next df8091ebb6fd5e74 +nl 109 up-next 6f27e3d2b1d1b4a2 +nl 110 up-next df8091ebb6fd5e74 +nl 111 up-next 334b1a1942fc8efa +nl 112 up-next 334b1a1942fc8efa +nl 113 up-next 334b1a1942fc8efa +nl 114 up-next 334b1a1942fc8efa +nl 115 up-next 83c01e9226cb9d18 diff --git a/rust-modules/appkit/src/player_hud/command.rs b/rust-modules/appkit/src/player_hud/command.rs index 0fd621d5bb..51c42070ff 100644 --- a/rust-modules/appkit/src/player_hud/command.rs +++ b/rust-modules/appkit/src/player_hud/command.rs @@ -126,7 +126,8 @@ pub enum HudCommand { TogglePlay, /// A relative jump. Seek(SeekBy), - /// The previous episode (or a restart of this one, the Previous disc's glyph says which). + /// The previous episode of the queue. Never a restart of this one: that is + /// [`HudCommand::FromBeginning`]. Previous, /// The next episode. Next, @@ -151,7 +152,7 @@ impl HudCommand { match self { // a jump needs a known duration to clamp against (`commit_jump` returns without one) HudCommand::Seek(_) => model.duration_ns > 0, - // `Controls` already folds a restart target into `previous` (`Controls::for_item`) + // the playing row has a predecessor in the queue (`Controls::for_item`) HudCommand::Previous => model.controls.previous, HudCommand::Next => model.controls.next, HudCommand::ToggleSection(Section::Chapters) => model.chapters, @@ -273,7 +274,7 @@ mod tests { /// performer used to spell as a `match` on `Action`. #[test] fn each_classic_control_is_the_command_of_the_action_it_always_performed() { - let episode = Controls::for_item(true, true, true, false); + let episode = Controls::for_item(true, true, true); let want = [ (BTN_PREVIOUS, Action::Previous, HudCommand::Previous), (BTN_NEXT, Action::Next, HudCommand::Next), @@ -299,15 +300,13 @@ mod tests { fn command_and_action_agree_for_every_row_the_discs_can_hold() { for previous in [false, true] { for next in [false, true] { - for restart in [false, true] { - let controls = Controls { previous, next, restart }; - for i in -1..=BTN_N { - assert_eq!( - controls.command(i), - controls.action(i).map(HudCommand::from), - "{controls:?} control {i}" - ); - } + let controls = Controls { previous, next }; + for i in -1..=BTN_N { + assert_eq!( + controls.command(i), + controls.action(i).map(HudCommand::from), + "{controls:?} control {i}" + ); } } } @@ -339,15 +338,15 @@ mod tests { assert!(HudCommand::Seek(by).enabled(&model(|m| m.duration_ns = 1)), "{by:?} with one"); } - // Previous is enabled by `previous || restart` (a restart target counts), Next by `next` - let controls = |previous, next, restart| model(|m| m.controls = Controls::for_item(true, previous, next, restart)); + // Previous is enabled by `previous` (a predecessor in the queue, and nothing else), Next by `next` + let controls = |previous, next| model(|m| m.controls = Controls::for_item(true, previous, next)); assert!(!HudCommand::Previous.enabled(&idle) && !HudCommand::Next.enabled(&idle), "not an episode"); - assert!(!HudCommand::Previous.enabled(&controls(false, true, false)), "first episode, nothing to restart"); - assert!(HudCommand::Previous.enabled(&controls(false, true, true)), "a restart target counts"); - assert!(HudCommand::Previous.enabled(&controls(true, false, false))); - assert!(!HudCommand::Next.enabled(&controls(true, false, true)), "the last episode has no Next"); - assert!(HudCommand::Next.enabled(&controls(false, true, false))); - assert!(!HudCommand::Next.enabled(&controls(false, false, true)), "neither neighbour: no pair"); + assert!(!HudCommand::Previous.enabled(&controls(false, true)), "first episode: there is no episode before it"); + assert!(HudCommand::Previous.enabled(&controls(true, false))); + assert!(HudCommand::Previous.enabled(&controls(true, true))); + assert!(!HudCommand::Next.enabled(&controls(true, false)), "the last episode has no Next"); + assert!(HudCommand::Next.enabled(&controls(false, true))); + assert!(!HudCommand::Next.enabled(&controls(false, false)), "neither neighbour: no pair"); // sections and the detail page need what they open assert!(!HudCommand::ToggleSection(Section::Chapters).enabled(&idle)); @@ -362,16 +361,14 @@ mod tests { fn a_discs_command_is_enabled_exactly_when_its_disc_has_a_stop() { for previous in [false, true] { for next in [false, true] { - for restart in [false, true] { - let controls = Controls::for_item(true, previous, next, restart); - let m = model(|m| m.controls = controls); - for i in 0..BTN_N { - assert_eq!( - CLASSIC_COMMANDS[i as usize].enabled(&m), - controls.enabled(i), - "{controls:?} control {i}" - ); - } + let controls = Controls::for_item(true, previous, next); + let m = model(|m| m.controls = controls); + for i in 0..BTN_N { + assert_eq!( + CLASSIC_COMMANDS[i as usize].enabled(&m), + controls.enabled(i), + "{controls:?} control {i}" + ); } } } diff --git a/rust-modules/appkit/src/player_hud/model.rs b/rust-modules/appkit/src/player_hud/model.rs index 2f46ff9112..36e15193da 100644 --- a/rust-modules/appkit/src/player_hud/model.rs +++ b/rust-modules/appkit/src/player_hud/model.rs @@ -62,8 +62,8 @@ pub struct HudModel { /// `plx_media::player::active_bitmap_key(plx_media::player::playpos_ns(ps))`: the `start_ns` key /// of the selected track's image cue active now, `None` between cues. pub bitmap_key: Option, - /// `Controls::now(ps)`: which of the Previous/Next pair the playing episode has, and whether - /// Previous restarts. The enabled predicates of `Previous` and `Next` read it + /// `Controls::now(ps)`: which of the Previous/Next pair the playing episode has (a predecessor + /// and a successor in the queue). The enabled predicates of `Previous` and `Next` read it /// ([`super::command::HudCommand::enabled`]). pub controls: super::state::Controls, /// `plx_appkit::chapters_panel::has_chapters(meta)`: the playing item has chapters. diff --git a/rust-modules/appkit/src/player_hud/render.rs b/rust-modules/appkit/src/player_hud/render.rs index d2566772b6..8549cc81c5 100644 --- a/rust-modules/appkit/src/player_hud/render.rs +++ b/rust-modules/appkit/src/player_hud/render.rs @@ -797,7 +797,6 @@ fn face_state(act: HudCommand, m: &HudModel) -> &'static str { match act { HudCommand::TogglePlay if m.paused => "paused", HudCommand::TogglePlay => "playing", - HudCommand::Previous if m.controls.restart => "restart", HudCommand::ToggleStats if m.stats => "on", HudCommand::ToggleStats => "off", _ => "default", @@ -808,7 +807,6 @@ fn face_state(act: HudCommand, m: &HudModel) -> &'static str { /// `Controls::icon`'s, so the replaced row and the old one cannot disagree. fn classic_glyph(act: Option, m: &HudModel) -> Icon { match act { - Some(HudCommand::Previous) if m.controls.restart => Icon::Restart, Some(HudCommand::Previous) => Icon::Previous, Some(HudCommand::Next) => Icon::Next, Some(HudCommand::OpenTracks(TrackTab::Audio)) => Icon::Audio, @@ -1018,8 +1016,7 @@ mod tests { } const FM: FixtureMeasure = FixtureMeasure; - const PAIR: Controls = Controls { previous: true, next: true, restart: false }; - const RESTART: Controls = Controls { previous: true, next: false, restart: true }; + const PAIR: Controls = Controls { previous: true, next: true }; /// Every primitive a frame hands the painter: the command census, the text runs, the glyphs and the /// fills with their effective alpha, in draw order. @@ -1092,7 +1089,6 @@ mod tests { let slots = [ ("movie", ControlSlot::Discs(Controls::NONE)), ("pair", ControlSlot::Discs(PAIR)), - ("restart", ControlSlot::Discs(RESTART)), ("skip-intro", ControlSlot::Skip(prompt_for(marker(MarkerKind::Intro, false)))), ("skip-credits", ControlSlot::Skip(prompt_for(marker(MarkerKind::Credits, false)))), ("up-next", ControlSlot::UpNext(marker(MarkerKind::Credits, true))), @@ -1945,8 +1941,8 @@ mod tests { let pair = frame_census_with(&compiled, ControlSlot::Discs(PAIR), GROUP_SCRUB, 0, model()); let icons: Vec = pair.2.iter().map(|(i, _)| *i).collect(); assert!(icons.contains(&Icon::Previous) && icons.contains(&Icon::Next), "an episode pair: {icons:?}"); - let restart = frame_census_with(&compiled, ControlSlot::Discs(RESTART), GROUP_SCRUB, 0, model()); - assert!(restart.2.iter().any(|(i, _)| *i == Icon::Restart), "Previous's `restart` face"); + // Previous never takes its package's `restart` face: that state is no longer selected + assert!(!pair.2.iter().any(|(i, _)| *i == Icon::Restart), "a Previous disc is not a restart: {icons:?}"); } #[test] diff --git a/rust-modules/appkit/src/player_hud/render/tests/conformance.rs b/rust-modules/appkit/src/player_hud/render/tests/conformance.rs index b916a19008..e5f2e54e77 100644 --- a/rust-modules/appkit/src/player_hud/render/tests/conformance.rs +++ b/rust-modules/appkit/src/player_hud/render/tests/conformance.rs @@ -482,7 +482,7 @@ fn content_geometry_focus_and_pair_availability_are_independent() { r#""thickness": { "value": 6, "if": { "title.is_episode": 12 } }"#)); for pair in [false, true] { for episode in [false, true] { - let slot = ControlSlot::Discs(if pair { Controls { previous: true, next: true, restart: false } } else { Controls::NONE }); + let slot = ControlSlot::Discs(if pair { Controls { previous: true, next: true } } else { Controls::NONE }); let state = compiled.lookup_content(0, crate::hud_format::Slot::Discs, false, pair, episode, false).unwrap(); assert_eq!(state.control("prev").is_some(), pair, "pair={pair} episode={episode}"); let sized = crate::player_hud::geometry::geometry_content(&numeric, 0, slot, false, episode, false); diff --git a/rust-modules/appkit/src/player_hud/state.rs b/rust-modules/appkit/src/player_hud/state.rs index 70f038d364..8950b8c42c 100644 --- a/rust-modules/appkit/src/player_hud/state.rs +++ b/rust-modules/appkit/src/player_hud/state.rs @@ -38,25 +38,26 @@ pub const BTN_MORE: i32 = BTN_N - 1; #[derive(Clone, Copy, Debug, PartialEq, Eq)] pub enum Action { Previous, Next, Subtitles, Audio, More } -/// A frame's episode-button eligibility, availability and restart glyph, sampled once. +/// A frame's episode-button eligibility and availability, sampled once. **Previous is the previous +/// episode of the queue and nothing else**: it is enabled exactly when the playing row has a +/// predecessor (`queue_neighbors`), so on the first episode of a queue it is disabled. Restarting +/// the current episode is `HudCommand::FromBeginning` (the Info tab's action, and a More-menu row +/// in a package that folds it), never a second meaning of this button. #[derive(Clone, Copy, Debug, PartialEq, Eq)] pub struct Controls { pub previous: bool, pub next: bool, - pub restart: bool, } impl Controls { - pub const NONE: Self = Self { previous: false, next: false, restart: false }; + pub const NONE: Self = Self { previous: false, next: false }; pub fn now(ps: &plx_media::route::PlaybackSession) -> Self { if plx_media::route::cur_kind(ps) != "episode" { return Self::NONE; } let (previous, next) = plx_media::route::queue_neighbors(ps); - if !(previous || next) { return Self::NONE; } - let restart = plx_media::route::previous_restarts(plx_media::player::intended_pos_ns(ps)); - Self::for_item(true, previous, next, restart) + Self::for_item(true, previous, next) } - pub fn for_item(episode: bool, previous: bool, next: bool, restart: bool) -> Self { + pub fn for_item(episode: bool, previous: bool, next: bool) -> Self { if !episode || !(previous || next) { return Self::NONE; } - Self { previous: previous || restart, next, restart } + Self { previous, next } } /// Whether the Previous/Next pair is on the row at all: [`Self::for_item`] only ever builds a /// pair with at least one of its two buttons live. @@ -85,7 +86,7 @@ impl Controls { } pub fn icon(self, i: i32) -> plx_ui::icons::Icon { use plx_ui::icons::Icon; - match i { BTN_PREVIOUS if self.restart => Icon::Restart, BTN_PREVIOUS => Icon::Previous, + match i { BTN_PREVIOUS => Icon::Previous, BTN_NEXT => Icon::Next, BTN_AUDIO => Icon::Audio, BTN_MORE => Icon::More, _ => Icon::Cc } } } diff --git a/rust-modules/appkit/src/player_hud/tests.rs b/rust-modules/appkit/src/player_hud/tests.rs index e6a38ed606..e6a568941d 100644 --- a/rust-modules/appkit/src/player_hud/tests.rs +++ b/rust-modules/appkit/src/player_hud/tests.rs @@ -85,7 +85,7 @@ fn long_hud_titles_fit_before_every_control_slot() { let compiled = super::loaded::load_classic("test", plx_platform::i18n::current(), measure).compiled; let slots = [ ControlSlot::Discs(Controls::NONE), - ControlSlot::Discs(Controls { previous: true, next: true, restart: true }), + ControlSlot::Discs(Controls { previous: true, next: true }), slot_for(Some(marker(MarkerKind::Intro, false)), false, NextEpisodeMode::Countdown), slot_for(Some(marker(MarkerKind::Credits, true)), true, NextEpisodeMode::Countdown), ]; @@ -1495,7 +1495,7 @@ fn the_layouts_band_is_classics_in_every_slot() { let compiled = super::loaded::classic_for_test(); let slots = [ ControlSlot::Discs(Controls::NONE), - ControlSlot::Discs(Controls { previous: true, next: true, restart: true }), + ControlSlot::Discs(Controls { previous: true, next: true }), slot_for(Some(marker(MarkerKind::Intro, false)), false, NextEpisodeMode::Countdown), slot_for(Some(marker(MarkerKind::Credits, true)), true, NextEpisodeMode::Countdown), ]; @@ -1569,7 +1569,7 @@ mod playback_navigation_tests { #[test] fn episode_pair_uses_custom_gap_without_moving_existing_discs() { let compiled = super::super::loaded::classic_for_test(); - let pair = Controls::for_item(true, false, true, false); + let pair = Controls::for_item(true, false, true); let rects = |c: Controls| geometry(&compiled, 0, ControlSlot::Discs(c), false, false).items; let (pair_rects, none_rects) = (rects(pair), rects(Controls::NONE)); for i in BTN_SUBTITLES..=BTN_MORE { @@ -1586,19 +1586,32 @@ mod playback_navigation_tests { assert_eq!(pair.primary(), BTN_NEXT); assert_eq!(pair.action(BTN_PREVIOUS), None); assert_eq!(pair.icon(BTN_PREVIOUS), Icon::Previous); - let restarting = Controls::for_item(true, false, true, true); - assert_eq!(restarting.icon(BTN_PREVIOUS), Icon::Restart); - assert_eq!(restarting.action(BTN_PREVIOUS), Some(Action::Previous)); + } + + /// **Previous is the previous episode and nothing else.** Whatever the playhead, the disc is + /// enabled exactly when the queue has an episode before the playing one, and it always wears + /// the Previous glyph: there is no restart state to wear a different one. + #[test] + fn previous_is_enabled_by_a_predecessor_alone_and_always_wears_the_previous_glyph() { + let middle = Controls::for_item(true, true, true); + assert_eq!(middle.action(BTN_PREVIOUS), Some(Action::Previous)); + assert_eq!(middle.icon(BTN_PREVIOUS), Icon::Previous); + let last = Controls::for_item(true, true, false); + assert_eq!(last.action(BTN_PREVIOUS), Some(Action::Previous)); + assert_eq!(last.icon(BTN_PREVIOUS), Icon::Previous); + let first = Controls::for_item(true, false, true); + assert_eq!(first.action(BTN_PREVIOUS), None, "the first episode of the queue has no Previous"); + assert!(!first.enabled(BTN_PREVIOUS)); } #[test] fn episode_pair_is_absent_for_movies_extras_and_single_items() { - for controls in [Controls::for_item(false, true, true, true), - Controls::for_item(true, false, false, true)] { + for controls in [Controls::for_item(false, true, true), + Controls::for_item(true, false, false)] { assert_eq!(controls, Controls::NONE); assert_eq!(controls.primary(), BTN_SUBTITLES); } - let last = Controls::for_item(true, true, false, false); + let last = Controls::for_item(true, true, false); assert_eq!(last.primary(), BTN_PREVIOUS); assert_eq!(last.action(BTN_NEXT), None); } @@ -1606,8 +1619,8 @@ mod playback_navigation_tests { /// **Under `builtin:classic`, the row resolved through the compile is the row resolved through `Controls`.** /// The page now asks the package's control at an index what a press performs and whether it has a stop; -/// for classic's five discs that must be exactly classic's table, for every valuation of the pair and of -/// Previous's restart target, and the cursor parks where `Controls::primary` parks it. +/// for classic's five discs that must be exactly classic's table, for every valuation of the pair, and the +/// cursor parks where `Controls::primary` parks it. #[test] fn classic_resolved_through_the_compile_is_classics_table() { use super::command::HudCommand; @@ -1619,16 +1632,14 @@ fn classic_resolved_through_the_compile_is_classics_table() { for episode in [false, true] { for previous in [false, true] { for next in [false, true] { - for restart in [false, true] { - let controls = Controls::for_item(episode, previous, next, restart); - let slot = ControlSlot::Discs(controls); - for i in -1..=BTN_N + 2 { - let want: Option = controls.command(i); - assert_eq!(row_command(compiled, 0, slot, &model, i), want, "{controls:?} item {i}"); - assert_eq!(row_enabled(compiled, 0, slot, &model, i), controls.enabled(i), "{controls:?} item {i}"); - } - assert_eq!(row_primary(compiled, 0, slot, &model), controls.primary(), "{controls:?}"); + let controls = Controls::for_item(episode, previous, next); + let slot = ControlSlot::Discs(controls); + for i in -1..=BTN_N + 2 { + let want: Option = controls.command(i); + assert_eq!(row_command(compiled, 0, slot, &model, i), want, "{controls:?} item {i}"); + assert_eq!(row_enabled(compiled, 0, slot, &model, i), controls.enabled(i), "{controls:?} item {i}"); } + assert_eq!(row_primary(compiled, 0, slot, &model), controls.primary(), "{controls:?}"); } } } diff --git a/rust-modules/appkit/src/player_hud/travel.rs b/rust-modules/appkit/src/player_hud/travel.rs index 17535362f1..6346d0c543 100644 --- a/rust-modules/appkit/src/player_hud/travel.rs +++ b/rust-modules/appkit/src/player_hud/travel.rs @@ -202,7 +202,7 @@ mod tests { } fn order_of(compiled: &Compiled, open: u8) -> Order { - let slot = ControlSlot::Discs(Controls { previous: false, next: false, restart: false }); + let slot = ControlSlot::Discs(Controls { previous: false, next: false }); order(compiled, open, slot, true, false) } @@ -279,7 +279,7 @@ mod tests { /// the tab group and this order all ask, and a disc reported there would be a second stop at the same rect. #[test] fn a_section_a_disc_opens_has_no_tab() { - let slot = ControlSlot::Discs(Controls { previous: false, next: false, restart: false }); + let slot = ControlSlot::Discs(Controls { previous: false, next: false }); let reference = reference(); for open in [0, CHAPTERS, INFO, CHAPTERS | INFO] { let geo = geometry(&reference, open, slot, true, false); @@ -298,7 +298,7 @@ mod tests { #[test] fn each_group_links_to_its_neighbours_both_ways() { let mut out = Vec::new(); - let order = order(&classic_for_test(), 0, ControlSlot::Discs(Controls { previous: false, next: false, restart: false }), false, false); + let order = order(&classic_for_test(), 0, ControlSlot::Discs(Controls { previous: false, next: false }), false, false); order.links(&mut out); let link = |from, dir, to| Link { from, dir, to }; assert_eq!(out.len(), 4); @@ -350,7 +350,7 @@ mod tests { /// tabs' is the pills', and rising-tabs' pills stand wholly above the page in both its states, classic's wholly below. #[test] fn the_spans_the_lint_reads_are_the_orders_and_the_tabs_clear_the_page() { - let slot = ControlSlot::Discs(Controls { previous: false, next: false, restart: false }); + let slot = ControlSlot::Discs(Controls { previous: false, next: false }); for (compiled, tabs_above) in [(classic_for_test(), false), (rising_tabs(), true)] { for open in [CHAPTERS, INFO] { let (state, skip) = frame_state(&compiled, open, slot, true, false).expect("the state"); diff --git a/rust-modules/data/src/metadata.rs b/rust-modules/data/src/metadata.rs index 24fa466b43..be8b4e7c1a 100644 --- a/rust-modules/data/src/metadata.rs +++ b/rust-modules/data/src/metadata.rs @@ -1738,6 +1738,22 @@ pub struct Detail { } impl Detail { + /// **The show an episode played from this page belongs to**, which the play queue is built + /// from so the episodes BEFORE it are in the queue: a show page's own key, a season page's + /// parent, an episode page's grandparent, empty for anything else (a movie). The `show_rk` field + /// alone is the grandparent key and stays empty on a show page and on a season page. + pub fn episodes_show_rk(&self) -> &str { + if self.is_show { + &self.rk + } else if self.kind == "season" { + &self.parent_rk + } else if self.kind == "episode" { + &self.show_rk + } else { + "" + } + } + /// **Does this item have a FILE of its own?** — the rule behind the *Track information* sheet /// (`screens::tracks_panel`) and the Languages column's press gate on the detail page. /// @@ -6154,6 +6170,27 @@ mod episode_tests { } } +#[cfg(test)] +mod episodes_show_rk_tests { + use super::*; + + /// **The show an episode played from a page belongs to, per kind of page**: the play queue is + /// built from it (`route::request_play`), so the episodes before the played one are in the + /// queue and Previous has somewhere to go. Only the pages that play episodes have one. + #[test] + fn each_page_that_plays_episodes_names_its_show() { + let show = Detail { is_show: true, kind: "show".into(), rk: "50".into(), ..Default::default() }; + assert_eq!(show.episodes_show_rk(), "50", "a show page's own key"); + let season = Detail { kind: "season".into(), rk: "60".into(), parent_rk: "50".into(), ..Default::default() }; + assert_eq!(season.episodes_show_rk(), "50", "a season page's parent"); + let episode = Detail { kind: "episode".into(), rk: "101".into(), show_rk: "50".into(), ..Default::default() }; + assert_eq!(episode.episodes_show_rk(), "50", "an episode page's grandparent"); + let movie = Detail { kind: "movie".into(), rk: "9".into(), show_rk: "50".into(), ..Default::default() }; + assert_eq!(movie.episodes_show_rk(), "", "a movie has no show, whatever a stale field holds"); + assert_eq!(Detail::default().episodes_show_rk(), ""); + } +} + #[cfg(test)] mod rating_tests { use super::*; diff --git a/rust-modules/media/src/player/mod.rs b/rust-modules/media/src/player/mod.rs index 5fe8cdabcd..2bc4b572ff 100644 --- a/rust-modules/media/src/player/mod.rs +++ b/rust-modules/media/src/player/mod.rs @@ -2928,7 +2928,7 @@ mod tests { let mut meta = plx_data::stores::metadata::MetadataStore::default(); assert!(crate::route::request_play(&mut ps, &mut meta, plx_platform::devcaps::caps(), plx_plex::plex::ServerId::from_raw(0), - "film", "/library/parts/1/file.mkv", "h264", "aac", "Film", "", None)); + "film", "/library/parts/1/file.mkv", "h264", "aac", "Film", "", None, "")); assert!(crate::route::arm_play_resume(&mut ps, 12_000_000_000)); let attempt = crate::route::RouteStartAttempt::fixture(); ps.begin_attempt_view(attempt, attempt_view::Backend::Uri); diff --git a/rust-modules/media/src/route/decision.rs b/rust-modules/media/src/route/decision.rs index d907ba923d..1551ef1fb3 100644 --- a/rust-modules/media/src/route/decision.rs +++ b/rust-modules/media/src/route/decision.rs @@ -66,6 +66,11 @@ struct PlaybackRequest { /// The version the viewer picked (Play Version), `None` for the default. Kept on the request, /// so a retry or quality change of this playback resolves the same version. version: Option, + /// The show an EPISODE belongs to (`grandparentRatingKey`), empty for a movie, a clip, or an + /// item whose caller did not know it. The play queue is built from it + /// ([`ResolveEnv::queue_show_rk`]), so a retry or a quality change of this playback builds the + /// same queue. + show_rk: String, } #[derive(Clone, Copy, Debug, PartialEq, Eq)] @@ -645,7 +650,8 @@ mod uri_failure_recovery_tests { session.install_backends(plx_playback_contract::BackendSet::BOTH); session.request = Some(PlaybackRequest { sid: ServerId::UNSET, rk: "42".into(), part: "/library/parts/7".into(), vcodec: "h264".into(), acodec: "aac".into(), - title: "Film".into(), ctx: String::new(), preview: false, version: None }); + title: "Film".into(), ctx: String::new(), preview: false, version: None, + show_rk: String::new() }); let mut policy = BackendPolicy::for_request(false); policy.qualify(BackendKind::Uri, OfferId::URI_ORIGINAL, Qualification::Qualified); assert!(policy.admitted(BackendKind::Uri, OfferId::URI_ORIGINAL)); @@ -955,7 +961,7 @@ mod backend_offer_session_tests { session.request = Some(PlaybackRequest { sid: session.cur_sid, rk: "42".into(), part: "/library/parts/7".into(), vcodec: "h264".into(), acodec: "ac3".into(), title: "Film".into(), - ctx: String::new(), preview: false, version: None, + ctx: String::new(), preview: false, version: None, show_rk: String::new(), }); assert_eq!(menu_enhancement_availability(&session), EnhancementAvailability::Offered(EnhancementRoute::Remux), @@ -970,7 +976,7 @@ mod backend_offer_session_tests { session.request = Some(PlaybackRequest { sid: ServerId::UNSET, rk: "42".into(), part: "/library/parts/7".into(), vcodec: "h264".into(), acodec: "ac3".into(), title: "Film".into(), - ctx: String::new(), preview: false, version: None, + ctx: String::new(), preview: false, version: None, show_rk: String::new(), }); let mut meta = plx_data::stores::metadata::MetadataStore::default(); let change = ViewerReconfigure::Speed(1500); @@ -8545,6 +8551,16 @@ impl TimelineStopFence { } } + /// [`Self::wait`] for at most `bound`; whether every stop up to `required` had completed by then. + fn wait_for(&self, required: u64, bound: std::time::Duration) -> bool { + let state = self.state.lock().unwrap_or_else(|e| e.into_inner()); + let (state, _) = self + .changed + .wait_timeout_while(state, bound, |s| s.completed < required) + .unwrap_or_else(|e| e.into_inner()); + state.completed >= required + } + fn complete(&self, generation: u64) { let mut state = self.state.lock().unwrap_or_else(|e| e.into_inner()); if generation > state.completed { @@ -8611,6 +8627,15 @@ pub fn timeline_stop_landed() -> bool { TIMELINE_STOP_FENCE.landed() } +/// Wait, for at most `bound`, until every `state=stopped` report announced up to generation +/// `required` (a value of [`timeline_stops_announced`]) has been attempted or abandoned, and say +/// whether it had. For a read that must see what that stop reported (the queue window's per-episode +/// resume points, [`queue_navigation::QueueContinuation::refreshed`]); a bound, because it runs on +/// the start's critical path and a slow server must not hold the start hostage. +pub(super) fn wait_for_timeline_stop(required: u64, bound: std::time::Duration) -> bool { + TIMELINE_STOP_FENCE.wait_for(required, bound) +} + /// How many `state=stopped` reports have ever been announced. A caller that remembers the value it /// last saw can tell whether an exit announced one of its own (the count moved) or ended a play /// that never started and reported nothing. @@ -9831,6 +9856,7 @@ pub fn with_queue(ps: &PlaybackSession, f: impl FnOnce(&[plx_plex::plex::Queu pub(super) fn resolve_playqueue( c: &plx_plex::plex::Client, rk: &str, + show_rk: &str, session: &str, cached: &str, continuous: bool, @@ -9854,7 +9880,7 @@ pub(super) fn resolve_playqueue( crate::player::log("playqueue: no machineIdentifier (skip)"); return QueueInfo::default(); } - match c.create_play_queue(effective, rk, session, continuous) { + match c.create_play_queue(effective, rk, show_rk, session, continuous) { Some(q) => { let up_next = q.next.as_ref().and_then(up_next_of); crate::player::log(&format!( @@ -9947,6 +9973,7 @@ impl ResolveEnv { resume_ns: ps.requested_resume_ns, src_kbps: resolve_src_kbps(meta.current(), sid, rk), omit_queue_continuous: false, + queue_show_rk: String::new(), queue_continuation: None, preview: false, audio_enhancements: crate::player::audio_enhancements(), @@ -10220,6 +10247,7 @@ pub fn request_play( title: &str, ctx: &str, version: Option, + show_rk: &str, ) -> bool { request_play_inner( ps, @@ -10235,6 +10263,7 @@ pub fn request_play( ctx: ctx.to_owned(), preview: false, version, + show_rk: show_rk.to_owned(), }, None, None, @@ -10262,6 +10291,7 @@ pub fn request_play_queue_item( request_play_with_queue(ps, meta, caps, PlaybackRequest { sid: row.sid, rk: row.rk.clone(), part: row.part.clone(), vcodec: row.vcodec.clone(), acodec: row.acodec.clone(), title: title.clone(), ctx: ctx.into(), preview: false, version: None, + show_rk: row.show_rk.clone(), }, None, None, false, crate::player::report::AttemptOrigin::Fresh, queue, false, None) } @@ -10292,6 +10322,7 @@ pub fn request_preview( ctx: plx_data::metadata::TRAILER_CONTEXT.to_owned(), preview: true, version: None, + show_rk: String::new(), }, None, None, @@ -10316,6 +10347,7 @@ fn request_env( ) -> ResolveEnv { let mut env = ResolveEnv::snapshot(ps, meta, request.sid, &request.rk); env.omit_queue_continuous = plx_data::metadata::context_omits_queue_continuous(&request.ctx); + env.queue_show_rk = request.show_rk.clone(); env.set_preview(request.preview); env.fresh_play = retry.is_none() && !request.preview; env.version = request.version; @@ -10975,9 +11007,16 @@ pub fn request_play_movie( &m.title, ctx, version, + episode_show_rk(m), ) } +/// The show a played row belongs to, for the queue built from it: only an EPISODE has one +/// (`PmsMovie::show_rk` also names a season's parent show, and a movie has none). +fn episode_show_rk(m: &PmsMovie) -> &str { + if m.kind == plx_data::pms::KIND_EPISODE { &m.show_rk } else { "" } +} + /// The server an item's ids belong to: its own when it has one, else the browsed surface. /// /// A row's `sid` is `UNSET` only before any server was registered (and in host tests, which build @@ -13994,6 +14033,7 @@ pub(crate) fn resolve_refused_preview_for_test(ps: &mut PlaybackSession) { ctx: plx_data::metadata::TRAILER_CONTEXT.into(), preview: true, version: None, + show_rk: String::new(), }); test_support::apply_plan(ps, plan, "trailer"); crate::route::resolve_as_preview_for_test(ps); @@ -14080,6 +14120,10 @@ mod memory_tests; #[path = "decision_queue_beside_tests.rs"] mod queue_beside_tests; +#[cfg(test)] +#[path = "decision_queue_predecessor_tests.rs"] +mod queue_predecessor_tests; + #[cfg(test)] #[path = "decision_track_memory_tests.rs"] mod track_memory_tests; diff --git a/rust-modules/media/src/route/decision_audio_enhancement_tests.rs b/rust-modules/media/src/route/decision_audio_enhancement_tests.rs index 7afc28c6c7..590523160f 100644 --- a/rust-modules/media/src/route/decision_audio_enhancement_tests.rs +++ b/rust-modules/media/src/route/decision_audio_enhancement_tests.rs @@ -4158,6 +4158,7 @@ fn admit_side_reader(ps: &mut PlaybackSession) { ctx: String::new(), preview: false, version: None, + show_rk: String::new(), }); } diff --git a/rust-modules/media/src/route/decision_plan_tests.rs b/rust-modules/media/src/route/decision_plan_tests.rs index 39610ae48d..de5db9b3aa 100644 --- a/rust-modules/media/src/route/decision_plan_tests.rs +++ b/rust-modules/media/src/route/decision_plan_tests.rs @@ -137,7 +137,7 @@ fn a_new_request_ends_the_standing_error_of_a_released_engine() { assert_eq!(unpresented_resume_ns(&ps), 60_000_000_000, "and a retry resumes from the playhead"); assert!(request_play( - &mut ps, &mut meta, plx_platform::devcaps::caps(), ServerId::from_raw(0), "film", "/library/parts/2/file.mkv", "h264", "aac", "Film", "", None, + &mut ps, &mut meta, plx_platform::devcaps::caps(), ServerId::from_raw(0), "film", "/library/parts/2/file.mkv", "h264", "aac", "Film", "", None, "", )); assert!(engine_failure(&ps).is_none(), "the new attempt ends the standing error"); assert_ne!(crate::player::state(&ps), crate::player::PlaybackState::Error); diff --git a/rust-modules/media/src/route/decision_queue_predecessor_tests.rs b/rust-modules/media/src/route/decision_queue_predecessor_tests.rs new file mode 100644 index 0000000000..5c885e707f --- /dev/null +++ b/rust-modules/media/src/route/decision_queue_predecessor_tests.rs @@ -0,0 +1,327 @@ +//! **A fresh play of an episode lands a queue that has the episodes BEFORE it**, so Previous has +//! somewhere to go (`queue_neighbors`, the HUD's Previous disc). +//! +//! The loopback PMS here answers `POST /playQueues` the way the two documented request shapes +//! differ, and the test grades what the app asked for, not a canned body: +//! +//! * an EPISODE uri with `continuous=1` (what the app used to send for every play) returns that +//! episode and the ones AFTER it, `playQueueSelectedItemOffset: 0` (docs/pms-api.md, "Up Next": +//! the queue is the show's remaining episodes); +//! * the SHOW uri with `key=/library/metadata/{episode}` returns the show's episodes in order with +//! the keyed one selected (docs/plex-openapi.json, `POST /playQueues`: "`key`: the key of the +//! first item to play"; python-plexapi's `PlayQueue.create(show, startItem=episode)` sends the +//! same two params). +//! +//! A server that ignores `key` and answers the show's own row, or the wrong selection, must never +//! cost the viewer their Up Next: the play falls back to the episode request. +use super::test_support::*; +use super::*; +use std::sync::atomic::{AtomicBool, Ordering}; +use std::sync::{Arc, Mutex}; +use std::time::Duration; + +const SHOW: &str = "50"; +/// The season's episodes: (ratingKey, playQueueItemID in the queue that holds them all). +const SEASON: [(&str, i64); 4] = [("101", 7001), ("102", 7002), ("103", 7003), ("104", 7004)]; + +fn episode_row(rk: &str, item_id: i64, index: usize) -> String { + format!( + r#"{{"ratingKey":"{rk}","playQueueItemID":{item_id},"type":"episode","title":"E{index}", + "grandparentRatingKey":"{SHOW}","grandparentTitle":"Show","parentIndex":1,"index":{index}, + "duration":1200000,"Media":[{{"videoCodec":"h264","audioCodec":"aac", + "Part":[{{"key":"/library/parts/{rk}/file.mkv"}}]}}]}}"# + ) +} + +/// A `/playQueues` answer holding `episodes` of the season (their positions in it), `selected` of +/// them being the one the server selected. +fn queue_answer(from: usize, selected: &str) -> Vec { + let rows: Vec = SEASON[from..] + .iter() + .enumerate() + .map(|(n, (rk, id))| episode_row(rk, *id, from + n + 1)) + .collect(); + let at = SEASON[from..].iter().position(|(rk, _)| *rk == selected).unwrap_or(0); + format!( + r#"{{"MediaContainer":{{"size":{},"playQueueID":88,"playQueueSelectedItemID":{}, + "playQueueSelectedItemOffset":{at},"playQueueTotalCount":{},"Metadata":[{}]}}}}"#, + rows.len(), + SEASON[from + at].1, + rows.len(), + rows.join(","), + ) + .into_bytes() +} + +/// How the loopback server treats the SHOW-based request. +#[derive(Clone, Copy, PartialEq)] +enum ShowQueue { + /// Honours `key`: the whole season, the keyed episode selected. + Honoured, + /// Ignores `key` and answers a queue that starts at the first episode, selected on it. + KeyIgnored, +} + +/// What the server answers to one request line. +fn answer(line: &str, show_queue: ShowQueue) -> Vec { + if line.contains("/decision?") { + return MDE_DIRECTPLAY.to_vec(); + } + if !line.starts_with("POST /playQueues") { + return EMPTY_MC.to_vec(); + } + let uri = query_param(line, "uri").map(|u| u.replace("%2F", "/").replace("%3A", ":")).unwrap_or_default(); + let ask = uri.rsplit('/').next().unwrap_or_default().to_owned(); + if ask == SHOW { + let key = query_param(line, "key").map(|k| k.replace("%2F", "/")).unwrap_or_default(); + let wanted = key.rsplit('/').next().unwrap_or_default(); + return match show_queue { + ShowQueue::Honoured => queue_answer(0, wanted), + ShowQueue::KeyIgnored => queue_answer(0, SEASON[0].0), + }; + } + // An episode uri: the episode and the ones after it, selected on the first. + let from = SEASON.iter().position(|(rk, _)| *rk == ask).unwrap_or(0); + queue_answer(from, &ask) +} + +struct Pms { + port: i32, + requests: Arc>>, + stop: Arc, + worker: Option>, +} + +impl Pms { + fn serving(show_queue: ShowQueue) -> Pms { + let listener = std::net::TcpListener::bind("127.0.0.1:0").expect("bind"); + listener.set_nonblocking(true).expect("nonblocking"); + let port = listener.local_addr().unwrap().port() as i32; + let requests = Arc::new(Mutex::new(Vec::::new())); + let stop = Arc::new(AtomicBool::new(false)); + let (log, stopping) = (Arc::clone(&requests), Arc::clone(&stop)); + let worker = std::thread::spawn(move || { + while !stopping.load(Ordering::Acquire) { + match plx_base::testnet::accept(&listener) { + Ok((mut socket, _)) => { + socket.set_nonblocking(false).expect("blocking"); + let line = drain_http(&mut socket); + log.lock().unwrap().push(line.clone()); + write_json(&mut socket, &answer(&line, show_queue)); + } + Err(e) if e.kind() == std::io::ErrorKind::WouldBlock => { + std::thread::sleep(Duration::from_millis(2)) + } + Err(e) => panic!("accept: {e}"), + } + } + }); + Pms { port, requests, stop, worker: Some(worker) } + } + + fn queue_posts(&self) -> Vec { + self.requests.lock().unwrap().iter().filter(|l| l.starts_with("POST /playQueues")).cloned().collect() + } + + /// How many window reads (`GET /playQueues/{id}`) the server has been asked for. + fn window_reads(&self) -> usize { + self.requests.lock().unwrap().iter().filter(|l| l.starts_with("GET /playQueues/")).count() + } +} + +impl Drop for Pms { + fn drop(&mut self) { + self.stop.store(true, Ordering::Release); + if let Some(worker) = self.worker.take() { + let _ = worker.join(); + } + plx_plex::plex::reset_servers_for_test(); + } +} + +/// The process-global registry guard, the session and the loopback server of one test. Field order +/// is drop order: the server resets the registry, so the guard that licenses that write outlives it. +struct Fx { + pms: Pms, + ps: PlaybackSession, + _guard: plx_base::testlock::Serial, +} + +impl Fx { + fn new(show_queue: ShowQueue) -> Fx { + let mut ps = PlaybackSession::IDLE; + let guard = fresh_registry(&mut ps); + Fx { pms: Pms::serving(show_queue), ps, _guard: guard } + } + + /// A fresh play of episode `rk` of show `show_rk` (`""`: no show), resolved against the server: + /// the plan the start would land. + fn play(&self, rk: &str, show_rk: &str) -> Plan { + assert!(plx_net::net::global_init()); + let sid = plx_plex::plex::register_for_test("predecessor-queue", "127.0.0.1", self.pms.port, "token", "queue-client"); + let mut env = ResolveEnv::snapshot(&self.ps, plx_data::stores::metadata::MetadataStore::default().view(), sid, rk); + env.cached_item = Some(fourk_item(sid, vec![eac3_track()])); + env.queue_show_rk = show_rk.into(); + build_stream( + &plx_base::task::OffFrame::for_test(), + rk, + "/library/parts/36013/1/file.mkv", + "hevc", + "eac3", + &env, + ) + } +} + +/// The ratingKeys of a landed queue, and where the playing episode sits in it. +fn shape(plan: &Plan, playing: &str) -> (Vec, Option) { + let rks = plan.queue.iter().map(|r| r.rk.clone()).collect(); + let at = plx_plex::plex::queue_index_of(&plan.queue, plan.pq_item_id.parse().unwrap_or(0), plan.sid, playing); + (rks, at) +} + +/// **The regression.** Playing episode 102 fresh must land a queue with 101 before it: the HUD's +/// Previous is `index > 0` of exactly these rows (`queue_neighbors`). Before the fix the play +/// asked for the episode alone and the landed queue began at the playing row, so Previous was +/// enabled only as a restart of the current episode. +#[test] +#[cfg(feature = "devtriggers")] +fn a_fresh_play_of_a_middle_episode_lands_the_episodes_before_it() { + let fx = Fx::new(ShowQueue::Honoured); + let plan = fx.play("102", SHOW); + let (rks, at) = shape(&plan, "102"); + assert_eq!(rks, ["101", "102", "103", "104"], "{:?}", fx.pms.queue_posts()); + assert_eq!(at, Some(1), "the playing row, with a predecessor"); + assert_eq!(plan.pq_id, "88"); + assert_eq!(plan.pq_item_id, "7002", "the timeline reports the keyed row's own queue item id"); + assert_eq!(plan.up_next.as_ref().map(|u| u.rk.as_str()), Some("103"), "Up Next is the row after it"); + let posts = fx.pms.queue_posts(); + assert_eq!(posts.len(), 1, "one queue was created: {posts:?}"); + let decoded = |name: &str| query_param(&posts[0], name).map(|v| v.replace("%2F", "/").replace("%3A", ":")); + assert!(decoded("uri").is_some_and(|u| u.ends_with("/library/metadata/50")), "the SHOW is the uri: {posts:?}"); + assert_eq!(decoded("key").as_deref(), Some("/library/metadata/102"), "the episode is the key: {posts:?}"); + assert!(!posts[0].contains("continuous"), "the queue already spans the show, before and after: {posts:?}"); +} + +/// **A server that ignores `key` costs the viewer nothing.** The show-based answer selects the +/// wrong row, so it is not believed: the play asks for the episode alone, as it always did, and its +/// timeline ids, queue and Up Next are the ones that request returns. +#[test] +#[cfg(feature = "devtriggers")] +fn a_server_that_ignores_the_key_falls_back_to_the_episode_request() { + let fx = Fx::new(ShowQueue::KeyIgnored); + let plan = fx.play("102", SHOW); + let (rks, at) = shape(&plan, "102"); + assert_eq!(rks, ["102", "103", "104"], "{:?}", fx.pms.queue_posts()); + assert_eq!(at, Some(0), "no predecessor, but a coherent queue"); + assert_eq!(plan.pq_item_id, "7002"); + assert_eq!(plan.up_next.as_ref().map(|u| u.rk.as_str()), Some("103")); + let posts = fx.pms.queue_posts(); + assert_eq!(posts.len(), 2, "the show-based ask, then the episode ask: {posts:?}"); + assert!(posts[0].contains("key="), "{posts:?}"); + assert!(!posts[1].contains("key=") && posts[1].contains("continuous=1"), "{posts:?}"); +} + +/// **A play that names no show asks for the episode alone**, exactly the request every play sent +/// before: a movie, a trailer and an extra have no show, and an extra's `continuous` stays off. +#[test] +#[cfg(feature = "devtriggers")] +fn a_play_with_no_show_asks_for_the_item_alone() { + let fx = Fx::new(ShowQueue::Honoured); + let plan = fx.play("102", ""); + let (rks, at) = shape(&plan, "102"); + assert_eq!(rks, ["102", "103", "104"], "{:?}", fx.pms.queue_posts()); + assert_eq!(at, Some(0)); + let posts = fx.pms.queue_posts(); + assert_eq!(posts.len(), 1, "{posts:?}"); + assert!(!posts[0].contains("key="), "{posts:?}"); + assert!(posts[0].contains("continuous=1"), "{posts:?}"); +} + +/// **The first episode of the show has nothing before it**, so its Previous stays disabled: the +/// show-keyed queue starts on it. +#[test] +#[cfg(feature = "devtriggers")] +fn the_first_episode_of_the_show_lands_no_predecessor() { + let fx = Fx::new(ShowQueue::Honoured); + let plan = fx.play("101", SHOW); + let (rks, at) = shape(&plan, "101"); + assert_eq!(rks, ["101", "102", "103", "104"], "{:?}", fx.pms.queue_posts()); + assert_eq!(at, Some(0), "the first row: queue_neighbors reports no previous"); +} + +/// Four queue rows of the season (item ids 7001..=7004) on `sid`. +fn season_rows(sid: plx_plex::plex::ServerId) -> Vec { + SEASON + .iter() + .map(|(rk, id)| plx_plex::plex::QueueRow { item_id: *id, sid, rk: (*rk).to_owned(), ..Default::default() }) + .collect() +} + +/// **A handoff's continuation is captured AFTER the outgoing stop is announced, and carries that +/// count**: the handoff stops the episode it leaves before it asks for the next one, so the window +/// re-read ([`the_window_reread_waits_for_the_outgoing_stop_within_its_bound`]) waits for it. A +/// `capture` that recorded no stop (`0`, what a continuation built from rows alone holds) would +/// read the rows while that report is still on its way, and Previous-then-Next would resume the +/// episode just left from the position it had before the viewer left it. +#[test] +#[cfg(feature = "devtriggers")] +fn a_captured_continuation_waits_for_the_stop_announced_before_it() { + let mut ps = PlaybackSession::IDLE; + let _guard = fresh_registry(&mut ps); + let rows = season_rows(plx_plex::plex::ServerId::from_raw(1)); + ps.queue = rows.clone(); + ps.pq_id = "88".into(); + let outgoing = TIMELINE_STOP_FENCE.announce(); + let announced = TIMELINE_STOP_FENCE.announced(); + let continuation = crate::route::QueueContinuation::capture(&ps, &rows[0]).expect("the row is in the queue"); + outgoing.finish(); + assert!( + continuation.stop() >= announced && announced > 0, + "captured after the announce, it waits for it: {} < {announced}", + continuation.stop() + ); +} + +/// **A handoff re-reads the queue window only after the episode it leaves has reported where it +/// was**, and never waits longer than its bound. The rows carry each episode's `viewOffset`: read +/// before the outgoing `stopped` report lands, the episode just left would resume from the position +/// it had before the viewer left it when stepped back to (Previous, then Next). +#[test] +#[cfg(feature = "devtriggers")] +fn the_window_reread_waits_for_the_outgoing_stop_within_its_bound() { + let fx = Fx::new(ShowQueue::Honoured); + assert!(plx_net::net::global_init()); + let sid = plx_plex::plex::register_for_test("predecessor-queue", "127.0.0.1", fx.pms.port, "token", "queue-client"); + let client = plx_plex::plex::client_for(sid).expect("the registered server"); + let rows = season_rows(sid); + let continuation = crate::route::QueueContinuation::from_queue("88".into(), rows.clone(), &rows[1]) + .expect("the selected row is in the queue"); + + // The outgoing episode's stop is announced and still on its way. + let stop = TIMELINE_STOP_FENCE.announce(); + let waiting = continuation.clone().awaiting(TIMELINE_STOP_FENCE.announced()); + let read = std::thread::scope(|scope| { + let reader = scope.spawn(|| { + let started = std::time::Instant::now(); + let _ = waiting.refreshed_within(client, Duration::from_secs(20)); + started.elapsed() + }); + std::thread::sleep(Duration::from_millis(250)); + assert_eq!(fx.pms.window_reads(), 0, "the window was read while the outgoing stop was in flight"); + stop.finish(); + reader.join().expect("the reader") + }); + assert!(read >= Duration::from_millis(250), "it waited for the stop: {read:?}"); + assert_eq!(fx.pms.window_reads(), 1, "and read the window once it landed"); + + // A stop that never lands costs the start the bound and nothing more. + let stuck = TIMELINE_STOP_FENCE.announce(); + let waiting = continuation.awaiting(TIMELINE_STOP_FENCE.announced()); + let started = std::time::Instant::now(); + let _ = waiting.refreshed_within(client, Duration::from_millis(120)); + let waited = started.elapsed(); + drop(stuck); + assert!(waited >= Duration::from_millis(120) && waited < Duration::from_secs(10), "{waited:?}"); + assert_eq!(fx.pms.window_reads(), 2, "the window is still read once the bound passed"); +} diff --git a/rust-modules/media/src/route/decision_resolve_route_tests.rs b/rust-modules/media/src/route/decision_resolve_route_tests.rs index 565d829399..b6389ae36e 100644 --- a/rust-modules/media/src/route/decision_resolve_route_tests.rs +++ b/rust-modules/media/src/route/decision_resolve_route_tests.rs @@ -1561,6 +1561,7 @@ fn a_refused_retry_keeps_its_position_and_full_request_for_the_next_quality() { ctx: "S01 E02".into(), preview: false, version: None, + show_rk: String::new(), }; { let s = &mut ps; { s.request = Some(request.clone()); diff --git a/rust-modules/media/src/route/decision_timeline_tests.rs b/rust-modules/media/src/route/decision_timeline_tests.rs index f1f6ef538f..90c5bfbdd8 100644 --- a/rust-modules/media/src/route/decision_timeline_tests.rs +++ b/rust-modules/media/src/route/decision_timeline_tests.rs @@ -456,6 +456,7 @@ fn a_session_resolved_for_a_preview_never_reports_a_timeline() { ctx: plx_data::metadata::TRAILER_CONTEXT.into(), preview: true, version: None, + show_rk: String::new(), }); apply_plan( &mut ps, @@ -494,7 +495,7 @@ fn a_film_request_retires_the_previous_trailers_preview_mark() { assert!(preview_request(&ps), "the trailer's session is marked"); assert!(request_play( - &mut ps, &mut meta, plx_platform::devcaps::caps(), ServerId::from_raw(0), "film", "/library/parts/2/file.mkv", "h264", "aac", "Film", "", None, + &mut ps, &mut meta, plx_platform::devcaps::caps(), ServerId::from_raw(0), "film", "/library/parts/2/file.mkv", "h264", "aac", "Film", "", None, "", )); assert!(!preview_request(&ps), "a film request is not a preview, whatever the last landing was"); @@ -716,7 +717,7 @@ fn a_playback_request_clears_a_finished_flag_no_stop_took() { ps.finished = true; assert!(request_play( - &mut ps, &mut meta, plx_platform::devcaps::caps(), ServerId::from_raw(0), "film", "/library/parts/2/file.mkv", "h264", "aac", "Film", "", None, + &mut ps, &mut meta, plx_platform::devcaps::caps(), ServerId::from_raw(0), "film", "/library/parts/2/file.mkv", "h264", "aac", "Film", "", None, "", )); assert!(!ps.finished, "the request retired the flag"); diff --git a/rust-modules/media/src/route/decision_track_memory_tests.rs b/rust-modules/media/src/route/decision_track_memory_tests.rs index 2954de83e3..592f18df94 100644 --- a/rust-modules/media/src/route/decision_track_memory_tests.rs +++ b/rust-modules/media/src/route/decision_track_memory_tests.rs @@ -949,6 +949,7 @@ fn request(sid: ServerId, preview: bool) -> PlaybackRequest { ctx: String::new(), preview, version: None, + show_rk: String::new(), } } diff --git a/rust-modules/media/src/route/decision_version_tests.rs b/rust-modules/media/src/route/decision_version_tests.rs index 0ae7278598..7c0cdb44fd 100644 --- a/rust-modules/media/src/route/decision_version_tests.rs +++ b/rust-modules/media/src/route/decision_version_tests.rs @@ -722,7 +722,7 @@ fn a_qualified_dts_original_lands_from_the_real_resolve_worker_before_feed_decis let sid = plx_plex::plex::register_for_test("qualified-dts", "127.0.0.1", port, "token", "qualified-dts-client"); let mut meta = plx_data::stores::metadata::MetadataStore::default(); assert!(request_play(&mut ps, &mut meta, plx_platform::devcaps::caps(), sid, - "rk-v", "/library/parts/3134/1/file.mkv", "h264", "dca", "Film", "", None)); + "rk-v", "/library/parts/3134/1/file.mkv", "h264", "dca", "Film", "", None, "")); let deadline = std::time::Instant::now() + std::time::Duration::from_secs(15); let start = loop { if let Some(start) = pump_play(&mut ps, &mut meta) { break start; } @@ -768,7 +768,7 @@ fn a_certified_av1_uri_original_does_not_inherit_feeds_video_refusal() { av1: false, ..plx_platform::devcaps::Caps::assumed() }); assert!(request_play(&mut ps, &mut meta, feed_cannot_decode_av1, sid, - "rk-v", "/library/parts/3134/1/file.mkv", "av1", "aac", "Film", "", None)); + "rk-v", "/library/parts/3134/1/file.mkv", "av1", "aac", "Film", "", None, "")); let finished = wait_resolve_workers_for_test(std::time::Duration::from_secs(15)); let start = pump_play(&mut ps, &mut meta); done.send(()).unwrap(); @@ -827,7 +827,7 @@ fn an_accepted_pms_decision_produces_an_owned_uri_offer_from_its_output() { "token", "qualified-pms-client"); let mut meta = plx_data::stores::metadata::MetadataStore::default(); assert!(request_play(&mut ps, &mut meta, plx_platform::devcaps::caps(), sid, - "rk-v", "/library/parts/3134/1/file.mkv", "h264", "dca", "Film", "", None)); + "rk-v", "/library/parts/3134/1/file.mkv", "h264", "dca", "Film", "", None, "")); assert!(wait_resolve_workers_for_test(std::time::Duration::from_secs(15))); let start = pump_play(&mut ps, &mut meta).expect("accepted PMS output start"); assert!(start.uri_offer.is_some(), "the actual resolve must carry its owned URI offer"); @@ -904,7 +904,7 @@ fn viewer_off_re_resolves_a_burned_pms_uri_without_reusing_its_subtitle() { av1: false, ..plx_platform::devcaps::Caps::assumed() }); assert!(request_play(&mut ps, &mut meta, feed_cannot_decode_av1, sid, - "rk-v", "/library/parts/3134/1/file.mkv", "av1", "dca", "Film", "", None)); + "rk-v", "/library/parts/3134/1/file.mkv", "av1", "dca", "Film", "", None, "")); assert!(wait_resolve_workers_for_test(std::time::Duration::from_secs(15))); let first = pump_play(&mut ps, &mut meta).expect("accepted burned PMS output"); let burned = first.uri_offer.expect("accepted server burn is a qualified URI output"); @@ -966,7 +966,7 @@ fn native_hls_master_offer_requires_the_fetched_multi_variant_response() { "token", "native-master-client"); let mut meta = plx_data::stores::metadata::MetadataStore::default(); assert!(request_play(&mut ps, &mut meta, plx_platform::devcaps::caps(), sid, - "rk-v", "/library/parts/3134/1/file.mkv", "h264", "dca", "Film", "", None)); + "rk-v", "/library/parts/3134/1/file.mkv", "h264", "dca", "Film", "", None, "")); assert!(wait_resolve_workers_for_test(std::time::Duration::from_secs(15))); let start = pump_play(&mut ps, &mut meta); assert_eq!(start.as_ref().is_some_and(|landed| landed.uri_offer.is_some()), qualifies, @@ -1006,7 +1006,7 @@ fn unknown_uri_qualification_keeps_the_existing_feed_decision_path() { let sid = plx_plex::plex::register_for_test("unknown-dts", "127.0.0.1", port, "token", "unknown-dts-client"); let mut meta = plx_data::stores::metadata::MetadataStore::default(); assert!(request_play(&mut ps, &mut meta, plx_platform::devcaps::caps(), sid, - "rk-v", "/library/parts/3134/1/file.mkv", "h264", "dca", "Film", "", None)); + "rk-v", "/library/parts/3134/1/file.mkv", "h264", "dca", "Film", "", None, "")); let finished = wait_resolve_workers_for_test(std::time::Duration::from_secs(15)); let start = pump_play(&mut ps, &mut meta); done.send(()).unwrap(); @@ -1048,7 +1048,7 @@ fn expired_original_skips_its_spent_recipe_but_qualifies_unspent_pms_uri(backend "token", "expiring-dts-client"); let mut meta = plx_data::stores::metadata::MetadataStore::default(); assert!(request_play(&mut ps, &mut meta, plx_platform::devcaps::caps(), sid, - "rk-v", "/library/parts/3134/1/file.mkv", "h264", "dca", "Film", "", None)); + "rk-v", "/library/parts/3134/1/file.mkv", "h264", "dca", "Film", "", None, "")); // This is an existing episode, not a fresh catalog. The both-backend variant has // already crossed from Feed to URI; both variants have spent URI Original and learned @@ -1193,7 +1193,7 @@ fn unknown_pms_output_in_uri_only_retires_the_accepted_encoder_once() { "token", "unknown-pms-client"); let mut meta = plx_data::stores::metadata::MetadataStore::default(); assert!(request_play(&mut ps, &mut meta, plx_platform::devcaps::caps(), sid, - "rk-v", "/library/parts/3134/1/file.mkv", "h264", "dca", "Film", "", None)); + "rk-v", "/library/parts/3134/1/file.mkv", "h264", "dca", "Film", "", None, "")); assert!(wait_resolve_workers_for_test(std::time::Duration::from_secs(15))); assert!(pump_play(&mut ps, &mut meta).is_none()); assert_eq!(ps.play_verdict, Some(PlayVerdict::NoQualifiedBackend)); @@ -1227,7 +1227,7 @@ fn the_picked_version_rides_the_request_and_the_landing() { let _g = fresh_registry(&mut ps); let mut meta = plx_data::stores::metadata::MetadataStore::default(); assert!(request_play(&mut ps, &mut meta, plx_platform::devcaps::caps(), ServerId::from_raw(0), "rk-v", "/library/parts/3134/1/file.mkv", - "h264", "aac", "Episode", "", Some(plx_plex::plex::MediaId(3082)))); + "h264", "aac", "Episode", "", Some(plx_plex::plex::MediaId(3082)), "")); assert_eq!(ps.request.as_ref().and_then(|r| r.version), Some(plx_plex::plex::MediaId(3082))); let env = request_env(&ps, meta.view(), ps.request.as_ref().unwrap(), None); assert_eq!(env.version, Some(plx_plex::plex::MediaId(3082))); diff --git a/rust-modules/media/src/route/mod.rs b/rust-modules/media/src/route/mod.rs index f1d049f668..25116c6c3e 100644 --- a/rust-modules/media/src/route/mod.rs +++ b/rust-modules/media/src/route/mod.rs @@ -15,7 +15,7 @@ mod uri_offer; pub use uri_offer::{QualifiedUriOffer, UriRecoveryNotice, UriSourceRecipe}; mod decision; mod queue_navigation; -pub use queue_navigation::{QueueContinuation, QueueStep, previous_restarts, queue_neighbor, queue_neighbors}; +pub use queue_navigation::{QueueContinuation, QueueStep, queue_neighbor, queue_neighbors}; mod flight; mod plan; mod prefs; diff --git a/rust-modules/media/src/route/plan.rs b/rust-modules/media/src/route/plan.rs index a956fc1b1f..0d3208c130 100644 --- a/rust-modules/media/src/route/plan.rs +++ b/rust-modules/media/src/route/plan.rs @@ -2,7 +2,11 @@ //! the plain data types they need. Nothing here reads a `static mut`, touches [`super::decision`]'s //! `SESSION`/`PLAYER_CONTROL`/`PLAY_SLOT`/`ENCODER_CLEANUP`/`SCROBBLE_JOIN`/`TIMELINE_STOP_FENCE`/ //! `QUALITY`, or calls `task::spawn*` — that is what makes [`build_stream`] safe to run on the -//! resolve worker. Everything else in the former `route.rs` (session state, the synchronized +//! resolve worker. (One bounded wait is the exception that is not named here: a handoff's queue +//! re-read, [`build_stream`]'s `queue.refreshed` on its `beside` leg, waits up to 1.5 s through +//! `queue_navigation` for the outgoing episode's `stopped` report, the shared stop fence, before +//! it reads the rows; it never runs on the main thread.) Everything else in the former +//! `route.rs` (session state, the synchronized //! `PlayerControl`, PMS/native I/O, the encoder/scrobble/timeline machinery) lives in //! [`super::decision`]. `ci/check-deps.sh`'s `wall` gate holds this file, and its child //! module `plan_memory.rs` (the track memory's pure helpers), to zero @@ -1452,6 +1456,11 @@ pub struct ResolveEnv { pub src_kbps: i64, /// Trailer sessions omit `continuous=1` so EOS cannot Up-Next into a sibling extra. pub omit_queue_continuous: bool, + /// The show the played EPISODE belongs to, empty for anything else (and for a caller that did not + /// know it). The queue is built from the show with the episode as its start, so the episodes + /// BEFORE it are in the queue and Previous has somewhere to go + /// ([`plx_plex::plex::Client::create_play_queue`]). + pub queue_show_rk: String, /// An explicit queue-item handoff reuses the existing queue instead of POSTing a new tail. pub queue_continuation: Option, /// Hero preview. Skip the PlayQueue entirely, and refuse anything that is not a direct play. @@ -1819,7 +1828,7 @@ pub(super) fn build_stream(off: &plx_base::task::OffFrame, rk: &str, part: &str, Some(plx_base::task::beside(s, "route-queue", move || { env.queue_continuation.as_ref().filter(|queue| queue.matches(env.sid, rk)) .map(|queue| queue.refreshed(client).resolve()).unwrap_or_else(|| resolve_playqueue( - client, rk, &session, &env.machine_id, !env.omit_queue_continuous, + client, rk, &env.queue_show_rk, &session, &env.machine_id, !env.omit_queue_continuous, )) })) } else { diff --git a/rust-modules/media/src/route/plan_audio_enhancement_tests.rs b/rust-modules/media/src/route/plan_audio_enhancement_tests.rs index bd94c1d96c..f588f45319 100644 --- a/rust-modules/media/src/route/plan_audio_enhancement_tests.rs +++ b/rust-modules/media/src/route/plan_audio_enhancement_tests.rs @@ -602,6 +602,7 @@ fn cold_start_plan_lands_as_client_over_remux_with_a_reader_target() { ctx: String::new(), preview: false, version: None, + show_rk: String::new(), }); assert_eq!(subtitle_presenter(&ps), SubtitlePresenter::ClientOverRemux); assert!(!subtitles_burned(&ps)); diff --git a/rust-modules/media/src/route/queue_navigation.rs b/rust-modules/media/src/route/queue_navigation.rs index 3b6c96a1df..4153342d71 100644 --- a/rust-modules/media/src/route/queue_navigation.rs +++ b/rust-modules/media/src/route/queue_navigation.rs @@ -2,6 +2,11 @@ use super::{cur_rk, cur_sid, pq_id, pq_item_id, with_queue, PlaybackSession}; use plx_plex::plex::{queue_index_of, QueueRow, ServerId}; use std::sync::Arc; +use std::time::Duration; + +/// The longest the window re-read waits for the outgoing episode's `stopped` report to land +/// (see [`QueueContinuation::refreshed`]): a stop on a slow server must not hold a start hostage. +const STOP_WAIT: Duration = Duration::from_millis(1500); /// An owned continuation of the server's existing queue, captured before request_play retires /// the outgoing state. Shared across resolve snapshots without copying a show's rows per frame. @@ -10,6 +15,11 @@ pub struct QueueContinuation { id: String, rows: Arc>, selected: usize, + /// How many `state=stopped` reports had been announced when the continuation was captured, + /// which is after the outgoing episode's own (the handoff stops it before it requests the + /// next): the window re-read waits for these to land, so the rows it reads carry the position + /// that stop reported. `0` for a continuation built from rows alone. + stop: u64, } impl QueueContinuation { @@ -26,9 +36,21 @@ impl QueueContinuation { Some(selected) } - fn from_queue(id: String, rows: Vec, target: &QueueRow) -> Option { + pub(super) fn from_queue(id: String, rows: Vec, target: &QueueRow) -> Option { let selected = Self::select(&id, &rows, target)?; - Some(Self { id, rows: Arc::new(rows), selected }) + Some(Self { id, rows: Arc::new(rows), selected, stop: 0 }) + } + + /// This continuation, its window re-read set to wait for the stops announced up to `stop`. + #[cfg(test)] + pub(super) fn awaiting(self, stop: u64) -> Self { + Self { stop, ..self } + } + + /// The stop count the window re-read waits for. + #[cfg(test)] + pub(super) fn stop(&self) -> u64 { + self.stop } /// The live queue is lent, not owned, so its rows are copied only once `target` is found. @@ -36,7 +58,12 @@ impl QueueContinuation { with_queue(ps, |rows| { let id = pq_id(ps); let selected = Self::select(&id, rows, target)?; - Some(Self { id, rows: Arc::new(rows.to_vec()), selected }) + Some(Self { + id, + rows: Arc::new(rows.to_vec()), + selected, + stop: super::decision::timeline_stops_announced(), + }) }) } @@ -46,10 +73,24 @@ impl QueueContinuation { fn with_window(&self, rows: Vec) -> Self { Self::from_queue(self.id.clone(), rows, &self.rows[self.selected]) + .map(|window| Self { stop: self.stop, ..window }) .unwrap_or_else(|| self.clone()) } + /// This queue re-read around its selected item. + /// + /// **After the outgoing episode's `stopped` report has landed**: the rows carry each episode's + /// `viewOffset`, and a handoff (Previous, Next) re-reads them while that report is still on its + /// way, so a row read before it lands holds the position from before the viewer left. Stepping + /// back to the episode just left would then resume it from that stale point, and its next + /// report would write that over the real one. The wait is the shared stop fence + /// (`decision::TIMELINE_STOP_FENCE`), bounded by [`STOP_WAIT`]. pub(super) fn refreshed(&self, client: &plx_plex::plex::Client) -> Self { + self.refreshed_within(client, STOP_WAIT) + } + + pub(super) fn refreshed_within(&self, client: &plx_plex::plex::Client, bound: Duration) -> Self { + super::decision::wait_for_timeline_stop(self.stop, bound); client .play_queue_window(&self.id, self.rows[self.selected].item_id) .map(|rows| self.with_window(rows)) @@ -76,11 +117,6 @@ pub enum QueueStep { Next, } -/// At five seconds the Previous button becomes Restart. Before it, it asks for the prior item. -pub fn previous_restarts(position_ns: i64) -> bool { - position_ns >= 5_000_000_000 -} - /// The row now playing among `rows`, by the queue's identity rule (item id, rating key as the /// fallback). fn current_index(ps: &PlaybackSession, rows: &[QueueRow]) -> Option { @@ -120,14 +156,6 @@ mod tests { neighbor(rows, queue_index_of(rows, item_id, sid, rk)?, step) } - #[test] - fn queue_navigation_previous_has_a_five_second_boundary() { - assert!(!previous_restarts(0)); - assert!(!previous_restarts(4_999_999_999)); - assert!(previous_restarts(5_000_000_000)); - assert!(previous_restarts(90_000_000_000)); - } - #[test] fn queue_navigation_queue_handoff_retains_the_predecessor_and_original_queue_ids() { let sid = ServerId::from_raw(1); diff --git a/rust-modules/plex/src/plex/timeline.rs b/rust-modules/plex/src/plex/timeline.rs index 53ccc331ca..16e75bcb97 100644 --- a/rust-modules/plex/src/plex/timeline.rs +++ b/rust-modules/plex/src/plex/timeline.rs @@ -68,14 +68,26 @@ impl Client { /// POST /playQueues for one item. Best-effort: None on failure — the timeline still works, /// just without the queue ids (and the player without an Up Next). /// - /// `continuous=1` is what makes the response carry the show's remaining episodes after the - /// one being played, each as a FULL `Metadata` row (thumb, S/E, duration, viewOffset, and - /// `Media[0].Part[0].key` + codecs — everything `route::request_play` needs). The Up Next control - /// screen is therefore free: it reads [`PlayQueueResult::next`] from the queue this playback - /// already had to create, instead of asking the server what plays next. + /// **An episode's queue is the SHOW's, with the episode selected in it** (`show_rk` non-empty): + /// the request names the show as the `uri` and the episode as the `key` + /// ([`show_queue_query`]). The queue then holds the episodes BEFORE the one playing as well as + /// after it, which is what Previous needs: the episode-only request below answers the playing + /// episode and its successors and nothing before, so a play started fresh had no predecessor + /// row and the Previous disc could only ever be a restart of the current episode. This shape + /// is spec- and client-derived, not observed on a live server (`docs/pms-api.md`, "A show-keyed PlayQueue"), + /// so the answer is accepted only when its selected row IS the requested episode + /// ([`PlayQueueResult::selects`]); a server that ignored `key`, or refused, falls back to the + /// episode request, which is exactly what every play sent before. /// - /// Trailer extras omit `continuous`: a continuous extras queue can be sibling clips, and EOS - /// must not Up-Next into a featurette. `opt_int` drops the param when `continuous` is false. + /// The episode request's `continuous=1` is what makes the response carry the show's remaining + /// episodes after the one being played, each as a FULL `Metadata` row (thumb, S/E, duration, + /// viewOffset, and `Media[0].Part[0].key` + codecs — everything `route::request_play` needs). + /// The Up Next control screen is therefore free: it reads [`PlayQueueResult::next`] from the + /// queue this playback already had to create, instead of asking the server what plays next. + /// + /// Trailer extras omit `continuous` (and pass no `show_rk`): a continuous extras queue can be + /// sibling clips, and EOS must not Up-Next into a featurette. `opt_int` drops the param when + /// `continuous` is false. /// /// The WHOLE window is kept, as [`QueueRow`]s — the queue this round trip already paid for is /// the queue a list can draw and jump around in, and throwing it away meant re-asking the @@ -84,15 +96,23 @@ impl Client { &self, machine_id: &str, rating_key: &str, + show_rk: &str, session: &str, continuous: bool, ) -> Option { - let uri = format!( - "server://{machine_id}/com.plexapp.plugins.library/library/metadata/{rating_key}" - ); + if continuous && !show_rk.is_empty() { + let q = self.playback_identity(show_queue_query(machine_id, show_rk, rating_key, session)); + let asked = self + .post_json(&q.build()) + .map(|mc| PlayQueueResult::of(mc, self.id(), rating_key)) + .filter(|queue| queue.selects(rating_key, self.id())); + if let Some(queue) = asked { + return Some(self.complete_before(queue)); + } + } let q = QueryBuilder::new("/playQueues") .str("type", "video") - .str("uri", &uri) + .str("uri", &item_uri(machine_id, rating_key)) .opt_int("continuous", i64::from(continuous)) .int("shuffle", 0) .int("repeat", 0) @@ -104,6 +124,42 @@ impl Client { rating_key, )) } + + /// A queue the server says has rows before the selected one, but whose answer OPENS on the + /// selected row (a window that starts where playback does), is read again around its selected + /// item with the existing window read ([`Client::play_queue_window`]), so Previous is not lost + /// to the page the create answered with. A read that fails, or that does not hold the selected + /// row, leaves the answer as it was. + fn complete_before(&self, queue: PlayQueueResult) -> PlayQueueResult { + if !queue.opens_on_selected_with_rows_before() { + return queue; + } + match self.play_queue_window(&queue.id.to_string(), queue.selected_item_id) { + Some(rows) => queue.with_window(rows), + None => queue, + } + } +} + +/// The `uri` naming one library item on the server `machine_id`, the form `POST /playQueues` takes +/// (the simple `/library/metadata/{rk}` form answers HTTP 400, `docs/pms-api.md`, "PlayQueue for an +/// extra"). +fn item_uri(machine_id: &str, rating_key: &str) -> String { + format!("server://{machine_id}/com.plexapp.plugins.library/library/metadata/{rating_key}") +} + +/// The `POST /playQueues` that builds an episode's queue from its SHOW: the show is the `uri`, the +/// episode the `key` ("the key of the first item to play", `docs/plex-openapi.json`), no +/// `continuous` (the queue already spans every episode, before and after the keyed one). Its own +/// function so the wire shape is graded without a server. +fn show_queue_query(machine_id: &str, show_rk: &str, rating_key: &str, session: &str) -> QueryBuilder { + QueryBuilder::new("/playQueues") + .str("type", "video") + .str("uri", &item_uri(machine_id, show_rk)) + .str("key", &format!("/library/metadata/{rating_key}")) + .int("shuffle", 0) + .int("repeat", 0) + .str("X-Plex-Session-Identifier", session) } /// One retained row of the play queue: everything a queue list draws, plus everything @@ -316,6 +372,84 @@ mod tests { "trailer sessions must not send continuous=1" ); } + + /// **The wire shape of an episode's show-based queue**: the SHOW is the `uri`, the episode the + /// `key` (as a library path), and there is no `continuous` (the queue spans the show before and + /// after the keyed episode). The episode-only request keeps `continuous=1` and the episode as + /// its `uri`, so the two cannot be confused. + #[test] + fn an_episodes_queue_is_asked_of_its_show_with_the_episode_as_the_key() { + let show = show_queue_query("machine", "50", "102", "sess").build(); + assert!(show.starts_with("/playQueues?"), "{show}"); + assert!(show.contains("uri=server%3A%2F%2Fmachine%2Fcom.plexapp.plugins.library%2Flibrary%2Fmetadata%2F50"), "{show}"); + assert!(show.contains("key=%2Flibrary%2Fmetadata%2F102"), "{show}"); + assert!(!show.contains("continuous"), "{show}"); + let episode = QueryBuilder::new("/playQueues") + .str("uri", &item_uri("machine", "102")) + .opt_int("continuous", 1) + .build(); + assert!(episode.contains("metadata%2F102") && !episode.contains("key="), "{episode}"); + assert!(episode.contains("continuous=1"), "{episode}"); + } + + /// A queue body whose selected row (`playQueueSelectedItemID`) is `selected` of rows + /// `101..=104` (item ids 7001..=7004), `offset` rows before it in the whole queue, the window + /// starting at row `from`. + fn season(selected: i64, offset: i64, from: usize) -> PlayQueueResult { + let rows: Vec = (from..4) + .map(|n| format!(r#"{{"playQueueItemID":{},"ratingKey":"{}","type":"episode","index":{}}}"#, 7001 + n, 101 + n, n + 1)) + .collect(); + result( + &format!( + r#"{{"MediaContainer":{{"size":{},"playQueueID":88,"playQueueSelectedItemID":{selected}, + "playQueueSelectedItemOffset":{offset},"playQueueTotalCount":4,"Metadata":[{}]}}}}"#, + rows.len(), + rows.join(",") + ), + "102", + ) + } + + /// **A show-based queue is believed only when it selects the episode asked for.** A server that + /// ignored `key` answers the first row, a stale one some other episode: neither may stand for + /// the playing episode (its queue item id is what the timeline reports, and its successor is + /// Up Next). + #[test] + fn a_show_queue_is_accepted_only_when_it_selects_the_requested_episode() { + assert!(season(7002, 1, 0).selects("102", Q), "the keyed episode is the selected row"); + assert!(!season(7001, 0, 0).selects("102", Q), "the key was ignored: the first row is selected"); + assert!(!season(7003, 2, 0).selects("102", Q), "another row is selected"); + assert!(!season(9999, 0, 0).selects("102", Q), "a selected id no row has"); + assert!(!season(7002, 1, 0).selects("102", ServerId::from_raw(3)), "a queue of another server is not this episode's"); + } + + /// The window read is spent only when the create answered with a window that opens ON the + /// selected row while the server says rows precede it: the predecessors it lost. + #[test] + fn a_window_that_opens_on_the_selected_row_with_rows_before_it_is_read_again() { + assert!(season(7002, 1, 1).opens_on_selected_with_rows_before(), "opens on 102, one row before it"); + assert!(!season(7002, 1, 0).opens_on_selected_with_rows_before(), "the predecessor is already in the window"); + assert!(!season(7001, 0, 0).opens_on_selected_with_rows_before(), "nothing before the first row"); + assert!(!season(7002, 0, 1).opens_on_selected_with_rows_before(), "the server reports no rows before it"); + } + + /// The re-read window replaces the rows (and the successor), keeps the selection and the queue + /// id, and is refused when it does not hold the selected row. + #[test] + fn a_re_read_window_replaces_the_rows_and_never_the_selection() { + let opened = season(7002, 1, 1); + assert_eq!(rks(&opened.items), ["102", "103", "104"]); + let window: Vec = (0..4) + .map(|n| QueueRow { item_id: 7001 + n, sid: Q, rk: (101 + n).to_string(), ..Default::default() }) + .collect(); + let full = season(7002, 1, 1).with_window(window.clone()); + assert_eq!(rks(&full.items), ["101", "102", "103", "104"]); + assert_eq!((full.id, full.selected_item_id), (88, 7002)); + assert_eq!(full.next.as_ref().map(|n| n.rk.as_str()), Some("103")); + let without = season(7002, 1, 1).with_window(window[..1].to_vec()); + assert_eq!(rks(&without.items), ["102", "103", "104"], "a window that lost the selected row is ignored"); + } + /// A response body through the SHIPPED mapping — the same call `create_play_queue` makes once /// its POST returns, so these tests cannot pass on a projection the app does not use. fn result(body: &str, rating_key: &str) -> PlayQueueResult { @@ -567,9 +701,49 @@ pub struct PlayQueueResult { /// continuous queue is just itself — verified live: total count 1). A copy of the `items` row /// after the selected one; one lean row is worth not making every caller redo the lookup. pub next: Option, + /// `playQueueSelectedItemOffset`: how many rows the WHOLE queue holds before the selected one + /// (0 when the server omitted it). A window that opens on the selected row while this is above + /// zero has lost the predecessors the server says exist ([`Self::opens_on_selected_with_rows_before`]). + before: i64, } impl PlayQueueResult { + /// Whether the selected row of this answer IS `rating_key` on `sid`: the check a queue built + /// from the show with the episode as its `key` must pass before the app believes it. A server + /// that ignored the `key` answers a queue selected on some other row, whose item ids the + /// timeline would then report for the wrong episode and whose Up Next would be wrong. + pub(super) fn selects(&self, rating_key: &str, sid: ServerId) -> bool { + // By the queue item id alone: the rating-key fallback of `queue_index_of` would believe a + // `playQueueSelectedItemID` that names no row, and the timeline reports that id. + self.selected_item_id != 0 + && self.items.iter().any(|row| { + row.item_id == self.selected_item_id + && super::same_item((row.sid, &row.rk), (sid, rating_key)) + }) + } + + /// The server says rows precede the selected one, and the returned window starts ON it. + fn opens_on_selected_with_rows_before(&self) -> bool { + self.before > 0 + && self.selected_item_id != 0 + && self.items.first().is_some_and(|row| row.item_id == self.selected_item_id) + } + + /// This result over `rows`, a window read of the same queue around the selected row. Only a + /// window that holds the selected row replaces the answer: an expired or mismatched one cannot + /// discard the selection. + fn with_window(self, rows: Vec) -> PlayQueueResult { + let Some(selected) = self.items.iter().find(|row| row.item_id == self.selected_item_id) else { + return self; + }; + let (sid, rk) = (selected.sid, selected.rk.clone()); + if !rows.iter().any(|row| row.item_id == self.selected_item_id && row.rk == rk) { + return self; + } + let next = next_after(&rows, self.selected_item_id, sid, &rk).cloned(); + PlayQueueResult { items: rows, next, ..self } + } + /// The WHOLE response→result mapping, kept out of `create_play_queue` so the host tests grade /// the code that ships instead of a copy of it (only the request build is left up there). fn of( @@ -597,6 +771,7 @@ impl PlayQueueResult { remaining, items, next, + before: mc.play_queue_selected_item_offset.max(0), } } } diff --git a/rust-modules/screens/src/detail/extras.rs b/rust-modules/screens/src/detail/extras.rs index 16b0f5bf70..8f6a091d06 100644 --- a/rust-modules/screens/src/detail/extras.rs +++ b/rust-modules/screens/src/detail/extras.rs @@ -52,6 +52,7 @@ pub fn play(d: &Detail, key: u32) -> Option { title: extra.hud_title(d.title.as_str()).to_string(), context: extra_play_context(extra).into(), version: None, + show_rk: String::new(), }) } diff --git a/rust-modules/screens/src/detail/mod.rs b/rust-modules/screens/src/detail/mod.rs index 5e35c8ef5d..ba8fe40f75 100644 --- a/rust-modules/screens/src/detail/mod.rs +++ b/rust-modules/screens/src/detail/mod.rs @@ -4491,6 +4491,7 @@ impl DetailScreen { title: title.to_string(), context: plx_data::metadata::TRAILER_CONTEXT.into(), version: None, + show_rk: String::new(), }; self.content(fx, ContentReq::Play { play, resume_ns: 0 }); } @@ -4605,6 +4606,9 @@ impl DetailScreen { title: d.title.clone(), context: String::new(), version, + // An episode page plays its own episode: its show keys the queue. A movie + // page has none. + show_rk: d.episodes_show_rk().to_owned(), }, |m| PlayIntent::Movie { item: m.clone(), version }, ); @@ -4685,6 +4689,7 @@ impl DetailScreen { title, context, version, + show_rk: d.episodes_show_rk().to_owned(), }; self.content(fx, ContentReq::Play { play, resume_ns }); true diff --git a/rust-modules/screens/src/player/compiled_geometry_tests.rs b/rust-modules/screens/src/player/compiled_geometry_tests.rs index ed346146a8..617c155439 100644 --- a/rust-modules/screens/src/player/compiled_geometry_tests.rs +++ b/rust-modules/screens/src/player/compiled_geometry_tests.rs @@ -291,7 +291,7 @@ fn the_row_parks_on_the_primary_control_of_the_package() { let compiled = reference(); let mut page = page_with(Arc::clone(&compiled)); assert_eq!(page.row_primary(), ordinal(&compiled, "play"), "a movie: no pair, so Play"); - page.slot = ControlSlot::Discs(plx_appkit::player_hud::Controls::for_item(true, true, true, false)); + page.slot = ControlSlot::Discs(plx_appkit::player_hud::Controls::for_item(true, true, true)); let episode = compiled.lookup(0, Slot::Discs, false, true, false).expect("the episode form"); let next = episode.row_ids.iter().position(|r| r.as_deref() == Some("next")).unwrap() as i32; assert_eq!(page.row_primary(), next, "an episode with a Next: Next"); diff --git a/rust-modules/screens/src/player/hud_pins_tests.rs b/rust-modules/screens/src/player/hud_pins_tests.rs index 9d0884f828..04a3c88042 100644 --- a/rust-modules/screens/src/player/hud_pins_tests.rs +++ b/rust-modules/screens/src/player/hud_pins_tests.rs @@ -124,7 +124,7 @@ fn marker(kind: plx_data::metadata::MarkerKind, final_seg: bool) -> plx_data::me fn slot_of(occupant: Occupant) -> ControlSlot { use plx_data::metadata::MarkerKind::{Credits, Intro}; use plx_media::route::NextEpisodeMode::Countdown; - let discs = |previous, next| ControlSlot::Discs(Controls { previous, next, restart: false }); + let discs = |previous, next| ControlSlot::Discs(Controls { previous, next }); let slot = match occupant { Occupant::Plain => ControlSlot::Discs(Controls::NONE), Occupant::Episode => discs(true, true), diff --git a/rust-modules/screens/src/player/mod.rs b/rust-modules/screens/src/player/mod.rs index 1383c12b68..ef752d910c 100644 --- a/rust-modules/screens/src/player/mod.rs +++ b/rust-modules/screens/src/player/mod.rs @@ -568,7 +568,7 @@ impl PlayerScreen { // opening or closing a section is a whole-page change with no spring behind it mix(u64::from(self.hud.sections.bits())); mix(match self.eff_slot() { - ControlSlot::Discs(controls) => 4 + u64::from(controls.pair()) + 2 * u64::from(controls.previous) + 4 * u64::from(controls.next) + 8 * u64::from(controls.restart), + ControlSlot::Discs(controls) => 4 + u64::from(controls.pair()) + 2 * u64::from(controls.previous) + 4 * u64::from(controls.next), ControlSlot::Skip(_) => 2, ControlSlot::UpNext(_) => 3, ControlSlot::AutomaticUpNext(_) => 32, @@ -3532,7 +3532,7 @@ mod step_ladder_tests { assert!(failure_n >= 1, "Back is always offered"); let cases: [(&str, ControlSlot, Busy, Vec); 5] = [ ("discs", ControlSlot::Discs(player_hud::Controls::NONE), Busy::None, [(2..5).map(|i| ELEM_CONTROL_BASE + i).collect::>(), vec![ELEM_TAB_BASE]].concat()), - ("episode", ControlSlot::Discs(player_hud::Controls { previous: true, next: true, restart: false }), + ("episode", ControlSlot::Discs(player_hud::Controls { previous: true, next: true }), Busy::None, [row(5), vec![ELEM_TAB_BASE]].concat()), ("skip", skip, Busy::None, [row(1), vec![ELEM_TAB_BASE]].concat()), ("up next", up_next, Busy::None, [row(2), vec![ELEM_TAB_BASE]].concat()), diff --git a/rust-modules/screens/src/player/overlay_tests.rs b/rust-modules/screens/src/player/overlay_tests.rs index 6e78ab1e8d..6b1abd2af1 100644 --- a/rust-modules/screens/src/player/overlay_tests.rs +++ b/rust-modules/screens/src/player/overlay_tests.rs @@ -337,7 +337,7 @@ fn a_folded_row_closes_more_first_and_asks_the_one_performer() { .expect("the minimal fixture loads") .compiled; let model = plx_appkit::player_hud::model::HudModel { - controls: plx_appkit::player_hud::Controls::for_item(true, true, true, false), + controls: plx_appkit::player_hud::Controls::for_item(true, true, true), chapters: true, detail: true, ..Default::default() diff --git a/rust-modules/screens/src/player/seek_jump.rs b/rust-modules/screens/src/player/seek_jump.rs index 94a00b9431..e1a58e9d2d 100644 --- a/rust-modules/screens/src/player/seek_jump.rs +++ b/rust-modules/screens/src/player/seek_jump.rs @@ -272,8 +272,8 @@ mod tests { let cx: Cx<'_, TestHost> = Cx { views: &playback.session, tick: Tick::default(), measure: &FixtureMeasure, focus: Default::default(), press: Default::default(), owner: InputOwner::Entry(EntryId(9)) }; for (controls, primary, right, left) in [ - (player_hud::Controls::for_item(true, false, true, false), 1, 2, 1), - (player_hud::Controls::for_item(true, true, false, false), 0, 2, 0), + (player_hud::Controls::for_item(true, false, true), 1, 2, 1), + (player_hud::Controls::for_item(true, true, false), 0, 2, 0), (player_hud::Controls::NONE, 2, 3, 2), ] { page.slot = ControlSlot::Discs(controls); diff --git a/rust-modules/screens/src/registry.rs b/rust-modules/screens/src/registry.rs index c9dbbadc9b..ada44a5020 100644 --- a/rust-modules/screens/src/registry.rs +++ b/rust-modules/screens/src/registry.rs @@ -1294,6 +1294,10 @@ pub enum PlayIntent { context: String, /// The version picked on the Play Version panel, `None` for the default (`Media[0]`). version: Option, + /// The show the item's episode belongs to, `""` for anything that is not an episode (a + /// trailer, a movie, an extra). The queue builder keys the PlayQueue off the show so the + /// episodes before this one are in it, which is what the HUD's Previous needs. + show_rk: String, }, /// The alternative source the page had selected (`route::request_play_movie`). Owned rather /// than a `&'static` catalog borrow: it is held inside `PageAction`/`AppFx` across a frame diff --git a/rust-modules/src/app/content.rs b/rust-modules/src/app/content.rs index 498003edaf..cbd10292c0 100644 --- a/rust-modules/src/app/content.rs +++ b/rust-modules/src/app/content.rs @@ -152,10 +152,11 @@ pub(super) fn request_play_intent( ) -> bool { match play { plx_screens::registry::PlayIntent::Item { - sid, rk, part, vcodec, acodec, title, context, version, + sid, rk, part, vcodec, acodec, title, context, version, show_rk, } => { let ok = plx_media::route::request_play( session, meta, super::playback::start_caps(), *sid, rk, part, vcodec, acodec, title, context, *version, + show_rk, ); if ok { note_extra_now_playing(meta, *sid, rk, context); @@ -360,6 +361,7 @@ mod held_feature_tests { title: "Official Trailer".into(), context: context.into(), version: None, + show_rk: String::new(), } } diff --git a/rust-modules/src/app/input.rs b/rust-modules/src/app/input.rs index 53f6ccbc29..3b4e6824af 100644 --- a/rust-modules/src/app/input.rs +++ b/rust-modules/src/app/input.rs @@ -1116,6 +1116,7 @@ pub(super) unsafe fn apply_item_action( title, context: plx_data::metadata::TRAILER_CONTEXT.to_string(), version: None, + show_rk: String::new(), }; // A press-and-hold context menu can reach here within the same beat as opening it, // which only STARTED the open preview's abandonment (`ContentReq::ItemMenu`'s own diff --git a/rust-modules/src/app/playback.rs b/rust-modules/src/app/playback.rs index b906708439..56938cafa3 100644 --- a/rust-modules/src/app/playback.rs +++ b/rust-modules/src/app/playback.rs @@ -505,7 +505,7 @@ pub(crate) fn request_loaded_hero(ps: &mut plx_media::route::PlaybackSession, me request_episode(ps, meta, &d, &ep, None).then(|| plx_data::metadata::resume_ns(ep.resume_ms, ep.dur_ms)) } else { plx_media::route::request_play(ps, meta, start_caps(), plx_media::route::item_sid(d.sid), &d.rk, &d.part, - &d.vcodec, &d.acodec, &d.title, "", None) + &d.vcodec, &d.acodec, &d.title, "", None, d.episodes_show_rk()) .then(|| plx_data::metadata::resume_ns(d.resume_ms, d.dur_ms)) } } @@ -528,7 +528,7 @@ fn request_episode(ps: &mut plx_media::route::PlaybackSession, meta: &mut plx_da let title = if ep.title.is_empty() { &d.title } else { &ep.title }; let context = plx_ui::fmt::episode_context(&d.title, ep.season, ep.index); plx_media::route::request_play(ps, meta, start_caps(), plx_media::route::item_sid(d.sid), &ep.rk, &ep.part, - &ep.vcodec, &ep.acodec, title, &context, version) + &ep.vcodec, &ep.acodec, title, &context, version, d.episodes_show_rk()) } /// Leaving playback (Stop / BACK / EOS / Info's jump-to-detail): retire every in-player panel. diff --git a/rust-modules/src/app/playback/queue_transport.rs b/rust-modules/src/app/playback/queue_transport.rs index d082557f3f..247ecda444 100644 --- a/rust-modules/src/app/playback/queue_transport.rs +++ b/rust-modules/src/app/playback/queue_transport.rs @@ -1,8 +1,13 @@ //! Explicit queue navigation and Up Next share one stop → resolve → start ritual. use super::*; -use plx_media::route::{previous_restarts, queue_neighbor, QueueStep}; +use plx_media::route::{queue_neighbor, QueueStep}; use plx_plex::plex::QueueRow; +/// Previous and Next both ask for the neighbouring queue row and play it through [`play`]; neither +/// is ever a seek. The outgoing episode is stopped by the replacement itself +/// (`stop_for_replacement` → `final_report`), which reports the playhead it really reached, so its +/// resume point survives the step. Restarting the current episode is `from_beginning` (the Info +/// tab's action, and a More-menu row in a package that folds it), not a Previous. pub(super) fn step( ps: &mut plx_media::route::PlaybackSession, pa: &mut plx_media::player::adapter::PlayerAdapter, @@ -10,12 +15,6 @@ pub(super) fn step( pages: &mut plx_ui::dispatch::Dispatcher, bridge: &mut super::super::bridge::Bridge, ) { - if direction == QueueStep::Previous && previous_restarts(plx_media::player::intended_pos_ns(ps)) - { - request_seek(ps, pa, 0); - resume_if_paused(ps, pa); - return; - } let Some(row) = queue_neighbor(ps, direction) else { return; }; diff --git a/rust-modules/src/app/run.rs b/rust-modules/src/app/run.rs index 777a20676e..64f2c6562a 100644 --- a/rust-modules/src/app/run.rs +++ b/rust-modules/src/app/run.rs @@ -2776,7 +2776,7 @@ fn follow_control_row(app: &mut App, ctrl: plx_appkit::player_hud::ControlSlot, player.hud.nav = HudNav::HOME; // The ring goes back to the SCRUBBER for the engine as much as for `nav`: `PlayerScreen::key_ok` // acts on the element the engine holds, so a stale seat on the stand-in's item (an enabled - // Previous / Restart on an episode, which reconcile leaves alone) made the next OK + // Previous on an episode, which reconcile leaves alone) made the next OK // press a disc where `main`'s `nav`-driven OK toggled pause. On a movie `main` did not // honour this contract either: reconcile moved the stale item to Subtitles and mirrored // it into `nav`, so OK opened that menu. Following the contract is the one deliberate @@ -4893,7 +4893,7 @@ mod lifecycle_regression_tests { assert_eq!(reqs, vec![plx_screens::registry::PlayerReq::Transport(None)], "OK toggles play"); } - /// The same on an EPISODE, where the first disc (Previous / Restart) IS enabled, so the engine's + /// The same on an EPISODE, where the first disc (Previous) IS enabled, so the engine's /// reconcile has nothing to move: before the departure re-seated the engine, the ring said /// "scrubber" while the engine held that disc, and OK restarted the episode. #[test] @@ -4902,7 +4902,7 @@ mod lifecycle_regression_tests { use plx_data::metadata::{Marker, MarkerKind}; let _serial = plx_base::testlock::serial(); let credits = Marker { kind: MarkerKind::Credits, start_ms: 0, end_ms: 60_000, final_seg: true }; - let discs = ControlSlot::Discs(Controls::for_item(true, true, true, true)); + let discs = ControlSlot::Discs(Controls::for_item(true, true, true)); assert!(discs.enabled(plx_appkit::player_hud::BTN_PREVIOUS), "the fixture: the first disc is live"); let (focus, engine, reqs) = depart_and_press_ok(ControlSlot::UpNext(credits), discs); assert_eq!(focus, GROUP_SCRUB, "nav"); @@ -5129,11 +5129,12 @@ mod lifecycle_regression_tests { ); } - /// **Previous and Next are not interchangeable in the performer.** With the playhead past the - /// restart threshold, Previous restarts the item (`queue_transport::step(Previous)` seeks to 0); - /// Next, with no queue behind it, asks for no seek at all. A swap of the two steps fails both ways. + /// **Previous and Next are queue steps and never seeks.** With the playhead well into the item + /// (it used to be past Previous's five-second restart threshold, where `queue_transport::step` + /// seeked to 0 and rewrote the episode's resume point), neither step, with no queue behind or + /// after the item, asks for a seek: an item with no neighbour has nothing to step to. #[test] - fn previous_steps_the_queue_back_and_next_steps_it_forward() { + fn previous_and_next_with_no_neighbour_seek_nowhere() { use plx_appkit::player_hud::command::HudCommand; let _serial = plx_base::testlock::serial(); let mut app = app(); @@ -5143,7 +5144,7 @@ mod lifecycle_regression_tests { perform_on_a_mounted_player(&mut app, HudCommand::Next, 100); assert_eq!(globals.seek_to(), -1, "Next, with nothing queued after it, seeks nowhere"); perform_on_a_mounted_player(&mut app, HudCommand::Previous, 200); - assert_eq!(globals.seek_to(), 0, "Previous past the threshold restarts the item"); + assert_eq!(globals.seek_to(), -1, "Previous, with nothing queued before it, seeks nowhere: it is not a restart"); } /// **A section no control opens is reached through More, and focus ends on a drawn control.** The package places @@ -5959,6 +5960,7 @@ mod lifecycle_regression_tests { "First", "", None, + "", )); self.entered.recv_timeout(Duration::from_secs(3)).unwrap(); } @@ -6210,6 +6212,7 @@ mod lifecycle_regression_tests { "Episode 1", "", None, + "", )); self.entered.recv_timeout(Duration::from_secs(3)).unwrap(); self.resolve(); @@ -7738,6 +7741,7 @@ mod lifecycle_regression_tests { "Replacement", "", None, + "", )); rig.accept_start(); assert!(plx_media::route::play_pending()); @@ -9814,6 +9818,191 @@ mod lifecycle_regression_tests { rig.requests.lock().unwrap(), ); } + + // ---- Previous and Next walk the queue, and never rewrite the episode they leave ------------- + // + // The Previous disc used to be a RESTART from five seconds in (a seek to 0 that rewrote the + // playhead and the viewOffset of the episode the viewer meant to leave), and was disabled again + // before it: a play started fresh had a queue that began at the playing episode, so there was + // no episode before it to go to. These play an episode of a three-episode season fresh, the way + // a Continue Watching tile or the detail page's Play does, and press the controls. + + /// One episode `n` of the season fixture as a PlayQueue row (ratingKey `n`, queue item `10 + n`). + fn season_row(n: usize) -> String { + format!( + r#"{{"ratingKey":"{n}","playQueueItemID":{},"type":"episode","title":"Episode {n}", + "grandparentRatingKey":"50","grandparentTitle":"Show","parentIndex":1,"index":{n}, + "duration":60000, + "Media":[{{"videoCodec":"h264","audioCodec":"aac","width":1280,"height":720, + "Part":[{{"id":{n},"key":"/library/parts/{n}/file.mkv","Stream":[ + {{"id":1,"streamType":1,"codec":"h264"}}, + {{"id":2,"streamType":2,"codec":"aac","selected":true}}]}}]}}]}}"#, + 10 + n + ) + } + + /// The fixture's answer to `POST /playQueues` for a show-keyed request (`key=` names the episode + /// to start on: the WHOLE season, that episode selected, the episodes before it included) or an + /// episode-only one (that episode and the ones after it), and to `GET /playQueues/1` (the whole + /// season). `/decision` direct-plays; the timeline and the scrobble answer empty. + fn season_body(request: &str) -> String { + if request.contains("/decision?") { + return queue_body(request, true).to_owned(); + } + if request.contains("/:/") { + return r#"{"MediaContainer":{}}"#.into(); + } + let after = |marker: &str| { + request.split_once(marker).and_then(|(_, rest)| rest.split(|c: char| !c.is_ascii_digit()).next()?.parse::().ok()) + }; + let (first, selected) = if request.starts_with("POST") { + match (after("key=%2Flibrary%2Fmetadata%2F"), after("metadata%2F")) { + (Some(keyed), _) => (1, keyed), + (None, Some(episode)) => (episode, episode), + _ => (1, 1), + } + } else { + (1, 1) + }; + let rows: Vec = (first..=3).map(season_row).collect(); + format!( + r#"{{"MediaContainer":{{"machineIdentifier":"lifecycle-fixture","size":{}, + "playQueueID":1,"playQueueSelectedItemID":{},"playQueueSelectedItemOffset":{}, + "playQueueTotalCount":{},"Metadata":[{}]}}}}"#, + rows.len(), + 10 + selected, + selected - first, + rows.len(), + rows.join(",") + ) + } + + /// Episode `rk` of the season fixture played FRESH (a show-keyed request, no queue handed over), + /// its plan landed and a real Engine over it with the playhead at `pos_ms` of 60 s. + fn playing_season_episode_at(rig: &mut Rig, rk: &str, pos_ms: i64) { + assert!(plx_media::route::request_play( + &mut rig.app.player.session, + rig.app.bridge.metadata_mut(), + plx_platform::devcaps::caps(), + rig.sid, + rk, + &format!("/library/parts/{rk}/file.mkv"), + "h264", + "aac", + "Episode", + "", + None, + "50", + )); + rig.entered.recv_timeout(Duration::from_secs(3)).unwrap(); + rig.release.send(()).unwrap(); + poll_until("the season fixture's plan did not land", || { + plx_media::route::pump_play(&mut rig.app.player.session, rig.app.bridge.metadata_mut()).is_some() + }); + rig.accept_start(); + frame(&mut rig.app, 16); + let _ = plx_platform::tv::launch::take_all(); + assert!(rig.app.adapters.player.is_live(), "the fixture must create the actual Engine"); + plx_play_feed::shared::FEED_SHARED.duration_ns.store(60_000_000_000, Ordering::Relaxed); + plx_play_feed::shared::FEED_SHARED.playpos_ns.store(pos_ms * 1_000_000, Ordering::Relaxed); + publish_feed_fixture_facts(&mut rig.app.player.session, |facts| { + facts.duration_ns = Some(60_000_000_000); + facts.position_ns = Some(pos_ms * 1_000_000); + facts.started = Some(true); + }); + } + + /// The `state=stopped` timeline reports episode `rk` sent, once the stop's worker has finished. + fn stop_reports(rig: &Rig, rk: &str) -> Vec { + plx_media::route::drain_scrobble(); + let needle = format!("ratingKey={rk}&"); + rig.requests + .lock() + .unwrap() + .iter() + .filter(|l| l.contains("/:/timeline") && l.contains(&needle) && l.contains("state=stopped")) + .cloned() + .collect() + } + + /// **Previous goes to the episode before, and the episode it leaves reports where it was.** E2 + /// of the season is played fresh and the viewer is 30 s into it. The queue the fresh play built + /// holds E1, so the Previous disc is enabled (the old disc was a Restart that seeked to 0); a + /// press stops E2 at the playhead it really reached (`time=30000`, never a rewound 0 and no + /// seek at all) and asks for E1. + #[test] + fn previous_plays_the_episode_before_and_the_one_it_leaves_reports_its_real_position() { + use plx_appkit::player_hud::command::HudCommand; + let _serial = plx_base::testlock::serial(); + let Some(mut rig) = Rig::serving_with(|request: &str| season_body(request)) else { + eprintln!("SKIPPED previous_plays_the_episode_before_and_the_one_it_leaves_reports_its_real_position: \ + lifecycle fixture worker thread could not be spawned"); + return; + }; + playing_season_episode_at(&mut rig, "2", 30_000); + let controls = plx_appkit::player_hud::Controls::now(&rig.app.player.session); + assert!(controls.previous, "a FRESH play of E2 has E1 before it: {:?}", rig.requests.lock().unwrap()); + assert!(controls.next, "and E3 after it"); + let globals = SeekGlobals::set(&mut rig.app, 30_000_000_000, 60_000_000_000); + + perform_on_a_mounted_player(&mut rig.app, HudCommand::Previous, 100); + + assert!(plx_media::route::play_pending(), "Previous asked for the episode before"); + assert_eq!(globals.seek_to(), -1, "Previous is a queue step: it seeks nowhere"); + let stops = stop_reports(&rig, "2"); + assert_eq!(stops.len(), 1, "the outgoing episode was stopped once: {:?}", rig.requests.lock().unwrap()); + assert!(stops[0].contains("time=30000&"), "it reports the playhead it reached, not 0: {}", stops[0]); + } + + /// **The first episode of the queue has no Previous.** E1 of the season, played fresh, is the + /// first row: the disc is disabled, and a Previous press (the Previous key reaches the same + /// performer) changes nothing: no stop report, no request, no seek. + #[test] + fn previous_on_the_first_episode_of_the_queue_is_disabled_and_does_nothing() { + use plx_appkit::player_hud::command::HudCommand; + let _serial = plx_base::testlock::serial(); + let Some(mut rig) = Rig::serving_with(|request: &str| season_body(request)) else { + eprintln!("SKIPPED previous_on_the_first_episode_of_the_queue_is_disabled_and_does_nothing: \ + lifecycle fixture worker thread could not be spawned"); + return; + }; + playing_season_episode_at(&mut rig, "1", 30_000); + let controls = plx_appkit::player_hud::Controls::now(&rig.app.player.session); + assert!(!controls.previous, "nothing before the first episode: {:?}", rig.requests.lock().unwrap()); + assert!(controls.next, "E2 follows"); + assert!(!controls.enabled(plx_appkit::player_hud::BTN_PREVIOUS), "the disc has no focus stop"); + assert_eq!(controls.command(plx_appkit::player_hud::BTN_PREVIOUS), None, "and nothing to press"); + let globals = SeekGlobals::set(&mut rig.app, 30_000_000_000, 60_000_000_000); + + perform_on_a_mounted_player(&mut rig.app, HudCommand::Previous, 100); + + assert!(!plx_media::route::play_pending(), "no episode before E1 to request"); + assert_eq!(globals.seek_to(), -1, "and it is not a restart either"); + assert!(stop_reports(&rig, "1").is_empty(), "E1 plays on: {:?}", rig.requests.lock().unwrap()); + } + + /// **Next still steps forward, and the episode it leaves reports where it was.** The same E2 at + /// 30 s: Next stops it at `time=30000` and requests E3. + #[test] + fn next_plays_the_episode_after_and_the_one_it_leaves_reports_its_real_position() { + use plx_appkit::player_hud::command::HudCommand; + let _serial = plx_base::testlock::serial(); + let Some(mut rig) = Rig::serving_with(|request: &str| season_body(request)) else { + eprintln!("SKIPPED next_plays_the_episode_after_and_the_one_it_leaves_reports_its_real_position: \ + lifecycle fixture worker thread could not be spawned"); + return; + }; + playing_season_episode_at(&mut rig, "2", 30_000); + let globals = SeekGlobals::set(&mut rig.app, 30_000_000_000, 60_000_000_000); + + perform_on_a_mounted_player(&mut rig.app, HudCommand::Next, 100); + + assert!(plx_media::route::play_pending(), "Next asked for E3"); + assert_eq!(globals.seek_to(), -1); + let stops = stop_reports(&rig, "2"); + assert_eq!(stops.len(), 1, "{:?}", rig.requests.lock().unwrap()); + assert!(stops[0].contains("time=30000&"), "{}", stops[0]); + } } #[cfg(test)] diff --git a/rust-modules/src/app/run_uri_boot_tests.rs b/rust-modules/src/app/run_uri_boot_tests.rs index 20df93b583..58a78cfabd 100644 --- a/rust-modules/src/app/run_uri_boot_tests.rs +++ b/rust-modules/src/app/run_uri_boot_tests.rs @@ -1625,7 +1625,7 @@ mod uri_boot_tests { // The viewer asks for the title again. assert!(plx_media::route::request_play(&mut rig.app.player.session, rig.app.bridge.metadata_mut(), plx_platform::devcaps::caps(), rig.sid, "1", - "/library/parts/1/file.mkv", "h264", "aac", "First", "", None)); + "/library/parts/1/file.mkv", "h264", "aac", "First", "", None, "")); let mut resolved = None; poll_until("the new request did not resolve", || { resolved = plx_media::route::pump_play(&mut rig.app.player.session, diff --git a/rust-modules/src/dev/scenarios.rs b/rust-modules/src/dev/scenarios.rs index 2cb5db09eb..3f5b1cd4a1 100644 --- a/rust-modules/src/dev/scenarios.rs +++ b/rust-modules/src/dev/scenarios.rs @@ -1621,7 +1621,7 @@ fn play_await_tick(app: &mut App, fr: &mut Frame) { } #[cfg(feature = "devtriggers")] plx_base::eventlog::log(&format!("plxnative-play: rk={rk} server={} start", sid.raw())); - if plx_media::route::request_play(&mut app.player.session, app.bridge.metadata_mut(), crate::app::playback::start_caps(), sid, &rk, &part, &vc, &ac, &title, "", None) { + if plx_media::route::request_play(&mut app.player.session, app.bridge.metadata_mut(), crate::app::playback::start_caps(), sid, &rk, &part, &vc, &ac, &title, "", None, "") { let resume = plx_data::metadata::resume_ns(resume_ms, dur_ms); crate::app::playback::start_playback(&mut app.player.session, &mut app.adapters.player, diff --git a/rust-modules/src/focusprobe.rs b/rust-modules/src/focusprobe.rs index 5ffdd3c9cd..a6b1b89b59 100644 --- a/rust-modules/src/focusprobe.rs +++ b/rust-modules/src/focusprobe.rs @@ -317,8 +317,8 @@ fn push_player(ps: &plx_media::route::PlaybackSession, s: &mut String, overlay: ControlSlot::Discs(controls) => controls, ControlSlot::Skip(_) | ControlSlot::UpNext(_) | ControlSlot::AutomaticUpNext(_) => plx_appkit::player_hud::Controls::NONE, }; - let _ = write!(s, " pair={} previous={} next={} restart={}", - b(controls.pair()), b(controls.previous), b(controls.next), b(controls.restart)); + let _ = write!(s, " pair={} previous={} next={}", + b(controls.pair()), b(controls.previous), b(controls.next)); // Whether this item HAS chapters at all — a fact about the item, not about a panel, which is // why it stays here while each overlay's own open flag and cursor arrive on `content` from the // surface (`app::bridge::content_probe`). diff --git a/tests/mock_pms.py b/tests/mock_pms.py index 0b77ac8bef..aacb62815c 100755 --- a/tests/mock_pms.py +++ b/tests/mock_pms.py @@ -662,6 +662,14 @@ def band_order(rows, values, desc, nulls_last): # ---------------------------------------------------------------- the generated library ---- +def show_episodes_of(items, show_rk): + """Every episode in `items` whose show is `show_rk`, in (season, episode) order.""" + show = [x for x in items.values() + if x["type"] == "episode" and x["grandparentRatingKey"] == show_rk] + show.sort(key=lambda x: (x["parentIndex"], x.get("index", 0))) + return show + + class Library: """Two sections — movies (key 1) and shows (key 2) — with people, genres, collections, seasons, episodes, media parts, chapters, markers, watch state and blur colours. Generated @@ -1309,11 +1317,17 @@ def continuous_queue(self, it): its show after it, in (season, episode) order, as PMS answers; anything else, itself.""" if it["type"] != "episode": return [it] - show = [x for x in self.items.values() - if x["type"] == "episode" and x["grandparentRatingKey"] == it["grandparentRatingKey"]] - show.sort(key=lambda x: (x["parentIndex"], x.get("index", 0))) + # Read through `items` alone: tests/test_hud_preview.py calls this unbound on a bare + # namespace that holds only `items` and `images`. + show = show_episodes_of(self.items, it["grandparentRatingKey"]) return show[show.index(it):] + def show_episodes(self, show_rk): + """Every episode of the show `show_rk`, in (season, episode) order: the rows of a PlayQueue + built from the SHOW (`uri` = the show, `key` = the episode to start on), which holds the + episodes before the keyed one as well as after it.""" + return show_episodes_of(self.items, show_rk) + # --- item builders ------------------------------------------------------------------- def _blur(self, rng): @@ -3175,6 +3189,7 @@ def __init__(self, lib): self.lock = threading.Lock() self.link = link_conditioner.Conditioner() # `--link`; a no-op until a profile is set self.play_queues = {} # retained snapshots for GET /playQueues/?center= + self.play_queue_selected = {} # queue id -> the playQueueItemID its POST selected self.wire_log = [] # `GET /_mock/wire`: per-request arrival/finish times under the link self.requests = [] # (path, status) in arrival order, for the harness # `/:/eventsource/notifications` (the PMS SSE feed): one queue per open stream, fed by @@ -3886,11 +3901,24 @@ def paged(rows): uri = q.get("uri", "") rk = uri.rsplit("/", 1)[-1] it = lib.items.get(int(rk)) if rk.isdigit() else None + key_rk = q.get("key", "").rsplit("/", 1)[-1] + keyed = lib.items.get(int(key_rk)) if key_rk.isdigit() else None + selected = 0 # Only the demo library follows `continuous=1` past the item, and the verification show # under `--media-next-episode`: the seeded library's queue of one is what the # harness's recorded cases were taken against. if it is None: queue = [] + elif (it["type"] == "show" and keyed is not None and keyed["type"] == "episode" + and keyed["grandparentRatingKey"] == str(it["ratingKey"]) + and (isinstance(lib, CatalogLibrary) + or (getattr(lib, "media_next_episode", False) + and str(it["ratingKey"]) == str(VERIFY_SHOW_RK)))): + # The queue the app builds an episode's from: the SHOW as the `uri` and the episode + # as the `key` ("the first item to play"), the episodes before it included, the + # keyed one selected (docs/pms-api.md, "A show-keyed PlayQueue"). + queue = lib.show_episodes(str(it["ratingKey"])) + selected = queue.index(keyed) elif q.get("continuous") == "1" and ( isinstance(lib, CatalogLibrary) or (getattr(lib, "media_next_episode", False) @@ -3901,19 +3929,21 @@ def paged(rows): rows = [dict(x, playQueueItemID=n) for n, x in enumerate(queue, start=1)] with self.lock: self.play_queues[1] = rows - return j(self.container(Metadata=rows, playQueueID=1, playQueueSelectedItemID=1, - playQueueSelectedItemOffset=0, playQueueTotalCount=len(rows), + self.play_queue_selected[1] = selected + 1 + return j(self.container(Metadata=rows, playQueueID=1, playQueueSelectedItemID=selected + 1, + playQueueSelectedItemOffset=selected, playQueueTotalCount=len(rows), playQueueVersion=1)) if segs[:1] == ["playQueues"] and len(segs) == 2: queue_id = int(segs[1]) if segs[1].isdigit() else 1 with self.lock: rows = list(self.play_queues.get(queue_id, [])) + selected = self.play_queue_selected.get(queue_id, 1) center = int(q.get("center", "1")) if q.get("center", "1").isdigit() else 1 window = int(q.get("window", "50")) if q.get("window", "50").isdigit() else 50 at = next((n for n, row in enumerate(rows) if row["playQueueItemID"] == center), 0) page = rows[max(0, at - window):at + window + 1] return j(self.container(Metadata=page, playQueueID=queue_id, playQueueTotalCount=len(rows), - playQueueSelectedItemID=1, playQueueSelectedItemOffset=0)) + playQueueSelectedItemID=selected, playQueueSelectedItemOffset=selected - 1)) if p.startswith("/library/parts/"): part_id = int(segs[2]) if len(segs) > 2 and segs[2].isdigit() else -1 if method == "PUT": @@ -4734,6 +4764,28 @@ def part_streams(base, rk): assert window["playQueueSelectedItemID"] == 1, "centering a window must not select it" assert queue_keys(next_base, V3_RATING_KEY) == [V3_RATING_KEY] assert queue_keys(next_base, V1_RATING_KEY, continuous=False) == [V1_RATING_KEY] + # A queue built from the SHOW with the episode as its `key` (what the app asks for an episode, + # so Previous has a row to go to): the whole show in order, the keyed episode selected. + def show_queue(base, show_rk, key_rk): + uri = urllib.parse.quote(f"server://machine/com.plexapp.plugins.library/library/metadata/{show_rk}") + key = urllib.parse.quote(f"/library/metadata/{key_rk}") + request = urllib.request.Request( + base + f"/playQueues?type=video&uri={uri}&key={key}&shuffle=0&repeat=0", method="POST") + return json.load(urllib.request.urlopen(request))["MediaContainer"] + + keyed = show_queue(next_base, VERIFY_SHOW_RK, V3_RATING_KEY) + assert [int(row["ratingKey"]) for row in keyed["Metadata"]] == [V1_RATING_KEY, V3_RATING_KEY] + assert (keyed["playQueueSelectedItemID"], keyed["playQueueSelectedItemOffset"]) == (2, 1), \ + "the keyed episode is selected, with the first one before it" + window = json.load(urllib.request.urlopen(next_base + "/playQueues/1?center=2&window=5&own=0"))["MediaContainer"] + assert [row["playQueueItemID"] for row in window["Metadata"]] == [1, 2] + assert window["playQueueSelectedItemID"] == 2, "a window read keeps the POST's selection" + first = show_queue(next_base, VERIFY_SHOW_RK, V1_RATING_KEY) + assert first["playQueueSelectedItemOffset"] == 0, "the first episode has nothing before it" + movie = show_queue(next_base, VERIFY_SHOW_RK, V2_RATING_KEY)["Metadata"] + assert [int(row["ratingKey"]) for row in movie] == [VERIFY_SHOW_RK], \ + "a key that is not one of the show's episodes does not make a queue of the show" + assert queue_keys(next_base, V1_RATING_KEY) == [V1_RATING_KEY, V3_RATING_KEY], "the episode ask is unchanged" assert queue_keys(next_base, V2_RATING_KEY) == [V2_RATING_KEY], "a movie never follows a show" first_part, first_streams = part_streams(next_base, V1_RATING_KEY) second_part, second_streams = part_streams(next_base, V3_RATING_KEY) diff --git a/tests/playback_navigation_sim.py b/tests/playback_navigation_sim.py index 4ebbf4d633..17754e85c7 100644 --- a/tests/playback_navigation_sim.py +++ b/tests/playback_navigation_sim.py @@ -72,14 +72,20 @@ def control_row(): queue_posts = [path for path, _ in pms.requests if path.split("?")[0] == "/playQueues"] assert len(queue_posts) == 1, f"queue replaced during item navigation: {queue_posts}" + # Back on the first episode of the queue, past the five seconds the old Previous restarted + # from: Previous is no longer a restart, and with no episode before this one it is + # disabled. The row parks on Next, Left finds no Previous stop, and no seek to 0 is asked. wait(r"loop=.*route=player .*pos=(?:[5-9]|[1-9][0-9]+)s", stamp) control_row() + latest = sim.latest(lambda _, line: line.startswith("focus route=player "))[1] + assert "previous=0" in latest and "next=1" in latest, f"the first episode has no Previous: {latest}" stamp = sim.key("left") - wait(r"^focus route=player .*f=1 btn=0", stamp) - stamp = sim.key("ok") - wait(r"seek\(in-place\): av_seek t=0 ", stamp) - wait(r"loop=.*route=player .*pos=[12]s", stamp) - print("PASS Previous restarts the current item after five seconds", flush=True) + time.sleep(0.3) + assert not any(re.search(r"^focus route=player .*f=1 btn=0 ", line) + for at, line in sim.snapshot() if at >= stamp), "Left reached a disabled Previous" + assert not any("seek(in-place): av_seek t=0 " in line + for at, line in sim.snapshot() if at >= stamp), "Previous restarted the item" + print("PASS Previous is disabled on the first episode and never restarts it", flush=True) # UP from the control row explicitly dismisses the HUD. The focus probe uses the last # input time, so its timer-only sample cannot establish an automatic timeout by itself. From c8e83207622e108bd60ac3bc7bc7520a493f73e3 Mon Sep 17 00:00:00 2001 From: Gleb Linnik Date: Thu, 12 Nov 2026 16:26:32 +0300 Subject: [PATCH 3/7] Release note and audit for v0.10.0: what the third candidate adds (#1350) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit ## What The v0.10.0 release note and audit, revised for the third candidate, which is promoted to the release. - **Note:** a Fixed line for the styled-ASS work ([#1347](https://github.com/GLinnik21/plx-native/pull/1347), issue #1328), measured with a test file on the development set and not the reporter's file; the Previous line from #1349 now says the show-built queue was seen working against one real server; the Next and Previous feature line no longer speaks of a "queue". - **Audit:** the third candidate's range (#1347, #1349); the 2026-11-12 `--fps --mock` runs on the development set with their failures and the interleaved parent comparison; the second candidate's package installed over the stable install and byte-checked; per-change rows for #1347 and #1349; a known-issue line for Previous from a show's first episode (specials order undetermined); #1347's build changes; the action-required check re-run (no section owed). `ci/check-package.py --lint-note` and `--lint-audit` pass. 🤖 Generated with [Claude Code](https://claude.com/claude-code) --- docs/release-audits/v0.10.0.md | 11 ++++++++--- docs/release-notes/v0.10.0.md | 5 +++-- 2 files changed, 11 insertions(+), 5 deletions(-) diff --git a/docs/release-audits/v0.10.0.md b/docs/release-audits/v0.10.0.md index dd9671cd8c..9e2814aad2 100644 --- a/docs/release-audits/v0.10.0.md +++ b/docs/release-audits/v0.10.0.md @@ -6,6 +6,8 @@ The evidence behind [the v0.10.0 release note](https://github.com/GLinnik21/plx- **Revised for the second release candidate.** The first candidate (the tag `rc/v0.10.0-rc.1`, the commit `release: 0.10.0`) was cut from `origin/main` after the first draft above. The second is cut from `origin/main` plus the `fix-yellow-hero-polish` branch, which is merged into it; this revision adds what lies between the two (`git log rc/v0.10.0-rc.1..HEAD`): the nine fixes from the owner's test of the first candidate ([#1335](https://github.com/GLinnik21/plx-native/pull/1335)), the Yellow subtitle color ([#1342](https://github.com/GLinnik21/plx-native/pull/1342)) and its follow-ups, the audio-feed fix ([#1343](https://github.com/GLinnik21/plx-native/pull/1343)), the version-choice and throughput-probe fixes ([#1333](https://github.com/GLinnik21/plx-native/pull/1333), [#1339](https://github.com/GLinnik21/plx-native/pull/1339)), the Experimental engine's event-log reasons ([#1332](https://github.com/GLinnik21/plx-native/pull/1332)), the timed skip prompts, marker-access and Home hero spacing commits on the branch, and developer-tooling commits that change nothing a television runs. As before, the reviser built nothing and touched no television. +**Revised for the third candidate, which is promoted to the release.** The third candidate is cut from `origin/main` after [#1347](https://github.com/GLinnik21/plx-native/pull/1347) (styled ASS subtitles: attached fonts, karaoke on every video frame, straight colour over video; closes [issue #1328](https://github.com/GLinnik21/plx-native/issues/1328)), [#1349](https://github.com/GLinnik21/plx-native/pull/1349) (Previous always plays the previous episode, and a fresh play's queue is built from the show) and this revision. Unlike the two revisions above, this one was written by the session that ran the device work it records below, on 2026-11-12, and it adds that work; nothing else above was re-checked. + ## Device test evidence The only grade of whether video plays, and the only thing here a machine cannot produce. **The release-level device suites for 0.10.0 had not been run when this was written.** @@ -16,8 +18,8 @@ The only grade of whether video plays, and the only thing here a machine cannot | server | for 0.10.0 as a release: not recorded when this was written. Individual pull requests name `tests/mock_pms.py` (the synthetic mock) and a live Plex Media Server as a guest identity | | suite | `./tests/run.py --server` — not run when this was written | | suite | `./tests/run.py` (synthetic pipeline tier, no Plex) — not run on a 0.10.0 package when this was written | -| perf | `./tests/run.py --fps` and `--fps-player` — not run on a 0.10.0 package when this was written. Per-change runs are below | -| by hand | a 0.9.0 install updated in place to this package, sign-in kept, profile kept — not run when this was written | +| perf | `./tests/run.py --fps` and `--fps-player` — not run on a 0.10.0 package. On 2026-11-12 `./tests/run.py --fps --mock` ran on the development set, panel off, against a release-profile (`ARM_PROFILE=release`) debug-flavour build of `0c2bf808a`, the second candidate's code before its last review fixes, which changed no drawing code: 34 passed, 6 failed of 40, and 42 skipped because they have no mock catalog. Home and Library scroll scenes held a 60 fps median with no frame over 50 ms in any paged scroll. The failures were `hero-pong`, `grid-pong`, `home-tap`, `collection-hold`, `collection-tap` (one of two runs) and `item-menu`; `hero-pong`, `grid-pong`, `home-tap` and `item-menu` fail the same way on the parent commit `b59dc1ecb`. An interleaved run of `hero-pong` (five each) read 93–121 missed refreshes on `0686bf703` against 89–113 on `b59dc1ecb`, and `home-tap` 21–47 against 24–40, so the second candidate's changes did not move them; both scenes' absolute level has drifted far above their October baselines, a cause not determined here. `home-hero`, `home-fold`, `library-switch` and `library-collections`, given temporary mock catalogs that were not committed, passed three runs of three on a release-profile build of `0686bf703`; the timed skip prompt, in a scratch player scene, held a 60/s main loop while offered. These runs predate [#1347](https://github.com/GLinnik21/plx-native/pull/1347) and [#1349](https://github.com/GLinnik21/plx-native/pull/1349) | +| by hand | a 0.9.0 install updated in place to this package, sign-in kept, profile kept — not run. On 2026-11-12 the published second-candidate package (its sha256 matched the candidate's `rc.sha256`) was installed through appinstalld over the development set's existing stable install, and the installed binary was byte-identical to the package's payload; nothing about playback on that install, or whether its sign-in and profiles survived, was recorded | | host/static | no `make check`, `--no-default-features` check, ARM build or `ci/check-elf.sh` result for the release head is recorded in this audit when this was written. Pull-request bodies record them on their own branches and CI gates every merge; the release run's own gates produce the generated half below | | install flavour | the device runs recorded below mostly name a debug-flavour install (`com.beb.plxnative.debug`) unless a pull request says otherwise. Nothing records the stable id, or the release feature set, on a television for this version | @@ -43,6 +45,8 @@ The only grade of whether video plays, and the only thing here a machine cannot | Yellow subtitle color ([#1342](https://github.com/GLinnik21/plx-native/pull/1342)) and the follow-ups that recolour white styles in ASS/SSA and dim the Color choice | the maintainer's debug build, on a converted-SubRip-as-ASS track, as above | the panel and HDR look of the yellow and its outline, any styled script with several colors, and the dimmed choice; the real-libass pixel test is ignored by default and was run on a host only | | Timed skip prompts and Home hero title spacing (branch commits) | the maintainer's debug build, as above | the stable package, and any Plex Pass account | | Magic Remote Back and pointer ([#1288](https://github.com/GLinnik21/plx-native/pull/1288)) | host tests; no television run is recorded in the commit body | the pointer on a television | +| Styled ASS subtitles: attached fonts, karaoke on every video frame, straight colour over video ([#1347](https://github.com/GLinnik21/plx-native/pull/1347)) | per its body, a release build on platform release 4.10.2 playing a stress fixture (17 fonts, 83 MB) through the mock: a median of 26 subtitle updates per second at 24p (29.7 with the video re-encoded to 30p), composite p95 15–25 ms, process memory peak 287 MB with one renderer and 372 MB with two, UI heartbeat 50–52 fps during playback, and a translucent sign over video measured pale as authored; a firmware-compatibility review of its new link and calls | a real server, the reporter's file, the stable package; two app restarts in earlier runs of that work left no crash record and their cause is not determined | +| Previous plays the previous episode, the queue built from the show ([#1349](https://github.com/GLinnik21/plx-native/pull/1349)) | on 2026-11-12, a debug build on the development set against the maintainer's own server, driven by the maintainer: the play's queue arrived as the show's (30 rows, the playing episode selected in it), and Previous then Next moved to the episode before and back, per the event log; the maintainer checked it | whether Plex lists season 0 specials before the first episode (so Previous from it could open a special), the queue window on a long show, and the stable package | ## External compatibility reports known at release time @@ -63,6 +67,7 @@ No line moved for this release. The note carries the same first group as v0.9.0' - **Not confirmed on a television in the second candidate.** The Library symptoms behind [#1335](https://github.com/GLinnik21/plx-native/pull/1335) item 9: focus seats on the first tile after opening onto Collections or Recommended (a change with a host test that failed before it), and the flash of the old layout on Back to the grid, which did not reproduce in the host rigs and was left unchanged. The note says both are unseen on a television. - **The managed profile's Plex Pass** is read from the Home administrator's feature list by the owner's decision of 2026-11-11 ([#1335](https://github.com/GLinnik21/plx-native/pull/1335), `docs/pms-api.md`). No real managed account's answers were observed; the policy is not a measurement. A full (non-managed) Home member also takes the administrator's Pass, because the household roster has no flag to tell the two apart. - **Yellow subtitles recolour every pure-white style in a styled script,** so signs and typesetting drawn in white turn yellow too: a known cost of treating white as the default ink, reversing [#1342](https://github.com/GLinnik21/plx-native/pull/1342)'s "authored ASS is never recoloured" for those styles. A karaoke secondary color is recoloured only when it is itself pure white. +- **Previous from a show's first episode** may open a season 0 special if the server lists specials first in the show's queue; [#1349](https://github.com/GLinnik21/plx-native/pull/1349) did not determine the order, and the note does not mention it. A server that does not select the requested episode in a show-built queue gets the old episode-only queue, where Previous stays dimmed on a fresh play. - **The subtitle timing offset has no minus direction** ([issue #1284](https://github.com/GLinnik21/plx-native/issues/1284), open). - **webOS 3.x** is outside the supported range; [issue #862](https://github.com/GLinnik21/plx-native/issues/862) is open and the note does not claim 3.x starts on this package. - **The k5lp, k3lp and k6hp chip issues** ([#74](https://github.com/GLinnik21/plx-native/issues/74) and [#174](https://github.com/GLinnik21/plx-native/issues/174)) are closed (#174 as not planned, since every event came from 0.6.6 and none from 0.7.0 or later); the note keeps them as known issues because nothing proves them gone. @@ -87,7 +92,7 @@ No line moved for this release. The note carries the same first group as v0.9.0' | built by | GitHub Actions, from the tag, at a path identical on every runner | | toolchain | webOS NDK unchanged as far as this range records; Rust is the stable release `rust-modules/rust-toolchain.toml` pins (as v0.9.0's audit records after #816), unchanged in this range | -Changes to the build that a reviewer might look for: new bundled FFmpeg audio decoders and a resampler for the app's PCM paths ([#950](https://github.com/GLinnik21/plx-native/pull/950), [#1095](https://github.com/GLinnik21/plx-native/pull/1095)), the HarfBuzz, FriBidi and FreeType shaping calls and two bundled Noto faces for Arabic and Hebrew ([#1007](https://github.com/GLinnik21/plx-native/pull/1007)), the playback build matrix and the URI backend returning to the shipping binary but never selected by default ([#1276](https://github.com/GLinnik21/plx-native/pull/1276), [#1275](https://github.com/GLinnik21/plx-native/pull/1275)), an `exit()` interposer ([#1242](https://github.com/GLinnik21/plx-native/pull/1242)) and a WebSocket transport over libcurl. A new outbound host: the Player HUD gallery reads a registry at `huds.plxnative.com` ([#1243](https://github.com/GLinnik21/plx-native/pull/1243), [#1239](https://github.com/GLinnik21/plx-native/pull/1239)), accepting only a signed index, and fetches previews without credentials. A new outbound read since the first candidate: for a seated managed Home profile whose own plex.tv answer lacks Plex Pass, the app also reads the Home administrator's `/api/v2/user` and `/api/v2/features` on plex.tv, GET only, under the account token the session already stores for that account, accepted only when the roster names one administrator and the answer matches it, and used for these two reads and nothing else ([#1335](https://github.com/GLinnik21/plx-native/pull/1335), `docs/pms-api.md`). **The "was an action-required section owed?" check was re-run for this addition** over the diff between the first candidate and the second (`git diff rc/v0.10.0-rc.1 HEAD -- rust-modules src ci` with the README's pattern): the hits are test fixtures, developer-tooling scripts that use `/tmp` as test data, a colour-table identifier that contains the word token, and this one production read. There is no new file the app writes, permission, listener, FIFO or path outside its own directories, and no new host (plex.tv is already contacted for the account). The one change is that the account token is now sent on a managed profile's seat, to the same host, for a read. The judgement recorded in the commit is that no action-required section is owed and that the read is disclosed in the note's Changed section; it is a judgement, and the owner should confirm it. `PRIVACY.md` was not changed; its general statement that the app talks to Plex services for account features is what covers it. Whether any of these changed the payload's `DT_NEEDED` set or file list is what the generated half records; this paragraph does not claim it. +Changes to the build that a reviewer might look for: new bundled FFmpeg audio decoders and a resampler for the app's PCM paths ([#950](https://github.com/GLinnik21/plx-native/pull/950), [#1095](https://github.com/GLinnik21/plx-native/pull/1095)), the HarfBuzz, FriBidi and FreeType shaping calls and two bundled Noto faces for Arabic and Hebrew ([#1007](https://github.com/GLinnik21/plx-native/pull/1007)), the playback build matrix and the URI backend returning to the shipping binary but never selected by default ([#1276](https://github.com/GLinnik21/plx-native/pull/1276), [#1275](https://github.com/GLinnik21/plx-native/pull/1275)), an `exit()` interposer ([#1242](https://github.com/GLinnik21/plx-native/pull/1242)) and a WebSocket transport over libcurl. A new outbound host: the Player HUD gallery reads a registry at `huds.plxnative.com` ([#1243](https://github.com/GLinnik21/plx-native/pull/1243), [#1239](https://github.com/GLinnik21/plx-native/pull/1239)), accepting only a signed index, and fetches previews without credentials. A new outbound read since the first candidate: for a seated managed Home profile whose own plex.tv answer lacks Plex Pass, the app also reads the Home administrator's `/api/v2/user` and `/api/v2/features` on plex.tv, GET only, under the account token the session already stores for that account, accepted only when the roster names one administrator and the answer matches it, and used for these two reads and nothing else ([#1335](https://github.com/GLinnik21/plx-native/pull/1335), `docs/pms-api.md`). **The "was an action-required section owed?" check was re-run for this addition** over the diff between the first candidate and the second (`git diff rc/v0.10.0-rc.1 HEAD -- rust-modules src ci` with the README's pattern): the hits are test fixtures, developer-tooling scripts that use `/tmp` as test data, a colour-table identifier that contains the word token, and this one production read. There is no new file the app writes, permission, listener, FIFO or path outside its own directories, and no new host (plex.tv is already contacted for the account). The one change is that the account token is now sent on a managed profile's seat, to the same host, for a read. The judgement recorded in the commit is that no action-required section is owed and that the read is disclosed in the note's Changed section; it is a judgement, and the owner should confirm it. `PRIVACY.md` was not changed; its general statement that the app talks to Plex services for account features is what covers it. Since the second candidate, [#1347](https://github.com/GLinnik21/plx-native/pull/1347) moves the bundled libass facade to ABI 5 (fonts borrowed from the app instead of copied), links `-lrt` for `clock_gettime` on firmware 6.4 and older, lowers its helper renderers' priority with `setpriority`, and adds a framebuffer-fetch pass when the extension is present; its body records three passes of the firmware-compatibility review and a load check on 4.4.2 through 11.2.0. The action-required check was re-run over `rc/v0.10.0-rc.2..HEAD` plus [#1349](https://github.com/GLinnik21/plx-native/pull/1349): its hits are the word token in a GL-extension parser, file headers whose path contains `/media/` and test fixture hashes. There is no new file the app writes, permission, listener, FIFO, path or host, so no action-required section is owed. Whether any of these changed the payload's `DT_NEEDED` set or file list is what the generated half records; this paragraph does not claim it. diff --git a/docs/release-notes/v0.10.0.md b/docs/release-notes/v0.10.0.md index 45d646626a..5f6c66ab8b 100644 --- a/docs/release-notes/v0.10.0.md +++ b/docs/release-notes/v0.10.0.md @@ -8,7 +8,7 @@ This release lets you choose, install and replace the player's on-screen control - **Play Version.** A movie or episode with two or more versions on the server gets a **Play Version** row in its menu and a disc after Play on its page; a row plays that version. Thanks to [aulonm](https://github.com/aulonm). - **Playback speed.** The player's `…` menu has a **Playback speed** row with 0.5×, 0.75×, 1.25×, 1.5×, 1.75× and 2× beside Normal. The speed is applied in the app, not by the television, and a speed the set cannot play is not offered. Thanks to [dimitrije83](https://github.com/dimitrije83). - **Skip intros and credits automatically.** For a viewer with Plex Pass, Settings → Video & playback has per-profile **Skip intros** and **Skip credits** choices of Manual, Disabled or Automatic. Automatic skipping waits five seconds of actual playback. On the timed prompt the filled button is the action the countdown performs (Skip Intro, or Next Episode after credits), pressing it does that now, and a separate Watch Intro or Watch Credits button beside it cancels the countdown; focus starts on the filled one. The choice is hidden when your plex.tv account does not offer the markers, and a Plex Pass held by the viewer now counts ([danzig666](https://github.com/danzig666)). A managed Home profile, which has no Plex Pass of its own, takes its Home administrator's. The two Settings rows and an automatic offer no longer blink away for a moment every few minutes. -- **Next and Previous, and tap seeking.** An episode with a neighbour in its queue gets Previous and Next buttons in the control row. A tap on fast-forward or rewind jumps one Skip interval with the controls left hidden, quick taps add up, and a hold still scrubs. A per-profile **Instant D-pad seeking** option lets Left and Right jump without showing the controls first. Thanks to [stefan-ovezea](https://github.com/stefan-ovezea). +- **Next and Previous, and tap seeking.** An episode with an episode before or after it gets Previous and Next buttons in the control row. A tap on fast-forward or rewind jumps one Skip interval with the controls left hidden, quick taps add up, and a hold still scrubs. A per-profile **Instant D-pad seeking** option lets Left and Right jump without showing the controls first. Thanks to [stefan-ovezea](https://github.com/stefan-ovezea). - **When it will finish.** The player's HUD shows the local time the film or episode ends at; the **Estimated end time** switch on the Player HUD settings page is offered only while the HUD in use has room for it (a HUD package declares that in format 1.1). Thanks to [seergi893](https://github.com/seergi893). - **More audio plays directly.** FLAC, ALAC and integer PCM are decoded in the app and sent to the television as PCM; stereo files above 48 kHz are resampled by the app instead of being re-encoded lossily by the server. Opus direct-plays on sets that can take it. A DTS core is decoded to stereo where the set has no DTS decoder, but only under Force Direct Play for one- and two-channel tracks, and as the last step in Auto after the server has refused to convert a refused file; a surround DTS file is otherwise still converted by the server. Audio the app decodes is delayed to line up with the picture; that was timed on the development set, with speakers and with Bluetooth, and the other LG audio stacks use values read from firmware that nobody has timed. Surround is not mixed down to stereo by default. Thanks to [paulorichards](https://github.com/paulorichards) and [Dr-Amp](https://github.com/Dr-Amp). - **Continue Watching is a row of stills.** On Home and on Recommended, each tile is a 16:9 still with cover art, a one-line footer with an episode label and time left, and a play glyph on focus. The Home hero shows playback progress in its Resume pill and, for every movie, show and episode, a line with the type, year, length, genres and age rating, whether or not you have started it. Its page dots sit inside the page and slide with it, the Play pill's tint follows the page that is arriving instead of popping, the background wash stays steady through a quick run of page flips, and a title drawn in type rather than as a logo now keeps the same gap above that line. A fullscreen autoplay trailer on a Detail page shows its clock and can be sought. @@ -41,7 +41,8 @@ This release lets you choose, install and replace the player's on-screen control - **A version you chose with Play Version could be re-encoded** at a quality choice it fitted, because its own bitrate was not read. - **Opening the Library straight onto Collections or Recommended** now puts focus on the first tile instead of leaving it on the top bar. A related symptom, a flash of the old layout when going Back to the grid, could not be reproduced here and has not been changed; neither has been seen on a television. - **A set with four cores and 2 GiB of memory** decodes posters with three workers instead of two. -- **Previous now always goes to the previous episode.** Five seconds into an episode the button used to restart it, which rewound the progress Plex had saved for it, and before that it was dimmed, because an episode started from Continue Watching or a page had no earlier episode in its queue. The queue is now built from the show, so the earlier episodes are in it, and Restart is no longer a button face; **From beginning** on the player's Info tab (and in the `…` menu of a HUD that lists it) starts the episode over. The show-based queue request follows Plex's published API and was not observed against a real server: a server that does not answer it as expected leaves the queue as before, with Previous dimmed on that episode. +- **Styled ASS and SSA subtitles with embedded fonts and heavy karaoke,** such as anime openings, now use the fonts attached to the file, animate on every video frame instead of a few times a second, and a translucent sign over the picture is no longer drawn dark; this was measured with a test file on the development set, not with the reporter's file ([issue #1328](https://github.com/GLinnik21/plx-native/issues/1328)). Thanks to [josmellop](https://github.com/josmellop). +- **Previous now always goes to the previous episode.** Five seconds into an episode the button used to restart it, which rewound the progress Plex had saved for it, and before that it was dimmed, because an episode started from Continue Watching or a page had no earlier episode in its queue. The queue is now built from the show, so the earlier episodes are in it, and Restart is no longer a button face; **From beginning** on the player's Info tab (and in the `…` menu of a HUD that lists it) starts the episode over. The show-based queue was seen working against one real server; a server that does not answer it as expected leaves the queue as before, with Previous dimmed on that episode. ## Changed From bd3ffbbecdc85ef779a799e111cc25bc93efc6ca Mon Sep 17 00:00:00 2001 From: Gleb Linnik Date: Thu, 12 Nov 2026 17:27:15 +0300 Subject: [PATCH 4/7] Release note v0.10.0: a redesigned Home and ASS we can in the title (#1352) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit ## What The v0.10.0 release note's title and Home line now say what a viewer sees (a redesigned Home, what you were watching one press away) instead of the design name "stills", and the styled-ASS fix leads with "ASS we can". Wording only; `ci/check-package.py --lint-note` passes. 🤖 Generated with [Claude Code](https://claude.com/claude-code) --- docs/release-notes/v0.10.0.md | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/docs/release-notes/v0.10.0.md b/docs/release-notes/v0.10.0.md index 5f6c66ab8b..e568913695 100644 --- a/docs/release-notes/v0.10.0.md +++ b/docs/release-notes/v0.10.0.md @@ -1,6 +1,6 @@ -# v0.10.0 — a player HUD you can replace, Play Version and playback speed, and a Home built on stills +# v0.10.0 — a player HUD you can replace, Play Version and playback speed, a redesigned Home, and ASS we can -This release lets you choose, install and replace the player's on-screen controls, adds Play Version, playback speed and automatic intro skipping, rebuilds Home's Continue Watching as a row of stills, and plays far more audio formats by decoding them in the app. It also adds Russian, Polish, Dutch and Catalan, draws Arabic and Hebrew, and fixes a long list of seeks and resumes that hung. It is the first release since 0.9.0, which was cut on 2026-10-23. +This release lets you choose, install and replace the player's on-screen controls, adds Play Version, playback speed and automatic intro skipping, redesigns Home so what you were watching is one press away, and plays far more audio formats by decoding them in the app. It also adds Russian, Polish, Dutch and Catalan, draws Arabic and Hebrew, and fixes a long list of seeks and resumes that hung. It is the first release since 0.9.0, which was cut on 2026-10-23. ## What's new @@ -11,7 +11,7 @@ This release lets you choose, install and replace the player's on-screen control - **Next and Previous, and tap seeking.** An episode with an episode before or after it gets Previous and Next buttons in the control row. A tap on fast-forward or rewind jumps one Skip interval with the controls left hidden, quick taps add up, and a hold still scrubs. A per-profile **Instant D-pad seeking** option lets Left and Right jump without showing the controls first. Thanks to [stefan-ovezea](https://github.com/stefan-ovezea). - **When it will finish.** The player's HUD shows the local time the film or episode ends at; the **Estimated end time** switch on the Player HUD settings page is offered only while the HUD in use has room for it (a HUD package declares that in format 1.1). Thanks to [seergi893](https://github.com/seergi893). - **More audio plays directly.** FLAC, ALAC and integer PCM are decoded in the app and sent to the television as PCM; stereo files above 48 kHz are resampled by the app instead of being re-encoded lossily by the server. Opus direct-plays on sets that can take it. A DTS core is decoded to stereo where the set has no DTS decoder, but only under Force Direct Play for one- and two-channel tracks, and as the last step in Auto after the server has refused to convert a refused file; a surround DTS file is otherwise still converted by the server. Audio the app decodes is delayed to line up with the picture; that was timed on the development set, with speakers and with Bluetooth, and the other LG audio stacks use values read from firmware that nobody has timed. Surround is not mixed down to stereo by default. Thanks to [paulorichards](https://github.com/paulorichards) and [Dr-Amp](https://github.com/Dr-Amp). -- **Continue Watching is a row of stills.** On Home and on Recommended, each tile is a 16:9 still with cover art, a one-line footer with an episode label and time left, and a play glyph on focus. The Home hero shows playback progress in its Resume pill and, for every movie, show and episode, a line with the type, year, length, genres and age rating, whether or not you have started it. Its page dots sit inside the page and slide with it, the Play pill's tint follows the page that is arriving instead of popping, the background wash stays steady through a quick run of page flips, and a title drawn in type rather than as a logo now keeps the same gap above that line. A fullscreen autoplay trailer on a Detail page shows its clock and can be sought. +- **A redesigned Home: pick up where you left off at a glance.** Continue Watching sits right under the hero, one press away instead of a scroll down. On Home and on Recommended, each tile is a wide 16:9 picture with cover art, a one-line footer with an episode label and time left, and a play glyph on focus. The Home hero shows playback progress in its Resume pill and, for every movie, show and episode, a line with the type, year, length, genres and age rating, whether or not you have started it. Its page dots sit inside the page and slide with it, the Play pill's tint follows the page that is arriving instead of popping, the background wash stays steady through a quick run of page flips, and a title drawn in type rather than as a logo now keeps the same gap above that line. A fullscreen autoplay trailer on a Detail page shows its clock and can be sought. - **Library, reorganised around the sort.** The Library header's sort control carries the current sort, the poster captions can show the sort's value, and the A-Z rail on the right now follows the other sorts too: rating, duration, unplayed, plays, date added, date viewed, last episode added and release year. A TV library's Seasons and Episodes views get the rail as well. The library list and **Favorite libraries** live in the Library options menu, and Sort & Filter holds only sorting and filtering. A Library remembers whether you left it on Recommended or Collections, per profile. Thanks to [MozgOFF](https://github.com/MozgOFF) and [isaymon](https://github.com/isaymon). - **Settings, redesigned.** The root is four groups (Libraries, Playback, System, Support) with a sentence about the focused row, and Diagnostics is one row that opens its own page. **Under posters** now lets you tick up to four details per kind (movies: year, length, genre, video quality, age rating, score; shows: year, seasons, genre, age rating, score) with a two-poster preview. **Favorite libraries** shows its preview as soon as you open it, and the **Player HUD** page previews the HUD you have focused. - **Dolby Vision Auto or Off, per profile.** Settings → Video & playback has a Dolby Vision setting for sets whose firmware misreports it and for viewers who prefer HDR10. With Off, a Profile 8.1 file plays its HDR10 base. Thanks to [MozgOFF](https://github.com/MozgOFF). @@ -41,7 +41,7 @@ This release lets you choose, install and replace the player's on-screen control - **A version you chose with Play Version could be re-encoded** at a quality choice it fitted, because its own bitrate was not read. - **Opening the Library straight onto Collections or Recommended** now puts focus on the first tile instead of leaving it on the top bar. A related symptom, a flash of the old layout when going Back to the grid, could not be reproduced here and has not been changed; neither has been seen on a television. - **A set with four cores and 2 GiB of memory** decodes posters with three workers instead of two. -- **Styled ASS and SSA subtitles with embedded fonts and heavy karaoke,** such as anime openings, now use the fonts attached to the file, animate on every video frame instead of a few times a second, and a translucent sign over the picture is no longer drawn dark; this was measured with a test file on the development set, not with the reporter's file ([issue #1328](https://github.com/GLinnik21/plx-native/issues/1328)). Thanks to [josmellop](https://github.com/josmellop). +- **ASS we can.** Styled ASS and SSA subtitles with embedded fonts and heavy karaoke, such as anime openings, now use the fonts attached to the file, animate on every video frame instead of a few times a second, and a translucent sign over the picture is no longer drawn dark; this was measured with a test file on the development set, not with the reporter's file ([issue #1328](https://github.com/GLinnik21/plx-native/issues/1328)). Thanks to [josmellop](https://github.com/josmellop). - **Previous now always goes to the previous episode.** Five seconds into an episode the button used to restart it, which rewound the progress Plex had saved for it, and before that it was dimmed, because an episode started from Continue Watching or a page had no earlier episode in its queue. The queue is now built from the show, so the earlier episodes are in it, and Restart is no longer a button face; **From beginning** on the player's Info tab (and in the `…` menu of a HUD that lists it) starts the episode over. The show-based queue was seen working against one real server; a server that does not answer it as expected leaves the queue as before, with Previous dimmed on that episode. ## Changed From 710b13295d969293e7e32d9d7400ae356af844c4 Mon Sep 17 00:00:00 2001 From: Gleb Linnik Date: Thu, 12 Nov 2026 17:40:03 +0300 Subject: [PATCH 5/7] Release note v0.10.0: yellow subtitles in the title; HUDs are submitted in plxnative-huds (#1353) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit ## What The v0.10.0 release note's title now names yellow subtitles, which people asked for. Its Player HUD line says that HUDs for the gallery are made and submitted in the separate [plxnative-huds](https://github.com/GLinnik21/plxnative-huds) repository, with a link to its authoring guide. Wording only; `--lint-note` passes. 🤖 Generated with [Claude Code](https://claude.com/claude-code) --- docs/release-notes/v0.10.0.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/docs/release-notes/v0.10.0.md b/docs/release-notes/v0.10.0.md index e568913695..6413b4db0f 100644 --- a/docs/release-notes/v0.10.0.md +++ b/docs/release-notes/v0.10.0.md @@ -1,10 +1,10 @@ -# v0.10.0 — a player HUD you can replace, Play Version and playback speed, a redesigned Home, and ASS we can +# v0.10.0 — a player HUD you can replace, Play Version and playback speed, yellow subtitles, a redesigned Home, and ASS we can This release lets you choose, install and replace the player's on-screen controls, adds Play Version, playback speed and automatic intro skipping, redesigns Home so what you were watching is one press away, and plays far more audio formats by decoding them in the app. It also adds Russian, Polish, Dutch and Catalan, draws Arabic and Hebrew, and fixes a long list of seeks and resumes that hung. It is the first release since 0.9.0, which was cut on 2026-10-23. ## What's new -- **A player HUD you can choose and replace.** Settings → Video & playback → **Player HUD** lists the built-in HUD and any installed HUD packages, per profile. **Browse HUDs** opens a gallery of published HUDs with a preview and separate Install and Use buttons; a package can also be installed from the HUD folder on the television or from an address. The built-in HUD is now itself a package drawn by the same renderer, so Chapters and Info are sections of it, a **More** fold holds the rest, and menus open next to the control that opened them. How packages work, and what a package may not do, is in [HUD plugins](https://github.com/GLinnik21/plx-native/blob/main/docs/hud-plugins.md). +- **A player HUD you can choose and replace.** Settings → Video & playback → **Player HUD** lists the built-in HUD and any installed HUD packages, per profile. **Browse HUDs** opens a gallery of published HUDs with a preview and separate Install and Use buttons; a package can also be installed from the HUD folder on the television or from an address. The built-in HUD is now itself a package drawn by the same renderer, so Chapters and Info are sections of it, a **More** fold holds the rest, and menus open next to the control that opened them. How packages work, and what a package may not do, is in [HUD plugins](https://github.com/GLinnik21/plx-native/blob/main/docs/hud-plugins.md). HUDs for the gallery are made and submitted in a separate repository, [plxnative-huds](https://github.com/GLinnik21/plxnative-huds), not in this one; its [authoring guide](https://github.com/GLinnik21/plxnative-huds/blob/main/AUTHORING.md) is the place to start. - **Play Version.** A movie or episode with two or more versions on the server gets a **Play Version** row in its menu and a disc after Play on its page; a row plays that version. Thanks to [aulonm](https://github.com/aulonm). - **Playback speed.** The player's `…` menu has a **Playback speed** row with 0.5×, 0.75×, 1.25×, 1.5×, 1.75× and 2× beside Normal. The speed is applied in the app, not by the television, and a speed the set cannot play is not offered. Thanks to [dimitrije83](https://github.com/dimitrije83). - **Skip intros and credits automatically.** For a viewer with Plex Pass, Settings → Video & playback has per-profile **Skip intros** and **Skip credits** choices of Manual, Disabled or Automatic. Automatic skipping waits five seconds of actual playback. On the timed prompt the filled button is the action the countdown performs (Skip Intro, or Next Episode after credits), pressing it does that now, and a separate Watch Intro or Watch Credits button beside it cancels the countdown; focus starts on the filled one. The choice is hidden when your plex.tv account does not offer the markers, and a Plex Pass held by the viewer now counts ([danzig666](https://github.com/danzig666)). A managed Home profile, which has no Plex Pass of its own, takes its Home administrator's. The two Settings rows and an automatic offer no longer blink away for a moment every few minutes. From 07902e072addeb7bdae764090a8ad781c593b99c Mon Sep 17 00:00:00 2001 From: Gleb Linnik Date: Thu, 12 Nov 2026 17:50:46 +0300 Subject: [PATCH 6/7] Release note v0.10.0: the headline leads with what people asked for (#1354) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit ## What The v0.10.0 release note's title leads with the most-requested items this release answers: automatic intro skipping (discussion 826), yellow subtitles (1193), Next and Previous episodes (829), Play Version (521), then the replaceable player HUD and the styled-ASS fix. Wording only; `--lint-note` passes. 🤖 Generated with [Claude Code](https://claude.com/claude-code) --- docs/release-notes/v0.10.0.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/release-notes/v0.10.0.md b/docs/release-notes/v0.10.0.md index 6413b4db0f..a44b32dcee 100644 --- a/docs/release-notes/v0.10.0.md +++ b/docs/release-notes/v0.10.0.md @@ -1,4 +1,4 @@ -# v0.10.0 — a player HUD you can replace, Play Version and playback speed, yellow subtitles, a redesigned Home, and ASS we can +# v0.10.0 — automatic intro skipping, yellow subtitles, Next and Previous episodes, Play Version, a player HUD you can replace, and ASS we can This release lets you choose, install and replace the player's on-screen controls, adds Play Version, playback speed and automatic intro skipping, redesigns Home so what you were watching is one press away, and plays far more audio formats by decoding them in the app. It also adds Russian, Polish, Dutch and Catalan, draws Arabic and Hebrew, and fixes a long list of seeks and resumes that hung. It is the first release since 0.9.0, which was cut on 2026-10-23. From 015868116287136757e456797754fd9a846d42b4 Mon Sep 17 00:00:00 2001 From: "github-actions[bot]" Date: Thu, 12 Nov 2026 15:12:50 +0000 Subject: [PATCH 7/7] release: 0.10.0 --- ipkroot/ctl/control | 2 +- pkg/appinfo.json | 2 +- rust-modules/Cargo.lock | 2 +- rust-modules/Cargo.toml | 2 +- 4 files changed, 4 insertions(+), 4 deletions(-) diff --git a/ipkroot/ctl/control b/ipkroot/ctl/control index f5492f6ae2..6640c6e95e 100644 --- a/ipkroot/ctl/control +++ b/ipkroot/ctl/control @@ -1,5 +1,5 @@ Package: com.beb.plxnative -Version: 0.9.0 +Version: 0.10.0 Section: misc Priority: optional Architecture: arm diff --git a/pkg/appinfo.json b/pkg/appinfo.json index c82bf62da1..911e2da9f8 100644 --- a/pkg/appinfo.json +++ b/pkg/appinfo.json @@ -1,6 +1,6 @@ { "id": "com.beb.plxnative", - "version": "0.9.0", + "version": "0.10.0", "vendor": "beb", "type": "native", "main": "plxnative", diff --git a/rust-modules/Cargo.lock b/rust-modules/Cargo.lock index 2f1c7301e6..fb375cc1a7 100644 --- a/rust-modules/Cargo.lock +++ b/rust-modules/Cargo.lock @@ -992,7 +992,7 @@ dependencies = [ [[package]] name = "plxnative-modules" -version = "0.9.0" +version = "0.10.0" dependencies = [ "image", "libc", diff --git a/rust-modules/Cargo.toml b/rust-modules/Cargo.toml index 202980554c..d76401c786 100644 --- a/rust-modules/Cargo.toml +++ b/rust-modules/Cargo.toml @@ -37,7 +37,7 @@ name = "plxnative-modules" # rather than equal to it: build.rs publishes `PLX_VERSION`, which is this number exactly for a # RELEASE build and the next minor plus `-dev` for every other build, so a working tree does not # report itself as the last published release. -version = "0.9.0" +version = "0.10.0" edition = "2021" # The crate is linked into the C app as a staticlib (see [lib] below) and is never published to # crates.io, so these are not for a registry — they are so `cargo metadata` and any SBOM tool agree