From 2561618bdf1400544d701406e900d34b4b170381 Mon Sep 17 00:00:00 2001 From: Gleb Linnik Date: Sat, 14 Nov 2026 06:30:26 +0300 Subject: [PATCH 1/2] Check: run the hostsim unit pass only over the crates that declare hostsim `check-cargo-unit-hostsim` ran all 18 test binaries a second time, but seven crates declare no `hostsim` feature and have no `feature = "hostsim"` in their source: plx_net, plx_ui, plx_appkit, plx_screens, plx_media_codecs, plx_play_uri and plx_playback_contract. In CI run 43528579834 the two passes printed the same test names for every one of them (and for 7 more that stay), and the hostsim-only tests are in plxnative-modules (+208), plx_media (+280) and plx_play_feed (+17), all kept. What `hostsim` changes below them (base paths steerable, gfx glyph-cache capacity, the simulator's env-driven render scale and locale overrides, GL-context-only code) is either unset or unreachable in a unit run, or makes the hostsim pass the laxer one. The three crates that declare the feature only to forward it (plx_machine, plx_data, plx_session) stay. The recipe's `-p` list becomes HOSTSIM_SUITE, a subset of UNIT_SUITE. ci/test_test_crate.py pins the subset, the recipe, and that every crate left out has no `hostsim` feature and no hostsim cfg, so a crate that gains one fails the gate until it is added. ci/test_ci_split.py's literal pin and the agent-reference paragraph follow. In the hostsim job's log those four large binaries took 78 s (plx_screens), 58.7 (plx_appkit), 33.4 (plx_ui) and 20.4 (plx_net) of the 474 s binary total, and their test binaries are no longer compiled under hostsim either. Co-Authored-By: Claude Opus 5.5 --- Makefile | 12 ++++++++++-- ci/test_ci_split.py | 2 +- ci/test_test_crate.py | 33 ++++++++++++++++++++++++++++++++- docs/agent-reference.md | 7 +++++-- 4 files changed, 48 insertions(+), 6 deletions(-) diff --git a/Makefile b/Makefile index 990443d1b2..85c6ba9191 100644 --- a/Makefile +++ b/Makefile @@ -1689,6 +1689,14 @@ check-cargo-lint: lint # ci/test_test_crate.py pins the two equal. UNIT_RUN = python3 ../tools/cargo-test-parallel.py UNIT_SUITE = plxnative-modules plx_base plx_machine plx_platform plx_gfx plx_net plx_ui plx_plex plx_telemetry plx_data plx_session plx_media plx_playback_contract plx_play_uri plx_play_feed plx_media_codecs plx_appkit plx_screens +# The `hostsim` pass runs only the crates that DECLARE a `hostsim` feature (HOSTSIM_SUITE, a subset of +# UNIT_SUITE in the same order; ci/test_test_crate.py pins both the subset and the recipe). The seven that +# do not (plx_net, plx_ui, plx_appkit, plx_screens, plx_media_codecs, plx_play_uri, plx_playback_contract) +# have no `feature = "hostsim"` in their source and the same test names in both passes (14 crates did in +# CI run 43528579834; the three that differ are `plxnative-modules`, `plx_media` and `plx_play_feed`, +# all kept), so the second run of their binaries repeated the default pass. The same test also fails +# the gate when one of them gains a `hostsim` cfg or feature, which is the moment it must join the list. +HOSTSIM_SUITE = plxnative-modules plx_base plx_machine plx_platform plx_gfx plx_plex plx_telemetry plx_data plx_session plx_media plx_play_feed # The default-feature unit suite and everything that drives cargo through ci/ self-tests. check-cargo-unit-default: @@ -1740,7 +1748,7 @@ check-cargo-unit-default: # The `hostsim` unit suite (the host feed seam exists only there). check-cargo-unit-hostsim: - @# The SAME suite as `check-cargo-unit-default`'s `cargo test --lib`, again under `hostsim`, which is + @# The crates of HOSTSIM_SUITE (the ones that declare `hostsim`) again under `hostsim`, which is @# not a duplicate run: the host feed seam (`play-feed/src/ffi_host.rs`) only exists in that configuration, @# so every test that drives an AU through `sf_feed` is COMPILED OUT of the default pass and cannot fail it. The prime-livelock @# regression is one of those, and it guards the 94-second freeze of 2026-08-29 — a defect that @@ -1751,7 +1759,7 @@ check-cargo-unit-hostsim: cd rust-modules && PATH="$$HOME/.cargo/bin:$$PATH" PLXNATIVE_RUNTIME_DIR="$$d" \ $(TELEMETRY_ENV) \ $(HOST_ENV) \ - $(UNIT_RUN) -- cargo +$(RUST_TOOLCHAIN) test --lib -p plxnative-modules -p plx_base -p plx_machine -p plx_platform -p plx_gfx -p plx_net -p plx_ui -p plx_plex -p plx_telemetry -p plx_data -p plx_session -p plx_media -p plx_playback_contract -p plx_play_uri -p plx_play_feed -p plx_media_codecs -p plx_appkit -p plx_screens --features hostsim + $(UNIT_RUN) -- cargo +$(RUST_TOOLCHAIN) test --lib -p plxnative-modules -p plx_base -p plx_machine -p plx_platform -p plx_gfx -p plx_plex -p plx_telemetry -p plx_data -p plx_session -p plx_media -p plx_play_feed --features hostsim # OPT-IN incremental inner loop: the default-feature unit suite (the same `cargo test --lib` as # `check-cargo-unit-default`, same throwaway runtime root and telemetry env) with diff --git a/ci/test_ci_split.py b/ci/test_ci_split.py index 70ae28dadc..f6e04d180c 100644 --- a/ci/test_ci_split.py +++ b/ci/test_ci_split.py @@ -64,7 +64,7 @@ def rule(target): r"ci/test_no_host_staticlib\.py", r"ci/test_build_not_always_dirty\.py", ], - "check-cargo-unit-hostsim": [r"cargo \+\$\(RUST_TOOLCHAIN\) test --lib -p plxnative-modules -p plx_base -p plx_machine -p plx_platform -p plx_gfx -p plx_net -p plx_ui -p plx_plex -p plx_telemetry -p plx_data -p plx_session -p plx_media -p plx_playback_contract -p plx_play_uri -p plx_play_feed -p plx_media_codecs -p plx_appkit -p plx_screens --features hostsim"], + "check-cargo-unit-hostsim": [r"cargo \+\$\(RUST_TOOLCHAIN\) test --lib -p plxnative-modules -p plx_base -p plx_machine -p plx_platform -p plx_gfx -p plx_plex -p plx_telemetry -p plx_data -p plx_session -p plx_media -p plx_play_feed --features hostsim"], } diff --git a/ci/test_test_crate.py b/ci/test_test_crate.py index ad5bd552aa..e93b761fc9 100644 --- a/ci/test_test_crate.py +++ b/ci/test_test_crate.py @@ -200,11 +200,42 @@ def test_the_suite_variable_is_the_recipes_package_list(self): text = (ROOT / "Makefile").read_text() suite = re.search(r"^UNIT_SUITE\s*=\s*(.+)$", text, flags=re.M).group(1).split() flags = " ".join(f"-p {p}" for p in suite) - for target in ("check-cargo-unit-default", "check-cargo-unit-hostsim", "test-fast"): + for target in ("check-cargo-unit-default", "test-fast"): start = text.index(f"\n{target}:") recipe = text[start:start + 6000].split("\n\n")[0] self.assertIn(f"test --lib {flags}", recipe, target) + def test_the_hostsim_pass_is_exactly_the_crates_that_declare_hostsim(self): + """The `hostsim` pass runs HOSTSIM_SUITE, a subset of UNIT_SUITE (`make check`'s first run of + every binary is the default pass). The crates left out must be ones the feature cannot reach: no + `hostsim` feature in the manifest and no `feature = "hostsim"` in the source. A crate that + gains either fails here until it is added to HOSTSIM_SUITE (and the recipe's `-p` list).""" + text = (ROOT / "Makefile").read_text() + unit = re.search(r"^UNIT_SUITE\s*=\s*(.+)$", text, flags=re.M).group(1).split() + hostsim = re.search(r"^HOSTSIM_SUITE\s*=\s*(.+)$", text, flags=re.M).group(1).split() + self.assertEqual(hostsim, [p for p in unit if p in hostsim], "HOSTSIM_SUITE is UNIT_SUITE's order") + self.assertTrue(set(hostsim) <= set(unit)) + start = text.index("\ncheck-cargo-unit-hostsim:") + recipe = text[start:start + 6000].split("\n\n")[0] + self.assertIn("test --lib " + " ".join(f"-p {p}" for p in hostsim) + " --features hostsim", recipe) + manifests = {"plxnative-modules": ROOT / "rust-modules"} + for member in (ROOT / "rust-modules").iterdir(): + manifest = member / "Cargo.toml" + if manifest.is_file(): + name = re.search(r'^name\s*=\s*"([^"]+)"', manifest.read_text(), flags=re.M) + if name: + manifests[name.group(1)] = member + for crate in unit: + root = manifests[crate] + declares = re.search(r"^hostsim\s*=", (root / "Cargo.toml").read_text(), flags=re.M) is not None + sources = (root / "src").rglob("*.rs") + cfgs = [str(f.relative_to(ROOT)) for f in sources if 'feature = "hostsim"' in f.read_text(errors="replace")] + if crate in hostsim: + self.assertTrue(declares, f"{crate} is in HOSTSIM_SUITE but declares no `hostsim` feature, which `--features hostsim -p {crate}` needs") + else: + self.assertFalse(declares, f"{crate} declares `hostsim`: add it to HOSTSIM_SUITE and the recipe") + self.assertEqual(cfgs, [], f"{crate} has a hostsim cfg: add it to HOSTSIM_SUITE and the recipe") + def test_check_never_names_test_crate(self): text = (ROOT / "Makefile").read_text() for target in ("check-cargo-unit-default", "check-cargo-unit-hostsim"): diff --git a/docs/agent-reference.md b/docs/agent-reference.md index af63bc2538..90ba8d598a 100644 --- a/docs/agent-reference.md +++ b/docs/agent-reference.md @@ -222,8 +222,11 @@ the pinned Sentry Native cross-build), and `sshpass` (Homebrew; deploy/run use y CI runs the three cargo targets plus those two as five parallel jobs (`host-lint`, `host-unit-default`, `host-unit-hostsim`, `host-python`, `host-python-harness`) behind an aggregator named `host checks (NOT a device gate)`; `ci/test_ci_split.py` pins that no gate falls between them. The cargo half runs `cargo test --lib -p plxnative-modules -p plx_base -p plx_machine -p plx_platform -p plx_gfx -p plx_net -p plx_ui -p plx_plex -p plx_telemetry -p plx_data -p plx_session -p plx_media -p plx_playback_contract -p plx_play_uri -p plx_play_feed -p plx_media_codecs -p plx_appkit -p plx_screens` - **twice: once on the default feature set and once with `--features hostsim`**, which is not a - duplicate run. The host feed seam (`play-feed/src/ffi_host.rs`) exists ONLY in the hostsim + **on the default feature set, and again with `--features hostsim` for the crates that declare it + (`HOSTSIM_SUITE`: all of `UNIT_SUITE` except `plx_net`, `plx_ui`, `plx_appkit`, `plx_screens`, + `plx_media_codecs`, `plx_play_uri` and `plx_playback_contract`, which have no `hostsim` cfg and run the + same test names in both passes; `ci/test_test_crate.py` fails if one gains the feature or a cfg)**, + which is not a duplicate run. The host feed seam (`play-feed/src/ffi_host.rs`) exists ONLY in the hostsim configuration, so every test that drives an access unit through `sf_feed` is compiled out of the default pass and cannot fail it — which is how the prime-livelock regression (`player::engine::prime_livelock_tests`) sat outside the gate entirely while 1398 default-feature From 0b4f4bfc1da829022550a3cbde39c89673c7863e Mon Sep 17 00:00:00 2001 From: Gleb Linnik Date: Sat, 14 Nov 2026 06:42:12 +0300 Subject: [PATCH 2/2] Check: say exactly what the reduced hostsim pass drops and why The Makefile comment said the second run of the seven left-out crates repeated the default pass, and the guard docstring called them crates the feature cannot reach. Both were inexact: the old pass linked those crates' test binaries against hostsim-built plx_base, plx_gfx, plx_platform, plx_media and plx_play_feed (ENV_STEERABLE paths, glyph-cache sizes, ffi_host present). The crates' own test set is identical in the two passes and none of their tests observes those dependency-side differences, which the guard does not check; it checks that the crate declares no hostsim feature and has no feature = "hostsim" cfg in its src. Reword both. docs/agent-reference.md's testing section still said make check runs the whole host suite a second time under hostsim; it now says the crates that declare hostsim (HOSTSIM_SUITE) do. Co-Authored-By: Claude Opus 5.5 --- Makefile | 8 ++++++-- ci/test_test_crate.py | 10 +++++++--- docs/agent-reference.md | 7 ++++--- 3 files changed, 17 insertions(+), 8 deletions(-) diff --git a/Makefile b/Makefile index 85c6ba9191..bb7d53d69a 100644 --- a/Makefile +++ b/Makefile @@ -1694,8 +1694,12 @@ UNIT_SUITE = plxnative-modules plx_base plx_machine plx_platform plx_gfx plx_net # do not (plx_net, plx_ui, plx_appkit, plx_screens, plx_media_codecs, plx_play_uri, plx_playback_contract) # have no `feature = "hostsim"` in their source and the same test names in both passes (14 crates did in # CI run 43528579834; the three that differ are `plxnative-modules`, `plx_media` and `plx_play_feed`, -# all kept), so the second run of their binaries repeated the default pass. The same test also fails -# the gate when one of them gains a `hostsim` cfg or feature, which is the moment it must join the list. +# all kept). Their own test set is therefore identical in the two passes. The old second run did link +# their test binaries against hostsim-built `plx_base`, `plx_gfx`, `plx_platform`, `plx_media` and +# `plx_play_feed` (steerable `ENV_STEERABLE` paths, larger glyph-cache sizes, `ffi_host` present), but +# none of their tests observes those dependency-side differences; the guard does not check that, it +# checks that the crate declares no `hostsim` feature and has no `feature = "hostsim"` in its src, and +# fails when one gains either, which is the moment it must join the list. HOSTSIM_SUITE = plxnative-modules plx_base plx_machine plx_platform plx_gfx plx_plex plx_telemetry plx_data plx_session plx_media plx_play_feed # The default-feature unit suite and everything that drives cargo through ci/ self-tests. diff --git a/ci/test_test_crate.py b/ci/test_test_crate.py index e93b761fc9..f083c4ba58 100644 --- a/ci/test_test_crate.py +++ b/ci/test_test_crate.py @@ -207,9 +207,13 @@ def test_the_suite_variable_is_the_recipes_package_list(self): def test_the_hostsim_pass_is_exactly_the_crates_that_declare_hostsim(self): """The `hostsim` pass runs HOSTSIM_SUITE, a subset of UNIT_SUITE (`make check`'s first run of - every binary is the default pass). The crates left out must be ones the feature cannot reach: no - `hostsim` feature in the manifest and no `feature = "hostsim"` in the source. A crate that - gains either fails here until it is added to HOSTSIM_SUITE (and the recipe's `-p` list).""" + every binary is the default pass). A crate left out has an identical test set in both passes: + no `hostsim` feature in its manifest and no `feature = "hostsim"` in its source. What the guard + does not check is the dependency side: the old pass linked those crates' test binaries + against hostsim-built `plx_base`, `plx_gfx`, `plx_platform`, `plx_media` and `plx_play_feed` + (`ENV_STEERABLE` paths, glyph-cache sizes, `ffi_host` present), and none of their tests + observes those differences. A crate that gains the feature or a cfg fails here until it is + added to HOSTSIM_SUITE (and the recipe's `-p` list).""" text = (ROOT / "Makefile").read_text() unit = re.search(r"^UNIT_SUITE\s*=\s*(.+)$", text, flags=re.M).group(1).split() hostsim = re.search(r"^HOSTSIM_SUITE\s*=\s*(.+)$", text, flags=re.M).group(1).split() diff --git a/docs/agent-reference.md b/docs/agent-reference.md index 90ba8d598a..6bd6f0005f 100644 --- a/docs/agent-reference.md +++ b/docs/agent-reference.md @@ -1940,9 +1940,10 @@ There **is** a host unit suite, and it is not the real gate — both halves matt them is how this section used to be wrong in three files at once. **Tier 1 — `make check` (host).** `cd rust-modules && cargo test --lib -p plxnative-modules -p plx_base -p plx_machine -p plx_platform -p plx_gfx -p plx_net -p plx_ui -p plx_plex -p plx_telemetry -p plx_data -p plx_session -p plx_media -p plx_playback_contract -p plx_play_uri -p plx_play_feed -p plx_media_codecs -p plx_appkit -p plx_screens` (a bare `cargo test --lib` runs the application crate only and skips every layer crate: `plx_base`, `plx_machine`, `plx_platform`, `plx_gfx`, `plx_net`, `plx_ui`, `plx_plex`, `plx_telemetry`, `plx_data`, `plx_session`, `plx_media`, `plx_appkit`, `plx_screens`) runs the whole -host suite on the dev Mac, no TV involved — and `make check` runs it a SECOND time under -`--features hostsim`, because the host feed seam only exists there and the tests that need it are -compiled out of the first pass (see the build section). **Treat every test COUNT in this section as +host suite on the dev Mac, no TV involved — and `make check` runs the crates that declare `hostsim` +(`HOSTSIM_SUITE`, a subset of that list) a SECOND time under `--features hostsim`, because the host +feed seam only exists there and the tests that need it are compiled out of the first pass (the build +section's paragraph on the two passes says which crates are left out and why). **Treat every test COUNT in this section as already wrong, including the one in this sentence.** 386 measured 2026-08-13; 284 on 2026-08-02; a documented 59 before that, which was five times stale before anyone noticed — and the first version of this paragraph was stale within one *commit*, because two agents were adding tests to the same