Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
16 changes: 14 additions & 2 deletions Makefile
Original file line number Diff line number Diff line change
Expand Up @@ -1689,6 +1689,18 @@ check-cargo-lint: lint
# ci/test_test_crate.py pins the two equal.
UNIT_RUN = python3 ../tools/cargo-test-parallel.py
UNIT_SUITE = plxnative-modules plx_base plx_machine plx_platform plx_gfx plx_net plx_ui plx_plex plx_telemetry plx_data plx_session plx_media plx_playback_contract plx_play_uri plx_play_feed plx_media_codecs plx_appkit plx_screens
# The `hostsim` pass runs only the crates that DECLARE a `hostsim` feature (HOSTSIM_SUITE, a subset of
# UNIT_SUITE in the same order; ci/test_test_crate.py pins both the subset and the recipe). The seven that
# do not (plx_net, plx_ui, plx_appkit, plx_screens, plx_media_codecs, plx_play_uri, plx_playback_contract)
# have no `feature = "hostsim"` in their source and the same test names in both passes (14 crates did in
# CI run 43528579834; the three that differ are `plxnative-modules`, `plx_media` and `plx_play_feed`,
# all kept). Their own test set is therefore identical in the two passes. The old second run did link
# their test binaries against hostsim-built `plx_base`, `plx_gfx`, `plx_platform`, `plx_media` and
# `plx_play_feed` (steerable `ENV_STEERABLE` paths, larger glyph-cache sizes, `ffi_host` present), but
# none of their tests observes those dependency-side differences; the guard does not check that, it
# checks that the crate declares no `hostsim` feature and has no `feature = "hostsim"` in its src, and
# fails when one gains either, which is the moment it must join the list.
HOSTSIM_SUITE = plxnative-modules plx_base plx_machine plx_platform plx_gfx plx_plex plx_telemetry plx_data plx_session plx_media plx_play_feed

# The default-feature unit suite and everything that drives cargo through ci/ self-tests.
check-cargo-unit-default:
Expand Down Expand Up @@ -1740,7 +1752,7 @@ check-cargo-unit-default:

# The `hostsim` unit suite (the host feed seam exists only there).
check-cargo-unit-hostsim:
@# The SAME suite as `check-cargo-unit-default`'s `cargo test --lib`, again under `hostsim`, which is
@# The crates of HOSTSIM_SUITE (the ones that declare `hostsim`) again under `hostsim`, which is
@# not a duplicate run: the host feed seam (`play-feed/src/ffi_host.rs`) only exists in that configuration,
@# so every test that drives an AU through `sf_feed` is COMPILED OUT of the default pass and cannot fail it. The prime-livelock
@# regression is one of those, and it guards the 94-second freeze of 2026-08-29 — a defect that
Expand All @@ -1751,7 +1763,7 @@ check-cargo-unit-hostsim:
cd rust-modules && PATH="$$HOME/.cargo/bin:$$PATH" PLXNATIVE_RUNTIME_DIR="$$d" \
$(TELEMETRY_ENV) \
$(HOST_ENV) \
$(UNIT_RUN) -- cargo +$(RUST_TOOLCHAIN) test --lib -p plxnative-modules -p plx_base -p plx_machine -p plx_platform -p plx_gfx -p plx_net -p plx_ui -p plx_plex -p plx_telemetry -p plx_data -p plx_session -p plx_media -p plx_playback_contract -p plx_play_uri -p plx_play_feed -p plx_media_codecs -p plx_appkit -p plx_screens --features hostsim
$(UNIT_RUN) -- cargo +$(RUST_TOOLCHAIN) test --lib -p plxnative-modules -p plx_base -p plx_machine -p plx_platform -p plx_gfx -p plx_plex -p plx_telemetry -p plx_data -p plx_session -p plx_media -p plx_play_feed --features hostsim

# OPT-IN incremental inner loop: the default-feature unit suite (the same `cargo test --lib` as
# `check-cargo-unit-default`, same throwaway runtime root and telemetry env) with
Expand Down
2 changes: 1 addition & 1 deletion ci/test_ci_split.py
Original file line number Diff line number Diff line change
Expand Up @@ -64,7 +64,7 @@ def rule(target):
r"ci/test_no_host_staticlib\.py",
r"ci/test_build_not_always_dirty\.py",
],
"check-cargo-unit-hostsim": [r"cargo \+\$\(RUST_TOOLCHAIN\) test --lib -p plxnative-modules -p plx_base -p plx_machine -p plx_platform -p plx_gfx -p plx_net -p plx_ui -p plx_plex -p plx_telemetry -p plx_data -p plx_session -p plx_media -p plx_playback_contract -p plx_play_uri -p plx_play_feed -p plx_media_codecs -p plx_appkit -p plx_screens --features hostsim"],
"check-cargo-unit-hostsim": [r"cargo \+\$\(RUST_TOOLCHAIN\) test --lib -p plxnative-modules -p plx_base -p plx_machine -p plx_platform -p plx_gfx -p plx_plex -p plx_telemetry -p plx_data -p plx_session -p plx_media -p plx_play_feed --features hostsim"],
}


Expand Down
37 changes: 36 additions & 1 deletion ci/test_test_crate.py
Original file line number Diff line number Diff line change
Expand Up @@ -200,11 +200,46 @@ def test_the_suite_variable_is_the_recipes_package_list(self):
text = (ROOT / "Makefile").read_text()
suite = re.search(r"^UNIT_SUITE\s*=\s*(.+)$", text, flags=re.M).group(1).split()
flags = " ".join(f"-p {p}" for p in suite)
for target in ("check-cargo-unit-default", "check-cargo-unit-hostsim", "test-fast"):
for target in ("check-cargo-unit-default", "test-fast"):
start = text.index(f"\n{target}:")
recipe = text[start:start + 6000].split("\n\n")[0]
self.assertIn(f"test --lib {flags}", recipe, target)

def test_the_hostsim_pass_is_exactly_the_crates_that_declare_hostsim(self):
"""The `hostsim` pass runs HOSTSIM_SUITE, a subset of UNIT_SUITE (`make check`'s first run of
every binary is the default pass). A crate left out has an identical test set in both passes:
no `hostsim` feature in its manifest and no `feature = "hostsim"` in its source. What the guard
does not check is the dependency side: the old pass linked those crates' test binaries
against hostsim-built `plx_base`, `plx_gfx`, `plx_platform`, `plx_media` and `plx_play_feed`
(`ENV_STEERABLE` paths, glyph-cache sizes, `ffi_host` present), and none of their tests
observes those differences. A crate that gains the feature or a cfg fails here until it is
added to HOSTSIM_SUITE (and the recipe's `-p` list)."""
text = (ROOT / "Makefile").read_text()
unit = re.search(r"^UNIT_SUITE\s*=\s*(.+)$", text, flags=re.M).group(1).split()
hostsim = re.search(r"^HOSTSIM_SUITE\s*=\s*(.+)$", text, flags=re.M).group(1).split()
self.assertEqual(hostsim, [p for p in unit if p in hostsim], "HOSTSIM_SUITE is UNIT_SUITE's order")
self.assertTrue(set(hostsim) <= set(unit))
start = text.index("\ncheck-cargo-unit-hostsim:")
recipe = text[start:start + 6000].split("\n\n")[0]
self.assertIn("test --lib " + " ".join(f"-p {p}" for p in hostsim) + " --features hostsim", recipe)
manifests = {"plxnative-modules": ROOT / "rust-modules"}
for member in (ROOT / "rust-modules").iterdir():
manifest = member / "Cargo.toml"
if manifest.is_file():
name = re.search(r'^name\s*=\s*"([^"]+)"', manifest.read_text(), flags=re.M)
if name:
manifests[name.group(1)] = member
for crate in unit:
root = manifests[crate]
declares = re.search(r"^hostsim\s*=", (root / "Cargo.toml").read_text(), flags=re.M) is not None
sources = (root / "src").rglob("*.rs")
cfgs = [str(f.relative_to(ROOT)) for f in sources if 'feature = "hostsim"' in f.read_text(errors="replace")]
if crate in hostsim:
self.assertTrue(declares, f"{crate} is in HOSTSIM_SUITE but declares no `hostsim` feature, which `--features hostsim -p {crate}` needs")
else:
self.assertFalse(declares, f"{crate} declares `hostsim`: add it to HOSTSIM_SUITE and the recipe")
self.assertEqual(cfgs, [], f"{crate} has a hostsim cfg: add it to HOSTSIM_SUITE and the recipe")

def test_check_never_names_test_crate(self):
text = (ROOT / "Makefile").read_text()
for target in ("check-cargo-unit-default", "check-cargo-unit-hostsim"):
Expand Down
14 changes: 9 additions & 5 deletions docs/agent-reference.md
Original file line number Diff line number Diff line change
Expand Up @@ -222,8 +222,11 @@ the pinned Sentry Native cross-build), and `sshpass` (Homebrew; deploy/run use y
CI runs the three cargo targets plus those two as five parallel jobs
(`host-lint`, `host-unit-default`, `host-unit-hostsim`, `host-python`, `host-python-harness`) behind an aggregator named
`host checks (NOT a device gate)`; `ci/test_ci_split.py` pins that no gate falls between them. The cargo half runs `cargo test --lib -p plxnative-modules -p plx_base -p plx_machine -p plx_platform -p plx_gfx -p plx_net -p plx_ui -p plx_plex -p plx_telemetry -p plx_data -p plx_session -p plx_media -p plx_playback_contract -p plx_play_uri -p plx_play_feed -p plx_media_codecs -p plx_appkit -p plx_screens`
**twice: once on the default feature set and once with `--features hostsim`**, which is not a
duplicate run. The host feed seam (`play-feed/src/ffi_host.rs`) exists ONLY in the hostsim
**on the default feature set, and again with `--features hostsim` for the crates that declare it
(`HOSTSIM_SUITE`: all of `UNIT_SUITE` except `plx_net`, `plx_ui`, `plx_appkit`, `plx_screens`,
`plx_media_codecs`, `plx_play_uri` and `plx_playback_contract`, which have no `hostsim` cfg and run the
same test names in both passes; `ci/test_test_crate.py` fails if one gains the feature or a cfg)**,
which is not a duplicate run. The host feed seam (`play-feed/src/ffi_host.rs`) exists ONLY in the hostsim
configuration, so every test that drives an access unit through `sf_feed` is compiled out of the
default pass and cannot fail it — which is how the prime-livelock regression
(`player::engine::prime_livelock_tests`) sat outside the gate entirely while 1398 default-feature
Expand Down Expand Up @@ -1937,9 +1940,10 @@ There **is** a host unit suite, and it is not the real gate — both halves matt
them is how this section used to be wrong in three files at once.

**Tier 1 — `make check` (host).** `cd rust-modules && cargo test --lib -p plxnative-modules -p plx_base -p plx_machine -p plx_platform -p plx_gfx -p plx_net -p plx_ui -p plx_plex -p plx_telemetry -p plx_data -p plx_session -p plx_media -p plx_playback_contract -p plx_play_uri -p plx_play_feed -p plx_media_codecs -p plx_appkit -p plx_screens` (a bare `cargo test --lib` runs the application crate only and skips every layer crate: `plx_base`, `plx_machine`, `plx_platform`, `plx_gfx`, `plx_net`, `plx_ui`, `plx_plex`, `plx_telemetry`, `plx_data`, `plx_session`, `plx_media`, `plx_appkit`, `plx_screens`) runs the whole
host suite on the dev Mac, no TV involved — and `make check` runs it a SECOND time under
`--features hostsim`, because the host feed seam only exists there and the tests that need it are
compiled out of the first pass (see the build section). **Treat every test COUNT in this section as
host suite on the dev Mac, no TV involved — and `make check` runs the crates that declare `hostsim`
(`HOSTSIM_SUITE`, a subset of that list) a SECOND time under `--features hostsim`, because the host
feed seam only exists there and the tests that need it are compiled out of the first pass (the build
section's paragraph on the two passes says which crates are left out and why). **Treat every test COUNT in this section as
already wrong, including the one in this sentence.** 386 measured 2026-08-13; 284 on 2026-08-02; a
documented 59 before that, which was five times stale before anyone noticed — and the first version
of this paragraph was stale within one *commit*, because two agents were adding tests to the same
Expand Down
Loading