Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
18 changes: 11 additions & 7 deletions .github/actions/host-rust/action.yml
Original file line number Diff line number Diff line change
Expand Up @@ -44,17 +44,21 @@ runs:

- name: Build the host crates with line tables only
# Nothing in CI attaches a debugger to a host test binary, and a panic backtrace needs only the
# line table. The default `dev`/`test` profile writes full DWARF for every crate, dependencies
# included, which is most of what rustc's codegen, the linker and the cache archive carry for
# the 18 test binaries. This is the per-user cargo config, so it reaches the bare `cargo`
# lines and every `make` target of the host jobs alike. A dependency built with the old
# line table. A `dev` build writes full DWARF for every crate, dependencies included, which is
# most of what rustc's codegen, the linker and the cache archive carry (the 18 test binaries
# too, before rust-modules/Cargo.toml gave `[profile.test]` line tables of its own: `cargo test`
# builds with `test`, which inherits `dev`). This is the per-user cargo config, so it reaches
# the bare `cargo` lines and every `make` target of the host jobs alike, and it trims what the
# manifest does not: the `dev` builds (clippy, `cargo check`) and the third-party crates'
# debug info (`debug = false`). A dependency built with the old
# profile is a different unit to cargo, and rust-cache's key does not see this file (it hashes
# the manifests, the lockfile, the toolchain files and the workspace's `.cargo/config.toml`,
# not the home one), so the cache step below carries its own `prefix-key`: without it the
# full-DWARF cache restores as an exact hit, every dependency is rebuilt, and the post step
# finds the cache up to date and saves nothing. It is NOT the manifest: a developer's checkout
# keeps full DWARF (docs/agent-reference.md, the lane policy), and the profiles the ARM binary
# is built with do not inherit `dev`, so the shipped library is untouched.
# finds the cache up to date and saves nothing. The `dev` half is NOT in the manifest: a developer's checkout
# keeps full DWARF in `dev` builds (docs/agent-reference.md, the lane policy; its host tests
# get line tables from the manifest's `[profile.test]`), and the profiles the ARM binary is
# built with do not inherit `dev`, so the shipped library is untouched.
shell: bash
run: |
mkdir -p "$HOME/.cargo"
Expand Down
8 changes: 5 additions & 3 deletions .github/workflows/ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -446,9 +446,11 @@ jobs:
- name: make check-python-rest
run: make check-python-rest

# tests/test_harness.py alone: about 190 s of what used to be a 360 s `check-python` job. Most of
# it is its `DepGates` class running ci/check-deps.sh over private copies of the tree on every
# core of the runner, so it gains nothing from sharing a runner with the other gates.
# tests/test_harness.py alone, split off a 360 s `check-python` job. Most of it is its `DepGates`
# class: about 70 self-tests that each run only the rules they plant a violation for
# (`ci/check-deps.sh --only`, #1386) over private copies of the tree, spread over every core of
# the runner, so it gains nothing from sharing a runner with the other gates. The step took
# 144-196 s on main's commits of 2026-11-14 after #1386 (380-500 s on the ones before it).
# `make check-python` is still the two halves, run side by side locally.
host-python-harness:
name: host harness tests
Expand Down
18 changes: 9 additions & 9 deletions Makefile
Original file line number Diff line number Diff line change
Expand Up @@ -1957,15 +1957,15 @@ check-python:
harness='$(MAKE) --no-print-directory check-python-harness'

check-python-harness:
@# The harness's own host unit tests (tests/test_harness.py, stdlib unittest). THE MOST
@# EXPENSIVE STEP IN `check` BY FAR — 661 s of a 1100 s run, measured 2026-10-01 on a cold lane
@# at load ~5, and 630 s of that is the `DepGates` class running the whole of
@# `ci/check-deps.sh` (11 s a run) once or more per test to prove each structure gate still
@# catches a planted violation. Those self-tests now edit a private copy of the gate's inputs
@# and run on a thread pool (`PLX_TEST_JOBS`, default min(8, cpus)): the module takes 175 s
@# instead of 661 s. (It was 980 s of 1217 s before `check-deps.sh` stopped forking a process
@# per candidate line.) Measure before budgeting, and if this number needs to come down
@# further, `check-deps.sh` itself is the place. run.py
@# The harness's own host unit tests (tests/test_harness.py, stdlib unittest): the harness
@# branch of `check-python` and the whole of CI's `host-python-harness` job. Most of it is the
@# `DepGates` class: about 70 self-tests, each planting ONE violation in a private copy of the
@# gate's inputs and running `ci/check-deps.sh --only <the rules it plants for>` to prove that
@# rule still catches it (since #1386 a self-test runs its own rules, not all 43), on a thread
@# pool (`PLX_TEST_JOBS`, default min(8, cpus)). Measured 2026-11-14 on CI: this step took
@# 144-196 s on the commits since #1386 and 380-500 s on the ones before it. Measure before
@# budgeting; `check-deps.sh`'s per-rule cost, then the self-test count, is where more would
@# come from. run.py
@# decides WHAT gets driven on the one television and had no test of any kind until 2026-08-22.
@# What it pins is the code path a full manifest.local.json never enters: an `item` key this
@# installation cannot resolve SKIPS the cases that need it instead of killing the run. A
Expand Down
2 changes: 1 addition & 1 deletion ci/proofs/platform-generated-enums.json
Original file line number Diff line number Diff line change
Expand Up @@ -26,7 +26,7 @@
},
"text_hashing": "sha256-utf8-bytes-crlf-to-lf-local-root-version-v1",
"manifests": {
"Cargo.toml": "fbeb900767c6d3a7091b0366cdaf79753f45384be9b39a649dd8e7decbf40fb6",
"Cargo.toml": "c57b1c2d662d460a6d71f6de158397f1cb2f2fecb167bfa738631d77b661248b",
"appkit/Cargo.toml": "1e3a33f392110f269537b12f12879c6bc15309aca304b189d6b0346edf8a1186",
"base/Cargo.toml": "ebe0b95553bd44f33f422f7d8e22b75ddeb53c7ee81bbe738f675690694568f7",
"data/Cargo.toml": "90a763a8a9e22bda7e95f8ede4dc10ec6e92a62b90bd0fd2e622c617d75ec6c7",
Expand Down
4 changes: 2 additions & 2 deletions ci/proofs/serde-json-immutable.json
Original file line number Diff line number Diff line change
Expand Up @@ -373,7 +373,7 @@
},
"lock_sha256": "c56af60124266be5ee0b7dfa7ae81b4f0ff38b047fe672f40a009f6e1c22d3d3",
"manifests": {
"Cargo.toml": "fbeb900767c6d3a7091b0366cdaf79753f45384be9b39a649dd8e7decbf40fb6",
"Cargo.toml": "c57b1c2d662d460a6d71f6de158397f1cb2f2fecb167bfa738631d77b661248b",
"appkit/Cargo.toml": "1e3a33f392110f269537b12f12879c6bc15309aca304b189d6b0346edf8a1186",
"base/Cargo.toml": "ebe0b95553bd44f33f422f7d8e22b75ddeb53c7ee81bbe738f675690694568f7",
"data/Cargo.toml": "90a763a8a9e22bda7e95f8ede4dc10ec6e92a62b90bd0fd2e622c617d75ec6c7",
Expand Down Expand Up @@ -602,7 +602,7 @@
],
"edition": "2021",
"lib": "ui/src/lib.rs",
"lib_sha256": "c77775a6faa9583579ee576ea9f9a51a7b9c92d7ef692161761c1e471faf3873"
"lib_sha256": "616bd784cbf479441275a483f83ed2d88df05db406731bb2ead7a8d70845e557"
}
},
"text_hashing": "sha256-utf8-bytes-crlf-to-lf-local-root-version-v1"
Expand Down
2 changes: 1 addition & 1 deletion ci/readonly_dependencies.py
Original file line number Diff line number Diff line change
Expand Up @@ -18,7 +18,7 @@
import tarfile

CERTIFICATE_PATH = Path(__file__).with_name('proofs') / 'serde-json-immutable.json'
CERTIFICATE_SHA256 = '7a299a63a440720a7f7313fe985519efb77b6f6a2963b5b0d92fc3bc3be987be'
CERTIFICATE_SHA256 = '29e62dc168d2691cebe44a1ce21b59d186002fed59df18bdb2c2b86d76f0497d'
SCHEMA = 'plx.readonly.locked-json.v1'
REGISTRY = 'registry+https://github.com/rust-lang/crates.io-index'
SUPPORTED = ['alloc', 'arbitrary_precision', 'default', 'float_roundtrip', 'raw_value', 'std', 'unbounded_depth']
Expand Down
2 changes: 1 addition & 1 deletion ci/readonly_generated.py
Original file line number Diff line number Diff line change
Expand Up @@ -13,7 +13,7 @@
from readonly_dependencies import _local_sha, _sha, _json, _manifest_paths, _configs, _workspace_hashes, WORKSPACE_HASHING

CERTIFICATE_PATH = Path(__file__).with_name('proofs') / 'platform-generated-enums.json'
CERTIFICATE_SHA256 = 'b61be9b470960a80bbb097dfa466fa5fd4229f89ef8f1234bb302edabf9f8396'
CERTIFICATE_SHA256 = 'c737368bf0642a51f34c3424f9545eaa8becb72dc2206ae5f1a40ddcc3c7f67e'
SOURCES = ['platform/Cargo.toml', 'platform/src/lib.rs', 'platform/src/i18n/mod.rs',
'platform/build.rs', 'platform/build_support/catalog.rs', 'build_support/install_identities.rs',
'machine/src/lib.rs', 'machine/src/machine.rs']
Expand Down
5 changes: 4 additions & 1 deletion docs/agent-reference.md
Original file line number Diff line number Diff line change
Expand Up @@ -753,7 +753,10 @@ the pinned Sentry Native cross-build), and `sshpass` (Homebrew; deploy/run use y
`tools/build-gc.sh` now installs `.claude/worktrees/.cargo/config.toml` with
`incremental = false`, which every cargo reads and which stops above the main checkout. Since
2026-09-18 the same file also sets `[profile.dev] debug = "line-tables-only"` and `debug = false`
for third-party packages in lanes (main keeps full DWARF). A lane's target dirs are also no longer
for third-party packages in lanes (main keeps full DWARF for `dev`: the simulator, clippy and
`cargo check`). A host `cargo test` build is the `test` profile, whose line tables
`rust-modules/Cargo.toml` sets for every checkout, main included; a debugger on a host test needs
`CARGO_PROFILE_TEST_DEBUG=2`. A lane's target dirs are also no longer
all built from scratch: in a linked worktree the Makefile seeds an ABSENT target dir with an APFS
clone of the third-party output (registry crates, once also the build-std sysroot, about 40% of a lane's
target bytes then) from `~/.cache/plxnative/cargo-seed/`, via `tools/cargo-seed.py`; the app crate is
Expand Down
4 changes: 2 additions & 2 deletions docs/module-layers.md
Original file line number Diff line number Diff line change
Expand Up @@ -1941,8 +1941,8 @@ The last no-LTO figure is over the 12,426,000 limit, which made Splits 9 and 10
because whole-program optimisation with a single codegen unit sees the same code the single crate
did. They also restore the cross-crate inlining the single crate had, which the per-frame UI code
leans on. Only release-profile builds pay for it (the ARM staticlib and the storage helper when built as `release`, the
Linux simulator of `make sim-linux` and `make macapp`); the dev profile that `make check`, the host
tests and the macOS simulator use is untouched. The price is build time: CI's ARM library build
Linux simulator of `make sim-linux` and `make macapp`); the `dev` and `test` profiles that the macOS
simulator, `make check` and the host tests use are untouched. The price is build time: CI's ARM library build
went from 2m40 to 4m15, and a local `ARM_PROFILE=release` one from 44 s to 94 s. A plain local `make` / `make deploy` no
longer pays it: the Makefile builds the ARM staticlib with `[profile.tvdev]` (no LTO, 16 codegen
units), which is faster and larger, and keeps `release` for CI, `RELEASE=1`, `SYMBOLS=1` and
Expand Down
25 changes: 22 additions & 3 deletions rust-modules/Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -299,9 +299,9 @@ serde = { version = "1", default-features = false, features = ["derive", "alloc"
# back the cross-crate inlining that the UI's per-frame code relied on.
#
# What it costs is build time, paid only by release-profile builds: CI's ARM library build went from
# 2m40 to 4m15 and a local `ARM_PROFILE=release` one from 44 s to 94 s. `make check` and the host tests run the dev profile
# and do not pay it, and neither does a plain local `make` / `make deploy`: the Makefile builds those
# with `[profile.tvdev]` below. Release-profile builds are CI, `RELEASE=1`, `SYMBOLS=1`,
# 2m40 to 4m15 and a local `ARM_PROFILE=release` one from 44 s to 94 s. `make check` and the host tests (the `test`
# profile, which inherits `dev`) do not pay it, and neither does a plain local `make` / `make deploy`:
# the Makefile builds those with `[profile.tvdev]` below. Release-profile builds are CI, `RELEASE=1`, `SYMBOLS=1`,
# `FLAVOR=stable|nightly` and `make ARM_PROFILE=release`.
[profile.release]
opt-level = 2
Expand All @@ -321,6 +321,25 @@ inherits = "release"
lto = "off"
codegen-units = 16

# The host UNIT-TEST build (`cargo test`: `make check`'s `check-cargo-unit-*`, `make test-fast`,
# `make test-crate`, `make test-hostsim`, `make build-bench`): every unit of such a build, the
# workspace libraries and the third-party crates under the test binaries included, is built with
# THIS profile and not with `dev` (`cargo test --unit-graph` names the profile of each unit). Line
# tables keep a panic backtrace's file:line and drop the variable-level DWARF; nothing here attaches
# a debugger to a test binary. Measured 2026-11-14 on a checkout with full-DWARF `dev` (the main
# checkout's setting), `make build-bench`, medians: the non-incremental edit rebuild of the test
# build took 14-22% less (plx_base leaf 39.8-43.5 s -> 33.9-35.9 s; plx_ui leaf 21.7-23.4 s ->
# 18.5-20.1 s) and `target/` + `target-fast/` shrank 19% (9.6 GB -> 7.8 GB); the incremental
# `make test-fast` loop was not clearly moved (a plx_ui edit 6.0 -> 6.2 s, a plx_base edit 16.1-21.8 s
# -> 15.5-16.1 s with the baseline drifting between runs). `[profile.dev]` is untouched: the
# simulator, clippy and `cargo check` keep full DWARF in a developer's checkout. Lanes and CI already
# trim `dev` through a cargo config (tools/build-gc.sh, .github/actions/host-rust/action.yml),
# which `test` inherits, so for them this line changes nothing (the unit graph is identical); the
# main checkout is where it takes effect. A debugger on a host test needs
# `CARGO_PROFILE_TEST_DEBUG=2` (`CARGO_PROFILE_DEV_DEBUG` no longer reaches a `cargo test` build).
[profile.test]
debug = "line-tables-only"

[dev-dependencies]
# The `cfg(test)` fixtures of the base layer (`testlock`, `testnet`, ...): a dependency is never
# built `cfg(test)`, so the tests here ask for them by feature.
Expand Down
32 changes: 20 additions & 12 deletions tools/build-gc.sh
Original file line number Diff line number Diff line change
Expand Up @@ -107,8 +107,8 @@ install_worktree_cargo_policy() {
#
# Cargo merges config upward from the working directory, so this file applies to every linked
# worktree under `.claude/worktrees/` and to NOTHING else: the main checkout lives above this
# directory and keeps its incremental cache and full DWARF, which is the policy the Makefile
# states beside `RUST_FEATFLAGS`. The Makefile can only enforce it for its own cargo invocations;
# directory and keeps its incremental cache and full DWARF in `dev` builds, which is the policy the
# Makefile states beside `RUST_FEATFLAGS`. The Makefile can only enforce it for its own cargo invocations;
# an agent running `cargo test` or `cargo check` by hand bypassed it, and 12.9 GB of lane
# incremental caches is what that cost.
#
Expand Down Expand Up @@ -137,17 +137,25 @@ incremental = false
# 11% cut to the whole tree and a 26% cut to the object files directly, from ONE crate's `--lib`
# closure. The saving is modest, plainly: most of the object-file bulk is code, not DWARF.
#
# `CARGO_PROFILE_DEV_DEBUG=2` in the environment still overrides this, the same escape hatch as
# `CARGO_INCREMENTAL=1` above: an env var outranks a config file, so a lane genuinely attaching
# `lldb`/`rust-lldb` to a host test can buy full DWARF back for itself without touching this file.
# `CARGO_PROFILE_DEV_DEBUG=2` in the environment still overrides this for a `dev` build (the
# simulator, clippy, `cargo check`), the same escape hatch as `CARGO_INCREMENTAL=1` above: an env
# var outranks a config file. A host TEST is another profile (since 2026-11-14): `cargo test` builds
# every unit, third-party crates included, with `[profile.test]`, and rust-modules/Cargo.toml sets
# that one to line tables itself, which a `[profile.dev]` setting (this file's or the
# environment's) no longer reaches. A lane genuinely attaching `lldb`/`rust-lldb` to a host test
# buys full DWARF back for itself with `CARGO_PROFILE_TEST_DEBUG=2`; third-party crates keep the
# `debug = false` below either way.
#
# Checked before landing that nothing else in the nearer config chain already sets this and would
# make the addition a no-op or a conflict: neither `rust-modules/Cargo.toml` (only
# `[profile.release]`) nor `rust-modules/.cargo/config.toml` (only `[target...]` rustflags and
# formerly `[unstable] build-std`) mentions `[profile.dev]` debug. Cargo resolves a manifest `[profile]`
# against a config `[profile]` by letting config win, and the nearest config wins over a farther
# one — both rules favour this file over anything upstream, which is the reason it is safe to add
# here rather than in the crate manifest.
# Checked before landing, for `dev` builds, that nothing else in the nearer config chain already
# sets this and would make the addition a no-op or a conflict: neither `rust-modules/Cargo.toml`
# (its profiles are `release`, `tvdev` and, since 2026-11-14, `test`; none is `dev`) nor
# `rust-modules/.cargo/config.toml` (only `[target...]` rustflags and formerly `[unstable]
# build-std`) mentions `[profile.dev]` debug. Cargo resolves a manifest `[profile]` against a
# config `[profile]` by letting config win, and the nearest config wins over a farther one — both
# rules favour this file over anything upstream, which is the reason it is safe to add here rather
# than in the crate manifest. A `cargo test` build takes its debug level from `[profile.test]`
# instead (the paragraph above), so for it the `debug` line below is redundant; the
# `debug = false` for third-party crates is what this file still adds there.
[profile.dev]
debug = "line-tables-only"
[profile.dev.package."*"]
Expand Down
2 changes: 1 addition & 1 deletion tools/cargo-seed.py
Original file line number Diff line number Diff line change
Expand Up @@ -26,7 +26,7 @@
* **It clones or it does nothing.** `clonefile(2)` is called directly (not `cp -c`, which quietly
falls back to a full copy, spending the very disk this exists to save). Another volume, a
filesystem without cloning, or any platform other than macOS: a no-op, never a copy.
* **Linked worktrees only.** The main checkout keeps its own incremental cache and full DWARF,
* **Linked worktrees only.** The main checkout keeps its own incremental cache and full DWARF in `dev` builds,
which no lane could share, so the Makefile does not call this there and the script refuses
too. `PLX_CARGO_SEED=off` disables it everywhere.
* **A key guards the seed**: the sha256 of the toolchain (`rustc +<pinned toolchain> -vV` and `rustc -vV`),
Expand Down
Loading