Skip to content

fix(build): stop two builds fighting over target/<profile>/deps/liboxidex.* - #649

Merged
swackhamer merged 3 commits into
mainfrom
claude/kind-matsumoto-f90315
Aug 10, 2026
Merged

fix(build): stop two builds fighting over target/<profile>/deps/liboxidex.*#649
swackhamer merged 3 commits into
mainfrom
claude/kind-matsumoto-f90315

Conversation

@swackhamer

Copy link
Copy Markdown
Collaborator

Two builds were fighting over target/<profile>/deps/liboxidex.*. Both are the
same bug class: [lib] crate-type = ["lib", "staticlib", "cdylib"] makes cargo
drop -C extra-filename, so every configuration of the lib shares one set of
filenames and whichever writer finishes last wins.

1. The C FFI test's nested build (finishes #639)

#639 stopped tests/ffi_c_integration.rs's nested cargo build --lib from
building the default feature graph over the rlib the doctests link — the
E0432 on both parsers::magika_detector doctests. It forwarded the feature
graph and recorded that the nested build now "writes nothing at all".

Measured on cargo test --all-features -v, that part is false. A nested cargo
inherits no profile either: it ran under dev while cargo test builds the
lib under test, which are different units here ([profile.test]
opt-level = 2, codegen-units = 4 vs [profile.dev]'s 0/16, read off the
rustc invocations). 44s into the FFI test it logged Compiling oxidex /
Finished `dev` profile, and target/debug/deps/liboxidex.rlib went
119,540,352 → 99,273,872 bytes underneath the doctests.

A test can't learn the profile name cargo invoked it under, so a nested build
can't be made to match one. It doesn't need to build at all: cargo builds the
lib as a dependency of every integration-test target, all three crate types in
one rustc call. The test now derives its link directory from
env::current_exe() and links what's already there.

It has to be target/<profile>/deps, not the profile root — cargo uplifts a lib
target only when it's a requested target. After
cargo clean -p oxidex && cargo test --all-features --no-run, deps/ holds all
three artifacts and target/debug/ holds none; the nested build was doing that
uplift, which is the only reason the old -L target/debug resolved.

2. cargo test --release races two lib units

Any --release invocation whose graph also contains test targets compiles
src/lib.rs twice: unwind (cargo forces it on test targets and their deps) and
abort (the three bins are final artifacts reading panic from
[profile.release]). Measured with
cargo test --release --all-features --features tag-comparison-binary --test ffi_c_integration --no-run -v:
two --crate-type lib invocations identical but for -C panic=abort, neither
carrying -C extra-filename, 4 output filename collision warnings
(rust-lang/cargo#6313), then a race. When abort wins, dependents fail with
"requires panic strategy abort … incompatible with … unwind" on unchanged
source. This is what the [profile.bench] comment already documents, with
[profile.release] as the abort source.

That cure — pin panic = "unwind" — doesn't transfer: [profile.release]'s
panic = 'abort' is what shipped binaries want. So the override is scoped to
the invocations that need it (unwind in the justfile). The bins those recipes
build are throwaway test-run artifacts; just build-bin-release,
cargo build --release, the tag-comparison bin recipes and the Release Build CI
job carry no prefix, so released binaries still abort. With the prefix the
same command reports 0 warnings and a single lib unit.

CI is unaffected either way — it runs no --release test step, which is why
this never showed as a red build, only as an intermittent local just test.

Not fixed here

The root enabler is the dropped -C extra-filename. The only complete cure is
moving the C ABI into its own crate so oxidex can return to
crate-type = ["lib"]; that touches cbindgen, the committed header, the python
bindings and the packaging assets, and CI can't validate it. Wants its own change.

Verification

  • cargo test --all-features (post-merge with origin/main): exit 0, 0 failures,
    doctests 221 passed / 0 failed. On a warm re-run the SHA-256 of
    target/debug/deps/liboxidex.rlib is identical before and after — nothing
    overwrites it now.
  • FFI suite 45.44s → 0.69s; the 44.75s was the nested rebuild.
  • cargo test --release --all-features --features tag-comparison-binary --test ffi_c_integration:
    exit 0, 41 C assertions pass, now linking the release cdylib that invocation
    actually built.
  • cargo fmt --check clean; cargo clippy --all-features --test ffi_c_integration -- -D warnings exit 0.
  • just --dry-run test shows the prefix expanded; shipping recipes verified unprefixed.

🤖 Generated with Claude Code

swackhamer and others added 3 commits August 9, 2026 18:25
#639 stopped tests/ffi_c_integration.rs's nested `cargo build --lib` from
building the DEFAULT feature graph over target/debug/deps/liboxidex.rlib,
which is what made `cargo test --all-features` fail with E0432 on both
parsers::magika_detector doctests. It forwarded the outer feature graph and
recorded that the nested build "resolves to the unit cargo has already
compiled, so it writes nothing at all".

Measured on `cargo test --all-features -v`, that last part is false. A nested
cargo inherits no profile either: it ran under `dev` while `cargo test` builds
the lib under `test`, and those are different units here because
[profile.test] sets opt-level = 2, codegen-units = 4 against [profile.dev]'s 0
and 16 -- both read off the rustc invocations, which also confirm the lib unit
carries no -C extra-filename. So 44s into the FFI test the nested build logged
"Compiling oxidex" / "Finished `dev` profile", and
target/debug/deps/liboxidex.rlib went from 119,540,352 bytes to 99,273,872 --
the same library, recompiled unoptimised, underneath the doctests about to
link it. Benign only because the features now match.

A test cannot learn the profile NAME cargo invoked it under, so a nested build
cannot be made to match one. Not building is the fix that holds: cargo builds
the lib as a dependency of every integration-test target, emitting all three
crate types in one rustc call, so the artifacts already exist. Derive the
directory from env::current_exe() and link there.

It has to be target/<profile>/deps, not the profile root -- cargo uplifts a lib
target only when it is a requested target of the invocation, and under
`cargo test` it is a dependency. Verified with `cargo clean -p oxidex && cargo
test --all-features --no-run`: afterwards deps/ holds all three artifacts and
target/debug/ holds none. The nested build was what uplifted them, which is the
only reason the old `-L target/debug` ever resolved.

Deriving from current_exe() also handles CARGO_TARGET_DIR and --target
<triple>, which the hard-coded target/debug never did, and makes the release
path link the cdylib that invocation actually built instead of a nested debug
one.

forwarded_features_cover_every_declared_feature goes with it: it guarded the
feature forwarding of a nested build that no longer exists.

Verified:
- `cargo test --all-features`: exit 0, 0 failed, doctests 221 passed. The
  SHA-256 of target/debug/deps/liboxidex.rlib is identical before and after
  the run -- nothing overwrites it now.
- FFI suite 45.44s -> 0.69s; the 44.75s was the nested rebuild.
- `cargo test --release --all-features --features tag-comparison-binary
  --test ffi_c_integration`: exit 0, 41 C assertions pass.
- `cargo fmt --check` clean; `cargo clippy --all-features
  --test ffi_c_integration -- -D warnings` exit 0.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Any --release invocation whose unit graph also contains test targets compiles
src/lib.rs twice:

  * once panic=unwind, because cargo forces unwind on test targets and their
    dependencies -- the rlib the test binaries link;
  * once panic=abort, for the `oxidex`, `tag-comparison` and `jpeg-tag-matrix`
    bins. Cargo builds the package's bins alongside its tests, and a bin is a
    final artifact, so it reads `panic` from [profile.release].

Measured with `cargo test --release --all-features --features
tag-comparison-binary --test ffi_c_integration --no-run -v`: two
`--crate-type lib` rustc invocations, identical but for `-C panic=abort` on
one, neither carrying -C extra-filename, and all three bins built with
`-C panic=abort`. Because `[lib] crate-type = ["lib", "staticlib", "cdylib"]`
makes cargo drop the filename hash, both units write
target/release/deps/liboxidex.{rlib,a,dylib} -- 4 "output filename collision"
warnings (rust-lang/cargo#6313) and then a race. When abort wins, dependents
fail with "requires panic strategy `abort` ... incompatible with ... `unwind`"
on unchanged source. This is the hazard [profile.bench] already documents,
with [profile.release] as the abort source.

The [profile.bench] cure -- pin panic = "unwind" -- does not transfer, because
[profile.release]'s panic = 'abort' is what shipped binaries want. Scoping the
override to the invocations that need it is the equivalent: the bins those
recipes build are throwaway test-run artifacts. `just build-bin-release`,
`cargo build --release`, the tag-comparison bin recipes and the Release Build
CI job carry no prefix, so released binaries still abort.

With the prefix the same command reports 0 collision warnings and a single lib
unit, and all three bins build unwind.

CI is unaffected either way: it runs no --release test step (`cargo nextest run
--all-features` and `cargo test --all-features` are both test-profile), which
is also why this never showed up as a red build -- only as an intermittent
local `just test`.

Not fixed here: the root enabler is the dropped -C extra-filename, and the only
complete cure is moving the C ABI into its own crate so `oxidex` can go back to
crate-type = ["lib"]. That touches cbindgen, the committed header, the python
bindings and the packaging assets, and CI cannot validate it, so it wants its
own change.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
@swackhamer
swackhamer merged commit c679005 into main Aug 10, 2026
10 checks passed
@swackhamer
swackhamer deleted the claude/kind-matsumoto-f90315 branch August 10, 2026 01:32
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant