refactor(packaging): three-grouping path-dep workspace (#403) - #636
Merged
Conversation
mfw78
force-pushed
the
carve/403-path-dep-workspace
branch
3 times, most recently
from
July 27, 2026 01:39
5c6e3d5 to
c64dffb
Compare
mfw78
marked this pull request as ready for review
July 27, 2026 01:51
Move all workspace members into nexum/ (L1), videre/ (L2) and shepherd/ (L3) repo-root dirs as a single cargo workspace with intra-grouping path-deps, so the tree keeps building as one unit with a shared Cargo.lock right up to the physical carve (#407). This is the refactor-now-cut-later step of the gated three-repo split (M5). Add the carve-groups dep-sync gate (scripts/check-carve-groups.sh, CI job, justfile recipe) enforcing the acyclic nexum <- videre <- shepherd invariant from the physical layout, so no upward edge can become a circular repo dependency at the carve. Document the transitional cross-repo dependency medium (path-deps -> git-tag pins -> crates.io) in docs/design/carve-workspace.md. WIT stays in a single root wit/, resolved via the manifest-dir ancestor walk; the per-group wit/ + wit/deps split is deferred to the wit-deps flip (#404/#405). The hardcoded wit_bindgen::generate! path-lists were re-based one level deeper by the move. Align the justfile with the already-optimized CI: test/test-e2e/ci use cargo nextest run, ci runs doctests separately and appends -D warnings to RUSTFLAGS/RUSTDOCFLAGS instead of clobbering the devshell mold flags. sccache is already wired via the flake devshell and the CI workflow env. Fix the test workspace-root locators the move broke: replace every hardcoded CARGO_MANIFEST_DIR.parent().parent() (valid only at the old crates/<x> depth) with a workspace_root() that walks to the topmost ancestor carrying a Cargo.toml, and group-prefix the module manifests those e2e tests load. These had silently skipped locally (no CI env) into a hollow green; CI's fail-loud path caught them. Re-validated with CI=1. Closes #403.
mfw78
force-pushed
the
carve/403-path-dep-workspace
branch
from
July 27, 2026 01:57
c64dffb to
32a1b9f
Compare
This was referenced Jul 27, 2026
mfw78
added a commit
that referenced
this pull request
Jul 27, 2026
refactor(packaging): reorganize into three-grouping path-dep workspace Move all workspace members into nexum/ (L1), videre/ (L2) and shepherd/ (L3) repo-root dirs as a single cargo workspace with intra-grouping path-deps, so the tree keeps building as one unit with a shared Cargo.lock right up to the physical carve (#407). This is the refactor-now-cut-later step of the gated three-repo split (M5). Add the carve-groups dep-sync gate (scripts/check-carve-groups.sh, CI job, justfile recipe) enforcing the acyclic nexum <- videre <- shepherd invariant from the physical layout, so no upward edge can become a circular repo dependency at the carve. Document the transitional cross-repo dependency medium (path-deps -> git-tag pins -> crates.io) in docs/design/carve-workspace.md. WIT stays in a single root wit/, resolved via the manifest-dir ancestor walk; the per-group wit/ + wit/deps split is deferred to the wit-deps flip (#404/#405). The hardcoded wit_bindgen::generate! path-lists were re-based one level deeper by the move. Align the justfile with the already-optimized CI: test/test-e2e/ci use cargo nextest run, ci runs doctests separately and appends -D warnings to RUSTFLAGS/RUSTDOCFLAGS instead of clobbering the devshell mold flags. sccache is already wired via the flake devshell and the CI workflow env. Fix the test workspace-root locators the move broke: replace every hardcoded CARGO_MANIFEST_DIR.parent().parent() (valid only at the old crates/<x> depth) with a workspace_root() that walks to the topmost ancestor carrying a Cargo.toml, and group-prefix the module manifests those e2e tests load. These had silently skipped locally (no CI env) into a hollow green; CI's fail-loud path caught them. Re-validated with CI=1. Closes #403.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
Reorganize the monorepo into the three prospective repo roots as a single cargo workspace with intra-grouping path-deps, so the tree keeps building as one unit with a shared
Cargo.lockright up to the physical carve (#407). This is the refactor-now-cut-later step of the gated three-repo split (M5). Closes #403.Every workspace member moves down exactly one level,
X-><group>/X:nexum/(L1, futurenexum-runtimerepo): universal runtime, SDK, macros, launcher, the barenexum-clibin; universal-package example modules (example,price-alert,balance-tracker,http-probe) and the runtime fixtures.videre/(L2, futurevidererepo): intent/venue SDK, host, macros, status-body; the genericechoreference venue andno-std-probe/flaky-venue.shepherd/(L3, the shepherd bundle):cow-venue,composable-cow, theshepherdcomposition-root bin, the cow reference modules (twap-monitor,ethflow-watcher),orderbook-mock, and the EthFlowbaseline-latencytool.Layout within a group is
<group>/crates/*,<group>/modules/*,<group>/tools/*. All moves aregit mv, so history is preserved per file.Dependency invariant + dep-sync gate
The grouping is strictly acyclic:
nexum <- videre <- shepherd, every cross-group edge points down. A new blocking CI jobcarve-groups(scripts/check-carve-groups.sh) enforces this: the physical layout is the source of truth (a crate's group is its top-level dir), and the check derives every internal edge's tier fromcargo metadata, failing on any upward edge that would become a circular repo dependency at the carve. No separate mapping to drift.WIT stays shared at root (deferred to #404/#405)
resolve_wit_package(nexum/crates/nexum-world) walks manifest-dir ancestors to the nearestwit/, so a single rootwit/keeps resolving for crates at any depth. Splittingwit/into per-groupwit/+wit/deps/needs the crate-local wit-deps flip and belongs to #404/#405, not here. This is a deliberate, code-driven deviation from the end-state layout. The hardcodedwit_bindgen::generate!path-lists that bypass the resolver were re-based one level deeper by the move.Cross-repo dependency medium
Documented in
docs/design/carve-workspace.md: path-deps now (one workspace, one lockfile) -> git-tag pins at the carve (#407) -> crates.io post-carve.Also updated
Cargo.tomlmembers, every crate's path-deps,engine.{e2e,soak,example,m2,m3,load}.tomlmanifest paths,Dockerfilemodule.toml COPY paths,justfile:27, and thecheck-venue-agnostic/check-cow-orderbook-onlygate scripts.crates//modules//tools/path references in README and docs re-pointed to the grouped layout (module-relative Cargo.toml snippets left unchanged, since intra-group relative paths are preserved).Justfile: nextest + sccache parity with CI
Folded in so the local
justrecipes match the already-optimized CI (ci.ymlsetsRUSTC_WRAPPER: sccache,SCCACHE_GHA_ENABLED, mold, and runscargo nextest run+ separatecargo test --doc):test/test-e2e/cinow usecargo nextest runinstead ofcargo test;ciruns the doctests separately (nextest does not cover them), mirroring CI one-to-one.cinow appends-D warningstoRUSTFLAGS/RUSTDOCFLAGSinstead of clobbering them, so it keeps the devshell's mold linker flags (flake.nix) for fast native linking.RUSTC_WRAPPER=sccache(+CARGO_INCREMENTAL=0) when sccache is on the host, so everynix develop -c just ...run is already cached. CI is likewise already on sccache.Test workspace-root locators (found via first CI run)
The first CI run surfaced 16 e2e failures the move had introduced: several test helpers computed the workspace root with a hardcoded
CARGO_MANIFEST_DIR.parent().parent()(valid only at the oldcrates/<x>depth) and referenced module manifests at their pre-movemodules/.../crates/...paths. Locally these had silently skipped (noCIenv → the skip branch) — a hollow green (cf. nullislabs/nexum-runtime#55) — while CI's fail-loud branch caught them. Fixed by:supervisor/tests.rs,builder.rs,test_utils/harness.rs,videre-host/tests/{zero_leak,platform}.rs) with aworkspace_root()that walks to the topmost ancestor carrying aCargo.toml. This survives the crate at any depth — the transitional grouping now, and the repo root after the carve — so these locators never need touching again through packaging: source cross-repo WIT from wit-deps and git tags #405/packaging: carve nexum-runtime, videre and shepherd into three repos #407. Also fixed a latentwasm.ancestors().nth(3)root miscalculation inbuilder.rs.modules/twap-monitor/module.toml→shepherd/modules/twap-monitor/module.toml).Re-validated with
CI=1set (the fail-loud path), so the formerly-skipped e2e tests actually run.Validation
cargo clippy --workspace --all-targets --all-features --locked -- -D warnings: green.just ci(fmt, clippy,carve-groups, rustdoc, all module wasms, nextest + doctests): green.CI=1 cargo nextest run --workspace --all-features: 864 passed, 0 failed, 2 skipped, zero hollow-green skips (the fail-loud path the earlier hidden skips came from).scripts/check-carve-groups.sh: OK.Rebased onto
origin/dev/m1(e86a3d88): the 3 intervening commits (journal-guard, typed local-store, composable-cow keeper) merged into the moved crates via rename detection; the 2 new files (nexum-sdk/src/store.rs,tests/store.rs) were relocated into the group. Re-validated green on the rebased tree; PR merges CLEAN.Not in scope
Per-group
wit/split and crate-local wit-deps (#404/#405); git-tag pin sourcing (#405); the physical carve itself (#407); version homogenization to 0.1.0 (#515, explicitly end-of-train). Thenexum-runtime/nexum-cli/nexum-launch/shepherd0.2.0version drift is left as-is for #515.