Skip to content

refactor(packaging): three-grouping path-dep workspace (#403) - #636

Merged
mfw78 merged 1 commit into
dev/m1from
carve/403-path-dep-workspace
Jul 27, 2026
Merged

refactor(packaging): three-grouping path-dep workspace (#403)#636
mfw78 merged 1 commit into
dev/m1from
carve/403-path-dep-workspace

Conversation

@mfw78

@mfw78 mfw78 commented Jul 27, 2026

Copy link
Copy Markdown
Contributor

What

Reorganize the monorepo into the three prospective repo roots as a single cargo workspace with intra-grouping path-deps, so the tree keeps building as one unit with a shared Cargo.lock right up to the physical carve (#407). This is the refactor-now-cut-later step of the gated three-repo split (M5). Closes #403.

Every workspace member moves down exactly one level, X -> <group>/X:

  • nexum/ (L1, future nexum-runtime repo): universal runtime, SDK, macros, launcher, the bare nexum-cli bin; universal-package example modules (example, price-alert, balance-tracker, http-probe) and the runtime fixtures.
  • videre/ (L2, future videre repo): intent/venue SDK, host, macros, status-body; the generic echo reference venue and no-std-probe/flaky-venue.
  • shepherd/ (L3, the shepherd bundle): cow-venue, composable-cow, the shepherd composition-root bin, the cow reference modules (twap-monitor, ethflow-watcher), orderbook-mock, and the EthFlow baseline-latency tool.

Layout within a group is <group>/crates/*, <group>/modules/*, <group>/tools/*. All moves are git mv, so history is preserved per file.

Dependency invariant + dep-sync gate

The grouping is strictly acyclic: nexum <- videre <- shepherd, every cross-group edge points down. A new blocking CI job carve-groups (scripts/check-carve-groups.sh) enforces this: the physical layout is the source of truth (a crate's group is its top-level dir), and the check derives every internal edge's tier from cargo metadata, failing on any upward edge that would become a circular repo dependency at the carve. No separate mapping to drift.

WIT stays shared at root (deferred to #404/#405)

resolve_wit_package (nexum/crates/nexum-world) walks manifest-dir ancestors to the nearest wit/, so a single root wit/ keeps resolving for crates at any depth. Splitting wit/ into per-group wit/ + wit/deps/ needs the crate-local wit-deps flip and belongs to #404/#405, not here. This is a deliberate, code-driven deviation from the end-state layout. The hardcoded wit_bindgen::generate! path-lists that bypass the resolver were re-based one level deeper by the move.

Cross-repo dependency medium

Documented in docs/design/carve-workspace.md: path-deps now (one workspace, one lockfile) -> git-tag pins at the carve (#407) -> crates.io post-carve.

Also updated

  • Root Cargo.toml members, every crate's path-deps, engine.{e2e,soak,example,m2,m3,load}.toml manifest paths, Dockerfile module.toml COPY paths, justfile:27, and the check-venue-agnostic/check-cow-orderbook-only gate scripts.
  • Stale crates//modules//tools/ path references in README and docs re-pointed to the grouped layout (module-relative Cargo.toml snippets left unchanged, since intra-group relative paths are preserved).

Justfile: nextest + sccache parity with CI

Folded in so the local just recipes match the already-optimized CI (ci.yml sets RUSTC_WRAPPER: sccache, SCCACHE_GHA_ENABLED, mold, and runs cargo nextest run + separate cargo test --doc):

  • test/test-e2e/ci now use cargo nextest run instead of cargo test; ci runs the doctests separately (nextest does not cover them), mirroring CI one-to-one.
  • ci now appends -D warnings to RUSTFLAGS/RUSTDOCFLAGS instead of clobbering them, so it keeps the devshell's mold linker flags (flake.nix) for fast native linking.
  • sccache needs no justfile change: the flake devshell shellHook already exports RUSTC_WRAPPER=sccache (+ CARGO_INCREMENTAL=0) when sccache is on the host, so every nix develop -c just ... run is already cached. CI is likewise already on sccache.

Test workspace-root locators (found via first CI run)

The first CI run surfaced 16 e2e failures the move had introduced: several test helpers computed the workspace root with a hardcoded CARGO_MANIFEST_DIR.parent().parent() (valid only at the old crates/<x> depth) and referenced module manifests at their pre-move modules/... / crates/... paths. Locally these had silently skipped (no CI env → the skip branch) — a hollow green (cf. nullislabs/nexum-runtime#55) — while CI's fail-loud branch caught them. Fixed by:

  • Replacing every hardcoded-depth root locator (supervisor/tests.rs, builder.rs, test_utils/harness.rs, videre-host/tests/{zero_leak,platform}.rs) with a workspace_root() that walks to the topmost ancestor carrying a Cargo.toml. This survives the crate at any depth — the transitional grouping now, and the repo root after the carve — so these locators never need touching again through packaging: source cross-repo WIT from wit-deps and git tags #405/packaging: carve nexum-runtime, videre and shepherd into three repos #407. Also fixed a latent wasm.ancestors().nth(3) root miscalculation in builder.rs.
  • Group-prefixing the module-manifest paths those tests load (e.g. modules/twap-monitor/module.tomlshepherd/modules/twap-monitor/module.toml).

Re-validated with CI=1 set (the fail-loud path), so the formerly-skipped e2e tests actually run.

Validation

  • cargo clippy --workspace --all-targets --all-features --locked -- -D warnings: green.
  • Full just ci (fmt, clippy, carve-groups, rustdoc, all module wasms, nextest + doctests): green.
  • CI=1 cargo nextest run --workspace --all-features: 864 passed, 0 failed, 2 skipped, zero hollow-green skips (the fail-loud path the earlier hidden skips came from).
  • scripts/check-carve-groups.sh: OK.

Rebased onto origin/dev/m1 (e86a3d88): the 3 intervening commits (journal-guard, typed local-store, composable-cow keeper) merged into the moved crates via rename detection; the 2 new files (nexum-sdk/src/store.rs, tests/store.rs) were relocated into the group. Re-validated green on the rebased tree; PR merges CLEAN.

Not in scope

Per-group wit/ split and crate-local wit-deps (#404/#405); git-tag pin sourcing (#405); the physical carve itself (#407); version homogenization to 0.1.0 (#515, explicitly end-of-train). The nexum-runtime/nexum-cli/nexum-launch/shepherd 0.2.0 version drift is left as-is for #515.

@mfw78
mfw78 force-pushed the carve/403-path-dep-workspace branch 3 times, most recently from 5c6e3d5 to c64dffb Compare July 27, 2026 01:39
@mfw78
mfw78 marked this pull request as ready for review July 27, 2026 01:51
Move all workspace members into nexum/ (L1), videre/ (L2) and shepherd/
(L3) repo-root dirs as a single cargo workspace with intra-grouping
path-deps, so the tree keeps building as one unit with a shared
Cargo.lock right up to the physical carve (#407). This is the
refactor-now-cut-later step of the gated three-repo split (M5).

Add the carve-groups dep-sync gate (scripts/check-carve-groups.sh, CI
job, justfile recipe) enforcing the acyclic nexum <- videre <- shepherd
invariant from the physical layout, so no upward edge can become a
circular repo dependency at the carve. Document the transitional
cross-repo dependency medium (path-deps -> git-tag pins -> crates.io) in
docs/design/carve-workspace.md.

WIT stays in a single root wit/, resolved via the manifest-dir ancestor
walk; the per-group wit/ + wit/deps split is deferred to the wit-deps
flip (#404/#405). The hardcoded wit_bindgen::generate! path-lists were
re-based one level deeper by the move.

Align the justfile with the already-optimized CI: test/test-e2e/ci use
cargo nextest run, ci runs doctests separately and appends -D warnings to
RUSTFLAGS/RUSTDOCFLAGS instead of clobbering the devshell mold flags.
sccache is already wired via the flake devshell and the CI workflow env.

Fix the test workspace-root locators the move broke: replace every
hardcoded CARGO_MANIFEST_DIR.parent().parent() (valid only at the old
crates/<x> depth) with a workspace_root() that walks to the topmost
ancestor carrying a Cargo.toml, and group-prefix the module manifests
those e2e tests load. These had silently skipped locally (no CI env) into
a hollow green; CI's fail-loud path caught them. Re-validated with CI=1.

Closes #403.
@mfw78
mfw78 force-pushed the carve/403-path-dep-workspace branch from c64dffb to 32a1b9f Compare July 27, 2026 01:57
@mfw78
mfw78 merged commit 2436c50 into dev/m1 Jul 27, 2026
8 checks passed
@mfw78
mfw78 deleted the carve/403-path-dep-workspace branch July 27, 2026 02:06
mfw78 added a commit that referenced this pull request Jul 27, 2026
refactor(packaging): reorganize into three-grouping path-dep workspace

Move all workspace members into nexum/ (L1), videre/ (L2) and shepherd/
(L3) repo-root dirs as a single cargo workspace with intra-grouping
path-deps, so the tree keeps building as one unit with a shared
Cargo.lock right up to the physical carve (#407). This is the
refactor-now-cut-later step of the gated three-repo split (M5).

Add the carve-groups dep-sync gate (scripts/check-carve-groups.sh, CI
job, justfile recipe) enforcing the acyclic nexum <- videre <- shepherd
invariant from the physical layout, so no upward edge can become a
circular repo dependency at the carve. Document the transitional
cross-repo dependency medium (path-deps -> git-tag pins -> crates.io) in
docs/design/carve-workspace.md.

WIT stays in a single root wit/, resolved via the manifest-dir ancestor
walk; the per-group wit/ + wit/deps split is deferred to the wit-deps
flip (#404/#405). The hardcoded wit_bindgen::generate! path-lists were
re-based one level deeper by the move.

Align the justfile with the already-optimized CI: test/test-e2e/ci use
cargo nextest run, ci runs doctests separately and appends -D warnings to
RUSTFLAGS/RUSTDOCFLAGS instead of clobbering the devshell mold flags.
sccache is already wired via the flake devshell and the CI workflow env.

Fix the test workspace-root locators the move broke: replace every
hardcoded CARGO_MANIFEST_DIR.parent().parent() (valid only at the old
crates/<x> depth) with a workspace_root() that walks to the topmost
ancestor carrying a Cargo.toml, and group-prefix the module manifests
those e2e tests load. These had silently skipped locally (no CI env) into
a hollow green; CI's fail-loud path caught them. Re-validated with CI=1.

Closes #403.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant