Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions .github/workflows/ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -59,6 +59,7 @@ jobs:
python3 -m unittest scripts/test_verify_real_repo_lane_scale.py -v
python3 -m unittest scripts/test_check_real_repo_lane_scale.py -v
python3 -m unittest scripts/test_check_real_framework_handoff.py -v
python3 -m unittest scripts/test_edit_real_framework_semantic.py -v
- name: Check real-repository harness syntax
run: bash -n scripts/verify-real-repo-lane-scale.sh

Expand Down
2 changes: 2 additions & 0 deletions .github/workflows/layered-workspaces.yml
Original file line number Diff line number Diff line change
Expand Up @@ -16,6 +16,8 @@ on:
- "scripts/verify-real-framework-handoff.sh"
- "scripts/check-real-framework-handoff.py"
- "scripts/test_check_real_framework_handoff.py"
- "scripts/edit-real-framework-semantic.py"
- "scripts/test_edit_real_framework_semantic.py"
- "scripts/verify-environment-adapter-plugin.sh"
- "scripts/verify-artifact-adapter-conformance.sh"
- "scripts/verify-artifact-real-tool-gates.sh"
Expand Down
58 changes: 47 additions & 11 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -12,22 +12,54 @@ All notable changes to Trail are documented in this file. Trail follows
Cargo/npm defaults globally. Go, pnpm/npm/Yarn/Bun, Python, and CMake commands
receive isolated framework-native caches and exact tool paths, while inactive
frameworks no longer leak variables into the command.
- Node dependency executables and CMake build trees now bind directly from the
- Managed execution now rejects environment-bearing materialized lanes before
emitting an impossible resolution command and recommends a new
`--workdir-mode auto` lane. Layered workspace backends remain required for
managed dependency and build projections.
- Node dependency executables, Python virtual environments, and CMake build trees now bind directly from the
lane's generated upper, avoiding metadata-heavy build/dependency traversal
through macOS NFS while preserving a mounted source path and lane-private
mutation. Python also exposes its path-correct interpreter through
`TRAIL_VENV_PYTHON`.
mutation. Python exposes the physical private environment through
`VIRTUAL_ENV`, `PATH`, and `TRAIL_VENV_PYTHON` while `.venv` remains visible
at its conventional lane path.
- macOS NFS lane mounts now retain attributes and negative lookups for up to 60
seconds within a mounted execution. Same-client mutations still invalidate
cached entries and synchronous writes remain enabled, while unchanged Go and
Node source/dependency walks avoid repeated userspace NFS round trips.
- Lane/root diff addition and deletion totals now come from the emitted text
diff rather than stable-line identity churn, so statistics agree with the
unified patch while line-identity inspection remains available separately.
- Built-in Claude terminal tasks now start with project instructions, plugins,
hooks, MCP servers, skills, and agents disabled. The new
- Built-in Claude and Codex terminal tasks now use contained launch profiles.
Claude disables project instructions, plugins, hooks, MCP servers, skills,
browser integration, and agents; Codex uses strict configuration, an explicit
lane root, workspace-write sandboxing, and an empty MCP map. Both receive an
isolated runtime home, an allowlisted environment, a lane Git shadow, and a
typed containment receipt. On macOS, `sandbox-exec` enforces declared writable
roots and protects the original checkout. The new
`--allow-project-integrations` flag restores the previous project-integrated
launch explicitly; custom commands after `--` remain unchanged.
launch explicitly; custom commands after `--` remain unchanged. Contained
Claude launches now preserve its documented OAuth token, OAuth-token file
descriptor, API-key, and workload-identity variables without copying or
discovering host keychain secrets. Claude's internal temporary directory is
redirected into Trail's private launch runtime, and `acceptEdits` permits
noninteractive built-in edits without disabling Bash or other higher-risk
permission checks. An exact allowlist imports Claude credential and provider
endpoint variables from user-level settings without importing hooks,
plugins, permissions, or other project/global integrations; explicit process
variables take precedence.
- Python `uv.lock` environments now select a checked `.python-version`
major/minor interpreter, install the frozen dependency set into a copy-based
lane-private venv, and report bounded redacted initializer diagnostics.
Hash-pinned `requirements.lock` and verified Trail-managed lock snapshots use
hash-required `uv pip sync`; unfrozen requirements and unsupported lock
formats fail with recovery guidance instead of producing an empty venv.
- Transparent-COW checkpoints now apply `.trailignore` to newly created journal
candidates before source recording, and classify `dist-node` as generated
output. Ignored or conventional build artifacts no longer enter source merely
because they were first observed by the native change journal.
- pnpm workspace roots now fail explicitly instead of attempting an incomplete
`--ignore-workspace` install. Independent non-workspace pnpm projects retain
frozen dependency-layer reuse.
- Managed execution now scopes environment discovery and synchronization to
the command's component root, projects a verified manifest-only Cargo lock
snapshot only for the command, and removes it before checkpointing. Unrelated
Expand Down Expand Up @@ -65,9 +97,12 @@ All notable changes to Trail are documented in this file. Trail follows
### Added

- Added an opt-in macOS real-framework qualification matrix for pinned bbolt,
date-fns, uuid, httpx, and LevelDB revisions. Each row produces checksummed
Agent A → B → C evidence for source-only handoff, parent-generation
inheritance, framework checks, cache/layer reuse, and lane-private outputs.
Polymarket CLOB TypeScript client, uuid, tappy, and LevelDB revisions. Each row
makes deterministic production-and-test source edits and produces checksummed
Agent A → B → C evidence for semantic ancestry, parent-generation
inheritance, real framework tests/builds, stable dependency identity,
cache/layer reuse, fresh lane-private outputs, incremental CMake recompilation,
stale-output rejection, and exact checkpoint paths.
- Added deterministic 10k/100k/1M artifact and source scale matrices for
1/5/20 lanes, fail-closed JSON evidence validation, and compositional
owning-host NFS/FUSE/Dokan qualification that distinguishes mounted backend
Expand Down Expand Up @@ -113,8 +148,9 @@ All notable changes to Trail are documented in this file. Trail follows
declarations remain external metadata, and repository v2 keeps its independent
desired-key v2 identity.
- Python environments can now bind an optional uv-generated, hash-bearing requirements
snapshot, warm a performance-only wheel/download cache, and still keep `.venv`, its
bytecode, and embedded path state entirely lane-private.
snapshot, warm a performance-only wheel/download cache, install that snapshot with
hash enforcement, and still keep `.venv`, its bytecode, and embedded path state
entirely lane-private.
- `trail.environment/v2` framework fixtures now compose Next.js and Vite build/state
components over the Node dependency component: `.next` and `.vite` remain lane-private,
while validated Vite `dist` content can use an independently keyed immutable layer.
Expand Down
4 changes: 2 additions & 2 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -203,8 +203,8 @@ target, platform, or build-policy changes remain hard compatibility misses.
Managed command bindings are adapter-owned rather than Cargo-specific. Active
Go components receive shared module/build caches and an exact `TRAIL_GO`;
Node components receive package-manager caches plus a direct private
`TRAIL_NODE_MODULES`; Python components receive a lane-private `VIRTUAL_ENV`
and `TRAIL_VENV_PYTHON`; and CMake components receive a direct lane-private
`TRAIL_NODE_MODULES`; Python components receive a direct lane-private
`VIRTUAL_ENV`, venv `PATH`, and `TRAIL_VENV_PYTHON`; and CMake components receive a direct lane-private
`TRAIL_CMAKE_BUILD_DIR`. Inactive frameworks inject no cache, tool, or output
variables into the command.

Expand Down
61 changes: 37 additions & 24 deletions docs/design/environment-adapter-contract.md
Original file line number Diff line number Diff line change
Expand Up @@ -24,14 +24,15 @@ generation activation. The first built-ins are:
lane so its source path names the stable lane workdir rather than disposable staging;
the writable build directory is bound directly from the generated upper so configure,
compilation, and tests do not route build-tree metadata through NFS/FUSE/Dokan;
- `trail/python-venv@1`: recognizes `pyproject.toml` and the common uv, Poetry, PDM,
Pipenv, and requirements lock/manifest files, provisions a layer-free lane-private
`.venv`, and keys it by every present dependency file plus the resolved Python
executable. An optional uv-generated hash-bearing requirements snapshot remains Trail
metadata and warms a shared performance-only wheel/download cache. Trail automatically
creates the virtual environment through an ephemeral candidate view at the lane's final
mountpoint, so scripts, bytecode, and prefix metadata stay private and embed the correct
absolute path without exposing partial state;
- `trail/python-venv@1`: recognizes `pyproject.toml`, `.python-version`, and common
lock/manifest markers, but installs only from `uv.lock`, hash-pinned
`requirements.lock`, or a verified Trail-managed hashed snapshot. Other recognized
formats return explicit unsupported-contract guidance. It provisions a layer-free
lane-private `.venv`, keys it by every present dependency file plus the selected
Python and uv executable identities, and warms a shared performance-only download
cache. Trail creates and installs the environment in an ephemeral candidate's
physical private upper, then exposes direct bindings so scripts, bytecode, and prefix
metadata stay private without routing venv I/O through the mounted source transport;
- `trail/oci-image@1`: reads `trail.oci.toml`, accepts only lowercase SHA-256
digest-pinned OCI references with an explicit platform, and records provider-owned
image identities without commands, caches, mounts, or manufactured directories;
Expand Down Expand Up @@ -1306,26 +1307,37 @@ delete a user-owned image, container, volume, or remote builder cache.

The experimental `trail/python-venv@1` built-in implements the safe baseline today. It
owns `<component>/.venv` as `writable_private`, publishes no shared layer, and preserves
the directory across a compatible re-sync. Synchronization automatically runs
`python -m venv --without-pip .venv` at the lane's final mountpoint:
the directory across a compatible re-sync. If `.python-version` is present, Trail
selects its CPython major/minor executable from `PATH`; otherwise it resolves
`python3`/`python`. Synchronization creates a copy-based virtual environment in the
physical lane-private generated upper and installs only from a frozen contract:

```sh
trail env sync component python.venv --lane <lane> --adapter trail/python-venv@1
# Optionally let a lockfile-aware tool populate the initialized private path:
trail lane exec <lane> -- uv sync --frozen
trail env sync component python-venv --lane <lane> --adapter trail/python-venv@1
trail lane exec <lane> -- "$TRAIL_VENV_PYTHON" -m pytest
```

Mounted initialization never runs against the active lane upper. Trail constructs an
The supported install contracts are `uv.lock`, a hash-pinned
`requirements.lock`, or a verified Trail-managed hash-bearing requirements snapshot.
`uv.lock` uses `uv sync --frozen --no-install-project`; requirements snapshots use
`uv pip sync --require-hashes`. A plain `requirements.txt`, Poetry/PDM/Pipenv lock, or
unhashed requirements file is rejected until an adapter can implement its exact frozen
installation semantics.

Initialization never runs against the active lane upper. Trail constructs an
ephemeral candidate view with the pinned source root, desired immutable bindings, and
prepared private seeds; source and unrelated writes land in disposable uppers. After
the command succeeds, Trail rejects every write outside the newly prepared private
outputs, copies those outputs back to staging, and performs the ordinary atomic
generation activation. A non-zero exit, undeclared write, process kill, or host crash
therefore leaves the predecessor generation and its private upper unchanged. Recovery
also removes abandoned candidate directories.
prepared private seeds. Host-expanded output references direct the interpreter and uv
to the candidate's physical private upper, avoiding metadata-heavy venv I/O through
FUSE/NFS/Dokan while source reads retain lane semantics. Source and unrelated writes
land in disposable uppers. After the command succeeds, Trail rejects every write
outside the newly prepared private outputs, copies those outputs back to staging, and
performs the ordinary atomic generation activation. A non-zero exit, undeclared write,
process kill, or host crash therefore leaves the predecessor generation and its private
upper unchanged. Recovery also removes abandoned candidate directories. Initializer
stderr is drained, bounded to 64 KiB, secret-redacted, and included on failure.

Real Linux/FUSE, macOS/NFS, and Windows/Dokan conformance creates two virtual
environments, verifies that `sys.prefix` identifies each mounted lane, mutates one
environments, verifies that `sys.prefix` identifies each direct private upper, mutates one
environment, and requires the other lane to remain unchanged. It also verifies that a
compatible re-sync preserves the private environment and creates no shared layer.
Additional native fixtures cover multi-component `env sync all`, initializer failure,
Expand All @@ -1335,9 +1347,10 @@ An optional resolver plan pins the complete source root, exact uv executable, of
hash-generation policy, and `pyproject.toml`, then stores the resulting
`requirements.lock` as verified environment metadata. The host projects it only into
attempt staging and runs hash-required `pip download` into a host-owned
`cache_shared_content` namespace. Evicting that cache affects performance only. Trail
still must not represent a path-bearing virtual environment, its bytecode, or embedded
scripts as a portable immutable artifact without relocation validation.
`cache_shared_content` namespace before the mounted initializer performs an offline,
hash-required `uv pip sync`. Evicting that cache affects performance only. Trail still
must not represent a path-bearing virtual environment, its bytecode, or embedded scripts
as a portable immutable artifact without relocation validation.

| Concern | Inputs | Policy/binding |
| --- | --- | --- |
Expand Down
4 changes: 2 additions & 2 deletions docs/design/layered-lane-workspaces.md
Original file line number Diff line number Diff line change
Expand Up @@ -744,8 +744,8 @@ The workspace host does not inject Cargo or npm variables unconditionally.
Each active built-in adapter declares its fixed policy values, resolved tools,
cache namespace paths, and output paths. Go receives `GOMODCACHE`, `GOCACHE`,
and `TRAIL_GO`; Node receives manager caches and a direct lane-private
`TRAIL_NODE_MODULES`; Python receives its mounted `VIRTUAL_ENV` and
`TRAIL_VENV_PYTHON`; CMake receives a direct writable-private
`TRAIL_NODE_MODULES`; Python receives a direct lane-private `VIRTUAL_ENV`,
`TRAIL_VENV_PYTHON`, and venv `PATH`; CMake receives a direct writable-private
`TRAIL_CMAKE_BUILD_DIR`. Cargo retains direct private `CARGO_TARGET_DIR` and
managed `CARGO_HOME`/compiler-cache bindings. Output bindings may bypass the
mounted transport only for private-seeded, writable-private, or disposable
Expand Down
10 changes: 6 additions & 4 deletions docs/design/universal-lane-environments.md
Original file line number Diff line number Diff line change
Expand Up @@ -1001,10 +1001,12 @@ backend:
- `trail/cmake-build@1` provisions a layer-free private build tree and defers configure
to the mounted lane so absolute cache paths remain valid;
- `trail/python-venv@1` provisions a layer-free private `.venv`, keys it by dependency
manifests/locks and interpreter identity, optionally binds a hash-bearing managed
requirements snapshot to a performance-only wheel/download cache, and automatically
initializes it through an ephemeral candidate view at the final mountpoint so embedded
prefixes and bytecode stay lane-private without weakening atomic generation activation;
manifests/locks and interpreter identity, accepts `uv.lock`, hash-pinned
`requirements.lock`, or a verified managed requirements snapshot, binds downloads to
a performance-only cache, and automatically initializes the venv in the candidate's
physical private upper. Managed commands use direct `VIRTUAL_ENV`, `PATH`, and
`TRAIL_VENV_PYTHON` bindings so embedded prefixes and metadata-heavy package/test I/O
stay lane-private without traversing NFS or weakening atomic generation activation;
- existing layer references are imported into an initial environment generation without
copying tree content.

Expand Down
25 changes: 16 additions & 9 deletions docs/guides/performance-and-scale-benchmarks.md
Original file line number Diff line number Diff line change
Expand Up @@ -122,9 +122,10 @@ scripts/cli-scale-bench.sh

Framework adapters have a separate opt-in macOS qualification matrix. Each row
fetches an exact upstream revision, uses the candidate Trail binary, and moves
three native NFS lanes through a source edit, environment synchronization, and
real framework check. The gate covers bbolt/Go, date-fns/pnpm, uuid/npm,
httpx/Python virtual environments, and LevelDB/CMake:
three transparent-COW lanes through deterministic production-and-test edits,
environment synchronization, and real framework checks. The pinned matrix uses
bbolt/Go, Polymarket's CLOB TypeScript client/pnpm, uuid/npm, tappy/Python, and
LevelDB/CMake:

```sh
TRAIL_BIN=/absolute/path/to/trail \
Expand All @@ -136,14 +137,20 @@ scripts/verify-real-framework-handoff.sh go
Valid selectors are `go`, `pnpm`, `npm`, `python`, and `cmake`. The output
and optional work directories must not exist. When omitted, the work directory
defaults to `<evidence>.work` and remains outside the uploaded evidence set.
macOS defaults to `nfs-cow`; prepared hosts can set
`TRAIL_FRAMEWORK_WORKDIR_MODE=fuse-cow` or `dokan-cow` to qualify another
transparent backend.
`evidence.json` pins the repository/revision, three
distinct source roots, active component keys, layer identities, cache
namespaces, lane-private output identities, and SHA-256 digests of every raw
Trail report. The checker requires each edit to checkpoint only `README.md`,
each child to inherit its parent's active generation before editing, and every
framework command to pass. Go additionally requires compatible-predecessor
seeding with nonzero avoided bytes; npm/pnpm require one exact immutable layer;
Python and CMake require three distinct writable-private outputs.
namespaces, lane-private output contracts, and SHA-256 digests of every raw
Trail report. The checker requires each edit to checkpoint only the expected
framework production and test paths, B to start from A's semantic checkpoint,
C to start from B's, and parent and edited semantics to pass. Go additionally
requires compatible-predecessor seeding with nonzero avoided bytes; npm/pnpm
require one exact immutable dependency layer; Python/CMake require fresh
lane-private outputs with stable dependency/build identity. The CMake row also
hashes affected and unaffected objects, requires selective recompilation, and
links/runs a marker executable so stale output cannot pass.

Dispatch `layered-workspaces.yml` with
`run_real_framework_handoffs=true` to run all five rows on clean macOS hosts
Expand Down
Loading
Loading