fix(sandbox): the container rule is not asked of Validate, and a make-based binding executes the tree - #418
Merged
tpouyer merged 2 commits intoSep 9, 2026
Conversation
…-based binding executes the tree In-Lockstep-Run: implement-propose-34408733138-20260909T223934Z-8d0d Ticket: #410
…t now requires Two things the run that opened this pull request could not do. The binding. `CommandValidate` declaring `EXECUTES_CODE` is correct and makes this repository's own `Validate` — bound with a bare `Sandbox()` — refused by `staged_refusal`, so every `/implement` and `/fix` here would have stopped checking what a model wrote. It fails closed, which is the right direction and the reason nobody would have noticed. It now shares the Test binding's sandbox, named once and bound twice rather than copied: the image carries `make`, `python`, `python3` and `uv`, measured rather than assumed, which is what `make lint typecheck` needs. An agent could not have written that line. `.lockstep/` is tier 1, so the acceptance criterion in #410 asking for it was one the framework forbids a model from satisfying — my error in writing the ticket, not the run's in skipping it. `test_own_ceilings.py` now derives the rule from the declaration instead of restating it, so a verb bound tomorrow to an adapter that executes is covered without that test being edited. The fail-open. `staged_refusal` reached `container.resolve` through a `getattr(..., None)` with an early `return None`, so a container it could not inspect was silently cleared — a crash turned into a pass on a security control, to spare a test double. The double is the thing that was wrong: `test_validate_tool.py`'s container answered only `has`, and since #410 the rule asks what the bound adapter declares. It now resolves to a `READS_REPO`-only adapter, which is the shape those tests mean, so the rule correctly does not fire and they stay about the tool's worktree. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Contributor
Author
in-lockstep review — intentsucceeded · $0.0851 · 3 in / 508 out tokens
|
Contributor
Author
in-lockstep review — performancesucceeded · $0.0744 · 3 in / 67 out tokens No findings. |
Contributor
Author
in-lockstep review — securitysucceeded · $0.0880 · 3 in / 677 out tokens
|
Contributor
Author
in-lockstep review — testssucceeded · $0.0912 · 3 in / 912 out tokens
|
tpouyer
deleted the
in-lockstep/implement/410/implement-propose-34408733138-20260909T223934Z-8d0d
branch
September 9, 2026 23:10
tpouyer
added a commit
that referenced
this pull request
Sep 10, 2026
…not a template for it (#420) `main` went red on run 34415733664: `run selfcheck`'s container job, `make lint typecheck exited 2`. #418's binding change is the trigger and this restores green. What it did, in order. `make lint` shells out to `uv run`; `uv run` found `/work/.venv` — the working tree's venv, writable because a working tree has to be, where `mounts` are `:ro` — and its interpreter symlink dangled, the image carrying CPython 3.11.14 where the runner had built that venv with 3.11.16 on a mutable image tag. uv's remedy for a broken venv is to delete and rebuild it, rebuilding wants a network the sandbox correctly does not have, and the deletion took the suite down with it in the same job: `test errored (pytest is not installed)`. The identical code was green on the pull request half an hour earlier; the only difference in the logs is `Downloaded newer image`. The general shape, which #419 holds: a contained command that repairs its environment in place can destroy state the rest of the run depends on, through the working tree rather than through a bind mount. And the specific one, recorded beside both bindings because it is the trap that produced this: `PytestTest` runs `python -m pytest` and site-packages are path-independent, so mounting the host's `.venv` works. `uv run mypy` executes a console script whose shebang carries the host's absolute venv path, which nothing mounted makes exist inside the image. Measured under `--network=none`: `UV_NO_SYNC` does not prevent the delete, and `UV_PROJECT_ENVIRONMENT` inside the image does but then falls through to `PATH`, which serves `ruff` and not `mypy`. `test_gate_sandbox_2_every_binding_that_executes_what_a_model_wrote_is_contained` goes with the binding. It is the right test and this repository does not pass it: `staged_refusal` now refuses this Validate for a model-staged run, so `/implement` and `/fix` here do not check what they wrote. A test asserting something the repository does not do is worse than the gap it hides, so the gap is stated in the module and in #419 and the test returns with the fix. Refs #419 Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Makes
GATE-SANDBOX-2—the rule that a binding executing model-authored code must run in a container—apply to Validate as it does to Test and Build. The issue:CommandValidatebound tomake lintexecutes recipes, and a model can author config files those recipes read (mypy.ini,ruff.toml,eslint.config.js), turning them into arbitrary execution on the host. The fix adds anEXECUTES_CODEcapability declaration to adapters;staged_refusalchecks for it before allowing materialisation; and both call sites (_checkedand_validate_runner) now refuse bindings that execute without a container.RuffValidatedeclaresREADS_REPOonly, so it is unaffected. Detection's bindings carry the appropriate declaration.What changed:
EXECUTES_CODEalongsideREADS_REPO, because a repository's ownmake lintruns recipes and a model can author the files those recipes read.READS_REPOonly, because ruff is a binary parser and its TOML config cannot execute arbitrary code.staged_refusalfunction now checks whether an adapter declaresEXECUTES_CODEand skips the container check if it does not, allowing bindings likeRuffValidateto proceed unaffected._checkedfunction callsstaged_refusal(ctx, Validate)before materialising the tree, refusing anyCommandValidatewithout a container.run_validatetool's_validate_runnerfunction callsstaged_refusal(ctx, Validate)before materialising, refusing the same way and returning an error the model sees.make …,npm run …) carryEXECUTES_CODEto indicate they execute model-authored files; pure tool bindings like ruff do not.design/gates.mdupdated to reflect that the container rule now applies to Validate when the binding declaresEXECUTES_CODE, citing fix(sandbox): the container rule is not asked of Validate, and a make-based binding executes the tree #410.test_validate_sandbox.pyverifies the declaration, the refusal at both call sites, and that read-only bindings are unaffected.Worth a closer look:
CommandValidate(cmd, sandbox=Sandbox())without an image will now be refused at validation time. This is the intended breaking change to close a security hole, but deployments using such bindings will need to update them to provide a container image.staged_refusalnow requires the container to have aresolvemethod; test doubles that only implementhaswill not be interrogated for capabilities and will not trigger the refusal. This is tolerant but worth noting for test design.CommandValidatepick up the new declaration immediately—there is no per-binding override here.Closes #410
The ticket body is untrusted input to a model that held write tools, so review this as you would a change from a stranger who had read your repository — the controls bound where it could write, not what it thought.
Tests: ✅ 2879 passed, 15 skipped, run against the staged change before it was proposed.
Checks: clean — the repository's own checks passed over this change
in-lockstep
{ "In-Lockstep-Run": "implement-propose-34408733138-20260909T223934Z-8d0d", "Ticket": "#410" }