fix(dogfood): uncontain Validate, and record why the Test binding is not a template for it - #420
Merged
Merged
Conversation
…not a template for it `main` went red on run 34415733664: `run selfcheck`'s container job, `make lint typecheck exited 2`. #418's binding change is the trigger and this restores green. What it did, in order. `make lint` shells out to `uv run`; `uv run` found `/work/.venv` — the working tree's venv, writable because a working tree has to be, where `mounts` are `:ro` — and its interpreter symlink dangled, the image carrying CPython 3.11.14 where the runner had built that venv with 3.11.16 on a mutable image tag. uv's remedy for a broken venv is to delete and rebuild it, rebuilding wants a network the sandbox correctly does not have, and the deletion took the suite down with it in the same job: `test errored (pytest is not installed)`. The identical code was green on the pull request half an hour earlier; the only difference in the logs is `Downloaded newer image`. The general shape, which #419 holds: a contained command that repairs its environment in place can destroy state the rest of the run depends on, through the working tree rather than through a bind mount. And the specific one, recorded beside both bindings because it is the trap that produced this: `PytestTest` runs `python -m pytest` and site-packages are path-independent, so mounting the host's `.venv` works. `uv run mypy` executes a console script whose shebang carries the host's absolute venv path, which nothing mounted makes exist inside the image. Measured under `--network=none`: `UV_NO_SYNC` does not prevent the delete, and `UV_PROJECT_ENVIRONMENT` inside the image does but then falls through to `PATH`, which serves `ruff` and not `mypy`. `test_gate_sandbox_2_every_binding_that_executes_what_a_model_wrote_is_contained` goes with the binding. It is the right test and this repository does not pass it: `staged_refusal` now refuses this Validate for a model-staged run, so `/implement` and `/fix` here do not check what they wrote. A test asserting something the repository does not do is worse than the gap it hides, so the gap is stated in the module and in #419 and the test returns with the fix. Refs #419 Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Contributor
in-lockstep review — intentsucceeded · $0.0226 · 3 in / 530 out tokens
|
Contributor
in-lockstep review — performancesucceeded · $0.0156 · 3 in / 50 out tokens No findings. |
Contributor
in-lockstep review — securitysucceeded · $0.0239 · 3 in / 592 out tokens
|
Contributor
in-lockstep review — testssucceeded · $0.0208 · 3 in / 407 out tokens
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Restores
main, which is red.mainwent red on run 34415733664,Restores
main, which is red.containerjob,uv run in-lockstep run selfcheck. My change in #418 containerised thisrepository's own
Validatebinding, and the reverting PR restores green while this issue holdswhat it cost to learn.
What happened
Read that in order, because two separate things go wrong and the second is worse:
make lintshells out touv run.uv runfound/work/.venv— the working tree's venv,writable because a working tree has to be, where
mountsare:ro— and its interpretersymlink dangled, because the image's Python was 3.11.14 and the runner had built that venv with
3.11.16.
ghcr.io/astral-sh/uv:python3.11-bookwormis a mutable tag, so which patch you getis whatever was pushed last.
sandbox correctly does not have, so lint failed — and the deletion took the suite down with it
in the same job:
test errored (pytest is not installed).The identical code was green on the pull request thirty minutes earlier
(34415227793):
validate succeeded,test succeeded. The only difference in the logs isStatus: Downloaded newer imageand a second interpreter version. So this was not a flake that will pass on a retry; it is
deterministic whenever the image's patch differs from the runner's, which is now.
The part worth keeping
A contained verb deleted state the rest of the job depended on. Not through a bind mount —
those are
:roand did their job — but through the working tree, which cannot be read-only. Anycontained command that repairs its environment in place can do this. It is not specific to uv and
it is not specific to Validate.
Why
Testworks andValidatecannot, on the same sandbox.PytestTestrunspython -m pytestwith the mounted.venv's site-packages onPYTHONPATH, and site-packages arepath-independent within one minor version.
make typecheckrunsuv run mypy, which executes aconsole script whose shebang carries the host's absolute venv path. Nothing mounted makes that
path exist inside the image. The Test binding is not a template Validate can copy; it is a trick
that works for module invocation and not for console scripts.
Measured rather than reasoned, in
ghcr.io/astral-sh/uv:python3.11-bookwormwith--network=none:UV_NO_SYNC=1aloneRemoved virtual environment at: /venv, then rebuildsUV_PROJECT_ENVIRONMENTat a path inside the imageuv runfalls through toPATHuv run ruffPATHuv run mypyFull analysis and the options are in #419. This PR is the revert only.
Verification
make checkgreen from the committed tree: 2889 passed, 6 skipped, 270 files unchanged by the formatter.uv run in-lockstep run selfchecklocally:validate succeededagain.