diff --git a/.github/THREAT_MODEL.md b/.github/THREAT_MODEL.md index db807efb1..c6497fe04 100644 --- a/.github/THREAT_MODEL.md +++ b/.github/THREAT_MODEL.md @@ -149,7 +149,7 @@ flowchart LR main["main branch
ruleset: pull request, checks"] pre["pre-release.yaml
release commit and tag"] tag["Tag *.*.*
tag ruleset"] - build["release.yaml build
no secrets"] + build["release.yaml build
no publishing credentials"] publish["release.yaml publish
environment: release"] verify["release.yaml verify jobs
no publishing credentials"] pypi[("PyPI
sdist, wheel, attestations")] @@ -181,12 +181,12 @@ flowchart LR class pypi,ghr,getv,boot output; ``` -| Boundary | Crossing | Controls | -| -------- | -------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | -| TB1 | Caller-supplied values into activation scripts and `pyvenv.cfg` | [Per-shell quoting][src-activation], [line boundary collapsing][src-text], [property tests][tests-property] and [fuzzing][atheris] | -| TB2 | Wheel bytes from an index through app-data into an environment | pip's TLS, SHA-256 comparison with PyPI's published digest, refusal when PyPI has none ([#3302][pr-3302]) | -| TB3 | Built artifacts from GitHub Actions to PyPI, GitHub Releases, get-virtualenv and bootstrap.pypa.io | [Trusted publishing][pypi-tp], [attestations][gh-attestations], [post-publish verification][workflow-release] of each destination | -| TB4 | Code from contributors and dependencies into a release | [Rulesets][gh-rulesets], [SHA-pinned actions][gh-sha-pinning], [embedded wheel hashes][src-embed-init], [wheel age gate][task-wheel-age], [pinned CI tool downloads][ci-tools] | +| Boundary | Crossing | Controls | +| -------- | -------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | +| TB1 | Caller-supplied values into activation scripts and `pyvenv.cfg` | [Per-shell quoting][src-activation], [line boundary collapsing][src-text], [property tests][tests-property] and [fuzzing][atheris] | +| TB2 | Wheel bytes from an index through app-data into an environment | pip's TLS, SHA-256 comparison with PyPI's published digest, refusal when PyPI has none ([#3302][pr-3302]) | +| TB3 | Built artifacts from GitHub Actions to PyPI, GitHub Releases, get-virtualenv and bootstrap.pypa.io | [Trusted publishing][pypi-tp], [attestations][gh-attestations], [immutable releases][gh-immutable], [post-publish verification][workflow-release] of each destination | +| TB4 | Code from contributors and dependencies into a release | [Rulesets][gh-rulesets], [SHA-pinned actions][gh-sha-pinning], [embedded wheel hashes][src-embed-init], [wheel age gate][task-wheel-age], [pinned CI tool downloads][ci-tools], [zipapp lock][pylock-zipapp] | ## Threats and existing mitigations @@ -218,9 +218,9 @@ risk to lowest. ### Repudiation -| ID | Threat | Risk | Mitigation | -| --- | --------------------------------------------------------------------------- | ---- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | -| R1 | A user cannot tell whether a published artifact came from the project's CI. | Low | [PyPI attestations][pypi-attestations] cover the sdist and wheel, and a [SLSA build provenance][slsa-provenance] attestation ships beside the zipapp. [GitHub attestations][gh-attestations] bind the CycloneDX SBOM and its [SPDX] rendering to both distributions ([#3299][pr-3299]) and the zipapp SBOM to the zipapp ([#3310][pr-3310]). The wheel carries its SBOM as [PEP 770][pep-770] describes. The release pins timestamps to [`SOURCE_DATE_EPOCH`][sde], so anyone can [rebuild the sdist][verify-rebuild] and compare it; wheels differ between build machines, see open items. [Verify a virtualenv release][verify] shows each check. We do not require signed commits. | +| ID | Threat | Risk | Mitigation | +| --- | --------------------------------------------------------------------------- | ---- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | +| R1 | A user cannot tell whether a published artifact came from the project's CI. | Low | [PyPI attestations][pypi-attestations] cover the sdist and wheel, and a [SLSA build provenance][slsa-provenance] attestation ships beside the zipapp. [GitHub attestations][gh-attestations] bind the CycloneDX SBOM and its [SPDX] rendering to both distributions ([#3299][pr-3299]) and the zipapp SBOM to the zipapp ([#3310][pr-3310]). The wheel carries its SBOM as [PEP 770][pep-770] describes. GitHub signs a release attestation over the assets of each immutable release. The release pins timestamps to [`SOURCE_DATE_EPOCH`][sde] and the zipapp's bundled wheels to a [PEP 751][pep-751] lock, and the SBOMs record the build tools, so anyone can [rebuild the sdist, wheel and zipapp][verify-rebuild] and compare the bytes ([#3306][pr-3306], [#3311][pr-3311]). [Verify a virtualenv release][verify] shows each check. We do not require signed commits. | ### Information disclosure @@ -262,7 +262,7 @@ close submissions with no human in the loop. T3, T5 and T6 cover compromised dependencies and embedded wheels, and S2 covers registry typosquatting. Tools that install virtualenv unpinned take each new release on the day PyPI publishes it. We cannot slow that down for them, so we -keep releases verifiable with attestations, an sdist and wheel that [rebuild byte for byte][ra-repro], SBOMs and +keep releases verifiable with attestations, an sdist, wheel and zipapp that [rebuild byte for byte][ra-repro], SBOMs and [published advisories][advisories]. The [CRA section][security-cra] of SECURITY.md points integrators to the same material, and the project reports its practices through [OpenSSF Scorecard][scorecard-virtualenv] and the [OpenSSF Best Practices badge][bestpractices-virtualenv]. @@ -347,8 +347,9 @@ with no token permissions and grants each job the permissions it needs and no mo to contents. The repository's [default workflow token][gh-token-permissions] is read-only, and GitHub [rejects actions not pinned to a full commit SHA][gh-sha-pinning]. Checkouts do not persist credentials, except in [pre-release.yaml][workflow-pre-release], which pushes the release commit and tag with a release App token, and the -upgrade publish job, which pushes with a deploy key. PyPI uploads use trusted publishing (S1). In the tool, the app-data -seeder [marks the extracted wheel image read-only][src-symlink] when it links packages by symlink. +upgrade publish job, which pushes the upgrade branch with the workflow token. PyPI uploads use trusted publishing (S1). +In the tool, the app-data seeder [marks the extracted wheel image read-only][src-symlink] when it links packages by +symlink. virtualenv uses fail-safe defaults on the network and in the batch activator. If the TLS handshake with the PyPI metadata API fails, virtualenv does not retry without verification; the unverified fallback needs the @@ -486,8 +487,10 @@ the workflows as I1 describes, and [scorecard.yaml][workflow-scorecard] runs [Op [pr-3306]: https://github.com/pypa/virtualenv/pull/3306 [pr-3309]: https://github.com/pypa/virtualenv/pull/3309 [pr-3310]: https://github.com/pypa/virtualenv/pull/3310 +[pr-3311]: https://github.com/pypa/virtualenv/pull/3311 [precommit-ci]: https://pre-commit.ci/ [precommit-config]: https://github.com/pypa/virtualenv/blob/main/.pre-commit-config.yaml +[pylock-zipapp]: https://github.com/pypa/virtualenv/blob/main/pylock.zipapp.toml [pypa-bootstrap]: https://github.com/pypa/bootstrap [pypi-2fa]: https://blog.pypi.org/posts/2024-01-01-2fa-enforced/ [pypi-attestations]: https://docs.pypi.org/attestations/ @@ -549,7 +552,7 @@ the workflows as I1 describes, and [scorecard.yaml][workflow-scorecard] runs [Op [usage-insecure]: https://virtualenv.pypa.io/en/latest/how-to/usage.html#allow-unverified-https-for-periodic-updates [usage-override-app-data]: https://virtualenv.pypa.io/en/latest/how-to/usage.html#override-app-data-location [verify]: https://virtualenv.pypa.io/en/latest/how-to/verify-release.html -[verify-rebuild]: https://virtualenv.pypa.io/en/latest/how-to/verify-release.html#rebuild-the-sdist +[verify-rebuild]: https://virtualenv.pypa.io/en/latest/how-to/verify-release.html#rebuild-the-release-files [workflow-check]: https://github.com/pypa/virtualenv/blob/main/.github/workflows/check.yaml [workflow-codeql]: https://github.com/pypa/virtualenv/blob/main/.github/workflows/codeql.yaml [workflow-pre-release]: https://github.com/pypa/virtualenv/blob/main/.github/workflows/pre-release.yaml diff --git a/docs/changelog/3330.doc.rst b/docs/changelog/3330.doc.rst new file mode 100644 index 000000000..c4e3fe039 --- /dev/null +++ b/docs/changelog/3330.doc.rst @@ -0,0 +1,3 @@ +Show how to check a release with ``gh release verify`` and ``gh release verify-asset``, and how to rebuild the zipapp +byte for byte with the build tool versions its SBOMs list. Drop the claims that the zipapp build does not pin the +distributions it bundles and that wheels differ between build machines - by :user:`gaborbernat`. diff --git a/docs/explanation.rst b/docs/explanation.rst index 9add9452c..4241b5025 100644 --- a/docs/explanation.rst +++ b/docs/explanation.rst @@ -598,12 +598,18 @@ involved. zipapp and the SBOM attestations. Verifying a file recomputes its digest and checks it against a signed statement whose identity you name. +**Immutable releases** + From 21.11.0 on, once the workflow publishes a GitHub release, GitHub refuses changes to its tag and assets and + signs a release attestation over the tag's commit and each asset's SHA-256. A stolen maintainer token cannot swap + the zipapp or an SBOM on a published release. The release attestation names no workflow, so pair it with the + provenance check. + **Reproducible builds** Provenance tells you which workflow run built a file, but you still trust that run. The release pins every timestamp - to ``SOURCE_DATE_EPOCH``, the commit time of the tag, so you can rebuild the sdist and the wheel from the tag - yourself and compare the bytes. The wheel's SBOM records the Python version and build backend the release used, - which a rebuild must match. It leaves out the machine that ran the build, which the provenance attestation already - names. + to ``SOURCE_DATE_EPOCH``, the commit time of the tag, so you can rebuild the sdist, the wheel and the zipapp from + the tag yourself and compare the bytes. The SBOMs record the Python version and build tools the release used, which + a rebuild must match, and a lock file pins the distributions the zipapp bundles. The SBOMs leave out the machine + that ran the build, which the provenance attestation already names. **The SBOMs** Dependency scanners find the packages a project declares, and virtualenv declares neither ``pip`` nor diff --git a/docs/how-to/verify-release.rst b/docs/how-to/verify-release.rst index b85f16cc4..7440a036e 100644 --- a/docs/how-to/verify-release.rst +++ b/docs/how-to/verify-release.rst @@ -4,7 +4,7 @@ The ``release.yaml`` workflow in `pypa/virtualenv `_ builds and publishes every release. The steps below check that a file you downloaded came out of that workflow, and show you what the release -bundles. The examples use release ``21.10.0``; replace it with the version you have. +bundles. The examples use release ``21.12.1``; replace it with the version you have. :doc:`../reference/release-artifacts` lists every file a release publishes, and :ref:`release-integrity` covers what these checks prove. @@ -20,21 +20,49 @@ Download the wheel and the sdist without installing them: .. code-block:: console - $ uvx pip download virtualenv==21.10.0 --no-deps --dest . - $ uvx pip download virtualenv==21.10.0 --no-deps --no-binary :all: --dest . + $ uvx pip download virtualenv==21.12.1 --no-deps --dest . + $ uvx pip download virtualenv==21.12.1 --no-deps --no-binary :all: --dest . Check each file against the attestation PyPI stores for it, one file per call: .. code-block:: console - $ uvx pypi-attestations verify pypi --repository https://github.com/pypa/virtualenv virtualenv-21.10.0-py3-none-any.whl - OK: virtualenv-21.10.0-py3-none-any.whl - $ uvx pypi-attestations verify pypi --repository https://github.com/pypa/virtualenv virtualenv-21.10.0.tar.gz - OK: virtualenv-21.10.0.tar.gz + $ uvx pypi-attestations verify pypi --repository https://github.com/pypa/virtualenv virtualenv-21.12.1-py3-none-any.whl + OK: virtualenv-21.12.1-py3-none-any.whl + $ uvx pypi-attestations verify pypi --repository https://github.com/pypa/virtualenv virtualenv-21.12.1.tar.gz + OK: virtualenv-21.12.1.tar.gz A modified file fails with ``subject does not match distribution digest``, and a file signed by another repository fails with ``provenance was signed by repository ...``. To check the copy on PyPI without downloading it first, prefix the -file name with ``pypi:``, as in ``pypi:virtualenv-21.10.0-py3-none-any.whl``. +file name with ``pypi:``, as in ``pypi:virtualenv-21.12.1-py3-none-any.whl``. + +************************************** + Verify the GitHub release and assets +************************************** + +Releases from 21.11.0 on are `immutable +`_: +GitHub signs a release attestation that binds the tag to its commit and to the SHA-256 of every asset, and refuses any +later change to the tag or the assets. Check that attestation: + +.. code-block:: console + + $ gh release verify 21.12.1 --repo pypa/virtualenv + Resolved tag 21.12.1 to sha1:befec5eae075d1c4cb00a41d0c72bcdb91bf4586 + Loaded attestation from GitHub API + ✓ Release 21.12.1 verified! + +Then check a file you downloaded against the digests the attestation lists. The command hashes the local file, so it +works for a copy under any name: + +.. code-block:: console + + $ gh release download 21.12.1 --repo pypa/virtualenv --pattern virtualenv.pyz + $ gh release verify-asset 21.12.1 virtualenv.pyz --repo pypa/virtualenv + ✓ Verification succeeded! virtualenv.pyz is present in release 21.12.1 + +GitHub signs the release attestation itself, so it shows the assets did not change after publication. It does not name +the workflow that built them; the provenance check below does. ******************* Verify the zipapp @@ -44,26 +72,25 @@ Download the zipapp and its provenance bundle from the GitHub release, then veri .. code-block:: console - $ gh release download 21.10.0 --repo pypa/virtualenv --pattern virtualenv.pyz --pattern virtualenv.pyz.intoto.jsonl + $ gh release download 21.12.1 --repo pypa/virtualenv --pattern virtualenv.pyz --pattern virtualenv.pyz.intoto.jsonl $ gh attestation verify virtualenv.pyz --repo pypa/virtualenv --bundle virtualenv.pyz.intoto.jsonl \ --signer-workflow pypa/virtualenv/.github/workflows/release.yaml -Add ``--source-ref refs/tags/21.10.0`` to also require that the build ran from the ``21.10.0`` tag. Without +Add ``--source-ref refs/tags/21.12.1`` to also require that the build ran from the ``21.12.1`` tag. Without ``--bundle``, ``gh`` fetches the attestation from GitHub instead of reading the local file. The zipapp at ``https://bootstrap.pypa.io/virtualenv.pyz`` comes from `pypa/get-virtualenv `_ and can trail the latest release. ``python virtualenv.pyz --version`` shows -which release you have. Verify it with ``gh attestation verify`` as above, leaving out ``--bundle``. A release without a -provenance bundle has no attestation to check; compare the file's SHA-256 with the digest GitHub lists for the release -asset instead: +which release you have. Verify it with ``gh release verify-asset`` for that release, or with ``gh attestation verify`` +as above, leaving out ``--bundle``. Releases before 21.7.11 have neither a release attestation nor a provenance bundle; +compare the file's SHA-256 with the digest GitHub lists for the release asset instead: .. code-block:: console - $ gh release view 21.10.0 --repo pypa/virtualenv --json assets --jq '.assets[] | .name + " " + .digest' - virtualenv.pyz sha256:345775312f24d272017d7c640b3184414fa1779152e28aa9f082491766d5fb38 - virtualenv.pyz.intoto.jsonl sha256:e2486ddcc38fa254afda37cd325dc14db45b42519235eb7e304388ea6afd3e2e + $ gh release view 21.7.10 --repo pypa/virtualenv --json assets --jq '.assets[] | .name + " " + .digest' + virtualenv.pyz sha256:06ee4ea84517e9b8565f7ee81c064d2de5e83dad3874d2babd7b94b2b40c4595 $ shasum -a 256 virtualenv.pyz - 345775312f24d272017d7c640b3184414fa1779152e28aa9f082491766d5fb38 virtualenv.pyz + 06ee4ea84517e9b8565f7ee81c064d2de5e83dad3874d2babd7b94b2b40c4595 virtualenv.pyz ************************ Read the embedded SBOM @@ -74,9 +101,9 @@ virtualenv bundles, with their hashes, licenses and the Python versions each one .. code-block:: console - $ python -m zipfile --extract virtualenv-21.10.0-py3-none-any.whl wheel + $ python -m zipfile --extract virtualenv-21.12.1-py3-none-any.whl wheel $ jq -r '.components[] | select(.hashes) | "\(.name) \(.version) Python \([.properties[] | select(.name == "virtualenv:seeded-for-python").value] | join(","))"' \ - wheel/virtualenv-21.10.0.dist-info/sboms/virtualenv.cdx.json + wheel/virtualenv-21.12.1.dist-info/sboms/virtualenv.cdx.json pip 26.0.1 Python 3.9 pip 26.2.1 Python 3.10,3.11,3.12,3.13,3.14,3.15,3.16 setuptools 82.0.1 Python 3.9 @@ -98,28 +125,26 @@ the release workflow: .. code-block:: console - $ gh attestation verify virtualenv-21.10.0-py3-none-any.whl -R pypa/virtualenv --predicate-type https://cyclonedx.org/bom + $ gh attestation verify virtualenv-21.12.1-py3-none-any.whl -R pypa/virtualenv --predicate-type https://cyclonedx.org/bom To confirm the attested SBOM matches the one inside the wheel, save the attested copy and compare the two. ``diff`` prints nothing when they match: .. code-block:: console - $ gh attestation verify virtualenv-21.10.0-py3-none-any.whl -R pypa/virtualenv --predicate-type https://cyclonedx.org/bom \ + $ gh attestation verify virtualenv-21.12.1-py3-none-any.whl -R pypa/virtualenv --predicate-type https://cyclonedx.org/bom \ --format json --jq '.[0].verificationResult.statement.predicate' > attested.cdx.json - $ diff <(jq -S . attested.cdx.json) <(jq -S . wheel/virtualenv-21.10.0.dist-info/sboms/virtualenv.cdx.json) + $ diff <(jq -S . attested.cdx.json) <(jq -S . wheel/virtualenv-21.12.1.dist-info/sboms/virtualenv.cdx.json) -Newer releases also attach the SBOM as ``virtualenv.cdx.json`` and an SPDX 2.3 rendering of it as -``virtualenv.spdx.json``, and attest the SPDX document against the wheel and the sdist as well. Replace ```` -with a release that has these assets. The CycloneDX asset must match the SBOM in the wheel, and the SPDX asset what the -release attested: +Releases from 21.11.0 on also attach the SBOM as ``virtualenv.cdx.json`` and an SPDX 2.3 rendering of it as +``virtualenv.spdx.json``, and attest the SPDX document against the wheel and the sdist as well. The CycloneDX asset must +match the SBOM in the wheel, and the SPDX asset what the release attested: .. code-block:: console - $ uvx pip download virtualenv== --no-deps --dest . - $ gh release download --repo pypa/virtualenv --pattern virtualenv.cdx.json --pattern virtualenv.spdx.json - $ unzip -p virtualenv--py3-none-any.whl '*.dist-info/sboms/virtualenv.cdx.json' | cmp - virtualenv.cdx.json - $ gh attestation verify virtualenv--py3-none-any.whl -R pypa/virtualenv --predicate-type https://spdx.dev/Document/v2.3 \ + $ gh release download 21.12.1 --repo pypa/virtualenv --pattern virtualenv.cdx.json --pattern virtualenv.spdx.json + $ unzip -p virtualenv-21.12.1-py3-none-any.whl '*.dist-info/sboms/virtualenv.cdx.json' | cmp - virtualenv.cdx.json + $ gh attestation verify virtualenv-21.12.1-py3-none-any.whl -R pypa/virtualenv --predicate-type https://spdx.dev/Document/v2.3 \ --format json --jq '.[0].verificationResult.statement.predicate' > attested.spdx.json $ diff <(jq -S . attested.spdx.json) <(jq -S . virtualenv.spdx.json) @@ -127,12 +152,12 @@ release attested: Read the zipapp SBOM ********************** -Newer releases describe the zipapp in its own CycloneDX SBOM. The zipapp carries it at its root, the release attaches it -as ``virtualenv.pyz.cdx.json``, and GitHub attests it against ``virtualenv.pyz``. Check all three agree: +Releases from 21.11.0 on describe the zipapp in its own CycloneDX SBOM. The zipapp carries it at its root, the release +attaches it as ``virtualenv.pyz.cdx.json``, and GitHub attests it against ``virtualenv.pyz``. Check all three agree: .. code-block:: console - $ gh release download --repo pypa/virtualenv --pattern virtualenv.pyz --pattern virtualenv.pyz.cdx.json + $ gh release download 21.12.1 --repo pypa/virtualenv --pattern virtualenv.pyz --pattern virtualenv.pyz.cdx.json $ unzip -p virtualenv.pyz virtualenv.pyz.cdx.json | cmp - virtualenv.pyz.cdx.json $ gh attestation verify virtualenv.pyz -R pypa/virtualenv --predicate-type https://cyclonedx.org/bom \ --format json --jq '.[0].verificationResult.statement.predicate' > attested.pyz.cdx.json @@ -144,42 +169,67 @@ List the distributions the zipapp bundles and the Python versions that load each $ jq -r '.components[] | "\(.name) \(.version) \([.properties[] | select(.name == "virtualenv:loaded-for-python").value] | join(","))"' \ virtualenv.pyz.cdx.json - -***************************** - Rebuild the sdist and wheel -***************************** + virtualenv 21.12.1 + distlib 0.4.3 3.14,3.13,3.12,3.11,3.10,3.9,3.8 + filelock 3.19.1 3.9,3.8 + filelock 3.32.6 3.14,3.13,3.12,3.11,3.10 + platformdirs 4.11.8 3.14,3.13,3.12,3.11,3.10 + platformdirs 4.4.0 3.9,3.8 + python-discovery 1.6.0 3.14,3.13,3.12,3.11,3.10,3.9,3.8 + typing_extensions 4.16.0 3.10,3.9,3.8 + +*************************** + Rebuild the release files +*************************** The release builds with ``SOURCE_DATE_EPOCH`` set to the commit time of the release tag, so rebuilding the tag yields the same sdist, byte for byte: .. code-block:: console - $ git clone --branch 21.10.0 https://github.com/pypa/virtualenv + $ git clone --branch 21.12.1 https://github.com/pypa/virtualenv $ cd virtualenv $ SOURCE_DATE_EPOCH=$(git log -1 --pretty=%ct) uv build --sdist --out-dir rebuild . - $ cmp rebuild/virtualenv-21.10.0.tar.gz ../virtualenv-21.10.0.tar.gz + $ cmp rebuild/virtualenv-21.12.1.tar.gz ../virtualenv-21.12.1.tar.gz ``cmp`` prints nothing when the files match. -The wheel of a release after 21.10.0 rebuilds byte for byte too, on any operating system and architecture, once the +The wheel of a release from 21.11.0 on rebuilds byte for byte too, on any operating system and architecture, once the Python patch version and the build backend versions match the ones the release used. Its SBOM lists both, so read them from the published wheel and pass them to the build: .. code-block:: console - $ unzip -p ../virtualenv--py3-none-any.whl '*.dist-info/sboms/virtualenv.cdx.json' > published.cdx.json + $ unzip -p ../virtualenv-21.12.1-py3-none-any.whl '*.dist-info/sboms/virtualenv.cdx.json' > published.cdx.json $ jq -r '.metadata.tools.components[] | select(.type == "platform") | .version' published.cdx.json 3.14.7 $ jq -r '.metadata.tools.components[] | select(.purl // "" | startswith("pkg:pypi/")) | "\(.name)==\(.version)"' \ published.cdx.json > build-constraints.txt $ SOURCE_DATE_EPOCH=$(git log -1 --pretty=%ct) uv build --wheel --python 3.14.7 \ --build-constraint build-constraints.txt --out-dir rebuild . - $ cmp rebuild/virtualenv--py3-none-any.whl ../virtualenv--py3-none-any.whl + $ cmp rebuild/virtualenv-21.12.1-py3-none-any.whl ../virtualenv-21.12.1-py3-none-any.whl Build from a git checkout, since the SBOM records the source commit and an sdist does not carry it. Wheels up to 21.10.0 recorded the machine that built them in the SBOM, so a rebuild of those differs in the SBOM and in ``RECORD``, which holds the SBOM's hash. -The zipapp rebuilds byte for byte with ``SOURCE_DATE_EPOCH=$(git log -1 --pretty=%ct) tox r -e zipapp`` on the Python -version its SBOM lists, but only while every package the build pulls from PyPI still resolves to the version the release -used. The zipapp build does not pin them; the zipapp SBOM and the wheel SBOM inside the zipapp list them. +The zipapp of a release from 21.11.0 on rebuilds byte for byte as well. The `pylock.zipapp.toml +`_ lock at the tag pins each distribution it bundles by +version and SHA-256, so only the build tools can drift. Two SBOMs list them: the zipapp SBOM holds the tools of the +``tox`` environment that assembles the zipapp, and the wheel SBOM inside the zipapp holds the build backend for that +wheel. Constrain both, and run on the CPython version the zipapp SBOM lists: + +.. code-block:: console + + $ jq -r '.metadata.tools.components[] | select(.purl // "" | startswith("pkg:pypi/")) | "\(.name)==\(.version)"' \ + ../virtualenv.pyz.cdx.json > zipapp-constraints.txt + $ unzip -p ../virtualenv.pyz 'virtualenv-*.dist-info/sboms/virtualenv.cdx.json' \ + | jq -r '.metadata.tools.components[] | select(.purl // "" | startswith("pkg:pypi/")) | "\(.name)==\(.version)"' \ + > wheel-constraints.txt + $ UV_CONSTRAINT=$PWD/zipapp-constraints.txt PIP_BUILD_CONSTRAINT=$PWD/wheel-constraints.txt \ + SOURCE_DATE_EPOCH=$(git log -1 --pretty=%ct) uvx --with tox-uv tox r -e zipapp -x 'env.zipapp.pass_env+=PIP_BUILD_CONSTRAINT' + $ cmp virtualenv.pyz ../virtualenv.pyz + +The release runs ``tox`` with the ``tox-uv`` plugin as well. The plugin passes ``UV_*`` variables into the environment, +and the ``-x`` override adds ``PIP_BUILD_CONSTRAINT`` for the ``pip wheel`` call that builds the virtualenv wheel inside +the zipapp. diff --git a/docs/reference/release-artifacts.rst b/docs/reference/release-artifacts.rst index ad5dbf255..355b7edc7 100644 --- a/docs/reference/release-artifacts.rst +++ b/docs/reference/release-artifacts.rst @@ -50,7 +50,8 @@ each file, and :ref:`release-integrity` explains what the checks prove. The bootstrap copies come from the ``public`` directory of `pypa/get-virtualenv `_, which the release workflow updates, and can trail the latest release. Releases published before an asset existed do not have it; ``gh release view {version} --repo pypa/virtualenv`` lists -what a release carries. Immutable releases are enabled on the repository, so assets cannot change after publication. +what a release carries. Releases from 21.11.0 on are immutable; GitHub rejects changes to their tag and assets after +publication. ************** Attestations @@ -78,9 +79,13 @@ what a release carries. Immutable releases are enabled on the repository, so ass - - CycloneDX SBOM, predicate ``https://cyclonedx.org/bom`` - ``virtualenv.pyz`` - GitHub attestations API, predicate equal to ``virtualenv.pyz.cdx.json`` + - - GitHub release attestation, predicate ``https://in-toto.io/attestation/release/v0.2`` + - the tag's commit and every GitHub release asset, from 21.11.0 on + - GitHub attestations API, read by ``gh release verify`` -The release workflow signs every attestation through `Sigstore `_ with the identity -``https://github.com/pypa/virtualenv/.github/workflows/release.yaml@refs/tags/{version}``. +The release workflow signs every attestation except the last through `Sigstore `_ with the +identity ``https://github.com/pypa/virtualenv/.github/workflows/release.yaml@refs/tags/{version}``. GitHub signs the +release attestation when it publishes an immutable release, with the identity ``https://dotcom.releases.github.com``. ******* SBOMs @@ -153,6 +158,9 @@ Wheels up to 21.10.0 recorded the build machine in their SBOM, so a rebuild of t ``RECORD``, which holds the SBOM's hash. The zipapp, built by `tasks/make_zipapp.py `_, needs -the same inputs, and its entries carry ``SOURCE_DATE_EPOCH`` as their timestamp and fixed permissions. Its build also -downloads the distributions it bundles and the backend for the wheel inside it from PyPI without pinning them, so a -rebuild matches only while those resolve to the versions the release used. +the same inputs, and its entries carry ``SOURCE_DATE_EPOCH`` as their timestamp and fixed permissions. The build takes +the distributions it bundles from `pylock.zipapp.toml +`_, a `PEP 751 `_ +lock that pins each wheel by SHA-256, and fails on a hash mismatch. A rebuild of a release from 21.11.0 on also needs +the versions of the tools that built it: the zipapp SBOM lists the packages of the build environment, and the wheel SBOM +inside the zipapp lists the backend that built that wheel.