Skip to content

✨ feat(build): make the embedded SBOM exhaustive - #3270

Merged
gaborbernat merged 4 commits into
pypa:mainfrom
gaborbernat:sbom-exhaustive
Sep 19, 2026
Merged

gaborbernat merged 4 commits into
pypa:mainfrom
gaborbernat:sbom-exhaustive

Conversation

@gaborbernat

@gaborbernat gaborbernat commented Sep 19, 2026 •

Copy link
Copy Markdown
Contributor

The SBOM that ships in the wheel today lists the bundled pip and setuptools wheels and little else, and validating it against the CycloneDX 1.6 JSON schema with cyclonedx-python-lib fails: the metadata.tools entry added in #3269 carries a vendor key, which was a field of the deprecated flat tools array and is not legal on a component. The hand-rolled field checks in tasks/validate_sbom.py had no way to catch that. 🔍 Measured against CISA's 2024 baseline attributes, the document also lacked an SBOM author, a lifecycle phase, supplier, relationship completeness, and the license and copyright of the primary component. PEP 770 names the build tools and environment as what a third party needs to verify build reproducibility, and the document recorded none of that.

The document now carries everything the build can state as fact. The root component comes from the project metadata and LICENSE: description, SPDX license expression, copyright line, maintainers, classifiers, keywords, [project.urls] mapped to reference types with the same label table cyclonedx-py uses, plus PyPI, the PyPI attestation location, changelog, security policy and advisories links, and the source revision from GITHUB_SHA or git rev-parse. The hook describes each bundled wheel from its own METADATA and RECORD: license, authors, summary, project URLs, Requires-Dist, Requires-Python, classifiers, the wheel's SHA-256, every file inside it with its hash and size, identity evidence stating how it was identified, and which Python versions receive it, from BUNDLE_SUPPORT. The declared runtime dependencies appear with their specifier and marker, without a version since they resolve at install time, and a compositions entry declares the root's dependency list incomplete for that reason. metadata.tools enumerates every distribution in the isolated build environment via importlib.metadata with the same fields and file listings, with the dependency graph between them derived from Requires-Dist, alongside the interpreter (implementation, compiler, build) and the OS (kernel, machine, os-release on Linux). A formulation workflow ties the tools to the produced wheel, and when the build runs in GitHub Actions it records the run URL, commit, ref and runner from an allowlist of variable names; the hook reads nothing else from the environment. The document also declares its own license and a build lifecycle phase. 🧾

Two accuracy problems surfaced along the way. codespell had rewritten the CycloneDX acknowledgement key to American spelling after the last schema validation ran, which the schema rejects; the key is now on codespell's ignore list. The timestamp fell back to hatchling's fixed 2020 value whenever SOURCE_DATE_EPOCH was unset, so the SBOM in the released 21.8.1 wheel claims a creation date of 2020-02-02; the hook now honors SOURCE_DATE_EPOCH and otherwise records the real time, and release.yaml pins SOURCE_DATE_EPOCH to the release commit so release artifacts become reproducible. The serial number is a uuid5 over the document body, so two builds with the same SOURCE_DATE_EPOCH and build environment produce a byte-identical file, which I confirmed by building twice and comparing. The build tool file listings skip console-script launchers since their shebang embeds the build environment's path, which made consecutive builds differ.

tasks/validate_sbom.py now validates the document with cyclonedx-python-lib's strict 1.6 schema validator, added to the pkg-meta group, and checks that every bom-ref is unique, including nested file components, and that every reference in dependencies and the workflow resolves. I confirmed it rejects a copy of the wheel with the vendor key re-injected. The document is about 1.3 MB uncompressed and 142 KB inside the wheel; the file listings for the four bundled wheels and the build environment account for most of it.

The SBOM shipped so far described the bundled wheels and nothing else, and
its metadata.tools entry carried a `vendor` key that is not valid on a
CycloneDX component, so the document did not conform to the 1.6 schema.
The hand-rolled validator could not see that; only a real schema
validation can.

Measured against CISA's 2024 baseline attributes the document also lacked
an SBOM author, lifecycle, supplier, relationship completeness, and the
primary component's license and copyright. PEP 770 names the build tools
and environment as what a third party needs to verify reproducibility, and
none of that was recorded either.

The document now describes the root component from the project metadata
and LICENSE, each bundled wheel from its own METADATA and bytes (no more
hardcoded license), the declared runtime dependencies with their
specifiers and markers, and every distribution in the isolated build
environment along with the dependency graph between them, the
interpreter and the OS. When built in GitHub Actions the run URL and
commit are recorded on the formulation workflow. Only an explicit
allowlist of environment variable names is ever read.

CI now validates the document with cyclonedx-python-lib's strict 1.6
schema validator and checks that every bom-ref resolves, so a malformed
document fails on the pull request rather than at release attestation.
Named groups for the RFC 4122 fields read better than a run of hex
quantifiers, and the error message no longer prints the raw pattern.
codespell had rewritten the CycloneDX `acknowledgement` key to American
spelling after the last schema validation ran, which the schema rejects
since license objects forbid unknown keys. The key is now on codespell's
ignore list.

The timestamp fell back to hatchling's fixed 2020 value whenever
SOURCE_DATE_EPOCH was unset, so released SBOMs claimed a creation date
years in the past. The hook now honors SOURCE_DATE_EPOCH and otherwise
records the real time, and the release workflow pins SOURCE_DATE_EPOCH
to the release commit so release artifacts are reproducible.

Every bundled wheel and build tool now lists its files with the RECORD
hashes and sizes, and bundled wheels carry identity evidence stating how
they were identified. Console-script launchers are excluded from the
build tool listings because their shebang embeds the build env path,
which made consecutive builds differ. The root component records the
exact source revision, the PyPI attestation location, classifiers and
keywords; the interpreter records its compiler and build; Linux hosts
record os-release; the document declares its own license.
@gaborbernat
gaborbernat merged commit 7549b43 into pypa:main Sep 19, 2026
64 checks passed
@gaborbernat gaborbernat added the security Fixes a vulnerability or hardens the supply chain, CI or release label Sep 22, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

bug enhancement security Fixes a vulnerability or hardens the supply chain, CI or release

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant