✨ feat(build): make the embedded SBOM exhaustive - #3270
Merged
Merged
Conversation
The SBOM shipped so far described the bundled wheels and nothing else, and its metadata.tools entry carried a `vendor` key that is not valid on a CycloneDX component, so the document did not conform to the 1.6 schema. The hand-rolled validator could not see that; only a real schema validation can. Measured against CISA's 2024 baseline attributes the document also lacked an SBOM author, lifecycle, supplier, relationship completeness, and the primary component's license and copyright. PEP 770 names the build tools and environment as what a third party needs to verify reproducibility, and none of that was recorded either. The document now describes the root component from the project metadata and LICENSE, each bundled wheel from its own METADATA and bytes (no more hardcoded license), the declared runtime dependencies with their specifiers and markers, and every distribution in the isolated build environment along with the dependency graph between them, the interpreter and the OS. When built in GitHub Actions the run URL and commit are recorded on the formulation workflow. Only an explicit allowlist of environment variable names is ever read. CI now validates the document with cyclonedx-python-lib's strict 1.6 schema validator and checks that every bom-ref resolves, so a malformed document fails on the pull request rather than at release attestation.
Named groups for the RFC 4122 fields read better than a run of hex quantifiers, and the error message no longer prints the raw pattern.
codespell had rewritten the CycloneDX `acknowledgement` key to American spelling after the last schema validation ran, which the schema rejects since license objects forbid unknown keys. The key is now on codespell's ignore list. The timestamp fell back to hatchling's fixed 2020 value whenever SOURCE_DATE_EPOCH was unset, so released SBOMs claimed a creation date years in the past. The hook now honors SOURCE_DATE_EPOCH and otherwise records the real time, and the release workflow pins SOURCE_DATE_EPOCH to the release commit so release artifacts are reproducible. Every bundled wheel and build tool now lists its files with the RECORD hashes and sizes, and bundled wheels carry identity evidence stating how they were identified. Console-script launchers are excluded from the build tool listings because their shebang embeds the build env path, which made consecutive builds differ. The root component records the exact source revision, the PyPI attestation location, classifiers and keywords; the interpreter records its compiler and build; Linux hosts record os-release; the document declares its own license.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The SBOM that ships in the wheel today lists the bundled pip and setuptools wheels and little else, and validating it against the CycloneDX 1.6 JSON schema with
cyclonedx-python-libfails: themetadata.toolsentry added in #3269 carries avendorkey, which was a field of the deprecated flat tools array and is not legal on a component. The hand-rolled field checks intasks/validate_sbom.pyhad no way to catch that. 🔍 Measured against CISA's 2024 baseline attributes, the document also lacked an SBOM author, a lifecycle phase, supplier, relationship completeness, and the license and copyright of the primary component. PEP 770 names the build tools and environment as what a third party needs to verify build reproducibility, and the document recorded none of that.The document now carries everything the build can state as fact. The root component comes from the project metadata and
LICENSE: description, SPDX license expression, copyright line, maintainers, classifiers, keywords,[project.urls]mapped to reference types with the same label tablecyclonedx-pyuses, plus PyPI, the PyPI attestation location, changelog, security policy and advisories links, and the source revision fromGITHUB_SHAorgit rev-parse. The hook describes each bundled wheel from its ownMETADATAandRECORD: license, authors, summary, project URLs,Requires-Dist,Requires-Python, classifiers, the wheel's SHA-256, every file inside it with its hash and size, identity evidence stating how it was identified, and which Python versions receive it, fromBUNDLE_SUPPORT. The declared runtime dependencies appear with their specifier and marker, without a version since they resolve at install time, and acompositionsentry declares the root's dependency list incomplete for that reason.metadata.toolsenumerates every distribution in the isolated build environment viaimportlib.metadatawith the same fields and file listings, with the dependency graph between them derived fromRequires-Dist, alongside the interpreter (implementation, compiler, build) and the OS (kernel, machine,os-releaseon Linux). Aformulationworkflow ties the tools to the produced wheel, and when the build runs in GitHub Actions it records the run URL, commit, ref and runner from an allowlist of variable names; the hook reads nothing else from the environment. The document also declares its own license and abuildlifecycle phase. 🧾Two accuracy problems surfaced along the way. codespell had rewritten the CycloneDX
acknowledgementkey to American spelling after the last schema validation ran, which the schema rejects; the key is now on codespell's ignore list. The timestamp fell back to hatchling's fixed 2020 value wheneverSOURCE_DATE_EPOCHwas unset, so the SBOM in the released 21.8.1 wheel claims a creation date of 2020-02-02; the hook now honorsSOURCE_DATE_EPOCHand otherwise records the real time, andrelease.yamlpinsSOURCE_DATE_EPOCHto the release commit so release artifacts become reproducible. The serial number is auuid5over the document body, so two builds with the sameSOURCE_DATE_EPOCHand build environment produce a byte-identical file, which I confirmed by building twice and comparing. The build tool file listings skip console-script launchers since their shebang embeds the build environment's path, which made consecutive builds differ.tasks/validate_sbom.pynow validates the document withcyclonedx-python-lib's strict 1.6 schema validator, added to thepkg-metagroup, and checks that everybom-refis unique, including nested file components, and that every reference independenciesand the workflow resolves. I confirmed it rejects a copy of the wheel with thevendorkey re-injected. The document is about 1.3 MB uncompressed and 142 KB inside the wheel; the file listings for the four bundled wheels and the build environment account for most of it.