Skip to content

Add PEP 751 pylock.toml resolved-dependency support - #208

Merged
bact merged 36 commits into
mainfrom
lock-formats-pep751
Sep 8, 2026
Merged

bact merged 36 commits into
mainfrom
lock-formats-pep751

Conversation

@bact

@bact bact commented Sep 4, 2026

Copy link
Copy Markdown
Owner

Summary

Add support for PEP 751 pylock.toml and other lock file formats

Related issue

Checklist

  • Tests pass (pytest)
  • Code formatted (ruff format)
  • Lints pass
    (ruff check src/ tests/,
    pylint src/ tests,
    mypy examples/ src/ tests/,
    pyright examples/ src/ tests/,
    pyrefly check examples/ src/ tests/)
  • CHANGELOG.md updated (if user-facing)
  • Docs updated (README.md / working-docs/ / docs/, if applicable)
  • Commits are signed off (git commit -s) -- see
    CONTRIBUTING.md

pylock.toml is build-backend-agnostic, so it's checked unconditionally
in read_pyproject() rather than gated behind [tool.poetry] detection
like poetry.lock.

Reuses ProjectMetadata.locked_dependencies and the
existing additive dependsOn/completeness wiring poetry.lock already
built.

When both poetry.lock and pylock.toml are present, pylock.toml
(the PEP 751 standard) takes priority, with a WARNING naming the
override.

Signed-off-by: Arthit Suriyawongkul <arthit@gmail.com>
@bact bact added the enhancement New feature or request label Sep 4, 2026
Signed-off-by: Arthit Suriyawongkul <arthit@gmail.com>
@codecov-commenter

codecov-commenter commented Sep 4, 2026

Copy link
Copy Markdown

⚠️ Please install the 'codecov app svg image' to ensure uploads and comments are reliably processed by Codecov.

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 99.97%. Comparing base (49f7faa) to head (b2cf21f).
⚠️ Report is 40 commits behind head on main.
❗ Your organization needs to install the Codecov GitHub app to enable full functionality.

Additional details and impacted files
@@           Coverage Diff            @@
##             main     #208    +/-   ##
========================================
  Coverage   99.97%   99.97%            
========================================
  Files         114      123     +9     
  Lines        8413     9271   +858     
  Branches     1476     1642   +166     
========================================
+ Hits         8411     9269   +858     
- Misses          1        2     +1     
+ Partials        1        0     -1     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

bact added 6 commits September 4, 2026 17:08
Signed-off-by: Arthit Suriyawongkul <arthit@gmail.com>
Signed-off-by: Arthit Suriyawongkul <arthit@gmail.com>
Signed-off-by: Arthit Suriyawongkul <arthit@gmail.com>
Signed-off-by: Arthit Suriyawongkul <arthit@gmail.com>
Signed-off-by: Arthit Suriyawongkul <arthit@gmail.com>
Signed-off-by: Arthit Suriyawongkul <arthit@gmail.com>

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

uv transitive resolution, empty-lock precedence, version validation, future dates, and oversized fixtures remain unresolved.

Once you've addressed the issues Copilot identified, you can request another Copilot review.

Pull request overview

Adds resolved-dependency extraction from PEP 751 and several ecosystem lock formats, integrating them into Source SBOM generation.

Changes:

  • Adds a priority cascade for pylock.toml, uv, Poetry, PDM, Pipenv, and pinned requirements.
  • Adds stage controls, provenance-aware document identity, and extensive tests.
  • Documents precedence and introduces real-world lock fixtures.
File summaries
File Description
CHANGELOG.md Records lock-format support.
pyproject.toml Excludes vendored fixtures from builds.
docs/cli.md Documents CLI lock handling.
docs/dependency-sources.md Explains sources and precedence.
docs/index.md Links dependency documentation.
docs/resources.md Adds PEP 751 resource.
working-docs/design/lock-files.md Updates lock-format status.
working-docs/design/roadmap.md Marks cascade formats complete.
working-docs/implementation/lock-file-cascade.md Documents cascade implementation.
working-docs/implementation/pep751-pylock-support.md Documents PEP 751 implementation.
src/pitloom/assemble/_model_generator.py Includes lock provenance in identity.
src/pitloom/assemble/spdx3/document.py Seeds UUID with lock provenance.
src/pitloom/cli/commands/embed_wheel.py Disables source locks during embedding.
src/pitloom/cli/options.py Skips locks during config lookup.
src/pitloom/core/models.py Extends deterministic UUID input.
src/pitloom/embed.py Guards build-stage metadata resolution.
src/pitloom/extract/_lock_common.py Adds shared lock helpers.
src/pitloom/extract/_locked_dependencies.py Implements source-priority cascade.
src/pitloom/extract/_pdm_lock.py Parses PDM locks.
src/pitloom/extract/_pipfile_lock.py Parses Pipenv locks.
src/pitloom/extract/_poetry_lock.py Reuses shared validation helpers.
src/pitloom/extract/_pylock.py Parses PEP 751 locks.
src/pitloom/extract/_pyproject.py Adds lock-stage control.
src/pitloom/extract/_requirements_txt.py Parses fully pinned requirements.
src/pitloom/extract/_uv_lock.py Parses uv locks and workspaces.
src/pitloom/extract/project.py Applies the cascade uniformly.
tests/assemble/test_deps_locked_dependencies.py Tests provenance-sensitive UUIDs.
tests/assemble/test_model_generator_doc_identity.py Verifies generated document identity.
tests/extract/test_lock_common.py Tests shared lock helpers.
tests/extract/test_locked_dependencies.py Tests cascade precedence.
tests/extract/test_pdm_lock.py Tests PDM extraction.
tests/extract/test_pipfile_lock.py Tests Pipenv extraction.
tests/extract/test_poetry_lock.py Expands Poetry regressions.
tests/extract/test_project.py Tests fallback and stage behaviour.
tests/extract/test_pylock.py Tests PEP 751 extraction.
tests/extract/test_pyproject.py Updates fallback invocation.
tests/extract/test_requirements_txt.py Tests pinned requirements.
tests/extract/test_uv_lock.py Tests uv extraction.
tests/extract/test_uv_lock_integration.py Tests uv cascade integration.
tests/extract/test_uv_lock_root_package.py Tests workspace-root selection.
tests/fixtures/real-world-projects/README.md Distinguishes fixture categories.
tests/fixtures/real-world-locks/README.md Catalogues real lock fixtures.
tests/fixtures/real-world-locks/uv/flask-3.1.3/pyproject.toml Supplies Flask metadata.
tests/fixtures/real-world-locks/uv/flask-3.1.3/uv.lock Supplies a marker-heavy uv lock.
tests/fixtures/real-world-locks/uv/flask-3.1.3/LICENSE.txt Preserves fixture licensing.
tests/fixtures/real-world-locks/uv/fastapi-cli-0.0.32/pyproject.toml Supplies FastAPI CLI metadata.
tests/fixtures/real-world-locks/uv/fastapi-cli-0.0.32/uv.lock Supplies a real uv lock.
tests/fixtures/real-world-locks/uv/fastapi-cli-0.0.32/LICENSE Preserves fixture licensing.
tests/fixtures/real-world-locks/uv/abi3audit-0.0.26/pyproject.toml Supplies abi3audit metadata.
tests/fixtures/real-world-locks/uv/abi3audit-0.0.26/uv.lock Supplies a real uv lock.
tests/fixtures/real-world-locks/uv/abi3audit-0.0.26/LICENSE Preserves fixture licensing.
tests/fixtures/real-world-locks/pylock/snowflake-cli-3.26.0/pyproject.toml Supplies Snowflake CLI metadata.
tests/fixtures/real-world-locks/pylock/snowflake-cli-3.26.0/pylock.toml Supplies a real PEP 751 lock.
tests/fixtures/real-world-locks/pylock/snowflake-cli-3.26.0/LICENSE Preserves fixture licensing.
tests/fixtures/real-world-locks/pylock/pipenv-2026.8.0/pyproject.toml Supplies Pipenv metadata.
tests/fixtures/real-world-locks/pylock/pipenv-2026.8.0/pylock.toml Supplies Pipenv-generated pylock data.
tests/fixtures/real-world-locks/poetry/tomlkit-0.15.1/pyproject.toml Supplies Poetry metadata.
tests/fixtures/real-world-locks/poetry/tomlkit-0.15.1/poetry.lock Covers an empty runtime set.
tests/fixtures/real-world-locks/poetry/pendulum-3.2.0/pyproject.toml Supplies hybrid Poetry metadata.
tests/fixtures/real-world-locks/poetry/pendulum-3.2.0/poetry.lock Supplies a real Poetry lock.
tests/fixtures/real-world-locks/poetry/pastel-0.2.1/pyproject.toml Supplies legacy Poetry metadata.
tests/fixtures/real-world-locks/poetry/pastel-0.2.1/poetry.lock Supplies a real Poetry lock.
tests/fixtures/real-world-locks/poetry/cleo-2.1.0/pyproject.toml Supplies Poetry-only metadata.
tests/fixtures/real-world-locks/poetry/cleo-2.1.0/poetry.lock Supplies a real Poetry lock.
tests/fixtures/real-world-locks/pdm/unearth-0.18.3/pyproject.toml Supplies Unearth metadata.
tests/fixtures/real-world-locks/pdm/unearth-0.18.3/pdm.lock Supplies a real PDM lock.
tests/fixtures/real-world-locks/pdm/unearth-0.18.3/LICENSE Preserves fixture licensing.
tests/fixtures/real-world-locks/pdm/pdm-2.29.0/pyproject.toml Supplies PDM metadata.
tests/fixtures/real-world-locks/pdm/pdm-2.29.0/pdm.lock Covers extras-variant PDM entries.
tests/fixtures/real-world-locks/pdm/pdm-2.29.0/LICENSE Preserves fixture licensing.
tests/fixtures/real-world-locks/pipfile/responder-2.0.0/setup.py Supplies setuptools metadata.
tests/fixtures/real-world-locks/pipfile/responder-2.0.0/Pipfile.lock Covers editable Pipenv entries.
tests/fixtures/real-world-locks/pipfile/requests-html-0.10.0/setup.py Supplies setuptools metadata.
tests/fixtures/real-world-locks/pipfile/requests-html-0.10.0/Pipfile.lock Supplies a real Pipenv lock.
tests/fixtures/real-world-locks/requirements/home-assistant-core-2026.9.0/pyproject.toml Supplies Home Assistant metadata.
tests/fixtures/real-world-locks/requirements/home-assistant-core-2026.9.0/requirements.txt Covers partially pinned rejection.
tests/fixtures/real-world-locks/requirements/home-assistant-core-2026.9.0/LICENSE.md Preserves fixture licensing.
tests/fixtures/real-world-locks/requirements/home-assistant-core-2026.9.0/homeassistant/backports/LICENSE.Python Satisfies declared licence files.
Review details
  • Files reviewed: 65/78 changed files
  • Comments generated: 6
  • Review effort level: Balanced

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

Comment thread src/pitloom/extract/_lock_common.py Outdated
Comment thread src/pitloom/extract/_locked_dependencies.py
Comment thread src/pitloom/extract/_uv_lock.py Outdated
Comment thread src/pitloom/extract/_pylock.py Outdated
Comment thread tests/fixtures/real-world-locks/pylock/pipenv-2026.8.0/pylock.toml
bact added 3 commits September 5, 2026 10:16
Signed-off-by: Arthit Suriyawongkul <arthit@gmail.com>
Signed-off-by: Arthit Suriyawongkul <arthit@gmail.com>
Signed-off-by: Arthit Suriyawongkul <arthit@gmail.com>

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

Marker precedence, empty-lock UUID identity, and malformed-encoding handling contain correctness defects.

Once you've addressed the issues Copilot identified, you can request another Copilot review.

Review details

Suppressed comments (1)

src/pitloom/extract/_lock_common.py:103

  • Reading malformed non-UTF-8 JSON raises UnicodeDecodeError, which is neither OSError nor JSONDecodeError. A corrupt optional Pipfile.lock therefore aborts the operation rather than warning and allowing the cascade to continue.
    except (OSError, json.JSONDecodeError) as exc:
        log.warning("Failed to parse %s: %s", lock_path, exc)
  • Files reviewed: 65/78 changed files
  • Comments generated: 6
  • Review effort level: Balanced

Comment thread src/pitloom/core/models.py Outdated
Comment thread src/pitloom/extract/_pylock.py Outdated
Comment thread src/pitloom/extract/_lock_common.py Outdated
Comment thread docs/dependency-sources.md Outdated
Comment thread working-docs/design/roadmap.md Outdated
Comment thread working-docs/implementation/lock-file-cascade.md Outdated
bact added 2 commits September 5, 2026 13:00
Signed-off-by: Arthit Suriyawongkul <arthit@gmail.com>

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

Malformed Poetry locks can suppress valid fallback sources, and pinned requirements currently overstate SPDX relationship completeness.

Once you've addressed the issues Copilot identified, you can request another Copilot review.

Review details
  • Files reviewed: 65/78 changed files
  • Comments generated: 3
  • Review effort level: Balanced

Comment thread src/pitloom/extract/_poetry_lock.py
Comment thread src/pitloom/extract/_locked_dependencies.py
Comment thread docs/dependency-sources.md Outdated
bact added 2 commits September 5, 2026 18:22
Signed-off-by: Arthit Suriyawongkul <arthit@gmail.com>
Signed-off-by: Arthit Suriyawongkul <arthit@gmail.com>

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔵 Needs a closer look

It introduces a broad, cross-cutting dependency-resolution cascade plus determinism/provenance changes that should get a final human review despite strong test coverage.

Review details

Suppressed comments (1)

src/pitloom/extract/_locked_dependencies.py:112

  • Docstring says the cascade picks the first extractor that yields a non-empty result, but the implementation treats an empty list as a valid, authoritative lock result (only None means “not applicable”). Update the wording to avoid contradicting the None vs [] semantics described later in the docstring.
    Tries each extractor-bearing entry of :data:`_LOCK_SOURCES` in
    priority order; the first one that yields a non-empty result wins.
  • Files reviewed: 69/82 changed files
  • Comments generated: 1
  • Review effort level: Lite

Comment thread src/pitloom/core/models.py Outdated
Signed-off-by: Arthit Suriyawongkul <arthit@gmail.com>
bact added 5 commits September 6, 2026 00:19
Signed-off-by: Arthit Suriyawongkul <arthit@gmail.com>
Signed-off-by: Arthit Suriyawongkul <arthit@gmail.com>
Signed-off-by: Arthit Suriyawongkul <arthit@gmail.com>
Signed-off-by: Arthit Suriyawongkul <arthit@gmail.com>
Signed-off-by: Arthit Suriyawongkul <arthit@gmail.com>

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

Legacy Poetry locks leak dev dependencies, while version enrichment and completeness can produce inaccurate SBOM metadata.

Once you've addressed the issues Copilot identified, you can request another Copilot review.

Review details
  • Files reviewed: 82/95 changed files
  • Comments generated: 8
  • Review effort level: Balanced

Comment thread src/pitloom/assemble/spdx3/deps.py
Comment thread src/pitloom/assemble/spdx3/deps_installed.py Outdated
Comment thread src/pitloom/assemble/spdx3/deps_installed.py
Comment thread src/pitloom/assemble/spdx3/document.py Outdated
Comment thread src/pitloom/extract/_poetry_lock.py Outdated
Comment thread docs/dependency-sources.md Outdated
Comment thread working-docs/design/lock-files.md Outdated
Comment thread working-docs/design/lock-files.md Outdated
bact added 2 commits September 6, 2026 16:56
Signed-off-by: Arthit Suriyawongkul <arthit@gmail.com>
Signed-off-by: Arthit Suriyawongkul <arthit@gmail.com>

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

Malformed lock data can suppress valid fallbacks or crash uv extraction, and several new documentation claims contradict the implementation.

Once you've addressed the issues Copilot identified, you can request another Copilot review.

Review details

Suppressed comments (4)

Previously missed (4) — in code that hasn't changed since the last review.

src/pitloom/extract/_pylock.py:156

  • PEP 751 requires both a non-empty created-by string and a present packages array. Defaulting a missing packages key to [] (and never checking created-by) makes a truncated lock-version-only file an authoritative empty result, so it can suppress every lower-priority usable lock file. Validate both required fields before returning a result.
    src/pitloom/extract/_uv_lock.py:290
  • A malformed truthy scalar such as extra = 1 reaches the comprehension below and raises TypeError because an integer is not iterable. This aborts the entire lock cascade instead of warning and skipping the malformed extras reference like the extractor does for other bad shapes; validate that non-string values are lists before iterating.
    docs/dependency-sources.md:120
  • This claim is too broad. Lock-only transitive packages receive the named lock provenance, but a direct dependency whose range is resolved through locked_versions receives only Version resolved: Project lock file, without the source filename or method. Narrow this paragraph to transitive dependencies or thread the lock provenance into direct dependency annotations.
    working-docs/implementation/lock-file-cascade.md:51
  • This implementation note is already stale: _LOCK_SOURCES registers extract_poetry_lock_dependencies here, not (None, None). The later claims that Poetry is only handled by _try_read_poetry() and never by the cascade are likewise incorrect for PEP 621/non-Poetry projects. Update the sample and the associated explanation to match the implemented registry.
  • Files reviewed: 85/98 changed files
  • Comments generated: 3
  • Review effort level: Balanced

Comment thread src/pitloom/assemble/spdx3/document.py Outdated
Comment thread skills/sbom-generate/SKILL.md Outdated
Comment thread working-docs/design/lock-files.md Outdated
bact added 13 commits September 7, 2026 05:52
Signed-off-by: Arthit Suriyawongkul <arthit@gmail.com>
Signed-off-by: Arthit Suriyawongkul <arthit@gmail.com>
Signed-off-by: Arthit Suriyawongkul <arthit@gmail.com>
Signed-off-by: Arthit Suriyawongkul <arthit@gmail.com>
Signed-off-by: Arthit Suriyawongkul <arthit@gmail.com>
Signed-off-by: Arthit Suriyawongkul <arthit@gmail.com>
Signed-off-by: Arthit Suriyawongkul <arthit@gmail.com>
Signed-off-by: Arthit Suriyawongkul <arthit@gmail.com>
Signed-off-by: Arthit Suriyawongkul <arthit@gmail.com>
Signed-off-by: Arthit Suriyawongkul <arthit@gmail.com>
Signed-off-by: Arthit Suriyawongkul <arthit@gmail.com>
Signed-off-by: Arthit Suriyawongkul <arthit@gmail.com>
Signed-off-by: Arthit Suriyawongkul <arthit@gmail.com>
@bact
bact merged commit 6a22618 into main Sep 8, 2026
17 checks passed
@bact
bact deleted the lock-formats-pep751 branch September 8, 2026 23:28
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

enhancement New feature or request

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants