Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions Makefile
Original file line number Diff line number Diff line change
Expand Up @@ -18,6 +18,7 @@ check-fmt: ## Verify template test formatting

lint: ## Run template test lint checks
$(UV) ruff check tests/
$(UV) --with interrogate interrogate --fail-under 100 tests/

typecheck: ## Run template test type checks
$(UV) --with pytest --with pytest-copier --with pyyaml --with syrupy --with make-parser ty check tests/
Expand Down
8 changes: 6 additions & 2 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -14,6 +14,8 @@ Run `copier copy` and answer the prompts to generate a project.
The test suite relies on the `pytest-copier` plugin and renders generated
projects that run Ruff, Pylint via a PyPy-backed runner, `ty`, pytest, and, when
the Rust extension is enabled, Clippy, Whitaker, and nextest-aware Rust tests.
Generated Python linting also runs Interrogate as a 100% docstring-coverage
gate.

Run the parent template tests through the repository `Makefile`. Run
`make help` to list the available parent Makefile targets. Parent gates include
Expand All @@ -30,8 +32,9 @@ when `act` and Docker are available. Parent repository CI runs this mode in a
separate act-validation workflow so rendered in-template GitHub workflows do not
hold up the normal template test workflow.

Generated projects install and run their own tooling, including Ruff, Pylint via
PyPy, `ty`, pytest, and, when Rust is enabled, Clippy, Whitaker, and nextest.
Generated projects install and run their own tooling, including Ruff,
Interrogate, Pylint via PyPy, `ty`, pytest, and, when Rust is enabled, Clippy,
Whitaker, and nextest.

You can also run `scripts/setup_test_deps.sh` to install parent test
dependencies into the current Python environment manually.
Expand All @@ -54,6 +57,7 @@ flowchart LR
All --> Test[test]

Lint --> RuffCheck[ruff check]
Lint --> Interrogate[interrogate --fail-under 100]
Lint --> PylintPyPy[pylint-pypy via PyPy]
Typecheck --> TyCheck[ty check]
Audit --> PipAudit[pip-audit]
Expand Down
2 changes: 1 addition & 1 deletion docs/adr-001-template-quality-gates.md
Original file line number Diff line number Diff line change
Expand Up @@ -16,7 +16,7 @@ failures are predictable.
Generated projects use `make all` as the public aggregate gate. The `lint`
target delegates to language-specific targets:

- `lint-python` runs Ruff and Pylint through the PyPy-backed runner.
- `lint-python` runs Ruff, Interrogate (`--fail-under 100`), and Pylint through the PyPy-backed runner.
- `lint-rust` is rendered only when `use_rust` is enabled and runs rustdoc,
Clippy, and Whitaker.

Expand Down
7 changes: 5 additions & 2 deletions docs/developers-guide.md
Original file line number Diff line number Diff line change
Expand Up @@ -12,7 +12,8 @@ template itself.
- `make help` — lists all `##`-annotated targets.
- `make check-fmt` — runs Ruff formatting checks against the parent template
test suite.
- `make lint` — runs Ruff lint checks against the parent template test suite.
- `make lint` — runs Ruff lint checks and `interrogate --fail-under 100` against
the `tests/` directory in the parent template test suite.
- `make typecheck` — runs `ty check` against the parent template test suite,
supplying the test dependencies through `uvx`.
- `make test` — runs the template test suite via `uvx`, supplying
Expand Down Expand Up @@ -50,7 +51,9 @@ runs `cargo audit` in the Rust extension crate.

The generated lint targets are split by language:

- `lint-python` runs Ruff and Pylint via the pinned PyPy-backed runner.
- `lint-python` runs Ruff, `interrogate --fail-under 100` for 100% docstring
coverage across `$(PYTHON_TARGETS)`, and Pylint via the pinned PyPy-backed
runner.
- `lint-rust` exists only when `use_rust` is enabled and runs rustdoc, Clippy,
and Whitaker.
- `lint` delegates to the applicable language-specific targets.
Expand Down
5 changes: 3 additions & 2 deletions template/AGENTS.md.jinja
Original file line number Diff line number Diff line change
Expand Up @@ -81,8 +81,9 @@
- `make check-fmt` runs Ruff formatting checks with
`ruff format --check $(PYTHON_TARGETS)`.
- `make lint` runs `make lint-python`; `make lint-python` runs
`ruff check $(PYTHON_TARGETS)` and the PyPy-backed Pylint runner against
`$(PYLINT_TARGETS)`.
`ruff check $(PYTHON_TARGETS)`, enforces 100% docstring coverage with
`interrogate --fail-under 100 $(PYTHON_TARGETS)`, and runs the
PyPy-backed Pylint runner against `$(PYLINT_TARGETS)`.
- `make typecheck` runs `ty check $(PYTHON_TARGETS)`.
- `make test` runs `pytest -v -n $(PYTEST_XDIST_WORKERS)` and honours
`WITH_ACT=1` through `RUN_ACT_VALIDATION=1`.
Expand Down
1 change: 1 addition & 0 deletions template/Makefile.jinja
Original file line number Diff line number Diff line change
Expand Up @@ -136,6 +136,7 @@ lint: lint-python{% if use_rust %} lint-rust{% endif %} ## Run linters

lint-python: build ## Run Python linters
$(UV_ENV) $(UV) run ruff check $(PYTHON_TARGETS)
$(UV_ENV) $(UV) run interrogate --fail-under 100 $(PYTHON_TARGETS)
$(PYLINT) $(PYLINT_TARGETS)
{% if use_rust %}

Expand Down
13 changes: 9 additions & 4 deletions template/docs/developers-guide.md
Original file line number Diff line number Diff line change
Expand Up @@ -9,6 +9,9 @@ The public entrypoint for formatting, linting, typechecking, and tests is
failure, and changes should be reconciled with the aggregate gate before being
considered complete.

`make lint` runs Ruff, `interrogate --fail-under 100 $(PYTHON_TARGETS)` for
100% docstring coverage across `$(PYTHON_TARGETS)`, and Pylint.

Run `make audit` as the dependency vulnerability gate. It runs `pip-audit` for
Python dependencies, and Rust-enabled projects also run `cargo audit` from the
`rust_extension` crate directory.
Expand All @@ -32,10 +35,12 @@ actions under `.github/`.

- `.github/workflows/ci.yml` runs on pushes to `main` and on pull requests. It
sets up Python 3.13, installs `uv`, validates the generated `Makefile` with
`mbake`, runs `make build`, `make check-fmt`, `make lint`, `make typecheck`,
and `make audit`, then delegates coverage generation to the shared coverage
action. When the Rust extension is enabled, it also sets up Rust, installs
Rust lint and test tools, and passes `rust_extension/Cargo.toml` to coverage.
`mbake`, runs `make build`, `make check-fmt`,
`make lint` (Ruff + `interrogate --fail-under 100 $(PYTHON_TARGETS)` + Pylint), `make
typecheck`, and `make audit`, then delegates coverage generation to the shared
coverage action. When the Rust extension is enabled, it also sets up Rust,
installs Rust lint and test tools, and passes `rust_extension/Cargo.toml` to
coverage.
- `.github/workflows/act-validation.yml` runs rendered workflow validation in a
separate workflow. It installs `act`, checks Docker availability, and runs
`make test WITH_ACT=1` outside the coverage path.
Expand Down
6 changes: 4 additions & 2 deletions template/docs/users-guide.md.jinja
Original file line number Diff line number Diff line change
Expand Up @@ -14,8 +14,10 @@ these targets in order:
- `test`: run pytest and, when Rust is enabled, Rust tests.
- `audit`: run `pip-audit` and, when Rust is enabled, `cargo audit`.

The `lint-python` target runs Ruff followed by Pylint via a PyPy-backed runner.
The Pylint runner is installed through `uv tool run` from the pinned
The `lint-python` target runs Ruff, then Interrogate with
`interrogate --fail-under 100 $(PYTHON_TARGETS)` to enforce 100% docstring
coverage for the Python targets, then Pylint via a PyPy-backed runner. The Pylint
runner is installed through `uv tool run` from the pinned
`pylint-pypy-shim` repository.

Pytest discovery is limited to the top-level `tests/` tree. Keep generated
Expand Down
1 change: 1 addition & 0 deletions template/pyproject.toml.jinja
Original file line number Diff line number Diff line change
Expand Up @@ -10,6 +10,7 @@ dependencies = []
[dependency-groups]
dev = [
"pytest",
"interrogate",
"pip-audit",
"ruff",
"pyright",
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,7 @@ name = "_{{ package_name }}_rs"
crate-type = ["cdylib", "rlib"]

[dependencies]
pyo3 = { version = "0.28.3", features = ["extension-module"] }
pyo3 = { version = "0.29.0", features = ["extension-module"] }

[lints.clippy]
pedantic = { level = "warn", priority = -1 }
Expand Down
2 changes: 2 additions & 0 deletions tests/helpers/agents_contracts.py
Original file line number Diff line number Diff line change
Expand Up @@ -98,7 +98,9 @@ def _assert_documented_command_flags(
],
"lint-python": [
("AGENTS.md", "ruff check $(PYTHON_TARGETS)"),
("AGENTS.md", "interrogate --fail-under 100 $(PYTHON_TARGETS)"),
("Makefile", f"ruff check {python_targets}"),
("Makefile", f"interrogate --fail-under 100 {python_targets}"),
],
"typecheck": [
("AGENTS.md", "ty check $(PYTHON_TARGETS)"),
Expand Down
4 changes: 4 additions & 0 deletions tests/helpers/makefile_contracts.py
Original file line number Diff line number Diff line change
Expand Up @@ -56,6 +56,7 @@ def _parse_makefile_rules(makefile: str) -> dict[str, list[str]]:
}

def normalise_target(match: re.Match[str]) -> str:
"""Replace hyphenated target names with parser-compatible aliases."""
target = match.group(1)
return normalised_targets.get(target, target) + ":"

Expand Down Expand Up @@ -104,6 +105,9 @@ def _assert_makefile_contracts(*, makefile: str, use_rust: bool) -> None:
assert "$(UV_ENV) $(UV) run pip-audit" in makefile, (
"expected generated audit target to run pip-audit"
)
assert "$(UV_ENV) $(UV) run interrogate --fail-under 100" in makefile, (
"expected generated lint target to enforce docstring coverage"
)
if use_rust:
assert "TEST_CMD :=" in makefile, (
"expected Rust variant to select nextest or cargo test"
Expand Down
11 changes: 10 additions & 1 deletion tests/helpers/pyproject_contracts.py
Original file line number Diff line number Diff line change
Expand Up @@ -32,7 +32,16 @@ def _assert_pyproject_contracts(
assert isinstance(dev_dependencies, list), (
"expected generated pyproject.toml to include a dev dependency group"
)
for dependency in ["pytest", "pip-audit", "ruff", "pyright", "ty", "pytest-xdist"]:
for dependency in [
"pytest",
"interrogate",
"pip-audit",
"ruff",
"pyright",
"ty",
"pytest-timeout",
"pytest-xdist",
]:
Comment thread
coderabbitai[bot] marked this conversation as resolved.
assert dependency in dev_dependencies, (
f"expected generated dev dependencies to include {dependency}"
)
Expand Down
1 change: 1 addition & 0 deletions tests/test_github_actions_integration.py
Original file line number Diff line number Diff line change
Expand Up @@ -173,6 +173,7 @@ def test_run_act_forwards_only_explicit_act_github_token(
captured_command: list[str] = []

def fake_run(command: list[str], **_: Any) -> subprocess.CompletedProcess[str]:
"""Capture the act subprocess command without executing it."""
captured_command.extend(command)
return subprocess.CompletedProcess(command, 0, "stdout", "stderr")

Expand Down
3 changes: 3 additions & 0 deletions tests/test_helpers.py
Original file line number Diff line number Diff line change
Expand Up @@ -456,6 +456,9 @@ def test_parent_makefile_test_target_uses_requisite_pytest_command() -> None:
assert "$(UV) ruff check tests/" in makefile, (
"expected parent Makefile lint target to run Ruff checks"
)
assert "$(UV) --with interrogate interrogate --fail-under 100 tests/" in makefile, (
"expected parent Makefile lint target to enforce docstring coverage"
)
assert "typecheck: ## Run template test type checks" in makefile, (
"expected parent Makefile to expose a documented typecheck target"
)
Expand Down
97 changes: 97 additions & 0 deletions tests/test_interrogate_lint.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,97 @@
"""Validate rendered docstring coverage lint behaviour.

These integration tests render a project from the Copier template and exercise
its public ``make lint`` target instead of inspecting only generated text. They
prove that a clean render passes the full Python lint tier, then mutate the
rendered package to introduce a Ruff-clean missing private docstring and verify
that Interrogate fails the target.

The module protects the template contract that generated projects enforce 100%
docstring coverage through their normal lint workflow.
"""

from __future__ import annotations

import re
import shutil
import subprocess
from pathlib import Path

from pytest_copier.plugin import CopierFixture

from tests.helpers.rendering import render_project


def test_generated_make_lint_enforces_interrogate_docstrings(
copier: CopierFixture, tmp_path: Path
) -> None:
"""Validate rendered ``make lint`` fails on missing private docstrings.

Parameters
----------
copier : CopierFixture
Fixture used to render the template into a temporary project.
tmp_path : Path
Temporary directory used as the generated project root.

Returns
-------
None
The test passes when a complete generated project passes ``make lint``
and a private helper without a docstring fails at the Interrogate gate.
"""
package_name = "lint_docstrings_pkg"
project = render_project(
tmp_path / "lint-docstrings",
copier,
project_name="LintDocstrings",
package_name=package_name,
use_rust=False,
)
make = shutil.which("make")
assert make is not None, "expected make to be available for generated tests"

passing_result = subprocess.run(
[make, "lint"],
cwd=project.path,
check=False,
capture_output=True,
text=True,
)
assert passing_result.returncode == 0, (
f"expected generated make lint to pass with complete docstrings:\n"
f"{passing_result.stdout}\n{passing_result.stderr}"
)

pure_module = project / package_name / "pure.py"
undocumented_helper = (
'\n\ndef _missing_interrogate_docstring() -> str:\n return "undocumented"\n'
)
pure_module.write_text(
pure_module.read_text(encoding="utf-8") + undocumented_helper,
encoding="utf-8",
)

failing_result = subprocess.run(
[make, "lint"],
cwd=project.path,
check=False,
capture_output=True,
text=True,
)
output = f"{failing_result.stdout}\n{failing_result.stderr}"
assert failing_result.returncode != 0, (
"expected generated make lint to fail when Interrogate finds a missing "
f"private docstring:\n{output}"
)
assert "RESULT: FAILED" in output, (
"expected generated make lint to run the Interrogate docstring gate"
)
actual_match = re.search(r"actual:\s+([0-9]+(?:\.[0-9]+)?)%", output)
assert actual_match is not None, (
f"expected Interrogate output to report actual docstring coverage:\n{output}"
)
actual_coverage = float(actual_match.group(1))
assert actual_coverage < 100.0, (
"expected Interrogate output to report reduced docstring coverage"
)
2 changes: 2 additions & 0 deletions tests/utilities.py
Original file line number Diff line number Diff line change
Expand Up @@ -39,6 +39,7 @@
def _resolved_socket_from_docker_host(
docker_host: str, allowed_dirs: tuple[Path, ...]
) -> Path | None:
"""Return a trusted Unix socket path parsed from ``DOCKER_HOST``."""
parsed = urlparse(docker_host)
if parsed.scheme != "unix" or parsed.netloc or not parsed.path:
return None
Expand All @@ -54,6 +55,7 @@ def _resolved_socket_from_docker_host(


def _user_podman_socket() -> Path | None:
"""Return the current user's Podman socket when it exists under runtime dir."""
socket_path = Path(f"/run/user/{os.getuid()}/podman/podman.sock")
if not socket_path.exists():
return None
Expand Down
Loading