From dbf7cf27587fdf11b8b5711ed7e035130bafe6e5 Mon Sep 17 00:00:00 2001 From: topcoder1 Date: Wed, 20 May 2026 12:04:13 -0700 Subject: [PATCH] feat(coverage-floor): add `extras:` input for selective extras install MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Adds a new optional input `extras` to the coverage-floor reusable workflow. Lets Python callers control which `[project.optional-dependencies]` extras get installed before pytest-cov runs. Values: - "all" (default) — `uv sync --all-extras`, preserves prior behavior - "none" — `uv sync` with no extras - comma-separated list — `uv sync --extra --extra ...` Motivation — burn 2026-05-19 (wxa_webcat): Coverage Floor failed silently for 8 consecutive PRs (#379-#387) because `uv sync --all-extras` triggered uv to clone webcat's `crawl` extra: webcrawl @ git+https://github.com/topcoder1/webcrawl.git AUTOMERGE_PAT (the only forwardable cross-org credential) doesn't have read scope on topcoder1/webcrawl. The `Configure git for cross-org clones` step ran successfully (rewrote URL with token) but the actual fetch returned 404, killing the `uv sync` step in 0 seconds. Workarounds that DIDN'T work or had unwanted tradeoffs: - Forwarding AUTOMERGE_PAT (already forwarded; PAT scope is the issue) - Broadening AUTOMERGE_PAT scope (security tradeoff — fleet-wide PAT growing in scope is exactly the kind of credential sprawl we want to avoid) - Repo-side pyproject restructure (changes how webcrawl is consumed in real environments, not just CI) This input lets the caller pass `extras: "dev,api,model,data,headless"` to skip the problematic `crawl` extra without changing any fleet-wide credentials. The pattern generalizes to any future repo with cross-org git deps. Backward compatibility: - Default `"all"` reproduces the current `--all-extras` behavior verbatim. - All 44 existing callers in the fleet continue to work without modification. - Self-test on this repo is unaffected (no extras anyway; default path). Follow-ups (separate PRs): - Update `callers/coverage-floor.yml` template in topcoder1/dotclaude to document the input with a commented example. - whois-api-llc/wxa_webcat caller-side PR to pass `extras: "dev,api,model,data,headless"` and unblock the coverage-floor gate there. - Consider adding graceful degradation: catch `uv sync` cross-org clone failures and retry with the extra dropped + warning, so future repos can self-recover without needing the caller to know which extra is the offender. Deferred — explicit caller control is the safer default. Co-Authored-By: Claude Opus 4.7 (1M context) --- .github/workflows/coverage-floor.yml | 64 ++++++++++++++++++++++++---- 1 file changed, 56 insertions(+), 8 deletions(-) diff --git a/.github/workflows/coverage-floor.yml b/.github/workflows/coverage-floor.yml index 8c707e4..3dec852 100644 --- a/.github/workflows/coverage-floor.yml +++ b/.github/workflows/coverage-floor.yml @@ -112,6 +112,28 @@ on: required: false type: string default: "" + extras: + description: | + (Python only) Which `[project.optional-dependencies]` extras to install before + running pytest-cov. Accepts: + - "all" (default) — `uv sync --all-extras`, preserves prior behavior + - "none" — `uv sync` with no extras + - comma-separated list like "dev,api,model" — `uv sync --extra dev --extra api --extra model` + + Use a targeted list when an extra contains cross-org git deps that uv can't clone + in CI (e.g. webcat's `crawl` extra pins `webcrawl @ git+https://github.com/topcoder1/webcrawl.git`, + which needs a PAT with read access to that repo; passing `extras: "dev,api,model,data,headless"` + skips the crawl extra entirely without changing AUTOMERGE_PAT's scope). + + Burn 2026-05-19 (wxa_webcat): coverage-floor failed silently for 6 days because + `--all-extras` triggered uv to clone `topcoder1/webcrawl` and AUTOMERGE_PAT lacked + read scope on that repo. Adding this input lets callers skip the problematic + extra without the security tradeoff of broadening the fleet PAT's scope. + + No-op for go/js callers (only the python branch honors this input). + required: false + type: string + default: "all" secrets: AUTOMERGE_PAT: description: "PAT used to push the seed branch and open the seed PR so that pull_request workflows actually fire (GITHUB_TOKEN doesn't trigger them — recursion prevention). Already deployed fleet-wide for `claude-author-automerge.yml`. Caller must explicitly forward this secret (see callers/coverage-floor.yml in topcoder1/dotclaude for the syntax). Required for the post-merge seed flow to land without manual unblock." @@ -306,19 +328,45 @@ jobs: working-directory: ${{ inputs.working_directory || '.' }} env: LANG_TYPE: ${{ steps.detect.outputs.language }} + EXTRAS: ${{ inputs.extras }} run: | set -euo pipefail case "$LANG_TYPE" in python) if [[ -f pyproject.toml ]]; then - # --all-extras installs every [project.optional-dependencies] - # extra (e.g. `dev`, `test`) in addition to [dependency-groups]. - # Without it, repos that put test deps only in optional- - # dependencies hit ModuleNotFoundError at pytest collection - # (burned 2026-05-12 wave-3 — domain-rank, ipgeo_core, - # crawl-infra all needed per-repo pyproject mirrors). Coverage - # measurement wants maximum dep coverage anyway. - uv sync --quiet --all-extras + # extras controls which [project.optional-dependencies] extras get + # installed before pytest-cov runs. Default ("all") preserves the + # original --all-extras behavior — repos that put test deps only + # in optional-dependencies need this so pytest collection doesn't + # hit ModuleNotFoundError (burned 2026-05-12 wave-3 — domain-rank, + # ipgeo_core, crawl-infra all needed per-repo pyproject mirrors). + # + # "none" skips extras entirely. A comma-separated list installs + # only the named extras. Use the list form when an extra contains + # cross-org git deps that uv can't clone in CI (burn 2026-05-19, + # wxa_webcat — webcat's `crawl` extra pulls webcrawl from topcoder1 + # which needs broader PAT scope than AUTOMERGE_PAT provides). + case "${EXTRAS:-all}" in + all) + uv sync --quiet --all-extras + ;; + none|"") + uv sync --quiet + ;; + *) + # Comma-separated list. Build --extra flags. + EXTRA_FLAGS=() + IFS=',' read -ra EXTRA_LIST <<< "$EXTRAS" + for e in "${EXTRA_LIST[@]}"; do + e_trimmed="$(echo "$e" | xargs)" # strip whitespace + if [[ -n "$e_trimmed" ]]; then + EXTRA_FLAGS+=("--extra" "$e_trimmed") + fi + done + echo "==> uv sync --quiet ${EXTRA_FLAGS[*]}" + uv sync --quiet "${EXTRA_FLAGS[@]}" + ;; + esac # Some repos have pyproject.toml as config-only (no [project] # deps section) and keep actual deps in requirements.txt # (inbox_superpilot/backend pattern). Install those on top so