ci: measure coverage and report to Coveralls - #11
Conversation
mina-release-toolkit's codecov.yml lists deb-toolkit/ under `ignore`, on the stated basis that this repo reports coverage from its own CI. It did not: there was no coverage job here, so the crate reported nothing anywhere. Adds a cargo-llvm-cov job uploading to Codecov under a deb-toolkit flag, mirroring the parent repo's workflow so the two report comparably. Two things the job has to match from ci.yml rather than defaulting: it pins ubuntu-22.04, because the tests/res GPG fixture uses a digest gpg 2.4 rejects; and it installs debsigs and debsig-verify, without which the integration tests skip themselves by design and everything they cover (signer, signature_verifier, viewer, and the packaging half of builder) would be reported uncovered rather than measured. Thresholds start soft — project auto with 1% slack, patch 70%, both advisory. The crate currently measures ~77% region / ~80% line, with a wide per-module spread, so there is not yet a basis for a hard gate.
Codecov rejects tokenless uploads outright. With fail_ci_if_error false, a missing CODECOV_TOKEN leaves the job green while publishing nothing — which is the state the sibling crates' coverage jobs in mina-release-toolkit have been in since May: green runs, badges, and no data behind them. Emit a warning annotation when the secret is absent so the condition is visible on the PR.
|
Heads-up on a blocker this surfaced — the coverage job is green, but the upload is being rejected: Codecov no longer accepts tokenless uploads. This is not specific to this PR. The same thing is happening in To make this actually publish: add I pushed 639b346 so the condition is at least visible rather than silent: the job now emits a warning annotation when the secret is missing. I kept |
Codecov requires an upload token even for public repos — tokenless uploads are accepted only for PRs from forks, and any other branch, including a plain feature branch, is rejected with "Token required - not valid tokenless upload". Both ways of obtaining that token are gated on GitHub organization ownership, and the people who need coverage here are org members with repo admin, not org owners. Waiting on an owner to provision a secret is what left mina-release-toolkit's three coverage jobs green and empty since May. Coveralls authenticates with the GITHUB_TOKEN that Actions injects automatically — its docs are explicit that it must not be added to the secrets store — and creates the repo on first upload. No secret, no approval, nothing to rotate. Drops codecov.yml; Coveralls keeps thresholds in its own UI rather than in-repo. Swaps the README badge.
|
Switched this from Codecov to Coveralls in 49aa531. Why: Codecov requires an upload token even for public repos — tokenless uploads are accepted only for PRs from forks, so an ordinary branch is rejected. Both the repo-level and global tokens are reachable only by a GitHub org owner, and the people who actually need coverage here (me included) are org Waiting on an owner to provision a secret is exactly what left Coveralls authenticates with the Changes: Trade-offs, stated honestly:
If an org owner would rather provision |
Coverage Report for CI Build 29901343755Warning No base build found for commit Coverage: 81.818%Details
Uncovered ChangesNo uncovered changes found. Coverage RegressionsRequires a base build to compare against. How to fix this → Coverage Stats
💛 - Coveralls |
Why
mina-release-toolkit'scodecov.ymllistsdeb-toolkit/underignore, with the comment that the submodules "report coverage from their own repos". This repo did not — there was no coverage job here, so the crate reported coverage nowhere. The parent has per-crate Codecov flags and badges for the three in-tree crates;deb-toolkitwas the gap.What
A
cargo-llvm-covjob uploading to Codecov under adeb-toolkitflag, plus acodecov.ymlmirroring the parent repo's conventions so the two report comparably.Two things the coverage job has to copy from
ci.ymlrather than taking defaults:ubuntu-22.04, notubuntu-latest— same reasonci.ymlpins it: the bundled GPG signing-key fixture undertests/res/uses a digest algorithm gpg 2.4 rejects. Coverage has to run the same tests CI runs, so it needs the same runner.debsigsanddebsig-verify— without them the integration tests skip themselves by design (see README) rather than fail. Everything they exercise —signer.rs,signature_verifier.rs,viewer.rs, and the packaging half ofbuilder.rs— would then be reported as uncovered. That understates coverage instead of measuring it.Baseline
Measured locally on
mainwith the full integration toolchain present:The spread across modules is wide:
session/apply.rssession/control.rssession/compression.rsbuilder.rssession/open.rssignature_verifier.rssigner.rscontent_verifier.rsmisc.rsBecause of that spread, thresholds start soft and advisory: project
autowith 1% slack, patch target 70%, bothif_ci_failed: ignore, andfail_ci_if_error: falseon the upload so a Codecov outage cannot turn a PR red. Worth tightening once there is trend data showing each module's natural baseline.Verification
cargo llvm-cov --all-features --workspace --lcov --output-path lcov.info— the exact command the workflow runs — was run locally and produces a validlcov.info. Both YAML files parse.lcov.infois added to.gitignoreso the artifact can't be committed by accident.Note
Overlaps #10 only in
README.md, and in a different hunk (badges at the top vs. the "Known limitation" section near the bottom), so the two merge cleanly in either order.