Skip to content

Report coverage for package source, with repo-relative LCOV paths - #9

Merged
davidanthoff merged 2 commits into
mainfrom
coverage-source-paths
Aug 20, 2026
Merged

Report coverage for package source, with repo-relative LCOV paths#9
davidanthoff merged 2 commits into
mainfrom
coverage-source-paths

Conversation

@davidanthoff

Copy link
Copy Markdown
Member

Part of julia-testitems/testitem-workflow#11 (Codecov reporting 0%). Companion to julia-testitems/TestItemControllers.jl#73; workflow-side fix is julia-testitems/testitem-workflow#13.

Warning

Blocked: cli.jl calls write_lcov(...; root=), which lands in TestItemControllers.jl#73. Merge that and release it before merging this, or main breaks against the registered TestItemControllers 1.7.0. [compat] TestItemControllers = "1.7" is deliberately left alone here — it needs bumping in the same commit that follows the TestItemControllers release.

Coverage covered the whole package, not its source

coverage_root_uris was built from package_uri, which is the folder holding Project.toml — not src. The test process filter keeps everything under a root, so test/, docs/ and any loose script landed in the report, where julia-processcoverage reports src and nothing else. Counting a package's own test files as covered source moves the reported percentage for no good reason.

The roots now name src and ext explicitly (ext is package code too). Plain string concatenation, not joinpath — these are URIs, and neither name needs escaping.

Only the app is narrowed. The identical computation in TestItemRuns keeps whole-package coverage: that is the library path behind VS Code and JuliaMCP, where "how well is my test file exercised" is a legitimate question.

SF: paths were absolute

write_lcov was called without a root, while the JUnit writer three lines above already passed root=abspath(opts.path). Codecov, Coveralls and genhtml match SF: paths against paths in the repository, and the absolute paths of a CI runner match nothing at all.

The action invokes juliati with test-path defaulting to . and cwd = GITHUB_WORKSPACE, so abspath(opts.path) is the repo root in the CI configuration. As with --junit-xml, pointing juliati at a subfolder makes paths relative to that subfolder.

The precompile workload passes root too, so the path that actually runs is the one that gets precompiled.

Tests

Full suite green, 31/31 on Windows against a locally dev'd TestItemControllers.jl#73.

  • run_tests collects coverage now asserts no /test/ file appears in result.coverage, and that write_lcov(...; root=fixture) emits SF:src/AppTestPkg.jl with no unix absolute path and no drive letter.
  • New --coverage-lcov writes repo-relative source paths, a CLI-level run through real_main asserting every SF: line starts with src/ and the file contains no backslash — the end-to-end shape a coverage service can match against a repository.

🤖 Generated with Claude Code

Two things made the LCOV `juliati --coverage-lcov` writes unusable to a
coverage service.

Coverage roots were the package root folders, so the test process kept every
instrumented file under the package — `test/`, `docs/`, loose scripts — where
`julia-processcoverage` reports `src` and nothing else. Counting a package's own
test files as covered source moves the reported percentage for no good reason.
The roots now name `src` and `ext` explicitly.

And the `SF:` paths were absolute, because `write_lcov` was called without a
root while the JUnit writer three lines above already passed one. Codecov,
Coveralls and `genhtml` match `SF:` paths against paths in the repository, and
the absolute paths of a CI runner match nothing at all — which is how a fully
covered package ends up reported as 0%.

Only the app is narrowed: TestItemRuns keeps whole-package coverage, since a
consumer asking how well a test file is exercised is asking something
legitimate.

Requires TestItemControllers with `write_lcov(...; root=)`.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
`cli.jl` calls `write_lcov(...; root=)`, which 1.8.0 is the first release to
have.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
@davidanthoff
davidanthoff marked this pull request as ready for review August 20, 2026 19:28
@davidanthoff

Copy link
Copy Markdown
Member Author

Unblocked: TestItemControllers 1.8.0 is registered and carries write_lcov(...; root=), so [compat] is bumped to "1.8" and this is out of draft.

@davidanthoff
davidanthoff merged commit 6daf427 into main Aug 20, 2026
16 of 18 checks passed
@davidanthoff
davidanthoff deleted the coverage-source-paths branch August 20, 2026 20:18
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant