Skip to content

Stop measuring vendored SDK code as this plugin's coverage - #1007

Merged
dkotter merged 3 commits into
developfrom
chore/exclude-vendor-from-coverage
Sep 9, 2026
Merged

Stop measuring vendored SDK code as this plugin's coverage#1007
dkotter merged 3 commits into
developfrom
chore/exclude-vendor-from-coverage

Conversation

@whyisjake

@whyisjake whyisjake commented Sep 5, 2026

Copy link
Copy Markdown
Member

Coverage measures includes/Vendor/, which is third-party SDK code copied verbatim from upstream php-ai-client. Those files carry paths this plugin never calls, so the score reflects upstream's test suite rather than ours.

phpcs.xml.dist and phpstan.neon.dist already exclude that directory. This adds the same exclusion for coverage: an <exclude> in phpunit.xml.dist so the files never enter the clover report, and a new codecov.yml so they are absent from Codecov's view of it.

The percentage can move in either direction, since this removes covered lines as well as uncovered ones.

Testing

npm run test:php passes at 1571 tests, which confirms PHPUnit still parses the config; a malformed <coverage> block fails the run outright. Codecov's validator accepts codecov.yml and compiles the matcher to (?s:includes/Vendor/.*)\Z.

Not verified locally: that the clover report itself omits the files. Neither wp-env container has Xdebug or PCOV, so no coverage run is possible here. CI will show it on this PR.


AI assistance: Yes. Tool(s): Claude Code (Opus 5). Used for: identifying the gap during a code review of #1004, both edits, and this description.

@whyisjake — this attestation is deliberately left for you. The WordPress AI Guidelines require the contributor to understand every line submitted and explain it under review; that is not something an agent can satisfy on your behalf. Replace this block before merging.

🤖 Generated with Claude Code

https://claude.ai/code/session_01BRSJEyQyYp3XTeum8fiq3L

Open WordPress Playground Preview

`includes/Vendor/AiClient/` holds third-party SDK classes copied verbatim from
upstream php-ai-client so the plugin can use capabilities before every
environment bundles them. Coverage currently measures them as if they were
written here, and because upstream code carries paths this plugin never calls,
they score far below the plugin's own code and drag the reported number away
from the thing it is supposed to describe.

The repository already treats that directory as not-our-code: `phpcs.xml.dist`
excludes `includes/Vendor/` and `phpstan.neon.dist` excludes
`includes/Vendor/AiClient/src/`. Coverage was the remaining gate still counting
it. The vendored README already documented this rationale alongside the other
two exemptions, describing an exclusion that did not yet exist.

`phpunit.xml.dist` keeps those files out of the clover report, and `codecov.yml`
keeps them out of the report Codecov renders from it. Both are needed: the
first fixes the data, the second fixes anything reading it.

This lowers the absolute number of covered lines as well as uncovered ones, so
the headline percentage may move in either direction. The point is that it
starts describing this plugin rather than upstream's test suite.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BRSJEyQyYp3XTeum8fiq3L
@whyisjake
whyisjake requested a review from a team September 5, 2026 06:21
@github-actions

github-actions Bot commented Sep 5, 2026

Copy link
Copy Markdown

The following accounts have interacted with this PR and/or linked issues. I will continue to update these lists as activity occurs. You can also manually ask me to refresh this list by adding the props-bot label.

If you're merging code through a pull request on GitHub, copy and paste the following into the bottom of the merge commit message.

Co-authored-by: whyisjake <whyisjake@git.wordpress.org>
Co-authored-by: justlevine <justlevine@git.wordpress.org>
Co-authored-by: dkotter <dkotter@git.wordpress.org>

To understand the WordPress project's expectations around crediting contributors, please review the Contributor Attribution page in the Core Handbook.

@github-actions

github-actions Bot commented Sep 5, 2026

Copy link
Copy Markdown

✅ WordPress Plugin Check Report

✅ Status: Passed

📊 Report

All checks passed! No errors or warnings found.


🤖 Generated by WordPress Plugin Check Action • Learn more about Plugin Check

@justlevine justlevine left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks so much for this PR @whyisjake 🙇

Please be mindful of doc-slop, both in in the PR description (that project maintainers need to read and evaluate) and in the diffs themselves (where it also becomes infectious to future AI-led contributions).

PS: Also big thanks for adding the AI Disclosure to the bottom of the PR description, it was immensely helpful to allow me not to worry about human "intent" and made it quick and easy to share feedback 🙌

Comment thread codecov.yml Outdated
Comment thread phpunit.xml.dist Outdated
Comment thread .github/.codecov.yml
The rationale for excluding vendored code from coverage was written out
three times: once in codecov.yml, once in phpunit.xml.dist, and once in
includes/Vendor/AiClient/README.md, each phrased differently. Three
copies of one reason drift apart, and eleven lines of comment over two
lines of config is the wrong ratio.

Both configs now point at the README, which is where the reasoning
already lives.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@whyisjake

Copy link
Copy Markdown
Member Author

Fair hit, thank you. The description ran 588 words for a 23-line change, and the diff carried eleven lines of comment over two lines of config.

Both are trimmed. The config comments now point at includes/Vendor/AiClient/README.md rather than restating the reasoning, which also removes a third copy of it that was free to drift from the other two.

The infectious point is the one that landed for me. A verbose comment in a config file is what the next contributor copies.

Kept the AI disclosure, since it was useful.

@whyisjake
whyisjake requested a review from justlevine September 9, 2026 16:57
Codecov reads the file from the repository root, dev/, or .github/, under
either codecov.yml or .codecov.yml. Putting it in .github/ places it with
dependabot.yml and dependency-review-config.yml, the other configs that
only GitHub reads.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@whyisjake
whyisjake requested a review from jeffpaul as a code owner September 9, 2026 16:59
@codecov

codecov Bot commented Sep 9, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 81.50%. Comparing base (a7657e1) to head (7e28ba3).
⚠️ Report is 1 commits behind head on develop.

Additional details and impacted files
@@              Coverage Diff              @@
##             develop    #1007      +/-   ##
=============================================
+ Coverage      75.55%   81.50%   +5.95%     
+ Complexity      3381     2923     -458     
=============================================
  Files            138      122      -16     
  Lines          13081    11653    -1428     
=============================================
- Hits            9883     9498     -385     
+ Misses          3198     2155    -1043     
Flag Coverage Δ
unit 81.50% <ø> (+5.95%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@justlevine justlevine left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🙌
(Proven working by the codecov report )

@dkotter dkotter added this to the 1.4.0 milestone Sep 9, 2026
@dkotter
dkotter merged commit 4d53643 into develop Sep 9, 2026
28 of 34 checks passed
@dkotter
dkotter deleted the chore/exclude-vendor-from-coverage branch September 9, 2026 20:59
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants