Skip to content

ci: parallelize coverage across two BVT groups - #26179

Merged
XuPeng-SH merged 25 commits into
mainfrom
xp/ci-race-coverage-artifacts
Jul 27, 2026
Merged

XuPeng-SH merged 25 commits into
mainfrom
xp/ci-race-coverage-artifacts

Conversation

@XuPeng-SH

@XuPeng-SH XuPeng-SH commented Jul 25, 2026 •

Copy link
Copy Markdown
Contributor

What this PR does / why we need it

Reworks PR coverage collection to remove the serial UT + full-BVT critical path while retaining Race UT and coverage breadth.

  • adds a separate non-race UT coverage producer; Race UT remains unchanged and does not block coverage merge
  • uses exactly two active BVT jobs: Compose + Proxy and Launch + Pessimistic
  • splits every non-optimistic BVT case into two complementary, timing-balanced groups; each run executes both groups once and a rerun swaps their deployment mapping
  • collects profiles in the producer jobs and merges exactly one UT plus two BVT profiles
  • skips merge when any producer fails, coverage secrets are unavailable, the PR is untrusted, or the PR is a draft
  • routes Compose SQL through Proxy and gracefully stops coverage-instrumented services
  • compacts the all-package UT profile before upload, reducing the observed artifact from about 622 MB to 3,519,245 bytes without changing merged block/hit results
  • fixes production object-storage retry cancellation and bounds QCloud multipart abort cleanup to 30 seconds
  • propagates test deadlines through object-storage constructors and operations so a remote outage cannot consume the package-level 10-minute timeout
  • adds the production entrypoint calls for Coverage UT and Coverage Merge; the same entrypoint disables legacy coverage atomically

Rollout dependency

Depends on matrixorigin/CI#404. Merge CI#404 first. Its enable_legacy_coverage input defaults to true, so the old entrypoint retains legacy coverage until this PR merges; this PR passes false in the same commit that enables the new producers.

.github/workflows/coverage-integration.yaml is validation-only and pins reviewed CI commit 6beb8d8bb8a93d62f6ff02591c3eafe287d536ce. Remove it after the final integration run passes and before merging this PR.

Which issue(s) this PR fixes:

Related performance issue: #25116

Validation

  • pkg/fileservice: full short package test passed
  • pkg/fileservice: full short -race test passed; final changed-helper race selection passed
  • go vet ./pkg/fileservice: passed
  • make err-check: passed
  • actionlint on MatrixOne entrypoint/integration workflows: passed
  • actionlint on changed CI coverage workflows: passed (known custom runner labels excluded)
  • BVT grouping: 1,113 non-optimistic scripts; group union/disjoint validation passed
  • Docker Compose launch-multi-cn config validation: passed
  • CI workflow schema validation in ci: split coverage into parallel producers CI#404: passed

@XuPeng-SH
XuPeng-SH requested a review from fengttt as a code owner July 25, 2026 23:39
@qodo-code-review

Copy link
Copy Markdown

Qodo reviews are paused for this user.

Troubleshooting steps vary by plan Learn more →

On a Teams plan?
Reviews resume once this user has a paid seat and their Git account is linked in Qodo.
Link Git account →

Using GitHub Enterprise Server, GitLab Self-Managed, or Bitbucket Data Center?
These require an Enterprise plan - Contact us
Contact us →

Comment thread .github/workflows/coverage-integration.yaml Fixed
Comment thread .github/workflows/coverage-integration.yaml Fixed
Comment thread .github/workflows/coverage-integration.yaml Fixed
Comment thread .github/workflows/coverage-integration.yaml Fixed
@XuPeng-SH
XuPeng-SH force-pushed the xp/ci-race-coverage-artifacts branch from bb6cf54 to 156d012 Compare July 26, 2026 11:54
@XuPeng-SH XuPeng-SH changed the title ci: merge race UT and BVT coverage collection ci: parallelize coverage across two BVT groups Jul 26, 2026
Comment thread .github/workflows/coverage-integration.yaml Fixed
Comment thread .github/workflows/coverage-integration.yaml Fixed
Comment thread .github/workflows/coverage-integration.yaml Fixed
Comment thread .github/workflows/coverage-integration.yaml Fixed
Comment thread .github/workflows/coverage-integration.yaml Fixed
Comment thread .github/workflows/coverage-integration.yaml Fixed
Comment thread .github/workflows/coverage-integration.yaml Fixed

This branch was previously deployed

1 inactive deployment
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

size/L Denotes a PR that changes [500,999] lines

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants