Skip to content

OAPE-696: Add E2E coverage reporting with Codecov integration - #111

Open
PillaiManish wants to merge 3 commits into
openshift:mainfrom
PillaiManish:e2e-coverage-integration
Open

OAPE-696: Add E2E coverage reporting with Codecov integration#111
PillaiManish wants to merge 3 commits into
openshift:mainfrom
PillaiManish:e2e-coverage-integration

Conversation

@PillaiManish

@PillaiManish PillaiManish commented May 12, 2026

Copy link
Copy Markdown
Member

Summary

  • Adds a coverage-instrumented Dockerfile (Dockerfile.coverage) that builds the operator with Go's -cover flags and FIPS compliance
  • Adds hack/e2e-coverage.sh with setup and collect subcommands for CI and local use
  • Adds Makefile targets (build-coverage, docker-build-coverage, docker-push-coverage, e2e-coverage-collect)
  • No operator code changes -- zero impact on production builds

How it works

  1. CI builds a coverage-instrumented image from Dockerfile.coverage (uses make build-coverage which adds -cover -covermode=atomic -coverpkg=./...)
  2. hack/e2e-coverage.sh setup patches the live CSV to swap in the coverage image and set GOCOVERDIR=/tmp/e2e-cover
  3. E2E tests run against the instrumented operator
  4. hack/e2e-coverage.sh collect sends SIGTERM to flush coverage data (Go runtime writes on clean exit), waits for container restart, copies data via oc cp, converts to Go profile, and uploads to Codecov

Coverage data survives the container restart because the operator's CSV already defines an emptyDir volume at /tmp (medium: Memory), which persists across container restarts within the same pod. No PVC or extractor pod is needed.

Files

File Purpose
Dockerfile.coverage Mirrors Dockerfile.openshift but builds with coverage flags
hack/e2e-coverage.sh Coverage lifecycle script (setup + collect)
Makefile (appended) Local dev targets for coverage builds

Tested

  • Coverage binary compiles correctly with -cover -covermode=atomic -coverpkg=./...
  • Production build is unaffected (no operator code changes)
  • Full flow validated on a live OCP 4.20 cluster: setup -> SIGTERM flush -> container restart -> oc cp -> covdata convert -> coverage summary
  • Coverage successfully collected: pkg/operator 64.5%, assets 100%, pkg/version 100%

Follow-up (separate PR)

  • Release repo CI config: openshift/release#79231
  • Codecov Vault secret: configured at sscsi-ci-secrets/codecov-token

Summary by CodeRabbit

  • New Features
    • Added end-to-end test coverage support for the operator, including a coverage-instrumented image workflow.
    • Introduced CI/local automation to prepare coverage collection, flush coverage data after tests, and gather artifacts for reporting and upload.
  • Documentation
    • Expanded the README with step-by-step instructions for generating coverage, collecting results, viewing summaries locally, and optionally uploading to Codecov with integrity verification.

@coderabbitai

coderabbitai Bot commented May 12, 2026

Copy link
Copy Markdown

Note

Reviews paused

It looks like this branch is under active development. To avoid overwhelming you with review comments due to an influx of new commits, CodeRabbit has automatically paused this review. You can configure this behavior by changing the reviews.auto_review.auto_pause_after_reviewed_commits setting.

Use the following commands to manage reviews:

  • @coderabbitai resume to resume automatic reviews.
  • @coderabbitai review to trigger a single review.

Use the checkboxes below for quick actions:

  • ▶️ Resume reviews
  • 🔍 Trigger review

Walkthrough

Adds E2E coverage infrastructure for the operator, including a coverage-instrumented image, Makefile build and collection targets, deployment setup, coverage report generation, and optional Codecov uploads.

Changes

E2E Coverage Instrumentation and Collection

Layer / File(s) Summary
Coverage image build and Make targets
Dockerfile.coverage, Makefile
Builds the operator with Go coverage instrumentation, packages it with GOCOVERDIR=/tmp/e2e-cover, and adds image build, push, and collection targets.
Operator coverage setup
hack/e2e-coverage.sh
Discovers and patches the operator CSV with the coverage image and environment, then waits for the deployment rollout.
Coverage collection and documentation
hack/e2e-coverage.sh, README.md
Flushes pod coverage, copies artifacts, generates coverage-e2e.out, optionally verifies and uploads the report to Codecov, and documents CI and local usage.

Estimated code review effort: 3 (Moderate) | ~25 minutes

Sequence Diagram(s)

sequenceDiagram
    participant User as User/CI
    participant Kubernetes
    participant OperatorPod
    participant LocalArtifacts
    participant Codecov

    User->>Kubernetes: Patch CSV with coverage image and GOCOVERDIR
    Kubernetes->>OperatorPod: Roll out coverage-enabled operator
    User->>OperatorPod: Send SIGTERM to flush coverage
    User->>OperatorPod: Copy coverage data
    LocalArtifacts->>LocalArtifacts: Generate coverage-e2e.out with go tool covdata
    LocalArtifacts->>Codecov: Upload report when CODECOV_TOKEN is set
Loading

Suggested reviewers: jsafrane, dobsonj

🚥 Pre-merge checks | ✅ 14 | ❌ 1

❌ Failed checks (1 warning)

Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 0.00% which is insufficient. The required threshold is 80.00%. Write docstrings for the functions missing them to satisfy the coverage threshold.
✅ Passed checks (14 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title clearly matches the main change: adding E2E coverage reporting with Codecov integration.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
Stable And Deterministic Test Names ✅ Passed PR changes only Dockerfile.coverage, README.md, and hack/e2e-coverage.sh; no Ginkgo test titles were added or modified.
Test Structure And Quality ✅ Passed No Ginkgo test code was added or modified; the PR only changes Dockerfile, README, and a shell script, so the test-structure check is not applicable.
Microshift Test Compatibility ✅ Passed No new Ginkgo e2e tests were added; the PR only changes Dockerfile, Makefile, shell script, and README.
Single Node Openshift (Sno) Test Compatibility ✅ Passed PASS: The PR only adds Dockerfile/README/bash coverage tooling; no .go files or Ginkgo specs were changed, so there are no new tests to vet for SNO assumptions.
Topology-Aware Scheduling Compatibility ✅ Passed PR only changes Dockerfile/Makefile/README and a coverage script; no manifests/controllers add anti-affinity, spread constraints, nodeSelectors, or replica logic.
Ote Binary Stdout Contract ✅ Passed PR only adds Dockerfile/Makefile/shell/README files; no Go main/init/TestMain/RunSpecs setup code or stdout writes were added to OTE binaries.
Ipv6 And Disconnected Network Test Compatibility ✅ Passed PR only adds Dockerfile/Makefile/script/README changes; no new Ginkgo e2e tests were added, so IPv4/external-network test checks are not applicable.
No-Weak-Crypto ✅ Passed New files only use SHA-256/GPG verification; no MD5/SHA1/DES/RC4/3DES/Blowfish/ECB, custom crypto, or secret comparisons found.
Container-Privileges ✅ Passed No changed file adds privileged, hostPID/hostNetwork/hostIPC, SYS_ADMIN, or allowPrivilegeEscalation:true; the operator CSV already enforces runAsNonRoot=true and drop ALL.
No-Sensitive-Data-In-Logs ✅ Passed No sensitive values are logged: the script prints only status, image/branch names, and coverage paths; CODECOV_TOKEN is read but never echoed.
✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests

Comment @coderabbitai help to get the list of available commands.

@openshift-ci
openshift-ci Bot requested review from dobsonj and jsafrane May 12, 2026 12:12
@PillaiManish PillaiManish changed the title Add E2E coverage reporting with Codecov integration OAPE-696: Add E2E coverage reporting with Codecov integration May 12, 2026
@openshift-ci-robot openshift-ci-robot added the jira/valid-reference Indicates that this PR references a valid Jira ticket of any type. label May 12, 2026
@openshift-ci-robot

openshift-ci-robot commented May 12, 2026

Copy link
Copy Markdown

@PillaiManish: This pull request references OAPE-696 which is a valid jira issue.

Warning: The referenced jira issue has an invalid target version for the target branch this PR targets: expected the sub-task to target the "5.0.0" version, but no target version was set.

Details

In response to this:

Summary

  • Adds a coverage-instrumented Dockerfile (Dockerfile.coverage) that builds the operator with Go's -cover flags
  • Adds a SIGUSR1-based coverage flush handler (coverage_flush.go) gated behind a //go:build e2ecoverage tag, excluded from production builds
  • Adds hack/e2e-coverage.sh with setup and collect subcommands for CI and local use
  • Adds Makefile targets for local coverage image builds

How it works (no-PVC approach)

  1. CI builds a coverage-instrumented image from Dockerfile.coverage
  2. setup patches the live CSV to swap in the coverage image and set GOCOVERDIR
  3. E2E tests run against the instrumented operator
  4. collect sends SIGUSR1 to flush coverage data, copies it from the running pod via oc cp, converts to a Go profile, and uploads to Codecov

No PVC or extractor pod is needed -- coverage data is flushed on demand while the pod is still alive.

Tested

  • Coverage binary compiles correctly with -cover -covermode=atomic -coverpkg=./...
  • Production build is unaffected (coverage_flush.go excluded without e2ecoverage tag)
  • Full flow validated on a live OCP 4.20 cluster: setup -> SIGUSR1 flush -> oc cp -> covdata convert -> coverage summary

Follow-up (separate PR)

  • Codecov Vault secret setup (one-time manual step)
  • openshift/release CI config changes to wire up coverage steps in Prow jobs

Made with Cursor

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 3

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@cmd/secrets-store-csi-driver-operator/coverage_flush.go`:
- Around line 14-16: The GOCOVERDIR creation currently uses os.MkdirAll(dir,
0o777) and ignores errors; change this to create the directory with restrictive
permissions (e.g., 0o700 or 0o755 as appropriate) and handle the returned error
from os.MkdirAll instead of discarding it: call os.MkdirAll(dir, 0o700), check
the error, and propagate or log/fail fast (using the existing operator logger or
by returning the error) so coverage flush code won’t proceed when directory
creation fails; update the block that reads the GOCOVERDIR env var and the
os.MkdirAll call accordingly.

In `@hack/e2e-coverage.sh`:
- Around line 93-95: The current send of SIGUSR1 uses a hardcoded PID 1 (the
line invoking oc exec ... 'kill -USR1 1'), which can be wrong if the operator
isn't PID 1; update the script to locate the actual operator process inside the
container (using pidof or pgrep) and send SIGUSR1 to that PID, falling back to
PID 1 if no process is found. Inside the block that uses "${NAMESPACE}" and
"${pod}", run something like pidof <operator-binary-name> || pgrep -f
<operator-binary-name> to capture the PID(s) and then call kill -USR1 "$PID" (or
kill -USR1 1 if the lookup returns empty) so the script remains compatible with
single-binary containers while handling wrapper/init cases. Ensure the lookup
happens inside the oc exec command so the PID refers to the container namespace.

In `@Makefile`:
- Around line 80-85: The build-coverage target is generating invalid Go flag
syntax by naively concatenating "$(GO_BUILD_FLAGS),e2ecoverage" and the
suggested "-tags e2ecoverage" breaks FIPS by producing duplicate -tags; fix by
producing a single -tags value for the coverage build: detect if GO_BUILD_FLAGS
already contains a -tags clause and if so append ",e2ecoverage" to that existing
tag list, otherwise add a new "-tags e2ecoverage" flag; implement this logic in
the Makefile when constructing flags for the build-coverage target (use
GO_BUILD_FLAGS and create/compute a temporary coverage tags variable or a
modified FLAGS variable) so the final go build invocation has at most one -tags
flag and includes e2ecoverage.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Repository: openshift/coderabbit/.coderabbit.yaml

Review profile: CHILL

Plan: Enterprise

Run ID: f4068362-35f1-4771-9630-6d924234b1da

📥 Commits

Reviewing files that changed from the base of the PR and between c422160 and e6855e5.

📒 Files selected for processing (4)
  • Dockerfile.coverage
  • Makefile
  • cmd/secrets-store-csi-driver-operator/coverage_flush.go
  • hack/e2e-coverage.sh

Comment thread cmd/secrets-store-csi-driver-operator/coverage_flush.go Outdated
Comment thread hack/e2e-coverage.sh Outdated
Comment thread Makefile

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🧹 Nitpick comments (1)
Dockerfile.coverage (1)

6-9: ⚡ Quick win

Drop root default in the coverage runtime image.

The runtime stage has no explicit USER, so it defaults to root. Please set a non-root user for defense-in-depth.

Proposed hardening
 FROM registry.ci.openshift.org/ocp/4.22:base-rhel9
 COPY --from=builder /go/src/github.com/openshift/secrets-store-csi-driver-operator/secrets-store-csi-driver-operator /usr/bin/
 ENV GOCOVERDIR=/tmp/e2e-cover
+USER 65532
 ENTRYPOINT ["/bin/sh", "-c", "mkdir -p /tmp/e2e-cover && exec /usr/bin/secrets-store-csi-driver-operator \"$@\"", "--"]
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@Dockerfile.coverage` around lines 6 - 9, The runtime stage in
Dockerfile.coverage defaults to root; add a non-root user and switch to it
before ENTRYPOINT to improve hardening: create a dedicated user/group (or use an
existing non-root UID), ensure ownership/permissions of /tmp/e2e-cover and the
binary at /usr/bin/secrets-store-csi-driver-operator are adjusted (chown/chmod)
so the non-root user can write to GOCOVERDIR and execute the binary, then add a
USER instruction before ENTRYPOINT to run the container as that non-root user.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Nitpick comments:
In `@Dockerfile.coverage`:
- Around line 6-9: The runtime stage in Dockerfile.coverage defaults to root;
add a non-root user and switch to it before ENTRYPOINT to improve hardening:
create a dedicated user/group (or use an existing non-root UID), ensure
ownership/permissions of /tmp/e2e-cover and the binary at
/usr/bin/secrets-store-csi-driver-operator are adjusted (chown/chmod) so the
non-root user can write to GOCOVERDIR and execute the binary, then add a USER
instruction before ENTRYPOINT to run the container as that non-root user.

ℹ️ Review info
⚙️ Run configuration

Configuration used: Repository: openshift/coderabbit/.coderabbit.yaml

Review profile: CHILL

Plan: Enterprise

Run ID: 67bd0df7-10b1-406d-8f94-fae8b631578c

📥 Commits

Reviewing files that changed from the base of the PR and between e6855e5 and 7cb80c3.

📒 Files selected for processing (3)
  • Dockerfile.coverage
  • Makefile
  • hack/e2e-coverage.sh
🚧 Files skipped from review as they are similar to previous changes (1)
  • hack/e2e-coverage.sh

@PillaiManish

Copy link
Copy Markdown
Member Author

/retest

@mytreya-rh

Copy link
Copy Markdown
Contributor

/label tide/merge-method-squash

@openshift-ci openshift-ci Bot added the tide/merge-method-squash Denotes a PR that should be squashed by tide when it merges. label May 20, 2026
@PillaiManish

Copy link
Copy Markdown
Member Author

/retest

2 similar comments
@chiragkyal

Copy link
Copy Markdown
Member

/retest

@PillaiManish

Copy link
Copy Markdown
Member Author

/retest

@chiragkyal chiragkyal left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, with a few questions.

/cc @mytreya-rh

Comment thread hack/e2e-coverage.sh
fi
echo "Found CSV: ${csv}"

echo "Patching CSV with coverage image..."

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Which image is it exactly replacing?

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It replaces the standard secrets-store-csi-driver-operator image (built from Dockerfile.openshift via make → normal go build) with the coverage-instrumented image (built from Dockerfile.coverage via make build-coveragego build -cover -covermode=atomic -coverpkg=./...).

Both produce the same binary at /usr/bin/secrets-store-csi-driver-operator, but the coverage version is compiled with Go's coverage instrumentation. The JSON patch targets /spec/install/spec/deployments[0]/spec/template/spec/containers[0]/image — the first container of the first deployment in the CSV (which is the operator container).

The coverage image additionally sets GOCOVERDIR=/tmp/e2e-cover and wraps the entrypoint with mkdir -p /tmp/e2e-cover so coverage data is written to disk during execution.

Comment thread hack/e2e-coverage.sh
-o jsonpath='{.spec.install.spec.deployments[0].spec.template.spec.containers[0].env[?(@.name=="GOCOVERDIR")].name}' 2>/dev/null)
if [[ -z "${has_gocoverdir}" ]]; then
echo "Adding GOCOVERDIR env var to CSV..."
oc patch csv "${csv}" -n "${NAMESPACE}" --type=json -p "[

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I believe CSV patch won't be overwritten by OLM.

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Correct — this is safe. OLM does not reconcile/overwrite the CSV's spec.install.spec.deployments once the CSV is in the Succeeded phase. OLM creates the deployment from the CSV spec, and the CSV is essentially static after that. The JSON patch triggers OLM to update the deployment with the new image and env var, which is exactly what we want.

The only risk would be if OLM reinstalled/upgraded the operator during the test, but in a CI e2e run that doesn't happen.

Comment thread hack/e2e-coverage.sh
Comment on lines +141 to +147
local job_type="${JOB_TYPE:-local}"
if [[ "${job_type}" == "presubmit" ]]; then
echo "Detected presubmit (PR #${PULL_NUMBER:-unknown})"
[[ -n "${PULL_NUMBER:-}" ]] && codecov_args+=(--pr "${PULL_NUMBER}")
[[ -n "${PULL_PULL_SHA:-}" ]] && codecov_args+=(--sha "${PULL_PULL_SHA}")
[[ -n "${PULL_BASE_REF:-}" ]] && codecov_args+=(--branch "${PULL_BASE_REF}")
[[ -n "${REPO_OWNER:-}" && -n "${REPO_NAME:-}" ]] && codecov_args+=(--slug "${REPO_OWNER}/${REPO_NAME}")

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

How are we defining these env vars?

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

These are not defined by us — they are standard Prow job environment variables automatically injected into every CI job container by Prow:

Variable Description
JOB_TYPE presubmit, postsubmit, or periodic
PULL_NUMBER PR number (presubmit only)
PULL_PULL_SHA HEAD commit SHA of the PR
PULL_BASE_REF Base branch name (e.g., main)
PULL_BASE_SHA Base branch commit SHA
REPO_OWNER GitHub org (e.g., openshift)
REPO_NAME GitHub repo name

The script uses them to provide correct PR/commit context to the Codecov uploader so coverage reports show up on the right PR. When running locally (JOB_TYPE is unset, defaults to local), these are all skipped and Codecov auto-detects from git.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Did we check in any rehearsal whether they are getting populated correctly?

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes — these Prow env vars are standard and well-established. They're populated for every CI job automatically. You can verify in any Prow job log by looking at the environment. For example, in the rehearsal run from the release repo PR, the Prow container env includes JOB_TYPE=presubmit, PULL_NUMBER, REPO_OWNER=openshift, etc.

That said, the Codecov upload only happens if CODECOV_TOKEN is set (from the mounted secret). In the rehearsal, the upload was skipped since the secret wasn't available in the rehearsal namespace — which is expected and harmless. The env vars themselves are always present; they're part of Prow's core infrastructure, documented at https://docs.prow.k8s.io/docs/jobs/#job-environment-variables.

Comment thread Makefile

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Are these Make commands meant to be used locally?

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, the Makefile targets support both CI and local use:

  • make build-coverage — builds the coverage-instrumented binary. Used by Dockerfile.coverage in CI, but can also be run locally.
  • make docker-build-coverage — builds the coverage Docker image locally (convenience for dev testing).
  • make docker-push-coverage — pushes the coverage image to a registry (local workflow only).
  • make e2e-coverage-collect — runs hack/e2e-coverage.sh collect locally, for collecting coverage after you've run e2e tests against a dev cluster with the coverage image deployed.

In CI, only build-coverage is used directly (inside Dockerfile.coverage). The other targets are convenience wrappers for developers who want to test the coverage flow against a dev cluster without going through the full CI pipeline.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Except make build-coverage, others are not used in CI. If they are for local testing, I think some documentation on how to run and verify coverage would help.

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good point. I'll add a section to the README documenting how to run and verify coverage locally. Something like:

## E2E Coverage (Local)

1. Build and push the coverage image:
   make docker-build-coverage docker-push-coverage

2. Deploy the operator with the coverage image on your dev cluster:
   hack/e2e-coverage.sh setup

3. Run e2e tests:
   make test-e2e

4. Collect coverage:
   make e2e-coverage-collect

Will add this in the next push.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

the README update is still pending

@openshift-ci
openshift-ci Bot requested a review from mytreya-rh June 22, 2026 12:20
@PillaiManish

Copy link
Copy Markdown
Member Author

/retest

@PillaiManish

Copy link
Copy Markdown
Member Author

/retest-required

@PillaiManish

Copy link
Copy Markdown
Member Author

/retest

@PillaiManish

Copy link
Copy Markdown
Member Author

/test operator-e2e-fips
/test operator-e2e-aws
/test operator-e2e-azure
/test operator-e2e-gcp

@chiragkyal

Copy link
Copy Markdown
Member

/retest-required

@mytreya-rh mytreya-rh left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM
few minor comments

Comment thread hack/e2e-coverage.sh Outdated
echo "Found operator pod: ${pod}"

echo "Sending SIGTERM to flush coverage data (container will restart)..."
oc exec -n "${NAMESPACE}" "${pod}" -- /bin/sh -c 'kill -TERM 1' 2>/dev/null || true

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If we cannot successfully send the SIGTERM, the job should fail correct?
But here it continues..

Comment thread hack/e2e-coverage.sh Outdated
oc exec -n "${NAMESPACE}" "${pod}" -- /bin/sh -c 'kill -TERM 1' 2>/dev/null || true

echo "Waiting for container to restart and become ready..."
sleep 5

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Here, we are assuming that the container restarted in 5 seconds.
More reliable approach would be to check that the restartCount got incremented by one from the value that was before sending the SIGTERM

Comment thread Dockerfile.coverage Outdated
@@ -0,0 +1,11 @@
FROM registry.ci.openshift.org/ocp/builder:rhel-9-golang-1.25-openshift-4.22 AS builder

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

we should update these to the 5.0 and golang 1.26 base images

Comment thread Makefile

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

the README update is still pending

@PillaiManish
PillaiManish force-pushed the e2e-coverage-integration branch from 4fe4186 to 6c922db Compare July 15, 2026 10:11
PillaiManish and others added 2 commits July 15, 2026 15:45
Add coverage-instrumented build and collection scripts for E2E test
coverage reporting. Uses a no-PVC approach: SIGUSR1 flushes coverage
data from the running operator pod, which is then copied out via oc cp
and uploaded to Codecov.

Co-authored-by: Cursor <cursoragent@cursor.com>
Remove coverage_flush.go and use SIGTERM to flush coverage data
instead of SIGUSR1. Container restarts after SIGTERM, emptyDir
preserves coverage files, and oc cp runs from the restarted container.

Co-authored-by: Cursor <cursoragent@cursor.com>
@PillaiManish
PillaiManish force-pushed the e2e-coverage-integration branch 2 times, most recently from 27c2dbd to a02ffdd Compare July 15, 2026 10:18

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 5

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@Dockerfile.coverage`:
- Line 3: Replace the broad COPY instruction in Dockerfile.coverage with
explicit COPY statements for only the files and directories required by make
build-coverage, and add or tighten a restrictive .dockerignore to exclude
credentials, build outputs, and unrelated artifacts.
- Around line 6-11: Add a meaningful Docker HEALTHCHECK to the runtime stage
after the operator ENTRYPOINT, using a probe command available in the base RHEL9
image that verifies the operator process is running and tolerates its startup
behavior. Keep the existing ENTRYPOINT and labels unchanged.
- Around line 1-6: Update the final runtime stage after the builder stage to use
an approved UBI minimal or distroless image from catalog.redhat.com instead of
registry.ci.openshift.org/ocp/5.0:base-rhel9. Keep the existing builder stage
and its separation from the runtime stage unchanged.
- Around line 6-9: Add a supported non-root user in the Dockerfile before the
ENTRYPOINT, switch to that user with USER, and ensure /tmp/e2e-cover is created
with ownership or permissions allowing it to be written at runtime. Keep the
existing GOCOVERDIR and secrets-store-csi-driver-operator startup behavior
unchanged.

In `@hack/e2e-coverage.sh`:
- Around line 80-87: Update the pod lookup in the coverage collection flow so a
non-zero oc get result for no matching pods does not trigger set -e before the
empty-pod check. Explicitly tolerate or handle the lookup failure, then preserve
the existing [[ -z "${pod}" ]] custom error path and exit behavior.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Repository: openshift/coderabbit/.coderabbit.yaml

Review profile: CHILL

Plan: Enterprise

Run ID: 8f392ebc-ac62-4e8a-8fab-ea04f6e0cec2

📥 Commits

Reviewing files that changed from the base of the PR and between 6c922db and a02ffdd.

📒 Files selected for processing (4)
  • Dockerfile.coverage
  • Makefile
  • README.md
  • hack/e2e-coverage.sh

Comment thread Dockerfile.coverage
Comment on lines +1 to +6
FROM registry.ci.openshift.org/ocp/builder:rhel-9-golang-1.26-openshift-5.0 AS builder
WORKDIR /go/src/github.com/openshift/secrets-store-csi-driver-operator
COPY . .
RUN make build-coverage

FROM registry.ci.openshift.org/ocp/5.0:base-rhel9

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔒 Security & Privacy | 🟠 Major | 🏗️ Heavy lift

Use the approved runtime base image.

The final stage uses registry.ci.openshift.org/ocp/5.0:base-rhel9, but the container policy requires a UBI minimal or distroless runtime image from catalog.redhat.com. Replace it with an approved catalog image while keeping the builder stage separate.

🧰 Tools
🪛 Trivy (0.69.3)

[error] 1-1: Image user should not be 'root'

Specify at least 1 USER command in Dockerfile with non-root user as argument

Rule: DS-0002

Learn more

(IaC/Dockerfile)

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@Dockerfile.coverage` around lines 1 - 6, Update the final runtime stage after
the builder stage to use an approved UBI minimal or distroless image from
catalog.redhat.com instead of registry.ci.openshift.org/ocp/5.0:base-rhel9. Keep
the existing builder stage and its separation from the runtime stage unchanged.

Source: Path instructions

Comment thread Dockerfile.coverage
@@ -0,0 +1,11 @@
FROM registry.ci.openshift.org/ocp/builder:rhel-9-golang-1.26-openshift-5.0 AS builder
WORKDIR /go/src/github.com/openshift/secrets-store-csi-driver-operator
COPY . .

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔒 Security & Privacy | 🟠 Major | 🏗️ Heavy lift

Do not copy the entire build context.

COPY . . places every context file into the builder layer/cache, potentially including credentials or unrelated artifacts. Use explicit COPY statements for the files required by make build-coverage, together with a restrictive .dockerignore.

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@Dockerfile.coverage` at line 3, Replace the broad COPY instruction in
Dockerfile.coverage with explicit COPY statements for only the files and
directories required by make build-coverage, and add or tighten a restrictive
.dockerignore to exclude credentials, build outputs, and unrelated artifacts.

Source: Path instructions

Comment thread Dockerfile.coverage
Comment on lines +6 to +9
FROM registry.ci.openshift.org/ocp/5.0:base-rhel9
COPY --from=builder /go/src/github.com/openshift/secrets-store-csi-driver-operator/secrets-store-csi-driver-operator /usr/bin/
ENV GOCOVERDIR=/tmp/e2e-cover
ENTRYPOINT ["/bin/sh", "-c", "mkdir -p /tmp/e2e-cover && exec /usr/bin/secrets-store-csi-driver-operator \"$@\"", "--"]

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔒 Security & Privacy | 🟠 Major | ⚡ Quick win

Run the runtime image as non-root.

No USER is declared, and Trivy reports the image’s effective user as root. Add a non-root user supported by the base image and verify that /tmp/e2e-cover remains writable.

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@Dockerfile.coverage` around lines 6 - 9, Add a supported non-root user in the
Dockerfile before the ENTRYPOINT, switch to that user with USER, and ensure
/tmp/e2e-cover is created with ownership or permissions allowing it to be
written at runtime. Keep the existing GOCOVERDIR and
secrets-store-csi-driver-operator startup behavior unchanged.

Sources: Path instructions, Linters/SAST tools

Comment thread Dockerfile.coverage
Comment on lines +6 to +11
FROM registry.ci.openshift.org/ocp/5.0:base-rhel9
COPY --from=builder /go/src/github.com/openshift/secrets-store-csi-driver-operator/secrets-store-csi-driver-operator /usr/bin/
ENV GOCOVERDIR=/tmp/e2e-cover
ENTRYPOINT ["/bin/sh", "-c", "mkdir -p /tmp/e2e-cover && exec /usr/bin/secrets-store-csi-driver-operator \"$@\"", "--"]
LABEL io.k8s.display-name="OpenShift Secrets Store CSI Driver Operator" \
io.k8s.description="The Secrets Store CSI Driver Operator installs and maintains the Secrets Store CSI Driver on a cluster."

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🩺 Stability & Availability | 🟠 Major | ⚡ Quick win

Add a meaningful HEALTHCHECK.

The runtime image has no healthcheck, despite the container policy requiring one. Use a probe available in the runtime image and compatible with the operator’s startup behavior.

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@Dockerfile.coverage` around lines 6 - 11, Add a meaningful Docker HEALTHCHECK
to the runtime stage after the operator ENTRYPOINT, using a probe command
available in the base RHEL9 image that verifies the operator process is running
and tolerates its startup behavior. Keep the existing ENTRYPOINT and labels
unchanged.

Source: Path instructions

Comment thread hack/e2e-coverage.sh
Comment on lines +80 to +87
local pod
pod=$(oc get pods -n "${NAMESPACE}" -l "${POD_LABEL}" \
-o jsonpath='{.items[0].metadata.name}' 2>/dev/null)
if [[ -z "${pod}" ]]; then
echo "Error: no operator pod found with label ${POD_LABEL}"
echo "Coverage collection requires the operator pod to be running."
exit 1
fi

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🎯 Functional Correctness | 🟡 Minor | ⚡ Quick win

Pod lookup should not short-circuit the custom error path.
oc get ... -o jsonpath='{.items[0].metadata.name}' exits non-zero when no pod matches, so set -e stops the script before if [[ -z "${pod}" ]] can print the intended message. Handle the empty case explicitly instead of relying on that branch.

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@hack/e2e-coverage.sh` around lines 80 - 87, Update the pod lookup in the
coverage collection flow so a non-zero oc get result for no matching pods does
not trigger set -e before the empty-pod check. Explicitly tolerate or handle the
lookup failure, then preserve the existing [[ -z "${pod}" ]] custom error path
and exit behavior.

@PillaiManish

Copy link
Copy Markdown
Member Author

/retest

@PillaiManish

Copy link
Copy Markdown
Member Author

/retest

…update README and base images

- Remove || true from kill -TERM so the job fails if SIGTERM cannot
  be sent
- Use oc wait --for=jsonpath restartCount for reliable restart
  detection
- Add E2E Coverage section to README with usage instructions
- Update Dockerfile.coverage to golang-1.26-openshift-5.0 and
  ocp/5.0:base-rhel9 matching production Dockerfile

Co-authored-by: Cursor <cursoragent@cursor.com>
@PillaiManish
PillaiManish force-pushed the e2e-coverage-integration branch from a02ffdd to 1cf2651 Compare July 17, 2026 08:54
@mytreya-rh

Copy link
Copy Markdown
Contributor

/lgtm
/approve

@openshift-ci openshift-ci Bot added the lgtm Indicates that a PR is ready to be merged. label Jul 17, 2026
@openshift-ci

openshift-ci Bot commented Jul 17, 2026

Copy link
Copy Markdown
Contributor

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: mytreya-rh, PillaiManish

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Details Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@openshift-ci openshift-ci Bot added the approved Indicates a PR has been approved by an approver from all required OWNERS files. label Jul 17, 2026
@chiragkyal

Copy link
Copy Markdown
Member

/retest

2 similar comments
@chiragkyal

Copy link
Copy Markdown
Member

/retest

@PillaiManish

Copy link
Copy Markdown
Member Author

/retest

@openshift-ci

openshift-ci Bot commented Jul 28, 2026

Copy link
Copy Markdown
Contributor

@PillaiManish: The following tests failed, say /retest to rerun all failed tests or /retest-required to rerun all mandatory failed tests:

Test name Commit Details Required Rerun command
ci/prow/operator-e2e-aws 1cf2651 link true /test operator-e2e-aws
ci/prow/operator-e2e-gcp 1cf2651 link true /test operator-e2e-gcp
ci/prow/operator-e2e-vault-fips 1cf2651 link true /test operator-e2e-vault-fips

Full PR test history. Your PR dashboard.

Details

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. I understand the commands that are listed here.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

approved Indicates a PR has been approved by an approver from all required OWNERS files. jira/valid-reference Indicates that this PR references a valid Jira ticket of any type. lgtm Indicates that a PR is ready to be merged. tide/merge-method-squash Denotes a PR that should be squashed by tide when it merges.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants