Skip to content

feat(deploy-report-pages)!: replace GitHub Pages with GCS for Playwright reports #253

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
mckn wants to merge 1 commit into from
Closed

feat(deploy-report-pages)!: replace GitHub Pages with GCS for Playwright reports #253

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
62 changes: 30 additions & 32 deletions playwright-gh-pages/README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,15 +1,15 @@
# Publishing Playwright reports to Github Pages
# Publishing Playwright reports to Google Cloud Storage

When testing a Grafana plugin using the [`@grafana/plugin-e2e`](https://www.npmjs.com/package/@grafana/plugin-e2e?activeTab=readme) package, it is highly recommended to run tests against a matrix of Grafana versions (as demonstrated in this [example](https://grafana.com/developers/plugin-tools/e2e-test-a-plugin/ci) in the documentation). Each test run in this matrix generates an HTML report. By uploading these reports to a static site hosting service, they become immediately accessible for direct browsing, eliminating the need to download and serve them locally. This enhances productivity and fosters collaborative troubleshooting by making the results easily shareable and reviewable.
When testing a Grafana plugin using the [`@grafana/plugin-e2e`](https://www.npmjs.com/package/@grafana/plugin-e2e?activeTab=readme) package, it is highly recommended to run tests against a matrix of Grafana versions (as demonstrated in this [example](https://grafana.com/developers/plugin-tools/e2e-test-a-plugin/ci) in the documentation). Each test run in this matrix generates an HTML report. By uploading these reports to Google Cloud Storage, they become immediately accessible for direct browsing by Grafana staff, eliminating the need to download and serve them locally.

This set of GitHub Actions streamlines the process of managing Playwright test reports. It automates uploading reports as artifacts, publishing them to GitHub Pages, and providing links in pull request comments. These actions work seamlessly together, enhancing collaboration, traceability, and test result management.
This set of GitHub Actions streamlines the process of managing Playwright test reports. It automates uploading reports as artifacts, publishing them to GCS, and providing links in pull request comments.

## Overview

The workflow consists of two main actions:

1. **Upload Report Artifacts Action**: This action uploads test reports and summaries as GitHub artifacts. It can be used together with the `deploy-report-pages` action to publish the reports to GitHub Pages.
2. **Deploy to GitHub Pages Action**: This action publishes the test artifacts to GitHub Pages and comments on the pull request with the results and corresponding links. It also cleans up and deletes old reports based on the specified retention policy.
1. **Upload Report Artifacts Action**: Uploads test reports and summaries as GitHub artifacts. Used together with the `deploy-report-pages` action to publish the reports to GCS.
2. **Deploy Playwright Reports to GCS Action**: Uploads the test artifacts to Google Cloud Storage and comments on the pull request with the results and corresponding links.

## Features

Expand All @@ -19,38 +19,38 @@ The workflow consists of two main actions:
- Supports conditional uploading based on test outcomes.
- Structures reports in a well-organized directory format, ensuring uniqueness for each test setup.

- **Deploy to GitHub Pages Action**:
- Downloads test artifacts and publishes them to GitHub Pages.
- **Deploy Playwright Reports to GCS Action**:
- Downloads test artifacts and uploads them to Google Cloud Storage.
- Comments on the pull request with the test results and links to the reports.
- Supports retention of reports for a specified number of days.
- Reports are retained for **90 days** via the bucket's object lifecycle policy.

## GitHub Pages Branch Configuration
## GCS Bucket Configuration

The `deploy-report-pages` Action publishes reports to the `gh-pages` branch by default. However, if GitHub Pages isn't already configured to serve content from this branch, you'll need to take an extra step to ensure your reports are accessible. Follow these steps:
Reports are uploaded to a Grafana-managed GCS bucket. Each report is stored at the path:

1. Navigate to the **Settings** tab of your repository.
2. In the left-hand sidebar, click on **Pages**.
3. Under **Source**, select **Deploy from a branch**, then choose `gh-pages`.
```
gs://{bucket}/{owner}/{repo}/{YYYYMMDD}/{pr-number-or-run-id}/{matrix-dir}/
```

This setup must be completed **manually** before GitHub Pages can build and serve reports from the `gh-pages` branch. Once configured, GitHub will automatically update the site whenever new reports are deployed.
The bucket must be provisioned by Grafana infra with:

If you’re already using GitHub Pages for other content, ensure that it does not overwrite reports published by the `upload-report-artifacts` Action.
- **Object lifecycle rule** — auto-delete objects after 90 days.
- **IAM** — `roles/storage.objectViewer` granted to the Grafana Google Workspace group; uniform bucket-level access enabled.
- **WIF write access** — the GitHub Actions service account used by Grafana's org-wide Workload Identity provider must have object create/write on the bucket.

## GitHub Pages Visibility
## Viewing Reports

By default, all GitHub Pages sites are publicly accessible on the Internet. Published Playwright reports can include HTML output, screenshots, traces, and internal URLs, so review the contents before enabling this workflow for sensitive repositories. GitHub Enterprise customers can restrict access by configuring access control for private and internal repositories. For more details, refer to the official GitHub [documentation](https://docs.github.com/en/enterprise-cloud@latest/pages/getting-started-with-github-pages/changing-the-visibility-of-your-github-pages-site#about-access-control-for-github-pages-sites).
Report links in PR comments point to `https://storage.cloud.google.com/{bucket}/...`. Viewing them requires being signed into a Google account that is a member of the Grafana Google Workspace group. Users not in the group will see a 403 error.

## Permissions Needed

Use job-scoped permissions instead of granting write access to the entire workflow:

- Set `contents: read` at the workflow level so test jobs can check out the repository.
- Add `contents: write` only to the job that publishes reports to the Pages branch.
- Add `id-token: write` only to the job that publishes reports to GCS (required for Workload Identity Federation authentication).
- Add `pull-requests: write` only if you want `deploy-report-pages` to manage a pull request comment. This permission is not needed when `pr-comment-summary: false`.

The `playwright-gh-pages` actions do not use OIDC, so `id-token: write` is not required for report publishing.

If your workflow runs on `pull_request`, remember that pull requests from forks receive a read-only `GITHUB_TOKEN`. In that case the publish job may be unable to push the Pages branch or update the PR comment. Avoid switching these examples to `pull_request_target` just to get a write-capable token unless you have separately reviewed the security implications of running untrusted code with elevated permissions.
Note: `contents: write` is **not** required. This action does not push to any branch.

## Workflow usage

Expand Down Expand Up @@ -136,21 +136,20 @@ jobs:
with:
test-outcome: ${{ steps.run-tests.outcome }}

deploy-pages:
deploy-reports:
if: ${{ always() && !cancelled() && (github.event_name != 'pull_request' || github.event.pull_request.head.repo.fork == false) }}
needs: [playwright-tests]
runs-on: ubuntu-latest
permissions:
contents: write
id-token: write
pull-requests: write
steps:
- uses: actions/checkout@v4

# use deploy-report-pages Action to deploy the artifacts to GitHub Pages
# use deploy-report-pages Action to upload the artifacts to GCS
- name: Publish report
uses: grafana/plugin-actions/playwright-gh-pages/deploy-report-pages@main
with:
github-token: ${{ secrets.GITHUB_TOKEN }}
bucket: your-gcs-bucket-name
```

### Example using a per-plugin matrix
Expand Down Expand Up @@ -208,26 +207,25 @@ jobs:
test-outcome: ${{ steps.run-tests-latest.outcome }}
# repeat steps but for another Grafana version if necessary

deploy-pages:
deploy-reports:
if: ${{ always() && !cancelled() && (github.event_name != 'pull_request' || github.event.pull_request.head.repo.fork == false) }}
needs: [e2e]
runs-on: ubuntu-latest
permissions:
contents: write
id-token: write
pull-requests: write
steps:
- uses: actions/checkout@v4

# use deploy-report-pages Action to deploy the artifacts to GitHub Pages
# use deploy-report-pages Action to upload the artifacts to GCS
- name: Publish report
uses: grafana/plugin-actions/playwright-gh-pages/deploy-report-pages@main
with:
github-token: ${{ secrets.GITHUB_TOKEN }}
bucket: your-gcs-bucket-name
```

## Inputs

For details on what the available inputs for the Actions, refer to the [README](./deploy-report-pages/README.md) of `deploy-report-pages` and the [README](./upload-report-artifacts/README.md) of `upload-report-artifacts`
For details on available inputs for the Actions, refer to the [README](./deploy-report-pages/README.md) of `deploy-report-pages` and the [README](./upload-report-artifacts/README.md) of `upload-report-artifacts`.

If you want to skip publishing for forked pull requests while still allowing scheduled runs, pushes, and same-repository pull requests, use this condition on the publish job:

Expand Down
19 changes: 9 additions & 10 deletions playwright-gh-pages/deploy-report-pages/README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,17 +1,16 @@
# Publish to GitHub Pages Action
# Deploy Playwright Reports to GCS Action

This GitHub Action automates the process of publishing test artifacts to GitHub Pages and commenting on the pull request with the results and links. It is designed to work together with the `upload-report-artifacts` action.
This GitHub Action uploads Playwright test reports to Google Cloud Storage and comments on the pull request with the results and links. It is designed to work together with the `upload-report-artifacts` action.

## Usage

See full blown examples [here](../README.md).
See full examples [here](../README.md).

## Inputs

| Input Name | Description | Required | Default |
| -------------------- | ------------------------------------------------------------------------------ | -------- | ------------------------ |
| `github-token` | Token for the repository. Pass `${{ secrets.GITHUB_TOKEN }}` from the publish job. | Yes | N/A |
| `retention-days` | Number of days to retain the reports. | Yes | 30 |
| `pr-comment-summary` | Whether to manage a PR comment with the test results. Set this to `false` if you do not want the action to create or delete PR comments. | Yes | true |
| `artifact-pattern` | Pattern to match the artifacts. | Yes | `gf-playwright-report-*` |
| `pages-branch` | Branch to deploy the reports to. | Yes | `gh-pages` |
| Input Name | Description | Required | Default |
| -------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------- | -------- | ------------------------ |
| `github-token` | Token for the repository. Pass `${{ secrets.GITHUB_TOKEN }}` from the publish job. | Yes | N/A |
| `pr-comment-summary` | Whether to manage a PR comment with the test results. Set this to `false` if you do not want the action to create or delete PR comments. | Yes | `true` |
| `artifact-pattern` | Pattern to match the artifacts. | Yes | `gf-playwright-report-*` |
| `bucket` | GCS bucket name to upload reports to. | Yes | N/A |
84 changes: 23 additions & 61 deletions playwright-gh-pages/deploy-report-pages/action.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,14 +1,10 @@
name: 'Deploy to GH pages'
description: 'Deploys test artifacts to GitHub Pages and comments on the PR with the results and links.'
name: 'Deploy test reports'
description: 'Uploads Playwright test reports to Google Cloud Storage and comments on the PR with the results and links.'

inputs:
github-token:
description: 'Token for the repository. Can be passed in using `{{ secrets.GITHUB_TOKEN }}`.'
required: true
retention-days:
description: 'Number of days to retain the reports. Default is 30.'
required: true
default: 30
pr-comment-summary:
description: 'Whether to comment the PR with the test results. Default is true.'
required: true
Expand All @@ -17,10 +13,9 @@ inputs:
description: 'Pattern to match the artifacts. Default is "gf-playwright-report-*"'
required: true
default: 'gf-playwright-report-*'
pages-branch:
description: 'Branch to deploy the reports to. Default is "gh-pages".'
bucket:
description: 'GCS bucket name to upload reports to.'
required: true
default: 'gh-pages'

runs:
using: "composite"
Expand Down Expand Up @@ -54,6 +49,24 @@ runs:
echo "job_initiator=${{ github.run_id }}" >> "$GITHUB_OUTPUT"
fi

- name: Login to GCS
id: login-to-gcs
uses: grafana/shared-workflows/actions/login-to-gcs@6a97ef28077dbb66da3582aad604aef980cd1787 # login-to-gcs/v0.3.0
with:
bucket: ${{ inputs.bucket }}
use_wif_auth: 'true'
delete_credentials_file: 'false'

- name: Upload reports to GCS
uses: google-github-actions/upload-cloud-storage@6397bd7208e18d13ba2619ee21b9873edc94427a # v3.0.0
with:
path: all-reports
destination: ${{ steps.login-to-gcs.outputs.bucket }}/${{ github.repository }}/${{ steps.timestampid.outputs.timestamp }}/${{ steps.set-initiator.outputs.job_initiator }}
parent: false
predefinedAcl: ' '
gzip: false
process_gcloudignore: false

- name: Generate Playwright Test Results Table
id: generate-table
run: |
Expand All @@ -69,9 +82,8 @@ runs:
env:
GITHUB_REPOSITORY_OWNER: ${{ github.repository_owner }}
GITHUB_REPOSITORY_NAME: ${{ github.event.repository.name }}
TIMESTAMP: ${{ steps.timestampid.outputs.timestamp }}
JOB_INITIATOR: ${{ steps.set-initiator.outputs.job_initiator }}
DEFAULT_BRANCH: ${{ github.event.repository.default_branch }}
REPORT_BASE_URL: https://storage.cloud.google.com/${{ steps.login-to-gcs.outputs.bucket }}/${{ github.repository }}/${{ steps.timestampid.outputs.timestamp }}/${{ steps.set-initiator.outputs.job_initiator }}

- name: Write result table in Action summary
shell: bash
Expand All @@ -87,53 +99,3 @@ runs:
create-if-not-exists: true
comment-tag: gf-playwright-test-results
message: ${{ steps.generate-table.outputs.table }}

- name: Validate pages branch safety
shell: bash
run: |
if [[ "$PAGES_BRANCH" == "main" || "$PAGES_BRANCH" == "master" ]]; then
echo "ERROR: Cannot deploy to main/master branch"
echo "Please use a dedicated branch like 'gh-pages' for Pages deployment."
exit 1
fi
env:
PAGES_BRANCH: ${{ inputs.pages-branch }}

- name: Push the new files to github pages
uses: peaceiris/actions-gh-pages@4f9cc6602d3f66b9c108549d475ec49e8ef4d45e # v4.0.0
with:
publish_branch: ${{ inputs.pages-branch }}
github_token: ${{ inputs.github-token }}
publish_dir: all-reports
destination_dir: ${{ steps.timestampid.outputs.timestamp }}/${{ steps.set-initiator.outputs.job_initiator }}
force_orphan: true


- name: Checkout code
uses: actions/checkout@08c6903cd8c0fde910a37f88322edcfb5dd907a8 # v5.0.0
with:
ref: ${{ inputs.pages-branch }}

- name: Delete old reports
shell: bash
run: ${{ github.action_path }}/cleanup-folders.sh --retention-days $RETENTION_DAYS --folder-name .
env:
RETENTION_DAYS: ${{ inputs.retention-days }}

- name: Commit and push changed files
shell: bash
run: |
git config --local user.name grafana-plugins-platform-bot[bot]
git config --local user.email 144369747+grafana-plugins-platform-bot[bot]@users.noreply.github.com

git add -A
if git diff --staged --quiet; then
echo "No changes to commit"
else
git commit -m "Delete folders older than $RETENTION_DAYS days"
git push -f origin $PAGES_BRANCH
fi
env:
RETENTION_DAYS: ${{ inputs.retention-days }}
PAGES_BRANCH: ${{ inputs.pages-branch }}

Loading
Loading