-
-
Notifications
You must be signed in to change notification settings - Fork 8.7k
[build] Add weekly CI check for Plausible pageview usage #17900
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: trunk
Are you sure you want to change the base?
Changes from all commits
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,34 @@ | ||
| name: Plausible Usage | ||
| on: | ||
| schedule: | ||
| - cron: 30 12 * * 3 | ||
| workflow_dispatch: | ||
|
|
||
| permissions: | ||
| contents: read | ||
|
|
||
| jobs: | ||
| check: | ||
| name: Check Weekly Pageviews | ||
| runs-on: ubuntu-latest | ||
| if: github.event.repository.fork == false | ||
| steps: | ||
| - uses: actions/checkout@v6 | ||
| with: | ||
| persist-credentials: false | ||
| - name: Check Plausible Pageviews | ||
| id: check | ||
| env: | ||
| PLAUSIBLE_STATS_KEY: ${{ secrets.PLAUSIBLE_STATS_KEY }} | ||
| run: ./scripts/github-actions/check-plausible-usage.sh | ||
| - name: Slack Notification | ||
| if: failure() | ||
| uses: rtCamp/action-slack-notify@v2 | ||
| env: | ||
| SLACK_ICON_EMOJI: ":rotating_light:" | ||
| SLACK_COLOR: failure | ||
| SLACK_CHANNEL: selenium-tlc | ||
| SLACK_USERNAME: GitHub Workflows | ||
| SLACK_MESSAGE: ${{ steps.check.outputs.report || 'Plausible stats could not be read for last week' }} | ||
| MSG_MINIMAL: actions url | ||
| SLACK_WEBHOOK: ${{ secrets.SLACK_WEBHOOK_URL }} |
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,101 @@ | ||
| #!/usr/bin/env bash | ||
| # | ||
| # Fail when Plausible pageviews for the last seven complete days exceed a threshold. | ||
| # | ||
| # Selenium Manager telemetry is billed against a Plausible plan, and sustained | ||
| # overage locks the dashboard, so we watch the trailing week and raise an alarm | ||
| # with enough headroom to react. | ||
| # | ||
| # Requires PLAUSIBLE_STATS_KEY. PLAUSIBLE_SITE_ID, PLAUSIBLE_THRESHOLD and | ||
| # PLAUSIBLE_MONTHLY_LIMIT may be set to override the defaults. | ||
|
|
||
| set -euo pipefail | ||
|
|
||
| SITE_ID="${PLAUSIBLE_SITE_ID:-manager.selenium.dev}" | ||
| THRESHOLD="${PLAUSIBLE_THRESHOLD:-17000000}" | ||
| MONTHLY_LIMIT="${PLAUSIBLE_MONTHLY_LIMIT:-75000000}" | ||
|
|
||
| : "${PLAUSIBLE_STATS_KEY:?PLAUSIBLE_STATS_KEY is required}" | ||
|
|
||
| millions() { | ||
| awk -v n="$1" 'BEGIN { v = n / 1000000; printf (v == int(v) ? "%d" : "%.1f"), v }' | ||
| } | ||
|
|
||
| # Yesterday back six days; Plausible's period=7d rolls into today's partial data. | ||
| if date -u -d 'yesterday' +%F >/dev/null 2>&1; then | ||
| END="$(date -u -d 'yesterday' +%F)" | ||
| START="$(date -u -d '7 days ago' +%F)" | ||
| else # BSD/macOS date | ||
| END="$(date -u -v-1d +%F)" | ||
| START="$(date -u -v-7d +%F)" | ||
| fi | ||
|
|
||
| echo "Site: ${SITE_ID}" | ||
| echo "Window: ${START} to ${END}" | ||
| echo "Threshold: $(millions "$THRESHOLD")M per week, $(millions "$MONTHLY_LIMIT")M per month" | ||
|
|
||
| # The status code is appended so a 4xx body reaches the log instead of being | ||
| # swallowed by the failing assignment. | ||
| if ! response="$(curl --silent --show-error --write-out '\n%{http_code}' \ | ||
| --get "https://plausible.io/api/v1/stats/aggregate" \ | ||
| --header "Authorization: Bearer ${PLAUSIBLE_STATS_KEY}" \ | ||
| --data-urlencode "site_id=${SITE_ID}" \ | ||
|
Comment on lines
+39
to
+42
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. 2. Curl request can hang check-plausible-usage.sh calls Plausible via curl without any connection/overall timeout, so a stalled network request can block the scheduled workflow until GitHub’s job timeout and delay/prevent the Slack alert. Agent Prompt
|
||
| --data-urlencode "period=custom" \ | ||
| --data-urlencode "date=${START},${END}" \ | ||
| --data-urlencode "metrics=pageviews")"; then | ||
| echo "::error::Could not reach the Plausible API" | ||
| exit 1 | ||
| fi | ||
|
|
||
| http_code="${response##*$'\n'}" | ||
| body="${response%$'\n'*}" | ||
|
|
||
| echo "Response: HTTP ${http_code} ${body}" | ||
|
|
||
| if [[ "$http_code" != 200 ]]; then | ||
| echo "::error::Plausible API returned HTTP ${http_code}" | ||
| exit 1 | ||
| fi | ||
|
|
||
| pageviews="$(jq -r '.results.pageviews.value // empty' <<<"$body" 2>/dev/null || true)" | ||
|
|
||
| if [[ ! "$pageviews" =~ ^[0-9]+$ ]]; then | ||
| echo "::error::No pageviews value in the Plausible response" | ||
|
Comment on lines
+60
to
+63
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. 1. No tests for plausible script This PR adds a new weekly CI check with non-trivial parsing and budget/threshold calculations, but introduces no test coverage to validate behavior across success/error responses. Lacking tests makes the workflow brittle and increases the chance of silent regressions in monitoring/alerting logic. Agent Prompt
|
||
| exit 1 | ||
| fi | ||
|
|
||
| over=false | ||
| direction=under | ||
| if ((pageviews > THRESHOLD)); then | ||
| over=true | ||
| direction=over | ||
| fi | ||
|
|
||
| monthly="$(awk -v p="$pageviews" 'BEGIN { printf "%d", p / 7 * 30 }')" | ||
| percent="$(awk -v p="$pageviews" -v t="$THRESHOLD" 'BEGIN { printf "%.0f", (p - t) / t * 100 }')" | ||
|
|
||
| report="Plausible stats are ${percent#-}% ${direction} budget for last week" | ||
|
|
||
| echo "$report" | ||
| echo "On pace for $(millions "$monthly")M this month, against a $(millions "$MONTHLY_LIMIT")M limit" | ||
|
|
||
| if [[ -n "${GITHUB_OUTPUT:-}" ]]; then | ||
| { | ||
| echo "pageviews=${pageviews}" | ||
| echo "percent=${percent#-}" | ||
| echo "monthly=${monthly}" | ||
| echo "start=${START}" | ||
| echo "end=${END}" | ||
| echo "over=${over}" | ||
| echo "report=${report}" | ||
| } >>"$GITHUB_OUTPUT" | ||
| fi | ||
|
|
||
| if [[ -n "${GITHUB_STEP_SUMMARY:-}" ]]; then | ||
| echo "$report" >>"$GITHUB_STEP_SUMMARY" | ||
| fi | ||
|
|
||
| if [[ "$over" == true ]]; then | ||
| echo "::error::${report}" | ||
| exit 1 | ||
| fi | ||
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
3. Unvalidated numeric overrides
🐞 Bug≡ CorrectnessAgent Prompt
ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools