Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
34 changes: 34 additions & 0 deletions .github/workflows/plausible-usage.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,34 @@
name: Plausible Usage
on:
schedule:
- cron: 30 12 * * 3
workflow_dispatch:

permissions:
contents: read

jobs:
check:
name: Check Weekly Pageviews
runs-on: ubuntu-latest
if: github.event.repository.fork == false
steps:
- uses: actions/checkout@v6
with:
persist-credentials: false
- name: Check Plausible Pageviews
id: check
env:
PLAUSIBLE_STATS_KEY: ${{ secrets.PLAUSIBLE_STATS_KEY }}
run: ./scripts/github-actions/check-plausible-usage.sh
- name: Slack Notification
if: failure()
uses: rtCamp/action-slack-notify@v2
env:
SLACK_ICON_EMOJI: ":rotating_light:"
SLACK_COLOR: failure
SLACK_CHANNEL: selenium-tlc
SLACK_USERNAME: GitHub Workflows
SLACK_MESSAGE: ${{ steps.check.outputs.report || 'Plausible stats could not be read for last week' }}
MSG_MINIMAL: actions url
SLACK_WEBHOOK: ${{ secrets.SLACK_WEBHOOK_URL }}
101 changes: 101 additions & 0 deletions scripts/github-actions/check-plausible-usage.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,101 @@
#!/usr/bin/env bash
#
# Fail when Plausible pageviews for the last seven complete days exceed a threshold.
#
# Selenium Manager telemetry is billed against a Plausible plan, and sustained
# overage locks the dashboard, so we watch the trailing week and raise an alarm
# with enough headroom to react.
#
# Requires PLAUSIBLE_STATS_KEY. PLAUSIBLE_SITE_ID, PLAUSIBLE_THRESHOLD and
# PLAUSIBLE_MONTHLY_LIMIT may be set to override the defaults.

set -euo pipefail

SITE_ID="${PLAUSIBLE_SITE_ID:-manager.selenium.dev}"
THRESHOLD="${PLAUSIBLE_THRESHOLD:-17000000}"
MONTHLY_LIMIT="${PLAUSIBLE_MONTHLY_LIMIT:-75000000}"
Comment on lines +14 to +16

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Informational

3. Unvalidated numeric overrides 🐞 Bug ≡ Correctness

PLAUSIBLE_THRESHOLD and PLAUSIBLE_MONTHLY_LIMIT are documented as overridable but never validated as
numeric; invalid values can be coerced (e.g., to 0) or trigger arithmetic/awk errors, causing
incorrect over/under decisions or confusing failures.
Agent Prompt
### Issue description
`PLAUSIBLE_THRESHOLD` / `PLAUSIBLE_MONTHLY_LIMIT` are treated as numbers but are not validated before use. In bash arithmetic and awk, malformed values may be coerced or error out, leading to false alerts or hard-to-debug failures.

### Issue Context
The script already validates `pageviews` is numeric, but not the user-supplied numeric configuration.

### Fix Focus Areas
- scripts/github-actions/check-plausible-usage.sh[14-16]
- scripts/github-actions/check-plausible-usage.sh[62-76]

ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools


: "${PLAUSIBLE_STATS_KEY:?PLAUSIBLE_STATS_KEY is required}"

millions() {
awk -v n="$1" 'BEGIN { v = n / 1000000; printf (v == int(v) ? "%d" : "%.1f"), v }'
}

# Yesterday back six days; Plausible's period=7d rolls into today's partial data.
if date -u -d 'yesterday' +%F >/dev/null 2>&1; then
END="$(date -u -d 'yesterday' +%F)"
START="$(date -u -d '7 days ago' +%F)"
else # BSD/macOS date
END="$(date -u -v-1d +%F)"
START="$(date -u -v-7d +%F)"
fi

echo "Site: ${SITE_ID}"
echo "Window: ${START} to ${END}"
echo "Threshold: $(millions "$THRESHOLD")M per week, $(millions "$MONTHLY_LIMIT")M per month"

# The status code is appended so a 4xx body reaches the log instead of being
# swallowed by the failing assignment.
if ! response="$(curl --silent --show-error --write-out '\n%{http_code}' \
--get "https://plausible.io/api/v1/stats/aggregate" \
--header "Authorization: Bearer ${PLAUSIBLE_STATS_KEY}" \
--data-urlencode "site_id=${SITE_ID}" \
Comment on lines +39 to +42

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Remediation recommended

2. Curl request can hang 🐞 Bug ☼ Reliability

check-plausible-usage.sh calls Plausible via curl without any connection/overall timeout, so a
stalled network request can block the scheduled workflow until GitHub’s job timeout and
delay/prevent the Slack alert.
Agent Prompt
### Issue description
The Plausible API request uses `curl` without `--connect-timeout` and `--max-time` (and optionally retries). A hung TCP/TLS connection or slow upstream can stall the whole workflow run.

### Issue Context
This script is executed by a weekly scheduled workflow; the job only notifies Slack on failure, so a hung request delays the alert.

### Fix Focus Areas
- scripts/github-actions/check-plausible-usage.sh[39-45]
- .github/workflows/plausible-usage.yml[1-6]

ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools

--data-urlencode "period=custom" \
--data-urlencode "date=${START},${END}" \
--data-urlencode "metrics=pageviews")"; then
echo "::error::Could not reach the Plausible API"
exit 1
fi

http_code="${response##*$'\n'}"
body="${response%$'\n'*}"

echo "Response: HTTP ${http_code} ${body}"

if [[ "$http_code" != 200 ]]; then
echo "::error::Plausible API returned HTTP ${http_code}"
exit 1
fi

pageviews="$(jq -r '.results.pageviews.value // empty' <<<"$body" 2>/dev/null || true)"

if [[ ! "$pageviews" =~ ^[0-9]+$ ]]; then
echo "::error::No pageviews value in the Plausible response"
Comment on lines +60 to +63

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Remediation recommended

1. No tests for plausible script 📘 Rule violation ☼ Reliability

This PR adds a new weekly CI check with non-trivial parsing and budget/threshold calculations, but
introduces no test coverage to validate behavior across success/error responses. Lacking tests makes
the workflow brittle and increases the chance of silent regressions in monitoring/alerting logic.
Agent Prompt
## Issue description
The new Plausible usage check script adds behavior (API response parsing and threshold/budget calculations) without any accompanying tests.

## Issue Context
Compliance requires new behavior to be covered by tests where practical, preferring small/unit tests. For this script, you can make the core logic testable without hitting the real Plausible API by isolating calculation/parsing into functions and feeding representative sample JSON inputs.

## Fix Focus Areas
- scripts/github-actions/check-plausible-usage.sh[60-101]

ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools

exit 1
fi

over=false
direction=under
if ((pageviews > THRESHOLD)); then
over=true
direction=over
fi

monthly="$(awk -v p="$pageviews" 'BEGIN { printf "%d", p / 7 * 30 }')"
percent="$(awk -v p="$pageviews" -v t="$THRESHOLD" 'BEGIN { printf "%.0f", (p - t) / t * 100 }')"

report="Plausible stats are ${percent#-}% ${direction} budget for last week"

echo "$report"
echo "On pace for $(millions "$monthly")M this month, against a $(millions "$MONTHLY_LIMIT")M limit"

if [[ -n "${GITHUB_OUTPUT:-}" ]]; then
{
echo "pageviews=${pageviews}"
echo "percent=${percent#-}"
echo "monthly=${monthly}"
echo "start=${START}"
echo "end=${END}"
echo "over=${over}"
echo "report=${report}"
} >>"$GITHUB_OUTPUT"
fi

if [[ -n "${GITHUB_STEP_SUMMARY:-}" ]]; then
echo "$report" >>"$GITHUB_STEP_SUMMARY"
fi

if [[ "$over" == true ]]; then
echo "::error::${report}"
exit 1
fi
Loading