Skip to content

Scrub the database hostname from Sentry events - #5592

Open
felipeatom wants to merge 1 commit into
WordPress:mainfrom
felipeatom:fix/scrub-db-hostname-from-sentry
Open

Scrub the database hostname from Sentry events#5592
felipeatom wants to merge 1 commit into
WordPress:mainfrom
felipeatom:fix/scrub-db-hostname-from-sentry

Conversation

@felipeatom

Copy link
Copy Markdown

Fixes

Fixes #670

Description

send_default_pii=False and Sentry's server-side scrubbing only cover known PII fields by key. Infrastructure hostnames such as the database DNS name can still reach Sentry as free text — inside breadcrumbs, span descriptions or exception messages — where key-based scrubbing never sees them.

This adds a before_send / before_send_transaction hook that redacts the configured database hostname wherever it appears, as a substring, anywhere in an event.

A note on approach: the issue links to Sentry's server-side scrubbing docs, but doing this client-side is strictly safer — the value is redacted before the event leaves the process, so it is never stored in Sentry at all. It also keeps the rule in version control next to the rest of the Sentry config.

Details:

  • The scrubber lives in api/utils/sentry.py as a small, dependency-free helper so it can be unit-tested in isolation.
  • Non-sensitive defaults (localhost, 127.0.0.1, ::1, empty) are ignored, so local/dev never redacts noise.
  • Longer hostnames are redacted first, so a host that is a substring of another does not leave a partial value behind.

Testing Instructions

just api/test test/unit/utils/test_sentry.py — or, since the helper is dependency-free:

from api.utils.sentry import make_sensitive_value_scrubber, FILTERED

scrub = make_sensitive_value_scrubber(["db.internal.example.com"])
event = {"message": "could not connect to db.internal.example.com:5432"}
assert FILTERED in scrub(event)["message"]

The added unit tests cover redaction across a nested event, ignored default hosts, the no-op case, and the overlapping-hostname ordering.

Checklist

  • My pull request has a descriptive title (not a vague title like Update index.md).
  • My pull request targets the default branch of the repository (main).
  • My change is covered by tests where possible.

Note / follow-up

This covers the API (Django) side. The ingestion server initialises Sentry separately (ingestion_server/ingestion_server/api.py); happy to extend the same approach there in this PR or a follow-up — whichever the maintainers prefer.

send_default_pii=False and Sentry's key-based scrubbing don't cover
infrastructure hostnames that reach Sentry as free text — inside breadcrumbs,
span descriptions or exception messages. The database DNS name can leak there.

Add a before_send/before_send_transaction hook that redacts the configured
database hostname wherever it appears in an event, before it leaves the process
(client-side, so the value is never stored in Sentry at all). Non-sensitive
defaults like localhost are ignored.

Fixes WordPress#670
@felipeatom
felipeatom requested a review from a team as a code owner July 24, 2026 18:52
@felipeatom
felipeatom requested review from krysal and obulat and removed request for a team July 24, 2026 18:52
@openverse-bot openverse-bot added 🧱 stack: api Related to the Django API 🟩 priority: low Low priority and doesn't need to be rushed 🧰 goal: internal improvement Improvement that benefits maintainers, not users 💻 aspect: code Concerns the software code in the repository 🧱 stack: ingestion server Related to the ingestion/data refresh server labels Jul 24, 2026
@openverse-bot openverse-bot moved this to 👀 Needs Review in Openverse PRs Jul 24, 2026
@felipeatom

Copy link
Copy Markdown
Author

A quick note on the red CI, since none of it looks related to this change:

  • Build Docker images (ingestion_server) fails at ingestion_server/Dockerfile:36 on pipenv install --system --deploy --dev (ERROR:: Aborting deploy, i.e. Pipfile.lock out of sync with Pipfile). This PR only touches files under api/ — nothing in ingestion_server/, its Pipfile, or its Dockerfile — so this appears to be a pre-existing failure rather than something introduced here.
  • Run tests for the API shows up as skipped, so test/unit/utils/test_sentry.py didn't actually execute in CI (it seems to be gated behind the failed build above). For what it's worth, the added tests pass locally (7 passed), and the Run Django checks jobs are green, which exercises the settings wiring added in conf/settings/sentry.py.
  • The Check … label jobs are red because the PR is missing the required labels — I don't have permission to add those as an external contributor, so I'd appreciate a maintainer applying the appropriate priority / aspect / stack / goal labels when triaging.

Happy to rebase or adjust anything. Thanks for taking a look!

@felipeatom

Copy link
Copy Markdown
Author

Following up with a likely root cause for the Build Docker images (ingestion_server) failure, in case it's useful (it looks unrelated to this PR and repo-wide):

The failing step is pipenv install --system --deploy --dev, and the traceback ends in handle_lockfileDeployException ("Aborting deploy") — i.e. --deploy rejecting ingestion_server/Pipfile.lock because its hash no longer matches what the installed pipenv computes.

ingestion_server/Dockerfile installs pipenv unpinned:

&& pip install --upgrade pipenv
...
RUN pipenv install --system --deploy --dev

So the build always pulls the latest pipenv at build time. When a pipenv release changes lockfile hashing/validation, --deploy starts failing against the committed lock for every build — which fits this being a sudden, PR-independent break rather than anything in a specific diff.

Two directions, whichever the team prefers:

  • Pin pipenv in the Dockerfile (pip install "pipenv==<known-good>") so builds are reproducible, and/or
  • Regenerate Pipfile.lock with the current pipenv.

Happy to open a separate PR for whichever approach you'd like — I'd just want to build the image against the pinned version to confirm the fix before sending it.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

💻 aspect: code Concerns the software code in the repository 🧰 goal: internal improvement Improvement that benefits maintainers, not users 🟩 priority: low Low priority and doesn't need to be rushed 🧱 stack: api Related to the Django API 🧱 stack: ingestion server Related to the ingestion/data refresh server

Projects

Status: 👀 Needs Review

Development

Successfully merging this pull request may close these issues.

Configure sentry scrubbing for things like database DNS names

2 participants