Skip to content

ref(api): Log client_kind on the API access log - #123915

Draft
DominikB2014 wants to merge 2 commits into
masterfrom
ref/client-kind-access-log
Draft

ref(api): Log client_kind on the API access log#123915
DominikB2014 wants to merge 2 commits into
masterfrom
ref/client-kind-access-log

Conversation

@DominikB2014

Copy link
Copy Markdown
Contributor

client_kind attribution currently rides on spans only, and span attributes never reach the analytics warehouse. The warehouse promotes a fixed, hand-curated allow-list of columns off spans_gcs and drops every other attribute — there is no tags map, JSON blob, or key-value side table. So client_kind_test, client_host_test and user_agent.original are queryable in a trace and nowhere else.

The api.access log line is warehouse-visible: it lands in internal-sentry.getsentry_api_access_logs.stdout as jsonPayload, and is rolled up daily into api_logs_us.api_log_stdout — the table behind the API Trends dashboard, which today tells callers apart by regex-matching sentry-mcp in the user agent.

This adds the log path. The span is untouched — developers keep reading attribution off a trace; this is additive.

Changes

  • set_client_kind_attributes stashes the kind and host it already derives onto the underlying Django request. The DRF wrapper is gone by the time access_log middleware runs, which is the same reason convert_args assigns request._request.organization. Re-deriving in the middleware is not an option: the organization whose opt-in gates this is resolved during dispatch and out of scope by then.
  • access_log logs client_kind and client_host, absent for orgs that have not opted in.

Known gaps, deliberate

  • internal_service cannot appear here. _create_api_access_log returns early on token_type == "system", so this field describes the mix of external callers by construction.
  • Field naming. The span attributes carry a _test suffix to stay clear of a future real client_kind span attribute. The log is a fresh namespace with no such collision, so these are unsuffixed — the downstream column will not need renaming when the POC suffix goes. Happy to align them if reviewers prefer uniformity.
  • organization_id is still often None in the access log (pre-existing TODO in this file). Attribution is usable without it, but org-level breakdowns will be partial until that is fixed.

Follow-up, not in this PR

The warehouse rollup drops columns it is not told about. A one-line change in getsentry/etl (api_analysis_stdout.sql and api_analysis_stdout_mcp.sql) is needed to carry client_kind through the SELECT/GROUP BY. The grain is already (api_endpoint, user_agent, organization_id, dt), so adding a nine-value column barely moves row counts. Once that lands, API Trends can drop the user-agent regex for a derived field.

Note also that api_logs_us is US-region only (there is a cust_region column); an EU equivalent needs checking before quoting totals.

The client_kind attributes ride on spans today, and spans do not reach the
analytics warehouse: it promotes a fixed, hand-curated allow-list of columns off
`spans_gcs` and drops every other attribute, so nothing set through
`sentry_sdk.set_attribute` is queryable downstream. The `api.access` log line is
warehouse-visible -- it lands in `internal-sentry.getsentry_api_access_logs.stdout`
as `jsonPayload` and is rolled up into `api_logs_us.api_log_stdout`, the table
behind the API Trends dashboard, which today separates callers by regex-matching
`sentry-mcp` in the user agent.

Stash the kind and host `set_client_kind_attributes` already derives onto the
underlying Django request -- the DRF wrapper is gone by the time the middleware
runs -- and log them. The span is untouched: developers keep reading attribution
off a trace, this only adds the path that also reaches analysis.
@github-actions github-actions Bot added the Scope: Backend Automatically applied to PRs that change backend components label Sep 9, 2026
Set the attributes the way `convert_args` sets `organization`, with one comment
instead of a docstring. Registers both on the mypy plugin's `HttpRequest`, which
is what lets the `organization` assignment type-check today.
path=request.path,
caller_ip=request.META.get("REMOTE_ADDR"),
user_agent=request.META.get("HTTP_USER_AGENT"),
client_kind=client_kind.value if client_kind is not None else None,

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

~ can this be simplified to client_kind=getattr(request, "client_kind", None), like we do in the tests?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Scope: Backend Automatically applied to PRs that change backend components

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants