Skip to content

fix(reports): count only rows that have a value for the selected metric - #4478

Open
JeremieSamson wants to merge 1 commit into
umami-software:devfrom
JeremieSamson:fix/performance-sample-size
Open

JeremieSamson wants to merge 1 commit into
umami-software:devfrom
JeremieSamson:fix/performance-sample-size

Conversation

@JeremieSamson

@JeremieSamson JeremieSamson commented Aug 25, 2026

Copy link
Copy Markdown

Problem

In the performance report, the sample size shown next to a Web Vital does not
describe that metric.

getPerformance and getPerformanceMetrics compute percentiles with
percentile_cont(...) within group (order by <metric>) (Postgres) and
quantile(...)(<metric>) (ClickHouse). Both skip NULL rows. The count next to
them is count(*) / count(), which does not.

Web Vitals are sparse. CLS and INP are Chromium-only, so Safari, iOS and
Firefox pageviews store a website_event row with event_type = 5 and no
value for those columns. The two numbers therefore describe different
populations.

This is visible in the UI, which labels the number sampleSize:

{t(labels.sampleSize)}: {formatLongNumber(data.summary?.count || 0)}

Evidence

Self-hosted instance, Postgres, one website, 14 days, event_type = 5:

Metric Rows with a value Reported sample size
TTFB 1681 1681
FCP 1483 1681
LCP 1300 1681
CLS 345 1681
INP 112 1681

So the report claims an INP p75 of 88 ms over 1681 samples, when 112 rows
carry an INP value. The smaller the coverage, the more confident the number
looks.

Fix

Count the selected metric column instead of every row:

count(${metric}) as count

Same change on both engines, in getPerformance.ts (the summary the UI
labels "sample size") and in getPerformanceMetrics.ts (the count returned
for pages, titles, devices and browsers).

Verified on the same data:

 inp_p75 | count_before | count_after
---------+--------------+-------------
      88 |         1681 |         112

Notes

  • metric is validated by z.enum(['lcp','inp','cls','fcp','ttfb']) in
    schema.ts, so interpolating it here is no different from the existing
    percentile expressions in the same queries.
  • The Web Vitals columns are nullable on both engines, so count(column)
    excludes missing values on ClickHouse as well as on Postgres.
  • The UI already re-queries when the selected metric changes (metric: selectedMetric is part of the query parameters), so the summary count
    follows the selection with no UI change.
  • No existing tests cover these two query files.

View with [code]smith Autofix with [code]smith
Need help on this PR? Tag @codesmith-bot with what you need. Autofix is disabled.

The performance report showed count(*) next to percentiles computed with
percentile_cont/quantile, which skip NULL rows. Web Vitals are sparse:
CLS and INP are Chromium-only, so most pageviews carry no value for them.

The UI labels this number "sample size", so it claims to describe the
population behind the displayed percentile. On a real instance over 14
days it reported 1681 for an INP p75 actually computed from 112 rows.

Counting the selected metric column instead makes the number match the
percentile it sits next to. The metric name is validated by a zod enum,
and the columns are nullable on both engines.
@vercel

vercel Bot commented Aug 25, 2026

Copy link
Copy Markdown

@JeremieSamson is attempting to deploy a commit to the Umami Software Team on Vercel.

A member of the Team first needs to authorize it.

@greptile-apps

greptile-apps Bot commented Aug 25, 2026

Copy link
Copy Markdown
Contributor

Greptile Summary

This PR aligns reported Web Vital sample sizes with the non-null observations used to calculate percentile values.

  • Replaces total-row counts with selected-metric counts in the performance summary query.
  • Applies the same counting semantics to grouped performance metrics.
  • Keeps PostgreSQL and ClickHouse implementations behaviorally aligned.

Confidence Score: 5/5

The PR appears safe to merge, with no actionable correctness, security, or compatibility issues identified.

The selected metric is constrained to known nullable Web Vital columns, and both database implementations now count the same non-null population their percentile aggregates use.

Important Files Changed

Filename Overview
src/queries/sql/reports/getPerformance.ts Counts non-null values for the selected Web Vital in both PostgreSQL and ClickHouse summary queries, matching percentile populations.
src/queries/sql/reports/getPerformanceMetrics.ts Counts non-null selected-metric values in grouped performance queries across both supported database engines.

Reviews (1): Last reviewed commit: "fix(reports): count only rows that have ..." | Re-trigger Greptile

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant