feat(filters): support wildcards in filters and funnel/goal steps

[asm0dey]

Support wildcards in filters and funnel/goal steps

Fixes #4500

Rebased on dev after 26e236c ("Fix Goal/Funnel wildcard UI bug"). That commit fixed the combobox half of #4500 via the new allowCustomValue prop on LookupField, so the LookupField change this PR originally carried is gone. What remains is the same defect in one component allowCustomValue does not reach, plus the query-layer work.

Problem

Two separate defects, both surfaced by the same use case in #4500 — building a funnel from /blog/* to a file-download event.

1. Typed values that match no lookup item are silently discarded — still true in the funnel step-filter editor.

Base UI's Combobox.Root resyncs the input to the selected item's label when the popup closes in single-select mode with the input outside the popup:

// @base-ui/react/combobox/root/AriaCombobox.js
const stringVal = stringifyAsLabel(selectedValue, itemToStringLabel);
if (inputRef.current && inputRef.current.value !== stringVal) {
  const reason = stringVal === '' ? REASONS.inputClear : REASONS.none;
  setInputValue(stringVal, createChangeEventDetails(reason));
}

Nothing is ever selected when the typed text matches no item, so stringVal is '' and the typed value is wiped before submit.

26e236c solved this for LookupField by prepending the typed value to items behind allowCustomValue, which covers the funnel and goal step value fields. ValueSelect in EventDataFilterRow.tsx — the funnel step-filter editor — uses ComboBox directly and still has the defect, so a step filter value absent from the lookup data cannot be entered. This PR passes value alongside inputValue there, giving Base UI the same string to write back on close.

Which combobox is fixed where:

 FunnelEditForm / GoalEditForm                       -- step VALUE
   <LookupField allowCustomValue>                    -- fixed on dev by 26e236c9
     <ComboBox items={[value, ...items]}>

 EventDataFilterRow -> ValueSelect                   -- step FILTER value
-  <ComboBox items={values} inputValue={value}>      -- typed value wiped on close
+  <ComboBox items={values} value={value} inputValue={value}>

 CohortEditForm                                      -- not covered by either; see Notes
   <LookupField>

2. Wildcards escaped nothing, and were unavailable outside funnels and goals.

getFunnel.ts and getGoal.ts recognised a leading/trailing * via startsWith('*') || endsWith('*') and mapped it to % with value.replace(/^\*|\*$/g, '%') — but neither escaped SQL LIKE metacharacters. The tracker stores new URL(raw, location.href).toString(), so url_path and url_query routinely hold percent-encoded sequences: /blog/hello%20world, ?q=caf%C3%A9. Unescaped, %20 means "any run of characters, then 20", so /blog/hello%20world matched /blog/hellozzz20world. _ — a plain legal URL character, common in slugs — matched any single character. Wildcards were also unavailable entirely in the filter bar, segments, cohorts, and property filters.

What changed

Combobox fix (ValueSelect only)

• ValueSelect now passes value alongside inputValue, so Base UI's close-sync writes back the same string instead of ''.

New matches / doesNotMatch operators (wc / nwc)

• src/lib/wildcard.ts — one shared helper converting a value to a LIKE pattern, keeping the existing leading/trailing-* semantics: a * at the start and/or end becomes %, and everything between is escaped (\, %, _) so it matches literally. A * in the middle of a value stays a literal *.

  typed value	SQL operator	bound parameter	matches
  /blog/*	like	/blog/%	/blog/post-1
  */thanks	like	%/thanks	/checkout/thanks
  /blog/*/comments	=	/blog/*/comments	only a path containing a literal *
  /sale/100%*	like	/sale/100\%%	/sale/100%-off, not /sale/100zzz
  /a_b	=	/a_b	only /a_b, not /axb

  The last two rows are the escaping fix: on dev those % and _ acted as LIKE wildcards.

• Wired through OPERATORS, the zod schemas, isSearchOperator, and both query builders (prisma.ts, clickhouse.ts), including all three property-filter builders in each.

  OPERATORS.matches = 'wc' / doesNotMatch = 'nwc'      src/lib/constants.ts
  ├── useFilters, useOperatorLabels                    filter bar
  ├── PropertyFilterRecord                             property filters
  ├── OperatorSelect                                   funnel step-filter editor
  ├── parseFilterValue -> isSearchOperator             src/lib/params.ts
  ├── filterSchema, segmentSchema                      src/lib/schema.ts
  ├── prisma.ts     getFilterQuery ──────────────┐
  │                 getEventPropertyFilterQuery  ├──> wildcardToLikePattern
  │                 getSessionPropertyFilterQuery│
  │                 getPropertyFilterQuery ──────┘
  ├── clickhouse.ts  (the same four builders) ───────> wildcardToLikePattern
  └── getFunnel.ts, getGoal.ts  step values ─────────> hasWildcard + wildcardToLikePattern
  

• Exposed in the filter bar, property filters, and the funnel step-filter editor.

Funnels and goals

• Both now use the shared helper, so LIKE metacharacters in the value are escaped instead of acting as wildcards. Wildcard placement is unchanged: leading and/or trailing * only.

Values reach SQL only through bound parameters ({{param}} / {param:Type}); the helper transforms the value, never the SQL text. Backslash is the default LIKE escape character in both PostgreSQL and ClickHouse, so no ESCAPE clause is needed.

Case sensitivity follows each site's existing contains behaviour rather than introducing a new convention: filter and property-filter sites use ilike/ILIKE; funnel and goal step values keep case-sensitive like on both engines, so existing funnels' numbers do not move.

How to test

The /blog/* step value from #4500 now works on dev alone, via 26e236c. What this PR adds on top:

Step filters (the remaining combobox gap)

1. Website → Funnels → create a funnel.
2. Step 1: Triggered event, value file-download. Add a step filter: property file, operator Matches wildcard, value liberica-*.
3. Click away from the value field. The value stays liberica-* — on dev it goes blank.
4. Save and run.

Wildcard operators in the filter bar

• Filter bar → Path → "Matches wildcard" → /blog/*; the URL becomes ?path=wc.%2Fblog%2F* and survives a reload.
• */thanks (leading wildcard), and /blog/*/comments — the middle * is a literal, so this matches only a path that really contains *.
• A percent-encoded path, e.g. /blog/hello%20world under the filter /blog/* — the %20 is escaped and matches literally. On dev the funnel/goal equivalent also matched /blog/hellozzz20world.

Automated:

pnpm exec vitest run

Baseline on dev (336fbcb3c) is 759 passing with 4 pre-existing collection failures (boards, Empty, SharePage, SessionProfile — a vite:css plugin error on ChartAnnotationMarkers.module.css, present without this branch). With this branch: 796 passing, the same 4 pre-existing failures, no regressions. biome lint reports an identical 6 errors / 13 warnings / 11 infos before and after, and tsc --noEmit is clean on both.

An earlier revision of this branch was additionally verified against a live instance (Postgres via docker compose up -d db, pnpm dev, three seeded sessions hitting /blog/post-1..3 then file-download, plus one /pricing session as a control): the saved funnel reported 3 visitors → 3 visitors, 100%, with the /pricing session correctly excluded. That run exercised the LookupField path now owned by 26e236c; the query-layer behaviour it confirmed is unchanged here.

Notes for reviewers

• This changes results for existing saved funnels and goals in one narrow case. A step value whose literal text contains % or _ — including any percent-encoded path — is now escaped instead of behaving as a LIKE wildcard. That is the intended semantics, but it will move numbers on affected definitions (generally by removing spurious matches).
• Wildcard placement is deliberately unchanged from dev: leading and/or trailing * only. No mid-string globbing and no ? single-character wildcard were added; the existing regex operator covers those cases.
• A literal * at the very start or end of a step value still cannot be expressed — also unchanged from dev.
• getGoal.ts is included because leaving it would make a goal and a funnel step disagree about what the same value means — they share the same input component.
• Cohort value fields also use LookupField and were not given allowCustomValue in 26e236c, so a custom cohort value is still discarded there. Out of scope for wildcards in filters #4500 and untouched here — flagging it in case you want it as a follow-up.
• No database-backed test exists in this repo, so the LIKE-escaping behaviour is covered by unit tests asserting the generated SQL and bound parameters rather than by round-tripping against PostgreSQL or ClickHouse.

AI Disclosure

This pull request was prepared with the assistance of Claude Code (Claude Opus 5). The root-cause analysis, implementation, and tests were AI-drafted across a task-by-task plan with independent review passes; I directed the work and reviewed the change before submitting it. The test, lint, and typecheck results reported above were produced by running those commands on this branch and on a pristine dev for comparison.

One coverage gap stated plainly: jsdom cannot drive Base UI's popup lifecycle, so the ValueSelect change is not covered by a unit test asserting the close-and-reset behaviour itself. This repo has Playwright e2e infrastructure under tests/e2e/; happy to add a spec there before merging if you'd prefer the check to run in CI.

[vercel]

@asm0dey is attempting to deploy a commit to the Umami Software Team on Vercel.

A member of the Team first needs to authorize it.

[greptile-apps]

Greptile Summary

This PR adds glob-style wildcard operators across filters, property filters, funnels, and goals, while preserving arbitrary combobox input that is absent from lookup results.

• Adds shared conversion of * and ? into escaped SQL LIKE patterns.
• Implements wildcard filtering in both PostgreSQL and ClickHouse query builders.
• Extends filter schemas, URL parsing, operator controls, funnel steps, and tests.
• Leaves the new operator labels unavailable in non-English locale catalogs.

Confidence Score: 4/5

The wildcard implementation is functionally coherent, but the missing non-English catalog entries should be fixed before merging because localized users will see broken operator labels.

The new labels are resolved through whichever locale catalog is active, while only the English catalog defines them, causing visible missing-key fallback text in every other supported locale.

Files Needing Attention: src/components/messages.ts and public/intl/messages/*.json

Important Files Changed

Filename	Overview
src/lib/wildcard.ts	Adds the shared glob-to-LIKE conversion and escapes user-supplied LIKE metacharacters before translating glob characters.
src/lib/prisma.ts	Adds wildcard parameter conversion and case-insensitive matching throughout PostgreSQL column and property-filter builders.
src/lib/clickhouse.ts	Mirrors wildcard support across ClickHouse column and property-filter builders using bound query parameters.
src/queries/sql/reports/getFunnel.ts	Extends both database branches to recognize wildcards anywhere in step values and adds wildcard step-filter operators.
src/queries/sql/reports/getGoal.ts	Replaces endpoint-only asterisk handling with shared wildcard detection and conversion in both database branches.
src/components/input/LookupField.tsx	Synchronizes the combobox selected value with arbitrary typed input so unmatched values survive popup closure.
src/components/messages.ts	Adds wildcard label keys, but matching entries were added only to en-US.json and are missing from every other locale catalog.

Flowchart

%%{init: {'theme': 'neutral'}}%%
flowchart LR
  UI[Filter or funnel UI] --> Params[wc / nwc serialized parameters]
  Params --> Schema[Schema and parameter parsing]
  Schema --> Glob[wildcardToLikePattern]
  Glob --> Router{Configured event backend}
  Router --> PG[PostgreSQL ILIKE / LIKE]
  Router --> CH[ClickHouse ILIKE / LIKE]
  PG --> Results[Filtered analytics results]
  CH --> Results

Loading

_{Reviews (1): Last reviewed commit: "test(filters): cover live operators arra..." | Re-trigger Greptile}

[greptile-apps]

Wildcard labels missing translations

When a user selects either wildcard operator in any locale other than en-US, the active locale catalog lacks the new message keys, causing missing-key fallback text to appear instead of translated operator labels.

Knowledge Base Used: Frontend App Shell

Note: If this suggestion doesn't match your team's coding style, reply to this and let me know. I'll remember it for next time!

[asm0dey]

On the flagged i18n item — that was deliberate, but I'm happy to go either way.

The two new keys (label.matches, label.does-not-match) were added to en-US.json only, which matches how new strings seem to land here: the feature PR adds English, and a later sweep commit fills in the other 51 locales — e.g. cb9cb27fb ("add missing 2FA translations"), 0dcba7d38 ("sync translations for 2FA and misc messages/labels"), and 3e729bc0e, each touching all 51 files. dev currently has 153 missing entries in that same state, including label.note, label.notes, and label.add-note from 64e4cc1c3.

The mechanism Greptile describes is real, though — NextIntlClientProvider is configured with onError={() => null} and no fallback locale, so a missing key renders as the raw key path rather than the English string. It's just not something this PR introduces.

I'd rather not machine-translate "Matches wildcard" / "Does not match wildcard" into 51 languages and put them in front of your translators as a fait accompli — "wildcard" is a term of art and I'd be guessing at idiomatic phrasing for a good number of those locales. But I'm glad to do whichever you prefer:

1. Leave as-is and let it ride with your next translation sync (what I've assumed).
2. Add both keys to all 51 locale files in a separate follow-up PR, so this diff stays focused.
3. Add them here in this PR if you'd rather not carry the gap at all.

Just say which and I'll turn it around.

[asm0dey]

A reviewer question came up about whether wildcard filters need an index at long date ranges, so I measured it instead of guessing. No change to this PR is proposed — posting in case the numbers are useful.

Methodology

• PostgreSQL 15.19 (postgres:15-alpine) in Docker, 16-core host, NVMe (3.4 GB/s measured, dd O_DIRECT).
• Server defaults as shipped by the image: shared_buffers=128MB, work_mem=4MB, effective_cache_size=4GB, max_parallel_workers_per_gather=2, jit=on.
• website_event: 12.4M rows, 43 websites, 2.4M distinct url_path values, 365-day span, umami's full index set from prisma/schema.prisma, ANALYZE run before measuring.
• Queries are the shapes this PR emits, measured with EXPLAIN (ANALYZE); the aggregate is count(distinct session_id) as the funnel query does.
• Insert cost measured separately on a logged table (relpersistence='p') with synchronous_commit=on, in 300k-row batches.

1. Wildcards don't add a new cost class

At a 365-day range every shape sequentially scans, including plain equality:

Query	Rows matched	Time
url_path = '/blog/post-1'	2,000	230 ms
url_path LIKE '/blog/%' (this PR)	3,000,000	1,302 ms
url_path ILIKE '%blog%' (existing contains)	3,000,000	3,160 ms

The new anchored wildcard is ~2.4× cheaper than the contains operator already shipping. Cost is dominated by aggregating the matched rows, not by pattern matching. A pg_trgm GIN index made the broad case worse (1,302 → 1,765 ms); re-tested with work_mem=1GB to rule out a lossy bitmap — still slower, because /blog/% matches 3M rows and the index yields no selectivity.

2. There may be a worthwhile index — but for existing queries, not this PR

(website_id, created_at, url_path) can't use url_path as a range condition because it follows a range column, so once date selectivity drops the planner abandons it entirely. A (website_id, url_path text_pattern_ops) B-tree changes that:

Selective path, wide dates	as shipped	+ text_pattern_ops
LIKE '/blog/post-123%', 90-day span	276 ms (parallel seq scan)	13.7 ms
same, 3 disjoint months (OR of ranges)	259 ms (parallel seq scan)	6.7 ms

This helps existing funnel and goal queries too — the shape is "specific page, long or fragmented time range."

3. What it costs on writes

Index size: 201 MB for 12.4M rows (~17 bytes/row) at 2.4M distinct paths.

Insert throughput, 300k-row batches, logged table, synchronous_commit=on:

	baseline (umami index set)	+ text_pattern_ops
mean of 3 runs	4,861 ms	7,403 ms

≈ +52% insert time (per-run range +31% to +72%, high variance). On your hottest write path that is a real trade, and it is your call whether the read win justifies it.

Caveats

• Synthetic data on one machine; not production hardware or traffic patterns.
• One website holds 10M of the 12.4M rows, so website_id contributes less selectivity than a real multi-tenant table would — this likely understates the index's benefit.
• Single run per query; cache state not controlled between runs.
• text_pattern_ops serves only case-sensitive LIKE. Funnel and goal step values use like and benefit; the filter-bar operators use ilike and would need a lower(url_path) functional index instead.

Happy to open a separate migration PR for the index, or to leave it entirely — you have production numbers I don't.