Skip to content

Rust #325: narrow visibility queries in the database - #897

Merged
rmcdaniel merged 1 commit into
mainfrom
rust-325-visibility-sql
Nov 1, 2026
Merged

rmcdaniel merged 1 commit into
mainfrom
rust-325-visibility-sql

Conversation

@rmcdaniel

Copy link
Copy Markdown
Member

Part of #325.

Problem

A structured list query used to read too much. It read every queried attribute row in the namespace. It then scanned the namespace's runs in list order, 500 at a time, deciding each in native code until it had the page. A selective query in a large namespace therefore read the whole namespace. Published PHP 2.11.7 applies the query in SQL.

Change

  • Compiled::narrowing. It gives, per OR group, conditions every run matching that group satisfies:

    • status and status bucket;
    • run type, workflow ID, run ID and queue equality or IN;
    • a present attribute row for each queried key;
    • keyword equality or IN;
    • integer comparisons and IN.

    The list statement adds them as one condition, with every value bound.

  • Text comparisons. These run under the column's own collation. On MySQL that is the schema's utf8mb4_unicode_ci, the collation Rust #325: compare visibility query text as each backend does #890's comparison keys follow, so a pair the matcher treats as equal is equal in the database too.

  • Presence only. Comparisons whose database form could differ narrow only to the attribute being present: times, floats, booleans, keyword lists, long text and text inequality. A run-time comparison doesn't narrow at all. A group with nothing to narrow leaves the query unnarrowed, as before.

  • Answers unchanged. The matcher still decides every candidate, so the database can only leave out runs the query does not select.

  • Attribute rows. These are now read per candidate page, for that page's runs only.

Checks

  • New unit test. a_selective_query_reads_its_candidates_not_the_whole_namespace uses 650 runs, more than one scan page. Keyword, keyword-and-integer and integer IN queries each read exactly their matches. A time-only query still reads one scan page.
  • Narrowing test. narrowing_holds_for_every_match_and_stays_open_when_a_group_cannot_narrow pins which checks narrow.
  • Fixtures (local, SQLite). I recorded search-attributes, start-search-attributes and visibility-collation against published 2.11.7 and against native built from this branch. compare accepts each pair. CI covers PostgreSQL, MySQL and visibility-current-runs.
  • Rust suites. Library 369 and integration 647, all passing.
  • Node unit suite. 6459 pass. The only failure is the known local PhpImageContextTest.

Checkpoint.

🤖 Generated with Claude Code

A structured list query read every queried attribute row in the
namespace and scanned its runs 500 at a time until it had the page, so a
selective query in a large namespace read the whole namespace. The list
statement now adds a narrowing every matching run satisfies: status, run
text columns, attribute presence, keyword equality and integer
comparisons, with text under the column's own collation. The matcher
still decides each candidate, with only that page's attribute rows, so
the answers are unchanged.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@rmcdaniel

Copy link
Copy Markdown
Member Author

Security read-only review of exact head 36276aa: no additional tenant-boundary or SQL-injection finding. Every request-derived namespace, filter, search-attribute key, literal, page limit and offset is bound; the only dynamically selected columns and integer operators come from closed internal variants; candidate reads retain both run and workflow-instance namespace predicates; search attributes are reached through workflow_runs.id, which is a primary key in every backend schema; and the existing compiled matcher remains the final inclusion decision. This is a source-design review only, not a substitute for the owner’s differential fixtures or current CI, and it adds no separate Security hold.

@sonarqubecloud

sonarqubecloud Bot commented Nov 1, 2026

Copy link
Copy Markdown

@rmcdaniel
rmcdaniel added this pull request to the merge queue Nov 1, 2026
Merged via the queue into main with commit 4dcca8d Nov 1, 2026
58 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant