Skip to content

v2 spec: an expression the INSERT evaluates never sees the schema's own settings - #632

Merged
EricAndrechek merged 6 commits into
v2from
v2-insert-eval-settings
Oct 21, 2026
Merged

EricAndrechek merged 6 commits into
v2from
v2-insert-eval-settings

Conversation

@EricAndrechek

@EricAndrechek EricAndrechek commented Oct 12, 2026 •

Copy link
Copy Markdown
Member

At the INSERT, the schema's own settings now reach only the parse. The library already works this way: the artifact producer reports (measured) that the published build 20261021.025722, the one SDK 2.0.0-dev.11 binds, runs an INSERT on an ABI v2 schema under the INSERT call's layers, with the schema's own settings taking part only in the parse, minus the type gates. This PR brings the spec in line with the library. The text only changes prose, so the fingerprint doesn't move.

What

spec/abi-v2/docs.md:

  • "Settings." (the layering): at the INSERT, the schema's own settings reach only the parse, just as a server's CREATE-time settings never reach its INSERT.
    • The parse class keeps all five layers. These are the settings ClickHouse's own FormatFactory::getFormatSettings reads (input_format_*, date_time_input_format, input_format_allow_errors_*, …). The one exception is a type gate among them (Nullable(Tuple)'s, for one), which binds the CREATE only.
    • Everything else the INSERT does runs under the call's own settings, the server profile and chs_set_defaults only, over the build's own. Every type gate stays at the build's default unless one of those three sets it. That covers:
      • every expression it evaluates: a DEFAULT or MATERIALIZED fill, a CHECK, a TTL, a partition key the server rebuilds at INSERT (such as one with a modulo), a projection, a Values body's expressions, and the creator of any aggregate function in them;
      • its limits, max_partitions_per_insert_block and throw_on_max_partitions_per_insert_block among them.
    • The sorting key and the skip indices run under the server profile and chs_set_defaults only, because a server builds them under its global context. An aggregate function's creator there still reads the call's settings.
    • The consequences:
      • A gated CAST or a gated aggregate function in such an expression (a Values body's included) is refused as the server refuses it (455, 44 or 63) when its gate is set only in the schema's own settings.
      • A partition limit set at the schema level doesn't reach the INSERT.
      • A caller whose real INSERT session carries such a setting passes it in the call's settings or the server profile.
    • Measured by the artifact producer, against the server and the library:
      • FORMAT Values (CAST(1 AS LowCardinality(UInt32))), with the gate only in the schema's settings, is refused with 455;
      • a 2-partition body under a schema-level limit of 1 is stored;
      • a 101-partition body under a schema-level throw_on_max_partitions_per_insert_block=0 is refused with 252, at the server's default limit of 100.
    • Unchanged: session_timezone keeps the zone path.
  • chs_filter_create, chs_filter_eval_body and chs_block_create: an evaluation's settings are the INSERT session's. They govern parsing the body and the expressions the INSERT evaluates, and still never reach the WHERE.

Merged with v2 (a merge commit, no rebase). Four hunks in docs.md conflicted. Each resolution keeps v2's text and applies this PR's change on top of it:

  • Settings: v2's object form for bytes.
  • chs_filter_create: v2's documented-pathological decline case.
  • chs_filter_eval_body: columns, refused_at and options.
  • chs_block_create: options and the recorded refusal.

include/v2/chtypes.h and docs/reference/abi-v2.md were regenerated with gen.py --major 2 --write, not hand-resolved.

Other text brought into line with the new layering:

  • docs/guides/settings.md has a new section, "The compile profile at the INSERT (ABI v2)". It also now says an evaluation's settings are the INSERT's, never the WHERE's.
  • docs/guides/filters.md: an evaluation's settings also govern the INSERT's expressions.
  • docs/guides/batches.md: on ABI v2, the partition limit comes from the call's settings, the server profile and the setup defaults, never the compile profile.
  • docs/reference/rust.md, and the Filter.rows doc comments in all four bindings: "parse settings only" becomes "INSERT settings". Only comments change, no code.
  • The four CHANGELOGs each get one bullet under Changed.

Measured

  • Fingerprint: unchanged, sha256:d53239933966e15a3c363e631a94fbba1231d84c4300337115d3ae5377bb41c2 both on v2 before the merge and on this head.
  • Gates, each exit 0:
    • gen.py --major 2 --check;
    • check-no-hand-decls.py --scope all;
    • markdownlint-cli2@0.18.1 and dprint@0.57.4 check on the changed Markdown;
    • lint-spelling and lint-cited-paths;
    • ruff check on the edited Python file, and gofmt -l on the edited Go file;
    • lint-public, run last.

Also: a server profile shapes the compile at create

Two sentences in spec/abi-v2/docs.md said a server's settings never change how a schema compiles: one in the Settings preamble, and one in chs_server_create. Since build 20261021.025722, the library runs the CREATE at posture create in a session carrying chs_set_defaults' settings and then the server profile, both under the schema's own settings. So every profile setting reaches the CREATE, flatten_nested, the type gates and the aggregate gates among them. At posture load, the stored definition is taken as written, as a server attaches it at startup. That was measured by the artifact producer, and by the release gate on the published build. The Settings sentence now says so, and chs_server_create keeps its rule (the setting is listed in filter_declined_settings at layer server) without the false reason. The fingerprint is unchanged (sha256:d53239933966…41c2 before and after), and the header diff is comment lines only.

🤖 Generated with Claude Code

EricAndrechek and others added 3 commits October 12, 2026 18:50
…wn settings

A DEFAULT or MATERIALIZED fill, a CHECK, a TTL, a partition key the server
rebuilds at INSERT, a projection and any aggregate function's creator in them
run under the call's own settings, the server profile and chs_set_defaults
only, with type gates at the build's defaults unless one of those sets them;
the sorting key and skip indices under the server profile and
chs_set_defaults only. A server's CREATE-time settings never reach its INSERT,
so a gated CAST or aggregate function there, gated only in the schema's own
settings, is refused as the server refuses it (455, 44, 63). Parsing keeps all
five layers, and session_timezone keeps the zone path. An evaluation's
settings now say they govern the INSERT's own expressions too.

Prose only: the fingerprint is unchanged.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Whether the expressions a Values body evaluates while it is read keep the
schema's own settings is still being measured against the server, so the
Settings paragraph no longer says either way. Parsing keeps all five layers.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Measured against the server: a Values body's CAST does not get the schema's
type gates (455), and a schema-level partition limit does not reach the
INSERT. So only the settings FormatFactory::getFormatSettings reads keep the
schema's layer, a type gate among them excepted; every INSERT-time
evaluation and every INSERT limit runs under the call's settings, the server
profile and chs_set_defaults.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
@EricAndrechek

Copy link
Copy Markdown
Member Author

Landing checklist (handoff 2026-10-13; #633 item 2):

  1. Trigger: the artifact producer reports that the dev build which makes INSERT-time evaluation stop reading the schema's own settings is on staging (its batch 17). Until then, the published library still layers the schema's settings under those expressions, so this stays a draft.

  2. If v2 spec: rule r8 — no input crashes the process (lands with the next spec batch) #630 landed first: merge v2 in, then run scripts/abi-v1/gen.py --major 2 --write and --check. Both PRs move include/v2/chtypes.h. The fingerprint stays d60a681e….

  3. Merge on the maintainers' clearance, once the required checks are green.

  4. The consumer notice for that build names the behaviour change:

    • a gated CAST or aggregate function in a DEFAULT, MATERIALIZED, CHECK or Values body, gated only in the schema's settings, is refused (455/44/63);
    • a schema-level max_partitions_per_insert_block / throw_on_max_partitions_per_insert_block no longer reaches the INSERT.

    A caller whose real INSERT session carries such a setting passes it in the call's settings or the server profile.

Brings #632 up to v2 (87 commits behind, merge base 28af53d). The library
already layers INSERT-time settings this way since artifact build
20261021.025722, so the spec catches up with it.

spec/abi-v2/docs.md, four conflicting hunks, resolved by taking v2's text
and then applying #632's change to it:
- Settings (preamble): kept v2's object form for bytes, and inserted #632's
  INSERT-time layering after the type-gate exception. At the INSERT, the
  schema's own settings reach only the parse.
- chs_filter_create: kept v2's documented-pathological decline case. An
  evaluation's settings now govern parsing the body and the expressions
  the INSERT itself evaluates.
- chs_filter_eval_body: kept v2's columns, refused_at and options. The
  settings are the INSERT session's: the parse and the INSERT's expressions.
- chs_block_create: kept v2's options and the recorded refusal. Same
  settings sentence as chs_filter_eval_body.

include/v2/chtypes.h and docs/reference/abi-v2.md are regenerated with
gen.py --major 2 --write and were not hand-resolved. The fingerprint does
not move: sha256:d53239933966e15a3c363e631a94fbba1231d84c4300337115d3ae5377bb41c2
on origin/v2 and on this merge.

Other v2 text brought into line with the new layering:
- docs/guides/settings.md: new section "The compile profile at the INSERT
  (ABI v2)". An evaluation's settings are the INSERT's, never the WHERE's.
- docs/guides/filters.md: an evaluation's settings also govern the
  INSERT's expressions.
- docs/guides/batches.md: on ABI v2, the partition limit comes from the
  call, the server profile and the setup defaults, never the compile
  profile.
- docs/reference/rust.md and the Filter.rows doc comments in all four
  bindings: "parse settings only" now reads "INSERT settings" (comments
  only, no code change).
- One CHANGELOG bullet under Changed, the same in all four bindings.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@EricAndrechek EricAndrechek changed the title v2 spec: an expression the INSERT evaluates never sees the schema's own settings (lands with the library build) v2 spec: an expression the INSERT evaluates never sees the schema's own settings Oct 21, 2026
@EricAndrechek
EricAndrechek marked this pull request as ready for review October 21, 2026 07:34
EricAndrechek and others added 2 commits October 21, 2026 03:38
The spec said a server's settings "never change how a schema compiles"
and that chs_schema_create succeeds on a server "because a profile never
changes a schema compile". Since build 20261021.025722 the library runs
the CREATE at posture create in a session carrying chs_set_defaults'
settings and then the server profile, both under the schema's own, so
every profile setting reaches the CREATE: flatten_nested, the type
gates and the aggregate gates among them. At posture load the stored
definition is taken as written, as a server attaches it at startup.
Both sentences now say so; the second keeps its rule (the setting is
listed in filter_declined_settings at layer server) without the false
reason. Measured by the artifact producer in that change's proof and
in the release gate on the published build. Prose only: regenerated,
header diff is comment lines only, ABI v2 fingerprint unchanged.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
No conflicts; gen.py --major 2 --check 0; ABI v2 fingerprint unchanged
(sha256:d53239933966...41c2).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@EricAndrechek
EricAndrechek merged commit ab701e7 into v2 Oct 21, 2026
54 checks passed
@EricAndrechek
EricAndrechek deleted the v2-insert-eval-settings branch October 21, 2026 08:28
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant