Skip to content

docs (v2): the Spans change compiles silently in Go; how a column_defaults value is read #667

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
EricAndrechek merged 2 commits into from
Oct 21, 2026
Merged

docs (v2): the Spans change compiles silently in Go; how a column_defaults value is read #667

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
06dc3ac
docs (v2): the Spans change compiles silently in Go; check each entry
EricAndrechek Oct 21, 2026
30ef4eb
docs (v2): how a column_defaults value is read, with measured examples
EricAndrechek Oct 21, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
14 changes: 14 additions & 0 deletions docs/guides/batches.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -342,6 +342,20 @@ let batch = schema.rows(Format::JsonEachRow, body, &options)?;

</details>

### How a `value` is read

The library reads a `value` with the same reader that reads that column's field in the body: **the body format's own field encoding**, including its quoting and escapes. Measured by a downstream user on build `20261021.025722` (darwin-arm64):

| body format | `value` | result |
| -------------------------- | ------------------- | -------------------------------------------------------------------- |
| CSV | `"a,""b"` | read: the string `a,"b` |
| CSV | `a,b` (unquoted) | refused: ClickHouse error 27 (`CANNOT_PARSE_INPUT_ASSERTION_FAILED`) |
| TSV | `a\tb` (the escape) | read: `a`, a tab, `b` |
| TSV | `a`, a raw tab, `b` | refused: error 27 |
| JSONEachRow, into `UInt32` | `"42"` or `42` | both read as 42 |

A value that doesn't read is refused when the call starts, as a usage error, before any byte of the body is read. So write each `value` exactly as it would appear in a field of the body you send.

## Next

- [`transformations.md`](transformations.md) — what the accepted rows were silently changed into.
Expand Down
44 changes: 22 additions & 22 deletions docs/reference/bindings-v1.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -420,28 +420,28 @@ The transform entry keeps v0's shape (`column`, `input`, `stored`, `reason`, `ro

### `BatchResult`, from the `batch` document and the export buffer

| field | from | type |
| --------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | -------------------------------------------------- |
| `outcome` | `outcome`, final | `Outcome` |
| `err_code`, `err_msg` | `code`, `err` (rule 5) | int32, bytes |
| `reason` | `reason`: the catalog id of a declined batch, present exactly where `code` is `CHS_CODE_DECLINED` | optional ASCII text |
| `refused_at`, `refused_row` | `refused_at` (a `refusal_site` value) and `refused_row`: where a refusal or decline sits, present whenever the outcome is not `accepted` or `accepted_poisoned`; `refused_row` for `row`, below `rows_read` | optional `RefusalSite`, optional unsigned 64-bit |
| `rows` | `rows`, each decoded as a `RowResult` | list of `RowResult` |
| `rows_read` | `rows_read` | unsigned 64-bit |
| `rows_skipped` | `rows_skipped` | unsigned 64-bit |
| `transformed` | `transformed`: every transform in the batch with its `row`, storage transforms (TTL) included, as the library lists them | list of `Transform` |
| `unsupported_settings` | `unsupported_settings`, name objects (rule 5): the call's own settings the library declined (ABI v2) | list of bytes |
| `engine_rows` | `engine_rows`: each stored row after the engine's insert-time merge, a list of cells | optional list of rows, each a list of `EngineCell` |
| `at_merge` | `at_merge`: what an `OPTIMIZE TABLE ... FINAL` of the part would do to its rows (ABI v2) | list of `AtMergeEntry` |
| `payload` | the `out_export` buffer, present only for an export that ran (absent when `export_declined` is non-empty) | optional payload |
| `spans` | `row_spans`: one entry per `rows` entry, index-aligned, each exported row's place in `payload` and null for a row not exported | optional list of optional `Span` |
| `export_columns` | `export_columns`: the export's one column list, name objects (rule 5) | optional list of bytes |
| `export_declined` | `export_declined` (rule 5) | bytes |
| `export_declined_reason` | `export_declined_reason`: the catalog id of a declined export | optional ASCII text |
| `rows_passed`, `rows_cut` | `rows_passed`, `rows_cut`, with an attached filter | unsigned 64-bit |
| `partition_count` | `partition_count` | optional unsigned 64-bit |
| `unconsumed` | `unconsumed`: the byte ranges the reader's error recovery skipped | list of `Span` |
| `framing` | `framing`: what the reader decided about the body's framing | optional `Framing` |
| field | from | type |
| --------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | -------------------------------------------------- |
| `outcome` | `outcome`, final | `Outcome` |
| `err_code`, `err_msg` | `code`, `err` (rule 5) | int32, bytes |
| `reason` | `reason`: the catalog id of a declined batch, present exactly where `code` is `CHS_CODE_DECLINED` | optional ASCII text |
| `refused_at`, `refused_row` | `refused_at` (a `refusal_site` value) and `refused_row`: where a refusal or decline sits, present whenever the outcome is not `accepted` or `accepted_poisoned`; `refused_row` for `row`, below `rows_read` | optional `RefusalSite`, optional unsigned 64-bit |
| `rows` | `rows`, each decoded as a `RowResult` | list of `RowResult` |
| `rows_read` | `rows_read` | unsigned 64-bit |
| `rows_skipped` | `rows_skipped` | unsigned 64-bit |
| `transformed` | `transformed`: every transform in the batch with its `row`, storage transforms (TTL) included, as the library lists them | list of `Transform` |
| `unsupported_settings` | `unsupported_settings`, name objects (rule 5): the call's own settings the library declined (ABI v2) | list of bytes |
| `engine_rows` | `engine_rows`: each stored row after the engine's insert-time merge, a list of cells | optional list of rows, each a list of `EngineCell` |
| `at_merge` | `at_merge`: what an `OPTIMIZE TABLE ... FINAL` of the part would do to its rows (ABI v2) | list of `AtMergeEntry` |
| `payload` | the `out_export` buffer, present only for an export that ran (absent when `export_declined` is non-empty) | optional payload |
| `spans` | `row_spans`: one entry per `rows` entry, index-aligned, each exported row's place in `payload` and null for a row not exported Check each entry before reading it: in Go, Python and TS, reading a field of a nil entry compiles but fails at run time. | optional list of optional `Span` |
| `export_columns` | `export_columns`: the export's one column list, name objects (rule 5) | optional list of bytes |
| `export_declined` | `export_declined` (rule 5) | bytes |
| `export_declined_reason` | `export_declined_reason`: the catalog id of a declined export | optional ASCII text |
| `rows_passed`, `rows_cut` | `rows_passed`, `rows_cut`, with an attached filter | unsigned 64-bit |
| `partition_count` | `partition_count` | optional unsigned 64-bit |
| `unconsumed` | `unconsumed`: the byte ranges the reader's error recovery skipped | list of `Span` |
| `framing` | `framing`: what the reader decided about the body's framing | optional `Framing` |

| type | fields, each from the same-named key unless noted |
| -------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
Expand Down
3 changes: 2 additions & 1 deletion go/CHANGELOG.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -9,6 +9,7 @@ The four bindings in this repository are released together and give one answer,
### Added

- `column_defaults` is covered by unit tests (ABI v2): the exact `insert_options` JSON for a UTF-8 value, a non-UTF-8 value (`value_b64`), an empty value (sent as `""`) and two entries in order, and the decode of a `default_override` cell (`SourceDefaultOverride`, stored). The batches guide gains a usage section. No behavior changed.
- Docs: `docs/guides/batches.md` shows how a `column_defaults` value is read, in the body format's own field encoding (CSV quoting, TSV escapes, a JSON string or number), with a value that doesn't read refused at the call before any body byte.
- `CallError.Reason` (ABI v2): the error's catalog id, read through `chs_error_reason` by the generated layer and passed through verbatim (ASCII, empty if none), on all four call classes.
- A fetch-only call (#492): `(*Registry).Fetch(ctx, request) (Resolved, error)` resolves, fetches, verifies and installs the build a request names into the cache, as `chtypes fetch` does, and opens nothing: no library is loaded and the process setup is untouched, so it needs no `Setup`. It honors the registry's fetch options (offline, the cache, and under the production channel the bases, the trust and the lock), and a concurrent `Fetch` and open of one request share one fetch. The `Resolved` names the installed version, build, digests and library path; errors are the fetch codes, and a done `ctx` is `ctx.Err()`.
- `chtypes resolve <spelling> [--json]` (#493): what a line or version resolves to on every platform the registry offers, one `resolved <version> <platform> <build> <manifest>` line each, or with `--json` one JSON array of `{platform, version, build, manifest}` objects, each signed statement verified exactly as a fetch verifies it and no layer downloaded; `--offline` answers from the cache. The four CLIs print the same bytes, so one parser reads them (a CI cache key, for example).
Expand Down Expand Up @@ -74,7 +75,7 @@ The four bindings in this repository are released together and give one answer,
- **Enforcement gate lifted** for `26.8` on `darwin-arm64`.
- **Enforcement gate lifted** for `26.9` on `darwin-arm64`.
- `docs/limitations.md` is re-checked against artifact build `20261012.142240` (v2-dev.13), the library half measured on that build: functions that need a server, in a column expression, and a body the server may read serially or in parallel under a non-zero error budget are listed under "Declined by design"; the known divergence for a column expression that calls a function needing a server is retired, since that build declines those schemas (on 26.8 and 26.9 the `AI*` and `ai*` functions compile, as the page says); and the `lossy` raw-NUL TSV gap is marked fixed on that build and still present in the 1.0 libraries. `docs/guides/filters.md` gains a section on subcolumn filters and a JSON value typed by its batch.
- **Breaking (ABI v2 dev): `BatchResult.Spans` is `[]*Span`**, one entry per `Rows` entry and index-aligned with it: `row_spans` carries `null` for a row the export did not carry (a skipped row, or a row a filter cut), which the entry reads as nil. It was `[]Span` of the exported rows only.
- **Breaking (ABI v2 dev): `BatchResult.Spans` is `[]*Span`**, one entry per `Rows` entry and index-aligned with it: `row_spans` carries `null` for a row the export did not carry (a skipped row, or a row a filter cut), which the entry reads as nil. It was `[]Span` of the exported rows only. **This change compiles silently:** code written for `[]Span` that reads `r.Spans[i].Off` still builds, because Go dereferences the pointer for you, and it panics at run time on a nil entry. Check each entry for nil before reading it.
- **Breaking (ABI v2 dev): `AtMergeEntry.Row` is `AtMergeEntry.PartRow`**, the document's `part_row`.
- A document's `code` can be `CodeDeclined` (-2) for a row, batch, filter result or verdict this build declined, and every decline names its `Reason`; a `Source` the description does not list reads the description's fallback (`skipped`), not stored.

Expand Down
Loading
Loading