fix: derive comprehension JSON access from the range, not variable names - #194
Merged
Merged
Conversation
Two hardcoded name lists remained after #193, both in the comprehension path. Removing them leaves none in the converter. isJSONObjectFieldAccess treated field access on a comprehension variable as JSON extraction whenever the variable was named attr, item, element, obj, feature or review. Both directions were wrong. A jsonb[] column iterated as "row" was treated as a composite and produced row.status; an array of a composite type iterated as "item" produced item->>'name' for a real column. The name leaked outside comprehensions too: with a schema declaring name as text, item.name.size() rendered as LENGTH(item->>'name'). A comprehension variable is now bound along with whether its range yields JSON, decided by the element type of the range, and a variable bound that way is treated exactly like one declared through WithJSONVariables for as long as the body is being written. That reuses the existing JSON path machinery rather than duplicating it, which nested access needs: r.metadata.active over a jsonb[] now routes through the path builder to r->'metadata'->>'active', where the name-based branch extracted the first segment as text and then dotted into it, producing r->>'metadata'.active. The binding is scoped like iterVars, since comprehensions nest and an inner one may reuse a name the outer bound to a different kind of collection. isNumericJSONField carried nineteen field names — level, score, price, rating, megapixels, vram, helpful and so on — deciding whether an extracted JSON value was wrapped in ::numeric. visitCall already drives that from the compared type, which is where v3.7.0 left it when it removed the matching heuristic from visitIdent, so the list was redundant. (item->>'price')::numeric > 10 is unchanged, now because 10 is numeric rather than because the field is called price. With the branch gone, visitSelect loses its three-way switch. Two tests pinned the old behaviour and are corrected: a filter over a jsonb[] column expected r.metadata.active for a document, and the issue-85 test expected item->>'name' for a text column. BREAKING: field access on a comprehension variable follows the schema. Iterating a composite array yields column access and iterating a jsonb range yields extraction, whichever the variable is called. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Why
Two hardcoded name lists survived #193, both in the comprehension path. Removing them leaves none in the converter:
$ grep -rn '\[\]string{"' --include='*.go' . | grep -v _test | grep -v /examples/ $isJSONObjectFieldAccess— variable namesField access on a comprehension variable extracted from a JSON document whenever the variable was named
attr,item,element,obj,featureorreview. Both directions were wrong:jsonb[]rowrow.status(composite access on a document)row->>'status'itemitem->>'name'(extraction from a row)item.nameIt leaked outside comprehensions too — with a schema declaring
nameastext,item.name.size()rendered asLENGTH(item->>'name')purely because the variable was calleditem.isNumericJSONField— field namesNineteen field names (
level,score,price,rating,megapixels,vram,helpful, …) decided whether an extracted JSON value got wrapped in::numeric.visitCallalready drives that from the compared type — which is exactly where v3.7.0 left it when it removed the matching heuristic fromvisitIdent. The list was redundant.How
A comprehension variable is now bound together with whether its range yields JSON, decided by the element type: a
jsonb[]column unnests to documents, an array of a composite type unnests to rows.The key move is that such a variable is then treated exactly like one declared through
WithJSONVariables, scoped to the comprehension body. That reuses the existing JSON path machinery instead of duplicating it — which nested access needs:The binding is scoped like
iterVars, because comprehensions nest and an inner one may reuse a name the outer bound to a different kind of collection. There's a test for that.With the object-access branch gone,
visitSelectloses its three-way switch and becomes a straight "JSON path or column".Two tests corrected
Both pinned the old behaviour:
filter with nested field access on JSON— range isjsonb[], soris a documentr.metadata.activer->'metadata'->>'active'TestIssue85_SizeOutsideComprehension—nameis atextcolumnLENGTH(item->>'name')LENGTH(item.name)The second's own comment ("item.name will be treated as a struct field, generating item->>'name' for JSON") described a
textcolumn, so it was already confused.Notably
(item->>'price')::numeric > 10fromdocs/json-support.mdis unchanged — it just gets there becauseitemsis declaredjsonband10is numeric, rather than because the variable is calleditemand the field is calledprice.Tests
New
comprehension_json_vars_test.go: a jsonb range with an unlisted variable name; a composite range with a formerly-listed name; a scalar array; the numeric cast without the name list; nested access; the documentedjsonbarray-value example; all six formerly-listed names outside a comprehension getting no special treatment; and nested comprehensions where the inner rebinds the outer's name to a different kind of range.Docker isn't available in my environment, so the 10
pg/testcontainer tests fail locally — identically onmain, so it's environmental.pg/provider_testcontainer_test.goholds exact-SQL assertions over real comprehension queries, so that job is the gate. Everything else is clean:go build,go vet,golangci-lint(0 issues),go test -short.Breaking
Field access on a comprehension variable follows the schema: iterating a composite array yields column access, iterating a
jsonbrange yields extraction, whichever the variable is called. Callers who relied on a variable name to force JSON treatment should declare the range as JSON inWithSchemas.Pre-existing, not touched
docs/json-support.md:145documentsjsonb_array_elements(order.items)for ajsonbarray value, but the converter emitsUNNEST— identically before and after this change. Separate issue.🤖 Generated with Claude Code