[nick-boss-tech]

🤖 AI text below 🤖 (reviewed by Nick Shanin)

https://issues.apache.org/jira/browse/SOLR-8003

What happens today

A glob cannot take a raw value transformer, so fl=*_json:[json] returns a 400.

The parser accepts a glob only when a space, a comma or the end of the string follows it (lines 305-316 at the merge base). A raw transformer works on one named field, so every _json field must be named.

What this change does

A glob followed by a raw value transformer writes each matching field raw.

fl=*_json:[json] and fl=su*:[json] are accepted. Each returned field whose name matches is written raw under its own name (lines 318-347). A bare * before a raw transformer returns a 400 (lines 320-325). The JSON and XML writers check the mark for each field (lines 122-135).

This is the follow-up to #5188, which adds the ReturnFields methods the tests for this change use. It is built on main, so it does not depend on #5188 to compile. Three ReturnFields unit tests that use those methods are left for after #5188 merges. Merge #5188 first. This PR then applies with one conflict, in TestExportWriter.java, where both PRs add tests at the end of the file. The production code has no conflicts in either order.

Proof

The raw-glob cases pass with this change and fail on main.

• Verified 2026-10-23 at a24414bf3d9: TestRawTransformer 17/17, TestExportWriter 22/22.
• On main (b73f03976fa), 15 of 17 TestRawTransformer cases and 2 of 22 TestExportWriter cases fail, each with a 400 from the parser. The cases that pass on main use no glob.
• Four planted breaks, run on the combined tree earlier: ignoring globs, not registering the raw glob, ignoring the mark in the writer, and erroring on a glob match in export. Each fails tests.

A choice to check

A renamed field under a raw glob comes back plain.

In fl=p:payload_json,*_json:[json], the field is returned as p, which does not match *_json. The guide matches globs against returned names (line 222). p:[json f=payload_json] already writes a renamed field raw (test).

• A (implemented): a glob matches the returned name. A renamed field is plain unless it is named with f=.
• B: a glob also matches the source name, so p:payload_json under *_json:[json] is raw. The output would depend on a name the response no longer returns.

Decision ladder:

1. Does a glob describe names in the response? Yes: A. No: B.
2. If B: should a user match a field by a name it no longer has? Yes: B. No: A.

Recommendation: A. The guide describes globs by returned names, and p:[json f=payload_json] covers the renamed case.

Was this the right call?

Limits

• /export ignores the raw mark. Its output with a raw glob matches the plain glob.
• A raw glob takes effect only in the JSON and XML writers, and only when the transformer's writer matches the response's. *_json:[xml] on wt=json returns plain values.
• Performance: the writer checks each raw glob against each field of each document. The glob matcher splits the pattern on every call, the same cost the existing fl globs already have. Caching the split is a possible follow-up.
• Not covered: nested child documents, lazy field loading, /get, highlighting, term vectors.
• The three ReturnFields unit tests for SOLR-11364: fl= fix some minor issues, mostly with globs #5188's methods follow once SOLR-11364: fl= fix some minor issues, mostly with globs #5188 merges.

Changelog: changelog/unreleased/SOLR-8003-fl-glob-raw-transformer.yml

Ref Guide: document-transformers.adoc, the [json] / [xml] section, documents the glob form and its limits (commit 874766b7625, docs only; the code proof above is at a24414bf3d9).

AI-generated description. Nick Shanin directed the work and takes responsibility for this contribution.

[nick-boss-tech]

🤖 AI text below 🤖 (reviewed by Nick Shanin)

@dsmiley, this is the follow-up to #5188. Please review the raw-glob change in SolrReturnFields and the writer check first.

Recommendation: review the raw-glob path first. It is the part that changes behavior.

• Merge order: SOLR-11364: fl= fix some minor issues, mostly with globs #5188 first. This PR then applies with one conflict, in TestExportWriter.java, where both PRs add tests at the end of the file. The production code has no conflicts in either order.
• This PR does not include your ReturnFields methods. Three tests in ReturnFieldsTest use wantsLuceneField and isExplicitLuceneField, so they are left out here. They come in after SOLR-11364: fl= fix some minor issues, mostly with globs #5188 merges.
• The base fails 15 of 17 TestRawTransformer cases and 2 of 22 TestExportWriter cases. Main plus this change passes them all (body).
• One choice needs your view: a renamed field under a raw glob comes back plain. The body explains the two options and the recommendation.

Bottom line: if the raw-glob design looks right, the rest is ready for review.

[dsmiley]

@nick-boss-tech Has anyone requested this in the last 5 years and/or re-communicated the need? If not, please don't invest in a solution to an old feature-request without that confirmation.

[nick-boss-tech]

Closing for lack of interest.