[dsmiley]

Code that decides which index fields to load (SolrDocumentFetcher, DocsStreamer, ExportWriter) had to piece the answer together from getLuceneFieldNames(), wantsField() and hasPatternMatching(). That went wrong in a few cases, mostly when fl mixes globs with explicitly named fields, aliases or transformers. ReturnFields now answers those questions directly:

• wantsLuceneField(field): whether an index field's value is needed, either to return it or for a transformer. False for a name that belongs only to an alias or transformer, unless * was requested.
• isExplicitLuceneField(field): whether the field was named explicitly rather than only matched by a glob or *.

SolrReturnFields now keeps its explicitly named fields when globs are present. getLuceneFieldNames() still returns null in that case, as before.

Effects:

• Efficiency: with globs, fields named only by an alias or transformer, and loaded fields nothing asked for, are no longer loaded or copied into the document.
• /select fix: an explicitly named useDocValuesAsStored=false field next to a glob (e.g. fl=val_*,_version_) was silently dropped and is now returned. The ref guide says such fields "can still be explicitly requested by name in the fl param, but will not match glob patterns" (see SOLR-10437).
• /select fix: a field a transformer reads (e.g. [json f=…], [geo f=…]) wasn't loaded when fl also had a glob, and now is.
• /export fix: a transformer or alias named like an index field (e.g. body_t:[hl]) no longer makes /export fetch that field, which failed when the field lacked docValues.
• /export behavior change: an explicitly named field that can't be exported now errors even next to a glob or *, instead of being silently skipped.

🤖 Generated with Claude Code

[dsmiley]

Journey:

• SOLR-9957: /export supports document transformers in fl #5113 (/export support doc transformers ): merged
• SOLR-3479: New [hl] document transformer for inline highlighting #5126 ([hl]) Tried with /export and it worked one way but not other, and there was a double-load perf issue... so I dug deeper...
• Considered fixing the newly committed /export to be a bit better. But it was gnawing at me that most users of ReturnFields are very clumsy/confusing, checking certain sets and null checks in a complicated dance. So I set out to improve ReturnFields generally...
• Discovered the improvements had also fixed some minor bugs (listed in the PR description) aside from Export.

I could file a JIRA but I feel the changes are minor. I should create a changelog.

[dsmiley]

bug fixes

[nick-boss-tech]

I checked on the CI failure here and it looks like it just needs a rerun, #5187 fixed it but CI ran before it merged.

[epugh]

retriggered

[kotman12]

Just reading wantsLuceneField it is hard to see how that encodes: "index field to return or for use by transformer". wantsField OTOH is just about what gets returned. So the difference between these two is just the word Lucene which misses the important part which is these fields need to be read/loaded for further use even if they aren't returned. Perhaps usesLuceneField? My preference actually would be to drop Lucene altogether, i.e.uses(Index|Real|Source)Field but I can see Lucene is already used in SolrReturnFields so this is good for consistency.

[dsmiley]

I proposes use of the word "loads". So loadsLuceneField or perhaps just loadsField as I'd be happy to drop the word "Lucene" as well.

wantsField ought to be renamed to returnsField with an important note in the javadocs that the name returned might be a synthetic label sourced by anything (same field, another field, a transformer output computed from anything). We can do this leaving the current name deprecated.

[kotman12]

loadsField is fine, I suppose it is implied if it is getting "loaded" that it is from somewhere, i.e. the index. I agree that returnsField is much better.

[nick-boss-tech]

Quick Review:
The changes consistently distinguish fields that must be loaded from synthetic output names, while preserving transformer inputs and explicit docValues requests alongside globs.
The existing API-naming discussion remains open. Optional improvements are additional alias/transformer docValues coverage and mentioning the stricter /export validation in the changelog.

[dsmiley]

I went further with the renames, retaining the original public methods as deprecated.