[nick-boss-tech]

🤖 AI text below 🤖 (posted on behalf of Nick Shanin)

Ready for review, but held as a draft due to the project's open-PR cap; I'll mark it open on request.

https://issues.apache.org/jira/browse/SOLR-16655

What happens today

Field mutating update processors skip child documents, so child fields keep the values they were sent with.

These processors change only the fields of the root document. Child documents are skipped, both anonymous children and child documents given as a field value. So a trim, regex replace, remove-blank or date parse step does not run on a child's field.

What this change does

Child documents are walked at any depth, whether they arrive as a field value or anonymously; the selector decides which fields change inside each document it reaches.

• Each child document in a field value is walked at any depth, under selected and unselected parent fields alike (field-value children). The selector still decides which of the child's own fields change.
• Anonymous child documents are walked at any depth (anonymous walk).
• A document reachable by two paths is changed once, since not every mutation is safe to repeat (identity check).
• Upgrade note: after upgrading, pipelines that send child documents will see these processors change child field values. Examples are trim, regex replace, remove-blank and date parsing. Check the chains that send child documents before upgrading.

Proof

The focused tests pass at this head, and the two tests that pin the new traversal fail on the production code it replaced.

• At head ea300ef237c: FieldMutatingUpdateProcessorTest 37 of 37, ParsingFieldUpdateProcessorsTest 44 of 44, TestNestedUpdateProcessor 9 of 9 and MinimalSchemaTest 3 of 3 pass. The module checks pass.
• With only FieldMutatingUpdateProcessor.java restored to the previous head's version (5e2317443f41) and this head's tests kept, both pinning tests fail: testTrimFieldsRecurseIntoUnselectedParentFields fails because the child's selected field stays padded under the unselected parent, and TestNestedUpdateProcessor.testParseSelectedDatesInNestedDocuments fails on the child's unparsed date with an invalid-date error.
• FieldMutatingUpdateProcessorTest testTrimFieldsRecurseIntoUnselectedParentFields pins the new rule. A child under an unselected field is walked, and its selected fields are trimmed.
• Other nested tests in that class, such as testTrimAllRecurseIntoNestedDocuments, check changed values inside child documents, and testMutatingProcessorsVisitSharedNestedDocumentsOnce and testRemovedChildFieldIsNotTraversed cover shared child instances and removed fields.
• ParsingFieldUpdateProcessorsTest testParseDateNestedLabelledChildDocuments checks a parsed date inside a child given as a field value.
• TestNestedUpdateProcessor sends update requests through a chain, for labelled and anonymous children, and checks the indexed documents.

A choice to check

The other route keeps the previous rule for labelled children: changed only under a field the selector names.

An earlier version of this change walked a labelled child only when its parent field was selected, and left the child alone otherwise. This branch walks every surviving child document instead, so a selector that names a field inside children applies wherever that field appears. The upgrade note above then covers children under unselected fields too.

Was walking child documents under unselected parent fields the right semantics?

Limits

SOLR-16655 lands first, and SOLR-12705 rebases onto it.

• SOLR-12705 also changes FieldMutatingUpdateProcessor.java. This change adds the child-document descent and the identity check to mutateDocument, and moves the field loop there from processAdd with no change to what the loop does. SOLR-12705 is the broader atomic operands rework in the same file. So this change lands first, and SOLR-12705 rebases onto it. A merge of the two heads conflicts in that file, and the combined tree has not been run. Each change was tested at its own head. A follow-up submission is planned to check the combined tree after SOLR-12705 rebases.
• Most tests run the processor on one document in memory. TestNestedUpdateProcessor is the exception: it sends update requests and checks what gets indexed.

Changelog: changelog/unreleased/SOLR-16655-field-mutating-urps-nested-docs.yml

AI assistance

AI agents assisted with research, implementation, review, and drafting. Nick Shanin directed the work and takes responsibility for this contribution.