Added fuzzy query translator and serializer to DSL analytics path

[ask-kamal-nayan]

Description

Adds FuzzyQueryTranslator and FuzzySerializer to the dsl-query-executor and analytics-backend-lucene sandbox plugins, enabling DSL fuzzy queries to be translated into delegated predicates and pushed down to Lucene for execution.

What: A translator that converts FuzzyQueryBuilder into a FUZZY RexCall with MAP operands, and a serializer that reconstitutes the FuzzyQueryBuilder on the data node from those operands.

Why: Edit-distance matching has no DataFusion equivalent. The fuzzy query must be pushed down to Lucene as a delegated predicate — Lucene builds a Levenshtein automaton that produces a match bitset directly, rather than attempting row-by-row evaluation in the analytics engine.

Where: Two components, both required for end-to-end operation:

1. FuzzyQueryTranslator in sandbox/plugins/dsl-query-executor — translates the DSL query builder into a RexCall whose operator name is "FUZZY", which resolves to ScalarFunction.FUZZY via name-based dispatch at ScalarFunction.fromSqlOperatorWithFallback().
2. FuzzySerializer in sandbox/plugins/analytics-backend-lucene + registration in QuerySerializerRegistry — reconstitutes the FuzzyQueryBuilder from the RexCall operands so the Lucene backend can execute the fuzzy query at the shard level.

Without the translator, the query wraps in UnresolvedQueryCall and fails at OpenSearchFilterRule. Without the serializer, ScalarFunction.FUZZY resolves but predicate serialization has no handler.

Supported Parameters

Parameter	Legacy Default	Behaviour
value (required)	—	Primary query term, carried as string literal in operand 1
fuzziness	AUTO	Edit-distance model; validated at translate time via Fuzziness.build() + asDistance() to fail fast on invalid values like "abc"
prefix_length	0	Number of leading characters held fixed (not subject to edits); must be ≥ 0
max_expansions	50	Maximum number of terms the fuzzy automaton expands; must be ≥ 1
transpositions	true	Whether adjacent-character swaps count as a single edit (Damerau-Levenshtein when true, plain Levenshtein when false)
rewrite	constant_score	MultiTermQuery rewrite method; passed through because top_terms_* variants change which documents match even in a non-scoring context

Non-default values are emitted as additional MAP operands at index 2+. Default values are omitted — the serializer lets FuzzyQueryBuilder apply its own defaults.

Fuzziness accepts "0", "1", "2", "AUTO", and "AUTO:x,y" (custom auto bounds). Values like "3" are accepted and clamped to 2 by Lucene at query time, preserving legacy parity with StringFieldType.fuzzyQuery (StringFieldType.java:103).

Rejected Parameters

Parameter	Why Rejected	Legacy Citation
boost	Delegated predicates produce a match bitset with no scoring; boost has no effect	AbstractQueryBuilder.toQuery lines 130–136: BoostQuery wrapping
_name	Query naming is diagnostic metadata with no mechanism to carry through the analytics pipeline	AbstractQueryBuilder.toQuery lines 137–139: named query registration

Known Divergences from Legacy _search

#	Behaviour	Legacy _search	Analytics DSL Path	Reason
1	boost	Affects BM25 score ranking	Rejected with ConversionException	Delegated predicate yields match bitset, no scoring (DelegatedPredicateFunction.java:23-33)
2	_name	Stored in response for diagnostics	Rejected with ConversionException	No mechanism in analytics pipeline
3	Field-type gating	Rejects at field-mapper level with type-specific errors	Rejects all non-VARCHAR with a generic error	Calcite type system collapses keyword/text/match_only_text into VARCHAR; no field-mapper access at translate time
4	Unknown field	Returns MatchNoneQueryBuilder (FuzzyQueryBuilder.doRewrite line 347)	Throws ConversionException	ConversionContext.getField throws if field absent from schema (ConversionContext.java:102-107)
5	Non-string value	Accepts int/long/float/boolean via Object constructor	Accepts only string (value toString'd at translate time)	Cosmetic — FuzzyQueryBuilder toString's value before Lucene processing regardless
6	search.allow_expensive_queries	When false, fuzzy queries are refused at query-build time (StringFieldType.fuzzyQuery:92, KeywordFieldMapper.KeywordFieldType.fuzzyQuery:736); the setting is declared at SearchService.java:221 (default true, dynamic) and delivered as a BooleanSupplier via QueryShardContext.allowExpensiveQueries() (QueryShardContext.java:373)	Not honoured — LuceneAnalyticsBackendPlugin.java:282 passes a hardcoded always-true supplier	Pre-existing engine-wide gap affecting every delegated predicate, not introduced by fuzzy. Fix belongs in a separate family-wide change that threads the cluster setting through the analytics backend context.
7	_name handling across family	term, terms, range, and fuzzy all reject _name with a ConversionException; prefix and wildcard silently ignore it (with an explanatory comment)	Rejects with ConversionException (consistent with term/terms/range)	Fuzzy sits with the majority; prefix/wildcard are the outliers that silently drop it. The family should converge on one policy — tracked as cross-cutting work, not resolved here.
8	flat_object / version field types	Support fuzzy natively (each type implements fuzzyQuery())	Unreachable — OpenSearchSchemaBuilder.mapFieldType() returns null for these types, excluding them from the Calcite schema entirely	Storage/schema-layer limitation affecting all query types, not fuzzy-specific. These fields are invisible to the analytics engine regardless of predicate.

Testing

Category	File	Tests
Translator unit tests	FuzzyQueryTranslatorTests.java	18
Serializer round-trip tests	QuerySerializerRegistryTests.java (additions)	7
Golden files	fuzzy_default_params.json, fuzzy_all_params.json	2 plan-shape assertions
Integration tests	DslQueryIT.java	2 (@AwaitsFix)
Module totals (all passing)	dsl-query-executor / analytics-backend-lucene	164 / 323

Translator tests cover: rejection of non-default boost, _name, non-VARCHAR fields, unknown fields, invalid fuzziness values (including non-numeric like "abc"), negative prefix_length, zero max_expansions, null value, empty-string value; acceptance of AUTO:4,7 custom auto bounds and fuzziness "3" (clamped); correct operator name, field/query operand shape, non-default param emission, and default-param omission.

Serializer tests cover: defaults round-trip, custom fuzziness, all-params round-trip, rewrite pass-through, transpositions=false isolation, missing-field error, and unrecognized-param tolerance.

Golden files assert the full Calcite logical plan shape for a fuzzy query with default parameters and with all five optional parameters overridden.

Integration tests (testFuzzyQueryOnKeywordField, testFuzzyQueryOnTextField) are parked with @AwaitsFix pending the analytics E2E pipeline (fragment conversion + shard execution + Arrow Flight drain), matching sibling ITs for range, prefix, wildcard, and bool queries.

Check List

• Functionality includes testing
• API changes companion pull request created, if applicable
• Public documentation issue/PR created, if applicable

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

[github-actions]

PR Reviewer Guide 🔍

Here are some key observations to aid the review process:

🧪 PR contains tests

🔒 No security concerns identified

✅ No TODO sections

🔀 No multiple PR themes

⚡ Recommended focus areas for review Unrelated Registration This PR is described as adding fuzzy support, but the same diff also adds Map.entry(ScalarFunction.SARG_PREDICATE, new SargSerializer()) to the registry. If SargSerializer was previously registered elsewhere, this could produce a duplicate-key error at map construction (Map.ofEntries throws on duplicate keys). If it wasn't registered, then a new, unrelated feature is being introduced without accompanying tests or description. Please verify this line is intended for this PR and does not conflict with an existing registration. Map.entry(ScalarFunction.SARG_PREDICATE, new SargSerializer()), Map.entry(ScalarFunction.FUZZY, new FuzzySerializer()) Possible NPE fuzzyQuery.fuzziness() is dereferenced via fuzziness.asString() and fuzziness.equals(...) without a null check. If a FuzzyQueryBuilder is constructed with fuzziness(null) (the setter appears to allow null in some paths), this will throw a NullPointerException before the intended ConversionException validation path runs. Consider guarding for null or documenting the assumption. Fuzziness fuzziness = fuzzyQuery.fuzziness(); String fuzzinessStr = fuzziness.asString(); if (!fuzziness.equals(FuzzyQueryBuilder.DEFAULT_FUZZINESS)) { operands.add(makeParamMap(ctx, "fuzziness", fuzzinessStr)); }

[github-actions]

PR Code Suggestions ✨

Explore these optional code suggestions:

Category	Suggestion	Impact
Possible issue	Use BytesRefs.toString for safe value extraction Using valueObj.toString() on non-String types (e.g., BytesRef, Number, Boolean) may not produce a serialization-safe or semantically correct string for the downstream fuzzy query. For example, BytesRef.toString() returns a debug representation like [6c 61 70 74 6f 70] rather than the UTF-8 string. Consider using BytesRefs.toString(valueObj) or explicitly handling supported types to ensure correct value extraction. sandbox/plugins/dsl-query-executor/src/main/java/org/opensearch/dsl/query/FuzzyQueryTranslator.java [117-125] // Extract and validate payload String fieldName = fuzzyQuery.fieldName(); Object valueObj = fuzzyQuery.value(); if (valueObj == null) { throw new ConversionException("Fuzzy query value must not be null"); } -String value = valueObj.toString(); -if (value.isEmpty()) { +String value = org.opensearch.common.lucene.BytesRefs.toString(valueObj); +if (value == null || value.isEmpty()) { throw new ConversionException("Fuzzy query value must not be empty"); } Suggestion importance[1-10]: 6 __ Why: Valid concern: BytesRef.toString() returns a debug hex-like representation, which would produce incorrect fuzzy query values if a non-String value is stored. Using BytesRefs.toString provides safer conversion.	Low
	Guard against null fuzziness before dereferencing fuzziness may be null if never set on FuzzyQueryBuilder (depending on builder defaults), which would cause a NullPointerException when calling asString() or equals(). Guard against null before dereferencing to prevent an NPE for queries that omit fuzziness entirely. sandbox/plugins/dsl-query-executor/src/main/java/org/opensearch/dsl/query/FuzzyQueryTranslator.java [79-83] // Optional params — emit only non-defaults Fuzziness fuzziness = fuzzyQuery.fuzziness(); -String fuzzinessStr = fuzziness.asString(); -if (!fuzziness.equals(FuzzyQueryBuilder.DEFAULT_FUZZINESS)) { - operands.add(makeParamMap(ctx, "fuzziness", fuzzinessStr)); +if (fuzziness != null && !fuzziness.equals(FuzzyQueryBuilder.DEFAULT_FUZZINESS)) { + operands.add(makeParamMap(ctx, "fuzziness", fuzziness.asString())); } Suggestion importance[1-10]: 4 __ Why: FuzzyQueryBuilder.fuzziness() typically returns DEFAULT_FUZZINESS (AUTO) rather than null, so this is a defensive change with limited actual impact, but a small correctness safeguard.	Low
General	Reject unrecognized parameters instead of ignoring Silently ignoring unrecognized params can mask misspelled or unsupported options (e.g., "fuzzines" typo), producing queries that don't behave as the user expects with no error signal. Consider throwing IllegalArgumentException on unknown params, or at minimum logging a warning, to match the strict validation applied to individual param values. sandbox/plugins/analytics-backend-lucene/src/main/java/org/opensearch/be/lucene/serializers/FuzzySerializer.java [42-46] case "transpositions" -> fuzzyQb.transpositions(parseStrictBoolean("transpositions", entry.getValue())); case "rewrite" -> fuzzyQb.rewrite(entry.getValue()); -default -> { - /* ignore unrecognized params for forward compatibility */ } +default -> throw new IllegalArgumentException(functionName() + " unrecognized parameter: " + entry.getKey()); Suggestion importance[1-10]: 4 __ Why: The comment explicitly documents that unknown params are ignored for forward compatibility, so this suggestion contradicts the intentional design choice, though it does raise a valid usability concern.	Low

[github-actions]

❌ Gradle check result for bb30c10: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?