Skip to content

[BUG] spath path=log.level returns NULL for literal flat keys containing dots; documented bracket escape is non-functional #5686

Description

@ahkcs

Describe the bug

spath input=body path=log.level returns NULL when body is JSON whose literal flat key is "log.level" (ECS-flattened logs). spath treats the dot as nested navigation (loglevel) with no way to match a literal dotted key. The documented bracket-escape workaround (path="['log.level']", spath.md Example 4) does not work at runtime.

To Reproduce

Document: {"log.level":"ERROR","message":"boom"} stored in a body field.

source=logs | spath input=body path=log.level | head 5       -> log.level = null
source=logs | spath input=body path=message   | head 5       -> message = "boom"  (works)
source=logs | spath input=body path="['log.level']" | head 5 -> still null (docs say this should work)

Expected behavior

Either the bracket/quoted form should extract the literal key log.level"ERROR", or spath should try a literal-key match. At minimum the documented Example 4 must work.

Root cause

spath (Calcite-only; V2 Analyzer.visitSpath throws getOnlyForCalciteException) desugars to json_extract(body, "log.level"). JsonUtils.convertToJsonPath copies every . verbatim as a JSONPath descent separator, so log.level$.log.level → nested lookup → NULL. convertToJsonPath special-cases only {/}/. — it has no bracket-awareness, so the interior dots in ['log.level'] still split; the bracket escape is effectively dead. It is also never executed in CI (spath.md isn't wired into the doctest runner). The sibling splitter expandJsonPath (split("\\.")) shares the flaw. Blast radius: json_extract/json_set/json_delete/json_append/json_extend all route through convertToJsonPath.

Workarounds (current)

  • rex to extract the dotted key, or
  • spath auto-extract mode (omit path=): JsonExtractAllFunctionImpl reads parser.currentName() as a literal key so {"log.level":...} is preserved (read-back via result.log.level re-splits, so access is still awkward).
  • The bracket syntax is not a working workaround.

Proposed fix

Make convertToJsonPath (and expandJsonPath) bracket-aware: ['log.level']$['log.level'] (Calcite jsonpath supports bracket selectors). This makes the already-documented escape work. Add an executing IT seeding {"log.level":"ERROR"} asserting path="['log.level']""ERROR", and wire spath.md into the doctest runner.

Metadata

Metadata

Assignees

No one assigned

    Labels

    PPLPiped processing languagebugSomething isn't working

    Type

    No type

    Projects

    Status
    Not Started

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions