Skip to content

fix(tagLabel): lenient appliedAt date deserialization#27771

Merged
Khairajani merged 15 commits intomainfrom
fix_taglabel_appliedat_lenient_parse
May 5, 2026
Merged

fix(tagLabel): lenient appliedAt date deserialization#27771
Khairajani merged 15 commits intomainfrom
fix_taglabel_appliedat_lenient_parse

Conversation

@Khairajani
Copy link
Copy Markdown
Contributor

@Khairajani Khairajani commented Apr 27, 2026

Summary

TagLabel.appliedAt is typed java.util.Date and the global Jackson ObjectMapper is configured with SimpleDateFormat("yyyy-MM-dd'T'HH:mm:ss.SSSSSS'Z'") — which strictly requires 6-digit fractional seconds on parse. Python clients using Pydantic emit datetimes via datetime.isoformat(), which omits the fractional part entirely when microsecond == 0 (~1/1000 odds per timestamp). The result is a payload like "2026-04-24T10:27:06Z" that the server cannot deserialize, surfacing as:

RuntimeException: Failed to convert JsonValue to target class
  caused by: Cannot deserialize value of type `java.util.Date` from String "2026-04-24T10:27:06Z":
             Unparseable date  (through reference chain: Table["columns"]→Column["tags"]→TagLabel["appliedAt"])

This was causing intermittent automator propagation failures in CI — every PATCH carrying a round-tripped appliedAt with microsecond=0 was being rejected. Same class of bug as the tag_usage.appliedAt precision fix in the 1.11.8 / 1.12.0 MySQL migrations, just on the wire side.

Fix

Register a Jackson mixin on TagLabel.appliedAt that wraps Jackson's normal Date path with one tiny pre-processing step: if the incoming value is the bare-second ISO form (yyyy-MM-ddTHH:mm:ssZ), pad the fractional with .000000 so the global SimpleDateFormat accepts it. Otherwise the value is handed to DeserializationContext#parseDate — the same path Jackson uses for every other Date field — so every wire form that worked before keeps working.

Cases the deserializer handles:

  • Bare-second ISO "2026-04-24T10:27:06Z" → padded → SDF parses as ms=0 (this PR's goal)
  • Server's own SDF emit "2026-04-24T10:27:06.000918Z" → SDF parses as ms=918 (round-trip preservation)
  • 3-digit ms ISO "2026-04-24T10:27:06.918Z" → SDF lenient → ms=918 (JS Date.toISOString())
  • JSON number 1777976050918new Date(getLongValue()) (matches Jackson default)
  • Numeric string "1777976050918" (10–19 digits) → Date(Long.parseLong) (JSON-Patch hop sometimes stringifies numbers)
  • null / empty / malformednull or JsonMappingException via handleWeirdStringValue

Mixin is added to the global OBJECT_MAPPER before any .copy(), so all five derived mappers inherit it. No generated code is touched. Serialization is unchanged (still always emits 6-digit fractional from the global format).

Test plan

  • JsonUtilsTest#testTagLabelAppliedAtAcceptsBareSecondPrecision — original PR goal
  • JsonUtilsTest#testTagLabelAppliedAtRoundTripPreservesMillis — server's own .000918Z round-trip preserves ms
  • JsonUtilsTest#testTagLabelAppliedAtAcceptsEpochMillis — JSON number + numeric string forms
  • JsonUtilsTest#testTagLabelAppliedAtMalformedRaisesMappingException — error path surfaces as JsonMappingException
  • mvn -pl openmetadata-spec install -DskipTests passes
  • mvn -pl openmetadata-service test -Dtest=JsonUtilsTest — 14/14 green
  • mvn spotless:apply clean
  • CI integration tests / playwright shards

@github-actions
Copy link
Copy Markdown
Contributor

Hi there 👋 Thanks for your contribution!

The OpenMetadata team will review the PR shortly! Once it has been labeled as safe to test, the CI workflows
will start executing and we'll be able to make sure everything is working as expected.

Let us know if you need any help!

@Khairajani Khairajani added the safe to test Add this label to run secure Github workflows on PRs label Apr 27, 2026
Accept ISO-8601 datetimes with or without fractional seconds for
TagLabel.appliedAt. Python clients omit fractional when a datetime's
microsecond is zero, emitting strings like "2026-04-24T10:27:06Z" that
the strict global SimpleDateFormat("yyyy-MM-dd'T'HH:mm:ss.SSSSSS'Z'")
rejects, causing PATCH operations to fail with "Failed to convert
JsonValue to target class".
@github-actions
Copy link
Copy Markdown
Contributor

github-actions Bot commented Apr 27, 2026

🟡 Playwright Results — all passed (18 flaky)

✅ 3980 passed · ❌ 0 failed · 🟡 18 flaky · ⏭️ 86 skipped

Shard Passed Failed Flaky Skipped
🟡 Shard 1 298 0 1 4
🟡 Shard 2 748 0 6 8
🟡 Shard 3 743 0 3 7
🟡 Shard 4 774 0 1 18
🟡 Shard 5 686 0 1 41
🟡 Shard 6 731 0 6 8
🟡 18 flaky test(s) (passed on retry)
  • Pages/AuditLogs.spec.ts › should apply both User and EntityType filters simultaneously (shard 1, 1 retry)
  • Features/ActivityAPI.spec.ts › Activity event is created when description is updated (shard 2, 1 retry)
  • Features/ActivityAPI.spec.ts › Activity event is created when tags are added (shard 2, 1 retry)
  • Features/ColumnBulkOperations.spec.ts › should show disabled edit button when no columns are selected (shard 2, 1 retry)
  • Features/Glossary/GlossaryHierarchy.spec.ts › should cancel move operation (shard 2, 1 retry)
  • Features/Glossary/GlossaryWorkflow.spec.ts › should display correct status badge color and icon (shard 2, 1 retry)
  • Features/Glossary/GlossaryWorkflow.spec.ts › should start term as Draft when glossary has reviewers (shard 2, 1 retry)
  • Features/RTL.spec.ts › Verify Following widget functionality (shard 3, 1 retry)
  • Features/Table.spec.ts › Tags term should be consistent for search (shard 3, 1 retry)
  • Flow/PersonaFlow.spec.ts › Set default persona for team should work properly (shard 3, 1 retry)
  • Pages/CustomProperties.spec.ts › Should display custom properties for apiCollection in right panel (shard 4, 1 retry)
  • Pages/EntityDataSteward.spec.ts › User as Owner Add, Update and Remove (shard 5, 1 retry)
  • Pages/Glossary.spec.ts › Column dropdown drag-and-drop functionality for Glossary Terms table (shard 6, 1 retry)
  • Pages/Lineage/DataAssetLineage.spec.ts › verify create lineage for entity - Table (shard 6, 1 retry)
  • Pages/Lineage/LineageFilters.spec.ts › Verify lineage schema filter selection (shard 6, 1 retry)
  • Pages/Lineage/LineageRightPanel.spec.ts › Verify custom properties tab IS visible for supported type: searchIndex (shard 6, 1 retry)
  • Pages/ODCSImportExport.spec.ts › Multi-object ODCS contract - object selector shows all schema objects (shard 6, 1 retry)
  • Pages/UserDetails.spec.ts › Admin user can edit teams from the user profile (shard 6, 1 retry)

📦 Download artifacts

How to debug locally
# Download playwright-test-results-<shard> artifact and unzip
npx playwright show-trace path/to/trace.zip    # view trace

Instant.parse misreads the server's own SimpleDateFormat output
(SSSSSS = left-padded ms, e.g. ".000918Z"), collapsing ms to 0 on
every PATCH round-trip and breaking tag mutations end-to-end.
Detect that form by its 6-digit "000"-leading fractional and parse
the digits as left-padded ms; fall through to Instant.parse for
no-fractional and Pydantic microsecond inputs.
Khairajani and others added 2 commits May 5, 2026 17:19
…put forms

The previous iteration replaced Jackson's default Date deserializer with
Instant.parse, which silently dropped support for JSON numbers and
numeric strings — wire forms that arrive in JSON Patch payloads. Tag
PATCHes started 4xx-ing on every entity in CI.

Delegate to ctxt.parseDate (the same global SimpleDateFormat path the
rest of Jackson uses) so every form that worked pre-PR keeps working.
Numeric tokens and stringified epoch-ms restore Jackson's prior
tolerance. The only addition is bare-second padding: if the value is
"…ssZ" (Pydantic with microsecond=0, the original PR goal), pad with
.000000 so the global format accepts it.
Per gitar-bot review. Avoids misinterpreting short all-digit strings
(e.g. compact "YYYYMMDD") as epoch milliseconds. Modern dates are
13 digits; 10 is the threshold for any year ≥ 2001.
@gitar-bot
Copy link
Copy Markdown

gitar-bot Bot commented May 5, 2026

Code Review ✅ Approved 2 resolved / 2 findings

Simplifies TagLabel.appliedAt deserialization to support both ISO formats and epoch-millisecond inputs, resolving potential failures when serializing datetimes without fractional seconds. No issues found.

✅ 2 resolved
Edge Case: LenientIsoDateDeserializer doesn't wrap DateTimeParseException

📄 openmetadata-spec/src/main/java/org/openmetadata/schema/utils/JsonUtils.java:912-918
Instant.parse(value) throws an unchecked DateTimeParseException for malformed input. While Jackson's infrastructure will catch it and surface an error, wrapping it in a ctxt.handleUnexpectedToken(...) or JsonMappingException would produce a cleaner, more consistent error message with the JSON path context (e.g., through reference chain: ...TagLabel["appliedAt"]) rather than an unwrapped runtime exception.

In practice this is low-impact because Jackson does catch RuntimeException during deserialization and re-wraps it, but explicit handling is the idiomatic Jackson pattern and gives better diagnostics.

Edge Case: looksLikeEpochMillis has no length guard against ambiguous inputs

📄 openmetadata-spec/src/main/java/org/openmetadata/schema/utils/JsonUtils.java:952-963
The looksLikeEpochMillis helper returns true for any all-digit string regardless of length. While ISO-8601 strings contain non-digit characters (so won't match), a hypothetical compact date like "20260424" (YYYYMMDD) would be misinterpreted as epoch millis (~234 days from epoch). For TagLabel.appliedAt this is unlikely in practice since inputs are either ISO-8601 or genuine epoch-ms, but a simple length guard (e.g. >= 10 digits, since epoch-ms for any date after ~2001 is 13 digits) would make intent clearer and prevent any future ambiguity.

Options

Display: compact → Showing less information.

Comment with these commands to change:

Compact
gitar display:verbose         

Was this helpful? React with 👍 / 👎 | Gitar

@sonarqubecloud
Copy link
Copy Markdown

sonarqubecloud Bot commented May 5, 2026

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

safe to test Add this label to run secure Github workflows on PRs

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants