parse datastream payloads through Fern serializers#122
Draft
bpapillon wants to merge 6 commits into
Draft
Conversation
Incoming WebSocket payloads (`rulesengine.Flags`, `Company`, `User`) were applied to the local cache via bare `message.data as Schematic.Foo` casts, with no transformation between the wire format and the in-memory type. Two stacked bugs fall out of that: 1. snake_case vs camelCase. The Go server emits snake_case JSON (`event_subtype`, `condition_type`, `credit_id`), and the Fern- generated TS types use camelCase. Without running incoming payloads through the serializer, every camelCase property read on a cached entity returns `undefined`. That silently breaks any downstream consumer that walks the cached structure — most painfully `findCreditCondition()` on the credit-lease check path, which loops over `condition.conditionType` and `condition.eventSubtype` looking for a match and never finds one. 2. Go nil-slice → JSON null. `json.Marshal` serializes a nil slice as `null`, but the Fern serializers declare these as required `list(...)` and reject null with an opaque "Expected list. Received null." error. The same flag tree that fails the camelCase reads above also has `condition_groups: null` on rules that don't carry any groups, so even after wiring the serializer in the parse rejects the whole payload. Fix: run flag / company / user payloads through `parseOrThrow` after walking the raw object to coerce known list-shaped wire fields from null to []. Parse failures degrade to a warn-log + skip rather than poisoning the whole connection.
… shape partialCompany/partialUser were writing snake_case keys back onto a camelCase cached entity, leaving the cache in a hybrid shape after any partial update. Update both to write camelCase target keys consistently so the cache stays in one canonical form. Test fixture updates: - Add `plan_version_ids: []` to all Company mocks. Required by the Fern schema and previously absent — once payloads route through parseOrThrow, the missing field caused ParseError and downstream test timeouts (the cache update silently failed and getCompany() hung waiting for a never-arriving WS response). - Compute expected post-parse shapes via asCompany/asUser/asFlag helpers. Wire fixtures stay snake_case (matching what the server sends); assertions compare against the camelCase canonical shape. - Switch partial-merge assertions from snake_case field names to camelCase to match the canonicalized merge output.
…onicalization
merge.test.ts: cached entities now model the post-parseOrThrow shape
(camelCase). Partial inputs stay snake_case (wire). Assertions check
the camelCase output keys partialCompany/partialUser now write.
datastream-client.test.ts: the two-level-caching metrics test was
providing a metric with only { eventSubtype, value } — parseOrThrow
rejects the company on the FULL message (RulesengineCompanyMetric
requires account_id/company_id/created_at/environment_id/event_subtype/
month_reset/period/value), so the cache never lands and the
subsequent getCompany hangs out the test timer. Provide a full metric.
rulesengine v0.1.16 migrated billing_product_ids, plan_ids, plan_version_ids, and entitlements on the wire Company struct to JSONSlice[T] — so they marshal as [] not null. But until the api ships v0.1.16 (api PR #5535) the WebSocket still ships null for these on any company without a plan/subscription, and parseOrThrow on the SDK side rejects the message. Existing NULLABLE_LIST_KEYS set missed the four names above; add them so the coerce helper keeps the SDK working against pre-bump APIs in the field. Once #5535 deploys the whole coerceNullArrays hack can be deleted in one followup.
rulesengine v0.1.16 wire types use JSONSlice[T] so nil slices marshal as [] not null, and the api bump (schematic-api #5535) is now live on main. parseOrThrow on the WS payloads no longer encounters null where a list is expected, so the coerce helper and its key allowlist have nothing to do. Pass message.data straight into the serializers.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Incoming WebSocket payloads (
rulesengine.Flag,Company,User) were applied to the cache via bareas Schematic.Foocasts. The Go server emits snake_case JSON; Fern's TS types and rulesengine-rust's WASM (#[serde(rename_all = "camelCase")]) both expect camelCase. So every camelCase property read on a cached entity returnedundefined— silently breaksfindCreditCondition()on the credit-lease check path, and the WASM eval receives input it interprets as all-defaults (empty rules, empty entitlements).Run FULL payloads through
parseOrThrowand havepartialCompany/partialUserwrite camelCase target keys on PARTIAL updates, so the cache stays in a single canonical shape. Parse failures degrade to a warn-log + skip rather than poisoning the connection.