credit lease SDK primitives: check / trackWithReservation / prewarm#121
Draft
bpapillon wants to merge 10 commits into
Draft
credit lease SDK primitives: check / trackWithReservation / prewarm#121bpapillon wants to merge 10 commits into
bpapillon wants to merge 10 commits into
Conversation
bpapillon
force-pushed
the
credit-lease-sdk-primitives
branch
6 times, most recently
from
fe61b92 to
c07650f
Compare
New opt-in client surface (`SchematicClient.prewarm` / `check` / `trackWithReservation`) for credit-burndown gating against the lease API in SchematicHQ/schematic-api#5433. Routes datastream client-side eval through the v0.2.0 WASM `checkFlagWithOptions` entry so preflight `event_usage` substitutions work end-to-end. Lease balance and reservation table default to in-memory; supplying `creditLeases.redisClient` switches them to Redis-backed stores that share state across SDK pods via atomic Lua scripts (check-and-decrement on `tryReserve`, consume-and-refund on `consume`). Existing `checkFlag` / `identify` / `track` stay byte-compatible.
bpapillon
force-pushed
the
credit-lease-sdk-primitives
branch
from
c07650f to
0137a29
Compare
Incoming WebSocket payloads (`rulesengine.Flags`, `Company`, `User`) were applied to the local cache via bare `message.data as Schematic.Foo` casts, with no transformation between the wire format and the in-memory type. Two stacked bugs fall out of that: 1. snake_case vs camelCase. The Go server emits snake_case JSON (`event_subtype`, `condition_type`, `credit_id`), and the Fern- generated TS types use camelCase. Without running incoming payloads through the serializer, every camelCase property read on a cached entity returns `undefined`. That silently breaks any downstream consumer that walks the cached structure — most painfully `findCreditCondition()` on the credit-lease check path, which loops over `condition.conditionType` and `condition.eventSubtype` looking for a match and never finds one. 2. Go nil-slice → JSON null. `json.Marshal` serializes a nil slice as `null`, but the Fern serializers declare these as required `list(...)` and reject null with an opaque "Expected list. Received null." error. The same flag tree that fails the camelCase reads above also has `condition_groups: null` on rules that don't carry any groups, so even after wiring the serializer in the parse rejects the whole payload. Fix: run flag / company / user payloads through `parseOrThrow` after walking the raw object to coerce known list-shaped wire fields from null to []. Parse failures degrade to a warn-log + skip rather than poisoning the whole connection.
The wasm-bindgen-generated `dist/wasm/rulesengine.js` loads its `.wasm`
sibling at runtime via `fs.readFileSync(${__dirname}/rulesengine_bg.wasm)`.
That works in plain Node, but breaks the moment a downstream bundler
follows the require chain — webpack rewrites `__dirname` to point inside
the bundle output, where the `.wasm` sibling never gets copied. Symptom
in a Next.js consumer:
ENOENT: no such file or directory, open
'.next/dev/server/vendor-chunks/rulesengine_bg.wasm'
…and the SDK silently falls back to API-only checks, disabling DataStream
and credit-lease paths.
This adds a build step that reads the WASM binary, base64-encodes it,
and rewrites `dist/wasm/rulesengine.js` to instantiate from an inlined
`Buffer.from(BASE64, 'base64')` instead of touching the filesystem. The
standalone `.wasm` is then removed from `dist/` since nothing reads it
at runtime anymore.
Tarball delta is +138 KB (base64 overhead on the ~414 KB binary). For
consumers that use creditLeases / DataStream the net bundle size is
unchanged — the WASM bytes are in either form. For consumers that don't,
practical tree-shaking ends up the same regardless of inlining: the
require chain is reachable from the package's main entry point and the
WASM init runs as a module-level side effect. Materially reducing the
WASM cost for non-credit-lease consumers would require splitting credit
leases into a separate entry point — out of scope here.
If wasm-bindgen output ever stops matching the regex this script keys
on, the build throws with a clear pointer instead of silently shipping
a broken loader.
The verify-package step asserted on `dist/wasm/rulesengine_bg.wasm` presence, which the inlining step now intentionally removes. Replace those checks with a content-sentinel grep on the inlined comment so a future change that silently reverts to disk-based loading still fails CI, plus an explicit assertion that the standalone .wasm is gone.
… shape partialCompany/partialUser were writing snake_case keys back onto a camelCase cached entity, leaving the cache in a hybrid shape after any partial update. Update both to write camelCase target keys consistently so the cache stays in one canonical form. Test fixture updates: - Add `plan_version_ids: []` to all Company mocks. Required by the Fern schema and previously absent — once payloads route through parseOrThrow, the missing field caused ParseError and downstream test timeouts (the cache update silently failed and getCompany() hung waiting for a never-arriving WS response). - Compute expected post-parse shapes via asCompany/asUser/asFlag helpers. Wire fixtures stay snake_case (matching what the server sends); assertions compare against the camelCase canonical shape. - Switch partial-merge assertions from snake_case field names to camelCase to match the canonicalized merge output.
…onicalization
merge.test.ts: cached entities now model the post-parseOrThrow shape
(camelCase). Partial inputs stay snake_case (wire). Assertions check
the camelCase output keys partialCompany/partialUser now write.
datastream-client.test.ts: the two-level-caching metrics test was
providing a metric with only { eventSubtype, value } — parseOrThrow
rejects the company on the FULL message (RulesengineCompanyMetric
requires account_id/company_id/created_at/environment_id/event_subtype/
month_reset/period/value), so the cache never lands and the
subsequent getCompany hangs out the test timer. Provide a full metric.
rulesengine v0.1.16 migrated billing_product_ids, plan_ids, plan_version_ids, and entitlements on the wire Company struct to JSONSlice[T] — so they marshal as [] not null. But until the api ships v0.1.16 (api PR #5535) the WebSocket still ships null for these on any company without a plan/subscription, and parseOrThrow on the SDK side rejects the message. Existing NULLABLE_LIST_KEYS set missed the four names above; add them so the coerce helper keeps the SDK working against pre-bump APIs in the field. Once #5535 deploys the whole coerceNullArrays hack can be deleted in one followup.
rulesengine v0.1.16 wire types use JSONSlice[T] so nil slices marshal as [] not null, and the api bump (schematic-api #5535) is now live on main. parseOrThrow on the WS payloads no longer encounters null where a list is expected, so the coerce helper and its key allowlist have nothing to do. Pass message.data straight into the serializers.
Strict Fern parsing (post-PARSE_OPTS-removal) rejects keys not in the
schema. The test fixtures used `crm_product_ids` (never in
RulesengineCompany) and `traits: [{ key, value }]` (Trait schema has
`value` + optional `trait_definition`, no `key`). The lenient
parser silently passed these through; the strict parser throws, which
left the cache empty and triggered 15 phantom-timeout failures.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
SDK-side implementation of the credit lease + reservation primitive from the gist. Builds on the lease API (SchematicHQ/schematic-api#5433) exposed via b1f9940 and the v0.2.0 WASM
checkFlagWithOptionsentry from 8bd2d60.Three new methods on
SchematicClient(prewarm/check/trackWithReservation), opt-in via acreditLeasesconfig block. ExistingcheckFlag/identify/trackunchanged. Lease-bearing checks require DataStream so the SDK has the cached flag + company to find the credit condition and substitute the balance for WASM eval; without it,checkfalls through to a plaincheckFlag.Cross-pod state sharing is supported via
creditLeases.redisClient— when supplied, lease balance and the reservation table live in Redis, with atomic mutations driven by Lua scripts (check-and-decrement ontryReserve, consume-and-refund onconsume). Without it, the SDK falls back to single-pod in-memory stores.Draft — not yet exercised end-to-end against the real API + datastream. Open verification items:
credit_balancesshape on cachedRulesengineCompanyforexternal_ratedcredits,consumptionRateon the demo flag's condition, and liveacquire → check → trackround-trip.