Skip to content

fix: ingest registered-synchronizer traffic purchases past migration 0 (Scala) - #20

Open
salindne wants to merge 2 commits into
feat/dedicated-syncfrom
fix/dedicated-sync-ingestion-migration-id
Open

fix: ingest registered-synchronizer traffic purchases past migration 0 (Scala)#20
salindne wants to merge 2 commits into
feat/dedicated-syncfrom
fix/dedicated-sync-ingestion-migration-id

Conversation

@salindne

Copy link
Copy Markdown
Collaborator

What this does

Traffic bought for a registered synchronizer creates a MemberTraffic contract whose
migrationId is pinned to 0 by the buy choice (validateBuyMemberTrafficInputs): a registered
synchronizer upgrades via LSU and never hard-migrates, so it has no generation counter. The SV and
Scan ACS stores, however, ingest MemberTraffic behind
payload.migrationId == domainMigrationId, the decentralized synchronizer's generation. On any
network past migration 0, 0 == N is false and every registered-synchronizer purchase is
silently dropped at ingestion
: Amulet is burned, the contract exists on-ledger, and the
reconcile trigger never fires (it is ACS-sourced), MergeMemberTrafficContractsTrigger never
compacts, and Scan's getMemberTrafficStatus serves total_purchased = 0, all with no error and
no log line. Dev environments cannot see it: LocalNet, the integration-test harness, and a freshly
bootstrapped SV all sit at migration 0, where the comparison passes.

  • Both filters widened (SvDsoStore, ScanStore) to also accept records that carry an
    operator and the pinned migration id, the same registered-synchronizer definition
    SyncOperatorStore's filter uses (feat: sync operator app skeleton #16), so every store ingesting MemberTraffic shares one
    definition. The operator field is set only from a RegisteredSynchronizer, preserved through
    the merge choice behind an operator-agreement check, and None on the decentralized-sync path;
    the buy gate pins migrationId = 0 whenever an operator is set, so the added arm accepts
    exactly the registered-synchronizer records and nothing else.
  • The generation check is preserved. It exists to stop re-granting a member's lifetime
    purchases on a sequencer whose traffic state reset at a hard migration (the reconcile trigger
    sets an absolute cumulative limit and only ever raises it). A registered synchronizer's traffic
    state never resets that way, so the exemption cannot double-grant, and decentralized-synchronizer
    records (no operator) keep today's behaviour byte for byte.
  • No store-descriptor bump. The changed filter affects only ingestion of new events; nothing
    historical is missed (no registered-synchronizer purchases exist on any network yet), and a bump
    would force a full unfiltered ACS re-ingest with the app unavailable.
  • Not changed: the same comparison in ValidatorStore (ValidatorTopUpState) and
    UserWalletStore (BuyTrafficRequest). No code path can produce a registered-synchronizer
    contract of either type today (the top-up trigger targets only the active decentralized
    synchronizer; both wallet entry points hardcode optRegisteredSynchronizer = None). To be
    picked up with registered-synchronizer top-up support ([P2-E5.5] Validator auto-top-up on dedicated synchronizers ChainSafe/canton-extending-mainnet#40).

How it's verified

One discrimination test per store suite (DbScanStoreTest, DbSvDsoStoreTest): a store at
migration 1 is fed three contracts, with amounts chosen so a failing sum names the failure mode.
A current-generation purchase (counted, as today), a previous-generation purchase with no operator
(still dropped: the double-grant guard), and a registered-synchronizer purchase (counted: the
fix). The SV variant additionally names a different synchronizer id on the registered purchase,
exercising the per-synchronizer SQL scoping, and asserts listMemberTrafficContracts sees it (the
merge automation's read path).

Run fail-first: with the filters unchanged the new tests fail as the bug predicts.
DbScanStoreTest: 10 was not equal to 1010; DbSvDsoStoreTest: 0 was not equal to 1000 (the
registered purchase dropped at ingestion). With the fix, both full suites pass (31/31 and 74/74),
including the pre-existing getTotalPurchasedMemberTraffic test that encodes the old-generation
exclusion, unmodified. scalafmt clean.

Test plumbing: mkStore in both suites gains a defaulted migrationId parameter (the
UserWalletStoreTest idiom) and the memberTraffic builders gain defaulted
migrationId/operator parameters; no existing call site changes, no test-registry change.
A payload.migrationId != 0 record with an operator is not constructible on-ledger (the buy gate
rejects it, with a Daml negative test from #2), so store behaviour on that input is not asserted.

Tracked in

Implements the store/ingestion half of [P2-E1.5] ChainSafe/canton-extending-mainnet#83 (analysis:
ChainSafe/canton-extending-mainnet#60); the buy-gate half shipped in #2. Ratifies the migration-id
assumption SyncOperatorStore (#16) relies on.

…0 [ci]

A registered synchronizer's traffic purchases pin migrationId = 0 (the buy
gate enforces it; a registered synchronizer upgrades via LSU and never
hard-migrates). The SV and Scan ACS stores ingested MemberTraffic behind
payload.migrationId == domainMigrationId, i.e. the decentralized
synchronizer's generation, so on any network past migration 0 every
registered-synchronizer purchase was silently dropped at ingestion: the
reconcile trigger never fired, merge automation never compacted, and Scan
served total_purchased = 0, all without an error. Dev environments sit at
migration 0 and cannot reproduce it.

Widen both filters to exempt records that carry an operator and the pinned
migration id, the same registered-synchronizer definition the sync operator
app's store uses. The operator is set only from a RegisteredSynchronizer
and None on the decentralized-sync path. The generation check exists to
avoid re-granting lifetime purchases on a sequencer whose traffic state
reset at a hard migration; a registered synchronizer's traffic state never
resets that way, so the exemption cannot double-grant, and
decentralized-sync records keep today's behaviour unchanged.

Tests: one discrimination test per store suite, a store at migration 1
fed a current-generation purchase (counted), a previous-generation
purchase (still dropped), and a registered-synchronizer purchase (now
counted); fails pre-fix with the registered purchase at 0. mkStore and
the memberTraffic helpers gain defaulted migrationId/operator parameters;
no existing call site or test changes.

Implements the ingestion half of ChainSafe/canton-extending-mainnet#83
(analysis: ChainSafe/canton-extending-mainnet#60); the buy-gate half
shipped in #2.

Signed-off-by: Sebastian Lindner <33971232+salindne@users.noreply.github.com>
@sadiq1971

Copy link
Copy Markdown
Collaborator

Looks good

@moritzkiefer-da moritzkiefer-da left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

thx

vt.payload.dso == dso && vt.payload.migrationId == domainMigrationId
vt.payload.dso == dso &&
(vt.payload.migrationId == domainMigrationId ||
(vt.payload.operator.isPresent && vt.payload.migrationId == 0L))

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why do you need the payload.migrationId == 0 check? You already enforce in the Daml code that you can't have this fail.

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

thanks, dropping it. Simplified to just vt.payload.operator.isPresent.

// migrationId = 0 (see AmuletRules.validateBuyMemberTrafficInputs). The generation check
// below exists to stop re-granting a member's lifetime purchases on a sequencer whose
// traffic state reset at a migration; that cannot happen to a registered synchronizer, so
// its records are exempt. A record counts as registered-synchronizer traffic iff it

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think what we're missing is a comment that explains why you need to ingest them at all in Scan and SvDsoStore. You don't reconcile traffic here so that part seems irrelevant. I think the answer is probably:

  1. For the dso store you need it because you want to merge them.
  2. For the scan store you'll likely need it eventually so you can expose the traffic balance?

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Adding both reasons to the comment: merging in the DSO store, per-synchronizer purchase totals in Scan. And dropping the reconciliation framing since neither store does that.

…e stores ingest [ci]

Review feedback on #20:

Drop the redundant migrationId == 0 conjunct from the exemption arm. The
buy choice already rejects a non-zero migrationId whenever a registration
is supplied, and the operator is only ever set from that same
registration, so operator.isPresent already implies it.

Rewrite the comment to say why these stores ingest registered-synchronizer
records at all, which was missing: the merge automation compacts them in
the DSO store, and Scan serves per-synchronizer purchase totals from them.
Neither store reconciles that traffic; the operator's own node does.

Signed-off-by: Sebastian Lindner <33971232+salindne@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants