feat(analytics): record write-transaction commit latency#1688
Merged
Conversation
Emit transaction-commit-time as a distribution metric (mean + percentiles via the standard analytics action path) capturing each write commit's submit->settle duration. This is the same clock the overload check compares against, so a rising p99/p999 is the leading indicator for the 'Outstanding write transactions have too long of queue' (503) rejection — and unlike an in-flight count it reflects transaction size/duration and the shared storage-engine write path, not just per-thread submission concurrency. Recorded per commit attempt (retries re-arm the outstanding-commit clock, so per-attempt timing matches the overload semantics). Async commit path only; synchronous commits do not queue. Closes #592 Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This was referenced Jul 7, 2026
Contributor
|
Reviewed; no blockers found. |
…ni hardening - Skip the commit-recording assertion under HARPER_STORAGE_ENGINE=lmdb: LMDB writes route through the separate LMDBTransaction.commit() override (resources/LMDBTransaction.ts), which does not call the RocksDB-path latency hook — matching the existing RocksDB-only carve-outs in transaction.test.js. This is what failed CI (all 3 Node versions, via the npm run test:unit:all -> test:unit:lmdb pass). - setCommitLatencyRecorder now types the recorder as optional, matching actual call sites (tests unregister it with undefined). - recordCommitLatency guards commitResolution as thenable before calling .then(), per Gemini review feedback -- defensive against a future caller passing a non-Promise value, even though today's only caller (rocksdb-js's async Transaction.commit()) always returns a real Promise. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
kriszyp
marked this pull request as ready for review
July 7, 2026 16:39
heskew
approved these changes
Jul 17, 2026
heskew
left a comment
Member
There was a problem hiding this comment.
Latency recording is cheap on the commit path (one .then on the existing promise, early-return when unset, try-catch so analytics can't break a commit). Thenable guard added, circular import avoided via registration. Tests cover recording + unregistered-recorder no-throw. LGTM.
🤖 Reviewed via cross-model pipeline; approved by @heskew.
dawsontoth
added a commit
that referenced
this pull request
Jul 20, 2026
…ency call Unrelated to the two-phase deploy feature — fixes a pre-existing type error on main (introduced by #1688's commit-latency analytics) that main's other build steps swallow via `|| true`/`continue-on-error`, but the newly-added Next.js adapter integration workflow (#1385) runs the build without that tolerance and so fails on it (tsc exit 2) for every PR built on current main. `commitResolution` is declared with the wider `Promise<number | void> | void` (the abort() branch reassigns it), but at this call site it is the `commit()` promise already cast to `Promise<void>` on the line above. recordCommitLatency only awaits it for timing and never reads the resolved value, so the cast is type-only with no runtime effect — matching the author's existing cast and safety comment two lines up. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
kriszyp
added a commit
that referenced
this pull request
Jul 21, 2026
`npm run build` has been failing on `main` since #1688: `commitResolution` is declared `Promise<number | void> | void` (commit() resolves the RETRY_NOW_VALUE sentinel on a coordinated retry), so assignment narrowing lands on `Promise<number | void>` and the `as Promise<void>` cast doesn't survive to the `recordCommitLatency()` call. The helper only reads settlement timing, so `Promise<unknown>` is the honest parameter type. Harper's own workflows tolerate the non-zero `tsc` exit (`noEmitOnError` is off), but the downstream Next.js adapter integration builds harper and treats it as a failure, so any PR touching adapter-relevant files goes red. Refs #1688 Co-Authored-By: Claude Sonnet <noreply@anthropic.com>
kriszyp
added a commit
that referenced
this pull request
Jul 22, 2026
…ecorder The `as Promise<void>` cast on `transaction.commit()` was ineffective: `commitResolution` is declared `Promise<number | void> | void`, so assignment narrowing re-widens to the `Promise<number | void>` union member and `recordCommitLatency(commitResolution, ...)` failed tsc (TS2345) — the org-wide red main build. `commit()` genuinely resolves to `number | void` (the coordinated-retry RETRY_NOW_VALUE sentinel), so widen `recordCommitLatency`'s parameter to `Promise<number | void>` and drop the misleading cast. The recorder only times the settle; it ignores the resolved value, so per-attempt recording is unchanged. Regression from #1688. Co-Authored-By: Claude Opus <noreply@anthropic.com>
kriszyp
added a commit
that referenced
this pull request
Jul 23, 2026
…ecorder The `as Promise<void>` cast on `transaction.commit()` was ineffective: `commitResolution` is declared `Promise<number | void> | void`, so assignment narrowing re-widens to the `Promise<number | void>` union member and `recordCommitLatency(commitResolution, ...)` failed tsc (TS2345) — the org-wide red main build. `commit()` genuinely resolves to `number | void` (the coordinated-retry RETRY_NOW_VALUE sentinel), so widen `recordCommitLatency`'s parameter to `Promise<number | void>` and drop the misleading cast. The recorder only times the settle; it ignores the resolved value, so per-attempt recording is unchanged. Regression from #1688. Co-Authored-By: Claude Opus <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Adds
transaction-commit-time, a distribution metric (mean + p90/p95/p99/p999 via the standard analytics action path) recording each write transaction's submit→settle duration.Purpose
#592 asks for a leading indicator before write-heavy workloads hit
Outstanding write transactions have too long of queue, please try again later(503). That rejection fires when the oldest outstanding commit has been outstanding longer thanstorage.maxTransactionQueueTime(45s default) — a latency condition, not a count.transaction-commit-timeis measured on that exact clock (outstandingCommitStart→ settle), so a rising p99/p999 is the direct leading indicator: it reflects transaction size/duration and the shared storage-engine write path, not just per-thread submission concurrency.Recorded per commit attempt — a transient-conflict retry rejects and re-issues
commit(), and the overload check's ownoutstandingCommitre-arms per attempt the same way, so this matches the failure semantics exactly.Design note (dependency inversion)
resources/DatabaseTransaction.tsdoes not import the analytics/server modules. It exposessetCommitLatencyRecorder(recorder);resources/analytics/write.tsregisters the recorder at load. This mirrors the existingreplicationConfirmationcallback-registration pattern in the same file and avoids a circular import between the storage layer and the analytics/server layer. The recorder is wrapped so a metrics failure can never throw into the floating.then()or disturb the commit path.Attention
.then(), a circular-import risk (DatabaseTransaction→Server), and a test binding issue. All three are resolved by the registration-based redesign above (verified by re-running build/lint/tests); no open concerns remain from that review.Generated by Claude Opus 4.8.