Skip to content

feat(analytics): record write-transaction commit latency#1688

Merged
kriszyp merged 2 commits into
mainfrom
kris/txn-commit-latency-592
Jul 17, 2026
Merged

feat(analytics): record write-transaction commit latency#1688
kriszyp merged 2 commits into
mainfrom
kris/txn-commit-latency-592

Conversation

@kriszyp

@kriszyp kriszyp commented Jul 7, 2026

Copy link
Copy Markdown
Member

Summary

Adds transaction-commit-time, a distribution metric (mean + p90/p95/p99/p999 via the standard analytics action path) recording each write transaction's submit→settle duration.

Purpose

#592 asks for a leading indicator before write-heavy workloads hit Outstanding write transactions have too long of queue, please try again later (503). That rejection fires when the oldest outstanding commit has been outstanding longer than storage.maxTransactionQueueTime (45s default) — a latency condition, not a count. transaction-commit-time is measured on that exact clock (outstandingCommitStart → settle), so a rising p99/p999 is the direct leading indicator: it reflects transaction size/duration and the shared storage-engine write path, not just per-thread submission concurrency.

Recorded per commit attempt — a transient-conflict retry rejects and re-issues commit(), and the overload check's own outstandingCommit re-arms per attempt the same way, so this matches the failure semantics exactly.

Design note (dependency inversion)

resources/DatabaseTransaction.ts does not import the analytics/server modules. It exposes setCommitLatencyRecorder(recorder); resources/analytics/write.ts registers the recorder at load. This mirrors the existing replicationConfirmation callback-registration pattern in the same file and avoids a circular import between the storage layer and the analytics/server layer. The recorder is wrapped so a metrics failure can never throw into the floating .then() or disturb the commit path.

Attention

  • Companion docs PR: docs(analytics): document transaction-commit-time metric documentation#572
  • Sibling PR (secondary/complementary metric, not required for this one): feat(analytics): expose write/read transaction queue depth (count) metrics #1689 — a per-thread in-flight transaction count. Kris flagged that count doesn't capture transaction size/duration or the shared write-path queue the way latency does, so it's split out as an independent, optional PR.
  • Cross-model review (Codex + Gemini) caught three real issues in an earlier revision of this diff — an unhandled-rejection risk on the floating .then(), a circular-import risk (DatabaseTransactionServer), and a test binding issue. All three are resolved by the registration-based redesign above (verified by re-running build/lint/tests); no open concerns remain from that review.

Generated by Claude Opus 4.8.

Emit transaction-commit-time as a distribution metric (mean + percentiles via
the standard analytics action path) capturing each write commit's submit->settle
duration. This is the same clock the overload check compares against, so a rising
p99/p999 is the leading indicator for the 'Outstanding write transactions have
too long of queue' (503) rejection — and unlike an in-flight count it reflects
transaction size/duration and the shared storage-engine write path, not just
per-thread submission concurrency.

Recorded per commit attempt (retries re-arm the outstanding-commit clock, so
per-attempt timing matches the overload semantics). Async commit path only;
synchronous commits do not queue.

Closes #592

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
gemini-code-assist[bot]

This comment was marked as resolved.

@claude

claude Bot commented Jul 7, 2026

Copy link
Copy Markdown
Contributor

Reviewed; no blockers found.

…ni hardening

- Skip the commit-recording assertion under HARPER_STORAGE_ENGINE=lmdb: LMDB
  writes route through the separate LMDBTransaction.commit() override
  (resources/LMDBTransaction.ts), which does not call the RocksDB-path
  latency hook — matching the existing RocksDB-only carve-outs in
  transaction.test.js. This is what failed CI (all 3 Node versions, via the
  npm run test:unit:all -> test:unit:lmdb pass).
- setCommitLatencyRecorder now types the recorder as optional, matching
  actual call sites (tests unregister it with undefined).
- recordCommitLatency guards commitResolution as thenable before calling
  .then(), per Gemini review feedback -- defensive against a future caller
  passing a non-Promise value, even though today's only caller
  (rocksdb-js's async Transaction.commit()) always returns a real Promise.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@kriszyp
kriszyp marked this pull request as ready for review July 7, 2026 16:39

@heskew heskew left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Latency recording is cheap on the commit path (one .then on the existing promise, early-return when unset, try-catch so analytics can't break a commit). Thenable guard added, circular import avoided via registration. Tests cover recording + unregistered-recorder no-throw. LGTM.

🤖 Reviewed via cross-model pipeline; approved by @heskew.

@kriszyp
kriszyp merged commit 4209c75 into main Jul 17, 2026
52 checks passed
@kriszyp
kriszyp deleted the kris/txn-commit-latency-592 branch July 17, 2026 20:07
dawsontoth added a commit that referenced this pull request Jul 20, 2026
…ency call

Unrelated to the two-phase deploy feature — fixes a pre-existing type error on
main (introduced by #1688's commit-latency analytics) that main's other build
steps swallow via `|| true`/`continue-on-error`, but the newly-added Next.js
adapter integration workflow (#1385) runs the build without that tolerance and
so fails on it (tsc exit 2) for every PR built on current main.

`commitResolution` is declared with the wider `Promise<number | void> | void`
(the abort() branch reassigns it), but at this call site it is the `commit()`
promise already cast to `Promise<void>` on the line above. recordCommitLatency
only awaits it for timing and never reads the resolved value, so the cast is
type-only with no runtime effect — matching the author's existing cast and
safety comment two lines up.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
kriszyp added a commit that referenced this pull request Jul 21, 2026
`npm run build` has been failing on `main` since #1688: `commitResolution` is
declared `Promise<number | void> | void` (commit() resolves the RETRY_NOW_VALUE
sentinel on a coordinated retry), so assignment narrowing lands on
`Promise<number | void>` and the `as Promise<void>` cast doesn't survive to the
`recordCommitLatency()` call. The helper only reads settlement timing, so
`Promise<unknown>` is the honest parameter type.

Harper's own workflows tolerate the non-zero `tsc` exit (`noEmitOnError` is
off), but the downstream Next.js adapter integration builds harper and treats it
as a failure, so any PR touching adapter-relevant files goes red.

Refs #1688

Co-Authored-By: Claude Sonnet <noreply@anthropic.com>
kriszyp added a commit that referenced this pull request Jul 22, 2026
…ecorder

The `as Promise<void>` cast on `transaction.commit()` was ineffective:
`commitResolution` is declared `Promise<number | void> | void`, so
assignment narrowing re-widens to the `Promise<number | void>` union
member and `recordCommitLatency(commitResolution, ...)` failed tsc
(TS2345) — the org-wide red main build.

`commit()` genuinely resolves to `number | void` (the coordinated-retry
RETRY_NOW_VALUE sentinel), so widen `recordCommitLatency`'s parameter to
`Promise<number | void>` and drop the misleading cast. The recorder only
times the settle; it ignores the resolved value, so per-attempt recording
is unchanged.

Regression from #1688.

Co-Authored-By: Claude Opus <noreply@anthropic.com>
kriszyp added a commit that referenced this pull request Jul 23, 2026
…ecorder

The `as Promise<void>` cast on `transaction.commit()` was ineffective:
`commitResolution` is declared `Promise<number | void> | void`, so
assignment narrowing re-widens to the `Promise<number | void>` union
member and `recordCommitLatency(commitResolution, ...)` failed tsc
(TS2345) — the org-wide red main build.

`commit()` genuinely resolves to `number | void` (the coordinated-retry
RETRY_NOW_VALUE sentinel), so widen `recordCommitLatency`'s parameter to
`Promise<number | void>` and drop the misleading cast. The recorder only
times the settle; it ignores the resolved value, so per-attempt recording
is unchanged.

Regression from #1688.

Co-Authored-By: Claude Opus <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants