Skip to content

Make @expiresAt attribute authoritative over the table expiration default#1812

Merged
kriszyp merged 3 commits into
mainfrom
kris/expiresat-field-authoritative
Jul 21, 2026
Merged

Make @expiresAt attribute authoritative over the table expiration default#1812
kriszyp merged 3 commits into
mainfrom
kris/expiresat-field-authoritative

Conversation

@kriszyp

@kriszyp kriszyp commented Jul 14, 2026

Copy link
Copy Markdown
Member

Fixes #1810. Reported internally (Jeff Darnton): a record-level @expiresAt field can only shorten-by-eviction, never extend past the table @table(expiration:) default.

Summary

The @expiresAt schema attribute's value was never fed into the stored expiry metadata that governs read-hiding and the cleanup sweep — it only armed a separate index-pruning sweep (runRecordExpirationEviction) that removes records already past due. So a far-future field value was ignored by both read-hiding and the table sweep, and could never extend past the table default. (And per #1481 that pruning sweep is LMDB-only, so on RocksDB the field enforced nothing at all.)

This resolves the stored expiresAt in the _writeUpdate commit path as:

options.expiresAt ?? context.expiresAt ?? (record @expiresAt field, if finite and >= 0) ?? table default

enforcing the invariant that the stored expiry metadata reflects the record's effective expiration. Because read-hiding and the getRange-based cleanup sweep (shouldEvict) are engine-agnostic, this also enforces field-derived expiry on RocksDB for new writes — addressing the correctness half of #1481 without depending on the LMDB-only pruning sweep (that sweep's index.getValuesgetRange fix, which also covers pre-existing records, remains tracked in #1481).

Where to look

  • resources/Table.ts — the new field-resolution block sits after recordToStore is merged, so it reads back exactly what the pruning sweep later sees. Negative field values are guarded (Number.isFinite(x) && x >= 0) because a negative epoch collides with the -1 "no expiration" metadata sentinel.
  • Precedence intentionally keeps options.expiresAt / context.expiresAt (the documented runtime override) ahead of the field.

Deliberate scope boundary (please sanity-check)

  • Direct writes only. Cache/source fills persist via a separate recordUpdater path and derive expiry from sourceContext.expiresAt (source freshness / table default), not the field. For a caching table the source's freshness signal is the authoritative TTL, so I left it as-is and documented it (Document the @expiresAt schema directive documentation#585, DESIGN.md). Cross-model review agreed this is a correct boundary, not a gap — flagging so a human can confirm the product intent.
  • Replicated writes carry options.expiresAt = event.expiresAt, so the receiver honors the origin's computed expiry rather than re-resolving the field. Correct-by-design for convergence; this PR doesn't touch that path.

Coverage notes (from the cross-model review)

Docs companion: HarperFast/documentation#585.

Generated by KrAIs (Claude Opus 4.8). Reviewed cross-model (Codex + Gemini + Harper-domain adjudication); no blockers.

@gemini-code-assist gemini-code-assist Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request updates the record-level TTL evaluation logic to make the schema-defined @expiresAt attribute authoritative over the table-level expiration default, allowing it to both extend and shorten the expiration time. It also adds comprehensive unit tests to verify this behavior under various scenarios. The review feedback points out a potential issue with using Number() directly for coercion, as values like booleans or empty strings can be coerced to 0 (causing immediate expiration) and ISO strings are ignored, and suggests a more robust parsing implementation.

Comment thread resources/Table.ts Outdated
@claude

claude Bot commented Jul 14, 2026

Copy link
Copy Markdown
Contributor

Reviewed; no blockers found.

@kriszyp
kriszyp marked this pull request as ready for review July 14, 2026 23:42
Comment thread resources/DESIGN.md Outdated
kriszyp added a commit that referenced this pull request Jul 15, 2026
Removes the empty trailing `|     |` cell noted in PR #1812 review by
cb1kenobi; the table has two columns.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
kriszyp added a commit that referenced this pull request Jul 15, 2026
Removes the empty trailing `|     |` cell noted in PR #1812 review by
cb1kenobi; the table has two columns.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
@kriszyp
kriszyp force-pushed the kris/expiresat-field-authoritative branch from f0190f3 to 9892389 Compare July 15, 2026 22:51
kriszyp and others added 3 commits July 17, 2026 06:19
…ault (#1810)

The record-level @expiresAt field was never fed into the stored expiry
metadata that governs read-hiding and the cleanup sweep — it only armed a
separate index-pruning sweep that removes already-past records. So a field
value could never extend past the table @table(expiration:) default, and on
RocksDB (where that sweep is LMDB-only, #1481) it never enforced expiry at all.

Resolve the stored expiresAt in the _writeUpdate commit path as
options.expiresAt ?? context.expiresAt ?? (record @expiresAt field, if finite
and >= 0) ?? table default. Read-hiding and the getRange-based cleanup sweep
are engine-agnostic, so field-derived expiry is now enforced on both engines
for direct put/patch.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…ean/empty)

Address Gemini review: Number() coerced booleans→0/1 and empty strings→0,
silently expiring records at epoch 0. Coerce only genuine timestamp shapes
(number/bigint epoch, Date, numeric/ISO string via Date.parse); everything
else falls through to NaN → table default.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Removes the empty trailing `|     |` cell noted in PR #1812 review by
cb1kenobi; the table has two columns.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
@kriszyp
kriszyp force-pushed the kris/expiresat-field-authoritative branch from 59127e0 to 42c33dc Compare July 17, 2026 12:21
@kriszyp
kriszyp merged commit 140bfaf into main Jul 21, 2026
93 of 94 checks passed
@kriszyp
kriszyp deleted the kris/expiresat-field-authoritative branch July 21, 2026 22:59
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

@expiresAt field not authoritative over table @table(expiration:) default (never fed into expiry metadata)

2 participants