Skip to content

fix(resources): Table.clear() also clears secondary index dbis#1906

Open
kriszyp wants to merge 1 commit into
mainfrom
fix/table-clear-secondary-indexes
Open

fix(resources): Table.clear() also clears secondary index dbis#1906
kriszyp wants to merge 1 commit into
mainfrom
fix/table-clear-secondary-indexes

Conversation

@kriszyp

@kriszyp kriszyp commented Jul 23, 2026

Copy link
Copy Markdown
Member

What

Table.clear() (resources/Table.ts) only cleared the primary store — secondary-index DBIs were left populated with entries pointing at records the clear removed.

Why

This surfaced as a review finding on the harper-1839-sql-selectall-pk-sort work: it's harmless for that test harness (it re-writes the same ids with the same values), but it's a sharp edge for any caller of the exported clear() API on an indexed table:

  • Stale index entries accumulate and are never freed (storage leak).
  • If a caller clear()s and then re-writes the same id with a different indexed value (an intended usage, not a misuse), the old value's stale index entry keeps pointing at that id. An equality search on the old value then incorrectly returns that record, because the equality-index path (resources/search.ts) doesn't re-check the record's current attribute value against the search value — it trusts the index.
  • A stale entry pointing at a fully-missing record is not itself a correctness bug: Table.transformEntryForSelect re-resolves every candidate against the primary store and silently SKIPs misses — but that also means the previous behavior was masking the real risk rather than avoiding it.

Fix

clear() now also clears every secondary-index dbi, using the same clearAsync()/clear() pattern runIndexing already uses when rebuilding an index from scratch (resources/databases.ts).

 static clear() {
-    return primaryStore.clear();
+    // clear the primary store and every secondary index dbi (same pattern used by
+    // runIndexing when rebuilding from scratch), so clear() doesn't leave stale
+    // index entries pointing at records that no longer exist.
+    const promises = [primaryStore.clear()];
+    for (const key in indices) {
+        const index = indices[key];
+        promises.push(index.clearAsync ? index.clearAsync() : index.clear());
+    }
+    return Promise.all(promises);
 }

Testing

New unit test unitTests/resources/tableClearSecondaryIndex.test.js:

  • Confirms the primary store and an indexed attribute's search results are both empty after clear().
  • Reproduces the real regression: put(id, tag: 'before')clear()put(id, tag: 'after') → searching for the stale old value 'before' must return 0 rows. Verified this test fails (returns 1 stale row) against the pre-fix code and passes after the fix.

Full test:unit:resources suite passes (1253 passing, 0 failing) after the change.

🤖 Generated with Claude Code

Table.clear() only cleared the primary store, leaving secondary-index
DBIs populated. This doesn't corrupt query results directly (the query
pipeline always re-validates a candidate against the primary store and
skips it if gone), but it does leak: index entries never freed, and if
a caller clears() then re-writes the same id with a DIFFERENT indexed
value (a documented, intended usage), the old value's stale index
entry keeps pointing at that id, so an equality search on the old value
incorrectly returns it.

Fixes it by clearing every index dbi alongside the primary store,
mirroring the clearAsync/clear pattern runIndexing already uses when
rebuilding an index from scratch.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>

@gemini-code-assist gemini-code-assist Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request updates the Table.clear() method to clear all secondary indexes alongside the primary store, preventing stale index entries. It also introduces a regression test suite to verify this behavior. The review feedback correctly identifies a race condition in the new test where only the last asynchronous Tbl.put operation is awaited, which could lead to flaky test runs in slower environments. Awaiting each write sequentially is recommended to ensure test stability.

Comment on lines +42 to +46
let last;
for (let i = 0; i < N; i++) {
last = Tbl.put({ id: 'k-' + i, tag: 'even' });
}
await last;

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

Awaiting only the last promise (last) in the loop does not guarantee that the previous N - 1 asynchronous Tbl.put operations have completed. Since each put runs as an independent immediate transaction, they execute concurrently and can resolve out of order. On slower or resource-constrained CI environments, this race condition can cause the subsequent Tbl.search to find fewer than N records, leading to flaky test failures.

Awaiting each put sequentially in the loop ensures all records are fully written before the search is executed.

		for (let i = 0; i < N; i++) {
			await Tbl.put({ id: 'k-' + i, tag: 'even' });
		}

@claude

claude Bot commented Jul 23, 2026

Copy link
Copy Markdown
Contributor

Reviewed; no blockers found.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants