Skip to content

Clarify snapshot consistency and revalidate example batches - #22

Open
ianmacartney wants to merge 3 commits into
mainfrom
ian/find-snapshot-guarantees
Open

ianmacartney wants to merge 3 commits into
mainfrom
ian/find-snapshot-guarantees

Conversation

@ianmacartney

@ianmacartney ianmacartney commented Sep 18, 2026 •

Copy link
Copy Markdown
Member

Work queries may read an older database snapshot than their worker mutations, while each round sees the committed writes from that worker's previous rounds. The README and API comments distinguish these guarantees: previous deletes and claims are visible to the next query, and re-fetching is needed when processing depends on concurrent changes by other writers. Immutable events can pass their values through the batch, including when the worker deletes them after processing.

The queue, LLM, and e2e examples re-fetch IDs and validate current rows before applying effects. Regression tests cover edits, deletes, and direct replay of previously processed arguments. The aggregate example documents its immutable event assumption and exclusive cursor. Guidance also explains why patching by ID does not validate values returned by an older query.

Validation for the documentation follow-up: npm run build, npm run typecheck, npm run lint, Markdown formatting, and git diff --check passed. All five edited TypeScript files were verified to contain comment-only changes.

@ianmacartney ianmacartney changed the title fix(examples): validate batch candidates in mutation snapshots Handle stale query snapshots and benchmark batch refetching Sep 18, 2026

ianmacartney commented Sep 18, 2026 •

Copy link
Copy Markdown
Member Author

@pkg-pr-new

pkg-pr-new Bot commented Sep 18, 2026 •

Copy link
Copy Markdown

Open in StackBlitz

npm i https://pkg.pr.new/@convex-dev/batch-worker@22

commit: e7d291c

@cubic-dev-ai cubic-dev-ai Bot left a comment •

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

2 issues found across 16 files

Reply with feedback, questions, or to request a fix.

Re-trigger cubic

Prompt for AI agents (unresolved issues)

Check if these issues are valid — if so, understand the root cause of each and fix them. If appropriate, use sub-agents to investigate and fix each issue separately.


<file name="benchmark.mjs">

<violation number="1" location="benchmark.mjs:125">
P2: When a batch-size or padding list contains a duplicate, this line reuses the same `caseId`, so the later per-case check sees twice the expected trials and aborts. Include a unique case index in `caseId` or reject duplicate inputs.</violation>

<violation number="2" location="benchmark.mjs:236">
P2: When the final log chunks are still buffered, this asynchronous kill is followed immediately by writing `logs.jsonl`, truncating the promised raw evidence. Wait for the stream to flush and close before writing the file.</violation>
</file>

Comment thread benchmark.mjs Outdated
Comment thread benchmark.mjs Outdated
@ianmacartney
ianmacartney force-pushed the ian/find-snapshot-guarantees branch from 066836b to 9bef49a Compare September 18, 2026 06:38
@ianmacartney ianmacartney changed the title Handle stale query snapshots and benchmark batch refetching Clarify snapshot consistency and revalidate example batches Sep 18, 2026
@ianmacartney

Copy link
Copy Markdown
Member Author

FYI @mikecann I may not clean this up and land it until I get back, but some of what I said in our interview is true today but may not be true in the future

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant