Skip to content

feat: expose Cell-backed Streams for current table generations - #3

Merged
forhappy merged 7 commits into
mainfrom
codex/cell-streams
Sep 28, 2026
Merged

forhappy merged 7 commits into
mainfrom
codex/cell-streams

Conversation

@forhappy

@forhappy forhappy commented Sep 28, 2026 •

Copy link
Copy Markdown
Contributor

Scope

  • Pin ExtendDB's closed-shard continuation contract to the crabbuild-owned fork commit in fork PR #1. Upstream PR #372 reviews the same commit; BeyondDB does not wait for an official release.
  • Journal item mutations and committed transaction resolution atomically with item changes in account or data Cells. Split imports and rejected operations do not emit records.
  • Expose stream creation, discovery, shard lineage, paginated reads, and terminal continuation through ExtendDB's DynamoDB Streams API.
  • Keep an immutable stream generation catalog when DeleteTable removes the live table record. ListStreams, DescribeStream, and GetRecords can serve the old generation after deletion and name reuse, with 24-hour visibility for catalog entries and records. A deleting routed stream stays pollable until its owners retire.
  • Page ListStreams from the catalog's ordered index, including multiple generations of one table name.

Proof

  • cargo fmt --check
  • cargo build --locked
  • cargo clippy --locked --all-targets -- -D warnings
  • cargo check --locked
  • RUST_MIN_STACK=16777216 cargo test --locked --test account_cell
  • cargo test --locked --test server_binary signed_stream_records_survive_hard_server_restart -- --ignored --nocapture
  • Earlier branch proof: elastic split integration test.

All Cargo commands used this checkout's external CARGO_TARGET_DIR. The build, lint, and both focused tests were rerun after switching to the crabbuild fork URL. The lockfile changed only seven ExtendDB Git sources. The signed process test uses the DynamoDB SDK for CreateTable/PutItem/DeleteTable and AWS CLI for stream discovery and reads, across a hard restart and table name reuse.

Remaining work

This is a draft Streams slice. UpdateTable stream enable/disable and view-type transitions, physical collection of expired catalog and journal rows, TTL service identity, full iterator and response-size qualification, and broad signed SDK coverage remain open. The account catalog still has a finite single-Cell budget. The 10,000-Cell and multi-TB targets are unqualified. The new SQL schema changes the unreleased version-1 digest; BeyondDB has no release tag or upgrade contract yet.

@forhappy forhappy changed the title feat: journal stream changes in owner Cells feat: expose Cell-backed Streams for current table generations Sep 28, 2026
@forhappy
forhappy marked this pull request as ready for review September 28, 2026 06:02
@forhappy
forhappy merged commit b4ab66b into main Sep 28, 2026
1 check passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant