Skip to content

docs(rfc):memory capacity and sharding - #1387

Open
MaoMengww wants to merge 2 commits into
oceanbase:masterfrom
MaoMengww:rfc/memory-capacity-and-sharding
Open

docs(rfc):memory capacity and sharding#1387
MaoMengww wants to merge 2 commits into
oceanbase:masterfrom
MaoMengww:rfc/memory-capacity-and-sharding

Conversation

@MaoMengww

Copy link
Copy Markdown
Contributor

Which issue or RFC does this PR close?

Closes #1321

Rationale for this change

Currently, because the manifest persists a full snapshot on every write, storage and write costs grow super-linearly, and in extreme cases the system can run beyond its practical limits. RFC 0014 explicitly listed Memory splitting, inactive tombstone compaction, and a shared routing manifest as unresolved. To contain these extreme cases, this RFC caps unbounded growth while preserving the manifest semantics, offering a concise solution under the conditions it defines.

What changes are included in this PR?

  • Adds synchronized English and Chinese RFC documents.
  • Introduces entry-count-bounded shards (memory_manifest_max_entries, default 200) and a routing domain derived from the configured root ID through strict root.sNNNN suffixes; no schema migration, no entry migration, and no change to the flat-v1 persistence shape.
  • Defines write routing: unconditional pure adds can automatically split to the next-ordinal shard; writes carrying expected_revision, citation-driven revise/retire/reactivate, and organize retain their owner shard and return the stable memory_capacity_exceeded error when capacity is insufficient.
  • Extends flush to single-plan semantics covering the entire routing domain: one candidate extraction over the active union, global deduplication by canonical bytes, and per-shard commits submitted together with the Source cursor in the same outer transaction, protected by a routing-domain barrier.
  • Defines candidate identity as (memory_artifact_id, entry_id) to prevent cross-shard target misidentification.
  • Defines bounded reads through exact immutable per-shard Revision snapshots and cursors that cannot be replayed across routing domains, as well as tombstone compaction audited by drop changes with no cooldown period.
  • Fully documents capacity/compaction, flush transactions and concurrency, API semantics, compatibility, drawbacks, alternatives, prior art, unresolved questions, and acceptance criteria.

Are there any user-facing changes?

No released behavior changes.

How was this change tested?

make check

AI usage statement

Claude Code was used in this change to analyze the Memory service implementation and the related design discussions, and to assist in drafting and cross-checking the synchronized English and Chinese RFC documents. The proposal was submitted by the author for review and revision by maintainers within the RFC process.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

bug: Memory append storage and latency grow superlinearly with entry history

1 participant