Skip to content

Prototype: dense order key codec for manual ordering - #12284

Closed
rymsha wants to merge 1 commit into
masterfrom
claude/order-key-codec
Closed

Prototype: dense order key codec for manual ordering#12284
rymsha wants to merge 1 commit into
masterfrom
claude/order-key-codec

Conversation

@rymsha

@rymsha rymsha commented Aug 23, 2026

Copy link
Copy Markdown
Contributor

Prototype of the core primitive for a new manual-ordering scheme, developed in design discussion with @sry-enonic. Self-contained in core-internal, nothing wired to storage or API — the point of this PR is the algorithm and its properties, reviewable in isolation.

What it replaces, eventually

_manualordervalue resolves each value against the sibling set and splits fixed gaps:

  • assigning a value queries neighbors (refresh + search on the create path);
  • between() halves a 2^31 gap — 30 targeted inserts into one slot, then it silently mints an exact duplicate of the lower neighbor;
  • the allocator is deterministic, so two branches performing the same insertion mint the identical value by construction and collide when sibling sets merge (observed in production);
  • switching a parent to manual order rewrites every child to seed values.

The primitive

An order key is position '.' discriminator, stored plain, sorted lexicographically (ES not_analyzed string semantics):

  • position — base-62 fraction, digits sort in value order; the separator sorts below every digit so a longer fraction sorts after its prefix, matching numeric order;
  • discriminator — the id of the node holding the key: two live keys are unequal by construction, whatever branch, layer or server minted them — collision-freedom without coordination or a site registry (Logoot with site = the node itself);
  • birth key encodes the inverted creation instant: creation and move-to-first are the same zero-read operation, and un-reordered siblings keep today's newest-first order;
  • between/before/after need only the anchor keys the caller already holds — no sibling load, no rebalancing ever (dense keys), no renumbering of untouched nodes;
  • a directional run consumes space linearly (boundary steps), not by halving: 10 000 inserts in one direction leave the position length unchanged (test);
  • targeted hammering of one gap survives 1000+ levels and then fails with a clean exception at the position-length cap instead of corrupting (test) — versus 30-then-silent-duplicate today.

The token layer

Keys leave the server wrapped: '1' + key + '~' + 16 hex chars of truncated HMAC-SHA512 under a subkey derived once from a master key, bound to the scope (parent) the token was issued under — the same construction RedirectChecksumService already uses, with a derivation step separating the two uses of the master key. Clients can replay positions they were handed — anchors for a reorder — but cannot mint one, so exact placement stays out of reach of standard means. Wrong scope, wrong version, truncation, tampering and oversize are each rejected; the stored key never carries the tag.

The golden-token test pins the exact wire format against an independently computed reference value, so the tag implementation can change without ambiguity about compatibility. (Earlier revisions of this branch used a hand-written SipHash-2-4 for per-item speed; with token minting scoped to reorder-capable listings only, the standard platform MAC costs the same in practice and the hand-written primitive was dropped.)

Not in this PR

Storage wiring, _orderkey mapping/field, ChildOrder integration (the stored childOrder expression is the intended per-parent migration flag), API surface, key-node lookup for the master key (/keys/generic-hmac-sha512), migration of existing manually-ordered parents.

🤖 Generated with Claude Code

https://claude.ai/code/session_01ANkY5TN9xn2cFKHk1sfX4B

@codacy-production

codacy-production Bot commented Aug 23, 2026

Copy link
Copy Markdown

Codacy's Analysis Summary

0 new issue (≤ 0 issue)
0 new security issue
96 complexity
More details

AI Reviewer: run a review on demand. To trigger the first review automatically, go to your organization or repository integration settings. AI can make mistakes. Always validate suggestions.

Run reviewer

TIP This summary will be updated as you push new changes. Give us feedback

A manual order value today is a long resolved against the sibling set:
assigning one queries the neighbors, inserting between two of them
halves a fixed gap that runs out after thirty targeted inserts and then
silently duplicates the lower neighbor, and two branches performing the
same insertion mint the identical value by construction, colliding when
their sibling sets merge.

An order key is a base-62 fraction followed by the id of the node that
holds it. Between two distinct keys there is always a third, so nothing
is ever renumbered; the id suffix makes concurrently minted keys unequal
by construction, whatever branch, layer or server minted them; and a
birth key encodes the inverted creation instant, so creating a node and
moving one first are the same read-free operation, and the relative
operations need only the anchor keys the caller already holds. A run of
inserts in one direction consumes space linearly rather than by halving,
and a position never grows past a cap that turns hostile input into a
clean refusal instead of index growth.

Keys are stored plain and leave the server as tokens: a version
character, the key, and a truncated HMAC-SHA512 tag - the construction
the redirect checksums already use - under a subkey derived once from a
master key and bound to the scope the token was issued for. A client can
replay positions it was handed but cannot mint one, which keeps exact
placement out of reach of standard means, and verification tolerates
fumbles loudly: wrong scope, wrong version, truncation and tampering are
each rejected. A golden token pins the exact wire format against an
independently computed reference value.

Nothing is wired to storage yet: this is the primitive and its
properties, testable in isolation.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01ANkY5TN9xn2cFKHk1sfX4B
@rymsha
rymsha force-pushed the claude/order-key-codec branch from 89c394b to b1a82b7 Compare August 23, 2026 14:11
@codecov

codecov Bot commented Aug 23, 2026

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 85.09317% with 24 lines in your changes missing coverage. Please review.
✅ Project coverage is 87.01%. Comparing base (b0341bd) to head (b1a82b7).

Files with missing lines Patch % Lines
...nonic/xp/core/internal/orderkey/OrderKeyCodec.java 85.12% 9 Missing and 9 partials ⚠️
.../xp/core/internal/orderkey/OrderKeyTokenCodec.java 85.00% 5 Missing and 1 partial ⚠️
Additional details and impacted files
@@             Coverage Diff              @@
##             master   #12284      +/-   ##
============================================
- Coverage     87.01%   87.01%   -0.01%     
- Complexity    20754    20801      +47     
============================================
  Files          2593     2595       +2     
  Lines         69055    69216     +161     
  Branches       5723     5749      +26     
============================================
+ Hits          60091    60226     +135     
- Misses         6290     6307      +17     
- Partials       2674     2683       +9     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

rymsha commented Aug 23, 2026

Copy link
Copy Markdown
Contributor Author

Folded into #12285, which now carries both the codec and its node-layer integration as one PR. Closing.


Generated by Claude Code

@rymsha rymsha closed this Aug 23, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants