feat: remove and replace indexes in place (sde-index protocol 2)

[krzysztof-smartdataengines]

Remove and replace indexes in place: sde-index protocol 2

A design that drops an index in force - one no read uses while every write maintains it, or one
replaced by a better one - used to need a fresh copy and a cutover, whose write pause grows with the
table (past about 400 000 rows on PostgreSQL it exceeds the cutover's 30 s budget and rolls back).
Protocol 2 of the signed in-place authorization removes indexes on the live tables instead.

Measured before the design

docs/qualification/in-place-index-drop/ (PostgreSQL 15.19, ClickHouse 24.8.14.39, a writer
inserting every 5 ms):

• DROP INDEX CONCURRENTLY on 100 000 rows: 6.3 ms, longest gap between writes 7.9 ms (10.2 ms
  before). ClickHouse ALTER TABLE ... DROP INDEX ... SETTINGS alter_sync = 0: 13.2 ms, gone from
  the catalogue at once, reads still answer.
• A concurrent drop waits for transactions holding a lock on the table - an open transaction that
  has read it - and not for older snapshots as a concurrent build does (a snapshot alone: the
  drop finished in 6.5 ms). The first run of the probe had conflated the two; the record says so.
• A stopped drop leaves the index valid = false, ready = true (unused, still maintained); a
  second drop removes it.

What changed

• Packet (both loaders): protocol 2 - the prepared indexes are the ones in force without the
  removed ones, byte for byte and in order, followed by new ones (possibly none); at least one is
  removed. IndexPlan.removed, IndexPlan.protocol, INDEX_CHANGE_PROTOCOL. Protocol 1 is
  unchanged. Vectors migration/180-187 (acceptances and a refusal per changed rule);
  migration/149 now refuses protocol 3. 342 vectors.
• Operator: every removed index is checked on its table in the declared shape before any DDL;
  what is new is built and qualified as before; the decision is recorded and the next map published;
  only then each index is removed, one resumable step each - DROP INDEX CONCURRENTLY IF EXISTS (the IF EXISTS only for an earlier drop, stopped client-side by the budget, finishing in
  the server meanwhile), on ClickHouse a pending materialization killed and DROP INDEX with
  alter_sync = 0; the catalogue is read back. After the decision an absent index or another object
  under the name means ours is gone, and the other object is left alone.
• Receipt: protocol 2 adds removed - a row per removed index once built, none once abandoned.
• State: a project directory holding a protocol-2 record uses storage contract 5; operators
  knowing contracts 1 to 4 refuse it (measured on the real previous operator: exit 2, the directory
  unchanged).
• Docs: docs/in-place-index.md (protocol 2, removal, evidence), docs/format-contract.md §7j,
  docs/physical-design.md, conformance/README.md, README.

Evidence

• python/tests/test_index_change_live.py on both engines: a removal while a process on the map in
  force writes; a replacement; recovery after every step after the decision (and nothing of the map
  in force touched before publication); abandonment before the decision removing nothing in force;
  an index in force absent, of another shape or another sort order refused before any DDL; a removal
  held by an open reader until the budget ends and resumed; a resumed removal waiting behind the
  stopped drop that then finishes first; a killed operator mid-removal; a foreign object under the
  name after the decision left alone; a drop that leaves its index not reported done; a pending
  ClickHouse materialization of the removed index killed.
• python/tests/test_cutover_project.py: state contract 5 written and required.
• make check with both live engines on this head (fd54687): Python 2010 passed and the ten
  optional orderbook skips, TypeScript 953 passed.
• Mutations: 21 entries - 19 killed as expected, both controls surviving.

🤖 Generated with Claude Code