Skip to content

Rust #325: encode plain input in document order, record key order - #917

Merged
rmcdaniel merged 4 commits into
mainfrom
rust-325-key-order-followups
Nov 2, 2026
Merged

rmcdaniel merged 4 commits into
mainfrom
rust-325-key-order-followups

Conversation

@rmcdaniel

Copy link
Copy Markdown
Member

Closes two #325 known gaps in docs/rust-server-port.md: Key order inside plain JSON input and the Visibility query key order entry, now named Search attribute validation order.

Plain input. Published PHP 2.11.7 takes a plain input's values in document order (PayloadEnvelopeResolver::resolve, array_values). Avro::toDatum then encodes each nested object as an Avro map in document order. An integer-keyed object that is not a list in document order fails with invalid_map_key; on a start that answers 422 validation_failed under workflow_type. Native encoded nested maps with sorted keys and read {"1":"a","0":"b"} as ["b","a"].

  • New plain_input::ordered applies toDatum's rules in document order, using Rust #325: keep registered query defaults in stored key order #906's preserve_order. The ordered Avro writer then encodes it, so stored blobs are PHP's exact bytes.
  • A non-list integer-keyed object is refused. A start gets PHP's validation failure. Signal, update, query, message, standalone-activity and bridge-start paths keep native's 422 invalid_workflow_arguments, where PHP's signal answers 500 control_plane_internal_error; this difference is recorded in the doc.

Ordered writer. encode_ordered_value used Apache's datum writer over the binding schema and then read its own blob back. On deep-json-bodies' 511-level input those two steps overflowed a 64 MiB stack. The writer now emits the protocol Value's Avro layout directly and skips the read-back, so a 511-level value encodes in a 2 MiB stack. A new unit test checks it byte for byte against Apache's binding writer. The existing test against PHP's published bytes still passes. Query routing, the other caller, writes the same bytes.

Search attribute errors. #906 already made the validator visit request keys in document order. This PR records it: several invalid attributes on a start and on an upsert_search_attributes command are reported in request order.

Fixture key-order, recorded against published 2.11.7 on SQLite:

PHP passed it twice. Native passed it once on SQLite, with a build from before the writer change; the new writer emits the same bytes.

Checks run locally:

  • Rust on SQLite: 376 library and 655 execution tests pass. After merging current main, the codec, plain_input, key_order, deep_json_bodies and query_contract tests pass again.
  • node --test passes for the new contract test, ServerParityDeepJsonBodiesContractTest and ServerParityPartsTest.
  • php -l passes on the probes.
  • test-regression-corpus-policy.py shows the same 46 local failures on a clean origin/main worktree, so they're environmental.

CI covers MySQL, PostgreSQL and the full corpus.

Not closed here: the service-catalog entry "an update that only reorders a document's keys writes nothing". serde_json map equality ignores order even with preserve_order.

🤖 Generated with Claude Code

rmcdaniel and others added 2 commits November 1, 2026 15:06
Published PHP takes a plain input's values in document order and encodes
nested objects as Avro maps in that order. An integer-keyed object that is
not a list in document order fails its encoder with invalid_map_key; a
start answers 422 validation_failed under workflow_type. Native encoded
nested maps with sorted keys and read {"1":"a","0":"b"} as ["b","a"].

Plain input now goes through plain_input::ordered and the ordered Avro
writer, so stored blobs are PHP's bytes. A non-list integer-keyed object is
refused, on a start with PHP's validation failure.

The ordered writer now emits the protocol Value's Avro layout directly and
no longer reads its blob back: Apache's binding writer plus the read-back
overflowed a 64 MiB stack on a 511-level input. A unit test checks the
direct writer byte for byte against Apache's.

The new key-order fixture, recorded against published 2.11.7, covers
nested maps, object input order, the refusal, a signal, and search
attribute errors in request order on a start and an upsert, which #906's
preserve_order already fixed.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@rmcdaniel

Copy link
Copy Markdown
Member Author

Security source review complete for exact head 56fe1f5 (runtime commit 912bdc3). I found no security defect and no Security source hold remains.

The new direct Avro writer is reached from plain JSON input only after the existing request boundary caps the body at 2 MiB and nesting at PHP-compatible depth 511. Conversion and writing stay behind the existing one-permit codec semaphore and blocking task, so the change removes the prior deep Apache writer/read-back stack amplification without opening unbounded parallel work. The implementation writes each protocol union branch, zig-zag long, little-endian double, length-prefixed byte/string, and terminated array/map block directly; the added reference comparison covers empty/nonempty collections, UTF-8, byte lengths, i64 extrema, negative zero, a large double and nested values. Non-finite doubles remain refused.

Plain-input conversion preserves document order but rejects any canonical integer key in a non-list nested object before persistence. The start regression also proves no run is created on that refusal. Top-level object values continue through array-values semantics. The conformance fixture contains synthetic values only, workflow permissions stay contents-read, and no credential, production, or live-provider surface changes.

Local read-only checks: git diff --check passed and the isolated worktree stayed clean. Reviewed SHA-256 values: wire_order.rs 9e9e1344f0be7099802fdd6478e193889ebadd81b7294a359260f8912ddd3a5c; plain_input.rs cb15e2abdce5dc8e3260f180e938d364b1097f189aecedd82880fbbbd2f9f35f; execution.rs 9aa574a139502420603fd4e281a6d830d4826d3baa77a82cca92f6f77cb036cb; key_order.rs 11386d8f752577a865d30c9c31e8b58e05a59629220001720082f70fd01a5527. At review time repository contracts, public-boundary scans and the native debug build were green; remaining exact-head CI is ordinary owner qualification, not a Security blocker.

The corpus requires every raw-HTTP probe that polls to retry the published
PHP reference's documented 503 backend_lock_pressure refusal
(ServerParityRunnerTest).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@rmcdaniel
rmcdaniel added this pull request to the merge queue Nov 1, 2026
@github-merge-queue
github-merge-queue Bot removed this pull request from the merge queue due to a conflict with the base branch Nov 1, 2026
…llowups

# Conflicts:
#	docs/rust-server-port.md
@sonarqubecloud

sonarqubecloud Bot commented Nov 1, 2026

Copy link
Copy Markdown

@rmcdaniel
rmcdaniel added this pull request to the merge queue Nov 1, 2026
@github-merge-queue
github-merge-queue Bot removed this pull request from the merge queue due to failed status checks Nov 1, 2026
@rmcdaniel
rmcdaniel added this pull request to the merge queue Nov 1, 2026
@rmcdaniel

Copy link
Copy Markdown
Member Author

Security focused re-review accepts current exact head 84050b186889a6b4c40a566b94dd9075367f2968; the prior runtime acceptance remains valid and no Security hold remains.

The only PR-specific delta after the accepted 56fe1f57 head is commit 8c6c0ae7, which attaches the already-shared LockPressurePollRetries middleware to the key-order raw HTTP client. This changes conformance transport behavior only, not Server runtime source. The middleware retries only its existing documented lock-pressure conditions, refuses non-replayable bodies, stays within the request timeout, rewinds seekable bodies, and records retries. Other HTTP failures remain visible.

Independent checks: isolated commit inspection and git diff --check passed. The current exact head has all 57 listed checks green, including Rust CodeQL, the backend matrix, Sonar, and the public boundary. No Rust build was run on the lab host.

@github-merge-queue
github-merge-queue Bot removed this pull request from the merge queue due to no response for status checks Nov 1, 2026
@rmcdaniel
rmcdaniel added this pull request to the merge queue Nov 1, 2026
@rmcdaniel
rmcdaniel removed this pull request from the merge queue due to a manual request Nov 1, 2026
@rmcdaniel
rmcdaniel added this pull request to the merge queue Nov 1, 2026
@rmcdaniel

Copy link
Copy Markdown
Member Author

Queue order: dequeued and re-enqueued at the same head 84050b1 (November 1, about 22:45 UTC) so #927 merges first. Merge groups were failing the codec-deadline-health cleanup check on the Ubuntu 26 runner image, which #927 fixes. No change to this PR.

Merged via the queue into main with commit e6886b0 Nov 2, 2026
58 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant