Skip to content

batch_publish.md mocks the pre-version-3 batch response for every RSC22 test and pins X-Ably-Version: 2; two revoke_tokens.md mocks do the same #559

Description

@owenpearson

Summary

rest/unit/batch_publish.md is written against the batch response format the server sends
below protocol version 3: a flat array of per-channel results, with any failure turning the
whole response into a 400. Its two siblings, rest/unit/batch_presence.md (:17-43) and
rest/unit/auth/revoke_tokens.md (:12-38), each open with a "Server Response Format"
section saying that at X-Ably-Version >= 3 every batch response is a BatchResult
envelope. The sandbox agrees with them.

So an SDK that implements RSC22 as features.md writes it — sending the CSV2b protocol
version and returning BatchResults — cannot pass batch_publish.md without rewriting its
mocks. An SDK that passes them as written is parsing a response it will never receive.
revoke_tokens.md has the same fault in two of its own mocks, despite its header.

Line references are against 747796f, which is current main; paths are relative to uts/.


What the server sends

Measured against the sandbox on 2026-10-08, basic auth, JSON and msgpack alike. One key has
full capability; a second can publish to one channel and is refused 40160 on another, which
gives the mixed rows.

X-Ably-Version POST /messages, every channel succeeds POST /messages, one channel refused GET /presence, one channel refused
none, 1, 2 201, one flat array of {channel, messageId} across every spec, no serials 400 40020 "Batched response includes errors", the flat array as batchResponse 400 40020, batchResponse
3, 4, 5, 6 201, an array of {successCount, failureCount, results} envelopes, one per spec 201, the same 200, one envelope

Three details matter for the fixtures:

  • A single spec sent as a bare JSON object is answered exactly like an array of one: an
    array holding a single envelope. This is the array RSC22b describes ("This is not a
    feature of the REST API, whose response will still be an array, so if implementing this
    overload, the SDK will have to extract the element from the array") — an array of
    BatchResults, one per spec, not of per-channel results.
  • The legacy format has no serials. At version 2 a success entry is
    {"channel": …, "messageId": …} and nothing else. batch_publish.md's mocks attach
    serials to legacy-shaped entries, a combination no version of the server sends.
  • Legacy concatenates specs. Two specs at version 2 come back as one flat array of
    three entries (2 channels + 1 channel), with nothing marking which spec each belongs to.

Trimmed raw output, a single spec as a bare object, on a mixed batch:

--- X-Ably-Version: 5 → HTTP 201
[{"results": [{"channel": "channel6", "messageId": "DtlHXJK45S:0",
               "serials": ["01791462730867-000@e02yvmjLAC7g5n77110453:000"]},
              {"channel": "denied-N5i1WQmU",
               "error": {"code": 40160, "statusCode": 401, "message": "Unauthorized to publish to channel", …}}],
  "successCount": 1, "failureCount": 1}]

--- X-Ably-Version: 2 → HTTP 400
{"error": {"code": 40020, "statusCode": 400, "message": "Batched response includes errors", …},
 "batchResponse": [{"channel": "channel6", "messageId": "XqypqlUeVF:0"},
                   {"channel": "denied-N5i1WQmU", "error": {"code": 40160, …}}]}

1. rest/unit/batch_publish.md — every response fixture is in neither format

The file has no "Server Response Format" section. Its mocks take two shapes:

  • A bare per-channel result as the whole response body. RSC22c3 (:61-80,
    rest/unit/RSC22c/single-spec-single-result-0), and all six BPR/BPF sections
    (:249-367: BPR2a/success-channel-name-0, BPR2b/success-message-id-prefix-0,
    BPR2c/serials-array-0, BPR2c/serials-null-conflated-0,
    BPF2a/failure-channel-name-0, BPF2b/failure-error-info-0). For example, RSC22c3:

    And the mock is configured to respond with:
      {
        "channel": channel_name,
        "messageId": "msg123",
        "serials": ["serial1"]
      }
    When batchPublish is called with a single BatchPublishSpec
    Then a single BatchResult is returned (not an array)
    

    Neither format ever answers with a bare object. The section's own spec requirement — "the
    response is a single BatchResult (not an array)" — describes the SDK's return value, but
    RSC22b says the wire response "will still be an array".

  • A flat array of per-channel results. RSC22c4 (:82-102), and BatchResult1 (:373-395,
    rest/unit/RSC22c/partial-success-mixed-results-0), whose mixed success-and-failure
    array is answered by a version-2 server with a 400/40020, never a success:

    And the mock responds with mixed results:
      [
        { "channel": channel_name_allowed, "messageId": "msg1", "serials": ["s1"] },
        { "channel": channel_name_restricted, "error": { "code": 40160, ... } }
      ]
    

    RSC22c4 then asserts "an array of BatchResults is returned / And each result corresponds
    to the respective spec". From a flat array that cannot be done, because the server's
    legacy array does not mark spec boundaries.

RSC22c5 (:103-120) and RSC22_Batch2 (:541-560) say only "respond with results for each
channel", which leaves the shape to whatever the rest of the file implies.

Suggested fix. Add a "Server Response Format" section mirroring batch_presence.md:17-43:

With X-Ably-Version >= 3 (sent by all current SDKs), POST /messages returns HTTP 201 and
an array holding one BatchResult envelope per BatchPublishSpec sent — a single spec sent
as a bare object is answered with an array of one. All-success, mixed and all-failure
batches all return 201 in this format.

Then wrap each fixture in that shape. RSC22c3 becomes:

[{ "successCount": 1, "failureCount": 0,
   "results": [{ "channel": channel_name, "messageId": "msg123", "serials": ["serial1"] }] }]

RSC22c4 becomes an array of two envelopes, one per spec. BatchResult1 becomes a single
envelope with successCount: 1, failureCount: 1. The BPR/BPF sections become a single
envelope, read through result.results[0]. None of the assertions needs to change except
where a section reads the result object directly rather than through results.

2. rest/unit/batch_publish.md:497 — RSC22_Headers1 pins X-Ably-Version: 2

Then the captured request includes:
  - X-Ably-Version: 2
  - Ably-Agent: <library-agent-string>
  - Content-Type: application/json

RSC7e sends the CSV2b version (features.md:95), and version 2 is the one that gives the
legacy responses the rest of the file mocks. rest/unit/rest_client.md's RSC7e test (:67-68) checks
the header against a pattern rather than a literal, which is the same check this one needs.

The Content-Type: application/json line is the unpinned-protocol fault #527 already
records for batch_publish.md's RSC22c6. useBinaryProtocol appears nowhere in the file, so
TO3f makes the request msgpack.

Suggested fix. X-Ably-Version matching [0-9.]+, as at rest/unit/rest_client.md:68, or an
explicit >= 3. Then either pin useBinaryProtocol: false or assert the content type that
matches the configured protocol.

3. rest/unit/auth/revoke_tokens.md — two mocks stub a bare array under an envelope header

The file's header (:12-38) describes the envelope and says "All success returns HTTP 201
with this format". Two of its mocks then stub the legacy bare array with HTTP 200, while
asserting envelope fields:

  • RSA17c_1 (:179-213, rest/unit/RSA17c/all-success-result-0) responds
    200, [ {target: "clientId:alice", …}, {target: "clientId:bob", …} ] and asserts
    result.successCount == 2, result.failureCount == 0, result.results.length == 2.
  • TRS2_1 (:305-337, rest/unit/TRS2/success-result-attributes-0) responds with a
    one-element bare array and reads result.results[0].

RSA17c_2 (:214), in between, already stubs the envelope. Measured against the sandbox,
POST /keys/{keyName}/revokeTokens at version 5 answers 201 with the envelope for an
all-success batch and for a mixed one, as the header says. An SDK that parses the envelope
fails both tests on their fixtures; one that passes them is computing counts the server
already sends.

Suggested fix. Wrap both fixtures as RSA17c_2 does, with status 201.


Related

Found while implementing RSC22/RSC24 for ably-python.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions