Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
34 commits
Select commit Hold shift + click to select a range
1bd502b
Move this branch to 1.1.1, the build after the OAuth 2.0 API's 1.1.0
James-Chapman Nov 16, 2026
85cbe8a
Test use_state_after and state_after on /sync before implementing them
James-Chapman Nov 16, 2026
5eefc8e
Answer state_after on /sync when use_state_after is true
James-Chapman Nov 16, 2026
134bce3
Format the use_state_after QueryParams scenario
James-Chapman Nov 16, 2026
f8633be
Document use_state_after on /sync and the 1.1 series start (ADR-0762 …
James-Chapman Nov 16, 2026
2080b55
Insert the test's second forward extremity through the table's insert
James-Chapman Nov 16, 2026
13173b8
Add the 1.0.3 database upgrade fixture, written by the 1.0.3 release
James-Chapman Nov 16, 2026
44cfe4a
Open, migrate and serve the 1.0.3 database in the upgrade flow tests
James-Chapman Nov 16, 2026
234e032
Document the 1.0.3 upgrade fixture and its scenarios
James-Chapman Nov 16, 2026
291d19d
Find the state anchor by a seek, not a walk past the rows above the p…
James-Chapman Nov 16, 2026
08a04df
Send a left room's whole readable state_after when the join began aft…
James-Chapman Nov 16, 2026
b0ca0c3
Move the use_state_after scenarios that assert project choices out of…
James-Chapman Nov 16, 2026
df7fa64
Correct the sync row of the capability gaps and track the state_after…
James-Chapman Nov 16, 2026
8982bba
Keep the capability-gaps changelog bullet within the line limit
James-Chapman Nov 16, 2026
5e2dda7
Drop the 1.1 series-start ADR and move the use_state_after ADRs above…
James-Chapman Nov 16, 2026
a29d73d
Make the 1.0.3 upgrade fixture scenarios prove the v1 to v2 migration
James-Chapman Nov 16, 2026
ea9b976
Format the upgrade fixture scenarios
James-Chapman Nov 16, 2026
433ad83
Add server.turn.uris so voip/turnServer can return several TURN URIs
James-Chapman Nov 16, 2026
acf25cb
Advertise MatrixRTC LiveKit transports from server.rtc.livekit.servic…
James-Chapman Nov 16, 2026
df6419b
Hold server.turn.uris to the RFC 7065 grammar and refuse a service_ur…
James-Chapman Nov 16, 2026
038bb21
Test the MatrixRTC routes against voip/turnServer for methods and mod…
James-Chapman Nov 16, 2026
02ecb19
Record MatrixRTC transport discovery in the threat model
James-Chapman Nov 16, 2026
d4763fa
Include the Catch2 generators header the moderated-account scenario n…
James-Chapman Nov 16, 2026
f4cd774
Require a bracketed TURN host to be a valid IPv6 address
James-Chapman Nov 16, 2026
cd0cd41
State the use_state_after fork limit in the 1.1.1 release notes
James-Chapman Nov 17, 2026
8bd9914
Name the upgrade fixture's scratch files uniquely and check their writes
James-Chapman Nov 17, 2026
ebe8bd3
Require JSON members and variant pointers to exist before the TURN an…
James-Chapman Nov 17, 2026
d7730e8
Report a room's state or state_after as present whatever type its val…
James-Chapman Nov 17, 2026
a110e5f
Put the config parse in the WHEN of the RTC and TURN URI unit scenarios
James-Chapman Nov 17, 2026
2c39903
Annotate the state_after conformance file as the other conformance fi…
James-Chapman Nov 17, 2026
5039845
Prove the 1.0.3 sync token stays incremental and its stream token epo…
James-Chapman Nov 17, 2026
1fde7d5
Fit the 1.1.1 test bug-fix bullet to the changelog format's two lines
James-Chapman Nov 17, 2026
cec01d8
Explain what the long state_after, TURN URI, RTC and anchor scenarios…
James-Chapman Nov 17, 2026
a04ef00
Require the required state section to be an object holding an events …
James-Chapman Nov 17, 2026
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion AGENTS.md
Original file line number Diff line number Diff line change
Expand Up @@ -224,7 +224,7 @@ the same name in `include/`: `src/appservice/RegistrationYaml.cpp`, `src/databas
│ ├── smoke/ # Smoke tests
│ ├── password_cost/ # Proves the production Argon2id cost, in a binary without the tests' cheap hash
│ ├── complement/ # Docker image that runs the release package under Complement
│ ├── fixtures/ # Test fixtures: complement/ (JSON flow fixtures) and upgrade/ (README only; the 1.0.x cross-release database fixture is not generated yet, the build VM has no PostgreSQL server)
│ ├── fixtures/ # Test fixtures: complement/ (JSON flow fixtures) and upgrade/ (the 1.0.3 release's SQLite database, PostgreSQL dump and manifest)
│ ├── support/ # Test helpers (JsonTestSupport, MasterKey, RegistrationToken, TempDirectory, TlsMockServer)
│ ├── sanitizer/ # Sanitizer suppression files (tsan.supp)
│ └── tooling/ # Python tests for CI workflows, dependency wraps, example config, logger filter
Expand Down
38 changes: 38 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -3,6 +3,44 @@ Changes from 1.0.0, the first production release, onwards; the format is set out
Changes before 1.0.0 are in [CHANGELOG-pre-release.md](CHANGELOG-pre-release.md), which is no longer added to
([ADR-0740](docs/adr/0740-the-changelog-starts-at-1-0-0-and-the-pre-release-entries-are-a-frozen-record.md)).

# Version 1.1.1

## Features

* `GET /_matrix/client/v3/sync` honours `use_state_after` (Matrix v1.16): with it true each joined and left room
carries `state_after` and no `state` (ADR-0818, ADR-0819, ADR-0820).
* The cross-release upgrade fixture `tests/fixtures/upgrade/1.0.3/`, written by the 1.0.3 release, is opened, taken
through migration 002 and served on SQLite and PostgreSQL by the `[fixture]` scenarios (ADR-0216).
* MatrixRTC transport discovery (MSC4143): `server.rtc.livekit.service_url` is advertised as a `livekit` transport
on `/rtc/transports`, `/versions` and the well-known document (ADR-0822).
* `server.turn.uris` lists several TURN URIs for `GET /voip/turnServer`, in order; `server.turn.server` still works and
both together are refused (ADR-0821).
* The `use_state_after` scenarios asserting project choices, not spec rows, moved from the conformance suite to
`tests/integration/test_sync_state_after_flow.cpp`.

## Security Fixes

* None.

## Bug fixes

* `use_state_after`: the state anchor at `since` is a binary search of the room's index, so a client back after a long
absence no longer walks every row it missed under the runtime lock.
* `use_state_after`: a room joined and left inside one sync window now gets its whole readable state as of the leave in
`state_after`, not a delta that hid the create event, power levels and join rules (ADR-0820).
* The sync row of `docs/todos/capability-gaps.md` no longer says a limited sync gives no gap state, and tracks the
`state_after` limit after a state fork (ADR-0819).
* Tests: the upgrade fixture scenarios prove the 1.0.3 sync token stays incremental and its epoch survives
migration; scratch files are uniquely named; a missing JSON field fails one scenario, not the binary.

## Release notes

1.1.1 adds `use_state_after` to `/sync`, `server.turn.uris` and MatrixRTC transport discovery. The new keys
(`server.turn.uris`, `server.rtc.livekit.service_url`) are optional; no configuration change is needed. Group calls
still need your own LiveKit SFU and `lk-jwt-service`, and delayed events (MSC4140) are not implemented. Known limit
of `use_state_after`: after a state fork at the client's `since`, a key that state resolution later flips back to the
value recorded at `since` is not sent in `state_after`; a client repairs itself with `full_state=true` (ADR-0819).

# Version 1.1.0

## Features
Expand Down
14 changes: 13 additions & 1 deletion config/merovingian.conf.bsd.example
Original file line number Diff line number Diff line change
Expand Up @@ -131,18 +131,30 @@ server.http.ipv6_client_prefix_length=64
# server.identity_server.total_timeout_seconds=30

# TURN server for VoIP relay (optional)
# When server.turn.server is empty, GET /_matrix/client/v3/voip/turnServer
# When server.turn.server and server.turn.uris are both empty, GET /_matrix/client/v3/voip/turnServer
# returns an empty object so clients gracefully disable relay support.
# Either static credentials, issued to every user (only supply credentials that
# are acceptable for all users of this homeserver), or a coturn shared secret
# (use-auth-secret): each user then gets time-limited credentials of their own.
# Set one or the other; both is refused.
# server.turn.server=turn:turn.example.org:3478?transport=udp
# Or several URIs, comma-separated, in the order clients should try them (instead of server.turn.server; both is
# refused). Each must be turn: or turns:, a host, an optional :port and an optional ?transport=udp or ?transport=tcp
# (RFC 7065), for example turn:[2001:db8::1]:3478.
# server.turn.uris=turn:turn.example.org:3478?transport=udp,turns:turn.example.org:5349?transport=tcp
# server.turn.username=merovingian
# server.turn.password=set-in-owner-only-secret-file
# server.turn.shared_secret_file=@SYSCONFDIR@/merovingian/turn-secret
# server.turn.ttl_seconds=86400

# MatrixRTC group calls (optional)
# The HTTPS URL of an lk-jwt-service (Element's LiveKit authorisation service) that you run beside a LiveKit SFU.
# This server never contacts it: it only tells authenticated clients, and the /.well-known/matrix/client
# document, where it is. Empty means no group-call transport is advertised. Reloadable.
# Clients append /sfu/get or /get_token to it, so write it with no trailing slash and no query: a URL ending in /
# or holding ? is refused. https://host, https://host:8443 and https://host/jwt are fine.
# server.rtc.livekit.service_url=https://livekit-jwt.example.org/lk-jwt-service

# Listener exposure
# Keep TLS disabled only on loopback binds behind a declared reverse proxy.
# If binding to a public interface, set tls=true, reverse_proxy=false, and
Expand Down
14 changes: 13 additions & 1 deletion config/merovingian.conf.example
Original file line number Diff line number Diff line change
Expand Up @@ -104,18 +104,30 @@ server.http.ipv6_client_prefix_length=64
# server.identity_server.total_timeout_seconds=30

# TURN server for VoIP relay (optional)
# When server.turn.server is empty, GET /_matrix/client/v3/voip/turnServer
# When server.turn.server and server.turn.uris are both empty, GET /_matrix/client/v3/voip/turnServer
# returns an empty object so clients gracefully disable relay support.
# Either static credentials, issued to every user (only supply credentials that
# are acceptable for all users of this homeserver), or a coturn shared secret
# (use-auth-secret): each user then gets time-limited credentials of their own.
# Set one or the other; both is refused.
# server.turn.server=turn:turn.example.org:3478?transport=udp
# Or several URIs, comma-separated, in the order clients should try them (instead of server.turn.server; both is
# refused). Each must be turn: or turns:, a host, an optional :port and an optional ?transport=udp or ?transport=tcp
# (RFC 7065), for example turn:[2001:db8::1]:3478.
# server.turn.uris=turn:turn.example.org:3478?transport=udp,turns:turn.example.org:5349?transport=tcp
# server.turn.username=merovingian
# server.turn.password=set-in-owner-only-secret-file
# server.turn.shared_secret_file=/etc/merovingian/turn-secret
# server.turn.ttl_seconds=86400

# MatrixRTC group calls (optional)
# The HTTPS URL of an lk-jwt-service (Element's LiveKit authorisation service) that you run beside a LiveKit SFU.
# This server never contacts it: it only tells authenticated clients, and the /.well-known/matrix/client
# document, where it is. Empty means no group-call transport is advertised. Reloadable.
# Clients append /sfu/get or /get_token to it, so write it with no trailing slash and no query: a URL ending in /
# or holding ? is refused. https://host, https://host:8443 and https://host/jwt are fine.
# server.rtc.livekit.service_url=https://livekit-jwt.example.org/lk-jwt-service

# Listener exposure
# Keep TLS disabled only on loopback binds behind a declared reverse proxy.
# If binding to a public interface, set tls=true, reverse_proxy=false, and
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,41 @@
# `state_after` is the change since `since`, as of the response's position

* Status: accepted
* Date: 2026-11-16

## Context and Problem Statement

`GET /sync` with `use_state_after=true` answers `state_after`: "Updates to the state, between the time indicated by the
`since` parameter, and the **end** of the `timeline`", and a client "MUST only update their local state with events in this
list, and MUST NOT update their local state with events in `timeline`". The spec leaves open which point is the end of the
timeline when the timeline filter hides events, and how much a `limited` sync sends.

## Considered Options

* The end is the state recorded for the last delivered timeline event (rejected, see below)
* The end is the response's own position, `sync_position`, the point `next_batch` names
* A `limited` sync sends the whole state at the end of the timeline, as `state` does at its start (rejected, see below)
* A `limited` sync sends the changes since `since`, like any other incremental sync

## Decision Outcome

Chosen options: the response's position, and the changes since `since`.

The last delivered event is the wrong point because `next_batch` is past it. With `not_types: ["m.room.topic"]` on the
timeline, a topic change after the last delivered message lies before `next_batch`. The next sync starts after it, and the
client takes no state from the timeline, so a `state_after` as of that message would lose the change for good. Without a
filter the two points hold the same state, so this reading never contradicts the spec.

`state` sends the whole state on a `limited` sync because a client cannot otherwise tell what the gap changed. `state_after`
has no such gap: the delta ([ADR-0819](0819-a-state-after-delta-compares-the-two-states.md)) covers `since` to the position
whatever the timeline's size, so whole state would only repeat what the client has, on the requests that are already
largest. Whole state remains for what the spec names (no `since`, `full_state=true`) and for a room joined in the window,
where the client holds nothing to apply a delta to.

The parameter is parsed like `full_state`: `true` turns it on, anything else leaves it off. With it off, or absent, no
response changes. Sliding sync (MSC4186) has its own state model and is not changed.

### Consequences

* `state_after` can name an event after the last timeline event, when the timeline filter hid it. That event is not an
update a client could have got any other way.
59 changes: 59 additions & 0 deletions docs/adr/0819-a-state-after-delta-compares-the-two-states.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,59 @@
# A `state_after` delta compares the two states, and every change it finds is sent under lazy-loading

* Status: accepted
* Date: 2026-11-16

## Context and Problem Statement

An incremental `state_after` holds the state that differs between `since` and the response's position. State resolution can
move a key back to an older event: no event of the window is about that key, and its current event is older than `since`.
This is the case `use_state_after` exists for, since a client reading `state` and `timeline` cannot see it. `/sync` runs under
`runtime.mutex` for every syncing user, so the cost matters as much as the answer
([ADR-0435](0435-the-event-table-keeps-a-per-room-index-in-stream-order.md),
[ADR-0436](0436-an-event-is-judged-for-ignored-senders-without-parsing-it-first.md)).

## Considered Options

* List the state events in the window and send each key's current value (rejected: misses a key moved back)
* Read both whole states for every room with a new event (rejected: a room's state, twice, per user, per message)
* Compare state groups: nothing when they are one group, the rows between them on an unforked delta chain, and both whole
states only when the chain does not connect them

## Decision Outcome

Chosen option: compare state groups (`joined_state_changes` in `ClientServerRuntime.cpp`, `state_changes_along_chain` in
`StateGroupLookup.cpp`).

* A room with no event above `since` costs one look at its newest row and sends nothing.
* The state at `since` is the recorded state of the newest event at or below it (`state_anchor_event`, found by a binary
search of the room's stream-order index, so the rows a long absence missed cost nothing; a test reads
`state_anchor_rows_examined`). With one forward
extremity, at or below the position, the current state is that event's recorded state, so the two points are two groups:
the same group for a window of messages (no rows read), otherwise the rows between them
(`state_changes_along_chain`, which never reads a snapshot's rows). The extremity comes from the room's own tips
(`ForwardExtremityTable::tips_of_room`), never from a walk of every room's.
* A fork, a snapshot between the two points (at most `max_state_group_delta_depth` hops apart) or an unreadable chain falls
back to the state at the position minus the state at `since`, the room's own rows. An event is the value of one key, so
subtracting event IDs finds exactly the keys that differ.
* With no recorded state at `since`, all the state at the position is sent: an update the client holds is harmless.

Lazy-loading: `state_after` carries, besides all non-member changes, every member event in the delta (never redundant, in
the timeline or not, about a sender or not, since the client takes no state from the timeline), and each timeline sender's
member event as of the position, by key and subject to redundancy (`member_event_at_position`, with the cost rule of
[ADR-0454](0454-a-senders-member-is-looked-up-by-key-never-by-scanning-the-room-state.md)). The user's own member event on
an initial sync, `full_state=true` or a join in the window is sent even when the timeline carries the join, which
[ADR-0452](0452-the-state-filter-selects-lazy-loaded-members-on-sync.md) allows only for `state`.

The tracker's rules ([ADR-0450](0450-lazy-loaded-members-are-skipped-only-when-the-since-token-acknowledges-them.md)) are
unchanged, and it is shared with `state` responses under the same stream key. It compares event IDs: a client that switches
modes either holds the event named or is sent it again. What `state_after` sends is recorded as what `state` sends is.

### Consequences

* The state at `since` is what was recorded for one event, not what the response carrying that token said. If the graph was
forked at `since`, the resolved state the client was told can differ from that recorded state, and a key that resolution
later flipped to the recorded value is not sent. A backfilled event ordered at or below `since` can stand in the same way.
Storing each response's state per token would close this and was refused for its cost; `full_state=true` repairs a client.
* A window that crosses a snapshot reads both whole states, about once per `max_state_group_delta_depth` state changes. It
sends only what differs.
* A key that changed and changed back within the window may be sent, with the value the client already holds.
Original file line number Diff line number Diff line change
@@ -0,0 +1,49 @@
# A left room's `state_after` stops at what the user may read, and always holds their own membership

* Status: accepted
* Date: 2026-11-16

## Context and Problem Statement

A left room's `state_after` is "The state updates for the room up to the **end** of the timeline", and that timeline runs
"up to the point when the user left". A left room's incremental `state` holds only the leave, and only when the timeline does
not carry it ([ADR-0453](0453-sync-honours-full-state.md) kept that). A `state_after` client takes no state from the
timeline, and a user who left may not read the room after they left, whatever a later ban changes
([ADR-0357](0357-a-left-rooms-sync-state-is-gated-by-room-read-access.md)).

## Considered Options

* Reuse the joined room's delta, against current state (rejected: it tells a departed user what happened after they left)
* Keep `state`'s rule and send the leave alone on an incremental sync (rejected: "the state updates up to the end of the
timeline" hold what changed before the leave too)
* Read the state only as far as `room_read_access_for` allows, take away what the client held at `since`, and add the user's
own membership event

## Decision Outcome

Chosen option: the last (`build_left_room_state_after`).

* The room's state comes from the state anchor of `room_read_access_for`, capped at the event that ended the join.
* The user's own key is always the current member event, sent even when the timeline carries it. For a user who left and
was banned since, the room is as of the leave and the membership is the ban, the one thing the room's state may say about
after the leave.
* A `since` at or past the end of what the user may read (they left before it, and were banned after it) leaves only that
membership event.
* A user who never joined (`none`) gets only their own event, as on `state` (ADR-0357).
* A client holds the room's state at `since` only if it was joined then. When the join this departure ended began after
`since` (`RoomReadAccess::join_ordering` above `since`: a join and leave inside one window, or a rejoin after an earlier
departure), nothing is taken away and the whole readable state as of the leave is sent, as an initial sync would, still
capped by `room_read_access_for`; lazy-loading then treats the room as new to the client (the tracker is not consulted).
The join's ordering comes from the walk of the user's member events that already finds the boundary, so no scan of the
room's rows decides it. It is the rule `joined_membership_changed_since` applies to a joined room, read from the join
event because a departed user's current member event is the leave. Subtracting what the client "held" would have hidden
the create event, power levels and join rules from a client that is never told them another way.
* `full_state=true` does not make a left room's `state_after` whole: `full_state` is about "all rooms the user is a member
of" and the Left Room `state_after` does not name it, as ADR-0453 has for `state`.
* With lazy-loading, a sender's member event is looked up as of the leave. When the state group chain cannot be read it is
omitted, while a joined room lets the current member event stand in: omitting a member is a smaller harm than telling a
departed user a later one.

### Consequences

* A left room's state always costs two whole reads, of that room's rows. Leaves are rare and once per room per client.
Loading
Loading