Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
29 commits
Select commit Hold shift + click to select a range
0e9091d
Bump the version to 1.1.2 for delayed events
James-Chapman Nov 17, 2026
affbc57
Record the delayed-events decisions (ADR-0823 to ADR-0826)
James-Chapman Nov 17, 2026
b6b5f0c
Add migration 003 and the delayed-events table, store functions and h…
James-Chapman Nov 17, 2026
31ac219
Document migration 003 and the delayed-events table
James-Chapman Nov 17, 2026
ffa671f
Add the server.delayed_events.* configuration and snapshot it into th…
James-Chapman Nov 17, 2026
bd65c0b
Document the server.delayed_events.* keys in the user manual and exam…
James-Chapman Nov 17, 2026
5f5c835
Refuse a NUL byte in any text field of a delayed-event row
James-Chapman Nov 17, 2026
2b34b20
Add the delayed-events service functions and the shared send-path hel…
James-Chapman Nov 17, 2026
be9f60a
Run the delayed-events job, delete a deactivated account's rows and s…
James-Chapman Nov 17, 2026
64ac82f
Document delayed events in the changelog, API reference, threat model…
James-Chapman Nov 17, 2026
18c3f3a
Add the built-in delayed-event rate limits with their unit test, Comp…
James-Chapman Nov 17, 2026
96150e6
Add the delayed-events routes, the /versions and /capabilities entrie…
James-Chapman Nov 17, 2026
a8a6bcc
Describe delayed events in the user manual's calls section
James-Chapman Nov 17, 2026
0c16373
Make the being-sent mark exception-safe, log a failed prune, and refu…
James-Chapman Nov 17, 2026
fe5d5ea
Document that every body-sending route refuses the delay parameter
James-Chapman Nov 17, 2026
a79a715
Bring the delayed-events documents in line with the review's rulings
James-Chapman Nov 17, 2026
87700e2
Pace the delayed-events job to one send a run and keep the mark when …
James-Chapman Nov 17, 2026
de14cfd
Read delayed-event rows strictly at hydration, count the skipped ones…
James-Chapman Nov 17, 2026
faad02a
Test the one-send pacing, the strict hydration, the review's route ca…
James-Chapman Nov 17, 2026
a88e75d
Say which logger writes the skipped-rows start-up warning
James-Chapman Nov 17, 2026
236bc34
Count the delayed-events job's wait from when its run ends and skip d…
James-Chapman Nov 17, 2026
58255dd
Put the parse in WHEN in the delayed-events config tests
James-Chapman Nov 17, 2026
45b9d5e
Say what disabling delayed events leaves running and that the prune b…
James-Chapman Nov 17, 2026
631f247
Test the delayed-events failure paths no scenario ran
James-Chapman Nov 17, 2026
5a8b5db
Say in the 1.1.2 release notes that an older server refuses the migra…
James-Chapman Nov 17, 2026
1642111
Load the delayed-events rows through a named helper instead of a shor…
James-Chapman Nov 17, 2026
f601d24
Explain what the long delayed-events scenarios prove and why their st…
James-Chapman Nov 17, 2026
c72562a
Explain the skip rule and the row builder of the PostgreSQL reopen sc…
James-Chapman Nov 17, 2026
adf9d8e
Load delayed events outside the loader's chain and shorten the Postgr…
James-Chapman Nov 17, 2026
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
Expand Up @@ -112,6 +112,9 @@ look at every time. Read the `AGENTS.md` itself before judging a change to that
an OAuth client that has a session (ADR-0776, ADR-0777). Whatever replaces a device's credentials replaces its
`oauth_sessions` row in the same transaction; a grant's audit row is in its transaction (ADR-0792).
`src/database/AGENTS.md#OAuth 2.0 tables (migration 002, ADR-0776, ADR-0777)`
- The finalised-row bound and the 7-day prune apply to finalised `delayed_events` rows only; a scheduled row is bounded
by the per-user scheduled cap, never pruned by age. Only `DelayedEventTable` touches the rows, and the in-memory mirror changes only after the commit (one exception, ADR-0825). A path that deletes an
account's data deletes these rows too (ADR-0824).

## src/email

Expand Down Expand Up @@ -197,6 +200,10 @@ look at every time. Read the `AGENTS.md` itself before judging a change to that
API's own function; a password change, reset or deactivation calls `end_account_page_access`; a taken approval is
spent only after the keys are stored (ADR-0796 to ADR-0798).
`src/homeserver/AGENTS.md#OAuth 2.0 API (ADR-0812, ADR-0773)`
- A due delayed event is sent only by the `delayed-events` job or the `send` action, under the runtime lock, through
`send_event_as_user`, never before `delayed_since_ms + delay_ms`, after the account checks and the send-time rate
limit, with the "being sent" mark set across the send; a request carrying `org.matrix.msc4140.delay` never reaches an
immediate send; the content and state key are never logged (ADR-0823, ADR-0825, ADR-0826).
- Worker frames are untrusted. Relayed PDUs are re-verified with main's resolver **before** taking
`runtime.mutex`, and the envelope is rebuilt from the verified event (ADR-0171).
`#Federation worker relays are untrusted input`
Expand Down Expand Up @@ -339,6 +346,7 @@ look at every time. Read the `AGENTS.md` itself before judging a change to that
equals the generated step (ADR-0778). `migrations/AGENTS.md#Migration 002: the OAuth 2.0 tables (1.1.0, ADR-0776, ADR-0778)`
- Next number is the maximum plus one, contiguous. Check that parallel branches have not taken it. Format and
header follow `#File format`.
- Migration 003 adds `delayed_events` the same way, in its own per-version array (ADR-0778, ADR-0824).
- Every new table is classified for the federation worker. `#Federation worker classification`
- `NOT NULL` additions need a `DEFAULT`. No drops without approval. Update `docs/database-persistence.md`.

Expand Down
31 changes: 31 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -3,6 +3,37 @@ Changes from 1.0.0, the first production release, onwards; the format is set out
Changes before 1.0.0 are in [CHANGELOG-pre-release.md](CHANGELOG-pre-release.md), which is no longer added to
([ADR-0740](docs/adr/0740-the-changelog-starts-at-1-0-0-and-the-pre-release-entries-are-a-frozen-record.md)).

# Version 1.1.2

## Features

* Delayed events (MSC4140): schedule, restart, cancel, send and look up one, on the v1.20 paths, the
`org.matrix.msc4140` paths and `?org.matrix.msc4140.delay=` on `/send` and `/state` (ADR-0823).
* `/versions` and `/capabilities` advertise delayed events (`org.matrix.msc4140`, `m.delayed_events`)
(ADR-0823).
* A due delayed event is sent by the server as its user through the ordinary send path: authorised when sent,
never early, kept across a restart (ADR-0825).
* Migration 003 adds the `delayed_events` table; finalised events are kept per user up to a bound and 7 days
(ADR-0824).
* `server.delayed_events.enabled`, `max_delay_ms`, `max_scheduled_per_user` and `max_finalised_per_user`, with
built-in per-user rate limits (ADR-0826).

## Security Fixes

* None.

## Bug fixes

* None.

## Release notes

Delayed events for MatrixRTC, on top of 1.1.1. 1.1.2 migrates the database: on first start it applies migration 003,
which adds one empty table and changes no existing one. A 1.1.1 or older server refuses the migrated database, so
back it up before upgrading if you may need to go back. The feature is on by default; setting
`server.delayed_events.enabled = false` and restarting stops new scheduling, while events already scheduled are still
sent and can still be managed. Operators running Element Call keep `max_delay_ms` at 3600000 or more.

# Version 1.1.1

## Features
Expand Down
11 changes: 11 additions & 0 deletions config/merovingian.conf.bsd.example
Original file line number Diff line number Diff line change
Expand Up @@ -155,6 +155,17 @@ server.http.ipv6_client_prefix_length=64
# or holding ? is refused. https://host, https://host:8443 and https://host/jwt are fine.
# server.rtc.livekit.service_url=https://livekit-jwt.example.org/lk-jwt-service

# Delayed events (MSC4140, optional; restart required)
# On by default: MatrixRTC clients (Element Call) schedule a "leave" that the server sends if the client vanishes.
# false stops new scheduling; events already scheduled are still sent.
# server.delayed_events.enabled=true
# The longest delay accepted, in milliseconds: 1000..2678400000 (31 days). Element Call needs 3600000 or more.
# server.delayed_events.max_delay_ms=86400000
# Scheduled events one user may hold: 1..1000.
# server.delayed_events.max_scheduled_per_user=100
# Finalised (sent, cancelled or failed) events kept per user: 1..1000.
# server.delayed_events.max_finalised_per_user=100

# Listener exposure
# Keep TLS disabled only on loopback binds behind a declared reverse proxy.
# If binding to a public interface, set tls=true, reverse_proxy=false, and
Expand Down
11 changes: 11 additions & 0 deletions config/merovingian.conf.example
Original file line number Diff line number Diff line change
Expand Up @@ -128,6 +128,17 @@ server.http.ipv6_client_prefix_length=64
# or holding ? is refused. https://host, https://host:8443 and https://host/jwt are fine.
# server.rtc.livekit.service_url=https://livekit-jwt.example.org/lk-jwt-service

# Delayed events (MSC4140, optional; restart required)
# On by default: MatrixRTC clients (Element Call) schedule a "leave" that the server sends if the client vanishes.
# false stops new scheduling; events already scheduled are still sent.
# server.delayed_events.enabled=true
# The longest delay accepted, in milliseconds: 1000..2678400000 (31 days). Element Call needs 3600000 or more.
# server.delayed_events.max_delay_ms=86400000
# Scheduled events one user may hold: 1..1000.
# server.delayed_events.max_scheduled_per_user=100
# Finalised (sent, cancelled or failed) events kept per user: 1..1000.
# server.delayed_events.max_finalised_per_user=100

# Listener exposure
# Keep TLS disabled only on loopback binds behind a declared reverse proxy.
# If binding to a public interface, set tls=true, reverse_proxy=false, and
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,147 @@
# Delayed events are served on the stable, unstable and query-parameter forms that released clients send

* Status: accepted
* Date: 2026-11-17

## Context and Problem Statement

Delayed events (MSC4140) let a client ask the server to send an event later, and to cancel or postpone it. MatrixRTC
uses them to schedule a call membership leave, so that a participant whose client vanished is removed from the call.
The MSC merged on 2026-09-08 and the feature is in matrix-spec `main` (commit `7198bf6724`, 2026-11-13) as v1.20. It
is not in the vendored v1.19 specification, so it has no conformance scenarios here. Which request forms does the
server serve?

The released clients, read at their release tags on 2026-11-17, do not send the spec's stable paths:

* Element X iOS and Android 26.11.0 (matrix-rust-sdk `3ff03638`, ruma `0ed7fba3`) and Element Call 0.28.2 running
standalone send only the query form, `?org.matrix.msc4140.delay=` on the ordinary `/send` and `/state` routes.
Element X manages an event with `POST .../org.matrix.msc4140/delayed_events/{delayId}` and the action in the body.
* Element Web 1.12.35 (matrix-js-sdk 45.1.0) tries the unstable dedicated path
`.../org.matrix.msc4140/rooms/{roomId}/delayed_event/{eventType}/{txnId}` first and falls back to the query form on
`M_UNRECOGNIZED`. It manages with `POST .../org.matrix.msc4140/delayed_events/{delayId}/{send|cancel|restart}`.
Element Call 0.28.2 pins matrix-js-sdk commit `3f05dc4d`, which schedules with the query form and manages with the
same path-form `POST`.
* lk-jwt-service 0.10.0 manages a delegated leave as an application service acting for the user, on the same
unstable management path.
* Every client reads only the `org.matrix.msc4140` flag in `unstable_features`; matrix-js-sdk refuses to use the
feature without it.

## Decision Outcome

Chosen option: "serve every form a released client sends, plus the spec's stable forms", because a server that serves
only the stable paths schedules nothing for the clients that exist.

| # | Method and path | Who sends it |
|---|---|---|
| S1 | `PUT /_matrix/client/v3/rooms/{roomId}/delayed_event/{eventType}/{txnId}` | spec, stable; no released client yet |
| S2 | `PUT /_matrix/client/unstable/org.matrix.msc4140/rooms/{roomId}/delayed_event/{eventType}/{txnId}` | Element Web 1.12.35 |
| L1 | `PUT /_matrix/client/v3/rooms/{roomId}/send/{eventType}/{txnId}?org.matrix.msc4140.delay=<ms>` | Element X, standalone Element Call |
| L2 | `PUT /_matrix/client/v3/rooms/{roomId}/state/{eventType}/{stateKey}?org.matrix.msc4140.delay=<ms>`, and the no-state-key and empty-state-key forms the ordinary state route accepts | Element X, standalone Element Call; the path a call's leave is scheduled on today |
| M1 | `POST /_matrix/client/v1/delayed_events/{delayId}/{send\|cancel\|restart}` | spec, stable |
| M2 | `POST /_matrix/client/unstable/org.matrix.msc4140/delayed_events/{delayId}/{send\|cancel\|restart}` | matrix-js-sdk, lk-jwt-service |
| M3 | `POST /_matrix/client/unstable/org.matrix.msc4140/delayed_events/{delayId}` with body `{"action":"send"\|"cancel"\|"restart"}` | Element X; the js-sdk fallback |
| G1 | `GET /_matrix/client/v1/delayed_events/{delayId}` | spec, stable |
| G2 | `GET /_matrix/client/unstable/org.matrix.msc4140/delayed_events/{delayId}` | a matrix-js-sdk method |

M3 exists only under the unstable prefix; `POST /_matrix/client/v1/delayed_events/{delayId}` is not a route. A last
segment other than `send`, `cancel` or `restart` on M1 or M2 is not a route (`404 M_UNRECOGNIZED`).

Every route needs an access token and goes through the standard gate (`M_MISSING_TOKEN`, `M_UNKNOWN_TOKEN`, the
moderation gate for locked and suspended accounts). Application-service masquerade (`?user_id=`) works exactly as on
other routes; lk-jwt-service depends on it.

A request that carries `org.matrix.msc4140.delay` is always taken down the delayed branch, before the ordinary
route's idempotency lookup and before any send. It never falls through to an immediate send, including when the
feature is disabled (then it is `403 M_FORBIDDEN`). A delay that is not a decimal integer of at least 1, or is
repeated, is `400 M_INVALID_PARAM`.

The parameter is honoured only on `PUT .../send/{eventType}/{txnId}` and `PUT .../state/...`. Every other client
route that sends a room event from the caller's body (`POST /_matrix/client/v3/rooms/{roomId}/send` and
`PUT .../redact/{eventId}/{txnId}`) answers `400 M_INVALID_PARAM` when the parameter is present, before anything is
sent and whether or not the feature is enabled, so that no route can send at once a request that asked for a delay.
The parameter's name is percent-decoded before it is compared, so an encoded spelling is recognised. The
application-service `?ts=` parameter is ignored on a delayed request; the event is stamped with the time it is sent.

Management looks an event up by the authenticated user and the delay ID. A delay ID that does not exist and one that
belongs to another user get the same answer, byte for byte: `404 M_NOT_FOUND`, `Delayed event not found.`

Management outcomes, all success bodies `{}`:

| State of the delayed event | `restart` | `cancel` | `send` |
|---|---|---|---|
| scheduled | 200; the delay starts again from now | 200; finalised as cancelled | sends now; 200 and finalised as sent; on failure the status and error an ordinary send gives, and the event stays scheduled with its timer unchanged |
| sent | 409 | 409 | 200 |
| cancelled by the user | 409 | 200 | 409 |
| cancelled because sending failed | 409 | 200 | 409 |
| being sent right now | 409 | 409 | 409 |

The 409 errcode is `M_UNKNOWN`. Released clients treat 404 `M_NOT_FOUND` and any 409 alike as "gone"
(matrix-js-sdk `isDelayedEventGoneError`, lk-jwt-service).

`/versions` carries `org.matrix.msc4140` and `org.matrix.msc4140.stable`, both set to whether the feature is enabled.
`/capabilities` carries `m.delayed_events` and `org.matrix.msc4140.delayed_events`, each
`{"max_delay_ms", "max_scheduled"}`, zeros when disabled. The event a delayed event became carries
`unsigned.delay_id` and `unsigned.org.matrix.msc4140.delay_id` for the user who scheduled it only, and no
`transaction_id`.

## Pros and Cons of the Options

### Serve only the spec's stable paths

* Bad, because no released client calls them: Element X and standalone Element Call would schedule nothing.

### Do not serve the dedicated path and rely on the js-sdk fallback

* Bad, because the fallback needs exactly `M_UNRECOGNIZED`, is permanent for the client's lifetime, and the dedicated
path is where the spec and clients are going.

### Answer 404 for every finalised delayed event, as Synapse 1.165 does by deleting the row

* Bad, because it loses the spec's idempotent 200 for a repeated action and the stored error. Released clients treat
409 and 404 alike, so nothing is gained.

### Unauthenticated management, with the delay ID as a bearer capability

Earlier MSC drafts did this, and one other server still accepts it on the path form.

* Bad, because the final MSC and the spec require authentication, every released client sends a token, and
lk-jwt-service authenticates as an application service. The delay ID is an identifier, never a credential.

### The pre-1.157 Synapse error for a too-large delay

`M_UNKNOWN` with `org.matrix.msc4140.errcode`. It is the only form Element Call 0.28.2's pinned js-sdk recognises.

* Bad, because it is gone from the MSC and from Synapse. The server answers `400` with `M_DELAY_TOO_LARGE` on S1 and
`ORG.MATRIX.MSC4140_DELAY_TOO_LARGE` on S2, L1 and L2. The cost is confined to an operator who lowers the maximum
delay below what the client asks for (18 seconds; one hour with MSC4195 delegation): that client joins without a
delayed leave.

### The listing endpoint

`GET .../delayed_events` is MSC4486.

* Bad, because it is unmerged and no released code calls it. It stays `404 M_UNRECOGNIZED`.

### Cancel a pending delayed state event when another state event with the same type and state key is sent

Earlier drafts did this. Synapse stopped in 1.127 and 1.164.

* Bad, because the client sends its join immediately after scheduling its leave under the same key; the join would
cancel the leave.

## Consequences

* The unstable and query forms are retired only after released clients stop sending them. Check the client release
tags again when a spec release containing delayed events is vendored.
* The listing endpoint and sticky events (MSC4354) remain out of scope; the sticky member of the dedicated body is
ignored.
* Application-service masquerade on the management routes must keep working.
* A new route that sends an event from the caller's body must refuse the delay parameter the same way.
* The user manual tells operators to keep the maximum delay at one hour or more.

## Links

* Related: [ADR-0824](0824-a-delayed-event-is-a-stored-row-kept-after-it-is-finalised-bounded-per-user-and-deleted-with-the-account.md)
* Related: [ADR-0825](0825-a-due-delayed-event-is-sent-by-one-job-through-the-ordinary-send-path-never-early-and-at-least-once.md)
* Related: [ADR-0826](0826-delayed-events-are-on-by-default-with-a-maximum-delay-per-user-caps-and-per-user-rate-limits.md)
* Related: [ADR-0822](0822-the-rtc-transport-is-advertised-as-type-livekit-with-livekit-service-url.md)
Loading
Loading