diff --git a/.agent/skills/merovingian-review/SKILL.md b/.agent/skills/merovingian-review/SKILL.md new file mode 100644 index 000000000..2165f5383 --- /dev/null +++ b/.agent/skills/merovingian-review/SKILL.md @@ -0,0 +1,315 @@ +--- +name: merovingian-review +description: "Review a Merovingian pull request, branch or local diff against this project's binding rules: docs/coding-rules.md, the security coding rules, the per-module AGENTS.md invariants, the Matrix v1.19 spec, and the change obligations (CHANGELOG, docs, ADR, version bump, PR headings). It targets the defect classes this project has actually shipped. Use when asked to review a PR, branch, diff or commit in this repository, to check a change before merge, to verify a subagent's or another session's work, or to post review comments on a Merovingian PR." +--- + +# Merovingian review + +Review the change against what it claims, against the code around it, and against the rules this project has +written down. Most of those rules exist because a defect shipped without them. A finding is reportable only when +you can name the inputs or state that produce the wrong outcome. + +This skill is a checklist and a method. **It does not hold the rules.** The binding text is in +[docs/coding-rules.md](../../../docs/coding-rules.md), +[docs/security-coding-rules.md](../../../docs/security-coding-rules.md), the directory `AGENTS.md` files and the +spec under `docs/matrix-v1.19-spec/`. The reference files below point into those documents. If a reference here +disagrees with the document it cites, the document wins, and the disagreement goes in the report as a finding +against this skill. + +| Reference | Use it for | +|---|---| +| [references/module-invariants.md](references/module-invariants.md) | What a diff in each `src/` module, `migrations/`, `packaging/` or `scripts/` must not break | +| [references/rule-checks.md](references/rule-checks.md) | Grep-able rule checks and judgement checks from the coding and security rules | +| [references/recurring-defects.md](references/recurring-defects.md) | Defect classes this project has shipped and fixed, as questions to ask of a diff | +| [references/tests.md](references/tests.md) | Where a test belongs, required structure, what the gates catch and miss, how to run and read tests | +| [references/obligations.md](references/obligations.md) | CHANGELOG, docs, ADR, version bump, PR description: rules and commands | + +## Modes + +- **Review** (default): read and report. Edit nothing, push nothing, post nothing. +- **Fix**: only when asked ("review and fix", "apply them"). Report first. Then make the change each finding + needs, following `docs/coding-rules.md`. That means the correct change, not the smallest. Re-run the tests you + cited and report the real counts. +- **Post**: only when asked to post. Finish Review first, then follow [Posting](#posting). Nothing is posted + until the user has seen it and chosen it. + +## Step 1: Pin the target + +| Target | Commands | +|---|---| +| PR | `gh pr view --json title,body,baseRefName,headRefName,headRefOid,commits,files` then `gh pr diff ` | +| Branch | `git fetch origin` then `git diff --stat origin/main...HEAD` | +| Local work | `git diff`, `git diff --staged` and `git ls-files --others --exclude-standard`. Neither diff lists untracked files: review each one as an addition, and report one the change needs but would leave uncommitted | +| Another session's or subagent's work | Its branch or worktree, as a branch. Its report is a claim to test, not a summary to trust | + +Record the base and head SHAs. GitHub rewrites PR head commits after a squash or rebase, so take the SHA from the +PR, not from a local branch with the same name. A stacked PR is reviewed against its base PR's head, not +`main`. Say so in the report, and check that its merge order is stated. + +Read the PR description, linked issue and the commit messages before the code. Write down each claim the change +makes ("fixes X", "adds a cap", "tests pass on all platforms"). Every claim is either supported by the diff and +the evidence, or listed in the report as unsupported. + +## Step 2: Map the diff to the rules that govern it + +1. List the changed files: `git diff --name-only ...`. +2. For each top-level area the diff touches, read that directory's `AGENTS.md`: `src//AGENTS.md`, + `include/merovingian/AGENTS.md`, `src/AGENTS.md`, `migrations/AGENTS.md`, `tests//AGENTS.md`, + `packaging/AGENTS.md`, `scripts/AGENTS.md`, `docs/AGENTS.md`. Then take that module's section from + [module-invariants.md](references/module-invariants.md). The AGENTS file is binding. The section here is the + shortlist of what has gone wrong before. +3. Read the sections of `docs/coding-rules.md` and `docs/security-coding-rules.md` that + [rule-checks.md](references/rule-checks.md) names for the kinds of code in the diff. +4. For every Matrix-visible behaviour the change touches (endpoint, status code, errcode, event field, auth rule, + federation exchange), find the spec section in `docs/matrix-v1.19-spec/` **before** judging the code. Read the + passage for each actor involved: client, server, remote server, appservice. Something is an owner decision + only when the spec is silent or genuinely ambiguous, and you have read the conflicting passages. +5. Find the ADRs the change touches or contradicts. Search `docs/adr/index.md` for the subsystem's terms. A diff + that undoes an ADR's decision without superseding that ADR is a finding. + +## Step 3: Understand before critiquing + +- Read every hunk in its file, not on its own. Read the callers of every changed signature or semantic. +- Note which process each changed line runs in. The **main process** holds the signing key, the database, the + appservice registrations and email. The **federation worker** has none of these and is the most exposed process. + The **thumbnail worker** decodes untrusted images under a sandbox. Code that moves from one to another changes + its trust. +- Note which locks each changed line runs under: `runtime.mutex` via `RuntimeLock`, the federation state mutex + (not recursive), the store mutexes, the leaf mutexes in the parker and registries. A network call, SMTP send, + Argon2id hash or IPC round trip under one of these is the most common serious defect in this project's history. +- Flag scope creep: unrelated refactors, whole-file reformatting, a fix bundled with a feature. Reformatting churn + hides the real change and has been introduced by delegated work before. +- A new function, type, config key or handler must be **wired**: reachable from a real call path, registered + where the dispatch tables are built, and present in both processes where it must be. Declared-but-unwired code + has been reported as "done" before. Find the call site. + +## Step 4: Discover, with an evidence log + +Find first. Rank later (Step 7). A finding judged minor while you are still looking is one you stop +investigating. + +Every dimension gets a row, whether or not it produced anything. A dimension with no row is unexamined, not +clean. A row whose Method is "looked at it" is the same thing. + +| ID | Dimension | Examined | Method | Result | +|---|---|---|---|---| +| C1 | Correctness | `src/sync/SlidingSync.cpp:210-340` | every branch; partial failure; empty and max-size input | D1 | +| S1 | Security / trust boundary | `src/homeserver/WorkerPool.cpp:88-140` | relay path re-verifies signature before `runtime.mutex`? | clean | +| L1 | Locks and concurrency | same | every call under `RuntimeLock`; any I/O inside? | D2 | +| B1 | Bounds and cost | request arrays, regex, scans | who controls the size; cap and errcode; cost under lock | clean | +| SP1 | Spec | `client-server-api.md#...` | status, errcode and fields against the cited section | D3 | +| R1 | Rules | changed `.cpp`/`.hpp` | rule-checks greps; judgement checks | clean | +| T1 | Tests | new scenarios | fails if fix reverted? placement; tags; gates | gap, D4 | +| O1 | Obligations | CHANGELOG, docs, ADR, version, PR text | obligations.md commands | D5 | +| H1 | Recurring defects | whole diff | each class in recurring-defects.md that applies | clean | + +Work through: + +- **Correctness**: every branch and early return; state after step N of M fails; empty, one, maximum, zero, + negative; byte versus character length; percent-encoded versus decoded identifiers; time from the right clock. +- **Trust boundaries**: each input crossing one. That includes client bodies, headers and query strings, federation + PDUs, EDUs and queries, worker IPC frames, appservice and identity-server responses, config files, and database + rows written from any of these. Authorisation is checked at the operation, not only at the route. Defaults fail + closed. +- **Locks and concurrency**: the lock-scope questions above, plus check-then-act across a `RuntimeLockRelease`. + After re-locking, is everything the code relies on re-validated? Are pointers kept across the release? Check lock + order, and `std::terminate` from an exception escaping a pool task. +- **Bounds and algorithmic cost**: anything a client or remote can grow must be capped. State it owns is refused + at the cap, never evicted; a cache the server can refill is bounded by LRU eviction where its module doc says + so. Work under the runtime lock grows at most linearly with request input. A single pass over a table that the + module doc sanctions is allowed, such as `matching_directory_profiles` reading `store.users` once; a scan + repeated per item, per member or per comparison is a finding + (`src/homeserver/AGENTS.md#Work under the runtime lock is bounded by the request`). Also check regex, glob and + substring matching on caller-chosen input, comparators that rescan, and per-row work multiplied by per-member + work. +- **Spec**: status codes and errcodes as the spec states them, field names and presence. Every event in a sync + or sliding-sync response carries `event_id`, and so on. +- **Rules**: run the greps in [rule-checks.md](references/rule-checks.md) over the changed lines, then the + judgement checks. Do not re-report what `scripts/reject-unsafe.sh` and the meson gate tests already fail on, + unless the diff shows the gate was bypassed or edited. The list is in rule-checks.md. +- **Reuse**: every new helper claims nothing equivalent exists. Search `src/core/`, the module's own files and + the standard library by name and by concept before accepting it. This project has single-source helpers that + must not be re-derived: `EventId.hpp`, `RoomVersionPolicy`, `HistoryVisibility`, `room_read_access_for`, + `parse_size_limit`, `QueryParams`, `request_header`, `FoldedSubstringMatcher`, `merge_visible_key_signatures`, + `OutboundClient`. A helper used by a second file moves to its own file. It is never copied. +- **Tests**: see [tests.md](references/tests.md) and Step 5. +- **Obligations**: see [obligations.md](references/obligations.md). +- **Recurring defects**: go through [recurring-defects.md](references/recurring-defects.md) and ask each + applicable question of the diff. + +### Delegating discovery + +The root `AGENTS.md` asks the orchestrator to delegate reading and keep judgement. For a diff larger than a few +hundred lines: + +- Give each lower-tier reviewer one area: a module, or the tests, or the obligations. Its brief holds the base and + head SHAs, the files, the matching section of `module-invariants.md` or `rule-checks.md`, and the instruction + to read `docs/coding-rules.md` and the directory `AGENTS.md`. It returns discoveries with `file:line`, trigger + and outcome. It assigns no severity. +- Keep these for yourself: spec interpretation, security and trust-boundary verdicts, lock and concurrency + correctness, migration numbering, severity, and whether the change is done. +- **Verify every delegated discovery against the code before it becomes a finding.** Delegates invent APIs, + misread lock scopes, and write confident rationales for wrong conclusions. A discovery you have not confirmed + at its `file:line` is not reported. + +## Step 5: Tests: map behaviour to assertions + +For each new or changed behaviour (a status code, an errcode, a cap, a validation, a changed default, a new +relay), name the scenario that asserts it: + +| Behaviour | Scenario (`file:line`, tag) | Assertion | Fails if reverted? | +|---|---|---|---| +| Oversized `rooms` array, 400 `M_INVALID_PARAM` | `tests/integration/test_sliding_sync_flow.cpp:512` `[sliding-sync]` | status and errcode | yes, checked | +| Worker relay with a bad signature is refused | none | — | gap | + +- "Fails if reverted" is a check, not a guess. Apply the test change alone to a worktree at the base commit, + build, and run the binary with the tag filter. Otherwise write "not checked". A regression test that passes + without the fix proves nothing. This project has shipped such tests. +- A changed behaviour must update the test that pinned the old behaviour. An unchanged test asserting the old + behaviour means one of the two is wrong. Earlier tests have pinned leaks as correct behaviour. +- In conformance tests, a weakened, removed or commented-out `REQUIRE`, or an expected value changed without + citing the spec, is a Blocker. +- Placement, structure, tags and the pitfalls (Catch2 assertions off the main thread, live threads next to a + failing `REQUIRE` or `SKIP`, untagged PostgreSQL scenarios, minimum-cost Argon2 in a race test) are in + [tests.md](references/tests.md). + +## Step 6: Audit coverage independently + +"I reviewed every file" is self-certifying. Enumerate the hunks: + +```bash +git diff --unified=0 ... | grep '^@@' +git diff --stat ... +``` + +For a diff of more than about ten files, hand the enumeration to a fresh agent that has **not** seen your +findings. It lists every hunk with `file:line` and the dimension it belongs to, and returns every hunk and file +that has no evidence-log row. It does not hunt for bugs. Every uncovered hunk is either examined now or listed +under Verification as not reviewed. + +## Step 7: Severity, as a separate pass + +| Tag | Meaning in this project | Blocks merge | +|---|---|---| +| **[Blocker]** | Wrong result, auth or signature bypass, a worker-trust violation, data loss, a crash, a deadlock or timeout, an unbounded cost reachable by an unprivileged client or remote, a weakened conformance assertion, a broken build or test | Yes | +| **[Major]** | Real defect on a plausible path; incomplete fix; behaviour without a test that fails if reverted; missing CHANGELOG, docs, ADR or version bump the project rules require | Yes, unless the author shows it is wrong | +| **[Minor]** | Correct but fragile, unclear or wasteful; a missing negative-path test on non-security code; doc drift | No | +| **[Nit]** | Wording, naming, or style no tool owns | Never; collapse them | + +Calibration from this project's history: + +- **Availability is security here.** Every high finding in the 1.0.0 release-candidate audit was availability: + lock-held cost, uncapped growth, scans. Do not rank a client-reachable lock-held network call or an unbounded + scan under `runtime.mutex` below Blocker because "it's only DoS". +- Anything that lets the federation worker cause main to act without main's own verification is a Blocker. +- A finding whose outcome is security, authorisation or data loss and that ranks below Major is re-examined. +- Every severity has a one-sentence justification naming impact and likelihood. A severity without one is a guess. +- Never raise a nit to look thorough. Never soften a blocker to be agreeable. + +Do not report: + +- Formatting or include order that `scripts/format_code.py` or `tests/tooling/test_include_order.py` would settle. + You may note in one line that the format tool was not run on the changed files. +- A code reshape to satisfy a metric tool, such as dropping digit separators for CodeQL. That is a false + positive to dismiss, not a change to request. +- Design you merely disagree with, where an ADR records the decision and its rejected alternatives. Review the + implementation of the decision. Raise the design only when it causes a concrete defect. + +## Step 8: Verify + +Claims about tests are worth only what they were checked against. The rules come from the root `AGENTS.md` +"Verifying Work" section. Read that section. The short form: + +- The build and test command is the one the root `AGENTS.md` names: `python build.py vm` at the time of writing + (ADR-0752). It runs past ten minutes, so run it in the background and wait for it to finish. A run in progress + is not a result. The VM runs one suite at a time, and a second run from any worktree waits for its lock. The + VM builds the tree of the worktree you run it from, so run it from the worktree under review. +- `build.py`'s exit code is not evidence. Read the `Ok:`, `Fail:` and `Timeout:` counts at the end of + `build-vm/meson-logs/testlog.txt` (`build-vm-/` for `--vm `). `build.py vm` deletes the old copy + first, so a missing file is no result, never a pass; its output says whether the run never reached the tests + or the log could not be copied back. **A timeout is a failure.** +- `testlog.txt` holds output only for failing tests, so it cannot prove a passing scenario ran. Run the binary + with a tag filter: `python build.py vm --exec ./build-vm/tests/merovingian-unit-tests "[push]"` (`--exec` sets + the hardening opt-out). Catch2 registers `SCENARIO("x")` as `Scenario: x`, so filter by tag, not by name. + Confirm the reported test count is not zero. A zero-byte or missing binary "passes" silently. +- A sanitizer or fuzz run that aborted at startup is not clean. +- CI green on the PR is evidence only for the head SHA it ran on, and only for the jobs that ran. Name them. + +If you cannot run something, say what and why: no PostgreSQL, no BSD host, the suite too long for the session. +Never round a partial check up. + +## Step 9: Report + +```markdown +## Summary +Verdict: approve | approve with comments | request changes +Reviewed → . +One or two sentences: what the change does, and whether the diff supports what it claims. + +## Blockers +- **[Blocker]** `path:line` — defect. Trigger: … Outcome: … Fix: … Rule/spec: `doc#heading` or ADR-NNNN. + +## Major +## Minor and nits + +## Claims checked +| Claim (from PR/commit) | Supported? | Evidence | + +## Behaviour → test +| Behaviour | Scenario | Fails if reverted? | + +## Obligations +| Item | State | Evidence | +| CHANGELOG bullet in current version, right section | | | +| Docs per docs/AGENTS.md table | | | +| ADR (needed? numbered above all branches? index line?) | | | +| Version bump (or docs-only exemption) and all places agree | | | +| PR description headings | | | + +## Evidence log +| ID | Dimension | Examined | Method | Result | + +## Verification +- Commands, verbatim, and the counts read from the log. +- Not verified, and why. + +## Questions +``` + +Order findings by severity, then file order. Each cites the rule, spec section or ADR it rests on, so the author +can check it. Report only findings you confirmed at `file:line`. Report unverified suspicions as questions, +labelled as such. + +When this review is part of finishing your own change (`AGENTS.md`: "check the change against those rules and +report any unresolved conflict or unmet requirement"), the Obligations table and unresolved findings go in your +completion report. + +## Posting + +Posting is outward-facing and hard to retract. + +1. Give every drafted comment an ID: `B1`, `M1`, `m1`, `N1`, `Q1`. +2. Show every draft in full: ID, tag, anchor, exact text. +3. Ask which to post. Suggest blockers and majors as the default, but do not pre-apply it. Post only the IDs the + user names, and read the selection back before posting. +4. Each comment is one finding. The severity tag comes first, then the defect in one sentence with its trigger + and outcome, then the fix. Cite the rule or spec section. Questions carry `**[Question]**` and no severity. +5. Anchor to a changed line. GitHub rejects the whole review (HTTP 422) if any anchor is outside the diff. + Missing-code findings anchor to the nearest changed line and say what is missing. +6. Post with `gh api repos/{owner}/{repo}/pulls//reviews --input review.json`. Set `commit_id` to the PR head + SHA. Use `event` `REQUEST_CHANGES` only if a selected comment is a Blocker; otherwise use `COMMENT`. Check that + the returned comment count matches the selection. +7. Do not mention withheld findings or their counts in the review body. + +Write a bare `@name` only for a real GitHub login you mean to notify. Put `@`-prefixed code in backticks. + +## Anti-patterns + +- Reviewing the diff without its file, its callers or its process and lock context. +- Trusting a report, a green check or an exit code instead of reading the log and running the filtered binary. +- Accepting a test without checking it fails when the fix is reverted. +- Calling something an owner decision before reading the spec passages for every actor. +- Reporting what a gate already fails on, or asking for a reshape to satisfy a metric. +- Ranking an availability defect low because it "only" denies service. +- Copying rule text into findings instead of citing `doc#heading`, so the finding drifts with the doc. +- Posting anything the user did not select. diff --git a/.agent/skills/merovingian-review/references/module-invariants.md b/.agent/skills/merovingian-review/references/module-invariants.md new file mode 100644 index 000000000..03cec530b --- /dev/null +++ b/.agent/skills/merovingian-review/references/module-invariants.md @@ -0,0 +1,315 @@ +# Module invariants + +What a diff in each directory must not break. Each line is a check followed by its source, `path#heading`. The +directory's `AGENTS.md` is binding and holds more than this list. This is the shortlist of what reviewers have to +look at every time. Read the `AGENTS.md` itself before judging a change to that directory. + +## Cross-cutting (check on every diff) + +- **Lock scope around I/O.** No network call, SMTP send, Argon2id hash or IPC round trip runs under `runtime.mutex`, + a store lock or the federation state mutex. Use `RuntimeLockRelease` around the call only. After re-locking, + re-validate what the code depends on (`src/homeserver/AGENTS.md#The runtime lock and blocking calls`). +- **Caps on everything a client or remote can grow:** rows per user, array lengths, walk depths, matcher steps, + in-flight requests. State a client or remote owns is refused at the cap, never evicted to make room (ADR-0173, + ADR-0548). A cache of what the server can fetch or build again is bounded by eviction instead, where its module + doc prescribes it: cached remote media and thumbnails are deleted least recently used first + (`docs/media-repository.md#Runtime Behavior`, ADR-0120, ADR-0690). Work under the lock is at most linear in + request input (`src/homeserver/AGENTS.md#Work under the runtime lock is bounded by the request`). +- **Main-process-only:** signing, email, appservice registrations, database access, the policy-server hook. The + worker relays to main, and main re-verifies (`src/federation_worker/AGENTS.md#Rules — non-negotiable`). +- **No thread before hardening** (ADR-0082): pools, parker and log writers start after Landlock and seccomp. +- **Fail closed:** signing, discovery, IS and push resolution, decoder sandbox, key validation, namespace regex, + ClamAV, Landlock and seccomp. +- **Secrets** live in `SecretBuffer`, are compared in constant time, and are never logged. +- **Outbound** goes only through `OutboundClient` to SSRF-safe pinned addresses. +- **Indexes and maintained counters, not scans,** for anything a request touches. +- **Single-source helpers are not re-derived:** `EventId.hpp`, `RoomVersionPolicy`, `HistoryVisibility`, + `room_read_access_for`, `parse_size_limit`, `QueryParams`, `request_header`. + +## src/appservice + +- Namespace regexes are compiled once in `Namespace`'s constructor and matched with `std::regex_match`, never + `regex_search`, on length-bounded input. A malformed pattern matches nothing (ADR-0613). + `src/appservice/AGENTS.md#Namespaces` +- Cleartext `http` only to a loopback IP literal or an operator-listed host. The check runs at parse **and** in + `AppserviceClient` (ADR-0515). `#Security rules` +- `as_token`/`hs_token` in `SecretBuffer`, never logged, compared in constant time. `#Security rules` +- Identity assertion (`?user_id=`) stays inside the asserting service's namespaces. It never reaches account + management or admin routes (ADR-0611). Other services' exclusive namespaces are honoured (ADR-0688). `#Security rules` +- Fan-out takes an ADR-0079 slot before the first outbound call and releases `runtime.mutex` around it. A bridge + that is down never turns a 404 into a 502 (ADR-0181). `#Third-party lookups`, `#Ping` +- Appservice responses are untrusted. `instance_id` is minted by the homeserver. `#Third-party lookups` +- `ts` is validated and never forwarded in the rewritten query (ADR-0303). `#Timestamp massaging` +- The worker loads no registrations (ADR-0296). + +## src/auth + +- Tokens are hashed before storage, compared in constant time and never logged. Session lookup compares full + hashes in the caller's predicate, with no miss fall-back scan. `src/auth/AGENTS.md#Session lookup and retention` +- Revocation is one-way. A failed revoke stays retryable, and the session is marked revoked only after durable + writes (ADR-0407). `#Security rules` +- Argon2id runs outside `runtime.mutex` after `try_acquire()` admission, which answers 429 rather than a failed + attempt. After re-lock, re-validate. Never keep a `LocalUser` pointer across the release. + `#Argon2id admission and the runtime lock` +- The login throttle and the UIA password checks stay separate. `#Failed-login throttle` +- Identifiers split on the **first** `:`. The reserved localpart `remote-media` is refused on every + account-creation path (ADR-0178). `#Security rules` +- UIA sessions are capped and answer 429, never evicting (ADR-0683). Login on an existing device deletes earlier + rows (ADR-0590). Login tokens are never persisted (ADR-0548). `#UIAA` +- The production password cost is pinned by `tests/password_cost/` (ADR-0510). + +## src/canonicaljson + +- Sign or hash only `Serializer.hpp` output, with `signatures` and `unsigned` stripped via `Signable.hpp`. + `src/canonicaljson/AGENTS.md#Critical constraints` +- The parser keeps rejecting duplicate keys, non-canonical input, integers outside ±(2^53−1) and floats. + `#Canonical JSON rules (from spec)` + +## src/config + +- Validate at parse time. Other modules read only `RuntimeConfigSnapshot`. `src/config/AGENTS.md#Rules` +- Every byte limit goes through `parse_size_limit`. An unparsable value is rejected, because it would otherwise + parse to 0, which means "no limit". `#Size limit parsing` +- `ReloadPolicy.cpp` stays in step with the user manual's reloadability section. A new key goes into **both** + example configs, BSD included (ADR-0738). A documented key is enforced or removed (ADR-0498): grep for its reader. +- Effective config is logged without secrets or TLS key paths. + +## src/core + +- `SecretBuffer`, `FileDescriptor` and `Error` are the only forms for their jobs. Query parsing goes only through + `QueryParams`. `src/core/AGENTS.md#Rules` +- A semaphore slot is returned only by `SemaphorePermit`'s destructor. Capped parallel work uses + `run_bounded_parallel` (ADR-0267). +- A mutex in a copyable type is a `CopyableMutex` by value (ADR-0517). +- A client-chosen term in client-chosen text uses `FoldedSubstringMatcher` with `max_search_term_bytes`, never + `std::search` (ADR-0330). + +## src/crypto + +- libsodium only in `src/crypto/`, `src/events/`, `src/auth/` and `src/core/SecretBuffer.cpp` (ADR-0014). The + gate enforces this. +- Signing fails closed. Key material is never logged. Remote keys and key IDs are shape-validated before use. + `src/crypto/AGENTS.md#Security rules — non-negotiable` +- Master key identity and bytes come from **one** open. `#Security rules` rule 6 +- Do not bypass `Ed25519Provider`. The worker's provider refuses everything (ADR-0078). `#Provider interface pattern` +- `hmac_sha1` is for TURN only (ADR-0480). Add no new SHA-1 use. + +## src/database + +- Prepared statements with bound parameters. Never log parameters, and never retain committed statements outside + tests. `src/database/AGENTS.md#Security rules — non-negotiable` +- Every BLOB write sets `BoundValue::binary`. +- One transaction is one pooled lease. Never take a second lease while holding one: that deadlocks (ADR-0154). +- A read that combines several tables into one picture of a room runs in one read transaction (ADR-0161). + Federated join writes are single transactions (ADR-0714). +- Reads go by index, never scan. `PersistentStore` mutexes follow ADR-0517. A member added to `PersistentStore` + is added to every hand-written copy, move and serialise path too. +- Schema changes go only through `migrations/` (ADR-0224). Nothing but SQLite opens an SQLite file (ADR-0157). + +## src/email + +- Refuse, never repair: every address passes `is_safe_mailbox_address` before libcurl or any header. + `src/email/AGENTS.md#Rules — non-negotiable` +- TLS is required (`CURLUSESSL_ALL`) unless the server is a loopback literal and the operator set + `require_tls=false` (ADR-0220). +- The SMTP password is read per send into `SecretBuffer`. `send_smtp_message` is never called with a lock held. +- Content is the unread count and server name only (ADR-0219), or the 8-character code (ADR-0481). +- Main process only. + +## src/events + +- Event IDs come only from `EventId.hpp`, with the room version from `RoomVersionPolicy`. v1 and v2 stay + unsupported (ADR-0076). +- `event_id` stays in the signed payload (ADR-0397). +- Authorisation runs before persisting any local event or inbound PDU. Rules go into `IndexedAuthEvent`, never as + a per-event JSON walk (ADR-0693). `src/events/AGENTS.md#Authorization rules` +- Auth-chain walks are bounded and fail closed on missing events. `#State resolution` +- Redaction uses the power levels this server recorded. `power_levels_known=false` means nobody qualifies + (ADR-0586). +- Ingested PDUs are stored via `without_unsigned` (ADR-0402). + +## src/federation + +- Authenticate X-Matrix before touching the body. Verify every inbound PDU signature, our own server's included, + before it enters the graph. Check the restricted-join authorising signature on **every** admission path: + `/send`, send_join, worker relay, backfill (ADR-0171). `src/federation/AGENTS.md#Security rules — non-negotiable` +- Keys come only via `RemoteKeyCache`. Fetches go through admitted paths under `key_resolution_max_in_flight` + (ADR-0155). +- Authorise before persistence. Rejected and soft-failed events are never relayed as normal. +- Room-scoped reads check `origin_may_read_room`, with the origin taken from authentication, never from the query + or body. State at an event is the recorded state (ADR-0363). +- At most one delivery thread per destination. Never hold a `FederationDestination&` across a network call, and + keep no process-global resolver state on delivery threads (ADR-0354, ADR-0355). +- `FederationRuntimeState::mutex` is not recursive (ADR-0518). `#The federation state mutex is not recursive` +- Events build only on same-room events (ADR-0583). Received PDUs pass `pdu_format_violation` (ADR-0399). + +## src/federation_worker + +- The worker never holds the signing secret and never signs (ADR-0078). + `src/federation_worker/AGENTS.md#Rules — non-negotiable` +- The worker opens no database, on either backend. Its store is filled from main under one-use permits + (ADR-0733, ADR-0735). +- Landlock, then `apply_worker_hardening()`, before any thread (ADR-0082). They are separate controls with + separate opt-outs. + - `apply_worker_landlock()` runs whatever `federation.worker.apply_hardening` says, and sets + `PR_SET_NO_NEW_PRIVS` itself just before `landlock_restrict_self()`. Only + `federation.worker.allow_without_landlock=true` lets the worker start on a kernel without Landlock, with a + CRITICAL log on every start; any other Landlock failure is fatal (`#Rules — non-negotiable` item 3). + - `apply_worker_hardening()` on Linux applies, in order and each fail-closed: the core-dump policy + (`RLIMIT_CORE` 0 and `PR_SET_DUMPABLE` 0), `PR_SET_NO_NEW_PRIVS`, the capability-bounding-set drop, and the + worker seccomp filter (`src/platform/RuntimeHardening.cpp`). `federation.worker.apply_hardening=false` skips + all four. + - FreeBSD and OpenBSD have neither, so the default refuses to start (ADR-0041; the server exits 80). An + operator who confines the worker another way must set **both** opt-outs + (`docs/user-manual.md#Install from a distro package (recommended)`). + + A change that adds another way to run unsandboxed, folds one opt-out into the other, or skips a refusal is a + finding. `#Hardening` +- `local_pool` and `relay_pool` stay separate, routed by `federation_endpoint_requires_main_relay`. +- `room_sync` runs on `reload_pool` through `RoomReloadCoalescer` and `StoreReadGuard`. Hold no lock across IPC, + and copy what a table lends before releasing (ADR-0623). +- Never `channel->stop()` from a handler. Drain pools before stopping the channel. +- Relayed EDUs and queries are bound to the request main verified (ADR-0395, ADR-0595). +- **A new federation handler in main needs its worker path** (relay or snapshot), and the reverse. Check both + sides and the routing table. + +## src/homeserver + +- No blocking network call under `runtime.mutex`. `RuntimeLockRelease` wraps only the call. Copy the signing + secret into an owned `SecretBuffer` first. `src/homeserver/AGENTS.md#The runtime lock and blocking calls` +- Work under the lock is linear in request input. Per-user row counts are capped with the documented errcodes: + `400 M_TOO_LARGE` (ADR-0173). `#Work under the runtime lock is bounded by the request` +- Worker frames are untrusted. Relayed PDUs are re-verified with main's resolver **before** taking + `runtime.mutex`, and the envelope is rebuilt from the verified event (ADR-0171). + `#Federation worker relays are untrusted input` +- Every endpoint returning room events goes through `HistoryVisibility`. Every endpoint returning state or a + roster goes through `room_read_access_for` (ADR-0084), with a conformance scenario. + `#Read paths that return room events or state` +- Route matching compares the path without the query string (ADR-0420). +- Never call `handle_local_http_request()` with a real client body. `#Architecture boundaries` +- Large-body routes authenticate before reading the body and opt out of the default cap explicitly. + `#Body size limits` +- Each connection has exactly one owner, passed by move. Never touch a parked poll after `park()` + (ADR-0569/0570). +- Operator routes dispatch before the general token gate, so each judges a locked or suspended admin itself. + +## src/http + +- Rate limiting runs before auth. Per-client limits key on `client_address_key`. `src/http/AGENTS.md#Rules` +- `OutboundClient` never resolves hosts: it needs `pinned_addresses`. Invalid header names and values are refused + before any I/O (ADR-0180). URLs are logged without the query (ADR-0418). +- Headers are looked up through `request_header`. Admission via `InFlightBudget` never queues. + +## src/identity + +- IS hosts resolve to SSRF-safe pinned addresses. HTTPS only, allowlist checked at call sites. + `src/identity/AGENTS.md#Security rules (non-negotiable)` +- `id_access_token` is validated before it becomes a header (ADR-0180). +- Only the IS proves 3PID ownership. `parse_validated_3pid_response` fails closed. +- The test-only discovery seam stays empty in production. + +## src/ipc + +- Mutually authenticated key exchange, and no libsodium here. `src/ipc/AGENTS.md#Security model — non-negotiable` +- No signing frame. No access token, peer `Authorization` or `X-Matrix` header, or secret key in any frame: main + verifies the inbound signature and forwards only the verified identity. The one `Authorization` value that + crosses is main's own request-bound X-Matrix signature on an outbound request, not a reusable credential + (`docs/hardening.md`). Add narrowly typed requests instead of a generic one. +- Oversize frames fail visibly via `worker_ipc_frame_bytes` (ADR-0368). Frames are valid UTF-8 (ADR-0321). A + failed `write_frame` is never ignored (ADR-0322). +- Pending requests carry main-owned origins and digests, never worker JSON (ADR-0595). +- The reader thread only routes frames. Handlers run on the dispatch thread, and a handler never stops its own + channel. `#Thread safety - the reader/dispatch split` + +## src/media + +- The internal pipe format is built only in `ClientServerRuntime.cpp`. `src/media/AGENTS.md#Internal body format` +- Declared and sniffed MIME types are derived independently. Unknown types are quarantined. Content-Disposition + goes through `media_content_disposition` (ADR-0211). +- Readers of `pending_uploads` test `expires_at_ms` (ADR-0589). Remote cache records are classified by + `from_remote_fetch`, never by the owner string (ADR-0178). +- Quotas use maintained counters, not record scans, and are charged for deduplicated uploads too (ADR-0689). +- The decoder runs only after `apply_decoder_hardening()` and refuses without a sandbox (ADR-0419). The ClamAV + scanner fails closed (ADR-0565). + +## src/net + +- `SOCK_CLOEXEC`/`O_CLOEXEC` via `FileDescriptor`. Pool tasks never throw. `src/net/AGENTS.md#Rules` +- No worker waits on a quiet socket: `ConnectionParker` does. The parker mutex is a leaf. A per-client key gets at + most 1/16 of the parked budget (ADR-0719). +- Shutdown order: stop accepting, stop the parker, drain the pools. + +## src/observability + +- Never log secrets. One record is one physical line. Field values are capped. URLs are logged without the query. + `src/observability/AGENTS.md#Rules` +- Unauthenticated per-request audit events pass through `AuditRateGate` (ADR-0080). Audit text goes through + `bounded_utf8`. +- Writer threads start after hardening. The log file opens only through `LogFileSink`. +- Admin routes live in `admin_routes` (ADR-0504). There is no shutdown route (ADR-0505). + +## src/platform + +- Hardening runs before any socket opens. A self-check result of `unknown` or `disabled` blocks start-up. + `src/platform/AGENTS.md#Security rules — non-negotiable` +- Every filter is installed with `SECCOMP_FILTER_FLAG_TSYNC`, and any non-zero result is a failure. + `ignore_sigpipe()` is the first call in every `main()`. +- Profiles that allow `fcntl` or `ioctl` filter the commands. `TIOCSTI` and `TIOCLINUX` are never allowed + (ADR-0731). A child spawned under main's filter needs main's list to allow its calls (ADR-0643). Filters stack + across fork and exec. +- The BSD gate and decoder confinement travel together (ADR-0378). On the platforms the gate refuses, it must + refuse, not report success. +- A filter change needs a real-kernel test that **fails**, not skips, when the filter is wrong. + +## src/push + +- `PushRules.cpp` is pure. An unrecognised condition never matches. `src/push/AGENTS.md#Rules — non-negotiable` +- Gateway URL: HTTPS, path exactly `/_matrix/push/v1/notify`, resolved through filtered discovery. +- No network call under the caller's locks. Deliveries, registrations and retries are bounded. +- Globs are linear, never `std::regex`, with one `GlobBudget` per evaluation (ADR-0175, ADR-0607). +- User rules are bounded on write and on read. Email pushers go only to the user's own 3PID (ADR-0219). + +## src/rooms + +- Get `RoomVersionPolicy` before any auth, state resolution, event ID or redaction work. No version logic + elsewhere. `src/rooms/AGENTS.md#Room version policy` +- `room_creation_encryption_policy` is the only create-time decider (ADR-0445). `#Encryption policy` + +## src/sync + +- Every read of events or state goes through `HistoryVisibility` (one per request) and `room_read_access_for`. + The "shared" rule compares depth, not stream ordering (ADR-0308). + `src/sync/AGENTS.md#History visibility and room read access` +- Long-poll: take the cursor before rerunning and a new one after waking. A publish is scoped only when its + audience is certain (ADR-0437). `#Long-poll behaviour` +- `sync_pool` is separate from the main pool. `ParkedPollRegistry`'s mutex is a leaf. +- Redactions are withheld until they apply. Filter lists and examination budgets are capped (ADR-0656/0657). +- Token families: `/sync` `next_batch` is the composite token, while `/messages` and `prev_batch` use plain stream + orderings. Sliding-sync `pos` never regresses. `#Stream token format` + +## src/trust_safety + +- The engine decides; the caller acts. No hard-coded moderation decisions. `src/trust_safety/AGENTS.md#Rules` +- The policy-server hook is built by `resolve_policy_server_hook` (ADR-0361) and consulted in main only. +- `IgnoreList` is shared by homeserver, sync and push: a change must hold for every call site. + +## migrations + +- Never edit an applied migration. `001_initial_schema.sql` is frozen (ADR-0224). A change is a new numbered + file, plus its step in `upgrade_migration_catalog()` and its downgrade. `migrations/AGENTS.md#Safety rules` +- Next number is the maximum plus one, contiguous. Check that parallel branches have not taken it. Format and + header follow `#File format`. +- Every new table is classified for the federation worker. `#Federation worker classification` +- `NOT NULL` additions need a `DEFAULT`. No drops without approval. Update `docs/database-persistence.md`. + +## packaging + +- Versions match `meson.build`. Services drop privileges. The systemd unit keeps `PrivateTmp`, `NoNewPrivileges` + and `ProtectSystem=strict`. `packaging/AGENTS.md#Rules` +- No hard-coded paths. BSD packages install the BSD example config (ADR-0738). +- PostgreSQL: separate migration and runtime roles. The worker has no role (ADR-0733). + +## scripts + +- `reject-unsafe.sh`: fix the code, never add an exception. `scripts/AGENTS.md#Rules` +- Scripts are idempotent and print what they changed. diff --git a/.agent/skills/merovingian-review/references/obligations.md b/.agent/skills/merovingian-review/references/obligations.md new file mode 100644 index 000000000..cb52df884 --- /dev/null +++ b/.agent/skills/merovingian-review/references/obligations.md @@ -0,0 +1,95 @@ +# Change obligations + +What every branch must carry besides code. Sources: root `AGENTS.md`, +[docs/AGENTS.md](../../../../docs/AGENTS.md), [docs/versioning.md](../../../../docs/versioning.md) and +ADR-0000/0001/0543/0740/0741. Only the formats are checked by tooling (`test_changelog_format.py`, +`test_packages_workflow.py`). Whether each obligation was met is for the reviewer. + +## CHANGELOG.md + +- A bullet for this change under the right heading of the right version: `## Features` for new behaviour, + `## Security Fixes` for vulnerabilities and DoS paths (audit findings included), `## Bug fixes` for every other + correction (test and CI fixes included). `## Release notes` is one paragraph an operator must read before + upgrading. +- The bullet says what changed for the operator or client, not how, and ends with its ADR or finding ID. +- `CHANGELOG-pre-release.md` is frozen (ADR-0740). Any diff to it is a finding. +- A docs-only branch adds its bullets to the **current** version's entry and does not bump. + +## Version + +- One bump per branch, BUILD + 1, except documentation-only changes (ADR-0464). `AGENTS.md` and coding-guidance + edits count as documentation. Any code, build, packaging or test change means the bump applies. +- Every place agrees with `meson.build`. The list is in `docs/versioning.md`: `src/main.cpp`, + `src/db_migrate.cpp`, the FreeBSD manifest, the NetBSD Makefile, three RPM specs (each also with a new dated + `%changelog` entry), the `scripts/build-*.sh` scripts, and the CHANGELOG heading. +- The number must not be taken by another open branch. Check before approving. + +```bash +git show origin/main:meson.build | grep -m1 "version:"; grep -m1 "version:" meson.build +grep -n 'version = std::string_view' src/main.cpp src/db_migrate.cpp +grep -n '^Version:' packaging/{rpm,rhel,opensuse}/merovingian.spec +grep -nE '^VERSION=' scripts/build-*.sh +for r in $(git for-each-ref --format='%(refname)' refs/remotes/origin); do + git show "$r:meson.build" 2>/dev/null | grep -m1 -o "version: '[^']*'" | sed "s|^|$r |"; done | sort -k3 | tail +``` + +## Docs + +Every change to observable behaviour updates its owning doc in the same branch. Update existing files; do not add +per-feature notes. Table (from `docs/AGENTS.md`): + +| Change | Doc | +|---|---| +| Project-wide coding rule | `docs/coding-rules.md` | +| Security rule in a module `AGENTS.md` | `docs/security-coding-rules.md` too | +| Testing convention | `docs/testing-standards.md` | +| Auth, sessions, identity | `docs/auth-identity.md` | +| Crypto interface, keys, signing | `docs/crypto-boundary.md` | +| Schema, migrations, store interface | `docs/database-persistence.md` | +| Event parse, sign, hash, auth, state resolution | `docs/event-engine.md` | +| HTTP, TLS, rate limits, connections | `docs/http-transport.md` | +| Media | `docs/media-repository.md` | +| New attack surface or mitigated threat | `docs/threat-model.md` | +| Log fields, audit events | `docs/observability-audit.md` | +| Policy engine | `docs/trust-safety.md` | +| Module structure, cross-module dependencies | `docs/architecture.md` | +| Hardening, seccomp, sandbox, build flags | `docs/hardening.md` | +| Config key, default, reload policy, CLI flag, exit code, admin route | `docs/user-manual.md` | +| Logger module or diagnostic field | `docs/log-filtering.md`, `docs/debug-logging.md` | +| Dependency change | `docs/dependencies/*.md` | +| Client-Server endpoint | `docs/matrix-v1.19-client-server-api.md` | +| Gap closed or found | `docs/todos/capability-gaps.md`, `docs/todos/production-milestone.md` | +| New module | module `AGENTS.md`, its `CLAUDE.md` with `@AGENTS.md`, and rows in the root `AGENTS.md` tables | +| Doc added, removed or renamed under `docs/` | `mkdocs.yml` nav | +| Something the review skill cites changed | `.agent/skills/merovingian-review/` | + +The usual miss: the user manual and CHANGELOG are updated, but the owning guide is not. Also check for comments +or docs that the change makes false. + +## ADR + +- Needed when a future reader would otherwise undo the decision by accident. That covers a constraint the code + relies on but does not state, a rejected alternative that looks better in isolation, and a rule about how to + write future code. Not needed for a fact with no alternative, or for spec-mandated behaviour. +- Copied from `docs/adr/template.md`, with `* Status:` and `* Date:`. Numbered above every number that exists on + **any** branch, never reused. A line goes in `docs/adr/index.md` in the same commit. Supersession is marked, + never deleted. +- A diff that reverses an existing ADR's decision without superseding it is a finding. + +```bash +for r in $(git for-each-ref --format='%(refname)' refs/heads refs/remotes/origin); do + git ls-tree --name-only "$r" docs/adr/; done | grep -oE '[0-9]{4}-' | sort -u | tail -3 +git diff --name-status ... -- docs/adr +git diff ... | grep -oE 'ADR-[0-9]{4}' | sort -u | sed 's/ADR-//' | while read n; do + ls docs/adr/$n-*.md >/dev/null 2>&1 || echo "cited but missing: ADR-$n"; done +``` + +## Pull request description + +Headings: **Summary**, **What changed** (bullets), **Why it changed**, **CI tests** (bullets of modified tests, +then bullets of new tests). Check that the CI tests list matches the test files in the diff, and that every claim +in the description is supported (see SKILL.md Step 1). + +## Branch + +Not `main`. Based on a current `origin/main`, or on the PR it is stacked on, with the merge order stated. diff --git a/.agent/skills/merovingian-review/references/recurring-defects.md b/.agent/skills/merovingian-review/references/recurring-defects.md new file mode 100644 index 000000000..d3708cd51 --- /dev/null +++ b/.agent/skills/merovingian-review/references/recurring-defects.md @@ -0,0 +1,200 @@ +# Recurring defects + +The defect classes this project has shipped and then fixed, with the question that would have caught each. The +instances cite the ADR, version or PR, so a reviewer can read what happened. Ask every applicable question of +the diff, and record the classes you checked in the evidence log. + +## Locks, pools and concurrency + +**L1. Expensive work under `runtime.mutex`.** A sliding-sync sort comparator rescanned the store on every +comparison (ADR-0171/0172). Other instances: a quadratic multipart boundary scan (ADR-0176), an uncapped +account-data scan (ADR-0173), per-member regex compiles, push-glob allocation per character (ADR-0175), and a +quadratic expired-upload sweep (ADR-0589). Every high finding of the 1.0.0 release-candidate audit was of this +family (ADR-0693, ADR-0694, ADR-0713). +> For each loop, lookup or match the diff adds on a request path: what is its cost in attacker-controlled input +> and in total store size? Does it run under the lock? Where is the index, the cap and the linear matcher? + +**L2. A lock held across network, IPC or SQL waits.** The IPC reader ran handlers inline and deadlocked against +`pdu_ingest` (0.10.29). The recursive-release stall shipped three times (ADR-0002 to ADR-0006). The auth chain is +now written with the lock released (ADR-0714). +> Does any new call to the network, the worker or a slow store path happen with the lock held? After the +> release, does the code re-validate before acting? Does it use the one release primitive? + +**L3. A shared pool starved.** TLS long-polls exhausted the main pool and starved federation (0.9.1). Remote media +fetches took a dedicated pool (ADR-0121). Parked waits and per-key budgets: ADR-0570, ADR-0719, ADR-0720/0721. +> Which pool thread blocks, for how long, and who else shares that pool? Is there both a per-key cap and a +> global cap? + +**L4. Wake-ups not scoped.** Element X polled every 70–150 ms because wake-ups ignored what the connection asked +for (0.10.50–0.10.53). A destructor broadcast undid per-user scoping (ADR-0437). The relevance check was unbounded +(ADR-0609). +> Does a new publish wake only its audience? Does the generation move even when positions do not? + +**L5. Thread and resource lifetime.** Thread scopes published stack addresses (ADR-0529). A blob was released +twice (ADR-0636). `IpcChannel::stop()` needed `shutdown()` before `close()`. `std::jthread` is missing on BSD libc++ +(ADR-0167). +> Is every thread joined and every resource released on every exit path, including exceptions and early returns? + +## Worker and trust boundaries + +**W1. A main-side handler without its worker path, or the reverse.** `send_join`/`leave`/`knock` accepted on the +worker were never relayed, so joins were invisible to main (0.10.19). The worker's key resolver returned nothing, +which failed dozens of Complement tests with 403 and then 429 (ADR-0263). Backfill and worker-relay paths skipped +the restricted-join signature check (ADR-0171). The OTK claim relay was not bound to a request +(GHSA-3p5v-vrg8-f545, ADR-0595). +> Which process runs each new federation, key, EDU or membership handler? Does the worker have its relay or +> snapshot path? Is the relay bound to a main-verified request? Does a test drive it through a real worker +> subprocess? + +**W2. The worker given too much.** The worker could recover the signing key (0.12.12 audit; ADR-0015, ADR-0078). +It opened a database until 1.0.1 (ADR-0733 to ADR-0736). One bad frame ended the whole channel (ADR-0321, +ADR-0322). +> Does new code give the worker a credential, a database handle or a readable path? Does a bad frame fail one +> request or the channel? + +**W3. Sandbox filters wrong or silently absent.** An off-by-one in the BPF made worker seccomp allow everything, +and its real-kernel test skipped (0.12.19). Other instances: `fcntl` and `ioctl` commands (ADR-0642/0643), +`TIOCSTI` (ADR-0731), and a NetBSD decoder with no confinement that reported success (ADR-0419). Capsicum forbids +opens after `cap_enter` (ADR-0739). +> Is every new syscall, open or connect in a sandboxed process allowed by every filter that stacks? Does a +> real-kernel test fail, rather than skip, when the filter is wrong? Does an unsupported platform refuse? + +## Verification, authorisation and visibility + +**V1. A check on one ingestion path but not its siblings.** The restricted-join signature check was missing on +relay and backfill (ADR-0171). `authorising_user_member` was never filled in on one path (0.12.32). +`prev_events` were not bound to the PDU's room (ADR-0583). Alias and encryption content checks did not run for +every local writer (ADR-0588). +> List every path that stores an event or applies state: `/send`, send_join/leave/knock, worker relay, +> backfill, invites, local writers, appservices. Did the diff change one and leave the others? + +**V2. A structural check where a cryptographic or semantic one is needed.** `event_has_signature_from_server` +accepted any non-empty string. Every known unexpired key must verify (ADR-0183). Retired keys never authenticate +(ADR-0425). Redaction authority comes from room state (ADR-0586). +> Could a forged value of the right shape pass? Is the test's forged input signed with a wrong key? + +**V3. Visibility leaks.** A user who had left paged past the leave in `/messages` (ADR-0311, ADR-0357). Other +instances: `/members?at=` (ADR-0414), shared history served to a server with no user in the room (ADR-0585), and +lazy-loaded state of a left room (ADR-0450). +> What does a user who left, was banned or never joined get from this path? What does the spec say for each +> `history_visibility` value? + +**V4. Whose authority.** An appservice acted as server administrator (ADR-0611, ADR-0628). A join was sent by +someone other than the joining user (ADR-0612). A resident claim moved a local user's membership (ADR-0630). A +deactivated account's token still worked (ADR-0626). Operator routes dispatch before the token gate. +> Whose authority does this path run under? Is it checked on every route that reaches it, including routes +> dispatched before the general token gate? + +## Input and bounds + +**I1. Unbounded per-user or per-request collections.** Fallback keys (GHSA-3p9g-gpvx-2cx9, ADR-0594). Others: +account data (ADR-0173), aliases and devices (ADR-0582), invites and knocks (ADR-0619/0620, ADR-0707/0708), +createRoom initial state (ADR-0316), key backup (ADR-0606), filter lists (ADR-0657), sliding-sync `rooms` arrays +(ADR-0174, ADR-0663) and thumbnail output (ADR-0641). +> Where is the cap, what does hitting it return (`400 M_TOO_LARGE`, ADR-0173), and does a test hit it? + +**I2. Matcher and parser complexity.** Push globs (ADR-0175, ADR-0343, ADR-0607), search (ADR-0330), ACL globs +(ADR-0640), filter `*` (ADR-0541), directory search (ADR-0724), unread counts (ADR-0655). +> Is the matcher iterative and linear, with a step budget? Is there any `std::regex` or recursion on attacker +> input? + +**I3. Encoded versus decoded identifiers, and path versus path-plus-query.** Shard routing hashed the raw `%21` +room ID but the notifier used the decoded one, which gave a permanent `make_join` 404 (0.10.18). Thirty-four +handlers compared the target with the query string attached (ADR-0420). The `?filename=` upload route was missed +(0.9.13). +> Is each path identifier decoded exactly once, before every compare, hash and lookup? Does route matching use +> the path alone? + +**I4. A status code, error or field set the spec does not say.** Audit expectations of 403 contradicted the spec +(200 before the ban; 404). Unknown endpoints answer `404 M_UNRECOGNIZED` before auth (ADR-0230). +Federation errors are spec-shaped (ADR-0280). Membership answers carry spec fields only (ADR-0538). Transaction +limits use spec values (ADR-0539). +> Does the expected status, errcode or field come from a cited spec section, not from Synapse, Complement or an +> audit report? Where the spec is silent, is the project convention ADR followed? + +## Sync + +**S1. Missing or wrong response fields.** Timeline and `required_state` events lacked `event_id`, and Element X +silently dropped the messages (0.10.5x). `limited` and `prev_batch` were computed from the store size (0.5.16). +Sliding-sync timelines had no `limited` or `prev_batch` (0.12.45). `full_state` was parsed and ignored (0.12.73). +> Diff the response against the spec schema field by field. Does a test assert content? + +**S2. Token and ordering confusion.** To-device `stream_id` was stored as TEXT and sorted "10" before "9" +(ADR-0426). A token ahead of the streams poisoned the counter (ADR-0315). The restore epoch is now in tokens +(ADR-0592). The lazy-member ack keys on the exact `next_batch` (ADR-0450). +> Which token family does each field use? Is the ordering numeric? Can a client abort a long-poll and retry +> without losing data? + +## Persistence + +**P1. Not atomic, or memory and store disagree.** A federated state commit let a local write in between +(ADR-0696). Admin actions and their audit rows were separate (ADR-0629, ADR-0687). A join was committed in parts +(ADR-0714). `/keys/upload` stored and announced keys before judging the whole body (0.12.76). Hand-written +`PersistentStore` copy operations omitted later members (0.12.79). +> What is the state if write N of M fails or the process dies? Is it one transaction, with memory updated after +> commit? Did a new member reach every hand-written copy and serialise path? + +**P2. Identifier reuse and number collisions.** One transaction ID was reused across destinations although it +was the primary key (0.12.34). The `remote-media` localpart could forge the quota sentinel (ADR-0178). Parallel +work duplicated migration and ADR numbers (ADR-0543). +> Is the identifier unique for its key? Do migration and ADR numbers collide with other open branches? + +## Outbound and configuration + +**O1. Outbound hygiene.** CRLF injection via `id_access_token` (ADR-0180). Others: the URL-preview SSRF gate +(ADR-0213), appservice cleartext rules (ADR-0515), the four-redirect limit for `.well-known` (ADR-0483), and +failures charged to the wrong party (ADR-0081, ADR-0346, ADR-0347). +> Is the answer trusted? Is it bounded in bytes, time and redirects, and counted against an in-flight budget +> (ADR-0079)? Did `tests/tooling/test_outbound_answer_trust.py` run? + +**O2. Configuration nothing enforces.** `security.encryption.*` was documented but never read (ADR-0445/0446). +A migration lock key was declared and never used. A documented key is enforced or removed (ADR-0498). +> Grep for the identifier being **used**, not only defined. Does a new key have a reader, a test that flips it, +> and a doc row? + +## Build, platform and tests + +**B1. Green on one toolchain, red on another.** GCC-only `-Werror=null-dereference` under LTO, and `-Wshadow` on +lambda parameters. BSD `#if` blocks and fuzz targets are not in the WSL compile database. NetBSD `res_query` has +one global state (ADR-0355). An fd-sweep test closed libsodium's cached `/dev/urandom` descriptor on NetBSD. +> Is any changed code in a platform `#if` block, a file outside the compile database, or a GCC-only warning +> path? Was it compiled there? + +**T1. Tests that cannot fail.** A script ran tags after the build failed, so the old binary passed. PostgreSQL +scenarios "skipped" with success. An unreadable `/proc//fd` was counted as zero (1.0.1). An unsigned +difference was compared with zero (1.0.1). A network guard with hidden visibility passed its own scenario while +libcurl walked past it (0.12.77). +> Was each new test seen failing against the code without the fix? + +**T2. Tests pinning the old behaviour, or the author's assumption.** Six older assertions expected a stranger to +read shared-era history, while tag-filtered runs stayed green (0.12.83). Audit "Test:" lines were taken as spec. +> Does the diff edit an existing assertion? Is the new expectation cited from the spec? Was the full suite run? + +## Process + +**R1. Delegated-work defects.** Declared-but-unwired constants. Invented APIs. Wrong expectations written +confidently. Whole-file reformatting. Workstreams green alone but broken together. +> Is every new identifier called somewhere real? Are hunks confined to the regions the change needed? Was the +> merged tree built? + +**R2. Docs, ADR and CHANGELOG drift.** Branches update the user manual and CHANGELOG but not the owning guide: +threat model, auth-identity, trust-safety, media repository, capability gaps. Comments cite deleted audit reports. +ADRs cite numbers that have no file (ADR-0543). +> Which docs in the `docs/AGENTS.md` table does the change invalidate? See [obligations.md](obligations.md). + +## Decisions that look like bugs but are not + +Do not report these as defects. Each is recorded: + +- No HTTP shutdown route (ADR-0505). +- Cap hit answers `400 M_TOO_LARGE` (ADR-0173). An oversized `/send` event type stays 413. +- `make_join` with no tips answers 404 (ADR-0318). +- Refused send_join/leave/knock events are never stored (ADR-0591). +- `max_transaction_pdus` and `max_transaction_edus` accept only the spec values (ADR-0539). +- The worker opens no database (ADR-0733). +- A worker that cannot start makes the server exit with code 80 (ADR-0377). +- NetBSD is refused at the BSD gate on purpose (ADR-0378). +- `security.encryption.*` is enforced at room creation only (ADR-0446). +- Digit separators (`900'000U`) stay. A metric tool that miscounts them gets a false-positive dismissal, not a + code change. +- Related fixes that serve an open branch's goal share that branch and its version. Do not ask for a split. diff --git a/.agent/skills/merovingian-review/references/rule-checks.md b/.agent/skills/merovingian-review/references/rule-checks.md new file mode 100644 index 000000000..03870e0ff --- /dev/null +++ b/.agent/skills/merovingian-review/references/rule-checks.md @@ -0,0 +1,96 @@ +# Rule checks + +Checks drawn from [docs/coding-rules.md](../../../../docs/coding-rules.md) (CR) and +[docs/security-coding-rules.md](../../../../docs/security-coding-rules.md) (SCR), cited as `doc#heading`. The +documents are binding. These lines are for finding where to look. + +## Already enforced: do not re-report + +Report these only if the diff edits or bypasses the gate itself. + +| Gate | Runs in | Fails on | Misses | +|---|---|---|---| +| `scripts/reject-unsafe.sh` | CI (`ci.yml`, `static-analysis.yml`) and pre-commit | naked `new`/`delete`/`malloc`/`calloc`/`realloc`; `(void)` and `static_cast` discards; `shared_ptr` without `// SHARED_PTR: reviewed — `; `.unlock()` without `// LOCK_RELEASE: reviewed — `; libsodium outside its directories | whether the "reviewed" reason is **true**; patterns split across lines; files outside its roots | +| `check-catch2-bdd-tests.sh` | meson test | `TEST_CASE`, missing SCENARIO/GIVEN/WHEN/THEN words in `tests/unit` | per-scenario structure; conformance and integration files | +| `check-unit-test-registration.sh` | meson test | unit or conformance file not in `tests/meson.build` | integration files | +| `check-unit-test-isolation.sh` | meson test | unit file naming `start_runtime` and the other runtime builders | a helper that builds a runtime | +| `tests/tooling/test_include_order.py` | meson test | include grouping and order | — | +| `tests/tooling/test_changelog_format.py` | meson test | CHANGELOG heading and bullet format; newest entry equals `meson.build` version | whether the bullet describes **this** change | +| `tests/tooling/test_line_endings.py` | meson test | CRLF | — | +| clang-tidy | CI `static-analysis.yml` | its configured checks | — | + +**Not enforced in CI:** `check-staged-changelog-docs.sh` runs only in the local pre-commit hook, which may not be +installed. The format tool `scripts/format_code.py` runs by hand only. So a missing CHANGELOG bullet or doc +update, and formatting drift, are review items. Report formatting in one line, never per hunk. + +## Grep on the changed lines + +Run each pattern over `git diff -U0 ...` added lines (`^\+`), then read each hit in context. + +| Check | Pattern | Source | +|---|---|---| +| Raw fd held as `int` outside `FileDescriptor` | `\bint\s+\w*fd\b` | SCR#Memory safety and resource ownership | +| fd or socket without CLOEXEC | `\b(socket\|accept\|open\|pipe\|dup2?)\(` | SCR#Memory safety and resource ownership | +| Unchecked narrowing | `static_cast<(u?int\d+_t\|int\|short\|unsigned)>` on sizes | CR#Errors and function contracts | +| Destructor without `noexcept` | `~\w+\(\)` | CR#Errors and function contracts | +| `sodium_memzero` outside the crypto boundary (use `core::secure_zero`) | `sodium_memzero` | SCR#Memory safety and resource ownership | +| Non-constant-time secret compare | `memcmp\|std::equal\|==` near token, hash, secret | SCR#Cryptography | +| Log to stdout or stderr instead of `SingleLog` | `std::cout\|std::cerr\|fprintf\(stderr` | SCR#Secrets and logging | +| Config or env read outside the snapshot | `getenv\|secure_getenv` | SCR#Secrets and logging | +| Interpolated SQL, ad-hoc DDL, backend downcast | `ALTER TABLE\|dynamic_cast<.*(Sqlite\|Postgres)`; string-built SQL | SCR#Database | +| Bare SQLite prepare or step | `sqlite3_prepare_v2\|sqlite3_step` | SCR#Database | +| Clearing a revoked flag | `revoked\s*=\s*(false\|0)` | SCR#Database | +| Direct header indexing | `headers\[` | SCR#HTTP and network boundary | +| Ad-hoc HTTP or cleartext | `curl_easy_\|allow_cleartext_http` | SCR#HTTP and network boundary | +| Blocking socket mode, raw TLS I/O | `O_NONBLOCK\|SSL_read\|SSL_write\|TIMEO` | SCR#HTTP and network boundary | +| ID split on the last colon | `rfind\(':'\)` | SCR#Authentication and authorization | +| Hard-coded room version logic | `room_version\s*==` | SCR#Federation | +| Regex or naive search on caller input | `std::regex\|regex_search\|std::search\|\.find\(` | SCR#Sync; SCR#Push notifications | +| Seccomp without TSYNC | `seccomp\(` | SCR#Platform hardening | +| Recursive mutex swap | `recursive_mutex` | CR#Concurrency and portability | +| Banned concurrency primitives | `jthread\|stop_token\|stop_source\|counting_semaphore` | CR#Concurrency and portability | +| Mutex behind a pointer in a copyable type | `(unique\|shared)_ptr` | CR#Concurrency and portability | +| `mutable` cache of a non-thread-safe type | `\bmutable\b` | CR#Concurrency and portability | +| Test-only master key in production code | `support/MasterKey` outside `tests/` | SCR#Testing requirements for security-critical code | +| Unpinned third-party action | `uses: [^./][^@]+@v` | CR#Change workflow | +| `using namespace` in a header | `using namespace` in `include/` | CR#Design and interfaces | +| Bare project include | `#include "[^m]` | CR#Calls, formatting, and includes | +| A second type in one file; file stem differs from the type name | new `class\|struct\|enum` in a file that already has one | CR#Design and interfaces (ADR-0514, ADR-0596) | + +## Judgement checks + +| Question | Source | +|---|---| +| Is inbound federation authenticated before any body byte is read? Is a bad signature charged to the source address, not the claimed origin? | SCR#Federation | +| Does every inbound PDU path verify signatures against the sender's keys before the graph? | SCR#Federation | +| Does main re-verify everything the worker relays, bind relays to a main-verified digest or permit, and judge ACL and policy itself? | SCR#Federation; `docs/security-review-checklist.md#Federation worker IPC` | +| Is the event or state change authorised (room-version-aware) before it is persisted? Is authority taken from recorded state rather than the event's cited `auth_events`? | SCR#Authentication and authorization | +| Does local membership change only through `persist_composed_event`, in one transaction with the event and `current_state`? | SCR#Authentication and authorization | +| Is `runtime.mutex` released around every blocking call, with protected state untouched while released? | CR#Concurrency and portability; SCR#HTTP and network boundary | +| Does every client-triggered outbound call take an `admit_client_outbound_proxy` slot and run under one `OutboundDeadline`? | SCR#HTTP and network boundary (ADR-0079) | +| Is everything a client or remote can grow capped and refused, with cost at most linear under the lock? | SCR#HTTP and network boundary; SCR#Sync; SCR#Push notifications | +| Does a large-body route authenticate before reading, with an explicit opt-out from the 64 KiB cap? | SCR#HTTP and network boundary | +| Do identity, push, media and preview fetches resolve through `deny_ip_ranges`, over HTTPS? | SCR#Identity Service client; SCR#HTTP and network boundary | +| Are variable-length secrets hashed with domain separation before comparison? | SCR#Cryptography | +| Is a secret file checked and read through one descriptor? | SCR#Cryptography | +| Do client read paths use `HistoryVisibility` and `room_read_access_for`? Can a not-found or presence answer enumerate users? | SCR#Sync | +| Is every unauthenticated-triggerable audit event rate-gated and size-bounded? | SCR#Secrets and logging | +| Does every hardening step run before the first socket and thread? Does each isolated child get its own profile? | SCR#Platform hardening | +| Is every `noexcept`, `[[nodiscard]]`, `SHARED_PTR` and `LOCK_RELEASE` marker's rationale actually true? | CR#Ownership and resource lifetime; CR#What automated checks cover | +| Did a performance change skip validation, bounds, authorisation, signature checks or redaction? | SCR#Process | +| Does a new shared type have a concurrency scenario? | SCR#Testing requirements for security-critical code | +| If a module `AGENTS.md` security rule changed, is `docs/security-coding-rules.md` changed with it? Is a new grep-detectable rule added to `reject-unsafe.sh`? | `docs/AGENTS.md`; `security/AGENTS.md#Rules` | + +## Known inconsistencies in the guidance + +When a finding rests on one of these, read the code and cite what it actually does. + +- `docs/coding-rules.md` names `RuntimeLockRelease.hpp` for releasing the runtime lock, while + `docs/security-coding-rules.md` names `RequestLockScope.hpp`. Both headers exist. Check which one the code + path uses before citing either. +- `docs/security-coding-rules.md#Federation` states the inbound-event tip-status rule twice: per room for senders + without power (ADR-0695), and "never per room". The ADR is the tie-breaker. +- `[[nodiscard]]` is "all functions" in `src/AGENTS.md`, "returned results" in CR, and "prefer" in + `include/merovingian/AGENTS.md`. Hold a diff to CR. +- `src/federation/AGENTS.md` rule 1 says to answer 401 for a bad X-Matrix signature. Check the handler and the + spec before citing a status code. diff --git a/.agent/skills/merovingian-review/references/tests.md b/.agent/skills/merovingian-review/references/tests.md new file mode 100644 index 000000000..7e52343db --- /dev/null +++ b/.agent/skills/merovingian-review/references/tests.md @@ -0,0 +1,89 @@ +# Reviewing tests + +The binding text is [docs/testing-standards.md](../../../../docs/testing-standards.md) and the `tests/*/AGENTS.md` +files. This is the reviewer's cut of them. + +## Where a test belongs + +| Directory | Belongs | Does not belong | +|---|---|---| +| `tests/unit/` | One module in isolation; pure logic; SQLite only for database behaviour; concurrency scenarios for thread-safe types | Anything that builds a runtime (`start_runtime`, `start_client_server`, `wire_federation_callbacks`) or drives the client-server stack (ADR-0593); spec MUST/SHOULD checks; network | +| `tests/conformance/` | One Matrix v1.19 MUST or SHOULD per scenario, `test__conformance.cpp` | Internal-structure tests | +| `tests/integration/` | Two or more modules, a live DB, a listening server, full flows, `test__flow.cpp`; real dependencies, no mocks | Live internet, except opt-in under `build_live_tests` | +| `tests/fuzz/` | One parser fed raw bytes; asserts no crash or hang only | Correctness | +| `tests/smoke/` | CLI probes under 5 s, no external services | HTTP listeners, correctness | +| Fuzz regression | A SCENARIO in `tests/unit/test_.cpp` tagged `[fuzz][regression]`, plus the minimised input in `tests/fuzz/corpus//` | | + +Sources: `tests/unit/AGENTS.md#Unit vs. other test directories`, `tests/integration/AGENTS.md#What belongs here`, +`tests/conformance/AGENTS.md#Purpose`, `docs/testing-standards.md#Unit or integration`. + +## Required structure (check by hand where no gate does) + +- `SCENARIO` with `GIVEN`, `WHEN` and `THEN` in every Catch2 test file (unit, conformance, integration), not only + unit. Setup goes in GIVEN, one action in WHEN, assertions in THEN. The BDD gate checks only `tests/unit`, and + only that the words appear somewhere in the file. Smoke tests (CLI probes registered in + `tests/smoke/meson.build`) and fuzz targets are not Catch2, so the rule does not apply to them. +- Tags: unit `[module]` matching the directory; conformance `[conformance]` plus an area tag; fuzz regression + `[][fuzz][regression]`. **A scenario that reaches PostgreSQL carries `[postgresql]`.** That tag is the + whole of what the PostgreSQL CI job runs (ADR-0511). An untagged one skips in CI and passes having tested + nothing. `tests/tooling/test_ci_run_scope.py` reads only the scenario's own text, so it misses a helper that + opens the connection. +- Conformance: a citation block above every SCENARIO (Spec, Endpoint/Section, URL into `docs/matrix-v1.19-spec/`, + the MUST in a sentence). Never weaken, remove or comment out a `REQUIRE`. Never change an expected value + without citing the spec section. The spec outranks Synapse, Complement and audit reports. + `tests/conformance/AGENTS.md#Rules — Non-Negotiable` +- Security-sensitive code needs negative paths: invalid input, boundary sizes, and no partial state left on error. + `tests/unit/AGENTS.md#Security-sensitive code` +- A forged-signature test signs with the **wrong key**. An unsigned input proves only the structural check. +- A thread-safe type needs a concurrency scenario: many threads, a release barrier, a result checked per call. The + model is the scenario in `tests/unit/test_outbound_client.cpp`. +- Registration: unit and conformance files go in `tests/meson.build` (gated). Integration files go in the + `integration_tests` source list (**not gated**: check it). A new test binary that registers users needs the + `crypto_pwhash_str` wrap and `RealPasswordHashCost.cpp` (ADR-0510). +- A new `SKIP()` needs an entry in `tests/expected_skips.json` for every CI job that skips it (ADR-0546). +- A helper goes in `tests/support/` only when two or more files use it. + +## Pitfalls that have produced false greens + +- **Catch2 assertions are not thread-safe.** Collect results on worker threads, and assert on the main thread. +- **A joinable `std::thread` in scope when a `REQUIRE` fails or a `SKIP` fires calls `std::terminate`**, which kills + the whole binary. Mock-server threads use `ScopedThreadJoin` or `JoiningThreads`. Never use `std::jthread`. +- **Minimum-cost Argon2 cannot exercise a race** around the hash window. A race scenario holds + `RealPasswordHashCost` from its first line. `docs/testing-standards.md#Password hashing cost in the test binaries` +- **Blocking `::connect()`** in integration tests hangs on OpenBSD. Use `connect_within`, and bound reads and + writes. +- **Live PostgreSQL scenarios share one database.** A scenario asserting on all writes uses + `ScratchPostgresqlDatabase` (ADR-0227). Do not filter statement names to make it pass. +- **TLS test servers** trust the test CA, not a self-signed certificate. + `docs/testing-standards.md#Live join_room integration tests` +- **`wire_federation_callbacks` after `remote_key_resolver`** overwrites the test double. +- **Timing-dependent scenarios** use `test_frozen_event_clock_ms`, not loops. +- **An older test pinning the old behaviour.** When the change alters a visibility, auth or cap rule, search for + tests that asserted the old outcome, and run the **full** suite, not only the new tags. Tag-filtered runs have + been green while older assertions elsewhere failed. +- **Tests that check shape, not content.** A sync test that asserts a field exists does not prove the right events + are in it. +- **A real-kernel seccomp test that SKIPs** when the filter is wrong hides an allow-all filter. It must fail. +- **A fixture key that expired** made tests reach real hosts. Integration tests must pass with no internet. + +## Running and reading + +- Build and test with the command the root `AGENTS.md` names. Run it in the background and wait. Read the + `Ok:`, `Fail:` and `Timeout:` counts from `meson-logs/testlog.txt` in that build directory. A timeout is a + failure. Lines with a zero count may be omitted. +- The log is evidence only if its modification time is after the build started. A failed compile leaves the + previous log and the previous binaries in place. +- Prove a scenario ran with a tag filter: + `python build.py vm --exec ./build-vm/tests/merovingian-unit-tests "[push]"`. `--exec` sets + `MEROVINGIAN_TEST_DISABLE_HARDENING=1`, which any direct run of a test binary needs. Expect + `All tests passed (N assertions in M test cases)` with M at least 1. Empty output for a new tag means the binary + is stale. A zero-byte binary exits 0. +- Binaries and meson timeouts: `merovingian-unit-tests` (600 s), `merovingian-conformance-tests` (900 s), + `merovingian-integration-tests` (1800 s; 900 s on the BSDs), `merovingian-password-cost-tests`. +- A second suite running on the same machine causes unrelated timeouts. Re-run alone; do not raise the timeout. +- **Fails-if-reverted check:** make a worktree at the base commit, apply only the test changes + (`git diff ... -- tests/ | git apply`), build, run the tag. The new scenario must fail. If it + passes, it does not test the fix. +- Platform code inside `#if` blocks for BSD is not compiled on Linux. Syntax-check it with + `-fsyntax-only -U__linux__ -D__FreeBSD__` using the flags from `compile_commands.json`. GCC-only warnings + (`-Wshadow` on lambda parameters, `-Werror=null-dereference` under LTO) need a GCC build. diff --git a/AGENTS.md b/AGENTS.md index bae541926..a77e1c855 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -100,7 +100,7 @@ not hypothetical — each has already produced a "complete, all tests passing" report over work that was broken: - **"Tests pass" from a run the agent never read.** Read the `Ok:` / `Fail:` / - `Timeout:` counts out of `build-wsl/meson-logs/testlog.txt` yourself. + `Timeout:` counts out of `build-vm/meson-logs/testlog.txt` yourself. - **A green suite that never ran the new tests.** Confirm with a tag filter against the test binary — see "Verifying Work" below. - **A sanitizer or tool run that aborted at startup** and was reported as clean @@ -126,10 +126,21 @@ because each one has already cost this project real time. running, and never say you will "follow up with results later" — a session that ends does not follow up, and the work lands unverified. Run the suite, wait for it, read the outcome, *then* report. -- **`python build.py wsl` exceeds a 10-minute foreground limit.** Run it in the - background and wait for it to finish. -- **Read the real summary** from `build-wsl/meson-logs/testlog.txt` — the `Ok:` / - `Fail:` / `Timeout:` counts at the end. Report those numbers, not an impression. +- **Build and test on the Linux VM with `python build.py vm`, not WSL.** It syncs the + working tree, uncommitted edits included, and runs the suite on the VM; see + [Building on a VM](docs/dev-environment.md#building-on-a-vm) and + [ADR-0752](docs/adr/0752-build-py-vm-syncs-the-working-tree-over-ssh-and-runs-build-linux-sh.md). + `python build.py vm --vm freebsd` does the same on the FreeBSD VM (its `[vm.freebsd]` + settings), with its logs in `build-vm-freebsd/`. + It exceeds a 10-minute foreground limit: run it in the background and wait for it to + finish. A second run as the same VM user, from any worktree, waits for its lock rather than + competing; never run the suite there as a second user at the same time. +- **Read the real summary** from `build-vm/meson-logs/testlog.txt`, which `build.py vm` + copies back — the `Ok:` / `Fail:` / `Timeout:` counts at the end. Report those + numbers, not an impression. It deletes the old log first, so a missing file means no + result, never a pass: when `build.py vm` printed "the run wrote no testlog.txt" the + run never reached the tests, and when it printed "cannot copy ... back" the log is + still on the VM where it says; read it there before reporting anything. - **A TIMEOUT is a FAILURE.** "48 Ok, 0 Fail, 1 Timeout" is a failing suite. A hang is usually a deadlock, and a deadlock in a background task is a production shutdown bug, not a flaky test. @@ -137,14 +148,26 @@ because each one has already cost this project real time. name therefore cannot prove that scenario ran. To confirm a passing test executed, run the binary directly with a filter. - **Catch2 registers `SCENARIO("x")` as `"Scenario: x"`**, so filtering on the raw - scenario string matches nothing. Filter by tag instead: - `./build-wsl/tests/merovingian-unit-tests "[push]"`. + scenario string matches nothing. Filter by tag instead; the binaries are on the VM: + `python build.py vm --exec ./build-vm/tests/merovingian-unit-tests "[push]"`. - **Do not poll with `pgrep -f "meson test"` in a shell loop.** The loop's own command line contains that string, so it matches itself and spins forever. Bracket the first character (`pgrep -f "[m]eson test"`) or poll the log file's modification time. - **Verify delegated work yourself.** When a subagent reports green, confirm it from the log before committing. Trust the change, check the claim. +## Reviewing Changes + +Review a pull request, branch or diff, including delegated work and your own +change before reporting it complete, with +[.agent/skills/merovingian-review/SKILL.md](.agent/skills/merovingian-review/SKILL.md). +It checks the change against the coding rules, the module `AGENTS.md` +invariants, the spec, and the CHANGELOG, docs, ADR and version obligations. It +cites those documents rather than copying them +([ADR-0754](docs/adr/0754-agent-skills-are-committed-under-agent-skills-and-cite-the-binding-docs.md)). +Tools that load skills only from `.claude/skills/` do not find it on their own, +so read it directly. + ## Ignore list Ignore these dirs: - .claude/ @@ -214,7 +237,7 @@ the same name in `include/`: `src/appservice/RegistrationYaml.cpp`, `src/databas ├── meson.build # Root build definition ├── meson.options # Build options ├── mkdocs.yml # Documentation site (GitHub Pages); hooks in scripts/mkdocs_site_hooks.py -└── build.py # Unified build CLI (Linux, BSD, WSL) +└── build.py # Unified build CLI (Linux, BSD, WSL, remote Linux or BSD VM) ``` ### Key entry points @@ -326,11 +349,17 @@ Base: docs/matrix-v1.19-spec/index.md ## Build & Test Commands ```bash -# Build and test Windows +# Build and test from Windows, on the Linux VM (docs/dev-environment.md#building-on-a-vm) +python build.py vm + +# The same on the FreeBSD VM +python build.py vm --vm freebsd + +# Build and test inside WSL instead python build.py wsl # Development build via the unified CLI; a target is required -# (linux, bsd, wsl, deb, rpm, pkg, static) +# (linux, bsd, wsl, vm, deb, rpm, pkg, static) python build.py linux # Documentation site (see docs/dev-environment.md#documentation-site) diff --git a/CHANGELOG.md b/CHANGELOG.md index 9f405ae5b..b7732b7db 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -3,6 +3,82 @@ Changes from 1.0.0, the first production release, onwards; the format is set out Changes before 1.0.0 are in [CHANGELOG-pre-release.md](CHANGELOG-pre-release.md), which is no longer added to ([ADR-0740](docs/adr/0740-the-changelog-starts-at-1-0-0-and-the-pre-release-entries-are-a-frozen-record.md)). +# Version 1.0.2 + +## Features + +* `server.http.client_outbound_max_per_client` sets how many remote calls one client address may have in flight, + 1..128, never above `request_threads / 2`. The default is 3, up from a fixed 1 (ADR-0751). +* Agents review changes with `.agent/skills/merovingian-review/`, which checks module invariants, past defect classes, + tests and the CHANGELOG, docs, ADR and version obligations, citing the binding docs (ADR-0754). +* `python build.py vm` builds and tests on a Linux or FreeBSD VM over ssh, sending only changed files; it copies + `testlog.txt` back to `build-vm/`. `scripts/remote-host-setup.sh` prepares the VM (ADR-0752). +* `python build.py vm --vm ` uses the `[vm.]` settings and `build-vm-/`, so one machine can drive a + Linux VM and a FreeBSD VM (ADR-0752). +* `--jobs ` on `build-linux.sh`, `build-bsd.sh`, `build.py linux` and `build.py bsd` caps compile jobs and tests + (ADR-0752). + +## Security Fixes + +* A resident's make_join, make_leave or make_knock template can no longer put signatures or hashes of its choosing + into this server's membership event; they are dropped before signing, and an unparseable join is refused (ADR-0745). +* Every fix under Bug fixes keeps the audit fix it touches. Each one is narrowed to the case the spec requires, and is + covered by tests that cover the refusal as well (ADR-0745 to ADR-0751). + +## Bug fixes + +* A first join into a restricted room stores the resident's co-signature, verified exactly as for a join through + another resident, so it is no longer refused as unauthorised; an invalid one is still refused (ADR-0745). +* A first federated join stores the join's signatures in its `event_signatures` rows as well as its JSON, the + co-signature of a restricted join included; they were left empty, so a worker's snapshot had none (ADR-0745). +* A threaded receipt is accepted for an event whose relation chain reaches the thread root within 3 hops, such as a + reaction to a reference of the root (ADR-0746). +* A remote user who is joined, invited or knocking can leave, or reject an invite, through this server after its last + local member has left; strangers are still refused (ADR-0747). +* A federation read of a room, such as `/event_auth`, is answered by main while the worker is reloading the room, so + an event sent a moment ago is no longer reported as missing (ADR-0748). +* So are a remote user's `make_leave` and `make_knock`, so rejecting an invite a moment after it arrives builds a + leave that names the invite, and it is no longer refused with 403 (ADR-0759). +* A remote server's join, knock, leave or PDU for a room created a moment ago is served by main until the + federation worker has loaded the room, so it is no longer refused under a guessed room version (ADR-0756). +* So is a room the worker's start-up copy left out, a room first changed while the worker's channel was down, and a + whole `/send` transaction when any room it names is one of these, not only its first (ADR-0756). +* Every worker shard now reloads every changed room, and main tracks each shard's copy, so a `/send` naming rooms + of several shards is judged by one worker on current copies, never sent to main for that (ADR-0758). +* `docs/architecture.md` described the worker's room reload twice, once in a pre-1.0.1 form with a database read; it + now describes it once, through main, with main's providers under `main_store_read_lock` and `/hierarchy` on main. +* Outbound federation delivery never reuses a cached discovery failure, so a queued message reaches a server at the + first retry after it comes back, including after this server restarts (ADR-0749). +* The Complement image allows cleartext appservice URLs on `localhost`, where a Complement blueprint registers one; + the server's own rule and default are unchanged (ADR-0750). +* Tests name their scratch files uniquely and check each write, so a second account or a concurrent run on the + same machine no longer reads another run's leftover appservice registration and fails (ADR-0753, ADR-0755). +* Unit tests now cover a relation chain that stops short of the thread root (ADR-0746) and a `/send` body that names + no room (ADR-0756); only the cases that succeed were covered before. +* Two Complement tests whose expectations contradict the spec are expected failures in the skip list: an MXC-only + `avatar_url` (F-63, ADR-0406) and an omitted `end` at the start of a room (ADR-0470). +* `build.py linux --hardening` and `build.py bsd --hardening` work: both build scripts used to refuse the option they + were passed (ADR-0752). +* The live-binary startup test runs as a non-root user on the BSDs too: off Linux it accepts the unsandboxed worker, + as the startup hardening gate does (ADR-0041). It used to fail there. +* `docs/hardening.md` said `federation.worker.apply_hardening=false` alone runs the worker on FreeBSD and OpenBSD; + it needs `federation.worker.allow_without_landlock=true` too. It now lists the worker's hardening steps in order. +* On a host with an installed system config, the `--dry-run` smoke tests read the example config and the + default-summary test is skipped, not failed: the server reads that config instead of its defaults (ADR-0752). +* `build.py vm` resends a file an interrupted sync had deleted, and a retry after a sync interrupted inside a new + directory succeeds, even when the local files changed in between (ADR-0752). +* The VM setup steps in `docs/dev-environment.md` pass the setup key to `scp` and `ssh`, which do not read `vm.ini`. +* A hardened FreeBSD or OpenBSD build links as a PIE however it is built, not only as a package, and the ELF hardening + check reads the BSDs' `readelf`, so FreeBSD's release evidence no longer reports PIE and BIND_NOW missing (ADR-0757). + +## Release notes + +1.0.2 fixes the Complement regressions that followed the 1.0.1 security audit fixes. No configuration change is +needed. A remote member's own leave is now admitted in a room with no local member, which ADR-0591 had refused. +One client address may now have three remote calls in flight instead of one; set +`server.http.client_outbound_max_per_client=1` to keep the old limit. Windows developers can build and test on a +Linux or FreeBSD VM with `python build.py vm` instead of WSL. + # Version 1.0.1 ## Features diff --git a/build.py b/build.py index 6bad5e971..70e590a43 100644 --- a/build.py +++ b/build.py @@ -5,18 +5,51 @@ # Delegates to the shell scripts in scripts/ for each build target. import argparse +import configparser +import hashlib +import io import os import re +import secrets import shutil +import stat import subprocess import sys +import tarfile +from dataclasses import dataclass from pathlib import Path +from typing import BinaryIO, Mapping, Sequence REPO_ROOT = Path(__file__).resolve().parent SCRIPTS_DIR = REPO_ROOT / "scripts" PROFILES = ("debug", "release", "sanitizer", "coverage", "fuzz", "hardened") +# build.py vm (ADR-0752): the VM holds one checkout per local checkout under +# ~/merovingian-vm/, each with a manifest of what was last sent to it. +VM_REMOTE_ROOT = "merovingian-vm" +VM_MANIFEST = ".vm-sync/manifest" +VM_MANIFEST_NEXT = ".vm-sync/manifest.next" +VM_DEFAULT_BUILDDIR = "build-vm" +# One compile job per 1.5 GiB of VM memory unless --jobs says otherwise: a +# C++26 translation unit here can take more than 1 GiB to compile. +VM_MEMORY_PER_JOB_KIB = 1_572_864 +VM_TESTLOGS = ("testlog.txt", "testlog.json") +# The remote script exits with this status, and leaves VM_BASELINE_MARKER in its +# run directory, when the VM's manifest changed between build.py reading it and +# the script taking the lock; build.py then reads it again and resends. +VM_BASELINE_CHANGED = 75 +VM_BASELINE_MARKER = "baseline-changed" +VM_SYNC_ATTEMPTS = 3 +_VM_TOKEN_PATTERN = re.compile(r"^[0-9a-f]{16}$") +VM_SETTING_NAMES = ("host", "user", "port", "identity", "jobs") +_VM_HOST_PATTERN = re.compile(r"^[A-Za-z0-9][A-Za-z0-9.:-]*$") +_VM_USER_PATTERN = re.compile(r"^[A-Za-z_][A-Za-z0-9_.-]*$") +_VM_NAME_PATTERN = re.compile(r"^[a-z0-9][a-z0-9_-]*$") +_VM_BUILDDIR_PATTERN = re.compile(r"^build[A-Za-z0-9._-]*(/[A-Za-z0-9._-]+)*$") +_TESTLOG_SUMMARY_PATTERN = re.compile( + r"^(Ok|Expected Fail|Fail|Unexpected Pass|Skipped|Timeout):\s+\d+\s*$") + def quote_bash(value: str) -> str: return "'" + value.replace("'", "'\\''") + "'" @@ -86,9 +119,10 @@ def run_wsl(args: argparse.Namespace, extra: list[str]) -> int: return subprocess.call(wsl_cmd) -def add_common_dev_args(parser: argparse.ArgumentParser, builddir_default: str = "build"): +def add_common_dev_args(parser: argparse.ArgumentParser, builddir_default: str | None = "build", + builddir_help: str = ""): parser.add_argument("--builddir", default=builddir_default, - help=f"Meson build directory (default: {builddir_default})") + help=f"Meson build directory (default: {builddir_help or builddir_default})") parser.add_argument("--cc", default="clang", help="C compiler (default: clang)") parser.add_argument("--cxx", default="clang++", help="C++ compiler (default: clang++)") parser.add_argument("--wrap-mode", default="forcefallback", @@ -141,6 +175,8 @@ def build_linux(args: argparse.Namespace) -> int: cmd_args.append("--compile-only") if args.dry_run: cmd_args.append("--dry-run") + if args.jobs is not None: + cmd_args += ["--jobs", str(args.jobs)] return run_script(script, cmd_args) @@ -173,6 +209,8 @@ def build_bsd(args: argparse.Namespace) -> int: cmd_args.append("--compile-only") if args.dry_run: cmd_args.append("--dry-run") + if args.jobs is not None: + cmd_args += ["--jobs", str(args.jobs)] return run_script(script, cmd_args) @@ -180,6 +218,610 @@ def build_wsl(args: argparse.Namespace) -> int: return run_wsl(args, []) +class VmError(Exception): + """A vm setting, path or checkout that build.py refuses to work with.""" + + +@dataclass(frozen=True) +class VmSettings: + """Where and how build.py vm reaches the VM. jobs None means size by memory; + name is the --vm name ("" for the default [vm] section).""" + host: str + user: str + port: int + identity: str + jobs: int | None + name: str = "" + + +def positive_int(value: str) -> int: + """argparse type for --jobs: a whole number of at least 1.""" + if not value.isdigit() or int(value) < 1: + raise argparse.ArgumentTypeError(f"expected a positive whole number, not {value!r}") + return int(value) + + +def vm_config_path(env: Mapping[str, str]) -> Path: + """The per-user settings file: $MEROVINGIAN_VM_CONFIG, else ~/.config/merovingian/vm.ini. + + It lives outside the repository so every worktree reads the same file and a + host address is never committed.""" + override = env.get("MEROVINGIAN_VM_CONFIG", "") + if override: + return Path(override) + return Path.home() / ".config" / "merovingian" / "vm.ini" + + +def load_vm_settings(args: argparse.Namespace, env: Mapping[str, str], + config_path: Path) -> VmSettings: + """Resolve each vm setting from the command line, then MEROVINGIAN_VM_, + then config_path: its [vm] section, or [vm.] for the VM that --vm or + MEROVINGIAN_VM_NAME names. Raises VmError on a missing host or VM section, + or a value ssh could read as an option.""" + vm_name = (getattr(args, "vm", None) or env.get("MEROVINGIAN_VM_NAME", "")).strip() + if vm_name and not _VM_NAME_PATTERN.match(vm_name): + raise VmError(f"VM name {vm_name!r} must be lower-case letters, digits, '-' or '_'") + section = f"vm.{vm_name}" if vm_name else "vm" + file_values: dict[str, str] = {} + if config_path.is_file(): + parser = configparser.ConfigParser(interpolation=None) + try: + parser.read(config_path, encoding="utf-8") + except configparser.Error as error: + raise VmError(f"cannot read {config_path}: {error}") from error + if parser.has_section(section): + for name, value in parser.items(section): + if name not in VM_SETTING_NAMES: + raise VmError(f"{config_path}: unknown [{section}] setting {name!r}") + file_values[name] = value.strip() + elif vm_name: + known = ", ".join(s[3:] for s in parser.sections() if s.startswith("vm.")) or "none" + raise VmError(f"{config_path} has no [{section}] section (named VMs: {known})") + elif vm_name: + raise VmError(f"--vm {vm_name} needs a [{section}] section in {config_path}") + + def pick(name: str) -> str: + from_cli = getattr(args, name, None) + if from_cli not in (None, ""): + return str(from_cli) + from_env = env.get(f"MEROVINGIAN_VM_{name.upper()}", "").strip() + if from_env: + return from_env + return file_values.get(name, "") + + host = pick("host") + if not host: + raise VmError("no VM host: pass --host, set MEROVINGIAN_VM_HOST, " + f"or set host in the [{section}] section of {config_path}") + if not _VM_HOST_PATTERN.match(host): + raise VmError(f"VM host {host!r} is not a host name or IP address") + + user = pick("user") + if user and not _VM_USER_PATTERN.match(user): + raise VmError(f"VM user {user!r} is not a user name") + + port_text = pick("port") or "22" + if not port_text.isdigit() or not 1 <= int(port_text) <= 65535: + raise VmError(f"VM port {port_text!r} is not a port number") + + identity = pick("identity") + if identity: + identity = str(Path(identity).expanduser()) + if not Path(identity).is_file(): + raise VmError(f"VM identity file {identity} does not exist") + + jobs_text = pick("jobs") + jobs = None + if jobs_text: + if not jobs_text.isdigit() or int(jobs_text) < 1: + raise VmError(f"VM jobs {jobs_text!r} is not a positive whole number") + jobs = int(jobs_text) + + return VmSettings(host=host, user=user, port=int(port_text), identity=identity, jobs=jobs, + name=vm_name) + + +def vm_default_builddir(settings: VmSettings) -> str: + """build-vm for the default VM, build-vm- for a named one, so two VMs' + logs never overwrite each other locally.""" + return f"{VM_DEFAULT_BUILDDIR}-{settings.name}" if settings.name else VM_DEFAULT_BUILDDIR + + +def validate_vm_builddir(builddir: str) -> str: + """Return builddir if it is a relative path whose first part starts with + "build", the prefix the sync never sends; raise VmError otherwise.""" + parts = builddir.split("/") + if not _VM_BUILDDIR_PATTERN.match(builddir) or any(part in (".", "..") for part in parts): + raise VmError(f"--builddir {builddir!r} must be a relative path whose first " + "part starts with 'build', with no '.' or '..' parts") + return builddir + + +def vm_checkout_name(repo_root: Path) -> str: + """Name the VM-side checkout after the local one: its directory name plus a + hash of its full path, so two worktrees never share a remote checkout.""" + resolved = str(repo_root.resolve()) + key = resolved.lower() if os.name == "nt" else resolved + digest = hashlib.sha256(key.encode("utf-8")).hexdigest()[:12] + stem = re.sub(r"[^A-Za-z0-9._-]", "-", repo_root.name).strip(".-") or "checkout" + return f"{stem[:40]}-{digest}" + + +def vm_ssh_command(settings: VmSettings) -> list[str]: + """The ssh invocation for every call to the VM. It never asks for a + password and refuses a host key that is unknown or has changed.""" + command = ["ssh", "-o", "BatchMode=yes", "-o", "StrictHostKeyChecking=yes", + "-o", "ServerAliveInterval=30", "-o", "ServerAliveCountMax=4", + "-p", str(settings.port)] + if settings.identity: + command += ["-i", settings.identity, "-o", "IdentitiesOnly=yes"] + destination = f"{settings.user}@{settings.host}" if settings.user else settings.host + return command + ["--", destination] + + +def _check_vm_path(path: str) -> None: + """Refuse a path the manifest format or the VM checkout cannot hold safely.""" + if any(character in path for character in "\t\n\r"): + raise VmError(f"cannot sync {path!r}: tab or newline in the path") + if path.startswith("/") or any(part in ("", ".", "..") for part in path.split("/")): + raise VmError(f"cannot sync {path!r}: not a plain relative path") + + +def _git_paths(repo_root: Path, *git_args: str) -> list[str]: + """Run git ls-files with -z in repo_root and split its output.""" + result = subprocess.run(["git", "-C", str(repo_root), "ls-files", "-z", *git_args], + capture_output=True, check=True) + return [entry for entry in result.stdout.decode("utf-8").split("\0") if entry] + + +def vm_local_files(repo_root: Path) -> dict[str, tuple[int, int, int]]: + """Map each file to send to the VM to (mode, size, mtime). + + Sends every tracked file as it is on disk, with git's executable bit, and + every untracked file git does not ignore, except under a top-level name + starting with "build" or "." (build output, worktrees, tool caches). A + tracked file deleted on disk is left out. Raises VmError on a symbolic link + or submodule, which the sync does not carry.""" + files: dict[str, tuple[int, int, int]] = {} + for entry in _git_paths(repo_root, "--stage"): + meta, path = entry.split("\t", 1) + git_mode = meta.split(" ", 1)[0] + if git_mode not in ("100644", "100755"): + raise VmError(f"cannot sync {path!r}: git mode {git_mode} is not a regular file") + _check_vm_path(path) + try: + # lstat, not stat: a tracked file replaced on disk by a symbolic link + # must be refused, not followed to a target outside the checkout. + status = (repo_root / path).lstat() + except FileNotFoundError: + continue + if not stat.S_ISREG(status.st_mode): + raise VmError(f"cannot sync {path!r}: not a regular file on disk") + mode = 0o755 if git_mode == "100755" else 0o644 + files[path] = (mode, status.st_size, int(status.st_mtime)) + for path in _git_paths(repo_root, "--others", "--exclude-standard"): + top = path.split("/", 1)[0] + if path in files or top.startswith("build") or top.startswith("."): + continue + _check_vm_path(path) + status = (repo_root / path).lstat() + if not stat.S_ISREG(status.st_mode): + raise VmError(f"cannot sync {path!r}: not a regular file on disk") + executable = os.name != "nt" and bool(status.st_mode & stat.S_IXUSR) + files[path] = (0o755 if executable else 0o644, status.st_size, int(status.st_mtime)) + return files + + +def format_vm_manifest(files: Mapping[str, tuple[int, int, int]]) -> str: + """One "modesizemtimepath" line per file, sorted by path.""" + return "".join(f"{mode:o}\t{size}\t{mtime}\t{path}\n" + for path, (mode, size, mtime) in sorted(files.items())) + + +def parse_vm_manifest(text: str) -> dict[str, tuple[int, int, int]]: + """Read a manifest written by format_vm_manifest. A line that does not parse + is dropped, so its file counts as missing and is sent again.""" + files: dict[str, tuple[int, int, int]] = {} + for line in text.splitlines(): + fields = line.split("\t", 3) + if len(fields) != 4: + continue + mode, size, mtime, path = fields + try: + files[path] = (int(mode, 8), int(size), int(mtime)) + except ValueError: + continue + return files + + +def plan_vm_sync(local: Mapping[str, tuple[int, int, int]], + remote: Mapping[str, tuple[int, int, int]]) -> list[str]: + """The paths whose mode, size or mtime differ from what the VM last received.""" + return [path for path in sorted(local) if remote.get(path) != local[path]] + + +def plan_vm_stale(local: Mapping[str, tuple[int, int, int]], + remote: Mapping[str, tuple[int, int, int]]) -> list[str]: + """The paths the VM last received that this checkout no longer has.""" + return sorted(path for path in remote if path not in local) + + +def vm_manifest_digest(manifest: bytes) -> str: + """The SHA-256 the remote script compares the VM's manifest against, under its lock.""" + return hashlib.sha256(manifest).hexdigest() + + +def write_vm_sync_archive(stream: BinaryIO, repo_root: Path, + local: Mapping[str, tuple[int, int, int]], + send: list[str]) -> None: + """Stream a tar of the files in send, then the full manifest as + VM_MANIFEST_NEXT, which the VM installs only after every file arrived.""" + with tarfile.open(fileobj=stream, mode="w|", format=tarfile.PAX_FORMAT) as archive: + for path in send: + mode, size, mtime = local[path] + info = tarfile.TarInfo(path) + info.size = size + info.mtime = mtime + info.mode = mode + with open(repo_root / path, "rb") as source: + archive.addfile(info, source) + manifest = format_vm_manifest(local).encode("utf-8") + info = tarfile.TarInfo(VM_MANIFEST_NEXT) + info.size = len(manifest) + info.mode = 0o644 + archive.addfile(info, io.BytesIO(manifest)) + + +def vm_incoming_list(send: Sequence[str]) -> bytes: + """The list of the paths the archive will bring, as the VM reads it before the archive: + its length in bytes and a newline, then one path per line.""" + listing = "".join(f"{path}\n" for path in send).encode("utf-8") + return f"{len(listing)}\n".encode("ascii") + listing + + +def write_vm_sync_stream(stream: BinaryIO, repo_root: Path, + local: Mapping[str, tuple[int, int, int]], + send: list[str]) -> None: + """What build.py vm sends on the remote script's stdin: vm_incoming_list(send), then the archive.""" + stream.write(vm_incoming_list(send)) + write_vm_sync_archive(stream, repo_root, local, send) + + +# Run on the VM with python3 -c: copies the incoming list (vm_incoming_list) from stdin to stdout. os.read never +# returns more than it is asked for, so the archive that follows stays on stdin for tar. +_VM_READ_INCOMING = """import os, sys +def take(count): + data = b"" + while len(data) < count: + chunk = os.read(0, count - len(data)) + if not chunk: + sys.exit("build.py vm: the stream ended inside the list of incoming files") + data += chunk + return data +header = b"" +while not header.endswith(b"\\n"): + if len(header) > 20: + sys.exit("build.py vm: the stream does not start with a list of incoming files") + header += take(1) +try: + size = int(header) +except ValueError: + sys.exit("build.py vm: the stream does not start with a list of incoming files") +sys.stdout.buffer.write(take(size)) +""" + + +def vm_run_dir(token: str) -> str: + """The directory, relative to the VM checkout, that holds one run's own files: + its log snapshot and, when its baseline was stale, VM_BASELINE_MARKER.""" + if not _VM_TOKEN_PATTERN.match(token): + raise VmError(f"bad run token {token!r}") + return f".vm-sync/run-{token}" + + +def vm_remote_script(checkout: str, run_line: str, *, token: str, baseline_sha256: str, + stale: Sequence[str] = (), builddir: str = "", clean: bool = False, + size_jobs: bool = False) -> str: + """The POSIX sh script the remote host runs for one build.py vm call. + + It sets $os from uname and $build_script to build-linux.sh on Linux or + build-bsd.sh on FreeBSD, OpenBSD, NetBSD or DragonFly, refusing any other + system. It takes the ssh user's lock, so a second run as that user (from any + checkout) waits; everything after that happens under the lock: + + 1. The VM's manifest must still hash to baseline_sha256, the manifest + build.py planned the sync against. If another run changed it meanwhile, + the script leaves VM_BASELINE_MARKER in the run directory and exits + VM_BASELINE_CHANGED without touching the checkout, and build.py retries. + 2. With builddir it removes that build's old test logs (and the whole + directory with clean), before anything can fail. + 3. It deletes the stale paths, files this checkout no longer has, and any + directory that leaves empty, before unpacking, so a file that became a + directory (or the reverse) never blocks the archive. + 4. It reads the list of incoming paths that precedes the archive on stdin + (vm_incoming_list) and adds them, zeroed, to the manifest, so an unpack + cut off part-way leaves nothing the next run does not know to delete. + Then it unpacks the archive into ~/merovingian-vm/ and installs + the new manifest, which arrives last, only after that succeeds. + 5. With size_jobs it sets $jobs from the host's memory, runs run_line, and + with builddir copies the test logs into the run directory, so build.py + fetches this run's logs even after a later run has replaced the build's. + + The lock is flock(2) on fd 9, taken through Python's fcntl because FreeBSD, + OpenBSD and NetBSD ship no flock(1). A flock lock belongs to the open file, + so this shell's fd 9 keeps holding it after Python exits.""" + run_dir = vm_run_dir(token) + if not re.fullmatch(r"[0-9a-f]{64}", baseline_sha256): + raise VmError(f"bad manifest digest {baseline_sha256!r}") + stale_end = f"VM_STALE_{token}" + if stale_end in stale: + raise VmError(f"cannot sync a path named {stale_end!r}") + lines = [ + "set -eu", + "os=$(uname -s)", + 'case "$os" in', + " Linux) build_script=build-linux.sh ;;", + " FreeBSD|OpenBSD|NetBSD|DragonFly) build_script=build-bsd.sh ;;", + ' *) echo "build.py vm: unsupported remote system $os" >&2; exit 2 ;;', + "esac", + "py=", + "for candidate in python3 python3.14 python3.13 python3.12 python3.11 python3.10; do", + ' if command -v "$candidate" >/dev/null 2>&1; then py=$candidate; break; fi', + "done", + 'if [ -z "$py" ]; then', + " echo 'build.py vm: no python3 on the remote host; run scripts/remote-host-setup.sh' >&2", + " exit 2", + "fi", + f'root="$HOME"/{quote_bash(VM_REMOTE_ROOT)}', + f'checkout="$root"/{quote_bash(checkout)}', + 'mkdir -p "$checkout/.vm-sync"', + 'exec 9>>"$root/.lock"', + """if ! "$py" -c 'import fcntl; fcntl.flock(9, fcntl.LOCK_EX | fcntl.LOCK_NB)' 2>/dev/null; then""", + " echo 'build.py vm: another run holds the VM lock; waiting for it' >&2", + """ "$py" -c 'import fcntl; fcntl.flock(9, fcntl.LOCK_EX)'""", + "fi", + 'cd "$checkout"', + f"run_dir={run_dir}", + 'mkdir -p "$run_dir"', + f"baseline=$(cat {VM_MANIFEST} 2>/dev/null | " + """"$py" -c 'import hashlib, sys; print(hashlib.sha256(sys.stdin.buffer.read()).hexdigest())')""", + f'if [ "$baseline" != {baseline_sha256} ]; then', + " echo 'build.py vm: another run changed the VM checkout since this one read it' >&2", + f' : > "$run_dir/{VM_BASELINE_MARKER}"', + f" exit {VM_BASELINE_CHANGED}", + "fi", + # Before the checkout changes, zero the metadata of every entry in the + # manifest. If the sync then fails part-way, the next run plans against + # this, so it resends every file (a file deleted here and later restored + # with its old metadata included), yet still knows which paths are stale. + f"if [ -f {VM_MANIFEST} ]; then", + """ awk 'BEGIN { FS = "\\t"; OFS = "\\t" } NF >= 4 { print 0, 0, 0, $4 }' """ + f"{VM_MANIFEST} > .vm-sync/manifest.dirty", + f" mv -f .vm-sync/manifest.dirty {VM_MANIFEST}", + "fi", + ] + if builddir: + quoted = quote_bash(builddir) + if clean: + lines.append(f"rm -rf -- {quoted}") + lines += [f"rm -f -- {quoted}/meson-logs/{name}" for name in VM_TESTLOGS] + if stale: + lines += [ + "while IFS= read -r stale; do", + ' case "$stale" in', + " ''|/*|..|../*|*/..|*/../*) continue ;;", + " esac", + # A stale file that is now a directory was made by an earlier unpack, + # interrupted after it had begun filling the file's replacement: + # the archive refills it, so leave it. + ' if [ -d "$stale" ] && [ ! -L "$stale" ]; then continue; fi', + ' rm -f -- "$stale"', + ' parent=$(dirname -- "$stale")', + ' if [ "$parent" != . ]; then rmdir -p -- "$parent" 2>/dev/null || true; fi', + f"done <<'{stale_end}'", + *stale, + stale_end, + ] + lines += [ + # Before unpacking, add every path the archive will bring to the zeroed + # manifest. An unpack cut off part-way then leaves no file the manifest + # does not name, so the next run deletes those the checkout no longer has, + # whatever changed locally in between. + f"[ -f {VM_MANIFEST} ] || : > {VM_MANIFEST}", + f""""$py" -c '{_VM_READ_INCOMING}' > .vm-sync/incoming""", + """awk 'BEGIN { FS = "\\t"; OFS = "\\t" } """ + """FILENAME == ARGV[1] { if (NF >= 4) { seen[$4] = 1; print } next } """ + """!($0 in seen) { seen[$0] = 1; print 0, 0, 0, $0 }' """ + f"{VM_MANIFEST} .vm-sync/incoming > .vm-sync/manifest.dirty", + f"mv -f .vm-sync/manifest.dirty {VM_MANIFEST}", + "rm -f .vm-sync/incoming", + f"rm -f {VM_MANIFEST_NEXT}", + "tar -xf -", + f"if [ ! -f {VM_MANIFEST_NEXT} ]; then", + " echo 'build.py vm: the archive carried no manifest' >&2", + " exit 2", + "fi", + f"mv -f {VM_MANIFEST_NEXT} {VM_MANIFEST}", + ] + if size_jobs: + lines += [ + 'if [ "$os" = Linux ]; then', + " mem_kib=$(awk '/^MemTotal:/ { print $2 }' /proc/meminfo)", + " cpus=$(nproc)", + "else", + " mem_kib=$(( $(sysctl -n hw.physmem) / 1024 ))", + " cpus=$(sysctl -n hw.ncpu)", + "fi", + f"jobs=$((mem_kib / {VM_MEMORY_PER_JOB_KIB}))", + '[ "$jobs" -ge 1 ] || jobs=1', + '[ "$jobs" -le "$cpus" ] || jobs=$cpus', + 'echo "build.py vm: $jobs parallel jobs ($cpus CPUs, $((mem_kib / 1024)) MiB memory)" >&2', + ] + lines += ["set +e", f"( {run_line} )", "status=$?", "set -e"] + if builddir: + lines += [f'if [ -f {quoted}/meson-logs/{name} ]; then ' + f'cp -- {quoted}/meson-logs/{name} "$run_dir/{name}"; fi' + for name in VM_TESTLOGS] + lines.append('exit "$status"') + return "\n".join(lines) + "\n" + + +def _vm_build_line(args: argparse.Namespace, builddir: str, jobs: int | None) -> str: + """The call of the remote host's build script ($build_script, set by + vm_remote_script); uses $jobs from that script when jobs is None.""" + build_args = ["--builddir", builddir, "--cc", args.cc, "--cxx", args.cxx, + "--wrap-mode", args.wrap_mode] + if args.profile: + build_args += ["--profile", args.profile] + if args.buildtype: + build_args += ["--buildtype", args.buildtype] + if args.sanitize: + build_args += ["--sanitize", args.sanitize] + for flag, enabled in (("--coverage", args.coverage), ("--build-fuzz", args.build_fuzz), + ("--hardening", args.hardening), ("--no-tests", args.no_tests), + ("--setup-only", args.setup_only), + ("--compile-only", args.compile_only)): + if enabled: + build_args.append(flag) + quoted = " ".join(quote_bash(arg) for arg in build_args) + jobs_arg = quote_bash(str(jobs)) if jobs is not None else '"$jobs"' + return (f'exec env PATH="$HOME/.local/bin:$PATH" sh "./scripts/$build_script" ' + f"{quoted} --jobs {jobs_arg}") + + +_VM_FETCH_MISSING = 3 + + +def vm_fetch(ssh: list[str], remote_path: str) -> bytes | None: + """Read a file on the VM, relative to the ssh user's home. + + Returns None only when the file does not exist there. Raises VmError when + the file could not be read for any other reason, such as ssh refusing the + key, so a log that was written is never reported as missing.""" + quoted = quote_bash(remote_path) + script = f"cd && if [ -f {quoted} ]; then exec cat -- {quoted}; fi; exit {_VM_FETCH_MISSING}" + result = subprocess.run(ssh + ["sh -c " + quote_bash(script)], capture_output=True) + if result.returncode == _VM_FETCH_MISSING: + return None + if result.returncode != 0: + detail = result.stderr.decode("utf-8", "replace").strip() + raise VmError(f"cannot copy {remote_path} back from the VM " + f"(exit {result.returncode}): {detail}") + return result.stdout + + +def build_vm(args: argparse.Namespace) -> int: + """Sync this checkout to a Linux or BSD VM over ssh and build, test or run a + command there; see "Building on a VM" in docs/dev-environment.md.""" + try: + settings = load_vm_settings(args, os.environ, vm_config_path(os.environ)) + builddir = validate_vm_builddir(args.builddir or vm_default_builddir(settings)) + if args.exec_command is not None and not args.exec_command: + raise VmError("--exec needs a command to run") + local = vm_local_files(REPO_ROOT) + except (VmError, subprocess.CalledProcessError) as error: + print(f"error: {error}", file=sys.stderr) + return 2 + + ssh = vm_ssh_command(settings) + checkout = vm_checkout_name(REPO_ROOT) + remote_dir = f"{VM_REMOTE_ROOT}/{checkout}" + runs_tests = args.exec_command is None and not (args.no_tests or args.setup_only + or args.compile_only) + if args.exec_command is not None: + run_line = ("exec env MEROVINGIAN_TEST_DISABLE_HARDENING=1 " + + " ".join(quote_bash(arg) for arg in args.exec_command)) + script_options = {} + else: + run_line = _vm_build_line(args, builddir, settings.jobs) + script_options = {"builddir": builddir, "clean": args.clean, + "size_jobs": settings.jobs is None} + + local_logs = REPO_ROOT / builddir / "meson-logs" + if runs_tests and not args.dry_run: + for name in VM_TESTLOGS: + (local_logs / name).unlink(missing_ok=True) + + for _attempt in range(VM_SYNC_ATTEMPTS): + probe = subprocess.run( + ssh + [f"cat -- {quote_bash(remote_dir + '/' + VM_MANIFEST)} 2>/dev/null || true"], + capture_output=True) + if probe.returncode != 0: + sys.stderr.write(probe.stderr.decode("utf-8", "replace")) + print(f"error: cannot reach the VM at {ssh[-1]} (ssh exit {probe.returncode})", + file=sys.stderr) + return 2 + remote = parse_vm_manifest(probe.stdout.decode("utf-8", "replace")) + send = plan_vm_sync(local, remote) + token = secrets.token_hex(8) + run_dir = f"{remote_dir}/{vm_run_dir(token)}" + try: + script = vm_remote_script(checkout, run_line, token=token, + baseline_sha256=vm_manifest_digest(probe.stdout), + stale=plan_vm_stale(local, remote), **script_options) + except VmError as error: + print(f"error: {error}", file=sys.stderr) + return 2 + + print(f"build.py vm: sending {len(send)} of {len(local)} files to " + f"{ssh[-1]}:~/{remote_dir}", file=sys.stderr) + if args.dry_run: + print(script) + return 0 + + process = subprocess.Popen(ssh + ["sh -c " + quote_bash(script)], stdin=subprocess.PIPE) + try: + write_vm_sync_stream(process.stdin, REPO_ROOT, local, send) + except BrokenPipeError: + pass + finally: + try: + process.stdin.close() + except BrokenPipeError: + pass + status = process.wait() + try: + retry = (status == VM_BASELINE_CHANGED + and vm_fetch(ssh, f"{run_dir}/{VM_BASELINE_MARKER}") is not None) + logs = {} + if not retry and runs_tests: + logs = {name: vm_fetch(ssh, f"{run_dir}/{name}") for name in VM_TESTLOGS} + except VmError as error: + # The run may well have written its log; say so rather than "no testlog", + # and never exit 0 without a log the result can be read from. + print(f"error: {error}", file=sys.stderr) + print(f"build.py vm: the build exited {status}; its logs are still on the VM " + f"under ~/{run_dir}/", file=sys.stderr) + return status or 2 + subprocess.run(ssh + [f"rm -rf -- {quote_bash(run_dir)}"], capture_output=True) + if not retry: + break + print("build.py vm: reading the VM's manifest again and resending", file=sys.stderr) + else: + print(f"error: the VM checkout kept changing under {VM_SYNC_ATTEMPTS} attempts", + file=sys.stderr) + return 2 + + if runs_tests: + local_logs.mkdir(parents=True, exist_ok=True) + ignore_file = REPO_ROOT / builddir / ".gitignore" + if not ignore_file.exists(): + ignore_file.write_text("*\n", encoding="utf-8") + for name, data in logs.items(): + if data is not None: + (local_logs / name).write_bytes(data) + if logs.get("testlog.txt") is not None: + summary = [line for line in (local_logs / "testlog.txt").read_text( + encoding="utf-8", errors="replace").splitlines() + if _TESTLOG_SUMMARY_PATTERN.match(line)] + print(f"build.py vm: {local_logs / 'testlog.txt'}", file=sys.stderr) + for line in summary: + print(f"build.py vm: {line}", file=sys.stderr) + else: + print("build.py vm: the run wrote no testlog.txt", file=sys.stderr) + return status + + def build_deb(args: argparse.Namespace) -> int: return run_script(SCRIPTS_DIR / "build-deb.sh", []) @@ -210,6 +852,9 @@ def main() -> int: python build.py linux --profile hardened --builddir build-hardened python build.py bsd python build.py wsl --distro Ubuntu + python build.py vm + python build.py vm --vm freebsd + python build.py vm --exec ./build-vm/tests/merovingian-unit-tests "[push]" python build.py deb python build.py rpm python build.py pkg @@ -223,6 +868,8 @@ def main() -> int: linux_parser = subparsers.add_parser("linux", help="Linux development build") add_common_dev_args(linux_parser, "build") add_profile_args(linux_parser) + linux_parser.add_argument("--jobs", type=positive_int, + help="Most compile jobs and tests at once (default: Meson's)") # bsd bsd_parser = subparsers.add_parser("bsd", help="BSD development build") @@ -230,6 +877,8 @@ def main() -> int: add_profile_args(bsd_parser) bsd_parser.add_argument("--pkg-config", default="", help="pkg-config command (default: pkgconf, then pkg-config)") + bsd_parser.add_argument("--jobs", type=positive_int, + help="Most compile jobs and tests at once (default: Meson's)") # wsl wsl_parser = subparsers.add_parser("wsl", @@ -241,6 +890,28 @@ def main() -> int: wsl_parser.add_argument("--clean", action="store_true", help="Wipe the build directory before configuring") + # vm + vm_parser = subparsers.add_parser( + "vm", help="Build and test on a Linux or BSD VM over ssh (docs/dev-environment.md)") + add_common_dev_args(vm_parser, None, + f"{VM_DEFAULT_BUILDDIR}, or {VM_DEFAULT_BUILDDIR}- with --vm") + vm_parser.add_argument("--vm", help="Named VM: the [vm.] section of the settings file " + "(default: MEROVINGIAN_VM_NAME, then [vm])") + add_profile_args(vm_parser) + vm_parser.add_argument("--host", help="VM host name or address " + "(default: MEROVINGIAN_VM_HOST, then the settings file)") + vm_parser.add_argument("--user", help="ssh user on the VM") + vm_parser.add_argument("--port", help="ssh port on the VM (default: 22)") + vm_parser.add_argument("--identity", help="ssh private key file") + vm_parser.add_argument("--jobs", type=positive_int, + help="Most compile jobs and tests at once " + "(default: one per 1.5 GiB of VM memory)") + vm_parser.add_argument("--clean", action="store_true", + help="Wipe the VM's build directory before configuring") + vm_parser.add_argument("--exec", dest="exec_command", nargs=argparse.REMAINDER, + help="Sync, then run this command in the VM checkout instead " + "of building; must come last") + # deb subparsers.add_parser("deb", help="Build Debian .deb package") @@ -262,6 +933,7 @@ def main() -> int: "linux": build_linux, "bsd": build_bsd, "wsl": build_wsl, + "vm": build_vm, "deb": build_deb, "rpm": build_rpm, "pkg": build_pkg, diff --git a/config/merovingian.conf.bsd.example b/config/merovingian.conf.bsd.example index 8b0e9e5ad..03150b5a5 100644 --- a/config/merovingian.conf.bsd.example +++ b/config/merovingian.conf.bsd.example @@ -90,6 +90,11 @@ server.cors.max_age=86400 # once (addresses in server.trusted_proxies are exempt), and remote directory # and media proxying at most half. Restart required. server.http.request_threads=32 +# client_outbound_max_per_client (1..128) is how many of those remote calls +# (directory, profile and key lookups, federated joins, knocks and leaves) one +# client address may have in flight at once; more are answered 429. It never +# exceeds request_threads / 2. Restart required. +server.http.client_outbound_max_per_client=3 server.http.keep_alive=true server.http.keep_alive_idle_seconds=15 server.http.keep_alive_max_connections=256 diff --git a/config/merovingian.conf.example b/config/merovingian.conf.example index ca19cf2e6..e7849d50b 100644 --- a/config/merovingian.conf.example +++ b/config/merovingian.conf.example @@ -63,6 +63,11 @@ server.cors.max_age=86400 # once (addresses in server.trusted_proxies are exempt), and remote directory # and media proxying at most half. Restart required. server.http.request_threads=32 +# client_outbound_max_per_client (1..128) is how many of those remote calls +# (directory, profile and key lookups, federated joins, knocks and leaves) one +# client address may have in flight at once; more are answered 429. It never +# exceeds request_threads / 2. Restart required. +server.http.client_outbound_max_per_client=3 server.http.keep_alive=true server.http.keep_alive_idle_seconds=15 server.http.keep_alive_max_connections=256 diff --git a/docs/AGENTS.md b/docs/AGENTS.md index 8c00ac75d..dc1bb6946 100644 --- a/docs/AGENTS.md +++ b/docs/AGENTS.md @@ -36,6 +36,7 @@ new documents. | `todos/capability-gaps.md` / `todos/production-milestone.md` | A tracked gap is closed or a new one is found | | `../AGENTS.md` and the module's own `AGENTS.md` | A module is added: give it an `AGENTS.md`, a `CLAUDE.md` containing `@AGENTS.md`, and a row in the root `AGENTS.md` layout and index tables | | `../mkdocs.yml` | A document under `docs/` is added, removed, or renamed: add, remove, or update its `nav` entry (or its `exclude_docs` pattern if it should not be published). `docs-site-tooling` fails on a published document with no nav entry. See [dev-environment.md](dev-environment.md#documentation-site) | +| `../.agent/skills/merovingian-review/` | A rule, heading, ADR, test convention or obligation the review skill cites is added, changed, renamed, or removed. The skill points at the binding text and does not restate it ([ADR-0754](adr/0754-agent-skills-are-committed-under-agent-skills-and-cite-the-binding-docs.md)) | | `adr/index.md` | A design decision is made whose consequences outlive the change that prompted it — a constraint the code depends on but does not state locally, a rejected alternative that looks better in isolation, or a rule about how to write future code. Add a new ADR under `adr/`, and a line in `adr/index.md`. See [ADR-0000](adr/0000-record-architecture-decisions.md). | ## Do NOT create new documents for diff --git a/docs/adr/0079-client-triggered-outbound-proxying-runs-under-a-bounded-in-flight-budget.md b/docs/adr/0079-client-triggered-outbound-proxying-runs-under-a-bounded-in-flight-budget.md index e42a23b92..48e8d17b2 100644 --- a/docs/adr/0079-client-triggered-outbound-proxying-runs-under-a-bounded-in-flight-budget.md +++ b/docs/adr/0079-client-triggered-outbound-proxying-runs-under-a-bounded-in-flight-budget.md @@ -1,6 +1,6 @@ # Client-triggered outbound proxying runs under a bounded in-flight budget, not a separate pool -* Status: accepted; superseded for remote media by [ADR-0121](0121-remote-media-fetches-run-on-their-own-pool.md) (2026-10-07) +* Status: accepted; superseded for remote media by [ADR-0121](0121-remote-media-fetches-run-on-their-own-pool.md) (2026-10-07); per-client cap of one amended by [ADR-0751](0751-the-per-client-outbound-call-cap-is-configurable-and-defaults-to-three.md) (2026-11-14) * Date: 2026-09-30 Technical Story: 2026-09-29 security audit, findings HTTP-2 (unauthenticated diff --git a/docs/adr/0515-appservice-cleartext-http-only-to-loopback-literals-or-listed-hosts.md b/docs/adr/0515-appservice-cleartext-http-only-to-loopback-literals-or-listed-hosts.md index 4a63a9176..33666b0cd 100644 --- a/docs/adr/0515-appservice-cleartext-http-only-to-loopback-literals-or-listed-hosts.md +++ b/docs/adr/0515-appservice-cleartext-http-only-to-loopback-literals-or-listed-hosts.md @@ -1,6 +1,6 @@ # Appservice cleartext http only to loopback literals or listed hosts -* Status: accepted +* Status: accepted; the setting of Complement's image is amended by [ADR-0750](0750-complements-image-also-lists-localhost-as-a-cleartext-appservice-host.md) * Deciders: James Chapman * Date: 2026-11-06 diff --git a/docs/adr/0591-a-refused-send-join-leave-or-knock-is-never-stored-and-needs-local-interest.md b/docs/adr/0591-a-refused-send-join-leave-or-knock-is-never-stored-and-needs-local-interest.md index a263000fe..97ede6ffa 100644 --- a/docs/adr/0591-a-refused-send-join-leave-or-knock-is-never-stored-and-needs-local-interest.md +++ b/docs/adr/0591-a-refused-send-join-leave-or-knock-is-never-stored-and-needs-local-interest.md @@ -1,6 +1,6 @@ # A refused send_join, send_leave or send_knock is never stored, and is asked for only in a room with a local member -* Status: accepted (supersedes the part of [ADR-0372](0372-a-resent-send-join-is-answered-from-current-state.md) that answers a soft-failed join with state) +* Status: accepted (supersedes the part of [ADR-0372](0372-a-resent-send-join-is-answered-from-current-state.md) that answers a soft-failed join with state; the "no exception" sentence of point 1 is superseded by [ADR-0747](0747-a-remote-member-may-leave-through-a-server-whose-local-members-have-left.md)) * Deciders: James Chapman * Date: 2026-11-09 diff --git a/docs/adr/0622-a-clients-federated-join-knock-or-leave-holds-a-client-outbound-slot.md b/docs/adr/0622-a-clients-federated-join-knock-or-leave-holds-a-client-outbound-slot.md index 56b33bb0c..32acc508c 100644 --- a/docs/adr/0622-a-clients-federated-join-knock-or-leave-holds-a-client-outbound-slot.md +++ b/docs/adr/0622-a-clients-federated-join-knock-or-leave-holds-a-client-outbound-slot.md @@ -1,6 +1,6 @@ # A client's federated join, knock or leave holds a client-outbound slot -* Status: accepted +* Status: accepted; the per-client cap its slot is counted against is configurable since [ADR-0751](0751-the-per-client-outbound-call-cap-is-configurable-and-defaults-to-three.md) * Date: 2026-11-11 Technical Story: release-gate audit, resource-exhaustion.client-membership-exchange.federated-join-and-knock-hold-pool-threads-without-outbound-slot diff --git a/docs/adr/0737-receipts-are-read-through-room-and-user-indexes-and-a-threaded-receipt-must-be-related-to-its-thread.md b/docs/adr/0737-receipts-are-read-through-room-and-user-indexes-and-a-threaded-receipt-must-be-related-to-its-thread.md index 9cd50ce81..40414589f 100644 --- a/docs/adr/0737-receipts-are-read-through-room-and-user-indexes-and-a-threaded-receipt-must-be-related-to-its-thread.md +++ b/docs/adr/0737-receipts-are-read-through-room-and-user-indexes-and-a-threaded-receipt-must-be-related-to-its-thread.md @@ -1,6 +1,6 @@ # Receipts are read through room and user indexes, and a threaded receipt must be related to its thread -* Status: Accepted +* Status: Accepted (its admission rule for a threaded receipt is amended by [ADR-0746](0746-a-threaded-receipt-may-name-any-event-whose-relation-chain-reaches-the-root.md)) * Date: 2026-11-12 Technical Story: run-3 audit finding C23 (receipt rows grow without bound and every reader scans the whole table). diff --git a/docs/adr/0745-a-first-restricted-join-stores-the-residents-verified-co-signature.md b/docs/adr/0745-a-first-restricted-join-stores-the-residents-verified-co-signature.md new file mode 100644 index 000000000..18c6bcd58 --- /dev/null +++ b/docs/adr/0745-a-first-restricted-join-stores-the-residents-verified-co-signature.md @@ -0,0 +1,86 @@ +# A first join into a restricted room stores the resident's verified co-signature + +* Status: accepted +* Deciders: James Chapman +* Date: 2026-11-14 + +Technical Story: Complement regressions after the run-2 audit fixes (1.0.2): `TestRestrictedRoomsRemoteJoin`, +`TestRestrictedRoomsRemoteJoinLocalUser`, `TestRestrictedRoomsRemoteJoinFailOver` and their MSC3787 and knock +variants. + +## Context and Problem Statement + +A user of this server joins a restricted room this server holds nothing of. The make_join template names +`join_authorised_via_users_server`, a user of the resident's server. Auth rule 4.3.5 of room versions 8 and later +(rooms/v10.md) allows that join only when it is signed by the authorising user's server, and +`Authorization.cpp` enforces it (EVT-1). The resident adds that signature to the copy it returns as the answer's +`event`. The send_join v2 response field is "Required if the room is restricted and the joining user is authorised by +one of the conditions". + +Since F-45 (ADR-0369), `authorize_send_join_answer` checks the answer's state and auth chain against the join before +anything is stored. It was given the join this server signed itself, which has no signature from the authorising +server, so every first restricted join was refused with "the events in the answer do not authorise the join". The +resident's `event` was ignored on this path. Only a join through another resident (ADR-0287) used it. + +How does this server get a join that the auth rules accept, without trusting the resident's copy? + +## Decision Drivers + +* A join that no checked signature authorises must never be stored (EVT-1, ADR-0171). +* The resident chooses the content of its copy. The event ID and its signatures cover the redacted form, so a resident + could change a display name and keep both (ADR-0302). +* The same answer check must cover the first join and a join through another resident. + +## Considered Options + +* Take the resident's co-signature as a join through another resident does: merge only the authorising server's + verified signatures into this server's own signed join, verify the combined event, and authorise and store that. +* Exempt this server's own join from rule 4.3.5 when it checks the answer. +* Store the resident's `event` as returned. + +## Decision Outcome + +Chosen option: the first. `perform_federated_join` calls `verify_resident_signed_join` whenever its join names an +authorising user. That function requires the answer's `event` to have the reference-hash ID of the join this server +signed and to name an authorising user of another server. It merges in that server's signatures only +(`with_server_signatures`), and `filter_verified_send_join_events` must verify the combined event: this server's +signature and the authorising server's. The verified combined event is what `authorize_send_join_answer` judges and +what `join_room` stores and federates. An answer with no `event`, an `event` without the authorising server's +signature, or one signed by another key is refused with 502, and nothing is stored. + +### Positive Consequences + +* A first restricted join works again, and the stored join carries the signature every other server's auth check + requires. +* There is one way to accept a co-signed join, used by both join paths: `perform_federated_join` verifies, logs and + refuses the resident's copy in one place (`verify_resident_join`), so the two paths cannot drift apart. +* The stored join's `event_signatures` rows list every signature its JSON carries, the co-signature included, so + the rows and the event agree after a reload or in a worker's snapshot. +* The resident's template brings no `hashes` or `signatures` into the join: `claim_membership_event_template` drops + them, because signing adds to the signatures an event holds. The signed join is parsed before it is sent and + refused with 502 if it does not parse, so a join this server sends can always be stored with its signature rows. + +### Negative Consequences + +* A resident that does not return `event` for a restricted join cannot be joined through. The spec requires the field + in this case, so such a resident is not conformant. + +## Pros and Cons of the Options + +### Exempt this server's own join from rule 4.3.5 + +* Good, because it needs no extra verification step. +* Bad, because it stores a join that no other server will accept. The join is then rejected everywhere else, and this + server believes its user is in a room it is not in. +* Bad, because it reopens a version of the EVT-1 hole on the one path the check was meant to cover. + +### Store the resident's `event` as returned + +* Good, because it is the smallest change. +* Bad, because the resident could rewrite unsigned parts of the content under valid signatures (ADR-0302). + +## Links + +* Refines [ADR-0369](0369-a-send-join-answer-is-judged-by-the-receipt-checks-from-its-own-events.md) +* Uses [ADR-0302](0302-a-remote-signature-is-merged-into-this-servers-own-event-after-it-is-verified.md) and + [ADR-0287](0287-a-resident-with-no-local-authoriser-joins-its-user-through-another-resident.md) diff --git a/docs/adr/0746-a-threaded-receipt-may-name-any-event-whose-relation-chain-reaches-the-root.md b/docs/adr/0746-a-threaded-receipt-may-name-any-event-whose-relation-chain-reaches-the-root.md new file mode 100644 index 000000000..7ef704457 --- /dev/null +++ b/docs/adr/0746-a-threaded-receipt-may-name-any-event-whose-relation-chain-reaches-the-root.md @@ -0,0 +1,62 @@ +# A threaded receipt may name any event whose relation chain reaches the thread root + +* Status: accepted (amends the admission rule of [ADR-0737](0737-receipts-are-read-through-room-and-user-indexes-and-a-threaded-receipt-must-be-related-to-its-thread.md)) +* Deciders: James Chapman +* Date: 2026-11-14 + +Technical Story: Complement `TestThreadedReceipts` failed after the run-3 audit fix C23 (1.0.2). + +## Context and Problem Statement + +ADR-0737 accepted a receipt filed under a thread root only for the root itself or an event in that thread, found by +`sync::event_thread_id` (an `m.thread` relation, up to three hops). `TestThreadedReceipts` marks a thread read through +an annotation of a reference to the root. The annotation is in the main timeline by the thread rule, so it was refused +with 400. + +The spec's 400 condition for `POST /receipt` is "The `event_id` is not related to the `thread_id`", and it does not +define "related". What counts as related, without reopening the cost C23 closed? + +## Decision Drivers + +* C23: one user must not be able to file receipts under thousands of `thread_id`s of their choice. +* An event's relation chain is one path. Walking it is bounded by the hop limit the thread rule already uses. +* Clients mark threads read through reactions and references to the root, as the Complement test does. + +## Considered Options + +* Related means the root, an event in its thread, or an event whose `m.relates_to` chain, of any `rel_type`, reaches + the root within three hops (`sync::event_relates_to`). +* Related means in the thread only (ADR-0737 as written). +* Accept any `thread_id` that is a main-timeline event (the rule before C23). + +## Decision Outcome + +Chosen option: the first. `receipt_event_is_in_thread` still requires `thread_id` to name a main-timeline event of the +room. It then accepts the event when `event_thread_id` is that root or `event_relates_to` reaches it. Parents are looked +up in the same room only, and a parent this server does not hold ends the chain. Each event has one relation, so an +event is related to at most three roots besides its own thread, and the C23 cap of 256 threaded rows per user and room +is unchanged. + +### Positive Consequences + +* Reactions to, and references of, a thread root can carry that thread's receipt. + +### Negative Consequences + +* An event that references a root without being in its thread can move that thread's read marker. That is what the + client asked for, and it moves only the sender's own marker. + +## Pros and Cons of the Options + +### In the thread only + +* Good, because it is the narrowest reading. +* Bad, because it refuses receipts that clients send and the spec's word "related" allows. + +### Any main-timeline `thread_id` + +* Bad, because it is the C23 finding: a user chooses freely among every event of the room. + +## Links + +* Amends [ADR-0737](0737-receipts-are-read-through-room-and-user-indexes-and-a-threaded-receipt-must-be-related-to-its-thread.md) diff --git a/docs/adr/0747-a-remote-member-may-leave-through-a-server-whose-local-members-have-left.md b/docs/adr/0747-a-remote-member-may-leave-through-a-server-whose-local-members-have-left.md new file mode 100644 index 000000000..1e9553622 --- /dev/null +++ b/docs/adr/0747-a-remote-member-may-leave-through-a-server-whose-local-members-have-left.md @@ -0,0 +1,70 @@ +# A remote member may leave through a server whose local members have all left + +* Status: accepted (supersedes the "no exception" sentence of point 1 of [ADR-0591](0591-a-refused-send-join-leave-or-knock-is-never-stored-and-needs-local-interest.md)) +* Deciders: James Chapman +* Date: 2026-11-14 + +Technical Story: Complement invite-rejection regression after the run-4 audit fix F-12 (1.0.2). + +## Context and Problem Statement + +ADR-0591 point 1 makes the membership acceptor refuse `send_join`, `send_leave` and `send_knock` for a room in which +no user of this server is joined, invited or knocking. It added no exception for a flow this server started, and it +accepted that a remote invitee could no longer reject an invite once the local inviter had left. + +Complement tests exactly that. A local user invites a remote user, then leaves. The invitee's server asks this server +for `make_leave` / `send_leave`, because this server sent the invite and, for a pre-v12 room, the room ID names it. The +answer was 403, and the invitee was left with an invite they could not reject. The spec ("Leaving Rooms (Rejecting +Invites)") lists no refusal for this case. + +Can a leave be admitted without reopening F-12, where any server could make this server store membership events for +any room it knows? + +## Decision Drivers + +* F-12: a stranger must not be able to make this server store, recompute or publish anything. +* F-10: a refused or ended member must learn no room state. +* A remote user who holds a `join`, `invite` or `knock` row here must be able to end it. + +## Considered Options + +* Admit a `send_leave` whose sender, a user of another server, holds a `join`, `invite` or `knock` row in the room. + All other gates still run. +* Keep ADR-0591 as written and record the test as an expected failure. +* Admit every `send_leave` for a known room. + +## Decision Outcome + +Chosen option: the first. `membership_admission_refusal` lets a `send_leave` through when `remote_member_may_leave` +holds for the envelope's sender. The agreement gate (an `m.room.member` leave about its own sender, from a user of +the origin) and the outcome gate (auth must accept it against current state) still run, in that order, before +anything is stored. The leave turns the row into `leave`, so a second leave is refused: each row admits one stored +event, not one per request. The leave that ended the row is admitted again while it is still the sender's current +membership, so a retry after a lost answer reaches the resend branch (ADR-0591 point 3), which answers it `200` and +stores nothing. `send_leave` carries no state, so F-10 is not affected. `send_join` and `send_knock` keep +the gate unchanged. + +### Positive Consequences + +* A remote invitee can reject an invite after the local inviter has left. A remote member can leave a room whose local + members are gone. + +### Negative Consequences + +* A server whose user holds such a row can make this server store one leave, which this server then sends to the other + servers in the room. The row exists only because this server accepted that user earlier. + +## Pros and Cons of the Options + +### Keep ADR-0591 as written + +* Good, because the gate stays a single rule. +* Bad, because an invite this server sent could never be rejected, and the spec gives no reason to refuse. + +### Admit every `send_leave` for a known room + +* Bad, because it reopens F-12 for any user of any server. + +## Links + +* Supersedes part of [ADR-0591](0591-a-refused-send-join-leave-or-knock-is-never-stored-and-needs-local-interest.md) diff --git a/docs/adr/0748-main-serves-a-rooms-federation-reads-while-the-workers-reload-is-pending.md b/docs/adr/0748-main-serves-a-rooms-federation-reads-while-the-workers-reload-is-pending.md new file mode 100644 index 000000000..0ad9efc11 --- /dev/null +++ b/docs/adr/0748-main-serves-a-rooms-federation-reads-while-the-workers-reload-is-pending.md @@ -0,0 +1,76 @@ +# Main serves a room's federation reads while the worker's reload of the room is pending + +* Status: accepted +* Deciders: James Chapman +* Date: 2026-11-14 + +Technical Story: Complement `TestEventAuth`, failing since 0.12.90. + +## Context and Problem Statement + +A federation worker answers a room's federation requests from its snapshot of the room (ADR-0733). When main changes +a room it calls `notify_room_changed`: the room's sync status becomes `pending`, and the worker reloads the room and +reports `ok` or `failed`. `FederationProxy` sent requests to main only after a `failed` reload. In the `pending` window +the worker answered from its old snapshot. + +`TestEventAuth` sends a state event, sees it in `/sync`, and asks `GET /event_auth` for it a few milliseconds later. The +worker had not reloaded yet and answered 404 "Event not found" for an event this server had just sent. + +How should a request be served while a reload is pending? + +## Decision Drivers + +* An answer about a room must not be older than what this server has already shown its own clients. +* Main can answer every room read itself: it does when there is no worker, and after a failed reload. Its read + providers take `runtime.mutex` (ADR-0711). +* Writes (`/send`, the membership endpoints) must keep going through the worker, whose relays to main carry the checks + of ADR-0071 and ADR-0595. + +## Considered Options + +* While a reload is `pending`, main serves the room reads that `federation_worker_room_read_may_fallback_to_main` lists + (GET `state`, `state_ids`, `backfill`, `event_auth` and `timestamp_to_event`, and POST `get_missing_events`, which + v1.19 defines as POST though it only reads), and writes go to the worker. +* Hold the request until the worker reports the reload, up to a timeout. +* Leave it as it is, since a remote server retries a 404. + +## Decision Outcome + +Chosen option: the first. `federation_room_request_served_by_main` makes the decision. It returns true for every +request after a `failed` reload, and for the listed reads while a reload is `pending`. `FederationProxy::handle` +calls it after verifying the request and applying the federation policy, so main receives only verified requests, as +it does after a failed reload. A pending fallback is logged at debug level (`federation_proxy.room_sync_pending_main`). +A failed one stays at warning level. + +### Positive Consequences + +* A remote server never learns less about a room than this server's own clients already see. +* The predicate that lists main's room reads, written for this purpose and never called, is now used. + +### Negative Consequences + +* In a busy room every change sets `pending`, so more of the room's reads are answered by main and fewer by the + sandboxed worker. They are reads from verified origins, which main already answers when no worker is configured. +* `GET /event/{eventId}` names no room, so it is not covered and is still answered from shard 0's snapshot. + +## Pros and Cons of the Options + +### Hold the request until the reload is reported + +* Good, because the worker would still serve every read. +* Bad, because it ties up a request thread for the length of a reload, and a reload that never reports would hold the + thread until its timeout. + +### Leave it as it is + +* Bad, because a 404 for an event that exists is a wrong answer, and a server that does not retry keeps it. + +## Links + +* Builds on [ADR-0733](0733-the-federation-worker-opens-no-database-main-sends-it-what-it-reads.md) and + [ADR-0735](0735-a-worker-may-have-main-read-a-room-only-under-a-permit-main-issued-and-only-once.md) +* Uses [ADR-0711](0711-the-federation-read-providers-read-main-s-store-under-the-runtime-lock.md) +* Refined by [ADR-0756](0756-main-serves-every-request-for-a-room-the-worker-has-never-loaded.md): a room the worker + has never loaded is served by main in full, writes included +* Refined by [ADR-0759](0759-main-builds-make-leave-and-make-knock-templates-while-the-workers-reload-is-pending.md): + GET `make_leave` and `make_knock` are reads, built by main while a reload is pending diff --git a/docs/adr/0749-outbound-delivery-never-serves-a-remembered-discovery-failure.md b/docs/adr/0749-outbound-delivery-never-serves-a-remembered-discovery-failure.md new file mode 100644 index 000000000..97183d922 --- /dev/null +++ b/docs/adr/0749-outbound-delivery-never-serves-a-remembered-discovery-failure.md @@ -0,0 +1,71 @@ +# Outbound delivery never serves a remembered discovery failure + +* Status: accepted +* Deciders: James Chapman +* Date: 2026-11-14 + +Technical Story: Complement `TestToDeviceMessagesOverFederation/stopped_server`, failing since 0.12.90 or earlier. + +## Context and Problem Statement + +`runtime.cached_discovery` caches server discovery for 60 seconds, including failures, so that a failing server does +not cost a full `.well-known`, SRV and DNS cascade on every PDU of a large inbound transaction. The dispatch worker's +resolver used the cache's default policy, `remember_failures`. + +In the failing test, hs1 queues a to-device EDU for hs2 while hs2 is down, and hs1 is then restarted. On restart hs1 +replays the transaction at once. hs2 is still down, so discovery fails and the failure is cached. hs2 comes back five +seconds later. hs1's next retries, at about 8 and then 24 seconds, are answered from the cached failure without +asking the network, so the backoff doubles each time and the next attempt lands after the test's 30-second wait. The +transaction itself was durable and replayed, and nothing dropped it. + +Which cached discovery answers may outbound delivery use? + +## Decision Drivers + +* The dispatch worker already paces a failing destination. A destination that is backing off takes no thread and no + lookup (`destination_should_retry`), so a remembered failure saves nothing the backoff does not. +* A retry after the destination recovers must reach it, as for a client's join (ADR-0262). +* A success is still worth reusing: every transaction to a reachable peer would otherwise repeat the cascade. + +## Considered Options + +* The dispatch resolver uses `DiscoveryCachePolicy::successes_only`: a fresh success is reused, a remembered failure + is ignored and the network is asked again. +* Shorten the cache's lifetime for failures only. +* Clear a destination's cached failure when that server sends this server a request. + +## Decision Outcome + +Chosen option: the first. `federation::make_dispatch_resolver` builds the resolver `wire_federation_callbacks` gives +the worker. It asks `cached_discovery` with `successes_only`, or the raw network when there is no cache. What it +finds still replaces the cache entry for every other caller, as `CachedServerDiscovery::discover` does for any policy. +Key resolution, pushers and appservices keep `remember_failures`. + +### Positive Consequences + +* Delivery to a destination resumes at its first retry after the destination recovers. +* A sender's restart while a peer is down no longer loses the first minute of the peer's recovery. + +### Negative Consequences + +* A destination that stays down is looked up once per backoff step, every few seconds at first and every 300 s at + the cap, instead of once a minute. Each lookup is bounded by `dispatch_discovery_timeout_seconds` (10 s, ADR-0354). + +## Pros and Cons of the Options + +### A shorter lifetime for failures + +* Good, because every caller benefits. +* Bad, because it changes the cache for key resolution, where remembering a failure guards against a request flood, + and any lifetime still refuses a retry that falls inside it. + +### Clear a cached failure when the destination sends a request + +* Good, because it reacts to evidence that the peer is up. +* Bad, because an inbound origin is not proven until its signature verifies, so an unverified request could change + outbound behaviour. Many recovered peers also never send this server anything. + +## Links + +* Applies [ADR-0262](0262-a-join-reuses-server-discovery-but-never-a-remembered-failure.md) to outbound delivery +* Bounded by [ADR-0354](0354-federation-delivery-gives-each-destination-at-most-one-thread.md) diff --git a/docs/adr/0750-complements-image-also-lists-localhost-as-a-cleartext-appservice-host.md b/docs/adr/0750-complements-image-also-lists-localhost-as-a-cleartext-appservice-host.md new file mode 100644 index 000000000..f1ae6ccd9 --- /dev/null +++ b/docs/adr/0750-complements-image-also-lists-localhost-as-a-cleartext-appservice-host.md @@ -0,0 +1,61 @@ +# Complement's image also lists localhost as a cleartext appservice host + +* Status: accepted +* Deciders: James Chapman +* Date: 2026-11-14 + +Technical Story: Complement `TestJoinFederatedRoomFromApplicationServiceBridgeUser` and +`TestJumpToDateEndpoint` failing against the 1.0.1 package + +## Context and Problem Statement + +[ADR-0515](0515-appservice-cleartext-http-only-to-loopback-literals-or-listed-hosts.md) accepts an +`http://` appservice URL only to a loopback IP literal or to a host listed in +`appservice.allow_cleartext_hosts`, and records that Complement's image lists `host.docker.internal` +alone. Complement's `hs_with_application_service` blueprint registers its appservice at +`http://localhost:9000`. `localhost` is a name, not a literal, so the registration was refused when the +server started and the appservice's `as_token` was answered 401. Every test built on that blueprint +failed before it reached what it tests. Should the image list `localhost` too? + +## Decision Drivers + +* Complement must exercise the appservice API, not the registration loader. +* The server's rule must not change for this: a cleartext name is always an operator's written choice. +* Inside the Complement container, `localhost` is the container's own loopback. + +## Considered Options + +* List `localhost` in the Complement image's `appservice.allow_cleartext_hosts`. +* Treat the name `localhost` as loopback in the server. +* Record the blueprint's tests as expected failures. + +## Decision Outcome + +Chosen option: list `localhost` in the image, as `host.docker.internal,localhost`. Only +`tests/complement/merovingian.conf.template` changes. The default stays empty, and the server still +decides from the URL text alone, so ADR-0515's rule is unchanged. ADR-0515's sentence naming the image's +setting is amended by this record. + +### Positive Consequences + +* The blueprint's registration loads, so its tests reach the behaviour they check. + +### Negative Consequences + +* The test image's list is longer than the one host the bundled appservices use. It is a test + configuration, and its tooling test pins the exact list. + +## Pros and Cons of the Options + +### Treat `localhost` as loopback in the server + +* Bad, because ADR-0515 refuses it on purpose: a name is resolved through configuration that can change, + and the rule must be decided from the URL text alone. + +### Record the tests as expected failures + +* Bad, because the server is not wrong, and the tests would never check the appservice API. + +## Links + +* Amends [ADR-0515](0515-appservice-cleartext-http-only-to-loopback-literals-or-listed-hosts.md) diff --git a/docs/adr/0751-the-per-client-outbound-call-cap-is-configurable-and-defaults-to-three.md b/docs/adr/0751-the-per-client-outbound-call-cap-is-configurable-and-defaults-to-three.md new file mode 100644 index 000000000..df288b21a --- /dev/null +++ b/docs/adr/0751-the-per-client-outbound-call-cap-is-configurable-and-defaults-to-three.md @@ -0,0 +1,78 @@ +# The per-client outbound call cap is configurable and defaults to three + +* Status: accepted +* Deciders: James Chapman +* Date: 2026-11-14 + +Technical Story: Complement `TestFederationRoomsInvite` and `TestJumpToDateEndpoint` subtests answered +`429 M_LIMIT_EXCEEDED` on `POST /join` since 0.12.92 + +## Context and Problem Statement + +[ADR-0079](0079-client-triggered-outbound-proxying-runs-under-a-bounded-in-flight-budget.md) bounds the +client-triggered calls this server makes to other servers: half of the request pool in total, and one per client +address, keyed by `rate_limit_client_key`. [ADR-0622](0622-a-clients-federated-join-knock-or-leave-holds-a-client-outbound-slot.md) +added federated joins, knocks and leaves to that budget, taken at once or answered 429. + +With one call per address, one client joining two remote rooms at once, or two users behind one NAT address doing +so, has every call but the first refused. Complement sends every request from one address and does not retry a 429, +so its parallel subtests' remote joins refused each other: 0 hits in the run before the audit, 3 to 5 in every run +since. The spec lists 429 for `POST /join`, so the server is not wrong, but one call per address is tighter than the +threat needs. How many calls may one address have in flight, and who decides? + +## Decision Drivers + +* ADR-0079's threat: a client naming a peer that never answers must not pin the request pool. The global cap of half + the pool is what keeps the other half free. +* The per-client cap stops one address from taking the whole global budget only while it is lower than the global + cap. With the minimum pool of 4 threads the global cap is 2, so the default of 3 is clamped to 2 and one address + can fill the budget. An operator may also set it as high as the global cap on purpose, for example for a test + harness that sends every request from one address. +* Legitimate clients do more than one remote call at once, and addresses are shared. + +## Considered Options + +* `server.http.client_outbound_max_per_client`, default 3, clamped to the global cap. +* Keep one per address, and record the Complement subtests as expected failures. +* Let a federated join, knock or leave wait for its client's slot (the ADR-0306 form). + +## Decision Outcome + +Chosen option: the configuration key, default 3. The owner chose the default. + +* `server.http.client_outbound_max_per_client`, 1..128, restart required. `client_outbound_proxy_policy_for_pool` + clamps it to the global cap (`request_threads / 2`, at least 1), so it can never exceed what the pool can spare. +* The global cap is unchanged, so the pool still keeps half of its threads for requests that never leave the server. +* Every route that takes a client-outbound slot uses the same cap: directory, alias, profile and key lookups, remote + media without the fetch pool, the hierarchy walk, `timestamp_to_event`, remote invites, and federated joins, knocks + and leaves. +* Complement's image sets 16, the global cap of the default pool, because it sends every request from one address. + +### Positive Consequences + +* A client can join a remote room while it looks another one up, and a small office behind one address is not limited + to one remote call at a time. +* An operator whose users share an address without a `server.trusted_proxies` entry can raise it. + +### Negative Consequences + +* One address may now hold three slots of the global budget instead of one, so it takes fewer addresses to fill the + budget with calls to a peer that never answers: 6 rather than 16 for the default global cap of 16. The global cap, + the deadlines and the per-address connection share (ADR-0072) still bound what they can hold. + +## Pros and Cons of the Options + +### Keep one per address, and record the subtests as expected failures + +* Bad, because which subtests fail depends on timing, so the entries would flap between failing and stale. +* Bad, because it keeps real clients' second concurrent remote call refused. + +### Let a join, knock or leave wait for its client's slot + +* Bad, because ADR-0622 rejected it: a waiter holds a request thread as well. + +## Links + +* Amends [ADR-0079](0079-client-triggered-outbound-proxying-runs-under-a-bounded-in-flight-budget.md) (the per-client + cap of one) +* Relates to [ADR-0622](0622-a-clients-federated-join-knock-or-leave-holds-a-client-outbound-slot.md) diff --git a/docs/adr/0752-build-py-vm-syncs-the-working-tree-over-ssh-and-runs-build-linux-sh.md b/docs/adr/0752-build-py-vm-syncs-the-working-tree-over-ssh-and-runs-build-linux-sh.md new file mode 100644 index 000000000..bcf347e43 --- /dev/null +++ b/docs/adr/0752-build-py-vm-syncs-the-working-tree-over-ssh-and-runs-build-linux-sh.md @@ -0,0 +1,148 @@ +# build.py vm syncs the working tree over ssh and runs build-linux.sh + +* Status: accepted +* Deciders: James Chapman +* Date: 2026-11-14 + +Technical Story: build and test on a Linux VM instead of WSL, and on a FreeBSD VM from the same machine + +## Context and Problem Statement + +Windows development built and tested through `python build.py wsl`, on the NTFS-backed `/mnt/c` checkout. That path +needs its own workarounds (`build-wsl.sh`'s curl and `make` shim handling, ADR-0158's line endings) and shares the +Windows host's memory with the editor and agents. The owner wants builds and tests on a Linux VM instead. The VM runs +under Hyper-V, but the developer account is not in Hyper-V Administrators, so the Hyper-V PowerShell API (`Get-VM`, +PowerShell Direct) is unavailable. Only ssh over the network reaches it. How does a Windows checkout, with uncommitted +edits, get built and tested there, with results that the verification rules in `AGENTS.md` can read? + +## Decision Drivers + +* Uncommitted work has to be tested; that is what a local run is for. +* Several worktrees, and several sessions, build from the same machine; one run must not overwrite another's sources + or its results, and two full suites at once time out unrelated tests. +* The verification rules read `/meson-logs/testlog.txt`; a stale log read as a fresh result is a false pass. +* The host is reached over the network, so its identity must be checked and nothing may prompt in an unattended run. +* Windows has Python, Git and OpenSSH, but no rsync. + +## Considered Options + +* ssh plus a tar stream built by `build.py`, with a manifest for incremental sync +* rsync over ssh +* `git push` to a repository on the VM, then check out there +* A shared folder (SMB or Hyper-V file share) mounted on the VM +* Hyper-V PowerShell Direct + +## Decision Outcome + +Chosen option: "ssh plus a tar stream built by `build.py`", because it needs nothing on Windows that a checkout does +not already have, carries uncommitted edits, and keeps the build on the VM's own file system. + +The rules this sets, each of which the code depends on: + +* **What is sent.** Every tracked file as on disk, with Git's mode, and every untracked file Git does not ignore, + except under a top-level name starting with `build` or `.`. The `--builddir` must therefore start with `build`, so + the logs copied back are never sent. A symbolic link or submodule is refused, not followed. +* **Incremental sync.** The VM keeps `.vm-sync/manifest` (mode, size, mtime, path). Only files that differ are sent; + files the old manifest lists and the new one does not are deleted before the archive is unpacked (so a file that + became a directory, or the reverse, cannot block it), never a path that is absolute or contains `..`, and any + directory that leaves empty goes too. A stale path that is now a real directory is left alone: only an interrupted + sync that already unpacked files into it can make one, and the archive fills it again. Before anything is deleted + or unpacked, the run zeroes the mode, size and mtime of every manifest line but keeps its path, so after an + interruption no file is listed as present and current, yet the stale list the next run plans still names + everything an earlier run sent. Deleting the manifest instead was rejected: a later run would then never delete + the files the interrupted run had left. Before unpacking, the run also adds every path the archive will bring, zeroed, + which `build.py` sends on stdin ahead of the archive as a length-prefixed list read with `os.read`, so a run cut off + part-way leaves no file the manifest does not name, and the next run deletes those its checkout no longer has, even + when the local files changed in between. Sending that list in the remote script was rejected: the script is one + ssh argument, which a first sync's list could make longer than the system allows. The new manifest arrives last + in the archive and is installed only after extraction succeeds, so an interrupted sync resends what it missed. +* **The baseline is checked under the lock.** `build.py` reads the manifest, plans against it, and sends its SHA-256 + with the run; the remote script compares it once it holds the lock. If another run changed the manifest in between, + the script touches nothing, leaves a marker in its run directory and exits 75, and `build.py` reads the manifest + again and resends, three times at most. Reading the manifest inside the locked session instead was rejected: the + archive would have to be planned mid-session over one ssh channel, a framing protocol the hash check does not need. + The marker, not the exit status alone, says to retry, so an `--exec` command that exits 75 is not run again. +* **Isolation.** One VM-side checkout per local checkout, named after its directory and a hash of its full path, and + one lock (`~/merovingian-vm/.lock`) held across sync, build and test for every run from every checkout. + The lock is in the ssh user's home, so it orders the runs of one VM user; every developer and agent using a VM + shares one VM user. A shared lock in `/tmp` was rejected: another local user could pre-create or replace it. +* **The lock is `flock(2)` taken through `python3`'s `fcntl`**, on a descriptor the remote shell holds open for the + whole run. `flock(1)` was rejected because FreeBSD, OpenBSD and NetBSD do not ship it; `lockf(1)` because Linux + does not, and because it locks only for the one command it runs. Both lock kinds are `flock(2)`, so a run with the + older `flock(1)` script and a run with this one still exclude each other. +* **Results.** `build-linux.sh`, not `build-wsl.sh`, runs on a Linux VM: the NTFS workarounds have nothing to work + around there. The remote host picks its build script from `uname -s`: `build-linux.sh` on Linux, `build-bsd.sh` on + FreeBSD, OpenBSD, NetBSD and DragonFly, and any other system is refused, never built with a guessed script. + Before a run that tests, both the local and the VM's `testlog.txt` and `testlog.json` are deleted, the VM's before + anything is unpacked, so a run that fails before testing leaves no log. Still under the lock, the run copies its logs into `.vm-sync/run-/`, a directory named by a random token of its own, and `build.py` + fetches them from there and then deletes it: a run that takes the lock as soon as this one releases it cannot + replace or delete them first. The exit status is the build's. +* **Host identity.** ssh runs with `StrictHostKeyChecking=yes` and `BatchMode=yes`: an unknown or changed host key + stops the run, and nothing prompts. Settings come from flags, then `MEROVINGIAN_VM_*`, then + `~/.config/merovingian/vm.ini`, outside the repository, so no host address is committed. A host or user that ssh + could read as an option is refused. +* **Named VMs.** `--vm ` (or `MEROVINGIAN_VM_NAME`) reads the `[vm.]` section and nothing from `[vm]`, so + a named VM never inherits the default VM's host or key; a name with no section is an error, not a fallback. Its + default build directory is `build-vm-`, so each VM's copied-back logs have their own place locally. +* **Memory.** Without `--jobs`, a run uses one compile job and one test process per 1.5 GiB of VM memory + (`/proc/meminfo` on Linux, `sysctl hw.physmem` on the BSDs), at most one per CPU, through the `--jobs` option this + adds to `build-linux.sh` and `build-bsd.sh`; Meson's default of one per CPU runs a small VM out of memory. +* **Forwarded options.** `build.py` passes the build scripts only options they parse; `test_build_vm.py` runs + both scripts with `--hardening` and `--jobs` and checks what reaches Meson, so an option one script refuses fails + there, not on a developer's VM. +* **The VM is not CI.** A VM run meets what a root CI job does not: a non-root user, and a host where a package + installed the system config. A smoke test that is not about the defaults passes `--config` with the example + config; the one test of the defaults skips (exit 77) when a system config exists; and the live-binary startup + test accepts the unsandboxed worker off Linux, as the startup hardening gate does (ADR-0041). +* **Setup.** `scripts/remote-host-setup.sh` prepares the remote host: on the Debian family, Meson 1.11+ in a virtual + environment in `~/.local/bin`, which `build.py vm` puts first on `PATH`; on FreeBSD, the FreeBSD CI job's packages + from `pkg`, whose Meson is current, plus `python3` for the lock. A VM without `python3` cannot be synced, so the + script is copied over and run on its own before the first sync. `scripts/wsl-setup.sh` stays for WSL only. + +### Positive Consequences + +* A full suite runs on Linux-native ext4 with the VM's memory, from any worktree, without WSL. +* The local `build-vm/meson-logs/testlog.txt` is read exactly as any other build's log. + +### Negative Consequences + +* If ssh drops mid-run, the build keeps running on the VM and holds the lock until it ends. +* Test binaries stay on the VM; a tag-filtered run goes through `build.py vm --exec`. +* An mtime-and-size manifest misses a change that keeps both, such as an edit within the same second that keeps the + size; `--clean` does not help there, but touching the file does. + +## Pros and Cons of the Options + +### ssh plus a tar stream built by build.py + +* Good, because it needs only Python's `tarfile` and the OpenSSH client Windows ships. +* Good, because it sends uncommitted edits and only what changed. +* Bad, because `build.py` owns a small sync protocol, covered by `build-vm-tooling`. + +### rsync over ssh + +* Good, because it is the standard tool for the job. +* Bad, because Windows has no rsync; Git for Windows does not ship it, and a Cygwin or MSYS2 copy maps paths and + modes in its own way. + +### git push to a repository on the VM + +* Good, because Git already moves exactly the tracked content. +* Bad, because it sends only commits, so every test run would need a commit, and untracked new files would be missed. + +### A shared folder mounted on the VM + +* Good, because nothing is copied. +* Bad, because the build would again run on a network or foreign file system, the problem WSL's `/mnt/c` has. + +### Hyper-V PowerShell Direct + +* Good, because it needs no network or ssh key. +* Bad, because it needs Hyper-V Administrators rights, which the developer account does not have, and works only for + a VM on the same Windows host. + +## Links + +* Related to [ADR-0158](0158-check-out-every-text-file-with-lf-endings.md), the line endings the sync relies on +* Related to [ADR-0546](0546-ci-fails-on-a-catch2-skip-its-job-does-not-list.md); `build.py vm` passes no + `--skip-job`, so its runs are not skip-gated diff --git a/docs/adr/0753-tests-never-write-a-fixed-name-in-the-shared-temporary-directory.md b/docs/adr/0753-tests-never-write-a-fixed-name-in-the-shared-temporary-directory.md new file mode 100644 index 000000000..c3a1cfdba --- /dev/null +++ b/docs/adr/0753-tests-never-write-a-fixed-name-in-the-shared-temporary-directory.md @@ -0,0 +1,66 @@ +# Tests never write a fixed name in the shared temporary directory + +* Status: accepted +* Deciders: James Chapman +* Date: 2026-11-14 + +Technical Story: the 1.0.2 branch's first `build.py vm` run as a second account on the build VM failed four +assertions in `test_appservice_query_hooks_flow.cpp` + +## Context and Problem Statement + +Several tests wrote their scratch files at fixed names directly in the system temporary directory, for example +`/tmp/merovingian-queryhook-alias.json`, and did not check the write. The build VM now has two accounts that run the +suite ([ADR-0752](0752-build-py-vm-syncs-the-working-tree-over-ssh-and-runs-build-linux-sh.md)). A file the first +account left in `/tmp` belongs to it, and the sticky bit stops the second account replacing it. The second +account's write therefore failed without a sound, and the homeserver loaded the earlier run's registration, which +named another mock port, so the mock never saw the query. Two suites running at once under one account would +overwrite each other's files in the same way. How should tests name their scratch files? + +## Decision Drivers + +* A test must read the file it wrote, never one another run left. +* A failed write must fail the test where it happens, not later as a missing request. + +## Considered Options + +* Keep the fixed names and delete any old file before writing. +* Give every scratch file or directory a name unique to the process and call, and check every write. +* Give each test binary a private directory under the temporary directory. + +## Decision Outcome + +Chosen option: "Give every scratch file or directory a name unique to the process and call, and check every write", +because it holds for any number of accounts and concurrent runs and needs no cleanup to be correct. +`merovingian::tests::unique_temporary_path(stem, extension)` in `tests/support/TempDirectory.hpp` returns +`---` under `temporary_directory()` without creating it. A file name with no +unique part (a process ID, counter, random value or `mkstemp` template) is not used for anything a test creates. A +test that writes a scratch file with a stream closes it and requires `good()` before using it. A path a test only +names to show it is missing may stay fixed, since nothing creates it. + +### Positive Consequences + +* Suites run side by side, under one account or several, without reading each other's files. +* A refused write fails at the write, naming the file. + +### Negative Consequences + +* Files that a test does not remove stay in the temporary directory under distinct names, rather than being + overwritten by the next run. Cleanup is unchanged from before. + +## Pros and Cons of the Options + +### Keep the fixed names and delete any old file before writing + +* Bad, because one account cannot delete another account's file in sticky `/tmp`, which is the failure itself. +* Bad, because two concurrent runs still race on the same name. + +### Give each test binary a private directory under the temporary directory + +* Good, because cleanup becomes one `remove_all`. +* Bad, because every helper and fixture that takes a path would need the directory passed through, for the same + guarantee the unique name already gives. + +## Links + +* Relates to [ADR-0752](0752-build-py-vm-syncs-the-working-tree-over-ssh-and-runs-build-linux-sh.md) diff --git a/docs/adr/0754-agent-skills-are-committed-under-agent-skills-and-cite-the-binding-docs.md b/docs/adr/0754-agent-skills-are-committed-under-agent-skills-and-cite-the-binding-docs.md new file mode 100644 index 000000000..f59dd80ba --- /dev/null +++ b/docs/adr/0754-agent-skills-are-committed-under-agent-skills-and-cite-the-binding-docs.md @@ -0,0 +1,66 @@ +# Agent skills are committed under .agent/skills and cite the binding docs + +* Status: accepted +* Date: 2026-11-14 + +## Context and Problem Statement + +Reviews of this project's changes keep missing the same things. Code holds `runtime.mutex` across I/O or unbounded +work. A handler is added to main without its federation worker path. A test passes with the fix reverted. A branch +updates the user manual but not the owning guide. A general-purpose review skill knows none of this. A +project-specific review skill does, and it has to live somewhere every session, worktree and contributor can reach. +It also has to stay consistent with the rules it checks against. + +## Considered Options + +* `.claude/skills/` in the repository +* The reviewer's own user-level skill directory +* `.agent/skills/`, committed with the code +* A skill that restates the coding, security and testing rules in full + +## Decision Outcome + +Chosen option: `.agent/skills/`, committed with the code. The first skill is +`.agent/skills/merovingian-review/`. The skill is a method and a checklist. Its reference files point at the +binding text by `path#heading` and ADR number, and they do not restate the rules. Where a reference and the +document it cites disagree, the document wins, and the disagreement is a defect in the skill. The same principle +is set for the root guidance in [ADR-0465](0465-root-agent-guidance-requires-the-coding-rules-without-copying-them.md). +The `docs/AGENTS.md` update table gives the skill a row, so a change to anything it cites updates it in the same +branch. + +### Positive Consequences + +* Every worktree, CI agent and contributor reviews against the same, versioned checklist. +* The skill is reviewed and changed through pull requests like the code it describes. +* A rule change lands in one binding document; the skill's pointer to it stays valid. + +### Negative Consequences + +* Tools that discover skills only under `.claude/skills/` do not find it on their own. The root `AGENTS.md` names + the skill, and a contributor may link it into their own skill directory. +* Heading-based citations break when a cited heading is renamed. A reviewer who finds a dead citation fixes the + skill in the same branch. + +## Pros and Cons of the Options + +### `.claude/skills/` in the repository + +* Good, because Claude Code discovers it with no setup. +* Bad, because `.claude/` is ignored by `.gitignore`. The skill would exist in one checkout only, and worktrees + under `.worktrees/` and `.claude/worktrees/` would not see it. + +### The reviewer's own user-level skill directory + +* Good, because every checkout on that machine sees it. +* Bad, because it is not versioned with the code. Other contributors and CI agents do not have it, and nothing + keeps it in step when a rule changes. + +### A skill that restates the rules in full + +* Good, because a reviewer reads one file. +* Bad, because two copies of a rule drift apart. The root `AGENTS.md` dropped its copies for that reason + (ADR-0465). + +## Links + +* Refines [ADR-0465](0465-root-agent-guidance-requires-the-coding-rules-without-copying-them.md) diff --git a/docs/adr/0755-no-std-format-the-netbsd-libc-has-no-format-header.md b/docs/adr/0755-no-std-format-the-netbsd-libc-has-no-format-header.md new file mode 100644 index 000000000..c6c0c1da7 --- /dev/null +++ b/docs/adr/0755-no-std-format-the-netbsd-libc-has-no-format-header.md @@ -0,0 +1,38 @@ +# No std::format: the NetBSD libc++ has no `` header + +* Status: accepted +* Deciders: James Chapman +* Date: 2026-11-14 + +Technical Story: `netbsd-build-and-test` on PR #592 failed to compile `tests/support/TempDirectory.hpp` with +"'format' file not found" + +## Context and Problem Statement + +`unique_temporary_path` ([ADR-0753](0753-tests-never-write-a-fixed-name-in-the-shared-temporary-directory.md)) built +its name with `std::format`. It compiled on Linux, FreeBSD and OpenBSD, but the NetBSD CI image's libc++ does not +ship the `` header, so the unit and integration test binaries failed to build there. Nothing in the tree had +included `` before. How should code build strings from numbers? + +## Considered Options + +* Do not include `` (`std::format`, `std::format_to`, `std::vformat`, `std::print`) in production code or + tests; build strings with `std::to_string`, `std::to_chars` and string concatenation. +* Use `` where it is available and fall back with a feature-test macro elsewhere. + +## Decision Outcome + +Chosen option: "Do not include ``", because the supported NetBSD standard library lacks it, and a +feature-test fallback would mean two code paths of which CI builds each on one platform only. `std::to_chars` (integer +overloads) and `std::to_string` are available on every supported platform and are what the rest of the tree uses. +This is the same kind of rule as the one for `std::jthread` and `std::stop_token` +([ADR-0167](0167-no-jthread-or-stop-token-use-std-thread-and-stop-flag.md)). + +### Negative Consequences + +* Formatting with width, padding or a radix is written by hand, for example `std::to_chars(..., value, 16)`. + +## Links + +* Relates to [ADR-0167](0167-no-jthread-or-stop-token-use-std-thread-and-stop-flag.md) +* Relates to [ADR-0753](0753-tests-never-write-a-fixed-name-in-the-shared-temporary-directory.md) diff --git a/docs/adr/0756-main-serves-every-request-for-a-room-the-worker-has-never-loaded.md b/docs/adr/0756-main-serves-every-request-for-a-room-the-worker-has-never-loaded.md new file mode 100644 index 000000000..f78fb7130 --- /dev/null +++ b/docs/adr/0756-main-serves-every-request-for-a-room-the-worker-has-never-loaded.md @@ -0,0 +1,99 @@ +# Main serves every request for a room the federation worker has never loaded + +* Status: accepted +* Deciders: James Chapman +* Date: 2026-11-14 + +Technical Story: Complement `TestKnockingInMSC3787Room`, failing on the Linux VM run of 1.0.2 at 7412b050. + +## Context and Problem Statement + +A federation worker holds the rooms main listed when it started, and loads a room again when main calls +`notify_room_changed` (ADR-0733). A room created after the worker started is in its store only once the first reload of +it has finished. Reloads run on one thread and are coalesced (ADR-0623), so that can take longer than a fast remote +server needs for invite, `make_join` and `send_join`. + +ADR-0748 sends a room's reads to main while a reload is pending and keeps its writes on the worker. That is right for a +room the worker holds: a slightly old copy does not change what the worker checks before it relays a write to main. +For a room the worker does not hold at all it is wrong. The worker's `room_version_resolver` finds no create event and +returns nothing, `handle_send_membership` and the `/send` path take room version 12, and the event ID and the +signatures are computed over the redacted event under that version's rules. For an event of an earlier room version +the event ID then differs from the one in the URL, and the worker answers 400 "membership event does not match +endpoint, path or origin". `TestKnockingInMSC3787Room` created a version 10 room and had Complement's server join it +about 250 ms later, and lost that race. + +How should a request for a room the worker has never loaded be served? + +## Decision Drivers + +* A worker must not judge an event under a room version it guessed. +* Main can serve every request for a room: it does when no worker is configured and after a failed reload. +* Requests for a room the worker holds should keep going through the sandboxed worker (ADR-0748). + +## Considered Options + +* Track per room whether the worker holds it (`RoomSyncStatus::held`). A room main has never notified is held (it was + in the worker's startup fill, or does not exist). A room's first notification marks it not held, and any successful + reload marks it held. While it is not held, main serves every request for it, as after a failed reload. +* Have the worker ask main for the room version when it has none. +* Hold the request on the worker until its first reload of the room has finished. + +## Decision Outcome + +Chosen option: the first. `WorkerPool::notify_room_changed` and `record_room_sync_result` keep the flag (per shard +since ADR-0758), and +`federation_room_request_served_by_main` returns true while it is false. Main already serves a room whose reload failed +in full, so this adds no path main does not have. + +Three more ways a worker can lack a room are covered the same way: + +* `notify_room_changed` writes the room's status before it checks the worker's channel, so a room first changed while + the channel is down is not held either. +* The worker's start-up fill leaves out a room main cannot send it, and it reports each one as a `room_sync_result` of + `failed` at generation 0, which main never assigns. Main records the room as not held + (`record_room_left_out_of_fill`). It takes the report only for a room in the listing it sent that channel, once + (`WorkerSnapshotService::consume_fill_omission`), so a worker cannot make main record rooms main never named. + The flag is kept per shard, and a report marks the room not held only on the shard that sent it + ([ADR-0758](0758-every-worker-shard-reloads-every-changed-room-and-main-tracks-each-shards-copy.md)). +* A `/send` transaction goes whole to the shard its first PDU picks, so the proxy checks that shard's status of every + room the transaction names (`federation_request_room_ids`) and serves the whole transaction from main if any of them + must be. Every shard reloads every changed room (ADR-0758), so the selected shard's copy of a room another shard owns + is current once its reload lands. + +The flag is conservative: main does not record which rooms the startup listing held, so a room that existed at startup +is also not held from its first notification until that reload succeeds. Main serves it for that one reload, once per +room per main process. Recording the listing would avoid that, at the cost of a second copy of the room list. + +### Positive Consequences + +* A remote server's first join, knock, leave or PDU for a room this server has just created is judged under the room's + real version. +* Rooms the worker holds keep ADR-0748's split: reads to main while a reload is pending, writes to the worker. + +### Negative Consequences + +* Main serves a room's requests in full during its first reload, including the first change of each room that existed + when the worker started. +* A room whose first reload's result never arrives, because the worker restarted, stays with main until its next change + is loaded. So does a room first changed while the worker's channel was down, even when the restarted worker's fill + copied it: the fill reports only the rooms it left out. +* A transaction naming one room the worker does not hold is served by main in full, its other rooms included. + +## Pros and Cons of the Options + +### Have the worker ask main for the room version + +* Good, because the worker would still serve the request. +* Bad, because the worker still has no state for the room: every read and every check that needs state would be wrong + in other ways, and the worker would need a new relay to main. + +### Hold the request until the first reload has finished + +* Bad, because it ties up a request thread for a reload, which ADR-0748 rejected for the same reason. + +## Links + +* Refined by [ADR-0758](0758-every-worker-shard-reloads-every-changed-room-and-main-tracks-each-shards-copy.md) +* Refines [ADR-0748](0748-main-serves-a-rooms-federation-reads-while-the-workers-reload-is-pending.md) +* Builds on [ADR-0733](0733-the-federation-worker-opens-no-database-main-sends-it-what-it-reads.md) and + [ADR-0623](0623-a-workers-room-reloads-are-coalesced-per-room-on-one-thread.md) diff --git a/docs/adr/0757-hardened-bsd-builds-link-pie-and-the-elf-gate-reads-elftoolchain-readelf.md b/docs/adr/0757-hardened-bsd-builds-link-pie-and-the-elf-gate-reads-elftoolchain-readelf.md new file mode 100644 index 000000000..2e94d1aee --- /dev/null +++ b/docs/adr/0757-hardened-bsd-builds-link-pie-and-the-elf-gate-reads-elftoolchain-readelf.md @@ -0,0 +1,69 @@ +# Hardened BSD builds link -pie, and the ELF gate reads elftoolchain readelf + +* Status: accepted +* Deciders: James Chapman +* Date: 2026-11-14 + +Technical Story: the FreeBSD release evidence of 1.0.2 reported PIE and BIND_NOW missing from +`build-freebsd-release/src/merovingian-server`. + +## Context and Problem Statement + +`scripts/collect-release-evidence.sh` runs `scripts/check-elf-hardening.sh` on the server that each platform's release +job builds. On FreeBSD it reported two failures, and they had different causes. + +PIE was really missing. `meson.build` added `-pie` to the hardened link flags on Linux only, and left the BSDs to the +packaging scripts: `build-freebsd-pkg.sh` passes `-pie` through `cpp_link_args`. FreeBSD's clang does not link a PIE by +default, so every other hardened FreeBSD build was `ET_EXEC`: `build-bsd.sh --profile hardened`, which the release +evidence job uses, `build.py vm --vm freebsd`, and any build an operator makes from source. Its objects were compiled +with `-fPIE`, and only the link step was missing. + +BIND_NOW was not missing. The gate matched GNU binutils' output, `(FLAGS_1) Flags: NOW`. The BSDs' `readelf` is +elftoolchain, which prints DT_FLAGS as `(FLAGS) BIND_NOW` and DT_FLAGS_1 as `(FLAGS_1) NOW`, so a binary linked with +`-Wl,-z,now` failed the check. The old pattern also used `\b`, which POSIX ERE does not define. + +## Decision Drivers + +* A hardened build must be hardened however it was built, not only when a packaging script built it. +* The gate must give the same answer for the same binary whichever platform's `readelf` reads it. +* The gate must still fail closed: lazy binding must not pass because the pattern is looser. + +## Considered Options + +* Add `-pie` in `meson.build` for every platform whose hardened build requests RELRO and BIND_NOW (Linux, FreeBSD, + OpenBSD), and match each `readelf`'s form of the dynamic flags. +* Pass `-pie` from `build-bsd.sh` as the packaging script does, and install GNU binutils on the BSD runners for the gate. + +## Decision Outcome + +Chosen option: `-pie` in `meson.build`, and a gate that reads both tools, because the build definition is the +one place every build goes through, and the system `readelf` is the one each platform ships. + +The pattern accepts a `(BIND_NOW)` entry, `BIND_NOW` as a word in the value of `(FLAGS)`, or `NOW` as a word in the +value of `(FLAGS_1)`. It uses POSIX ERE only. NetBSD stays out of the `-pie` and RELRO/BIND_NOW list, because its +server refuses to start whatever it is linked with (ADR-0378). + +### Positive Consequences + +* `build-bsd.sh --profile hardened`, `build.py vm --vm freebsd` and the release evidence produce the PIE that the + shipped `.pkg` already was. +* `tests/tooling/test_check_elf_hardening.py` runs the gate against GNU and elftoolchain output, with a fake `readelf`, + on every platform the tooling tests run on. + +### Negative Consequences + +* The FreeBSD `.pkg` build passes `-pie` twice, once from `meson.build` and once from `cpp_link_args`. The linker + accepts the duplicate. + +## Pros and Cons of the Options + +### Pass -pie from build-bsd.sh and use GNU readelf + +* Bad, because a build from `meson setup -Dhardening=true` alone would still be `ET_EXEC` on FreeBSD. +* Bad, because the gate would depend on a package the BSD hosts do not need for anything else, and would still fail + wherever only the system `readelf` is installed. + +## Links + +* Refines the build-time hardening in [docs/hardening.md](../hardening.md#build-time-toolchain-hardening) +* NetBSD's server refuses to start: [ADR-0378](0378-probe-elf-hardening-in-memory-and-keep-netbsd-refused.md) diff --git a/docs/adr/0758-every-worker-shard-reloads-every-changed-room-and-main-tracks-each-shards-copy.md b/docs/adr/0758-every-worker-shard-reloads-every-changed-room-and-main-tracks-each-shards-copy.md new file mode 100644 index 000000000..096282b1e --- /dev/null +++ b/docs/adr/0758-every-worker-shard-reloads-every-changed-room-and-main-tracks-each-shards-copy.md @@ -0,0 +1,81 @@ +# Every worker shard reloads every changed room, and main tracks each shard's copy + +* Status: accepted +* Deciders: James Chapman +* Date: 2026-11-14 + +Technical Story: CodeRabbit review of PR #592 on the ADR-0756 fix; owner decision "Fix properly in this PR". + +## Context and Problem Statement + +With more than one worker shard, a room's requests go to the shard its room ID hashes to (FNV-1a), and until now only +that shard was told when the room changed. But a `/send` transaction goes whole to the shard its first PDU picks +(ADR-0131), and may carry PDUs for rooms other shards own. That shard's copy of such a room was its start-up snapshot, +or nothing for a room created since, whatever main recorded about the owner's copy. So the worker could judge a PDU +against old state, or under a guessed room version (ADR-0756). + +The first fix served such a transaction from main whenever its rooms spanned shards. That breaks ADR-0131's sandbox +boundary: a remote server chooses which rooms its transaction names, so any remote could steer its transactions into +the main process by naming rooms of two shards. + +How should a worker answer for a room another shard owns? + +## Decision Drivers + +* A remote server must not be able to choose whether its request is handled by main or by the sandboxed worker + (ADR-0131). +* The worker that answers a request must hold a current copy of every room the request names, or main must answer it. +* One transaction ID is answered once, by one process. + +## Considered Options + +* Tell every shard of every room change, and keep each room's sync status per shard. A request is judged on the copy of + the shard it is sent to. +* Serve a transaction from main when its rooms belong to more than one shard. +* Split a transaction by room and send each part to the room's owning shard. + +## Decision Outcome + +Chosen option: the first. `WorkerPool::notify_room_changed` assigns one generation per change, marks the room +`pending` on every shard and sends `room_sync` to every healthy shard, with a reload permit on each channel (ADR-0735). +Each `room_sync_result` is recorded against the shard that sent it, and so is a start-up fill omission (ADR-0756). +`FederationProxy::handle` picks the shard first (`shard_for` of the request's routing room, shard 0 for none) and checks +`room_sync_status(shard, room)` for every room the request names; ADR-0748 and ADR-0756 then apply to that copy. A +result can only be recorded for a generation main issued, or for a room main listed to that channel, so a worker cannot +grow main's maps. + +`GET /hierarchy` stays with main (ADR-0712): the proxy judges only the room in a request's path, and a hierarchy reads +child rooms whose reloads may be pending or failed on the selected shard. + +### Positive Consequences + +* No request reaches main because of which rooms a remote server named; main answers only for a copy the selected + worker lacks or has not yet updated. +* A transaction naming rooms of several shards is handled by one worker, with current copies of all of them. + +### Negative Consequences + +* Every room change makes every shard reload the room: with `N` shards, `N` snapshot reads in main per change. The + default (`shards = 1`) is unchanged. Reloads stay coalesced per room on each worker (ADR-0623). +* Main keeps one status entry per shard for each room it has notified. + +## Pros and Cons of the Options + +### Serve a cross-shard transaction from main + +* Good, because it is a small change. +* Bad, because a remote server decides, by the rooms it names, that its transaction is parsed and judged by main + instead of the sandboxed worker, which is what ADR-0131 forbids. + +### Split a transaction by room across shards + +* Bad, because one transaction ID would be answered piecemeal by several workers, and the per-PDU results would have + to be merged in main. + +## Links + +* Refines [ADR-0756](0756-main-serves-every-request-for-a-room-the-worker-has-never-loaded.md) +* Keeps [ADR-0131](0131-keep-malformed-first-pdu-transactions-inside-worker-shard-zero.md) and + [ADR-0712](0712-a-space-hierarchy-request-is-answered-by-main-not-by-a-worker-snapshot.md) +* Builds on [ADR-0735](0735-a-worker-may-have-main-read-a-room-only-under-a-permit-main-issued-and-only-once.md) and + [ADR-0748](0748-main-serves-a-rooms-federation-reads-while-the-workers-reload-is-pending.md) diff --git a/docs/adr/0759-main-builds-make-leave-and-make-knock-templates-while-the-workers-reload-is-pending.md b/docs/adr/0759-main-builds-make-leave-and-make-knock-templates-while-the-workers-reload-is-pending.md new file mode 100644 index 000000000..6a17b7bb2 --- /dev/null +++ b/docs/adr/0759-main-builds-make-leave-and-make-knock-templates-while-the-workers-reload-is-pending.md @@ -0,0 +1,83 @@ +# Main builds make_leave and make_knock templates while the worker's reload of the room is pending + +* Status: accepted +* Deciders: James Chapman +* Date: 2026-11-15 + +Technical Story: Complement `TestFederationRoomsInvite/Parallel/Invited_user_can_reject_invite_over_federation_several_times`, +failed once in the full run at 33a5cb61 after passing four times. + +## Context and Problem Statement + +ADR-0748 has main serve a room's reads while a worker's reload of the room is `pending`, and keeps the room's writes on +the worker. It listed `state`, `state_ids`, `backfill`, `event_auth`, `timestamp_to_event` and `get_missing_events` as +the reads. `make_leave` and `make_knock` were left with the membership writes, so the worker kept building them from +its snapshot. `make_join` was not affected: the worker always has main build its template (ADR-0184). + +A `make_leave` template names the user's current membership event in `auth_events`. In the failed run, hs1 invited a +user on hs2, the invite returned 200, and the user rejected it about 300 ms later. hs1's worker had not reloaded the +room yet. Its template left out the invite, so the leave was a self-leave with no prior membership. When hs2 sent it +back, main judged it against those auth events and refused it with 403 ("self-leave requires current membership of +invite, join, or knock"). hs2's client got 502. `make_knock` has the same window: its template names the knocking +user's current membership. + +How should a leave or knock template be built while a reload is pending? + +## Decision Drivers + +* A template must not leave out a membership change this server has already committed and shown to its clients. +* Main can build every membership template itself: it does when no worker is configured, and its provider takes + `runtime.mutex`. +* Requests for a room the worker holds should keep going through the sandboxed worker when its copy is current + (ADR-0748, ADR-0756). + +## Considered Options + +* List GET `make_leave` and `make_knock` in `federation_worker_room_read_may_fallback_to_main`, so main builds them + while a reload is pending and the worker builds them otherwise. +* Have the worker always hand `make_leave` and `make_knock` to main, as it does `make_join` (ADR-0184). +* Have the leaving server retry `send_leave` with a fresh `make_leave` after a 403. + +## Decision Outcome + +Chosen option: the first. Both endpoints are GETs that build a template from room state and change nothing, so they +are reads in ADR-0748's sense, and `federation_room_request_served_by_main` already sends such reads to main while a +reload is pending. A PUT to either path is not a route the spec defines and does not fall back. The worker's local +provider is unchanged and still builds both when its copy is current. + +### Positive Consequences + +* A remote user can reject an invite, or leave, the moment this server shows them the invite, and the template names + the invite. The same holds for a knock template. +* No IPC frame or relay changes. Main receives only requests `FederationProxy` has already verified, as for the other + pending reads. + +### Negative Consequences + +* In a busy room more leave and knock templates are built by main instead of the sandboxed worker. They come from + verified origins, and main already builds them when no worker is configured. +* As for every ADR-0748 read, the fix holds only while `notify_room_changed` marks the room `pending` before a client + can see the change. A path that commits a membership change without notifying the worker would reopen the window. + +## Pros and Cons of the Options + +### Always relay make_leave and make_knock to main + +* Good, because the template could never be stale, whatever the reload state. +* Bad, because every leave and knock template would leave the sandbox, also when the worker's copy is current. +* Bad, because `membership_template_ingest` carries no endpoint. Main assumes `make_join`, so the frame would need a new + field that main must validate. + +### Retry send_leave after a 403 + +* Good, because it needs no change to request routing. +* Bad, because it fixes only this server as the leaving side. A remote homeserver that leaves through this server still + gets the stale template. +* Bad, because it hides a stale answer instead of not giving one. + +## Links + +* Refines [ADR-0748](0748-main-serves-a-rooms-federation-reads-while-the-workers-reload-is-pending.md) +* Keeps [ADR-0184](0184-federation-workers-relay-make-join-to-main.md) for `make_join` +* Applies per shard copy as in + [ADR-0758](0758-every-worker-shard-reloads-every-changed-room-and-main-tracks-each-shards-copy.md) diff --git a/docs/adr/index.md b/docs/adr/index.md index cbd67601f..5791492bc 100644 --- a/docs/adr/index.md +++ b/docs/adr/index.md @@ -626,3 +626,18 @@ * [0741 - Each changelog version has Features, Security Fixes, Bug fixes and Release notes](0741-each-changelog-version-has-features-security-fixes-bug-fixes-and-release-notes.md) * [0742 - Fixed build-and-test and package platform lists, and a tag ships what latest ships](0742-build-test-and-package-platform-lists-and-a-tag-ships-what-latest-ships.md) * [0743 - Analyses, then build and test jobs, then packages take runners](0743-analyses-then-build-and-test-then-packages-take-runners.md) +* [0745 - A first join into a restricted room stores the resident's verified co-signature](0745-a-first-restricted-join-stores-the-residents-verified-co-signature.md) +* [0746 - A threaded receipt may name any event whose relation chain reaches the thread root](0746-a-threaded-receipt-may-name-any-event-whose-relation-chain-reaches-the-root.md) +* [0747 - A remote member may leave through a server whose local members have all left](0747-a-remote-member-may-leave-through-a-server-whose-local-members-have-left.md) +* [0748 - Main serves a room's federation reads while the worker's reload of the room is pending](0748-main-serves-a-rooms-federation-reads-while-the-workers-reload-is-pending.md) +* [0749 - Outbound delivery never serves a remembered discovery failure](0749-outbound-delivery-never-serves-a-remembered-discovery-failure.md) +* [0750 - Complement's image also lists localhost as a cleartext appservice host](0750-complements-image-also-lists-localhost-as-a-cleartext-appservice-host.md) +* [0751 - The per-client outbound call cap is configurable and defaults to three](0751-the-per-client-outbound-call-cap-is-configurable-and-defaults-to-three.md) +* [0752 - build.py vm syncs the working tree over ssh and runs build-linux.sh](0752-build-py-vm-syncs-the-working-tree-over-ssh-and-runs-build-linux-sh.md) +* [0753 - Tests never write a fixed name in the shared temporary directory](0753-tests-never-write-a-fixed-name-in-the-shared-temporary-directory.md) +* [0754 - Agent skills are committed under .agent/skills and cite the binding docs](0754-agent-skills-are-committed-under-agent-skills-and-cite-the-binding-docs.md) +* [0755 - No std::format: the NetBSD libc++ has no `` header](0755-no-std-format-the-netbsd-libc-has-no-format-header.md) +* [0756 - Main serves every request for a room the federation worker has never loaded](0756-main-serves-every-request-for-a-room-the-worker-has-never-loaded.md) +* [0757 - Hardened BSD builds link -pie, and the ELF gate reads elftoolchain readelf](0757-hardened-bsd-builds-link-pie-and-the-elf-gate-reads-elftoolchain-readelf.md) +* [0758 - Every worker shard reloads every changed room, and main tracks each shard's copy](0758-every-worker-shard-reloads-every-changed-room-and-main-tracks-each-shards-copy.md) +* [0759 - Main builds make_leave and make_knock templates while the worker's reload of the room is pending](0759-main-builds-make-leave-and-make-knock-templates-while-the-workers-reload-is-pending.md) diff --git a/docs/architecture.md b/docs/architecture.md index 9a97b01a9..7289e71e8 100644 --- a/docs/architecture.md +++ b/docs/architecture.md @@ -518,18 +518,17 @@ This preserves ordering within a single room (all events for that room serialize `merovingian-fed-worker` runs a full `HomeserverRuntime` with its own `PersistentStore` copy, built at worker startup. Main remains authoritative for client-visible state; worker-local state is used only for room-scoped federation reads that main refreshes. -Room-scoped changes call `notify_room_changed(room_id)` after commit, including main's `membership_acceptor` once it has stored a `send_join`, `send_leave` or `send_knock` (since 0.12.59: the worker that relayed it answers the room's next `make_leave` from its own snapshot, and without the refresh built a template whose `auth_events` lacked the user's join). The proxy hashes the decoded room ID with FNV-1a modulo `federation.worker.shards`, sends a best-effort `room_sync` IPC frame to the owning shard, and the worker reloads that room with `database::reload_room()`. The reload covers the room, memberships, invites, events, state, the room's state groups with their entries and its events' state-group mappings, and event relation tables; it is not a full-store rehydrate. The state groups matter: federation reads (`timestamp_to_event`, worker-served `/backfill` and `/get_missing_events`) judge each event on the group recorded for it and fail closed without one, so a reload that left them behind made every event created after the worker started look invisible (404, or sent redacted). The reload is two steps (F-27, ADR-0360): `database::load_room_snapshot` reads the database with no lock held, and `database::apply_room_snapshot` swaps the room's rows in memory. In the worker `federation_worker::StoreReadGuard` puts one `std::shared_mutex` between them and the request threads: every provider that reads the store (ACL, room version, state, state_ids, event_auth, get_missing_events, backfill, timestamp_to_event, the local make_leave/make_knock template) holds it shared for its own call, and the swap holds it exclusively, so a reader sees a room before or after a reload, never during it. A relay transaction does not hold `runtime.mutex` across its round trips (`handle_federation_http_request` releases it before dispatch), so that mutex excludes nothing here. Main's providers take `runtime.mutex` themselves (`main_store_read_lock`, C11, ADR-0711), because every writer of main's store holds it. `GET /hierarchy` is not answered by a worker: it reads child rooms any shard may own, and a worker's copy of a room another shard owns is its start-up snapshot, so the proxy lets main answer it (`federation_request_should_bypass_worker`, C47, ADR-0712). Hydration and room reloads call `database::reconstruct_event_relations()` so stored `prev_event_ids`, `auth_event_ids`, and signatures are rebuilt from their normalized tables. The worker opens no database, on either backend (C44, [ADR-0733](adr/0733-the-federation-worker-opens-no-database-main-sends-it-what-it-reads.md)): a SHARED lock it held on main's SQLite file made every write in main wait its busy timeout and fail. Its configuration copy is switched to `database.backend=memory` before Landlock and the runtime start, and its store comes from main over the IPC channel. At start it asks for the room list (`room_list_request`, paged) and then each room (`room_snapshot_request`, paged), one at a time, and builds its store from the snapshots before hydration (`RuntimeStartOptions::fill_store`); the channel's request handler waits behind a gate until the runtime exists. Main answers on its snapshot pool (`WorkerSnapshotService`), never under `runtime.mutex`: it reads the room in one read transaction with `database::load_room_snapshot`, closes it, and only then sends the first page, so a worker that reads slowly holds nothing in main's database. Main reads a room for a worker only under a permit it issued: the listed rooms once each at start, and a `room_sync` generation once ([ADR-0735](adr/0735-a-worker-may-have-main-read-a-room-only-under-a-permit-main-issued-and-only-once.md)). A key-cache miss asks main for one server's public key rows (`server_keys_request`), answered from main's in-memory table. A frame that cannot be written ends the channel ([ADR-0734](adr/0734-a-failed-ipc-write-ends-the-channel.md)). -Room-scoped changes call `notify_room_changed(room_id)` after commit, including main's `membership_acceptor` once it has stored a `send_join`, `send_leave` or `send_knock` (since 0.12.59: the worker that relayed it answers the room's next `make_leave` from its own snapshot, and without the refresh built a template whose `auth_events` lacked the user's join). The proxy hashes the decoded room ID with FNV-1a modulo `federation.worker.shards`, sends a best-effort `room_sync` IPC frame to the owning shard, and the worker reloads that room by asking main for it at that generation (`StoreReadGuard::reload_room_from`). The reload covers the room, memberships, invites, events, state, the room's state groups with their entries and its events' state-group mappings, and event relation tables; it is not a full-store rehydrate. The state groups matter: federation reads (`timestamp_to_event`, worker-served `/backfill` and `/get_missing_events`) judge each event on the group recorded for it and fail closed without one, so a reload that left them behind made every event created after the worker started look invisible (404, or sent redacted). The reload is two steps (F-27, ADR-0360): the worker asks main for the snapshot with no lock held, and `database::apply_room_snapshot` swaps the room's rows in memory. In the worker `federation_worker::StoreReadGuard` puts one `std::shared_mutex` between them and the request threads: every provider that reads the store (ACL, room version, state, state_ids, event_auth, get_missing_events, backfill, timestamp_to_event, hierarchy, the local make_leave/make_knock template) holds it shared for its own call, and the swap holds it exclusively, so a reader sees a room before or after a reload, never during it. A relay transaction does not hold `runtime.mutex` across its round trips (`handle_federation_http_request` releases it before dispatch), so that mutex excludes nothing here. Main's providers are not wrapped: they read main's store under its own rules. Hydration and room reloads call `database::reconstruct_event_relations()` so stored `prev_event_ids`, `auth_event_ids`, and signatures are rebuilt from their normalized tables. +Room-scoped changes call `notify_room_changed(room_id)` after commit, including main's `membership_acceptor` once it has stored a `send_join`, `send_leave` or `send_knock` (since 0.12.59: the worker that relayed it answers the room's next `make_leave` from its own snapshot, and without the refresh built a template whose `auth_events` lacked the user's join). Main sends a best-effort `room_sync` IPC frame to every shard, not only the one the room's decoded ID hashes to (FNV-1a modulo `federation.worker.shards`): a `/send` goes whole to the shard its first PDU picks and may carry PDUs for rooms other shards own, so any shard may answer for any room (ADR-0758). Each worker reloads that room by asking main for it at that generation (`StoreReadGuard::reload_room_from`), and main keeps each shard's result apart: a request is judged on the copy held by the shard it is sent to. The reload covers the room, memberships, invites, events, state, the room's state groups with their entries and its events' state-group mappings, and event relation tables; it is not a full-store rehydrate. The state groups matter: federation reads (`timestamp_to_event`, worker-served `/backfill` and `/get_missing_events`) judge each event on the group recorded for it and fail closed without one, so a reload that left them behind made every event created after the worker started look invisible (404, or sent redacted). The reload is two steps (F-27, ADR-0360): the worker asks main for the snapshot with no lock held, and `database::apply_room_snapshot` swaps the room's rows in memory. In the worker `federation_worker::StoreReadGuard` puts one `std::shared_mutex` between them and the request threads: every provider that reads the store (ACL, room version, state, state_ids, event_auth, get_missing_events, backfill, timestamp_to_event, hierarchy, the local make_leave/make_knock template) holds it shared for its own call, and the swap holds it exclusively, so a reader sees a room before or after a reload, never during it. A relay transaction does not hold `runtime.mutex` across its round trips (`handle_federation_http_request` releases it before dispatch), so that mutex excludes nothing here. Main's providers take `runtime.mutex` themselves (`main_store_read_lock`, C11, ADR-0711), because every writer of main's store holds it. `GET /hierarchy` is not answered by a worker even though its provider is wrapped: it reads child rooms any shard may own, and the proxy judges only the room in the path, so a child room's pending or failed reload would go unseen; main answers it (`federation_request_should_bypass_worker`, C47, ADR-0712, ADR-0758). Hydration and room reloads call `database::reconstruct_event_relations()` so stored `prev_event_ids`, `auth_event_ids`, and signatures are rebuilt from their normalized tables. Shard selection must use the same decoded room ID string on both sides. Writer-side notifications use the internal room ID, while federation paths can percent-encode Matrix IDs; `federation_worker_room_id_from_request()` decodes path components before hashing. Routes without a room ID, including `GET /_matrix/federation/v1/query/directory` and `GET /_matrix/federation/v1/event/{eventId}`, cannot be sharded by room from the request path alone. Verified EDU-only `PUT /_matrix/federation/v1/send/{txnId}` transactions are handled in main rather than routed to shard 0, because they ultimately call main's `edu_sink` and include E2EE to-device key shares. -A room whose last `room_sync` reload failed is served by main until a reload succeeds (`federation_proxy.room_sync_failed_fallback`), so one room's transactions can run in either process. Main's dedupe ring is therefore the record of completed transactions: the proxy records each `/send` the worker answers with 200 there, and hands any `/send` already in it to main's handler, which answers the retry as a duplicate (ADR-0160). Main's read for a reload reads the room's tables in one read transaction, so the worker never serves a room in a state main never committed (ADR-0161). +A room whose last `room_sync` reload failed is served by main until a reload succeeds (`federation_proxy.room_sync_failed_fallback`), so one room's transactions can run in either process. While a reload is still pending, the room's reads (`state`, `state_ids`, `backfill`, `get_missing_events`, `event_auth`, `timestamp_to_event`, and the `make_leave` and `make_knock` templates) are served by main too (`federation_proxy.room_sync_pending_main`), and its writes go to the worker, so a remote server is never told an event this server has just sent does not exist, and a leave or knock template names an invite this server has just sent (ADR-0748, ADR-0759). A room the worker has never loaded (one created after it started, or first changed while its channel was down, until its first reload succeeds, and one its start-up copy left out, which the worker reports) is served by main in full, writes included, so the worker never judges an event under a room version it guessed; a `/send` naming any such room is served by main whole (ADR-0756). Main's dedupe ring is therefore the record of completed transactions: the proxy records each `/send` the worker answers with 200 there, and hands any `/send` already in it to main's handler, which answers the retry as a duplicate (ADR-0160). Main's read for a reload reads the room's tables in one read transaction, so the worker never serves a room in a state main never committed (ADR-0161). Inbound signing-key resolution separates network-free verification-key reads from complete peer discovery (FED-10). `cached_pdu_key_lookup` reads owned local or fresh persistent remote keys; it also supports known-peer key rotation without changing discovery/trust state. Core inbound discovery-capable sender and origin resolution uses the source-address rate budget, per-runtime in-flight cap and bounded server/key failure cache. Parallel PDU sender pre-resolution and authorization fallbacks share that wrapper; per-transaction parallelism is not an admission grant. Admission guards release only owned reservations, and task scheduling permits/callables remain valid through asynchronous failure and unwind. These are per-runtime controls, not an aggregate cross-shard limiter. Local sender/authorising-key denial returns retryable 503 without charging or resetting relay trust or inserting accepted-transaction dedup; actual failures retain penalties even in mixed transactions. EDUs wait for retry, while already-ingested PDUs rely on event-ID idempotency. Main IPC PDU/membership/invite re-verification delegates to the same admitted key-only helper. Main relay cache misses share its common unknown-source bucket because these frames have no main-validated client address; worker-supplied source hints never choose the bucket. Local denial defers before acceptance/mutation. Backfilled-event, snapshot and auth-chain verification now share the admitted key-only helper. `/send_join` state verification uses `resolve_join_state_key`: the same in-flight slots and failure cache, but it waits for a slot instead of being refused and is not charged a per-source rate, because a refused key would drop a member event from the joined room's state (ADR-0155). Request-local recovery provenance carries local denial to the existing main-overload result instead of ordinary missing state; further recovery calls stop. This slice passed fresh full-suite verification and independent source-only review. Additional sticky-denial/partial-progress tests and end-to-end worker socket-HTTP overload coverage remain before whole-finding closure; review is not transport runtime evidence (ADR-0135). No persisted or IPC format changes. See [ADR-0132](adr/0132-separate-cached-verification-keys-from-budgeted-discovery.md) and [ADR-0133](adr/0133-preserve-local-key-admission-denial-separately-from-remote-failure.md). -Workers relay operations that must observe main's current global state or must commit to main exactly once. Relayed hooks include `pdu_sink`, `edu_sink`, `membership_acceptor`, `invite_handler`, `one_time_keys_claim_provider`, `user_devices_provider`, `device_keys_query_provider`, `profile_query_provider`, `directory_query_provider` (`GET /query/directory` has no room ID, so a worker never learns of an alias created after it started), `public_rooms_provider` (`public_rooms_ingest`: the directory names no room and reads appservice registrations, ADR-0461), `event_query_provider`, and the `make_join` case of `membership_template_provider` (`membership_template_ingest`: a restricted room's allow conditions name rooms another shard owns, R04, ADR-0184). This keeps inbound PDUs/EDUs, federated joins/leaves/knocks, invites, one-time-key claims, device/profile reads, and event-by-ID reads consistent with the store that client `/sync` uses. Main does not trust the worker's room ACL or policy decision either (F-21, F-24, ADR-0358, ADR-0359): the relay handlers read the origin's federation policy rule and the room's current `m.room.server_acl` from main's own store, because a worker's copy of both can be stale (only the shard that owns a room is told when it changes, and `/send` is routed by a room ID the sender chooses). Main does not trust the worker's signature check on a relayed PDU: `handle_pdu_ingest_request`, `handle_membership_ingest_request` and `handle_invite_ingest_request` re-verify the sender server's signature with main's own key-only cache/admission helper (`resolve_verification_key`), before taking `runtime.mutex`, and build the envelope from the verified event (ADR-0071). +Workers relay operations that must observe main's current global state or must commit to main exactly once. Relayed hooks include `pdu_sink`, `edu_sink`, `membership_acceptor`, `invite_handler`, `one_time_keys_claim_provider`, `user_devices_provider`, `device_keys_query_provider`, `profile_query_provider`, `directory_query_provider` (`GET /query/directory` has no room ID, so a worker never learns of an alias created after it started), `public_rooms_provider` (`public_rooms_ingest`: the directory names no room and reads appservice registrations, ADR-0461), `event_query_provider`, and the `make_join` case of `membership_template_provider` (`membership_template_ingest`: a restricted room's allow conditions name rooms another shard owns, R04, ADR-0184). This keeps inbound PDUs/EDUs, federated joins/leaves/knocks, invites, one-time-key claims, device/profile reads, and event-by-ID reads consistent with the store that client `/sync` uses. Main does not trust the worker's room ACL or policy decision either (F-21, F-24, ADR-0358, ADR-0359): the relay handlers read the origin's federation policy rule and the room's current `m.room.server_acl` from main's own store, because a worker's copy of both can be stale (a reload lands after main's commit, and `/send` is routed by a room ID the sender chooses). Main does not trust the worker's signature check on a relayed PDU: `handle_pdu_ingest_request`, `handle_membership_ingest_request` and `handle_invite_ingest_request` re-verify the sender server's signature with main's own key-only cache/admission helper (`resolve_verification_key`), before taking `runtime.mutex`, and build the envelope from the verified event (ADR-0071). Worker request execution uses two pools. `local_pool` (`federation.worker.threads`) handles endpoints answered from the worker room snapshot: `make_leave`, `make_knock`, `backfill`, `state`, `state_ids`, `event_auth`, `get_missing_events`, `timestamp_to_event`, and the no-op EDU route. `relay_pool` (`federation.worker.relay_threads`) handles endpoints that can block on main or outbound HTTP: PDU-bearing transactions, `make_join`, `send_join`, `send_leave`, `send_knock`, invites, key/profile/directory/device relays, event-by-ID, and `outbound_http_request` (whose reply main uses only for the four signature-verified backfill fetches; every other outbound call runs on main's own client, ADR-0349). Every room-scoped read (`state`, `state_ids`, `backfill`, `get_missing_events` on the worker snapshot; event-by-ID relayed to main together with the verified origin) enforces the origin-in-room gate (FED-2), so it applies on whichever process answers. The split is driven by `federation::federation_endpoint_requires_main_relay(FederationEndpoint)` so slow relays cannot starve local room reads of worker threads. diff --git a/docs/coding-rules.md b/docs/coding-rules.md index 51bc6d7dd..381eac70c 100644 --- a/docs/coding-rules.md +++ b/docs/coding-rules.md @@ -184,6 +184,10 @@ See the [header guidance](../include/merovingian/AGENTS.md) and `tests/support/JoiningThreads.hpp` as appropriate. The supported BSD standard libraries do not all provide these facilities. See [ADR-0167](adr/0167-no-jthread-or-stop-token-use-std-thread-and-stop-flag.md). +- Do not include `` (`std::format`, `std::format_to`, `std::print`) in production + code or tests: the NetBSD libc++ does not ship it. Build strings with `std::to_string`, + `std::to_chars` and concatenation. See + [ADR-0755](adr/0755-no-std-format-the-netbsd-libc-has-no-format-header.md). - Use `core::PortableSemaphore`, not `std::counting_semaphore`. Use `core::run_bounded_parallel` for work that completes within the call. For work that outlives the call, take a `core::SemaphorePermit` immediately after diff --git a/docs/dev-environment.md b/docs/dev-environment.md index 2b804958d..3799ec271 100644 --- a/docs/dev-environment.md +++ b/docs/dev-environment.md @@ -143,6 +143,158 @@ client headers, Catch2, clang tooling, cppcheck, and a current Meson/Ninja virtual environment. The virtual environment is used because the distribution Meson in Ubuntu 24.04 is too old to configure the repo's C++26 build. +### Building on a VM + +`python build.py vm` builds and tests on a Linux or FreeBSD virtual machine (or +any such host) over ssh, from Windows or any other host with Python, Git and +OpenSSH. The VM's native file system avoids every NTFS workaround above, and a +build there does not compete with the editor for the Windows host's memory. +See [ADR-0752](adr/0752-build-py-vm-syncs-the-working-tree-over-ssh-and-runs-build-linux-sh.md). + +Each run: + +1. Lists the files to send: every tracked file as it is on disk (uncommitted + edits included, with Git's executable bit), and every untracked file Git does + not ignore, except under a top-level name starting with `build` or `.`. +2. Reads the manifest the VM kept from the last run and sends, as one tar stream + over ssh, only the files whose mode, size or modification time differ. Files + the last manifest listed and this one does not are deleted on the VM before + the archive is unpacked, so a file that became a directory, or the reverse, + never blocks it. If another run changed the manifest between this run reading + it and taking the lock, the VM touches nothing and `build.py vm` reads the + manifest again and resends (three attempts). A sync cut off part-way leaves + the manifest listing every path it had and every path it was sending, with + no metadata, so the next run resends each file and still deletes what it no + longer has. +3. Takes the ssh user's lock (`~/merovingian-vm/.lock`, through `python3`'s + `fcntl.flock`, since the BSDs have no `flock` command), so a second run as + that user, from any checkout, waits ("another run holds the VM lock"), then + runs the host's build script in + `~/merovingian-vm/-`, one directory per local + checkout or worktree: `scripts/build-linux.sh` where `uname -s` says `Linux`, + `scripts/build-bsd.sh` where it says `FreeBSD`, `OpenBSD`, `NetBSD` or + `DragonFly`; any other system is refused. The lock does not cover other users + on the VM: use one VM user for every run, since two suites at once time out + unrelated tests. +4. Copies `meson-logs/testlog.txt` and `testlog.json` back to the same build + directory locally, `build-vm/meson-logs/` by default (`build-vm-/` for + a named VM), and prints the `Ok:`, `Fail:` and `Timeout:` counts. It deletes + the local and the VM's previous logs before anything is unpacked, so a run + that fails before testing leaves no log, never an old one. Each run copies + its logs, still under the lock, into a directory of its own on the VM + (`.vm-sync/run-/`) and fetches them from there, so a run that starts + the moment this one ends cannot replace them first. If the logs cannot be + copied back (ssh refuses the key, say), it says so, names where they are on + the VM, and exits non-zero even when the build passed. + +The exit status is the build's. Because the VM-side build directory and the +local one share a name, `build-vm/meson-logs/testlog.txt` is read the same way +as any other build's log. + +One-time setup: + +1. Create a key for the build only and add its public half to the VM user's + `~/.ssh/authorized_keys`: + + ```sh + ssh-keygen -t ed25519 -N '' -C merovingian-vm-build -f ~/.ssh/merovingian_vm_ed25519 + ``` + +2. Connect once with `ssh` and check the host key fingerprint against the VM's + console (`ssh-keygen -lf /etc/ssh/ssh_host_ed25519_key.pub`). `build.py vm` + runs ssh with `StrictHostKeyChecking=yes` and `BatchMode=yes`: it never + accepts an unknown or changed host key and never prompts. +3. Write the settings to `~/.config/merovingian/vm.ini` (or the file + `MEROVINGIAN_VM_CONFIG` names). The file lives outside the repository, so + every worktree reads it and no address is committed: + + ```ini + [vm] + host = 192.0.2.10 + user = builder + identity = ~/.ssh/merovingian_vm_ed25519 + # port = 22 + # jobs = 4 + ``` + + `--host`, `--user`, `--port`, `--identity` and `--jobs` override the file, and + `MEROVINGIAN_VM_HOST`, `MEROVINGIAN_VM_USER`, `MEROVINGIAN_VM_PORT`, + `MEROVINGIAN_VM_IDENTITY` and `MEROVINGIAN_VM_JOBS` override it when no flag + is given. An unknown key in `[vm]` is an error. + + More VMs go in sections named `[vm.]` (lower-case letters, digits, `-` + and `_`), chosen with `--vm ` or `MEROVINGIAN_VM_NAME`. A named VM reads + only its own section, never `[vm]`, and a name with no section is an error. + Its default build directory is `build-vm-`, so each VM's logs stay + apart locally: + + ```ini + [vm.freebsd] + host = 192.0.2.11 + user = builder + identity = ~/.ssh/merovingian_bsd_ed25519 + ``` + +4. Install the packages on the VM before the first sync, as root or a user who + can use `sudo` or `doas`. `build.py vm` takes its lock through `python3`, so + a VM without it cannot be synced yet: copy the setup script over on its own, + run it there, then sync once to check. `scp` and `ssh` do not read `vm.ini`, + so name the key created above: + + ```sh + scp -i ~/.ssh/merovingian_vm_ed25519 scripts/remote-host-setup.sh builder@192.0.2.10: + ssh -i ~/.ssh/merovingian_vm_ed25519 -t builder@192.0.2.10 sh remote-host-setup.sh + python build.py vm --exec true + ``` + + On FreeBSD, `pkg install python3` from the console does as well for the + first sync; the setup script can then run from the synced checkout. + + `scripts/remote-host-setup.sh` prepares a remote build host, the VM or + another machine (`scripts/wsl-setup.sh` is for WSL only). It accepts Ubuntu, + Debian, distributions whose `ID_LIKE` names either, such as Linux Mint, and + FreeBSD. On the Debian family it installs with `apt-get` the compiler, linker + and every library `build-linux.sh` checks for, including libcurl, libpng and + libjpeg-turbo, and installs Meson 1.11 or later and Ninja in a virtual + environment linked into `~/.local/bin`, because Ubuntu 24.04's Meson is too + old for the C++26 build. `build.py vm` puts `~/.local/bin` first on `PATH` + for the build. On FreeBSD it installs with `pkg` the packages the FreeBSD CI + job installs, Meson and Ninja among them, plus `python3`. `--dry-run` prints + the commands; re-running it is safe. + +Usage: + +```sh +# Full build and test suite in build-vm +python build.py vm + +# The same on the [vm.freebsd] VM, in build-vm-freebsd +python build.py vm --vm freebsd + +# Other build-linux.sh options work as they do for linux +python build.py vm --builddir build-vm-asan --profile sanitizer +python build.py vm --clean --compile-only + +# Sync, then run one command in the VM checkout instead of building; --exec comes last +python build.py vm --exec ./build-vm/tests/merovingian-unit-tests "[push]" +``` + +`--exec` runs with `MEROVINGIAN_TEST_DISABLE_HARDENING=1`, as `meson test` does, +under the same lock. `--dry-run` reads the VM's manifest and prints the number of +files it would send and the script the VM would run, without sending anything. + +Without `--jobs`, a run compiles and tests with one job per 1.5 GiB of the VM's +memory (`/proc/meminfo` on Linux, `sysctl hw.physmem` on the BSDs), never more +than its CPUs: a C++26 translation unit here can take over +1 GiB to compile, and Meson's default of one job per CPU runs a small VM out of +memory. Give the VM memory rather than lowering `--jobs` where you can. + +The `--builddir` must be a relative path whose first part starts with `build`, +so the sync never sends it. PostgreSQL integration tests run only when +`MEROVINGIAN_TEST_POSTGRESQL_URI` is set on the VM, as on any other host. If +ssh drops mid-run, the build keeps running on the VM and holds the lock until it +ends; the next run waits for it. + Project-owned headers should use quoted includes. Use the rewrite helper to convert angle-bracket Merovingian includes in source trees: @@ -204,6 +356,9 @@ python build.py bsd # WSL (Windows Subsystem for Linux) python build.py wsl +# A Linux or FreeBSD VM over ssh (see "Building on a VM") +python build.py vm + # AddressSanitizer + UBSan through WSL python build.py wsl --builddir build-asan --buildtype debug --sanitize address,undefined @@ -277,7 +432,10 @@ sh scripts/build-linux.sh ``` The default build directory is `build`, with `CC=clang` and `CXX=clang++`. -Override these when using another compiler: +`--jobs ` caps both the compile jobs and the tests run at once (`build.py +linux --jobs` and `build.py vm` pass it on; `build-bsd.sh` and `build.py bsd` +take it too); without it Meson picks its own. +Override the compiler when using another one: ```sh sh scripts/build-linux.sh --builddir build-dev --cc clang-22 --cxx clang++-22 diff --git a/docs/hardening.md b/docs/hardening.md index b8d5a1a4e..7e08708c3 100644 --- a/docs/hardening.md +++ b/docs/hardening.md @@ -26,7 +26,7 @@ binary or in the service-manager configuration that ships with the packages. | FORTIFY_SOURCE | `hardening_compile_flags` (`-D_FORTIFY_SOURCE=3`) when `optimization != '0'` | Checked libc wrappers. | | Hidden visibility | `hardening_compile_flags` (`-fvisibility=hidden`) | Limits ELF symbol exposure. | | Trivial auto-var init | `hardening_compile_flags` (`-ftrivial-auto-var-init=zero`) | Uninitialised locals are zeroed. | -| Position-independent executable | `hardening_compile_flags` (`-fPIE`) and link args (`-pie`) | Enables ASLR. | +| Position-independent executable | `hardening_compile_flags` (`-fPIE`) and `hardening_link_flags` (`-pie`) on Linux, FreeBSD and OpenBSD | Enables ASLR. FreeBSD's clang does not link a PIE without `-pie` (ADR-0757). | | No-exec stack | `hardening_link_flags` (`-Wl,-z,noexecstack`) | ELF GNU_STACK note is non-executable. | | RELRO + BIND_NOW | `hardening_link_flags` (`-Wl,-z,relro -Wl,-z,now`) on Linux, FreeBSD and OpenBSD | Full RELRO; dynamic relocations resolved at load time. | | Static PIE fallback | `scripts/build-static-linux.sh` (`-static-pie`) | Fully static, position-independent musl build for Linux. | @@ -603,9 +603,10 @@ separate process on the same host: afterwards inherit Landlock and seccomp and cannot open the master key path. A kernel without seccomp or Landlock makes those scenarios SKIP with a message rather than pass. -* **Worker-specific seccomp + runtime hardening** (#319): the worker applies - `PR_SET_NO_NEW_PRIVS`, drops capabilities, sets resource limits, and installs - a stricter seccomp-bpf filter (on top of the inherited server filter) that +* **Worker-specific seccomp + runtime hardening** (#319): the worker applies, + in order and each fail-closed, the core-dump policy (`RLIMIT_CORE` 0 and + `PR_SET_DUMPABLE` 0), `PR_SET_NO_NEW_PRIVS` and the capability-bounding-set + drop, then installs a stricter seccomp-bpf filter (on top of the inherited server filter) that denies `execve`/`execveat`/spawn-oriented `clone` — the worker never execs or spawns, so those syscalls are unreachable in normal operation and kill the worker on attempted abuse. This sequence (`apply_worker_hardening()` in @@ -620,8 +621,11 @@ separate process on the same host: controls, and the worker refuses to start. The server's first spawn then fails its IPC handshake and the server exits with status 80, rather than ever accepting unsandboxed federation traffic. An operator who accepts the - risk can still run the worker there by setting - `federation.worker.apply_hardening=false` explicitly, or run a + risk can still run the worker there by setting both + `federation.worker.apply_hardening=false` and + `federation.worker.allow_without_landlock=true`: the worker applies Landlock + first, whatever `apply_hardening` says, and refuses to start without it + unless the second key is set. Or run a non-federating server, which starts no worker, with `security.federation.enabled=false` (ADR-0377). * **ASan exit-time leak check disabled in the worker (#319):** the worker @@ -937,7 +941,9 @@ Hardening is exercised by automated tests: binary's ELF headers (PIE, `PT_GNU_RELRO`, `DT_BIND_NOW`, non-executable `PT_GNU_STACK`) and fails closed if any is missing — the CI-time counterpart to the runtime ELF probe above, needing no live process so it - runs the same whether the job is root, non-root, or containerized. Run + runs the same whether the job is root, non-root, or containerized. It reads + both GNU binutils' and the BSDs' elftoolchain `readelf` output, which print + the dynamic flags differently (ADR-0757). Run against every shipped binary in the `ubuntu-hardened-listener-coverage` CI job and per-platform in `.github/workflows/release.yml` via `scripts/collect-release-evidence.sh`. diff --git a/docs/http-transport.md b/docs/http-transport.md index a083e4ab9..5a19b8e01 100644 --- a/docs/http-transport.md +++ b/docs/http-transport.md @@ -218,6 +218,7 @@ Configuration (`server.http.*`, restart required — read when listeners start): | `server.http.max_connections_per_ip` | `256` | Open connections one client key may hold, 1..65535 (see below). | | `server.http.ipv6_client_prefix_length` | `64` | Prefix length IPv6 clients are grouped by, 1..128. | | `server.http.request_threads` | `32` | Threads in the main request pool, 4..256. One client address may hold at most a quarter of them; client-triggered outbound proxying at most half (ADR-0079). | +| `server.http.client_outbound_max_per_client` | `3` | Client-triggered outbound calls one client address may have in flight, 1..128, clamped to the global cap (ADR-0751). | ### Per-client connection cap (ADR-0072) @@ -1143,7 +1144,8 @@ Rules: held by `HomeserverRuntime::client_outbound_budget`. Global cap half of the main request pool (`server.http.request_threads / 2`, so 16 by default; the other half of the pool always serves requests that never leave this - server); per-client cap 1, keyed by + server); per-client cap `server.http.client_outbound_max_per_client`, 3 by default + and never above the global cap (ADR-0751), keyed by `rate_limit_client_key` (the same key the rate limiter uses, so `trusted_proxies` and the IPv6 prefix grouping apply). Authenticated callers count too. Local `publicRooms`, local aliases and local media never take a slot. @@ -1172,13 +1174,14 @@ Rules: - **Lock.** The slot is taken while the runtime lock is held and before it is released for the call (`RuntimeLockRelease`); it is dropped on every exit path by its destructor. -- **Constants, not configuration.** The caps and deadlines live in +- **What is configurable.** The caps and deadlines live in `ClientOutboundProxyPolicy` (`include/merovingian/homeserver/ClientOutboundProxyPolicy.hpp`), - derived from the main request pool size (`client_outbound_proxy_policy_for_pool` in - `ClientOutboundProxy.hpp`, - applied by `start_runtime` from `server.http.request_threads`, ADR-0077). A - separate configuration key would let an operator raise the caps above the - pool size, which is the defect. Tests lower the deadline through + built by `client_outbound_proxy_policy_for_pool` in `ClientOutboundProxy.hpp` and applied + by `start_runtime`. The global cap is derived from `server.http.request_threads` + (ADR-0077) and has no key of its own: a key that could raise it above half the pool would + be the defect. The per-client cap is `server.http.client_outbound_max_per_client`, which + the policy clamps to the global cap, so it can share out that budget but never enlarge + it (ADR-0751). The deadlines are constants. Tests lower them through `runtime.client_outbound_proxy_policy`. - **Appservice query hooks.** `GET /profile/{userId}` and `GET /directory/room/{roomAlias}` take one slot for the whole loop over the diff --git a/docs/media-repository.md b/docs/media-repository.md index 2e614c110..c1d5f8d39 100644 --- a/docs/media-repository.md +++ b/docs/media-repository.md @@ -131,7 +131,7 @@ current in-process runtime path. requests for the same remote file share one fetch. Every step (discovery, the federation media request, the legacy fallback, a `Location` redirect follow) draws from one 30 s deadline (audit HTTP-2). Without a media fetch pool (tests, embedded callers) a fetch runs inline under the - ADR-0079 client-outbound budget: 1 per client address. + ADR-0079 client-outbound budget: 3 per client address by default (ADR-0751). - The origin of a remote fetch must be a Matrix server name, checked by `federation::server_name_is_valid` (appendices, "Server Name"). A name without a dot, such as `hs2` or `localhost`, is valid; until 0.12.57 the media check diff --git a/docs/platform-support.md b/docs/platform-support.md index d1de0fd02..e1483b695 100644 --- a/docs/platform-support.md +++ b/docs/platform-support.md @@ -194,7 +194,7 @@ release-blocking. - **HardenedBSD** — uses the FreeBSD `pkg` toolchain and `scripts/build-bsd.sh`. - **Other Linux distributions** — via the portable static tarball or a from-source build. -- **Windows** — development only, through WSL (`python build.py wsl`). Not a server deployment target. +- **Windows** — development only, through a Linux or FreeBSD VM over ssh (`python build.py vm`, `--vm ` for a second VM) or WSL (`python build.py wsl`). Not a server deployment target. - **macOS** — not a supported target. ## Per-platform hardening posture diff --git a/docs/security-coding-rules.md b/docs/security-coding-rules.md index 99e7f5463..924223271 100644 --- a/docs/security-coding-rules.md +++ b/docs/security-coding-rules.md @@ -601,7 +601,7 @@ quickly finding everything a given `AGENTS.md` file contributed. read the room's current `m.room.server_acl` from main's store for the transport origin and the verified sender's server. A new relay does the same. Why: a worker's store holds no policy rules (since 1.0.1 on either backend, ADR-0733), and a room's ACL only - after `room_sync` reaches the shard that owns it; `/send` is routed by a room ID the sender chooses. Until 0.12.70 a denied + after a shard's `room_sync` reload lands (every shard reloads every changed room since 1.0.2, ADR-0758); `/send` is routed by a room ID the sender chooses. Until 0.12.70 a denied origin or server kept federating (F-21, F-24; ADR-0358, ADR-0359). Source: `src/federation/AGENTS.md`, `src/federation_worker/AGENTS.md`. diff --git a/docs/testing-standards.md b/docs/testing-standards.md index e999143ba..6f96dfeea 100644 --- a/docs/testing-standards.md +++ b/docs/testing-standards.md @@ -23,6 +23,10 @@ This rule applies to: - Avoid implementation-detail-oriented naming. - Prefer one behavioral assertion group per test. - Security-sensitive code requires negative-path tests. +- A scratch file or directory a test creates gets a unique name from `merovingian::tests::unique_temporary_path()` + (`tests/support/TempDirectory.hpp`), never a fixed name in the shared temporary directory, and a test that writes + one checks the write succeeded before using it: another account's leftover file cannot be replaced, and a + concurrent run would overwrite it (ADR-0753). ## Unit or integration @@ -321,8 +325,8 @@ would pass with the test silently not run. Each CI job that runs Catch2 therefor name mapped to the groups it combines). `test` is the scenario name without the `Scenario: ` prefix; `message` is an optional substring the SKIP message must contain; `reason` says why this runner skips it. - `` is a key of the `jobs` object in `tests/expected_skips.json`. The build scripts take - `--skip-job ` (`build-linux.sh`, `build-bsd.sh`); `build-wsl.sh` has no `--skip-job`, so WSL runs are not - skip-gated. They set + `--skip-job ` (`build-linux.sh`, `build-bsd.sh`); `build-wsl.sh` has no `--skip-job`, and `build.py vm` passes none, so + WSL and VM runs are not skip-gated. They set `MEROVINGIAN_CATCH_REPORT_DIR`, so every test binary also writes an XML report there, and run `scripts/check_expected_skips.py` after the suite. A skip that is not listed for the job fails the job and is printed with the test name, message and source line. No report, a truncated report or a bad list also fails. diff --git a/docs/threat-model.md b/docs/threat-model.md index 1188d391e..01d155871 100644 --- a/docs/threat-model.md +++ b/docs/threat-model.md @@ -2359,7 +2359,8 @@ membership. No whole-finding or release acceptance is claimed. See proposed call. The answer is `404 M_NOT_FOUND`. - HTTP-2: every client-triggered outbound call (the routes above plus remote media when enabled) takes a slot in `http::InFlightBudget` before it leaves: 4 in flight overall (half - the pool) and 1 per client address (`rate_limit_client_key`, so `trusted_proxies` applies). + the pool) and 3 per client address by default (`server.http.client_outbound_max_per_client`, + ADR-0751; keyed by `rate_limit_client_key`, so `trusted_proxies` applies). Over either cap the answer is `429 M_LIMIT_EXCEEDED` with `retry_after_ms` 1000, at once. The total deadline, discovery included, is 10 s for directory lookups and 30 s for media, and the federation-worker round trip may exceed it by at most 2 s. The slot is an RAII object @@ -2380,17 +2381,21 @@ membership. No whole-finding or release acceptance is claimed. See proposed local part of the answer is still returned. A join by remote alias takes a slot (`429` when none is free) and runs under the 10 s directory deadline. Tests: `[f-17]` in `tests/integration/test_client_outbound_proxy_flow.cpp`. -- **Residual risk:** an attacker holding 4 slots costs the pool 4 of 8 threads for up to 10 s - (30 s for media) at a time, and each address needs to be distinct to hold more than one, so a - botnet can keep the 4 slots busy and lock legitimate remote lookups out with 429; the other 4 - threads stay available to everything that does not leave this server. Threads are still +- **Residual risk:** an attacker holding every slot of the global cap (half the pool) costs the + pool that half for up to 10 s (30 s for media) at a time. One address holds up to the + per-client cap, 3 by default (ADR-0751), so filling a global cap of 16 takes 6 addresses + rather than 16; an operator who sets the per-client cap equal to the global cap lets one + address fill it. A botnet can keep the slots busy and lock legitimate remote lookups out + with 429; the other half of the pool stays available to everything that does not leave + this server. Threads are still blocked while a call is in flight: the fix bounds how many, and a separate pool or an asynchronous proxy path was rejected for now. Outbound calls to operator-configured destinations that an unauthenticated client can trigger (identity-server `requestToken`) are not under the budget; the appservice `query_user` and `query_room_alias` hooks of `GET /profile` and `GET /directory/room` are, one slot per request (ADR-0417). DNS lookups inside server discovery are bounded by the resolver, not by the deadline. Clients behind a - shared address that is not a configured trusted proxy share one per-client slot. + shared address that is not a configured trusted proxy share that address's per-client slots + (3 by default). ### Outbound destination agreement (OUT-1, OUT-2) @@ -2708,7 +2713,8 @@ nothing, all from trusting the worker's view or the hook's transport. `pdu_ingest`, `membership_ingest` or `invite_ingest` frame is the worker's claim; a compromised worker can name an allowed origin, but not change what the signed event says (below). - **A /send routed to a shard with a stale snapshot passed the room ACL (F-24, ADR-0359).** The shard is chosen by the - first PDU's room ID, which the sender picks; only the owning shard learns of later state. A denied server with a joined + first PDU's room ID, which the sender picks; until 1.0.2 only the owning shard learned of later state (every shard + reloads every changed room since, ADR-0758, and a shard's copy still lags main's until its reload lands). A denied server with a joined user picked a shard whose snapshot lacked the ACL, and main's relay handler did not check. Mitigated: main reads the room's current `m.room.server_acl` for the transport origin and for the verified sender's server on every relayed PDU, membership event and invite; a refused PDU is reported with an `error` entry (`PduIngestionStatus::rejected_acl`). Residual: diff --git a/docs/user-manual.md b/docs/user-manual.md index ec035dcc6..dd6d0ea8f 100644 --- a/docs/user-manual.md +++ b/docs/user-manual.md @@ -286,7 +286,7 @@ The unified build script configures, compiles, and tests in one step: ```sh sh scripts/setup-dev-env.sh # install toolchain and configure build dir -python build.py linux # or `python build.py wsl` on Windows +python build.py linux # on Windows: `python build.py vm` (Linux or FreeBSD VM) or `python build.py wsl` ``` After a successful build, the binaries are at: @@ -531,6 +531,7 @@ its own requests finishes, while other clients are served. | Key | Default | When to change | |---|---|---| | `server.http.request_threads` | `32` | Threads in the main request pool that serves every listener, 4..256. Raise for a busy server with many cores; each client address may use a quarter of them, and remote directory and media proxying at most half. | +| `server.http.client_outbound_max_per_client` | `3` | Remote calls one client address may have in flight at once (directory, profile and key lookups, remote media without the fetch pool, and federated joins, knocks and leaves), 1..128, never more than `request_threads / 2`. A call over it is answered `429 M_LIMIT_EXCEEDED` with `retry_after_ms`, except `/keys/query` and `/keys/claim`, which still answer locally and list the remote servers they could not ask in `failures`. Raise when many users share one address and no `server.trusted_proxies` entry tells them apart (ADR-0751). | | `server.http.keep_alive` | `true` | Set `false` to restore strict one-request-per-connection behaviour (e.g. in front of a proxy that pools upstream connections itself). | | `server.http.keep_alive_idle_seconds` | `15` | Idle window per kept-alive connection, seconds, 1..300. Raise for chatty API clients that re-use connections; lower to close idle connections sooner. | | `server.http.keep_alive_max_connections` | `256` | Process-wide cap on connections parked awaiting a next request, 1..4096. Beyond the cap the server answers `Connection: close`. A parked connection holds no worker thread, so this bounds open descriptors and memory and need not match `request_threads`; raise it to let more clients re-use their connections. | @@ -1250,14 +1251,16 @@ and direct rooms with `default_for_new_rooms`. Remote lookups that a client triggers (`GET`/`POST /publicRooms?server=` and a room-alias lookup for a remote alias) are not governed by -`remote_timeout` alone: they are limited to 4 in flight at once across all -clients and 1 per client address, and end after at most 10 s, +`remote_timeout` alone: they are limited to half of `server.http.request_threads` +in flight at once across all clients (16 with the default of 32) and 3 per client +address (`server.http.client_outbound_max_per_client`), and end after at most 10 s, whichever of that and `remote_timeout` is shorter. A request over the limit is answered `429 M_LIMIT_EXCEEDED` with `retry_after_ms` 1000 and a `Retry-After` -header; a lookup that runs out of time is answered `502`. These limits are -fixed, not configuration keys: they exist to keep half of the request -pool free for requests that never leave this server. Behind a reverse proxy set -`server.trusted_proxies`, or every client shares one address and so one slot. +header; a lookup that runs out of time is answered `502`. The overall limit is +half of `server.http.request_threads`, not a key of its own: it exists to keep half +of the request pool free for requests that never leave this server. Behind a reverse +proxy set `server.trusted_proxies`, or every client shares one address and so one +client's slots. #### Federation join/leave budget — `security.federation.join_*` diff --git a/docs/versioning.md b/docs/versioning.md index 0753c62f4..d1627a937 100644 --- a/docs/versioning.md +++ b/docs/versioning.md @@ -141,7 +141,7 @@ than the others. nothing is added to ([ADR-0740](adr/0740-the-changelog-starts-at-1-0-0-and-the-pre-release-entries-are-a-frozen-record.md)). - `tests/tooling/test_packages_workflow.py` enforces that every file in the table above carries the current `meson.build` version. Run it (or `python build.py - wsl`) after a bump to catch any location that was missed. + vm`) after a bump to catch any location that was missed. ## Release process diff --git a/include/merovingian/config/HttpTransportConfig.hpp b/include/merovingian/config/HttpTransportConfig.hpp index c2b5d9dd4..3f22f504d 100644 --- a/include/merovingian/config/HttpTransportConfig.hpp +++ b/include/merovingian/config/HttpTransportConfig.hpp @@ -40,6 +40,13 @@ namespace merovingian::config // proxying at most request_threads / 2 in total // (ADR-0079). Range 4..256; default 16; restart // required. +// client_outbound_max_per_client — of those outbound calls (remote +// directory, media, keys and federated joins, +// knocks and leaves), how many one client address +// may have in flight at once; more are answered +// 429 at once. Never more than the global cap of +// request_threads / 2. Range 1..128; default 3; +// restart required (ADR-0751). struct HttpTransportConfig final { bool keep_alive{true}; @@ -48,6 +55,7 @@ struct HttpTransportConfig final std::uint32_t max_connections_per_ip{256U}; std::uint8_t ipv6_client_prefix_length{64U}; std::uint32_t request_threads{32U}; + std::uint32_t client_outbound_max_per_client{3U}; std::uint32_t sync_threads{128U}; std::uint32_t sync_max_in_flight{512U}; std::uint32_t sync_max_per_user{4U}; diff --git a/include/merovingian/federation/DiscoveryCachePolicy.hpp b/include/merovingian/federation/DiscoveryCachePolicy.hpp index f3260e700..1f91c73dc 100644 --- a/include/merovingian/federation/DiscoveryCachePolicy.hpp +++ b/include/merovingian/federation/DiscoveryCachePolicy.hpp @@ -15,7 +15,8 @@ enum class DiscoveryCachePolicy // A fresh success only. A remembered failure is ignored and the network // asked again, so a client's retry after the destination recovers is not // refused for the rest of the lifetime: the federated join and the other - // outbound calls a client request triggers. + // outbound calls a client request triggers, and outbound delivery, which + // its own backoff paces (ADR-0749). successes_only, }; diff --git a/include/merovingian/federation/DispatchWorker.hpp b/include/merovingian/federation/DispatchWorker.hpp index 6c98223b4..5ae0161bc 100644 --- a/include/merovingian/federation/DispatchWorker.hpp +++ b/include/merovingian/federation/DispatchWorker.hpp @@ -49,6 +49,17 @@ using DispatchResolver = std::function(std: // timeout is for other callers; discovery here runs before an attempt has even started. inline constexpr auto dispatch_discovery_timeout_seconds = std::uint32_t{10U}; +class CachedServerDiscovery; +class ServerDiscoveryNetwork; + +// The resolver production gives the dispatch worker: `cached` when it is given, otherwise `network`, with each lookup +// bounded by `timeout_seconds`. A failed discovery is nullopt. A remembered failure is never served +// (DiscoveryCachePolicy::successes_only). The worker's per-destination backoff already paces how often a failing +// destination is looked up. A remembered failure would refuse the first retries after the destination came back for +// the whole cache lifetime, as a sender's restart while the receiver was down showed (ADR-0749). +[[nodiscard]] auto make_dispatch_resolver(ServerDiscoveryNetwork* network, CachedServerDiscovery* cached, + std::uint32_t timeout_seconds) -> DispatchResolver; + // The per-destination bound a worker with `config` enforces: max_pending_per_destination when set, // otherwise a sixteenth of max_queue_depth and never below 16 (256 for the default 4096). [[nodiscard]] auto effective_max_pending_per_destination(DispatchWorkerConfig const& config) noexcept -> std::size_t; diff --git a/include/merovingian/homeserver/ClientOutboundProxy.hpp b/include/merovingian/homeserver/ClientOutboundProxy.hpp index 450756992..b08e803b2 100644 --- a/include/merovingian/homeserver/ClientOutboundProxy.hpp +++ b/include/merovingian/homeserver/ClientOutboundProxy.hpp @@ -24,13 +24,16 @@ struct HomeserverRuntime; // a result after its transport has timed out. inline constexpr std::chrono::seconds default_worker_ipc_margin{10}; -// The policy for a request pool of `pool_threads` threads. -[[nodiscard]] constexpr auto client_outbound_proxy_policy_for_pool(std::uint32_t pool_threads) noexcept +// The policy for a request pool of `pool_threads` threads, with `max_per_client` calls for one client address. The +// per-client cap is never zero and never above the global cap (ADR-0751). +[[nodiscard]] constexpr auto client_outbound_proxy_policy_for_pool( + std::uint32_t pool_threads, std::uint32_t max_per_client = default_client_outbound_max_per_client) noexcept -> ClientOutboundProxyPolicy { auto policy = ClientOutboundProxyPolicy{}; // Never zero: a zero cap would refuse every remote lookup on a tiny pool. policy.global_cap = std::max(1U, pool_threads / 2U); + policy.per_client_cap = std::clamp(max_per_client, 1U, policy.global_cap); policy.policy_server_global_cap = std::max(2U, pool_threads / 4U); policy.policy_server_per_surface_cap = std::max(1U, policy.policy_server_global_cap / 2U); return policy; @@ -55,8 +58,7 @@ inline constexpr std::chrono::seconds default_worker_ipc_margin{10}; // authenticated request: the thread it holds while waiting is bounded by the client's connection share // (ADR-0077) and by the deadline. [[nodiscard]] auto admit_client_outbound_proxy_waiting(HomeserverRuntime& runtime, std::string const& client_key, - RuntimeLock& guard, - OutboundDeadline const& deadline) + RuntimeLock& guard, OutboundDeadline const& deadline) -> std::optional; // The budget for offering remote invites (ADR-0307): one `POST /invite`, or all of a createRoom's. The diff --git a/include/merovingian/homeserver/ClientOutboundProxyPolicy.hpp b/include/merovingian/homeserver/ClientOutboundProxyPolicy.hpp index d39bc2962..26103e0fa 100644 --- a/include/merovingian/homeserver/ClientOutboundProxyPolicy.hpp +++ b/include/merovingian/homeserver/ClientOutboundProxyPolicy.hpp @@ -12,6 +12,8 @@ namespace merovingian::homeserver // from the configured value; this default is what a runtime built without a // configuration uses. inline constexpr std::uint32_t default_main_request_pool_threads{16U}; +// The default of server.http.client_outbound_max_per_client (ADR-0751). +inline constexpr std::uint32_t default_client_outbound_max_per_client{3U}; // Client-triggered outbound proxying (ADR-0079). // @@ -29,8 +31,12 @@ struct ClientOutboundProxyPolicy final // server. std::uint32_t global_cap{default_main_request_pool_threads / 2U}; // Calls in flight for one client address (rate_limit_client_key, which - // honours trusted_proxies and groups IPv6 by prefix). - std::uint32_t per_client_cap{1U}; + // honours trusted_proxies and groups IPv6 by prefix). Three, so a client + // joining two remote rooms while it looks one up is not refused, and + // clients behind one NAT address are not limited to a single call; set by + // server.http.client_outbound_max_per_client, never above global_cap + // (ADR-0751). + std::uint32_t per_client_cap{default_client_outbound_max_per_client}; // Total wall-clock budget for a directory lookup (publicRooms, alias): // server discovery plus the request. std::uint32_t directory_deadline_seconds{10U}; diff --git a/include/merovingian/homeserver/FederationProxy.hpp b/include/merovingian/homeserver/FederationProxy.hpp index 707d90ac9..aad1df667 100644 --- a/include/merovingian/homeserver/FederationProxy.hpp +++ b/include/merovingian/homeserver/FederationProxy.hpp @@ -3,14 +3,17 @@ #pragma once #include "merovingian/config/FederationWorkerConfig.hpp" +#include "merovingian/homeserver/ClientOutboundProxy.hpp" #include "merovingian/homeserver/LocalHttpRequest.hpp" #include "merovingian/homeserver/LocalHttpResponse.hpp" #include "merovingian/homeserver/RoomSyncStatus.hpp" -#include "merovingian/homeserver/ClientOutboundProxy.hpp" #include "merovingian/http/OutboundRequest.hpp" #include "merovingian/http/OutboundResult.hpp" +#include + #include +#include #include #include #include @@ -57,20 +60,33 @@ class FederationProxy final std::chrono::seconds ipc_margin = default_worker_ipc_margin) -> http::OutboundResult; - // Tells the worker shard that owns room_id to refresh its otherwise-stale - // copy of that room from the database. See WorkerPool::notify_room_changed. + // Tells every worker shard to refresh its otherwise-stale copy of room_id + // from main (ADR-0758). See WorkerPool::notify_room_changed. auto notify_room_changed(std::string_view room_id) -> void; // True when the underlying worker pool is up and all shards are healthy. // Exposed for tests that need to poll readiness instead of sleeping. [[nodiscard]] auto healthy() const noexcept -> bool; - // Returns the last observed room-sync status for the shard that owns room_id. + // Returns the last observed room-sync status of room_id on the shard that owns it. // A "failed" status means the worker's room snapshot is stale and must not be // trusted for room-scoped reads; room-scoped federation requests then fall back - // to the main process's authoritative in-memory state. + // to the main process's authoritative in-memory state. While a reload is + // "pending", the room's reads fall back the same way and its writes still go to + // the worker (federation_room_request_served_by_main, ADR-0748). A room the worker has never loaded is served + // by main whatever its state (`held`, ADR-0756). [[nodiscard]] auto room_sync_status(std::string_view room_id) const -> RoomSyncStatus; + // The same for room_id on one shard: what that shard's worker holds of the room. A request is judged on the copy + // of the shard it is sent to, which for a /send may not own every room it names (ADR-0758). + [[nodiscard]] auto room_sync_status(std::size_t shard, std::string_view room_id) const -> RoomSyncStatus; + + // Exposed for tests: the shard a request whose first room is room_id is sent to. + [[nodiscard]] auto shard_for_room(std::string_view room_id) const noexcept -> std::size_t; + + // Exposed for tests: the process ID of the worker that owns room_id, or -1 when that shard has none. + [[nodiscard]] auto worker_pid_for_room(std::string_view room_id) const noexcept -> pid_t; + private: HomeserverRuntime& runtime_; std::unique_ptr pool_{}; diff --git a/include/merovingian/homeserver/FederationRequestRouting.hpp b/include/merovingian/homeserver/FederationRequestRouting.hpp index fffac2c1e..34497847f 100644 --- a/include/merovingian/homeserver/FederationRequestRouting.hpp +++ b/include/merovingian/homeserver/FederationRequestRouting.hpp @@ -4,10 +4,12 @@ #include "merovingian/homeserver/LocalHttpRequest.hpp" #include "merovingian/homeserver/LocalHttpResponse.hpp" +#include "merovingian/homeserver/RoomSyncStatus.hpp" #include #include #include +#include namespace merovingian::homeserver { @@ -22,17 +24,36 @@ namespace merovingian::homeserver // string, which the caller routes to shard 0. [[nodiscard]] auto federation_worker_room_id_from_request(LocalHttpRequest const& request) -> std::string; +// Every distinct room a request concerns: for PUT /send/{txnId}, the room_id of each PDU in the body, in the order +// they first appear (at most federation::max_federation_transaction_pdus); otherwise the room ID in the path, if any. +// The shard the first PDU picks receives the whole transaction, so main checks that shard's sync status of every +// room in it before forwarding it to a worker (ADR-0756, ADR-0758). +[[nodiscard]] auto federation_request_room_ids(LocalHttpRequest const& request) -> std::vector; + // Returns true when a verified inbound federation request should stay in the // main process even while the worker pool is active. [[nodiscard]] auto federation_request_should_bypass_worker(LocalHttpRequest const& request) -> bool; // Returns true for room-scoped federation reads that the main process can // serve authoritatively when the worker's room snapshot is known to be stale. -// Only GET endpoints that consume room state (state, state_ids, backfill, -// get_missing_events, event_auth) qualify; writes must still reach the worker -// because they may trigger main-side relays and side effects. +// Only the reads that consume room state qualify: GET state, state_ids, +// backfill, event_auth and timestamp_to_event, and POST get_missing_events, +// which v1.19 defines as POST though it only reads (ADR-0748), and GET +// make_leave and make_knock, which build a template from room state and +// change nothing (ADR-0759). Writes must still reach the worker because they +// may trigger main-side relays and side effects. [[nodiscard]] auto federation_worker_room_read_may_fallback_to_main(LocalHttpRequest const& request) noexcept -> bool; +// Returns true when a request for a room must be served by the main process instead of the worker shard that holds +// the room, given that shard's room-sync status for it. After a "failed" reload the worker's copy of the room is +// stale, so every request for the room goes to main. While a reload is "pending", main has changed the room and the +// worker has not yet reloaded it. A read main can serve (federation_worker_room_read_may_fallback_to_main) then goes +// to main. Otherwise a server asking about an event this server has just sent is told it does not exist. Writes +// still go to the worker (ADR-0748), unless the worker has never loaded the room (`held` false): then every request +// for it goes to main, as after a failed reload (ADR-0756). +[[nodiscard]] auto federation_room_request_served_by_main(LocalHttpRequest const& request, + RoomSyncStatus const& status) noexcept -> bool; + // Returns true when the request target is exactly `/_matrix/key/v2/server` // (optionally followed by a query string). The key server endpoint must be // matched precisely: a substring match would let unrelated paths such as diff --git a/include/merovingian/homeserver/RoomService.hpp b/include/merovingian/homeserver/RoomService.hpp index cc89d94d9..e1051a10d 100644 --- a/include/merovingian/homeserver/RoomService.hpp +++ b/include/merovingian/homeserver/RoomService.hpp @@ -214,7 +214,9 @@ auto federate_accepted_membership_event(HomeserverRuntime& runtime, std::string_ // received by the resident server". Puts this server's name and clock on a // validated make_join or make_leave template before it is hashed and signed, and // drops any event_id (room versions 3 and later derive it from the event). The -// resident never chooses the timestamp this server signs. +// resident never chooses the timestamp this server signs. Drops `unsigned`, +// `hashes` and `signatures` too: signing adds to the signatures an event holds, +// so a template's own would be sent and stored as part of this server's event. auto claim_membership_event_template(canonicaljson::Object& event, std::string_view our_server_name, std::int64_t now_ms) -> void; diff --git a/include/merovingian/homeserver/RoomSyncStatus.hpp b/include/merovingian/homeserver/RoomSyncStatus.hpp index f2e541c6b..23075f9d7 100644 --- a/include/merovingian/homeserver/RoomSyncStatus.hpp +++ b/include/merovingian/homeserver/RoomSyncStatus.hpp @@ -18,6 +18,10 @@ struct RoomSyncStatus final { std::string state{}; // "pending", "ok", or "failed" std::uint64_t generation{0}; // monotonic per room_id, assigned by main + // Whether the worker holds a copy of the room at all. A room main notifies for the first time may have been + // created after the worker's startup fill, so it is not held until the worker reports a successful reload of it; + // until then every request for the room goes to main (ADR-0756). A room main never notified is held. + bool held{true}; }; } // namespace merovingian::homeserver diff --git a/include/merovingian/homeserver/WorkerPool.hpp b/include/merovingian/homeserver/WorkerPool.hpp index be0279fb5..3f05e069f 100644 --- a/include/merovingian/homeserver/WorkerPool.hpp +++ b/include/merovingian/homeserver/WorkerPool.hpp @@ -16,6 +16,8 @@ #include "merovingian/ipc/IpcChannel.hpp" #include "merovingian/net/ThreadPool.hpp" +#include + #include #include #include @@ -40,7 +42,7 @@ struct HomeserverRuntime; // /send transaction so it is persisted through main's pdu_sink). Exposed as a // free function for the same test-seam reason handle_membership_ingest_request // is. Takes the wire JSON a worker sends and returns the ingestion result; -// WorkerPool serializes it and, on acceptance, notifies the owning shard. +// WorkerPool serializes it and, on acceptance, notifies every shard (ADR-0758). [[nodiscard]] auto handle_pdu_ingest_request(HomeserverRuntime& runtime, std::string_view request_json) -> federation::PduIngestionResult; @@ -243,14 +245,12 @@ class WorkerPool final std::chrono::seconds ipc_margin = default_worker_ipc_margin) -> http::OutboundResult; - // Tells the worker shard that owns room_id to re-read that room from the - // database. The worker's PersistentStore is otherwise a snapshot taken - // once at worker startup — it never learns about rooms created or joined - // by the main process afterward (see docs/architecture.md, "Federation - // worker room staleness"). Fire-and-forget: no reply is expected, and a - // failure to deliver the notification (e.g. an unhealthy shard) is not - // surfaced to the caller, matching the "best-effort cache refresh, not a - // correctness-critical write" nature of this call. + // Tells every worker shard to re-read room_id from main. A worker's PersistentStore is otherwise a snapshot taken + // once at worker startup: it never learns about rooms created or joined by the main process afterward (see + // docs/architecture.md, "Federation worker room staleness"). Every shard is told, not only the one that owns the + // room, because a /send goes whole to the shard its first PDU picks and may carry PDUs for rooms other shards own + // (ADR-0758). Fire-and-forget: no reply is expected, and a failure to deliver the notification (e.g. an unhealthy + // shard) is not surfaced to the caller; that shard's status for the room stays pending, so main serves it. auto notify_room_changed(std::string_view room_id) -> void; // True when all configured workers are healthy. @@ -262,10 +262,17 @@ class WorkerPool final // Exposed for unit tests: which shard index would handle this room_id? [[nodiscard]] auto shard_for(std::string_view room_id) const noexcept -> std::size_t; - // Returns the last observed room-sync status for the shard that owns room_id. + // Exposed for tests: the process ID of the worker serving a shard, or -1 when it has none. + [[nodiscard]] auto worker_pid(std::size_t shard) const noexcept -> pid_t; + + // Returns the last observed room-sync status of room_id on the shard that owns it. // See FederationProxy::room_sync_status for semantics. [[nodiscard]] auto room_sync_status(std::string_view room_id) const -> RoomSyncStatus; + // Returns the last observed room-sync status of room_id on one shard: what the worker a request is sent to holds + // of that room (ADR-0758). A shard outside the pool holds nothing: failed and not held. + [[nodiscard]] auto room_sync_status(std::size_t shard, std::string_view room_id) const -> RoomSyncStatus; + private: config::FederationWorkerConfig cfg_{}; HomeserverRuntime& runtime_; @@ -283,18 +290,25 @@ class WorkerPool final std::string config_path_; std::vector> workers_{}; - // Per-room room_sync tracking. Guarded by room_sync_mu_. generation is - // monotonically increasing per room_id and assigned by main when it sends a - // room_sync notification; the worker echoes the same generation in its - // room_sync_result notification so main can match asynchronous results to - // the pending entry. + // Room_sync tracking, guarded by room_sync_mu_. generation is monotonically increasing per room_id and assigned by + // main when it notifies every shard of a change; each worker echoes it in its room_sync_result so main can match + // asynchronous results to the pending entry. Status is kept per shard (one map per shard, sized at construction): + // each worker reloads its own copy, and a request is judged on the copy of the worker it is sent to (ADR-0758). + // Entries exist only for rooms main notified or listed to a channel, so a worker cannot grow these maps. mutable std::mutex room_sync_mu_{}; - std::unordered_map room_sync_status_{}; + std::vector> room_sync_status_{}; std::unordered_map room_sync_generations_{}; - // Updates the tracked status for a room_id. Called on the IPC dispatch - // thread when a room_sync_result notification arrives from a worker. - auto record_room_sync_result(std::string_view room_id, std::string_view state, std::uint64_t generation) -> void; + // Updates the tracked status of room_id on one shard. Called on the IPC dispatch + // thread when a room_sync_result notification arrives from that shard's worker. + auto record_room_sync_result(std::size_t shard, std::string_view room_id, std::string_view state, + std::uint64_t generation) -> void; + + // Records that a shard's start-up fill left room_id out, so that worker holds no copy of it and main serves its + // requests for the room until a reload of it succeeds (ADR-0756). A pending reload keeps its state and decides + // when its result arrives. The caller has checked the room against the channel's listing + // (WorkerSnapshotService::consume_fill_omission). + auto record_room_left_out_of_fill(std::size_t shard, std::string_view room_id) -> void; }; } // namespace merovingian::homeserver diff --git a/include/merovingian/homeserver/WorkerSnapshotService.hpp b/include/merovingian/homeserver/WorkerSnapshotService.hpp index 0a0afa497..267f256bf 100644 --- a/include/merovingian/homeserver/WorkerSnapshotService.hpp +++ b/include/merovingian/homeserver/WorkerSnapshotService.hpp @@ -61,6 +61,11 @@ class WorkerSnapshotService final // no read. [[nodiscard]] auto answer(ChannelKey const& channel, std::string_view type, std::string_view json) -> std::string; + // Whether the worker on `channel` may report `room_id` as left out of its start-up fill: true only for a room in + // the listing this channel was sent, and at most once per room, so a worker cannot make main record a room main + // never listed to it (ADR-0756). + [[nodiscard]] auto consume_fill_omission(ChannelKey const& channel, std::string_view room_id) -> bool; + // Channels with state held, after dropping those whose channel is gone. For tests. [[nodiscard]] auto channels_tracked() -> std::size_t; @@ -86,6 +91,8 @@ class WorkerSnapshotService final std::vector list_pages{}; std::size_t next_list_page{0U}; std::unordered_set fill_permits{}; + // The listed rooms the worker has not yet reported as left out of its fill (consume_fill_omission). + std::unordered_set fill_omissions{}; std::unordered_map reload_permits{}; std::optional held{}; }; diff --git a/include/merovingian/sync/ReadMarkers.hpp b/include/merovingian/sync/ReadMarkers.hpp index 8447f4d85..3b0d9e7ed 100644 --- a/include/merovingian/sync/ReadMarkers.hpp +++ b/include/merovingian/sync/ReadMarkers.hpp @@ -70,4 +70,12 @@ struct ReadMarkers final [[nodiscard]] auto event_thread_id(database::PersistentStore const& store, canonicaljson::Object const& event, std::size_t* bytes_left = nullptr) -> std::string; +// Whether `event` relates to `target_event_id` through its `m.relates_to` chain, of any `rel_type`, within the same +// 3 hops as `event_thread_id`: its own relation, its parent's and its grandparent's. Parents are looked up in +// `room_id` only; one this store does not hold ends the chain. An event has one relation, so the chain is a single +// path and an event relates to at most three events. Spec (POST /receipt, 400): "The event_id is not related to the +// thread_id" (ADR-0746). +[[nodiscard]] auto event_relates_to(database::PersistentStore const& store, std::string_view room_id, + canonicaljson::Object const& event, std::string_view target_event_id) -> bool; + } // namespace merovingian::sync diff --git a/meson.build b/meson.build index a483526a1..0f334b178 100644 --- a/meson.build +++ b/meson.build @@ -2,7 +2,7 @@ project( 'merovingian', 'cpp', 'c', - version: '1.0.1', + version: '1.0.2', license: 'GPL-3.0-or-later', meson_version: '>=1.1.0', default_options: [ @@ -88,9 +88,10 @@ hardening_compile_flags = [ '-fcf-protection=full', ] -# -pie is supplied per-platform by the build scripts: -# Alpine/musl .deb → -static-pie (fully static PIE via cpp_link_args) -# Fedora .rpm / FreeBSD .pkg → -pie (dynamic-libc PIE via cpp_link_args) +# -pie is added below for every hardened Linux, FreeBSD and OpenBSD build; the +# packaging scripts add their own as well: +# Alpine/musl .deb: -static-pie (fully static PIE via cpp_link_args) +# Fedora .rpm / FreeBSD .pkg: -pie (dynamic-libc PIE via cpp_link_args) # -Wl,-z,noexecstack sets the PT_GNU_STACK ELF note; the kernel reads this on # both static and dynamic ELF binaries, so it applies universally. # -Wl,-z,relro/-z,now are requested wherever the startup self-check can pass, @@ -101,13 +102,14 @@ noexecstack_flag = '-Wl,-z,noexecstack' relro_now_flags = ['-Wl,-z,relro', '-Wl,-z,now'] hardening_link_flags = [noexecstack_flag] -if host_machine.system() == 'linux' +if host_machine.system() in ['linux', 'freebsd', 'openbsd'] # -pie makes the binary ET_DYN (PIE-linked). Combined with -fPIE on the objects - # this allows the ELF probe to confirm PIE at runtime. The packaging scripts - # (build-*.sh) may also pass -pie or -static-pie; the linker tolerates duplicates. + # this allows the ELF probe to confirm PIE at runtime. FreeBSD's clang does not + # link PIE by default, so a hardened BSD build that only the packaging script + # made PIE was ET_EXEC everywhere else, release evidence included (ADR-0757). + # The packaging scripts (build-*.sh) may also pass -pie or -static-pie; the + # linker tolerates duplicates. hardening_link_flags += ['-pie'] -endif -if host_machine.system() in ['linux', 'freebsd', 'openbsd'] hardening_link_flags += relro_now_flags endif diff --git a/packaging/freebsd/+MANIFEST b/packaging/freebsd/+MANIFEST index 79c983934..a1eabee9e 100644 --- a/packaging/freebsd/+MANIFEST +++ b/packaging/freebsd/+MANIFEST @@ -1,5 +1,5 @@ name: "merovingian" -version: "1.0.1" +version: "1.0.2" origin: net/merovingian comment: Secure Matrix Protocol homeserver desc: "Merovingian is a Matrix Protocol homeserver." diff --git a/packaging/netbsd/Makefile b/packaging/netbsd/Makefile index aebccdc3e..313c53462 100644 --- a/packaging/netbsd/Makefile +++ b/packaging/netbsd/Makefile @@ -1,4 +1,4 @@ -DISTNAME= merovingian-1.0.1 +DISTNAME= merovingian-1.0.2 CATEGORIES= net MASTER_SITES= ${MASTER_SITE_GITHUB:=OMG-Software/} GITHUB_PROJECT= Merovingian diff --git a/packaging/opensuse/merovingian.spec b/packaging/opensuse/merovingian.spec index ab681be97..b4ace1a06 100644 --- a/packaging/opensuse/merovingian.spec +++ b/packaging/opensuse/merovingian.spec @@ -1,5 +1,5 @@ Name: merovingian -Version: 1.0.1 +Version: 1.0.2 Release: 1%{?dist} Summary: Secure Matrix Protocol homeserver @@ -115,6 +115,9 @@ fi %{_sysconfdir}/merovingian/merovingian.conf.example %changelog +* Sat Nov 14 2026 James Chapman - 1.0.2-1 +- Merovingian 1.0.2 + * Thu Nov 12 2026 James Chapman - 1.0.1-1 - Merovingian 1.0.1 diff --git a/packaging/rhel/merovingian.spec b/packaging/rhel/merovingian.spec index 1b44cf01a..2c1ca4cb0 100644 --- a/packaging/rhel/merovingian.spec +++ b/packaging/rhel/merovingian.spec @@ -1,5 +1,5 @@ Name: merovingian -Version: 1.0.1 +Version: 1.0.2 Release: 1%{?dist} Summary: Secure Matrix Protocol homeserver @@ -100,6 +100,9 @@ fi %{_sysconfdir}/merovingian/merovingian.conf.example %changelog +* Sat Nov 14 2026 James Chapman - 1.0.2-1 +- Merovingian 1.0.2 + * Thu Nov 12 2026 James Chapman - 1.0.1-1 - Merovingian 1.0.1 diff --git a/packaging/rpm/merovingian.spec b/packaging/rpm/merovingian.spec index 0f423739e..e86b3a071 100644 --- a/packaging/rpm/merovingian.spec +++ b/packaging/rpm/merovingian.spec @@ -1,5 +1,5 @@ Name: merovingian -Version: 1.0.1 +Version: 1.0.2 Release: 1%{?dist} Summary: Secure Matrix Protocol homeserver @@ -104,6 +104,9 @@ fi %{_sysconfdir}/merovingian/merovingian.conf.example %changelog +* Sat Nov 14 2026 James Chapman - 1.0.2-1 +- Merovingian 1.0.2 + * Thu Nov 12 2026 James Chapman - 1.0.1-1 - Merovingian 1.0.1 diff --git a/scripts/AGENTS.md b/scripts/AGENTS.md index 4c1f32975..4aef756b9 100644 --- a/scripts/AGENTS.md +++ b/scripts/AGENTS.md @@ -8,10 +8,11 @@ called by it rather than directly. | Script | Purpose | |---|---| -| `build-linux.sh` | Linux release build (Meson + Ninja). `--coverage` builds run `meson test` with `--timeout-multiplier 2` unless `MESON_TEST_TIMEOUT_MULTIPLIER` sets one: gcov instrumentation took the conformance suite to 555 s of its 600 s limit, and other builds keep the plain limits that catch a hang. Covered by `tests/tooling/test_dependency_wraps.py` | +| `build-linux.sh` | Linux release build (Meson + Ninja). `--coverage` builds run `meson test` with `--timeout-multiplier 2` unless `MESON_TEST_TIMEOUT_MULTIPLIER` sets one: gcov instrumentation took the conformance suite to 555 s of its 600 s limit, and other builds keep the plain limits that catch a hang. `--jobs ` caps both `meson compile -j` and `meson test --num-processes` (a positive whole number; `build.py vm` sizes it from the VM's memory, ADR-0752). `--hardening` sets `-Dhardening=true`, as the `hardened` profile does; `build.py linux --hardening` passes it. Covered by `tests/tooling/test_dependency_wraps.py` and `tests/tooling/test_build_vm.py` | | `build-wsl.sh` / `build-wsl.ps1` | WSL build wrapper (runs the Linux build inside WSL from Windows) | +| `remote-host-setup.sh` | One-time setup of a remote build host (a VM or another machine) for `build.py vm` (ADR-0752); not for WSL, which has `wsl-setup.sh`. Accepts Ubuntu, Debian, distributions whose `ID_LIKE` names either, and FreeBSD (read from `/etc/os-release` without sourcing it; `--os-release` names another file), and refuses others before running anything. Runs as root, else through `sudo`, else `doas`. On the Debian family it installs with `apt-get` every library `build-linux.sh` checks for, plus libpng and libjpeg-turbo, and Meson 1.11+ and Ninja in a virtual environment linked into `~/.local/bin`, which `build.py vm` puts first on `PATH`; on FreeBSD it installs with `pkg` the FreeBSD CI job's packages plus `python3`, which `build.py vm`'s lock needs. `--dry-run` prints the commands. Covered by `tests/tooling/test_build_vm.py` | | `build-static-linux.sh` | Fully static Linux binary (musl) | -| `build-bsd.sh` | POSIX-sh build/test driver for BSD hosts (FreeBSD/NetBSD/OpenBSD): configures Meson with system libsodium/openssl/libpq/libcurl, supports named profiles (`debug`, `release`, `sanitizer`, `coverage`, `fuzz`, `hardened`), and runs `meson test` with hardening disabled in-process for the Catch2 runner and `--timeout-multiplier 2` by default (`MESON_TEST_TIMEOUT_MULTIPLIER` overrides it), because the BSD CI VMs' speed varies between runs | +| `build-bsd.sh` | POSIX-sh build/test driver for BSD hosts (FreeBSD/NetBSD/OpenBSD): configures Meson with system libsodium/openssl/libpq/libcurl, supports named profiles (`debug`, `release`, `sanitizer`, `coverage`, `fuzz`, `hardened`), and runs `meson test` with hardening disabled in-process for the Catch2 runner and `--timeout-multiplier 2` by default (`MESON_TEST_TIMEOUT_MULTIPLIER` overrides it), because the BSD CI VMs' speed varies between runs. Takes `--jobs ` and `--hardening` as `build-linux.sh` does; `build.py vm` runs it with `--jobs` on a BSD host (ADR-0752). Covered by `tests/tooling/test_build_vm.py` | | `build-deb.sh` | Builds a `.deb` package for the Ubuntu or Debian release it runs on. `Depends` is `adduser` (the postinst runs it) plus what `dpkg-shlibdeps` derives from every ELF file in the staging tree, so each release gets its own library package names and minimum versions (ADR-0650); an empty result fails the build. Needs `dpkg-dev` (pulled in by `build-essential`) | | `build-rpm.sh` | Builds an `.rpm` package (Fedora-family default spec) | | `build-opensuse-rpm.sh` | Builds an `.rpm` for OpenSUSE Tumbleweed or Leap from `packaging/opensuse/merovingian.spec` (OpenSUSE package names); forces a dist tag that names the release, so the filename shows its platform and doesn't collide with the Fedora/RHEL RPMs: `.tumbleweed` unless `MEROVINGIAN_RPM_DIST` names another (packages.yml passes `.leap16` for Leap 16.0, ADR-0650, ADR-0742). Refuses a tag that is not a dot, a lower-case letter, then lower-case letters, digits and dots | @@ -21,9 +22,9 @@ called by it rather than directly. | `ci-apt-update.sh` | CI only: refreshes the apt index on an Ubuntu runner or Ubuntu container with `apt-get update --error-on=any`, and when that fails points `/etc/resolv.conf` at Cloudflare and Google (replacing a host runner's systemd-resolved symlink; rewriting a container's bind-mounted file in place) and tries three more times (ADR-0650). Uses `sudo` unless root. Every Ubuntu job in `.github/workflows/` runs it after its checkout; `ubuntu-interim-build-and-test` refreshes before its checkout and carries the same fallback inline. Covered by `tests/tooling/test_ci_workflow.py` against a fake `apt-get` | | `ci-openbsd-pkg-add.sh` | CI only: `ci-openbsd-pkg-add.sh --require "command ..." [--require-pkg-config "module ..."] package ...` installs packages in the vmactions OpenBSD VMs, three tries through `cdn.openbsd.org` then three through `ftp.openbsd.org` (`PKG_PATH` for the VM's release and architecture). Judges each try by whether every `--require` command is on `PATH` and every `--require-pkg-config` module passes `pkg-config --exists`, because `pkg_add` exits 0 having installed nothing; fails naming what is missing (ADR-0650). Both OpenBSD jobs run it from `prepare`, which vmactions runs in the synced checkout. Covered by `tests/tooling/test_ci_workflow.py` against a fake `pkg_add` | | `check-catch2-bdd-tests.sh` | Rejects unit test files that use `TEST_CASE`, use comment-only Given/When/Then markers, or lack any of `SCENARIO`/`GIVEN`/`WHEN`/`THEN` (`test_main.cpp` exempt). One `awk` pass over all files: run per file it timed out on the OpenBSD VM. Covered by `tests/tooling/test_check_catch2_bdd.py` | -| `check_expected_skips.py` | CI skip gate (ADR-0546): reads the XML reports the Catch2 binaries write when `MEROVINGIAN_CATCH_REPORT_DIR` is set and fails on any skipped test case that `tests/expected_skips.json` does not list for the `--job`. Exit 0 ok, 1 unexpected skip (or a stale entry with `--strict-stale`), 2 unusable input (no report, truncated report, bad list, unknown job). `` is a key of the `jobs` object in `tests/expected_skips.json`. `build-linux.sh` and `build-bsd.sh` run it after the suite when given `--skip-job `; `build-wsl.sh` has no `--skip-job`, so WSL runs are not skip-gated; `build-bsd.sh` runs it with `python3`, else the interpreter in `meson`'s shebang, else a `python3.N` on `PATH` (FreeBSD and NetBSD install only a versioned Python, as meson's dependency), and fails the gate when it finds none. Covered by `tests/tooling/test_expected_skips.py` | +| `check_expected_skips.py` | CI skip gate (ADR-0546): reads the XML reports the Catch2 binaries write when `MEROVINGIAN_CATCH_REPORT_DIR` is set and fails on any skipped test case that `tests/expected_skips.json` does not list for the `--job`. Exit 0 ok, 1 unexpected skip (or a stale entry with `--strict-stale`), 2 unusable input (no report, truncated report, bad list, unknown job). `` is a key of the `jobs` object in `tests/expected_skips.json`. `build-linux.sh` and `build-bsd.sh` run it after the suite when given `--skip-job `; `build-wsl.sh` has no `--skip-job`, and `build.py vm` passes none, so WSL and VM runs are not skip-gated; `build-bsd.sh` runs it with `python3`, else the interpreter in `meson`'s shebang, else a `python3.N` on `PATH` (FreeBSD and NetBSD install only a versioned Python, as meson's dependency), and fails the gate when it finds none. Covered by `tests/tooling/test_expected_skips.py` | | `check-conformance-gate.sh` | Runs `meson test conformance-tests --print-errorlogs` in the build directory given as its argument (default `build`) and fails if that suite fails or the directory is missing. It does not detect a removed or commented-out test; CI runs it after each build | -| `check-elf-hardening.sh` | Build-time hardening gate: inspects a built ELF binary's headers with `readelf` and fails if PIE (`ET_DYN`), `PT_GNU_RELRO`, `DT_BIND_NOW`, or a non-executable stack (`PT_GNU_STACK` without `E`) is missing; the CI-time counterpart to the runtime self-check in `src/platform/`, since that check can only run inside a live, correctly-privileged `merovingian-server` process. Pass `--static` for the static-PIE binary, which has no `PT_DYNAMIC` section to apply `BIND_NOW` to | +| `check-elf-hardening.sh` | Build-time hardening gate: inspects a built ELF binary's headers with `readelf` and fails if PIE (`ET_DYN`), `PT_GNU_RELRO`, `DT_BIND_NOW`, or a non-executable stack (`PT_GNU_STACK` without `E`) is missing; the CI-time counterpart to the runtime self-check in `src/platform/`, since that check can only run inside a live, correctly-privileged `merovingian-server` process. Pass `--static` for the static-PIE binary, which has no `PT_DYNAMIC` section to apply `BIND_NOW` to. Accepts both GNU binutils' and the BSDs' elftoolchain `readelf` output, and uses POSIX ERE only (ADR-0757). Covered by `tests/tooling/test_check_elf_hardening.py` against a fake `readelf` | | `check-startup-hardening-gate.sh` | Runtime counterpart to `check-elf-hardening.sh` for shipped artifacts (audit OPS-5): boots a packaged `merovingian-server` against a throwaway loopback SQLite config with a fresh 0400 master key, and passes only when every `Hardening self-check:` line is `enabled`, their count matches `Startup hardening checks: N`, readiness is `ready=true blockers=0`, the federation worker is spawned and still up `--worker-settle` seconds (3) later, and the server exits 0 on `SIGTERM`. Run it as the unprivileged service user with an empty capability bounding set. Leaves the worker path to the binary's libexec default unless `--federation-worker` is given. Off Linux it sets `federation.worker.apply_hardening=false`, because the worker's in-process sandbox exists only on Linux. Run by the `deb`, `debian-pkg`, `rpm`, `freebsd-pkg` and `docker-image` jobs in `packages.yml`; covered by `tests/tooling/test_startup_hardening_gate.py` | | `check-staged-changelog-docs.sh` | Pre-commit guard requiring staged project changes to include `CHANGELOG.md` and relevant `docs/*.md` updates | | `check-unit-test-registration.sh` | Checks that every `tests/unit/test_*.cpp` and `tests/conformance/test_*.cpp` file is listed in `tests/meson.build`, and that no listed file is missing from disk | @@ -58,7 +59,7 @@ called by it rather than directly. ## Rules - **Do not call build scripts directly** unless you know what you are doing. - Use `python build.py` (or `python build.py wsl` on Windows) — it handles ordering, flags, and test runs. + Use `python build.py` (on Windows, `python build.py vm` for the Linux VM and `python build.py vm --vm freebsd` for the FreeBSD VM; build and test there, not in WSL) — it handles ordering, flags, and test runs. - All scripts must be idempotent — running them twice must not cause errors or duplicate work. - Scripts that modify the working tree (format, rewrite) must print what they changed. - `reject-unsafe.sh` runs as a pre-commit gate. If it blocks you, fix the code — do not diff --git a/scripts/build-bsd.sh b/scripts/build-bsd.sh index 095778b2c..70625357b 100644 --- a/scripts/build-bsd.sh +++ b/scripts/build-bsd.sh @@ -20,6 +20,7 @@ sanitize= coverage=false skip_job= hardening= +jobs= script_dir=$(CDPATH= cd "$(dirname "$0")" && pwd -P) repo_root=$(CDPATH= cd "$script_dir/.." && pwd -P) @@ -42,8 +43,11 @@ Options: --sanitize Meson b_sanitize value, for example address,undefined. --coverage Enable Meson coverage instrumentation. --build-fuzz Enable fuzz harness targets. + --hardening Enable hardening flags (-Dhardening=true). --skip-job After the tests, fail on any Catch2 skip that job in tests/expected_skips.json does not list (ADR-0546). + --jobs Run at most compile jobs and tests at once. + Default: Meson's own choice, one per CPU or more. --no-tests Build without running tests. --setup-only Configure/reconfigure Meson and stop. --compile-only Configure/reconfigure and compile, but do not test. @@ -163,11 +167,22 @@ while [ "$#" -gt 0 ]; do --build-fuzz) build_fuzz=true ;; + --hardening) + hardening=true + ;; --skip-job) shift [ "$#" -gt 0 ] || fail "--skip-job requires a job name" skip_job=$1 ;; + --jobs) + shift + [ "$#" -gt 0 ] || fail "--jobs requires a number" + case "$1" in + ""|*[!0-9]*|0*) fail "--jobs requires a positive whole number, not '$1'" ;; + esac + jobs=$1 + ;; --no-tests) run_tests=0 ;; @@ -255,7 +270,11 @@ if [ "$setup_only" -eq 1 ]; then exit 0 fi -run meson compile -C "$builddir" +if [ -n "$jobs" ]; then + run meson compile -C "$builddir" -j "$jobs" +else + run meson compile -C "$builddir" +fi if [ "$compile_only" -eq 1 ] || [ "$run_tests" -eq 0 ]; then exit 0 @@ -277,6 +296,10 @@ export MEROVINGIAN_TEST_DISABLE_HARDENING=1 # its 30 minutes because of this default (ADR-0509): change them together. timeout_multiplier="${MESON_TEST_TIMEOUT_MULTIPLIER:-2}" meson_test_timeout_args="--timeout-multiplier ${timeout_multiplier}" +meson_test_jobs_args="" +if [ -n "$jobs" ]; then + meson_test_jobs_args="--num-processes ${jobs}" +fi # With --skip-job each Catch2 binary also writes an XML report, and # scripts/check_expected_skips.py fails the build on a skipped test case the job @@ -293,7 +316,7 @@ fi test_status=0 # shellcheck disable=SC2086 -run env MEROVINGIAN_TEST_DISABLE_HARDENING=1 MEROVINGIAN_CATCH_REPORT_DIR="$skip_reports" meson test -C "$builddir" --print-errorlogs ${meson_test_timeout_args} || test_status=$? +run env MEROVINGIAN_TEST_DISABLE_HARDENING=1 MEROVINGIAN_CATCH_REPORT_DIR="$skip_reports" meson test -C "$builddir" --print-errorlogs ${meson_test_timeout_args} ${meson_test_jobs_args} || test_status=$? gate_status=0 if [ -n "$skip_job" ]; then diff --git a/scripts/build-deb.sh b/scripts/build-deb.sh index 9e3798c67..04f9ee573 100755 --- a/scripts/build-deb.sh +++ b/scripts/build-deb.sh @@ -1,9 +1,9 @@ #!/bin/sh # SPDX-License-Identifier: GPL-3.0-or-later -# Build for merovingian 1.0.1. +# Build for merovingian 1.0.2. set -e -VERSION="1.0.1" +VERSION="1.0.2" PKG_NAME="merovingian" STAGING="staging-deb" diff --git a/scripts/build-freebsd-pkg.sh b/scripts/build-freebsd-pkg.sh index f22e0abab..240e68bbe 100755 --- a/scripts/build-freebsd-pkg.sh +++ b/scripts/build-freebsd-pkg.sh @@ -1,9 +1,9 @@ #!/bin/sh # SPDX-License-Identifier: GPL-3.0-or-later -# Build for merovingian 1.0.1. +# Build for merovingian 1.0.2. set -e -VERSION="1.0.1" +VERSION="1.0.2" STAGING="staging-fbsd" # Clean any state (staged files, build dir) from cached FreeBSD VM runs. diff --git a/scripts/build-linux.sh b/scripts/build-linux.sh index 9e7e2ece5..9ab3d6b08 100644 --- a/scripts/build-linux.sh +++ b/scripts/build-linux.sh @@ -20,6 +20,7 @@ sanitize= coverage=false skip_job= hardening= +jobs= script_dir=$(CDPATH= cd "$(dirname "$0")" && pwd -P) repo_root=$(CDPATH= cd "$script_dir/.." && pwd -P) @@ -42,8 +43,11 @@ Options: --sanitize Meson b_sanitize value, for example address,undefined. --coverage Enable Meson coverage instrumentation. --build-fuzz Enable fuzz harness targets. + --hardening Enable hardening flags (-Dhardening=true). --skip-job After the tests, fail on any Catch2 skip that job in tests/expected_skips.json does not list (ADR-0546). + --jobs Run at most compile jobs and tests at once. + Default: Meson's own choice, one per CPU or more. --no-tests Build without running tests. --setup-only Configure/reconfigure Meson and stop. --compile-only Configure/reconfigure and compile, but do not test. @@ -152,11 +156,22 @@ while [ "$#" -gt 0 ]; do --build-fuzz) build_fuzz=true ;; + --hardening) + hardening=true + ;; --skip-job) shift [ "$#" -gt 0 ] || fail "--skip-job requires a job name" skip_job=$1 ;; + --jobs) + shift + [ "$#" -gt 0 ] || fail "--jobs requires a number" + case "$1" in + ""|*[!0-9]*|0*) fail "--jobs requires a positive whole number, not '$1'" ;; + esac + jobs=$1 + ;; --no-tests) run_tests=0 ;; @@ -240,7 +255,11 @@ if [ "$setup_only" -eq 1 ]; then exit 0 fi -run meson compile -C "$builddir" +if [ -n "$jobs" ]; then + run meson compile -C "$builddir" -j "$jobs" +else + run meson compile -C "$builddir" +fi if [ "$compile_only" -eq 1 ] || [ "$run_tests" -eq 0 ]; then exit 0 @@ -265,6 +284,10 @@ meson_test_timeout_args="" if [ -n "$timeout_multiplier" ]; then meson_test_timeout_args="--timeout-multiplier ${timeout_multiplier}" fi +meson_test_jobs_args="" +if [ -n "$jobs" ]; then + meson_test_jobs_args="--num-processes ${jobs}" +fi # With --skip-job each Catch2 binary also writes an XML report, and # scripts/check_expected_skips.py fails the build on a skipped test case the job @@ -281,7 +304,7 @@ fi test_status=0 # shellcheck disable=SC2086 -run env MEROVINGIAN_TEST_DISABLE_HARDENING=1 MEROVINGIAN_CATCH_REPORT_DIR="$skip_reports" meson test -C "$builddir" --print-errorlogs ${meson_test_timeout_args} || test_status=$? +run env MEROVINGIAN_TEST_DISABLE_HARDENING=1 MEROVINGIAN_CATCH_REPORT_DIR="$skip_reports" meson test -C "$builddir" --print-errorlogs ${meson_test_timeout_args} ${meson_test_jobs_args} || test_status=$? gate_status=0 if [ -n "$skip_job" ]; then diff --git a/scripts/build-netbsd-pkg.sh b/scripts/build-netbsd-pkg.sh index 875997064..701ebff62 100644 --- a/scripts/build-netbsd-pkg.sh +++ b/scripts/build-netbsd-pkg.sh @@ -1,6 +1,6 @@ #!/bin/sh # SPDX-License-Identifier: GPL-3.0-or-later -# Build for merovingian 1.0.1. +# Build for merovingian 1.0.2. # # The checked-in packaging/netbsd/Makefile is the pkgsrc recipe kept for # downstream porters. This script generates a framework-free package so CI @@ -11,7 +11,7 @@ # a staged tree using tar, which produces a package that pkg_add can install. set -e -VERSION="1.0.1" +VERSION="1.0.2" STAGE="staging-netbsd" PREFIX=/usr/pkg diff --git a/scripts/build-openbsd-pkg.sh b/scripts/build-openbsd-pkg.sh index 993144d27..d76bf87ef 100644 --- a/scripts/build-openbsd-pkg.sh +++ b/scripts/build-openbsd-pkg.sh @@ -1,6 +1,6 @@ #!/bin/sh # SPDX-License-Identifier: GPL-3.0-or-later -# Build for merovingian 1.0.1. +# Build for merovingian 1.0.2. # # Standalone pkg_create(1), no ports tree. The checked-in packaging/openbsd/PLIST # is the ports-framework packing list kept for downstream porters; this script @@ -8,7 +8,7 @@ # produce and validate an installable package on every run. set -e -VERSION="1.0.1" +VERSION="1.0.2" STAGE="staging-openbsd" PREFIX=/usr/local diff --git a/scripts/build-opensuse-rpm.sh b/scripts/build-opensuse-rpm.sh index 50ba800cd..fd29b83a6 100644 --- a/scripts/build-opensuse-rpm.sh +++ b/scripts/build-opensuse-rpm.sh @@ -1,6 +1,6 @@ #!/bin/sh # SPDX-License-Identifier: GPL-3.0-or-later -# Build an RPM package for merovingian 1.0.1 on OpenSUSE Tumbleweed or Leap. +# Build an RPM package for merovingian 1.0.2 on OpenSUSE Tumbleweed or Leap. # # Uses packaging/opensuse/merovingian.spec which adapts BuildRequires to # OpenSUSE package names (libopenssl-devel, postgresql-devel, libpng16-devel, @@ -12,7 +12,7 @@ # (.leap16, ADR-0650). set -e -VERSION="1.0.1" +VERSION="1.0.2" DIST="${MEROVINGIAN_RPM_DIST:-.tumbleweed}" case "${DIST}" in .[a-z]*) ;; diff --git a/scripts/build-rhel-rpm.sh b/scripts/build-rhel-rpm.sh index 590243413..977c66110 100644 --- a/scripts/build-rhel-rpm.sh +++ b/scripts/build-rhel-rpm.sh @@ -1,13 +1,13 @@ #!/bin/sh # SPDX-License-Identifier: GPL-3.0-or-later -# Build an RPM package for merovingian 1.0.1 targeting RHEL 10 / AlmaLinux 10. +# Build an RPM package for merovingian 1.0.2 targeting RHEL 10 / AlmaLinux 10. # # Differs from build-rpm.sh only in the spec file used: packaging/rhel/merovingian.spec # omits catch-devel from BuildRequires (Catch2 is not reliably available in EPEL 10 # and is not needed when -Dbuild_tests=false). set -e -VERSION="1.0.1" +VERSION="1.0.2" mkdir -p "${HOME}/rpmbuild/BUILD" mkdir -p "${HOME}/rpmbuild/BUILDROOT" diff --git a/scripts/build-rpm.sh b/scripts/build-rpm.sh index 479f8dfd5..e23336ea5 100755 --- a/scripts/build-rpm.sh +++ b/scripts/build-rpm.sh @@ -1,9 +1,9 @@ #!/bin/sh # SPDX-License-Identifier: GPL-3.0-or-later -# Build an RPM package for merovingian 1.0.1 using rpmbuild. +# Build an RPM package for merovingian 1.0.2 using rpmbuild. set -e -VERSION="1.0.1" +VERSION="1.0.2" # 1. Create rpmbuild directory tree mkdir -p "${HOME}/rpmbuild/BUILD" diff --git a/scripts/build-static-linux.sh b/scripts/build-static-linux.sh index 50aaa6304..2f9620722 100755 --- a/scripts/build-static-linux.sh +++ b/scripts/build-static-linux.sh @@ -3,7 +3,7 @@ # Build a portable Linux fallback tarball with musl-linked static PIE binaries. set -eu -VERSION="${MEROVINGIAN_VERSION:-1.0.1}" +VERSION="${MEROVINGIAN_VERSION:-1.0.2}" BUILD_DIR="${BUILD_DIR:-build-static-linux}" STAGING="staging-static-linux" PACKAGE_ROOT="merovingian-${VERSION}-linux-static-x86_64" diff --git a/scripts/check-elf-hardening.sh b/scripts/check-elf-hardening.sh index 7abfc13b8..44a05dbc1 100644 --- a/scripts/check-elf-hardening.sh +++ b/scripts/check-elf-hardening.sh @@ -118,7 +118,13 @@ if [ "$static_binary" -eq 1 ]; then pass "BIND_NOW: skipped (--static binary has no PT_DYNAMIC section)" else dynamic_flags=$(readelf -d "$binary" 2>/dev/null || true) - if printf '%s\n' "$dynamic_flags" | grep -qE '\(BIND_NOW\)|Flags:.*\bNOW\b'; then + # Immediate binding is a DT_BIND_NOW entry, DF_BIND_NOW in DT_FLAGS or + # DF_1_NOW in DT_FLAGS_1. GNU readelf prints the last as "(FLAGS_1) Flags: NOW", + # the elftoolchain readelf of the BSDs as "(FLAGS_1) NOW" and DT_FLAGS as + # "(FLAGS) BIND_NOW", so match each tool's form. POSIX ERE only: no \b, which + # not every BSD grep supports (ADR-0757). + if printf '%s\n' "$dynamic_flags" | + grep -qE '\(BIND_NOW\)|\(FLAGS\)[[:space:]](.*[[:space:]])?BIND_NOW([[:space:]]|$)|\(FLAGS_1\)[[:space:]](.*[[:space:]])?NOW([[:space:]]|$)'; then pass "BIND_NOW: dynamic section advertises immediate binding" else fail "BIND_NOW: no BIND_NOW flag in the dynamic section; rebuild with -Wl,-z,now" diff --git a/scripts/remote-host-setup.sh b/scripts/remote-host-setup.sh new file mode 100644 index 000000000..13f154caa --- /dev/null +++ b/scripts/remote-host-setup.sh @@ -0,0 +1,163 @@ +#!/bin/sh +# SPDX-License-Identifier: GPL-3.0-or-later +# +# One-time setup of a remote build host, a VM or another machine, for +# `python build.py vm` (ADR-0752). On a Debian-family Linux host it installs the +# build packages with apt-get and a current Meson/Ninja in a virtual environment +# linked into ~/.local/bin, which build.py vm puts first on PATH. On FreeBSD it +# installs the packages the FreeBSD CI job installs, Meson and Ninja included, +# with pkg. Run it on the remote host, in the checkout build.py vm synced, as +# root or as a user who can use sudo or doas. It is not for WSL, which has +# scripts/wsl-setup.sh. + +set -eu + +tools_venv=${MEROVINGIAN_TOOLS_VENV:-"$HOME/.local/share/merovingian-tools"} +bin_dir=${MEROVINGIAN_TOOLS_BIN:-"$HOME/.local/bin"} +os_release=/etc/os-release +dry_run=0 + +usage() { + cat <<'EOF' +Usage: + sh scripts/remote-host-setup.sh [options] + +Options: + --tools-venv Python virtual environment for Meson/Ninja. + Default: ~/.local/share/merovingian-tools. + --bin-dir Directory for Meson/Ninja symlinks. Default: ~/.local/bin. + --os-release Distribution description to check. Default: /etc/os-release. + --dry-run Print commands without running them. + --help Show this help text. + +Supports Ubuntu, Debian, distributions whose ID_LIKE names either (for +example Linux Mint), and FreeBSD. On FreeBSD --tools-venv and --bin-dir are +unused: pkg's Meson is current. After setup, run python build.py vm locally. +EOF +} + +fail() { + printf '%s\n' "$1" >&2 + exit 1 +} + +quote_arg() { + case "$1" in + *[!abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ0123456789_./:=,+-]*) + printf "'%s'" "$(printf '%s' "$1" | sed "s/'/'\\\\''/g")" + ;; + *) + printf '%s' "$1" + ;; + esac +} + +print_command() { + printf '+' + for arg in "$@"; do + printf ' ' + quote_arg "$arg" + done + printf '\n' +} + +run() { + print_command "$@" + if [ "$dry_run" -eq 0 ]; then + "$@" + fi +} + +while [ "$#" -gt 0 ]; do + case "$1" in + --tools-venv) + shift + [ "$#" -gt 0 ] || fail "--tools-venv requires a path" + tools_venv=$1 + ;; + --bin-dir) + shift + [ "$#" -gt 0 ] || fail "--bin-dir requires a path" + bin_dir=$1 + ;; + --os-release) + shift + [ "$#" -gt 0 ] || fail "--os-release requires a file" + os_release=$1 + ;; + --dry-run) + dry_run=1 + ;; + --help) + usage + exit 0 + ;; + *) + fail "unknown option: $1" + ;; + esac + shift +done + +[ -r "$os_release" ] || fail "cannot read $os_release to detect the distribution" +# Read only the two fields needed, without sourcing the file. +os_id=$(sed -n 's/^ID=//p' "$os_release" | tr -d "\"'") +os_id_like=$(sed -n 's/^ID_LIKE=//p' "$os_release" | tr -d "\"'") +case " $os_id $os_id_like " in + *" ubuntu "*|*" debian "*) + family=debian + ;; + " freebsd "*) + family=freebsd + ;; + *) + fail "scripts/remote-host-setup.sh supports Ubuntu, Debian, their derivatives and FreeBSD only (ID=$os_id)" + ;; +esac + +if [ "$(id -u)" -eq 0 ]; then + as_root= +elif command -v sudo >/dev/null 2>&1; then + as_root=sudo +elif command -v doas >/dev/null 2>&1; then + as_root=doas +elif [ "$dry_run" -eq 1 ]; then + as_root=sudo +else + fail "run scripts/remote-host-setup.sh as root, or install sudo or doas" +fi + +if [ "$family" = freebsd ]; then + # The FreeBSD CI job's packages (.github/workflows/ci.yml), plus python3 for + # build.py vm's lock and the skip gate. pkg's Meson is new enough. + packages="meson ninja llvm catch2 pkgconf git perl5 bison flex gmake openssl libsodium postgresql17-client python3 curl png libjpeg-turbo" + # shellcheck disable=SC2086 + run $as_root pkg install -y $packages + cat <=1.11" ninja +run mkdir -p "$bin_dir" +run ln -sf "$tools_venv/bin/meson" "$bin_dir/meson" +run ln -sf "$tools_venv/bin/ninja" "$bin_dir/ninja" + +cat < std::vector validate_count("server.http.media_fetch_threads", config.server().http.media_fetch_threads, 256U); validate_count("server.http.media_fetch_max_in_flight", config.server().http.media_fetch_max_in_flight, 1024U); validate_count("server.http.media_fetch_max_per_client", config.server().http.media_fetch_max_per_client, 256U); + validate_count("server.http.client_outbound_max_per_client", config.server().http.client_outbound_max_per_client, + 128U); validate_count("server.http.max_start_line_bytes", config.server().http.max_start_line_bytes, 8192U); validate_count("server.http.max_header_bytes", config.server().http.max_header_bytes, 65536U); validate_count("server.http.max_header_count", config.server().http.max_header_count, 200U); diff --git a/src/config/ConfigParser.cpp b/src/config/ConfigParser.cpp index 17690aff8..f5daedf14 100644 --- a/src/config/ConfigParser.cpp +++ b/src/config/ConfigParser.cpp @@ -538,6 +538,14 @@ namespace add_parse_finding(findings, std::string{key}, "expected unsigned integer"); } } + // ADR-0751: client-triggered outbound calls one client address may have in flight. + else if (key == "server.http.client_outbound_max_per_client") + { + if (!parse_u32_value(value, server.http.client_outbound_max_per_client)) + { + add_parse_finding(findings, std::string{key}, "expected unsigned integer"); + } + } // Request-framing limits applied before any body is read. else if (key == "server.http.max_start_line_bytes") { diff --git a/src/config/ReloadPlan.cpp b/src/config/ReloadPlan.cpp index 74bc5f097..d6c4a18cf 100644 --- a/src/config/ReloadPlan.cpp +++ b/src/config/ReloadPlan.cpp @@ -863,6 +863,10 @@ auto build_reload_plan(Config const& current, Config const& next) -> ReloadPlan { add_change(plan, "server.http.media_fetch_max_per_client"); } + if (current.server().http.client_outbound_max_per_client != next.server().http.client_outbound_max_per_client) + { + add_change(plan, "server.http.client_outbound_max_per_client"); + } if (current.server().http.max_start_line_bytes != next.server().http.max_start_line_bytes) { add_change(plan, "server.http.max_start_line_bytes"); diff --git a/src/db_migrate.cpp b/src/db_migrate.cpp index 72468eb5d..a55a54ff3 100644 --- a/src/db_migrate.cpp +++ b/src/db_migrate.cpp @@ -23,7 +23,7 @@ namespace { -constexpr auto version = std::string_view{"1.0.1"}; +constexpr auto version = std::string_view{"1.0.2"}; auto print_help() -> void { diff --git a/src/federation/AGENTS.md b/src/federation/AGENTS.md index 5db21125b..ca9be9833 100644 --- a/src/federation/AGENTS.md +++ b/src/federation/AGENTS.md @@ -116,7 +116,7 @@ Federation is the highest-risk surface: **all input comes from untrusted remote checks, redacting on a content-hash mismatch); see `docs/event-engine.md`, "Phase B2". The membership acceptor differs from the `/send` path in one respect (F-10, F-12, ADR-0591): it refuses a rejected or soft-failed event `403` and does not store it, and it first refuses a room with no local member (`room_has_current_local_interest`; `send_knock` 404, the others - 403), so nothing is stored, recomputed, published or notified for either. A resent event is answered only while it is + 403; a `send_leave` from a remote user who holds a `join`, `invite` or `knock` row there is admitted, ADR-0747), so nothing is stored, recomputed, published or notified for either. A resent event is answered only while it is still the sender's accepted, current `m.room.member`. 6. **Never relay a remote server's answer about users unfiltered.** Keep only the users that @@ -388,10 +388,10 @@ Federation is the highest-risk surface: **all input comes from untrusted remote | `ServerAcl.cpp`, `ServerAclGlob.cpp` | Parses and evaluates `m.room.server_acl` allow/deny lists. The glob is a two-pointer match; an ACL over 4096 patterns or with a pattern over 255 bytes, or a check over its step budget, refuses the server; parsed ACLs are cached per event, keyed with the JSON size and hash (ADR-0640). A request that checks several rooms makes one `ServerAclBudget` and passes it to every `room_server_acl_allows` (the receipt sink per EDU, a worker relay per frame); answers per (ACL event version, host) are cached (ADR-0653) | | `RemoteDeviceListCache.cpp` | The in-memory device lists of remote users who share a room with a local user (G14, ADR-0560, ADR-0561): fill tickets, update sequencing by `stream_id`/`prev_id`, caps per user and in all. Guarded by `runtime.mutex` | | `EduIdempotenceWindow.cpp` | Bounded in-memory replay window for `m.direct_to_device` `message_id`s, keyed on (origin, sender, message_id), plus `derive_direct_to_device_message_id` for our outbound IDs | -| `DispatchWorker.cpp` | Background outbound PDU/EDU delivery with per-destination retry and back-off | +| `DispatchWorker.cpp` | Background outbound PDU/EDU delivery with per-destination retry and back-off; `make_dispatch_resolver` is its discovery, which never serves a remembered failure (ADR-0749) | | `EventQuery.cpp` | Serves the room-scoped reads `event`, `state`, `state_ids`, `backfill`, `get_missing_events`, `timestamp_to_event` (seeks the store room-and-time index, steps over events `ServerEventFilter` hides, at most `max_time_search_examined` candidates; ADR-0299, ADR-0300), and owns the origin-in-room gate (`origin_may_read_room`) | | `OutboundMembership.cpp` | Outbound `make_join` / `make_leave` / `make_knock` calls | -| `CachedServerDiscovery.cpp` | TTL-bounded in-memory cache in front of server discovery. `DiscoveryCachePolicy::successes_only` is what a call a client request triggers (the federated join's `make_join`/`send_join`) uses, so a remembered failure never refuses a retry; `remember_failures` stays the default for key resolution, pushers and appservices | +| `CachedServerDiscovery.cpp` | TTL-bounded in-memory cache in front of server discovery. `DiscoveryCachePolicy::successes_only` is what a call a client request triggers (the federated join's `make_join`/`send_join`) uses, so a remembered failure never refuses a retry, and so does outbound delivery, which its own backoff paces (ADR-0749); `remember_failures` stays the default for key resolution, pushers and appservices | | `RuntimeFederationConfig.cpp` | Federation route registration and per-origin request caps | ## Adding a new federation endpoint diff --git a/src/federation/DispatchWorker.cpp b/src/federation/DispatchWorker.cpp index 43ab31f3f..4923b7f83 100644 --- a/src/federation/DispatchWorker.cpp +++ b/src/federation/DispatchWorker.cpp @@ -3,12 +3,15 @@ #include "merovingian/federation/DispatchWorker.hpp" +#include "merovingian/federation/CachedServerDiscovery.hpp" +#include "merovingian/federation/DiscoveryCachePolicy.hpp" #include "merovingian/federation/DispatchDestinationSnapshot.hpp" #include "merovingian/federation/DispatchWorkerConfig.hpp" #include "merovingian/federation/DispatchWorkerSummary.hpp" #include "merovingian/federation/OutboundCall.hpp" #include "merovingian/federation/OutboundTransactionResult.hpp" #include "merovingian/federation/OutboundTransactionSupport.hpp" +#include "merovingian/federation/ServerDiscovery.hpp" #include "merovingian/observability/LogEventSeverity.hpp" #include "merovingian/observability/SingleLog.hpp" #include "merovingian/observability/StructuredLogField.hpp" @@ -307,6 +310,26 @@ auto DispatchWorker::enqueue_batch(std::vector batch) -> st return accepted; } +auto make_dispatch_resolver(ServerDiscoveryNetwork* network, CachedServerDiscovery* cached, + std::uint32_t timeout_seconds) -> DispatchResolver +{ + return [network, cached, timeout_seconds](std::string_view server_name) -> std::optional { + if (cached == nullptr && network == nullptr) + { + return std::nullopt; + } + // ADR-0749: never a remembered failure. The backoff already paces a failing destination's lookups. + auto result = cached != nullptr + ? cached->discover(server_name, timeout_seconds, DiscoveryCachePolicy::successes_only) + : discover_server(server_name, *network, timeout_seconds); + if (!result.discovery_allowed) + { + return std::nullopt; + } + return result; + }; +} + auto effective_max_pending_per_destination(DispatchWorkerConfig const& config) noexcept -> std::size_t { if (config.max_pending_per_destination != 0U) diff --git a/src/federation_worker/AGENTS.md b/src/federation_worker/AGENTS.md index ca20190ea..7b8887cf6 100644 --- a/src/federation_worker/AGENTS.md +++ b/src/federation_worker/AGENTS.md @@ -52,7 +52,8 @@ one exception to the `merovingian/` include-path rule in `src/AGENTS.md`. until the runtime exists, and `RuntimeStartOptions::fill_store` asks main for the room list (`room_list_request`) and then each room (`room_snapshot_request`, page by page through `ipc::RoomSnapshotAssembler`), one room at a time, before hydration. A room list main refuses stops the worker (the supervisor restarts it); a room main - cannot send is logged and left out until its next `room_sync`. Main answers only under a permit it issued + cannot send is logged and left out until its next `room_sync`, and reported to main as a `failed` + `room_sync_result` of generation 0, so main serves the room meanwhile (ADR-0756). Main answers only under a permit it issued ([ADR-0735](../../docs/adr/0735-a-worker-may-have-main-read-a-room-only-under-a-permit-main-issued-and-only-once.md)). Never give the worker a database path, URI, descriptor or Landlock grant again, and never add a worker read that does not come through main. `RuntimeStartOptions::database_load_profile` stays @@ -103,7 +104,8 @@ one exception to the `merovingian/` include-path rule in `src/AGENTS.md`. (`StoreReadGuard::reload_room_from`) asks main for the room at the round's generation, with no lock held, and takes the guard's exclusive lock only to swap the room's rows in memory; every provider that reads the store (ACL, room version, state, state_ids, event_auth, get_missing_events, backfill, timestamp_to_event, - hierarchy, the local make_leave/make_knock template) is wrapped by `wrap_store_readers` and holds the + hierarchy, the local make_leave/make_knock template, which main builds instead while this shard's reload of the + room is pending, ADR-0759) is wrapped by `wrap_store_readers` and holds the shared lock for its own call, returning values it owns. A relay transaction does NOT hold `runtime.mutex` across its round trips (`handle_federation_http_request` releases it before dispatch), so that mutex excludes nothing here. Wrap before the overrides that relay to main, never @@ -127,8 +129,8 @@ one exception to the `merovingian/` include-path rule in `src/AGENTS.md`. client. Do not move another call onto the tunnel here, and do not add a worker-to-main frame whose content main would act on without a signature check. The reply is validated in `WorkerPool::send_outbound_request`. 10. **The worker's ACL and policy view is advisory (F-21, F-24, ADR-0358, ADR-0359).** It holds no - policy rules on either backend (ADR-0733), and learns of a room only when it owns - its shard. Main judges the origin's rule and the room's ACL again on everything relayed, and the + policy rules on either backend (ADR-0733), and its copy of a room lags main's until + that room's reload lands (every shard reloads every changed room, ADR-0758). Main judges the origin's rule and the room's ACL again on everything relayed, and the worker does not run the policy-server hook: the proxy asked it once before forwarding. 11. **A relayed EDU must be the one main verified in the request body (F-74, ADR-0395).** The `edu_sink` override @@ -160,11 +162,13 @@ anything that does, above the hardening calls in `main.cpp`. Before the event loop opens the database or starts threads, `main.cpp` first calls `platform::apply_worker_landlock()` (ADR-0062 part 3; see Rules item 3 above), then `platform::apply_worker_hardening()` (gated by `federation.worker.apply_hardening`, default -`true`): `PR_SET_NO_NEW_PRIVS`, capability drop, and a worker seccomp profile that denies -`execve`/`execveat` (`clone`/`clone3` stay allowed because the worker runs thread pools). On -platforms with no in-process equivalent this fails closed — the worker refuses to start rather -than run unsandboxed (ADR-0041). ASan builds disable exit-time LeakSanitizer in the worker because -the seccomp profile denies the `ptrace` it needs. +`true`), which applies in order, each fail-closed: the core-dump policy (`RLIMIT_CORE` 0 and +`PR_SET_DUMPABLE` 0), `PR_SET_NO_NEW_PRIVS`, the capability-bounding-set drop, and a worker seccomp profile that +denies `execve`/`execveat` (`clone`/`clone3` stay allowed because the worker runs thread pools). +`apply_worker_landlock()` sets `PR_SET_NO_NEW_PRIVS` on its own before `landlock_restrict_self()`, because it runs +whether or not `apply_hardening` is on. On platforms with no in-process equivalent, +`apply_worker_hardening()` fails closed — the worker refuses to start rather than run unsandboxed (ADR-0041). +ASan builds disable exit-time LeakSanitizer in the worker because the seccomp profile denies the `ptrace` it needs. The worker filter also denies `kill`, `tkill` and `setrlimit`, allows `tgkill` only on the worker's own thread group and `prlimit64` only to read its own limits (ISO-2, diff --git a/src/federation_worker/WorkerEventLoop.cpp b/src/federation_worker/WorkerEventLoop.cpp index d746c7d9a..83fc34e85 100644 --- a/src/federation_worker/WorkerEventLoop.cpp +++ b/src/federation_worker/WorkerEventLoop.cpp @@ -655,7 +655,8 @@ namespace } // The start-up fill. False only when main will not list its rooms: the worker then does not start, and the - // supervisor restarts it. A room main cannot send is logged and left out; its next room_sync fills it. + // supervisor restarts it. A room main cannot send is logged and left out, and reported to main as a failed result + // of generation 0, which main never assigns, so main serves the room until its next room_sync fills it (ADR-0756). [[nodiscard]] auto fill_store_from_main(ipc::IpcChannel& channel, database::PersistentStore& store) -> bool { auto const room_ids = room_ids_from_main(channel); @@ -672,6 +673,11 @@ namespace if (!snapshot.has_value()) { LOG_ERROR("Federation worker: room_id=" + room_id + " left out of the start-up copy"); + if (!channel.send_notification(ipc::serialize_room_sync_result(room_id, "failed", 0U))) + { + LOG_WARNING("Federation worker: could not tell main that room_id=" + room_id + + " was left out of the start-up copy"); + } continue; } snapshots.push_back(std::move(*snapshot)); @@ -932,9 +938,11 @@ auto WorkerEventLoop::run() -> void // ADR-0185). In a restricted room the template names an authorising user // only after the allow conditions are checked, and those conditions name // OTHER rooms, whose state this shard's snapshot may not hold or may hold - // stale (room_sync refreshes only the shard that owns a room). Main holds + // stale (a room_sync reload lags behind main's commit). Main holds // every room, so it builds the template. make_leave and make_knock need - // nothing outside the room being left or knocked on, so they stay local. + // nothing outside the room being left or knocked on, so they stay local; + // while this shard's reload of the room is pending, main builds them + // instead, so they name an invite main sent a moment ago (ADR-0759). runtime.federation.membership_template_provider = [channel_ptr, local_provider = runtime.federation.membership_template_provider]( federation::FederationEndpoint endpoint, std::string_view room_id, std::string_view user_id, diff --git a/src/homeserver/AGENTS.md b/src/homeserver/AGENTS.md index 934b75f1b..e1c7c7d84 100644 --- a/src/homeserver/AGENTS.md +++ b/src/homeserver/AGENTS.md @@ -56,9 +56,9 @@ This is the largest and most complex module — read this file carefully before | `TurnCredentials.cpp` | `make_turn_credentials`: the coturn REST API username and password for `GET /voip/turnServer` (ADR-0480) | | `EmailPush.cpp` | Kind `"email"` pushers (ADR-0219, ADR-0220): `owned_email_pusher_address` (the pushkey must be the account's own email 3PID), `remove_email_pushers_for_address`, and `EmailPushJob`, the one thread that sends due digests through `send_due_email_notifications`. It re-reads the pusher, the 3PID, the account and the badge count under `runtime.mutex` (one hold, one `PushBadgeBudget`: rows and glob steps bounded together, ADR-0722), then sends with the mutex released | | `RuntimeSigningKeyStore.cpp` | Production `SigningKeyStore` over the persisted server signing-key rows | -| `FederationProxy.cpp` | Forwards inbound federation requests to the out-of-process federation worker over encrypted IPC; `GET /_matrix/key/v2/server` stays local | +| `FederationProxy.cpp` | Forwards inbound federation requests to the out-of-process federation worker over encrypted IPC; `GET /_matrix/key/v2/server` stays local. A room whose reload failed is served by main, and so are its reads while a reload is pending (`federation_room_request_served_by_main`, ADR-0748); so is every request for a room the worker has never loaded (`RoomSyncStatus::held`, ADR-0756), and a `/send` is checked for every room it names (`federation_request_room_ids`), each on the copy held by the shard it is sent to (`room_sync_status(shard, room)`). Every shard reloads every changed room (ADR-0758). Main serves a request only for the selected shard's copy of a room it names (failed, pending read, or never loaded); never route a request to main because its rooms belong to different shards, which a remote server chooses (ADR-0131) | | `FederationRequestRouting.cpp` | Extracts the room ID from an inbound federation request for worker shard routing | -| `WorkerPool.cpp` | Federation worker shard selection (FNV-1a of the room ID) and the main-process side of worker IPC | +| `WorkerPool.cpp` | Federation worker shard selection (FNV-1a of the room ID) and the main-process side of worker IPC. `notify_room_changed` tells every shard and room-sync status is kept per shard (ADR-0758) | | `WorkerSnapshotService.cpp` | Main's answers to a federation worker's store requests (room list, room snapshot pages, public server keys; ADR-0733). A room is read only under a permit main issued on that channel, consumed once (ADR-0735); no lock and never `runtime.mutex` across a read. It reads through `database::SnapshotReaderPool`, whose readers `WorkerPool` opens before `cap_enter` (ADR-0739) | | `WorkerSupervisor.cpp` | Spawns and monitors the federation worker child; restarts it with exponential back-off, whether it exited or its main-side channel failed while it ran (it kills a live worker whose channel is unhealthy, ADR-0323) | | `MinimalWorkerEnv.cpp` | Minimal environment allowlist for the worker child (ADR-0042) | @@ -320,9 +320,9 @@ that swaps the network falls back to the raw network), and it must never serve a (a retried join after the peer recovers is not refused). Keep the lookup inside the `RuntimeLockRelease` scope. -The caps derive from the main request pool size (`server.http.request_threads`, applied by -`start_runtime` through `client_outbound_proxy_policy_for_pool`); they have no configuration of their -own. A new +The global cap derives from the main request pool size (`server.http.request_threads`) and the +per-client cap is `server.http.client_outbound_max_per_client` (default 3), clamped to the global cap; +`start_runtime` applies both through `client_outbound_proxy_policy_for_pool` (ADR-0751). A new route that proxies to a remote server without one of these three is the defect. See ADR-0079 and `docs/http-transport.md` "Client-triggered outbound proxying". @@ -474,6 +474,14 @@ other servers with joined members; never this server), offering only the room's and the membership is `join`. Any other stored copy leaves the join failed. - If no candidate succeeds the client gets the original 403, not an upstream error. +### A first restricted join takes the resident's co-signature the same way (ADR-0745) + +When this server holds nothing of the room, `perform_federated_join`'s join names an authorising user of the resident's +server whenever the make_join template did. It then calls `verify_resident_signed_join` on the answer's `event`, and the +verified combined event is what `authorize_send_join_answer` judges and what `join_room` stores. A missing, unsigned or +wrongly signed `event` is 502 with nothing stored. Never authorise or store this server's own copy of such a join: it +lacks the signature rule 4.3.5 requires, and every other server rejects it. + ## A resident's answer to our send_join is a receipt (F-45, ADR-0369) `perform_federated_join` runs the answer through `filter_verified_send_join_events` (room, signatures, then the content @@ -509,7 +517,8 @@ answered again from current state (`resent_send_join_state`, ADR-0372), never fr date from `make_join` and may predate a ban, but only while it is stored `accepted` and is the sender's current `m.room.member` state, otherwise 403 with no state (F-10, ADR-0591); it is never stored or sent on a second time. The same test covers a resent leave and knock. Before any of this the acceptor refuses a room with no local member -(`membership_admission_refusal`: `send_knock` 404, the others 403), then an event that is not the endpoint's membership +(`membership_admission_refusal`: `send_knock` 404, the others 403), except a `send_leave` from a remote user who holds a +`join`, `invite` or `knock` row there (`remote_member_may_leave`, ADR-0747: one stored leave per row), then an event that is not the endpoint's membership about its own sender from a user of the origin (`membership_agreement_refusal`, 400 `M_INVALID_PARAM`; it runs before the resend answer, so a stored leave relayed as `send_knock` is never answered), and an event that auth rejects or soft-fails is refused (`membership_outcome_refusal`, 403) and never stored (F-12). Both run in the acceptor, which the direct path and the worker relay share. `servers_in_room` is "before the join", so it counts the joiner only if the answer's state has them joined. Without `omit_members` the auth chain still repeats the state events; the spec allows the @@ -841,7 +850,9 @@ client, sender or remote server controls, and per-user rows that requests scan m under `RuntimeLockRelease{guard}`); the state array and the member rows in one each, with the lock held, because they write `current_state` and membership rows that a local send, an invite or a knock could also write. Release for them only after every such writer checks a per-room claim. Never commit an answer-sized loop of rows one by one. -- A resident's make_join, make_leave or make_knock template is claimed (`claim_membership_event_template`) and then its +- A resident's make_join, make_leave or make_knock template is claimed (`claim_membership_event_template`: our `origin` + and `origin_server_ts`; its `event_id`, `unsigned`, `hashes` and `signatures` dropped, since signing adds to the + signatures an event holds; ADR-0745) and then its content is rebuilt from an allow-list (`restrict_membership_template_content`): `membership`, and for a join the authorising user of another server. A `reason` or profile in the template is never signed as the user's; the user's own reason (knock) and profile (join) are added after. A join template naming a user of this server is refused (C18, ADR-0709). diff --git a/src/homeserver/ClientServerRuntime.cpp b/src/homeserver/ClientServerRuntime.cpp index 1e8f499a0..19fcb4ae7 100644 --- a/src/homeserver/ClientServerRuntime.cpp +++ b/src/homeserver/ClientServerRuntime.cpp @@ -5172,8 +5172,11 @@ namespace } // Whether `thread_id` names a thread root of `room_id` (an event in the main timeline) and the event `event_id` - // is that root or belongs to its thread (ADR-0737). An event this server does not hold in the room is related to - // nothing: the endpoint answers 400 for it. + // is related to it (ADR-0737, ADR-0746): it is that root, it belongs to its thread, or its relation chain reaches + // the root within the 3 hops of the thread rule, whatever the relation types (an annotation of a reference to + // the root). Spec (POST /receipt, 400): "The event_id is not related to the thread_id." An event has one + // relation chain, so it is related to at most three roots besides its own thread. An event this server does not + // hold in the room is related to nothing: the endpoint answers 400 for it. [[nodiscard]] auto receipt_event_is_in_thread(database::PersistentStore const& store, std::string_view room_id, std::string_view event_id, std::string_view thread_id) -> bool { @@ -5186,8 +5189,19 @@ namespace { return true; } - auto const event_thread = receipt_thread_of_event(store, room_id, event_id); - return event_thread.has_value() && *event_thread == thread_id; + auto const event = store.events.find_in_room(room_id, event_id); + if (event == store.events.end()) + { + return false; + } + auto const parsed = canonicaljson::parse_lossless(event->json); + auto const* object = std::get_if(&parsed.value.storage()); + if (parsed.error != canonicaljson::ParseError::none || object == nullptr) + { + return false; + } + return sync::event_thread_id(store, *object) == thread_id || + sync::event_relates_to(store, room_id, *object, thread_id); } // Sets one of the user's receipts and wakes /sync. Spec (Receipts): "Each user_id, diff --git a/src/homeserver/FederationProxy.cpp b/src/homeserver/FederationProxy.cpp index 6edc4dcb9..75c054cf2 100644 --- a/src/homeserver/FederationProxy.cpp +++ b/src/homeserver/FederationProxy.cpp @@ -4,17 +4,17 @@ #include "merovingian/homeserver/FederationProxy.hpp" #include "merovingian/core/QueryParams.hpp" -#include "merovingian/federation/SignedFederationRequest.hpp" #include "merovingian/federation/InboundRequest.hpp" +#include "merovingian/federation/SignedFederationRequest.hpp" #include "merovingian/homeserver/ClientServerClock.hpp" #include "merovingian/homeserver/ClientServerRuntime.hpp" +#include "merovingian/homeserver/FederationRequestRouting.hpp" #include "merovingian/homeserver/HomeserverRuntime.hpp" #include "merovingian/homeserver/LocalHttpRequest.hpp" #include "merovingian/homeserver/LocalHttpResponse.hpp" +#include "merovingian/homeserver/LocalHttpRouter.hpp" #include "merovingian/homeserver/RoomSyncStatus.hpp" #include "merovingian/homeserver/WorkerPool.hpp" -#include "merovingian/homeserver/FederationRequestRouting.hpp" -#include "merovingian/homeserver/LocalHttpRouter.hpp" #include "merovingian/observability/LogEventSeverity.hpp" #include "merovingian/observability/SingleLog.hpp" @@ -219,18 +219,28 @@ auto FederationProxy::handle(LocalHttpRequest const& request) -> LocalHttpRespon } auto const room_id = federation_worker_room_id_from_request(request); - if (!room_id.empty()) + // The shard the request goes to: the one its first room picks, shard 0 for a request with no usable room key + // (ADR-0131). Its worker's copy of every room the request names is what answers, so that copy is judged. + auto const selected_shard = pool_->shard_for(room_id); + // A failed reload leaves the worker's copy of the room stale; a pending one has not yet taken in a change main + // made, such as an event a client just sent. Main answers then (ADR-0748), and for a room the worker has never + // loaded (ADR-0756). A /send goes whole to the selected shard and may name rooms other shards own; every shard + // reloads every changed room, so each of those rooms is judged on the selected shard's copy (ADR-0758). + for (auto const& concerned_room_id : federation_request_room_ids(request)) { - auto const status = pool_->room_sync_status(room_id); - if (status.state == "failed") + auto const status = pool_->room_sync_status(selected_shard, concerned_room_id); + if (federation_room_request_served_by_main(verified_request, status)) { - observability::log_diagnostic("federation_proxy", "federation_proxy.room_sync_failed_fallback", - { - {"target", observability::sanitized_http_target(request.target), false}, - {"room_id", std::string{room_id}, false}, - {"generation", std::to_string(status.generation), false}, + auto const failed = status.state == "failed"; + observability::log_diagnostic( + "federation_proxy", + failed ? "federation_proxy.room_sync_failed_fallback" : "federation_proxy.room_sync_pending_main", + { + {"target", observability::sanitized_http_target(request.target), false}, + {"room_id", concerned_room_id, false}, + {"generation", std::to_string(status.generation), false}, }, - observability::LogEventSeverity::warning); + failed ? observability::LogEventSeverity::warning : observability::LogEventSeverity::debug); return handle_federation_http_request(runtime_, verified_request); } } @@ -276,6 +286,12 @@ auto FederationProxy::healthy() const noexcept -> bool return pool_ != nullptr && pool_->healthy(); } +// The test accessors below rely on the constructor: pool_ is set there and never cleared. +auto FederationProxy::worker_pid_for_room(std::string_view room_id) const noexcept -> pid_t +{ + return pool_->worker_pid(pool_->shard_for(room_id)); +} + auto FederationProxy::room_sync_status(std::string_view room_id) const -> RoomSyncStatus { if (pool_ == nullptr) @@ -285,4 +301,14 @@ auto FederationProxy::room_sync_status(std::string_view room_id) const -> RoomSy return pool_->room_sync_status(room_id); } +auto FederationProxy::room_sync_status(std::size_t shard, std::string_view room_id) const -> RoomSyncStatus +{ + return pool_->room_sync_status(shard, room_id); +} + +auto FederationProxy::shard_for_room(std::string_view room_id) const noexcept -> std::size_t +{ + return pool_->shard_for(room_id); +} + } // namespace merovingian::homeserver diff --git a/src/homeserver/FederationRequestRouting.cpp b/src/homeserver/FederationRequestRouting.cpp index dcae8d64c..8963d7807 100644 --- a/src/homeserver/FederationRequestRouting.cpp +++ b/src/homeserver/FederationRequestRouting.cpp @@ -4,12 +4,13 @@ #include "merovingian/homeserver/FederationRequestRouting.hpp" #include "merovingian/canonicaljson/ParseError.hpp" -#include "merovingian/canonicaljson/Value.hpp" #include "merovingian/canonicaljson/Parser.hpp" +#include "merovingian/canonicaljson/Value.hpp" #include "merovingian/core/QueryParams.hpp" #include "merovingian/federation/Transactions.hpp" #include "merovingian/homeserver/ClientServerRuntime.hpp" +#include #include #include #include @@ -151,6 +152,40 @@ namespace return *room_id; } + // Every distinct room_id the PDUs of a /send body name, in the order they first appear. A PDU without a string + // room_id names none: the worker refuses it on its own. At most max_federation_transaction_pdus are collected, + // the most a transaction the worker accepts can carry. + [[nodiscard]] auto room_ids_from_send_body(std::string_view body) -> std::vector + { + auto room_ids = std::vector{}; + auto const parsed = canonicaljson::parse_json(body); + if (parsed.error != canonicaljson::ParseError::none) + { + return room_ids; + } + auto const* root = std::get_if(&parsed.value.storage()); + auto const* pdus = root == nullptr ? nullptr : array_member(*root, "pdus"); + if (pdus == nullptr) + { + return room_ids; + } + for (auto const& pdu : *pdus) + { + auto const* object = std::get_if(&pdu.storage()); + auto const* room_id = object == nullptr ? nullptr : string_member(*object, "room_id"); + if (room_id == nullptr || std::ranges::find(room_ids, *room_id) != room_ids.end()) + { + continue; + } + if (room_ids.size() == federation::max_federation_transaction_pdus) + { + break; + } + room_ids.push_back(*room_id); + } + return room_ids; + } + [[nodiscard]] auto room_id_from_path_target(std::string_view target) -> std::string { for (auto const prefix : room_endpoint_prefixes()) @@ -195,6 +230,20 @@ auto federation_worker_room_id_from_request(LocalHttpRequest const& request) -> return room_id_from_path_target(request.target); } +auto federation_request_room_ids(LocalHttpRequest const& request) -> std::vector +{ + if (is_federation_send_target(request.target)) + { + return room_ids_from_send_body(request.body); + } + auto room_id = room_id_from_path_target(request.target); + if (room_id.empty()) + { + return {}; + } + return {std::move(room_id)}; +} + auto federation_request_should_bypass_worker(LocalHttpRequest const& request) -> bool { // EDU-only /send transactions relay back to main's edu_sink anyway. @@ -230,21 +279,45 @@ auto federation_request_should_bypass_worker(LocalHttpRequest const& request) -> auto federation_worker_room_read_may_fallback_to_main(LocalHttpRequest const& request) noexcept -> bool { + auto const path = request.target.substr(0U, request.target.find('?')); + // v1.19 defines get_missing_events as POST, though it only reads: its body names the events the asker has. + if (request.method == "POST") + { + return path.starts_with("/_matrix/federation/v1/get_missing_events/"); + } if (request.method != "GET") { return false; } - auto const path = request.target.substr(0U, request.target.find('?')); static auto const prefixes = std::vector{ - "/_matrix/federation/v1/state/", "/_matrix/federation/v1/state_ids/", - "/_matrix/federation/v1/backfill/", "/_matrix/federation/v1/get_missing_events/", - "/_matrix/federation/v1/event_auth/", "/_matrix/federation/v1/timestamp_to_event/", + "/_matrix/federation/v1/state/", + "/_matrix/federation/v1/state_ids/", + "/_matrix/federation/v1/backfill/", + "/_matrix/federation/v1/event_auth/", + "/_matrix/federation/v1/timestamp_to_event/", + // A make_leave or make_knock template names the user's current membership as an auth event, so a stale copy + // leaves out an invite main sent a moment ago and main refuses the send_leave built on it (ADR-0759). + // make_join needs no entry: the worker always has main build its template (ADR-0184). + "/_matrix/federation/v1/make_leave/", + "/_matrix/federation/v1/make_knock/", }; return std::ranges::any_of(prefixes, [&path](std::string_view prefix) { return path.starts_with(prefix); }); } +auto federation_room_request_served_by_main(LocalHttpRequest const& request, RoomSyncStatus const& status) noexcept + -> bool +{ + // A worker with no copy of the room judges it as unknown: it takes room version 12, so it computes another event ID + // than the sender's and refuses its send_join or PDU (ADR-0756). Main serves such a room as it does a failed one. + if (status.state == "failed" || !status.held) + { + return true; + } + return status.state == "pending" && federation_worker_room_read_may_fallback_to_main(request); +} + auto is_federation_key_server_endpoint(std::string_view target) noexcept -> bool { // The key server endpoint is the only inbound federation route that main diff --git a/src/homeserver/HomeserverRuntime.cpp b/src/homeserver/HomeserverRuntime.cpp index ac7cac310..7bc27f38c 100644 --- a/src/homeserver/HomeserverRuntime.cpp +++ b/src/homeserver/HomeserverRuntime.cpp @@ -10,33 +10,33 @@ #include "merovingian/canonicaljson/ParseError.hpp" #include "merovingian/canonicaljson/Parser.hpp" #include "merovingian/canonicaljson/Serializer.hpp" +#include "merovingian/crypto/GenericHash.hpp" #include "merovingian/crypto/RefusingEd25519Provider.hpp" #include "merovingian/crypto/RuntimeEd25519Provider.hpp" #include "merovingian/crypto/RuntimeMultiKeyEd25519Provider.hpp" -#include "merovingian/crypto/GenericHash.hpp" #include "merovingian/database/PostgresqlStore.hpp" -#include "merovingian/federation/RuntimeFederationConfig.hpp" #include "merovingian/federation/InboundRequest.hpp" +#include "merovingian/federation/RuntimeFederationConfig.hpp" #include "merovingian/federation/ServerDiscovery.hpp" +#include "merovingian/homeserver/AppserviceEphemeral.hpp" +#include "merovingian/homeserver/AuthService.hpp" +#include "merovingian/homeserver/ClientOutboundProxy.hpp" #include "merovingian/homeserver/FederationProxy.hpp" #include "merovingian/homeserver/InboundTypingUser.hpp" #include "merovingian/homeserver/KeyServerCache.hpp" #include "merovingian/homeserver/LocalDatabase.hpp" #include "merovingian/homeserver/LocalDatabaseScope.hpp" #include "merovingian/homeserver/LocalRoom.hpp" +#include "merovingian/homeserver/LocalServices.hpp" #include "merovingian/homeserver/LocalUser.hpp" #include "merovingian/homeserver/OutboundDeadline.hpp" +#include "merovingian/homeserver/RedactionService.hpp" +#include "merovingian/homeserver/RoomService.hpp" #include "merovingian/homeserver/RuntimeLockRelease.hpp" #include "merovingian/homeserver/RuntimeStartOptions.hpp" #include "merovingian/homeserver/RuntimeStartResult.hpp" -#include "merovingian/homeserver/TypingExpiry.hpp" -#include "merovingian/homeserver/AppserviceEphemeral.hpp" -#include "merovingian/homeserver/AuthService.hpp" -#include "merovingian/homeserver/ClientOutboundProxy.hpp" -#include "merovingian/homeserver/LocalServices.hpp" -#include "merovingian/homeserver/RedactionService.hpp" -#include "merovingian/homeserver/RoomService.hpp" #include "merovingian/homeserver/StreamTokenEpoch.hpp" +#include "merovingian/homeserver/TypingExpiry.hpp" #include "merovingian/media/LocalMediaBlob.hpp" #include "merovingian/media/LocalMediaRecord.hpp" #include "merovingian/media/LocalMediaRepository.hpp" @@ -51,12 +51,12 @@ #include "merovingian/observability/MetricLabel.hpp" #include "merovingian/observability/MetricSample.hpp" #include "merovingian/observability/MetricType.hpp" +#include "merovingian/observability/Observability.hpp" #include "merovingian/observability/SingleLog.hpp" #include "merovingian/observability/StructuredLogField.hpp" -#include "merovingian/observability/Observability.hpp" +#include "merovingian/platform/FileMetadataSupport.hpp" #include "merovingian/platform/HardeningPlanDecision.hpp" #include "merovingian/platform/HardeningSelfCheck.hpp" -#include "merovingian/platform/FileMetadataSupport.hpp" #include "merovingian/platform/RuntimeHardening.hpp" #include "merovingian/trust_safety/PolicyEngine.hpp" @@ -1013,7 +1013,8 @@ auto start_runtime(RuntimeStartOptions opts) -> RuntimeStartResult config.security().password_hashing.max_concurrent, 1U, auth::argon2id_admission_max_capacity)); // ADR-0079 caps are a fraction of the main request pool, which is now // configured (server.http.request_threads, ADR-0077). - runtime.client_outbound_proxy_policy = client_outbound_proxy_policy_for_pool(config.server().http.request_threads); + runtime.client_outbound_proxy_policy = client_outbound_proxy_policy_for_pool( + config.server().http.request_threads, config.server().http.client_outbound_max_per_client); runtime.listeners = net::make_runtime_listeners(config); if (runtime.listeners.empty()) { @@ -1603,8 +1604,8 @@ auto resolve_policy_server_hook(HomeserverRuntime& runtime, trust_safety::Policy // unreachable server is still judged by policy_server_allow_without_result. if (leader.has_value()) { - if (auto remembered = runtime.policy_decision_cache->stale(entity, std::chrono::steady_clock::now(), - cache_settings); + if (auto remembered = + runtime.policy_decision_cache->stale(entity, std::chrono::steady_clock::now(), cache_settings); remembered.has_value()) { return decision_with_current_config(std::move(*remembered)); diff --git a/src/homeserver/LocalHttpRouter.cpp b/src/homeserver/LocalHttpRouter.cpp index 93bf22c7b..cc43500fb 100644 --- a/src/homeserver/LocalHttpRouter.cpp +++ b/src/homeserver/LocalHttpRouter.cpp @@ -1326,20 +1326,56 @@ namespace }); } + // ADR-0747: whether the sender of `envelope`, a user of another server, holds a membership in `room_id` that a + // leave of their own ends: `join`, `invite` or `knock`. One leave turns the row into `leave`, so each such row + // admits one stored leave, never one per request. The leave that did so is admitted again while it is still the + // sender's current membership, so a retry after a lost answer reaches the resend branch, which answers it without + // storing anything (ADR-0591 point 3); any other leave is refused here. + [[nodiscard]] auto remote_member_may_leave(HomeserverRuntime const& runtime, std::string_view room_id, + federation::InboundPduEnvelope const& envelope) -> bool + { + auto const& user_id = envelope.sender; + if (user_id.empty() || server_name_from_user_id(user_id) == runtime.config.server().server_name) + { + return false; + } + auto const& store = runtime.database.persistent_store; + auto const* const membership = database::find_membership_row(store, room_id, user_id); + if (membership != nullptr && (membership->membership == "join" || membership->membership == "invite" || + membership->membership == "knock")) + { + return true; + } + auto const current_membership_id = + database::find_current_state_event_id(store, room_id, "m.room.member", user_id); + return current_membership_id.has_value() && *current_membership_id == envelope.event_id; + } + // F-12 (ADR-0591): the admission gate of send_join, send_leave and send_knock, before anything is stored or any // state is read for an answer. A room this server has no current local interest in is not one it can act as the // resident of, so a request for it is refused with nothing written, as the /send path refuses an unsolicited // room. Spec "Knocking upon a room" lists 404 for "the room ... is unknown to the receiving server"; send_join and // send_leave list no code for this case, so those are 403 M_FORBIDDEN. The membership acceptor is the one place // this runs, and it is the callback both the direct path and main's side of the worker relay call. + // The one exception (ADR-0747, superseding part of ADR-0591 point 1): a send_leave whose sender this server holds + // as joined, invited or knocking. This server is still the resident other servers name for that user (it sent + // the invite, or the room ID names it), the spec lists no refusal for a room its last member left ("Leaving Rooms + // (Rejecting Invites)"), and refusing left the user unable to leave at all. The row bounds what is stored, the + // agreement and outcome gates still run, and send_leave carries no state, so F-10 and F-12 stay closed. [[nodiscard]] auto membership_admission_refusal(HomeserverRuntime const& runtime, - federation::FederationEndpoint endpoint, std::string_view room_id) + federation::FederationEndpoint endpoint, std::string_view room_id, + federation::InboundPduEnvelope const& envelope) -> std::optional { if (room_has_current_local_interest(runtime, room_id)) { return std::nullopt; } + if (endpoint == federation::FederationEndpoint::send_leave && + remote_member_may_leave(runtime, room_id, envelope)) + { + return std::nullopt; + } if (endpoint == federation::FederationEndpoint::send_knock) { return federation::MembershipAcceptResult{ @@ -2455,7 +2491,8 @@ namespace } // F-12 (ADR-0591): admission first, before anything is stored or any state is read for an // answer. The one gate the direct path and main's side of the worker relay share. - if (auto const refusal = membership_admission_refusal(*rt, endpoint, room_id); refusal.has_value()) + if (auto const refusal = membership_admission_refusal(*rt, endpoint, room_id, envelope); + refusal.has_value()) { return *refusal; } @@ -3554,22 +3591,10 @@ namespace // SecretBuffer rather than an unpinned std::string. The runtime // retains its own SecretBuffer; both are wiped independently. dispatch_config.secret_key = core::SecretBuffer{runtime.database.signing_secret_key.bytes()}; - auto* discovery_ptr = discovery; - auto* cached_ptr = cached; auto const discovery_timeout = std::min(timeout > 0U ? timeout : 30U, federation::dispatch_discovery_timeout_seconds); - auto resolver = [discovery_ptr, cached_ptr, discovery_timeout]( - std::string_view server_name) -> std::optional { - // Prefer the TTL cache; fall back to the raw network for tests. - auto result = cached_ptr != nullptr - ? cached_ptr->discover(server_name, discovery_timeout) - : federation::discover_server(server_name, *discovery_ptr, discovery_timeout); - if (!result.discovery_allowed) - { - return std::nullopt; - } - return result; - }; + // A remembered discovery failure is never served to the dispatch worker (ADR-0749). + auto resolver = federation::make_dispatch_resolver(discovery, cached, discovery_timeout); auto clock = []() -> std::uint64_t { return static_cast(std::chrono::duration_cast( std::chrono::system_clock::now().time_since_epoch()) diff --git a/src/homeserver/RoomService.cpp b/src/homeserver/RoomService.cpp index 7c1449503..324cbbac4 100644 --- a/src/homeserver/RoomService.cpp +++ b/src/homeserver/RoomService.cpp @@ -5289,6 +5289,8 @@ namespace { std::string json{}; std::string refusal{}; + // The verified combined event `json` serializes. + canonicaljson::Value value{}; }; [[nodiscard]] auto verify_resident_signed_join(HomeserverRuntime& runtime, canonicaljson::Object const& answer, @@ -5348,7 +5350,7 @@ namespace { return {{}, "the signed join event could not be serialized"}; } - return {std::move(serialized.output), {}}; + return {std::move(serialized.output), {}, verified.front()}; } [[nodiscard]] auto federated_join_failure(std::string reason, std::uint16_t status) -> FederatedJoinOutcome @@ -5436,6 +5438,22 @@ namespace MakeJoinRaceState& state_; }; + // Logs the refusal of the resident's co-signed copy of this server's join, which the caller answers 502. Shared + // by a join through another resident and a first restricted join (ADR-0745). + auto log_resident_join_refusal(std::string_view user_id, std::string_view room_id, std::string_view remote_server, + std::string_view refusal) -> void + { + log_diagnostic("room.join.rejected", + { + {"actor", std::string{user_id}, false}, + {"room_id", std::string{room_id}, false}, + {"status", "502", false}, + {"remote_server", std::string{remote_server}, false}, + {"reason", std::string{refusal}, false} + }, + observability::LogEventSeverity::warning); + } + [[nodiscard]] auto perform_federated_join( HomeserverRuntime& runtime, std::string_view room_id, std::string const& user_id, std::string const& our_server, std::vector candidates, std::vector const& supported_versions, @@ -5717,6 +5735,18 @@ namespace }); return federated_join_failure("event_id computation failed", 500U); } + // The join is stored with signature rows parsed from it (join_room). One that does not parse is refused before + // it is sent, so this server never sends a join it could not store whole. + if (auto const parsed_join = events::parse_event_envelope(signed_value.value); !parsed_join.error.empty()) + { + log_diagnostic("room.join.rejected", { + {"actor", user_id, false}, + {"room_id", std::string{room_id}, false}, + {"status", "502", false}, + {"reason", parsed_join.error, false} + }); + return federated_join_failure("make_join template refused: " + parsed_join.error, 502U); + } // Send the signed join event via send_join. auto send_join_tx = federation::make_outbound_send_membership( federation::FederationEndpoint::send_join, remote_server, our_server, room_id, event_id_result.event_id, @@ -5782,15 +5812,7 @@ namespace event_id_result.event_id, signed_value.value); if (!resident_join.refusal.empty()) { - log_diagnostic("room.join.rejected", - { - {"actor", user_id, false}, - {"room_id", std::string{room_id}, false}, - {"status", "502", false}, - {"remote_server", remote_server, false}, - {"reason", resident_join.refusal, false} - }, - observability::LogEventSeverity::warning); + log_resident_join_refusal(user_id, room_id, remote_server, resident_join.refusal); return federated_join_failure(resident_join.refusal, 502U); } auto outcome = FederatedJoinOutcome{}; @@ -5802,6 +5824,28 @@ namespace outcome.resident_signed_event_json = std::move(resident_join.json); return outcome; } + // ADR-0745: a join into a restricted room this server holds nothing of names an authorising user of the + // resident's server, and the auth rules require that server's signature on the join (rooms/v10.md 4.2.1). + // The resident adds it to the copy it returns as the answer's `event` (send_join v2, "event": "Required if + // the room is restricted and the joining user is authorised by one of the conditions"). The co-signature is + // taken exactly as for a join through another resident: only the authorising server's verified signatures + // are merged into this server's own signed join, and the combined event must verify. That combined event is + // the one the answer is authorised against and the one stored. A missing, altered or unverifiable copy is + // refused: a join no checked signature authorises is never stored. + auto join_signed_event = std::move(signed_event); + auto join_value = signed_value.value; + if (!join_authorising_user_of(signed_value.value).empty()) + { + auto resident_join = verify_resident_signed_join(runtime, *send_obj, *policy, our_server, + event_id_result.event_id, signed_value.value); + if (!resident_join.refusal.empty()) + { + log_resident_join_refusal(user_id, room_id, remote_server, resident_join.refusal); + return federated_join_failure(resident_join.refusal, 502U); + } + join_signed_event.event_json = std::move(resident_join.json); + join_value = std::move(resident_join.value); + } // ADR-0241: the whole state array — other members' m.room.member events // included — is verified here and persisted by join_room before the join // returns. The join event's after-state group is built from it, and the @@ -5871,7 +5915,7 @@ namespace : canonicaljson::Array{}; // Step 4 of the receipt checks, with runtime.mutex still released (F-45, ADR-0369). auto authorized_answer = - authorize_send_join_answer(verified_state, verified_auth_chain, signed_value.value, *policy, room_id); + authorize_send_join_answer(verified_state, verified_auth_chain, join_value, *policy, room_id); if (authorized_answer.dropped > 0U || !authorized_answer.refusal.empty()) { log_diagnostic("room.join.answer_checked", @@ -5894,8 +5938,8 @@ namespace auto outcome = FederatedJoinOutcome{}; outcome.policy = *policy; outcome.remote_server = std::move(remote_server); - outcome.signed_event = std::move(signed_event); - outcome.signed_event_value = signed_value.value; + outcome.signed_event = std::move(join_signed_event); + outcome.signed_event_value = std::move(join_value); outcome.event_id_result = event_id_result; outcome.verified_state = std::move(verified_state); outcome.verified_auth_chain = std::move(verified_auth_chain); @@ -6514,6 +6558,13 @@ namespace join_pe.room_id = std::string{room_id}; join_pe.sender_user_id = *user_id; join_pe.json = signed_event.event_json; + // The event_signatures rows come from this list, not from the JSON: give them every signature the stored + // join carries, this server's own and, for a restricted room, the resident's verified co-signature + // (ADR-0745), as the snapshot and auth-chain writers do for theirs. + if (auto const parsed_join = events::parse_event_envelope(signed_event_value); parsed_join.error.empty()) + { + join_pe.signatures = parsed_join.event.signatures; + } join_pe.depth = join_depth; join_pe.stream_ordering = membership_stream; join_pe.prev_event_ids = std::move(join_prev_ids); @@ -10779,9 +10830,12 @@ auto claim_membership_event_template(canonicaljson::Object& event, std::string_v std::int64_t now_ms) -> void { // The resident's template is signed and stored as this server's own event, so an `unsigned` it carries - // (covered by no signature) is dropped with the `event_id` (ADR-0402). + // (covered by no signature) is dropped with the `event_id` (ADR-0402). So are its `hashes` and `signatures`: a + // template is unsigned, signing adds to whatever `signatures` the event holds, and this server's join must carry + // only the signatures it adds itself and those it verified (ADR-0745). std::erase_if(event, [](canonicaljson::ObjectMember const& member) { - return member.key == "event_id" || member.key == "unsigned"; + return member.key == "event_id" || member.key == "unsigned" || member.key == "hashes" || + member.key == "signatures"; }); replace_or_add_member(event, "origin", canonicaljson::Value{std::string{our_server_name}}); replace_or_add_member(event, "origin_server_ts", canonicaljson::Value{now_ms}); diff --git a/src/homeserver/WorkerPool.cpp b/src/homeserver/WorkerPool.cpp index 3e31eb496..9ef8bc84d 100644 --- a/src/homeserver/WorkerPool.cpp +++ b/src/homeserver/WorkerPool.cpp @@ -837,7 +837,7 @@ namespace // F-21 and F-24 (ADR-0358, ADR-0359): main, not a worker, decides what an admin's policy rule and a room's // m.room.server_acl allow. A worker's store is a snapshot (PostgreSQL workers hold no policy rules at all, and - // only the shard that owns a room is told when it changes), and a worker is the process most exposed to hostile + // a shard's copy of a room lags main's until its reload lands), and a worker is the process most exposed to hostile // input, so a decision it made is not relied on: everything a worker relays is judged again here, against // main's own store, for the transport origin (`origin`, the worker's own claim, which the proxy has already // authenticated and judged) and, once the event is verified, for the server its sender belongs to. @@ -971,7 +971,7 @@ auto handle_pdu_ingest_request(HomeserverRuntime& runtime, std::string_view requ } // The default sink reserves stream_ordering internally and returns it in // accepted_stream_ordering. It also publishes the sync notification, so the - // worker path only forwards the result back to the shard that owns this room. + // worker path only forwards the result back to the shard that relayed the PDU. return runtime.federation.pdu_sink(*relayed.envelope); } @@ -1501,6 +1501,8 @@ WorkerPool::WorkerPool(config::FederationWorkerConfig const& cfg, HomeserverRunt } auto const count = cfg_.shards > 0U ? cfg_.shards : 1U; + // One status map per shard, sized before any worker starts: a worker's handler records by its own index. + room_sync_status_.resize(count); workers_.reserve(count); for (auto i = std::uint32_t{0U}; i < count; ++i) { @@ -1695,7 +1697,22 @@ WorkerPool::WorkerPool(config::FederationWorkerConfig const& cfg, HomeserverRunt auto const result_room_id = json_get_str(json, "room_id"); auto const result_state = json_get_str(json, "state"); auto const result_generation = ipc::ipc_json_get_u64(json, "generation"); - record_room_sync_result(result_room_id, result_state, result_generation); + // The result is recorded against this shard only: each worker holds its own copy of every room, + // and a request is judged on the copy of the worker it is sent to (ADR-0758). Main never sends + // generation 0: the worker uses it to report a room its start-up fill left out (ADR-0756). It is + // taken only for a room main listed to this channel, once, so a worker cannot grow main's status + // maps with rooms main never named; any other result needs a generation main issued for the room. + if (result_generation == 0U) + { + if (result_state == "failed" && snapshot_service_->consume_fill_omission(ch, result_room_id)) + { + record_room_left_out_of_fill(shard_index, result_room_id); + } + } + else + { + record_room_sync_result(shard_index, result_room_id, result_state, result_generation); + } } else { @@ -1778,30 +1795,41 @@ auto WorkerPool::handle(LocalHttpRequest const& request, std::string_view room_i auto WorkerPool::notify_room_changed(std::string_view room_id) -> void { - auto const index = shard_for(room_id); - if (index >= workers_.size()) - { - return; - } - auto const ch = workers_[index]->channel_snapshot(); - if (!ch || !ch->healthy()) - { - return; - } - + // Every shard is told (ADR-0758): a /send goes whole to the shard its first PDU picks, so any shard may be the + // one that answers for this room. The status is written for every shard even when a notification cannot be sent: + // a room first notified while a worker's channel is down is one that worker may never have loaded, and main + // serves that shard's requests for the room until a reload of it succeeds there. auto const generation = [this, room_id]() { auto guard = std::lock_guard{room_sync_mu_}; - auto const next = ++room_sync_generations_[std::string{room_id}]; - room_sync_status_[std::string{room_id}] = RoomSyncStatus{"pending", next}; + auto const key = std::string{room_id}; + auto const next = ++room_sync_generations_[key]; + for (auto& shard_status : room_sync_status_) + { + auto& status = shard_status[key]; + // A first notification may be for a room created after the worker's fill, which it has never loaded: + // that room is held only once a reload of it succeeds (ADR-0756). A room already loaded stays held. + auto const held = !status.state.empty() && status.held; + status = RoomSyncStatus{"pending", next, held}; + } return next; }(); - // The permit is recorded before the notification is sent, so a worker that reloads at once finds it (ADR-0735). - snapshot_service_->permit_reload(ch, room_id, generation); - if (!ch->send_notification(ipc::serialize_room_sync_notification(room_id, generation))) + auto const notification = ipc::serialize_room_sync_notification(room_id, generation); + for (auto const& worker : workers_) { - LOG_WARNING("Federation worker pool: room_sync notification could not be sent for room_id=" + - std::string{room_id}); + auto const ch = worker->channel_snapshot(); + if (!ch || !ch->healthy()) + { + continue; + } + // The permit is recorded before the notification is sent, so a worker that reloads at once finds it + // (ADR-0735). + snapshot_service_->permit_reload(ch, room_id, generation); + if (!ch->send_notification(notification)) + { + LOG_WARNING("Federation worker pool: room_sync notification could not be sent for room_id=" + + std::string{room_id}); + } } } @@ -1843,19 +1871,39 @@ auto WorkerPool::shard_for(std::string_view room_id) const noexcept -> std::size return federation_worker_shard_for(room_id, cfg_.shards); } +auto WorkerPool::worker_pid(std::size_t shard) const noexcept -> pid_t +{ + // Every entry of workers_ is set in the constructor; stop() clears the vector, never an entry. + if (shard >= workers_.size()) + { + return -1; + } + return workers_[shard]->worker_pid(); +} + auto WorkerPool::room_sync_status(std::string_view room_id) const -> RoomSyncStatus +{ + return room_sync_status(shard_for(room_id), room_id); +} + +auto WorkerPool::room_sync_status(std::size_t shard, std::string_view room_id) const -> RoomSyncStatus { auto guard = std::lock_guard{room_sync_mu_}; - auto const it = room_sync_status_.find(std::string{room_id}); - if (it == room_sync_status_.end()) + if (shard >= room_sync_status_.size()) + { + return RoomSyncStatus{"failed", 0U, false}; + } + auto const& shard_status = room_sync_status_[shard]; + auto const it = shard_status.find(std::string{room_id}); + if (it == shard_status.end()) { return RoomSyncStatus{"ok", 0U}; } return it->second; } -auto WorkerPool::record_room_sync_result(std::string_view room_id, std::string_view state, std::uint64_t generation) - -> void +auto WorkerPool::record_room_sync_result(std::size_t shard, std::string_view room_id, std::string_view state, + std::uint64_t generation) -> void { auto guard = std::lock_guard{room_sync_mu_}; auto const it = room_sync_generations_.find(std::string{room_id}); @@ -1864,14 +1912,33 @@ auto WorkerPool::record_room_sync_result(std::string_view room_id, std::string_v // Stale or unmatched result for a room we no longer track; ignore it. return; } + // shard is the index of the worker that sent the result, always below the size set in the constructor. + auto& status = room_sync_status_.at(shard)[std::string{room_id}]; + // Any successful reload, even of an older generation, means the worker now holds a copy of the room (ADR-0756). + if (state == "ok") + { + status.held = true; + } if (generation == it->second) { - room_sync_status_[std::string{room_id}] = RoomSyncStatus{std::string{state}, generation}; + status.state = std::string{state}; + status.generation = generation; } // An older generation is also ignored; the pending entry for the latest // generation remains until its own result arrives or the room is re-synced. } +auto WorkerPool::record_room_left_out_of_fill(std::size_t shard, std::string_view room_id) -> void +{ + auto guard = std::lock_guard{room_sync_mu_}; + auto& status = room_sync_status_.at(shard)[std::string{room_id}]; + status.held = false; + if (status.state != "pending") + { + status.state = "failed"; + } +} + auto validate_worker_outbound_reply(http::OutboundRequest const& request, http::OutboundResult reply) -> http::OutboundResult { diff --git a/src/homeserver/WorkerSnapshotService.cpp b/src/homeserver/WorkerSnapshotService.cpp index e66acc381..cbb4a5e0c 100644 --- a/src/homeserver/WorkerSnapshotService.cpp +++ b/src/homeserver/WorkerSnapshotService.cpp @@ -71,6 +71,16 @@ auto WorkerSnapshotService::answer(ChannelKey const& channel, std::string_view t return refused(type, "unknown_type"); } +auto WorkerSnapshotService::consume_fill_omission(ChannelKey const& channel, std::string_view room_id) -> bool +{ + if (!channel) + { + return false; + } + auto const lock = std::lock_guard{mu_}; + return state_of(channel).fill_omissions.erase(std::string{room_id}) == 1U; +} + auto WorkerSnapshotService::channels_tracked() -> std::size_t { auto const lock = std::lock_guard{mu_}; @@ -125,6 +135,7 @@ auto WorkerSnapshotService::answer_room_list(ChannelKey const& channel, std::str } auto const lock = std::lock_guard{mu_}; auto& state = state_of(channel); + state.fill_omissions.insert(room_ids->begin(), room_ids->end()); state.fill_permits.insert(std::make_move_iterator(room_ids->begin()), std::make_move_iterator(room_ids->end())); auto first = std::move(pages.front()); if (pages.size() > 1U) diff --git a/src/ipc/AGENTS.md b/src/ipc/AGENTS.md index a4172a657..4268c73b6 100644 --- a/src/ipc/AGENTS.md +++ b/src/ipc/AGENTS.md @@ -27,8 +27,13 @@ and `merovingian-fed-worker`. There is no filesystem socket path: the fd is inhe (`homeserver::refuse_forbidden_worker_request`, no runtime lock), and the worker's provider is `crypto::RefusingEd25519Provider`. Never add a generic "sign these bytes" frame; if a worker ever needs a signature, add a narrowly typed request where main builds the payload itself. Callers must - never put an access token, `Authorization` / `X-Matrix` header, or the Ed25519 secret key into a - frame body (`docs/hardening.md`). + never put the Ed25519 secret key, a client access token, or a peer's inbound `Authorization` / + `X-Matrix` header into a frame body: main verifies the inbound signature and sends the worker only + the verified identity, and `fed_request` drops those headers (`is_credential_header_name`). The one + exception is direction-specific: an outbound request frame (`serialize_outbound_http_request`) + carries this server's own `Authorization: X-Matrix` signature, bound to that one request's method, + URI, destination and body, which is not a reusable peer or client credential (`docs/hardening.md`, + `docs/security-coding-rules.md`). 4. **Oversize frames fail the send and are logged, never silently dropped.** `max_frame_bytes` (`kIpcMaxFrameBytes`, 24 MiB) is derived identically on both ends from `security.federation.join_response_max_size` and the transport caps, by diff --git a/src/main.cpp b/src/main.cpp index fc3a7cce9..54ee2ee72 100644 --- a/src/main.cpp +++ b/src/main.cpp @@ -81,7 +81,7 @@ namespace { -constexpr auto version = std::string_view{"1.0.1"}; +constexpr auto version = std::string_view{"1.0.2"}; struct BootstrapConfigResult final { diff --git a/src/sync/AGENTS.md b/src/sync/AGENTS.md index 52363a345..a233d54fe 100644 --- a/src/sync/AGENTS.md +++ b/src/sync/AGENTS.md @@ -28,7 +28,7 @@ Spec authority: | `ReceiptContentBuilder.cpp` | `ReceiptContentBuilder`: the one builder of an `m.receipt` event's content (with `thread_id`), shared by `/sync` and sliding sync | | `BundledAggregations.cpp` | `BundledAggregations`: the one builder of `unsigned.m.relations` (`m.replace`, `m.thread`, `m.reference`) for an event served to a client (ADR-0430) | | `ViewerEventUnsigned.cpp` | `ViewerEventUnsigned`: adds the requesting user's `unsigned.membership` and, for the sending device, `unsigned.transaction_id` to a client event | -| `ReadMarkers.cpp` | `ReadMarkers`, `read_markers_for()`, `event_thread_id()`: where a user has read up to, unthreaded and per thread | +| `ReadMarkers.cpp` | `ReadMarkers`, `read_markers_for()`, `event_thread_id()`: where a user has read up to, unthreaded and per thread; `event_relates_to()`: whether an event's relation chain reaches an event within 3 hops (ADR-0746) | | `HistoryVisibility.cpp` | `HistoryVisibility`: the one per-request filter deciding which room events a user may see (Room History Visibility rules, state at the event, fail closed) | | `RoomReadAccess.cpp` | `room_read_access_for()` / `room_state_event_ids()`: the one predicate deciding which room state a user may read (current, as of departure, or none) | | `SlidingSync.hpp` (header-only) | Core sliding-sync connection-state, request and response types shared by the files above | @@ -428,8 +428,9 @@ call it, and `database::add_redaction_compat`). Never loop `store.receipts` in a request: use `database::receipts_in_room` (ephemeral `m.receipt` events, the sliding sync receipts extension and its wake check) or `database::receipts_of_user_in_room` (`read_markers_for`, the push badge -count). A threaded receipt (`thread_id` other than `main`) is accepted by the POST handler only for the thread root or -an event that belongs to that thread (`receipt_event_is_in_thread`), and `store_receipt` caps a user's threaded rows in +count). A threaded receipt (`thread_id` other than `main`) is accepted by the POST handler only for the thread root, +an event that belongs to that thread, or an event whose relation chain reaches the root within 3 hops +(`receipt_event_is_in_thread`, `event_relates_to`, ADR-0746), and `store_receipt` caps a user's threaded rows in a room. A test proves cost with `receipt_rows_examined`, not time. ## Classic sync reads per-room rows through the room indexes (ADR-0654) diff --git a/src/sync/ReadMarkers.cpp b/src/sync/ReadMarkers.cpp index 963d11084..2e9b8ec6a 100644 --- a/src/sync/ReadMarkers.cpp +++ b/src/sync/ReadMarkers.cpp @@ -189,4 +189,38 @@ auto event_thread_id(database::PersistentStore const& store, canonicaljson::Obje } } +auto event_relates_to(database::PersistentStore const& store, std::string_view room_id, + canonicaljson::Object const& event, std::string_view target_event_id) -> bool +{ + auto parent = canonicaljson::Value{}; + auto const* current = &event; + for (auto hop = std::size_t{0U}; hop < max_thread_relation_hops; ++hop) + { + auto const* relation = relation_of(*current); + auto const* related_id = relation != nullptr ? member_string(*relation, "event_id") : nullptr; + if (related_id == nullptr || related_id->empty()) + { + return false; + } + if (*related_id == target_event_id) + { + return true; + } + auto const stored = store.events.find_in_room(room_id, *related_id); + if (stored == store.events.end()) + { + return false; + } + auto parsed = canonicaljson::parse_lossless(stored->json); + auto const* parent_object = std::get_if(&parsed.value.storage()); + if (parsed.error != canonicaljson::ParseError::none || parent_object == nullptr) + { + return false; + } + parent = std::move(parsed.value); + current = std::get_if(&parent.storage()); + } + return false; +} + } // namespace merovingian::sync diff --git a/tests/complement/merovingian.conf.template b/tests/complement/merovingian.conf.template index 352974a55..806cac605 100644 --- a/tests/complement/merovingian.conf.template +++ b/tests/complement/merovingian.conf.template @@ -99,8 +99,17 @@ server.http.sync_max_per_device=16 # entrypoint installs them beside the master key and adds the key # (install-appservices.sh). # Complement registers appservices at http://host.docker.internal:, which is -# not a loopback literal; plain http to it is allowed by name (ADR-0515). -appservice.allow_cleartext_hosts=host.docker.internal +# not a loopback literal; plain http to it is allowed by name (ADR-0515). The +# hs_with_application_service blueprint registers http://localhost:9000, and +# `localhost` is a name, not a loopback literal, so it is listed too. Without it +# the registration is refused at start and its as_token answers 401 +# (TestJoinFederatedRoomFromApplicationServiceBridgeUser, TestJumpToDateEndpoint). +appservice.allow_cleartext_hosts=host.docker.internal,localhost +# Complement sends every request from one address and does not retry a 429, so +# its parallel subtests' remote joins would refuse each other at the default of +# three remote calls per client address (ADR-0751). Sixteen is the global cap of +# the default request pool. +server.http.client_outbound_max_per_client=16 security.federation.per_origin_transaction_rate=1000000000/1s security.federation.per_origin_pdu_rate=1000000000/1s security.federation.per_origin_edu_rate=1000000000/1s diff --git a/tests/complement/skip-list.toml b/tests/complement/skip-list.toml index 346147a1a..ebe72ea49 100644 --- a/tests/complement/skip-list.toml +++ b/tests/complement/skip-list.toml @@ -651,3 +651,22 @@ mode = "expect-fail" category = "non-spec" spec = "docs/matrix-v1.19-spec/client-server-api.md#get_matrixclientv3sync" reason = "Also requires unsigned.prev_sender, which v1.19's ClientEvent unsigned does not define; unsigned.prev_content (the invite's content, is_direct included) and replaces_state are returned (ADR-0304)." + +# The two entries below fail since an audit fix made this server follow the spec where the +# pinned Complement test does not (1.0.2). + +[[entry]] +test = "TestAvatarUrlUpdate" +package = "tests/csapi" +mode = "expect-fail" +category = "spec" +spec = "docs/matrix-v1.19-spec/client-server-api.md#put_matrixclientv3profileuseridkeyname" +reason = "Sets avatar_url to the string \"LemurLover\"; v1.19 says \"For avatar_url, the value MUST be an MXC URI string\", so it is refused 400 M_INVALID_PARAM (audit run-2 F-63, ADR-0406). The displayname case passes." + +[[entry]] +test = "TestRoomImageRoundtrip" +package = "tests/csapi" +mode = "expect-fail" +category = "spec" +spec = "docs/matrix-v1.19-spec/client-server-api.md#get_matrixclientv3roomsroomidmessages" +reason = "Requires `end` on a dir=b page that reaches the start of the room; v1.19 says that when no further events are available \"this property is omitted from the response\" (ADR-0470). The page holds the image event." diff --git a/tests/conformance/test_appservice_assertion_conformance.cpp b/tests/conformance/test_appservice_assertion_conformance.cpp index d962363ef..5612b18f4 100644 --- a/tests/conformance/test_appservice_assertion_conformance.cpp +++ b/tests/conformance/test_appservice_assertion_conformance.cpp @@ -15,6 +15,7 @@ #include "../support/JsonTestSupport.hpp" #include "../support/MasterKey.hpp" #include "../support/RegistrationToken.hpp" +#include "../support/TempDirectory.hpp" #include "merovingian/config/ClientRateLimitsConfig.hpp" #include "merovingian/config/Config.hpp" #include "merovingian/config/ListenersConfig.hpp" @@ -64,6 +65,8 @@ auto write_overlapping_registrations(std::filesystem::path const& path_a, std::f "rooms": [] } })"; + out.close(); + REQUIRE(out.good()); } { auto out = std::ofstream{path_b, std::ios::binary}; @@ -79,6 +82,8 @@ auto write_overlapping_registrations(std::filesystem::path const& path_a, std::f "rooms": [] } })"; + out.close(); + REQUIRE(out.good()); } } @@ -159,8 +164,8 @@ SCENARIO("an appservice cannot register, log in as, or assert a user in another // application service." GIVEN("service A (non-exclusive @.*) and service B (exclusive @_b_.*)") { - auto const path_a = std::filesystem::temp_directory_path() / "merovingian-conformance-auth7-a.json"; - auto const path_b = std::filesystem::temp_directory_path() / "merovingian-conformance-auth7-b.json"; + auto const path_a = merovingian::tests::unique_temporary_path("merovingian-conformance-auth7-a", ".json"); + auto const path_b = merovingian::tests::unique_temporary_path("merovingian-conformance-auth7-b", ".json"); auto const config = overlapping_services_config(path_a, path_b); auto started = merovingian::homeserver::start_client_server(config); REQUIRE(started.started); @@ -271,8 +276,8 @@ SCENARIO("an appservice cannot assert a user that does not exist or is deactivat // unregistered or deactivated user with 403 M_FORBIDDEN. GIVEN("service A (non-exclusive @.*) on a freshly started server") { - auto const path_a = std::filesystem::temp_directory_path() / "merovingian-conformance-auth12-a.json"; - auto const path_b = std::filesystem::temp_directory_path() / "merovingian-conformance-auth12-b.json"; + auto const path_a = merovingian::tests::unique_temporary_path("merovingian-conformance-auth12-a", ".json"); + auto const path_b = merovingian::tests::unique_temporary_path("merovingian-conformance-auth12-b", ".json"); auto const config = overlapping_services_config(path_a, path_b); auto started = merovingian::homeserver::start_client_server(config); REQUIRE(started.started); @@ -354,8 +359,8 @@ SCENARIO("identity assertion does not apply to Account Management", "[appservice // ?user_id=, so whoami stays assertable. GIVEN("service A with a registered virtual user, and an ordinary human user") { - auto const path_a = std::filesystem::temp_directory_path() / "merovingian-conformance-auth8-a.json"; - auto const path_b = std::filesystem::temp_directory_path() / "merovingian-conformance-auth8-b.json"; + auto const path_a = merovingian::tests::unique_temporary_path("merovingian-conformance-auth8-a", ".json"); + auto const path_b = merovingian::tests::unique_temporary_path("merovingian-conformance-auth8-b", ".json"); auto const config = overlapping_services_config(path_a, path_b); auto started = merovingian::homeserver::start_client_server(config); REQUIRE(started.started); @@ -477,8 +482,8 @@ SCENARIO("an appservice asserts a user on an endpoint matched by its whole path" { GIVEN("service A with a registered virtual user") { - auto const path_a = std::filesystem::temp_directory_path() / "merovingian-conformance-qs-a.json"; - auto const path_b = std::filesystem::temp_directory_path() / "merovingian-conformance-qs-b.json"; + auto const path_a = merovingian::tests::unique_temporary_path("merovingian-conformance-qs-a", ".json"); + auto const path_b = merovingian::tests::unique_temporary_path("merovingian-conformance-qs-b", ".json"); auto const config = overlapping_services_config(path_a, path_b); auto started = merovingian::homeserver::start_client_server(config); REQUIRE(started.started); @@ -529,8 +534,8 @@ SCENARIO("an appservice's device_id assertion reads the device table under the r { GIVEN("service A with a registered user, and this thread holding the runtime lock") { - auto const path_a = std::filesystem::temp_directory_path() / "merovingian-conformance-f66-a.json"; - auto const path_b = std::filesystem::temp_directory_path() / "merovingian-conformance-f66-b.json"; + auto const path_a = merovingian::tests::unique_temporary_path("merovingian-conformance-f66-a", ".json"); + auto const path_b = merovingian::tests::unique_temporary_path("merovingian-conformance-f66-b", ".json"); auto const config = overlapping_services_config(path_a, path_b); auto started = merovingian::homeserver::start_client_server(config); REQUIRE(started.started); diff --git a/tests/conformance/test_appservice_conformance.cpp b/tests/conformance/test_appservice_conformance.cpp index a8873c0c6..5f74ae887 100644 --- a/tests/conformance/test_appservice_conformance.cpp +++ b/tests/conformance/test_appservice_conformance.cpp @@ -1,5 +1,6 @@ // SPDX-License-Identifier: GPL-3.0-or-later #include "../support/InMemoryDatabaseConfig.hpp" +#include "../support/TempDirectory.hpp" // // +-------------------------------------------------------------------------+ // | APPLICATION SERVICE API — as_token AUTH & MASQUERADING | @@ -62,6 +63,8 @@ using namespace merovingian::tests; { auto out = std::ofstream{registration_path, std::ios::binary}; out << irc_bridge_registration_json(); + out.close(); + REQUIRE(out.good()); } auto security = merovingian::config::SecurityConfig{}; // A runtime refuses to mint a signing secret it cannot encrypt at rest @@ -95,7 +98,7 @@ SCENARIO("an appservice's as_token authenticates as its sender_localpart by defa // the sender_localpart property of the registration." GIVEN("a homeserver with one registered appservice") { - auto const path = std::filesystem::temp_directory_path() / "merovingian-conformance-as-default.json"; + auto const path = merovingian::tests::unique_temporary_path("merovingian-conformance-as-default", ".json"); auto const config = conformance_config_with_appservice(path); auto started = merovingian::homeserver::start_client_server(config); REQUIRE(started.started); @@ -128,7 +131,7 @@ SCENARIO("?user_id= masquerades as a user within the appservice's namespace", "[ // `user` namespaces." GIVEN("a homeserver with one registered appservice") { - auto const path = std::filesystem::temp_directory_path() / "merovingian-conformance-as-masquerade.json"; + auto const path = merovingian::tests::unique_temporary_path("merovingian-conformance-as-masquerade", ".json"); auto const config = conformance_config_with_appservice(path); auto started = merovingian::homeserver::start_client_server(config); REQUIRE(started.started); @@ -187,7 +190,7 @@ SCENARIO("m.login.application_service registers and logs in a passwordless virtu // password." GIVEN("a homeserver with one registered appservice") { - auto const path = std::filesystem::temp_directory_path() / "merovingian-conformance-as-register.json"; + auto const path = merovingian::tests::unique_temporary_path("merovingian-conformance-as-register", ".json"); auto const config = conformance_config_with_appservice(path); auto started = merovingian::homeserver::start_client_server(config); REQUIRE(started.started); @@ -266,7 +269,7 @@ SCENARIO("m.login.application_service logs an existing virtual user in without a { GIVEN("a virtual user already registered by the appservice") { - auto const path = std::filesystem::temp_directory_path() / "merovingian-conformance-as-login.json"; + auto const path = merovingian::tests::unique_temporary_path("merovingian-conformance-as-login", ".json"); auto const config = conformance_config_with_appservice(path); auto started = merovingian::homeserver::start_client_server(config); REQUIRE(started.started); @@ -343,7 +346,7 @@ SCENARIO("an exclusive namespace blocks a human from registering or creating an // defined the namespace as exclusive." GIVEN("a homeserver with an appservice owning an exclusive users and aliases namespace") { - auto const path = std::filesystem::temp_directory_path() / "merovingian-conformance-as-exclusivity.json"; + auto const path = merovingian::tests::unique_temporary_path("merovingian-conformance-as-exclusivity", ".json"); auto const config = conformance_config_with_appservice(path); auto started = merovingian::homeserver::start_client_server(config); REQUIRE(started.started); diff --git a/tests/conformance/test_membership_admission_outcome_conformance.cpp b/tests/conformance/test_membership_admission_outcome_conformance.cpp index 699534399..980d5c82b 100644 --- a/tests/conformance/test_membership_admission_outcome_conformance.cpp +++ b/tests/conformance/test_membership_admission_outcome_conformance.cpp @@ -18,8 +18,8 @@ #include "merovingian/federation/InboundPduEnvelope.hpp" #include "merovingian/federation/MembershipAcceptResult.hpp" #include "merovingian/homeserver/HomeserverRuntime.hpp" -#include "merovingian/homeserver/RuntimeStartResult.hpp" #include "merovingian/homeserver/LocalHttpRouter.hpp" +#include "merovingian/homeserver/RuntimeStartResult.hpp" #include @@ -617,3 +617,84 @@ SCENARIO("send_join, send_leave and send_knock for a room with no local member s } } } + +// Spec: Matrix Server-Server API v1.19 +// Section: "Leaving Rooms (Rejecting Invites)"; PUT /_matrix/federation/v2/send_leave/{roomId}/{eventId} +// URL: ../../docs/matrix-v1.19-spec/server-server-api.md#leaving-rooms-rejecting-invites +// +// A leave is sent to "a resident server", which "may be the server which sent the invite", and send_leave lists no +// refusal for a room whose last member of the receiving server has left. A remote user this server holds as joined, +// invited or knocking can therefore still leave through it (ADR-0747). Each such membership admits one stored leave: +// the leave ends it, a retry of that leave after a lost answer is answered again without being stored again +// (ADR-0591 point 3), and a stranger is refused as before (F-12). The refusals are this server's admission policy +// (ADR-0591, ADR-0747), not a status the spec requires. +SCENARIO("A remote member can leave through a server whose last local member has left", + "[federation][conformance][membership][f12][adr0747]") +{ + GIVEN("a room this server holds with no local member, in which a remote user is joined") + { + auto fixture = Fixture{"!adr0747:local.example.org", "@admin:remote.example.org"}; + auto const& genesis = fixture.genesis(); + auto const join_id = std::string{"$adr0747join:remote.example.org"}; + auto const join = fixture.member_event(join_id, bob, "join", {genesis.join_rules_id}, join_auth(genesis), 4); + seed_accepted_event(fixture.runtime(), fixture.room_id(), join_id, "m.room.member", bob, bob, join.json, + {genesis.join_rules_id}, 4U); + auto const leave_auth = std::vector{genesis.create_id, genesis.pl_id, join_id}; + + WHEN("the joined user's server sends their leave, retries it, and then sends another leave for them") + { + auto const leave_id = std::string{"$adr0747leave:remote.example.org"}; + auto const leave = fixture.member_event(leave_id, bob, "leave", {join_id}, leave_auth, 5); + auto const first = fixture.accept(FederationEndpoint::send_leave, leave); + auto const after_first = store_shape(fixture.runtime()); + auto const retry = fixture.accept(FederationEndpoint::send_leave, leave); + auto const after_retry = store_shape(fixture.runtime()); + auto const again = + fixture.member_event("$adr0747again:remote.example.org", bob, "leave", {join_id}, leave_auth, 6); + auto const second = fixture.accept(FederationEndpoint::send_leave, again); + + THEN("the leave is accepted and stored, its retry is answered, and the next one is refused") + { + // Spec MUST: 200 when the leave "was accepted into the graph". + CAPTURE(first.reason); + REQUIRE(first.accepted); + REQUIRE(first.status == 200U); + REQUIRE(fixture.event_stored(leave_id)); + // ADR-0591 point 3: a retry of the leave that is still the sender's membership is answered 200, and + // nothing is stored a second time. + CAPTURE(retry.reason); + REQUIRE(retry.accepted); + REQUIRE(retry.status == 200U); + REQUIRE(after_retry == after_first); + // ADR-0747 policy: the leave ended the membership, so the row admits no other leave; nothing is stored. + REQUIRE_FALSE(second.accepted); + REQUIRE(second.status == 403U); + REQUIRE(store_shape(fixture.runtime()) == after_first); + } + } + + WHEN("a user this server holds no membership for sends a leave, and a send_join and send_knock arrive") + { + auto const before = store_shape(fixture.runtime()); + auto const stranger = + fixture.member_event("$adr0747stranger:other.example.net", "@eve:other.example.net", "leave", + {genesis.join_rules_id}, {genesis.create_id, genesis.pl_id}, 5); + auto const rejoin = fixture.member_event("$adr0747rejoin:remote.example.org", bob, "join", {join_id}, + join_auth(genesis), 5); + auto const leave_result = fixture.accept(FederationEndpoint::send_leave, stranger); + auto const join_result = fixture.accept(FederationEndpoint::send_join, rejoin); + + THEN("each is refused 403 and nothing is stored") + { + // ADR-0591 and ADR-0747 policy: send_leave and send_join list no code for a room this server has no + // member in, and this server refuses it 403, storing nothing. + REQUIRE_FALSE(leave_result.accepted); + REQUIRE(leave_result.status == 403U); + REQUIRE_FALSE(join_result.accepted); + REQUIRE(join_result.status == 403U); + REQUIRE(join_result.state_json.empty()); + REQUIRE(store_shape(fixture.runtime()) == before); + } + } + } +} diff --git a/tests/integration/test_appservice_query_hooks_flow.cpp b/tests/integration/test_appservice_query_hooks_flow.cpp index ec3b330c7..9efaf8984 100644 --- a/tests/integration/test_appservice_query_hooks_flow.cpp +++ b/tests/integration/test_appservice_query_hooks_flow.cpp @@ -17,9 +17,10 @@ // | the outbound half existed as dead code before and still compiled. | // +-------------------------------------------------------------------------+ -#include "../support/ScopedThreadJoin.hpp" #include "../support/MasterKey.hpp" #include "../support/RegistrationToken.hpp" +#include "../support/ScopedThreadJoin.hpp" +#include "../support/TempDirectory.hpp" #include "../support/TlsMockServer.hpp" #include "merovingian/config/ClientRateLimitsConfig.hpp" #include "merovingian/config/Config.hpp" @@ -103,6 +104,8 @@ auto run_one_shot_appservice_server(merovingian::net::TcpAcceptor& acceptor, std }, "protocols": [] })"; + out.close(); + REQUIRE(out.good()); } auto security = merovingian::config::SecurityConfig{}; // A runtime refuses to mint a signing secret it cannot encrypt at rest @@ -143,7 +146,7 @@ SCENARIO("an unknown room alias inside an appservice namespace is queried before { GIVEN("an appservice claiming #_bridge_.* and a mock server that declines the alias") { - auto const path = std::filesystem::temp_directory_path() / "merovingian-queryhook-alias.json"; + auto const path = merovingian::tests::unique_temporary_path("merovingian-queryhook-alias", ".json"); auto acceptor = merovingian::net::TcpAcceptor{}; REQUIRE(acceptor.bind("127.0.0.1", 0U).ok); auto const port = acceptor.bound_port(); @@ -197,7 +200,7 @@ SCENARIO("A lookup naming an identifier over 255 bytes is refused before any app { GIVEN("an appservice claiming #_bridge_.* and @_bridge_.*, and a mock server") { - auto const path = std::filesystem::temp_directory_path() / "merovingian-queryhook-long.json"; + auto const path = merovingian::tests::unique_temporary_path("merovingian-queryhook-long", ".json"); auto acceptor = merovingian::net::TcpAcceptor{}; REQUIRE(acceptor.bind("127.0.0.1", 0U).ok); auto const port = acceptor.bound_port(); @@ -276,7 +279,7 @@ SCENARIO("an unknown alias outside every appservice namespace is not sent to any { GIVEN("the same appservice, claiming only #_bridge_.*") { - auto const path = std::filesystem::temp_directory_path() / "merovingian-queryhook-alias-outside.json"; + auto const path = merovingian::tests::unique_temporary_path("merovingian-queryhook-alias-outside", ".json"); auto acceptor = merovingian::net::TcpAcceptor{}; REQUIRE(acceptor.bind("127.0.0.1", 0U).ok); auto const port = acceptor.bound_port(); @@ -322,7 +325,7 @@ SCENARIO("an unknown user inside an appservice namespace is queried before 404", { GIVEN("an appservice claiming @_bridge_.* and a mock server that declines the user") { - auto const path = std::filesystem::temp_directory_path() / "merovingian-queryhook-user.json"; + auto const path = merovingian::tests::unique_temporary_path("merovingian-queryhook-user", ".json"); auto acceptor = merovingian::net::TcpAcceptor{}; REQUIRE(acceptor.bind("127.0.0.1", 0U).ok); auto const port = acceptor.bound_port(); diff --git a/tests/integration/test_appservice_thirdparty_flow.cpp b/tests/integration/test_appservice_thirdparty_flow.cpp index 40a77bbe2..1157d4eac 100644 --- a/tests/integration/test_appservice_thirdparty_flow.cpp +++ b/tests/integration/test_appservice_thirdparty_flow.cpp @@ -1,5 +1,6 @@ // SPDX-License-Identifier: GPL-3.0-or-later #include "../support/InMemoryDatabaseConfig.hpp" +#include "../support/TempDirectory.hpp" // // +-------------------------------------------------------------------------+ // | THIRD-PARTY LOOKUPS — END-TO-END APPSERVICE INTEGRATION | @@ -23,12 +24,12 @@ // | can still answer. | // +-------------------------------------------------------------------------+ -#include "../support/MockAppservice.hpp" -#include "../support/ScopedThreadJoin.hpp" #include "../support/AppserviceTestConfig.hpp" #include "../support/JsonTestSupport.hpp" #include "../support/MasterKey.hpp" +#include "../support/MockAppservice.hpp" #include "../support/RegistrationToken.hpp" +#include "../support/ScopedThreadJoin.hpp" #include "../support/TlsMockServer.hpp" #include "merovingian/config/ClientRateLimitsConfig.hpp" #include "merovingian/config/Config.hpp" @@ -36,9 +37,9 @@ #include "merovingian/config/LogModulesConfig.hpp" #include "merovingian/config/SecurityConfig.hpp" #include "merovingian/config/ServerConfig.hpp" +#include "merovingian/homeserver/ClientOutboundProxy.hpp" #include "merovingian/homeserver/ClientServerRuntime.hpp" #include "merovingian/homeserver/ClientServerStartResult.hpp" -#include "merovingian/homeserver/ClientOutboundProxy.hpp" #include "merovingian/net/TcpAcceptor.hpp" #include @@ -159,6 +160,8 @@ auto run_one_shot_appservice_server(merovingian::net::TcpAcceptor& acceptor, std "protocols": )" << protocols_json << R"( })"; + out.close(); + REQUIRE(out.good()); } auto config = thirdparty_test_config(); config.appservice().registration_files = {registration_path.string()}; @@ -176,7 +179,7 @@ SCENARIO("GET /thirdparty/protocol/{protocol} returns the appservice's Protocol { GIVEN("an appservice registered for 'irc' and a mock server that answers its protocol query") { - auto const path = std::filesystem::temp_directory_path() / "merovingian-thirdparty-protocol.json"; + auto const path = merovingian::tests::unique_temporary_path("merovingian-thirdparty-protocol", ".json"); auto acceptor = merovingian::net::TcpAcceptor{}; REQUIRE(acceptor.bind("127.0.0.1", 0U).ok); auto const port = acceptor.bound_port(); @@ -250,7 +253,7 @@ SCENARIO("GET /thirdparty/protocols aggregates protocols across every registered { GIVEN("one appservice registered for 'irc' with a mock server behind it") { - auto const path = std::filesystem::temp_directory_path() / "merovingian-thirdparty-protocols.json"; + auto const path = merovingian::tests::unique_temporary_path("merovingian-thirdparty-protocols", ".json"); auto acceptor = merovingian::net::TcpAcceptor{}; REQUIRE(acceptor.bind("127.0.0.1", 0U).ok); auto const port = acceptor.bound_port(); @@ -301,7 +304,7 @@ SCENARIO("GET /thirdparty/location aggregates Location results for a room alias" { GIVEN("an appservice whose mock server answers an alias lookup") { - auto const path = std::filesystem::temp_directory_path() / "merovingian-thirdparty-location-alias.json"; + auto const path = merovingian::tests::unique_temporary_path("merovingian-thirdparty-location-alias", ".json"); auto acceptor = merovingian::net::TcpAcceptor{}; REQUIRE(acceptor.bind("127.0.0.1", 0U).ok); auto const port = acceptor.bound_port(); @@ -360,7 +363,7 @@ SCENARIO("an unreachable appservice degrades to 404 instead of failing or hangin { GIVEN("an appservice registered for 'irc' whose URL points at a closed local port") { - auto const path = std::filesystem::temp_directory_path() / "merovingian-thirdparty-unreachable.json"; + auto const path = merovingian::tests::unique_temporary_path("merovingian-thirdparty-unreachable", ".json"); // Reserve a loopback port, then let the acceptor close before any // client connects — the port refuses connections, standing in for // an unreachable appservice. @@ -416,8 +419,10 @@ SCENARIO("one appservice being unreachable does not suppress another appservice' { GIVEN("two appservices both declaring 'irc': one unreachable, one backed by a real mock server") { - auto const unreachable_path = std::filesystem::temp_directory_path() / "merovingian-thirdparty-multi-a.json"; - auto const reachable_path = std::filesystem::temp_directory_path() / "merovingian-thirdparty-multi-b.json"; + auto const unreachable_path = + merovingian::tests::unique_temporary_path("merovingian-thirdparty-multi-a", ".json"); + auto const reachable_path = + merovingian::tests::unique_temporary_path("merovingian-thirdparty-multi-b", ".json"); auto closed_port = std::uint16_t{0U}; { @@ -438,6 +443,8 @@ SCENARIO("one appservice being unreachable does not suppress another appservice' << R"(","as_token":"irc-a-as-token","hs_token":"irc-a-hs-token",)" << R"("sender_localpart":"_irc_a_bot","namespaces":{"users":[],"aliases":[],"rooms":[]},)" << R"("protocols":["irc"]})"; + out.close(); + REQUIRE(out.good()); } { auto out = std::ofstream{reachable_path, std::ios::binary}; @@ -445,6 +452,8 @@ SCENARIO("one appservice being unreachable does not suppress another appservice' << R"(","as_token":"irc-b-as-token","hs_token":"irc-b-hs-token",)" << R"("sender_localpart":"_irc_b_bot","namespaces":{"users":[],"aliases":[],"rooms":[]},)" << R"("protocols":["irc"]})"; + out.close(); + REQUIRE(out.good()); } auto config = thirdparty_test_config(); config.appservice().registration_files = {unreachable_path.string(), reachable_path.string()}; @@ -493,7 +502,7 @@ SCENARIO("Third-party lookups that call an appservice take a client outbound pro { GIVEN("a registered appservice and a client outbound budget whose only slot another client holds") { - auto const path = std::filesystem::temp_directory_path() / "merovingian-thirdparty-budget.json"; + auto const path = merovingian::tests::unique_temporary_path("merovingian-thirdparty-budget", ".json"); auto acceptor = merovingian::net::TcpAcceptor{}; REQUIRE(acceptor.bind("127.0.0.1", 0U).ok); auto const port = acceptor.bound_port(); diff --git a/tests/integration/test_appservice_wave3_flow.cpp b/tests/integration/test_appservice_wave3_flow.cpp index a9e6d49c6..fb33732bd 100644 --- a/tests/integration/test_appservice_wave3_flow.cpp +++ b/tests/integration/test_appservice_wave3_flow.cpp @@ -13,27 +13,27 @@ // | so no scenario sleeps and no PeriodicJob thread runs. | // +-------------------------------------------------------------------------+ +#include "../support/AppserviceTestConfig.hpp" #include "../support/JoiningThreads.hpp" +#include "../support/JsonTestSupport.hpp" #include "../support/MockAppservice.hpp" #include "../support/TestAppservice.hpp" -#include "../support/AppserviceTestConfig.hpp" -#include "../support/JsonTestSupport.hpp" -#include "merovingian/canonicaljson/Value.hpp" #include "merovingian/canonicaljson/Serializer.hpp" +#include "merovingian/canonicaljson/Value.hpp" #include "merovingian/database/ExaminedRows.hpp" #include "merovingian/database/PersistentStore.hpp" #include "merovingian/federation/InviteRequest.hpp" +#include "merovingian/homeserver/AppserviceDelivery.hpp" +#include "merovingian/homeserver/AppserviceEphemeral.hpp" #include "merovingian/homeserver/ClientServerClock.hpp" #include "merovingian/homeserver/ClientServerRuntime.hpp" #include "merovingian/homeserver/ClientServerStartResult.hpp" #include "merovingian/homeserver/HomeserverRuntime.hpp" #include "merovingian/homeserver/KeyServerCache.hpp" #include "merovingian/homeserver/LocalHttpRequest.hpp" +#include "merovingian/homeserver/LocalHttpRouter.hpp" #include "merovingian/homeserver/LocalRoom.hpp" #include "merovingian/homeserver/StopFlag.hpp" -#include "merovingian/homeserver/AppserviceDelivery.hpp" -#include "merovingian/homeserver/AppserviceEphemeral.hpp" -#include "merovingian/homeserver/LocalHttpRouter.hpp" #include @@ -1177,6 +1177,8 @@ SCENARIO("the bare third-party lookups run under the client-outbound budget", auto config = merovingian::tests::appservice_test_config({bridge}); config.appservice().connect_timeout_seconds = 1U; config.appservice().total_timeout_seconds = 2U; + // ADR-0751: one call per client, so the holder's lookup is the client's whole share. + config.server().http.client_outbound_max_per_client = 1U; auto started = merovingian::homeserver::start_client_server(config); REQUIRE(started.started); auto& runtime = started.runtime; diff --git a/tests/integration/test_client_outbound_proxy_flow.cpp b/tests/integration/test_client_outbound_proxy_flow.cpp index 000ae0980..aabc16547 100644 --- a/tests/integration/test_client_outbound_proxy_flow.cpp +++ b/tests/integration/test_client_outbound_proxy_flow.cpp @@ -34,21 +34,21 @@ #include "merovingian/core/SocketHandle.hpp" #include "merovingian/database/ExaminedRows.hpp" #include "merovingian/database/PersistentStore.hpp" +#include "merovingian/federation/OutboundTransactionSupport.hpp" #include "merovingian/federation/ResolvedAddressSet.hpp" #include "merovingian/federation/ServerDiscoveryNetwork.hpp" #include "merovingian/federation/SrvRecord.hpp" #include "merovingian/federation/WellKnownServerResult.hpp" -#include "merovingian/federation/OutboundTransactionSupport.hpp" +#include "merovingian/homeserver/ClientOutboundProxy.hpp" #include "merovingian/homeserver/ClientServerClock.hpp" #include "merovingian/homeserver/ClientServerRuntime.hpp" #include "merovingian/homeserver/ClientServerStartResult.hpp" #include "merovingian/homeserver/HomeserverRuntime.hpp" #include "merovingian/homeserver/KeyServerCache.hpp" #include "merovingian/homeserver/OutboundDeadline.hpp" +#include "merovingian/homeserver/RoomService.hpp" #include "merovingian/homeserver/RuntimeStartResult.hpp" #include "merovingian/homeserver/TestOnlyForcedOutboundResolution.hpp" -#include "merovingian/homeserver/ClientOutboundProxy.hpp" -#include "merovingian/homeserver/RoomService.hpp" #include "merovingian/http/OutboundRequest.hpp" #include "merovingian/http/OutboundResult.hpp" #include "merovingian/net/TcpAcceptor.hpp" @@ -183,10 +183,12 @@ class CountingDiscoveryNetwork final : public merovingian::federation::ServerDis std::shared_ptr> m_calls; // SHARED_PTR: reviewed — test fixture member shared with caller }; -[[nodiscard]] auto proxy_test_config(bool remote_fetch_enabled) -> merovingian::config::Config +[[nodiscard]] auto proxy_test_config(bool remote_fetch_enabled, std::uint32_t outbound_max_per_client = 3U) + -> merovingian::config::Config { auto server = merovingian::config::ServerConfig{}; server.http.request_threads = 8U; + server.http.client_outbound_max_per_client = outbound_max_per_client; auto security = merovingian::config::SecurityConfig{}; security.secrets.master_key_file = merovingian::tests::shared_master_key_file(); @@ -459,10 +461,11 @@ SCENARIO("allow_remote=false stops a remote media fetch even when remote fetchin SCENARIO("Remote media fetches run under the in-flight budget and the media deadline", "[out-7][http-2][integration][media]") { - GIVEN("a homeserver with remote fetching enabled, a one second media deadline and a peer that never answers") + GIVEN("a homeserver with remote fetching enabled, one outbound call per client, a one second media deadline and a " + "peer that never answers") { REQUIRE(sodium_init() >= 0); - auto started = merovingian::homeserver::start_client_server(proxy_test_config(true)); + auto started = merovingian::homeserver::start_client_server(proxy_test_config(true, 1U)); REQUIRE(started.started); auto& runtime = started.runtime; runtime.homeserver.client_outbound_proxy_policy.media_deadline_seconds = 1U; @@ -611,21 +614,25 @@ SCENARIO("Twenty concurrent unauthenticated publicRooms?server= requests cannot } } -SCENARIO("One client address may hold one proxied lookup at a time", "[http-2][integration][concurrency]") +// ADR-0751: the per-client cap is server.http.client_outbound_max_per_client, applied by start_runtime. +SCENARIO("One client address may hold its configured number of proxied lookups at a time", + "[http-2][integration][concurrency][adr0751]") { - GIVEN("a peer that never answers") + GIVEN("a server configured for two outbound calls per client, and a peer that never answers") { REQUIRE(sodium_init() >= 0); - auto started = merovingian::homeserver::start_client_server(proxy_test_config(false)); + auto started = merovingian::homeserver::start_client_server(proxy_test_config(false, 2U)); REQUIRE(started.started); auto& runtime = started.runtime; + REQUIRE(runtime.homeserver.client_outbound_proxy_policy.per_client_cap == 2U); runtime.homeserver.client_outbound_proxy_policy.directory_deadline_seconds = 2U; auto const peer = SilentPeer{}; point_at(runtime, peer); - WHEN("a lookup is in flight from one address, then the same address asks again and a different one asks") + WHEN("two lookups are in flight from one address, then the same address asks again and a different one asks") { auto first = Answer{}; + auto second = Answer{}; auto other_client = Answer{}; auto same_client = Answer{}; auto in_flight = false; @@ -635,9 +642,12 @@ SCENARIO("One client address may hold one proxied lookup at a time", "[http-2][i threads.emplace_back([&]() { first = send(runtime, "GET", silent_rooms_target, "192.0.2.10"); }); + threads.emplace_back([&]() { + second = send(runtime, "GET", silent_rooms_target, "192.0.2.10"); + }); in_flight = wait_until( [&]() { - return runtime.homeserver.client_outbound_budget->active("192.0.2.10") == 1U; + return runtime.homeserver.client_outbound_budget->active("192.0.2.10") == 2U; }, 2000ms); same_client = send(runtime, "GET", silent_rooms_target, "192.0.2.10"); @@ -651,7 +661,7 @@ SCENARIO("One client address may hold one proxied lookup at a time", "[http-2][i 2000ms); } - THEN("the same address is refused and the different address is admitted") + THEN("the same address is refused its third and the different address is admitted") { REQUIRE(in_flight); REQUIRE(same_client.status == 429U); @@ -662,6 +672,7 @@ SCENARIO("One client address may hold one proxied lookup at a time", "[http-2][i THEN("the admitted lookups end in a bad gateway once the peer's silence hits the deadline") { REQUIRE(first.status == 502U); + REQUIRE(second.status == 502U); REQUIRE(other_client.status == 502U); } } diff --git a/tests/integration/test_join_room_flow.cpp b/tests/integration/test_join_room_flow.cpp index a22460925..2a25b04f2 100644 --- a/tests/integration/test_join_room_flow.cpp +++ b/tests/integration/test_join_room_flow.cpp @@ -58,7 +58,6 @@ #include "merovingian/federation/PduIngestionResult.hpp" #include "merovingian/federation/PduIngestionStatus.hpp" #include "merovingian/homeserver/AuthService.hpp" -#include "merovingian/federation/InboundIngestion.hpp" #include "merovingian/homeserver/ClientOutboundProxy.hpp" #include "merovingian/homeserver/ClientServerRuntime.hpp" #include "merovingian/homeserver/ClientServerStartResult.hpp" @@ -130,7 +129,8 @@ namespace auto run_resident_server(merovingian::net::TcpAcceptor& acceptor, merovingian::homeserver::TlsServerContext& tls_context, std::string const& make_join_response, std::string const& send_join_response, std::vector& captured_requests, - std::function const& before_send = {}) + std::function const& before_send = {}, + std::function const& send_response_for = {}) { for (auto request_index = 0; request_index < 2; ++request_index) { @@ -201,7 +201,11 @@ auto run_resident_server(merovingian::net::TcpAcceptor& acceptor, { before_send(request_bytes); } - std::ignore = connection.write(is_send ? send_join_response : make_join_response); + // `send_response_for`, when given, answers the send request from the event the joining server sent: a + // resident that co-signs a restricted join must answer with the very event it received. + std::ignore = connection.write(!is_send ? make_join_response + : send_response_for ? send_response_for(request_bytes) + : send_join_response); } } @@ -1829,7 +1833,6 @@ SCENARIO("A federated join seeds the join event's after-state group and forward room_chain.member_auth()); auto const make_join_body = std::string{R"({"room_version":"10","event":)"} + make_join_event + "}"; - // The state array is the room's state BEFORE the join — create and // power_levels only, no join event and no other members, matching // spec (server-server-api.md#joining-rooms). @@ -2195,7 +2198,6 @@ SCENARIO("Outbound joins defer concurrent PDUs until committed and discard them room_chain.member_auth()); auto const make_join_body = std::string{R"({"room_version":"10","event":)"} + make_join_event + "}"; - // The state array is the room's state BEFORE the join — create and // power_levels only, no join event and no other members, matching // spec (server-server-api.md#joining-rooms). @@ -2999,6 +3001,180 @@ SCENARIO("A joining server applies the receipt checks to a resident's send_join } } +// Spec: Matrix Server-Server API v1.19, PUT /_matrix/federation/v2/send_join, 200 response, `event`: "The membership +// event sent to other servers by the resident server including a signature from the resident server. Required if the +// room is restricted and the joining user is authorised by one of the conditions." +// URL: ../../docs/matrix-v1.19-spec/server-server-api.md#put_matrixfederationv2send_joinroomideventid +// rooms/v10.md, authorisation rule 4.3.5.2 (restricted): the event must be "signed by the server of the user ID in +// join_authorised_via_users_server". A first join into a restricted room this server holds nothing of stores this +// server's own join with only the resident's verified co-signature merged in, and refuses a join no checked +// signature authorises (ADR-0745). +SCENARIO("A first join into a restricted room stores the resident's verified co-signature", + "[adr0745][security][homeserver][federation][join][restricted][integration]") +{ + GIVEN("a resident server whose restricted room names its creator as the authorising user") + { + REQUIRE(sodium_init() >= 0); + auto started = merovingian::homeserver::start_runtime(registration_enabled_config()); + REQUIRE(started.started); + auto& runtime = started.runtime; + + auto const reg = merovingian::homeserver::register_local_user(runtime, "alice", "CorrectHorse7!", + merovingian::tests::registration_token); + REQUIRE(reg.ok); + auto const login = merovingian::homeserver::login_local_user(runtime, reg.value, "CorrectHorse7!", "DEVICE1"); + REQUIRE(login.ok); + auto const alice = reg.value; + + auto const certificate = merovingian::tests::tls_mock::write_test_tls_certificate(); + auto tls_context = merovingian::homeserver::make_tls_server_context(certificate.certificate_file, + certificate.private_key_file); + REQUIRE(tls_context.ok()); + auto acceptor = merovingian::net::TcpAcceptor{}; + REQUIRE(acceptor.bind("127.0.0.1", 0U).ok); + auto const port = acceptor.bound_port(); + REQUIRE(port > 0U); + runtime.test_forced_outbound_resolution[resident_server] = + merovingian::homeserver::TestOnlyForcedOutboundResolution{ + "localhost", port, {"127.0.0.1"}, certificate.certificate_pem}; + merovingian::homeserver::wire_federation_callbacks(runtime); + runtime.federation.remote_key_resolver = + [](std::string_view server_name, + std::string_view key_id) -> std::optional { + if (server_name != resident_server || key_id != resident_key_id) + { + return std::nullopt; + } + return resident_remote_runtime(); + }; + + // This server holds nothing of the room, so the join is a first join that no stored state can vouch for. Only + // the resident's key resolves, so a co-signature made with any other key cannot verify. + auto const room_id = std::string{"!restricted745:"} + resident_server; + auto const creator = std::string{"@creator:"} + resident_server; + auto const policy = *merovingian::rooms::find_room_version_policy("10"); + auto chain = resident_room_chain(room_id, creator, policy); + auto const quoted = [](std::string const& id) { + return "\"" + id + "\""; + }; + // The public join rules of the shared chain, replaced by restricted ones naming a room on the resident. + chain.join_rules = sign_test_event( + naming_auth_events( + std::string{R"({"type":"m.room.join_rules","state_key":"","sender":")"} + creator + R"(","room_id":")" + + room_id + R"(","depth":4,"origin_server_ts":900,"prev_events":[)" + quoted(chain.power_levels_id) + + R"(],"auth_events":[],"content":{"join_rule":"restricted","allow":[)" + R"({"type":"m.room_membership","room_id":"!allowed:resident.example.org"}]}})", + quoted(chain.create_id) + "," + quoted(chain.creator_join_id) + "," + quoted(chain.power_levels_id)), + policy, resident_server, resident_key_id, resident_key_seed); + chain.join_rules_id = reference_event_id(chain.join_rules, policy); + // make_join's template names the creator as the authorising user, so only the resident can sign for it. + auto const make_join_event = naming_auth_events( + std::string{R"({"type":"m.room.member","state_key":")"} + alice + R"(","room_id":")" + room_id + + R"(","sender":")" + alice + R"(","depth":5,"origin_server_ts":1000,"prev_events":[)" + + quoted(chain.join_rules_id) + R"(],"auth_events":[],"content":{"join_authorised_via_users_server":")" + + creator + R"(","membership":"join"}})", + chain.member_auth() + "," + quoted(chain.creator_join_id)); + auto const make_join_response = + json_http_response("200 OK", std::string{R"({"room_version":"10","event":)"} + make_join_event + "}"); + + // Answers send_join with the room's state and, as `event`, `event_for` applied to the join it received. + auto const answer_with = [&](std::function event_for) { + return [&chain, event_for = std::move(event_for)](std::string const& request) { + auto const received = event_from_send_request(request); + auto const event = event_for(received); + return json_http_response("200 OK", std::string{R"({"state":)"} + + canonicaljson_array_to_string(chain.events()) + + R"(,"auth_chain":[])" + + (event.empty() ? std::string{} : R"(,"event":)" + event) + "}"); + }; + }; + auto const serialized = [](merovingian::canonicaljson::Value const& value) { + auto const out = merovingian::canonicaljson::serialize_canonical(value); + REQUIRE(out.error == merovingian::canonicaljson::CanonicalJsonError::none); + return out.output; + }; + // The join this server sent, signed again under the resident's server name with the key `seed` derives. + auto const co_signed_with = [&](std::string const& seed) { + return [&policy, &serialized, seed](merovingian::canonicaljson::Value const& received) { + return serialized( + sign_test_event(serialized(received), policy, resident_server, resident_key_id, seed)); + }; + }; + // Runs one join against the resident, which answers make_join with the template and send_join as told. + auto const join_answered = [&](std::function const& send_response_for) { + auto captured_requests = std::vector{}; + auto server_thread = std::thread{[&]() { + run_resident_server(acceptor, *tls_context.context, make_join_response, std::string{}, + captured_requests, {}, send_response_for); + }}; + auto result = + merovingian::homeserver::join_room(runtime, login.value, room_id, {std::string{resident_server}}); + server_thread.join(); + return result; + }; + auto const alice_joined = [&] { + auto const lock = std::lock_guard{runtime.mutex}; + return std::ranges::any_of(runtime.database.persistent_store.memberships, [&](auto const& row) { + return row.room_id == room_id && row.user_id == alice && row.membership == "join"; + }); + }; + + WHEN("the resident answers with the join it received, co-signed with its key") + { + auto const result = join_answered(answer_with(co_signed_with(resident_key_seed))); + + THEN("the join succeeds and the stored join carries both this server's and the resident's signature") + { + CAPTURE(result.reason); + REQUIRE(result.ok); + REQUIRE(alice_joined()); + auto const lock = std::lock_guard{runtime.mutex}; + auto const& store = runtime.database.persistent_store; + auto const join_id = + merovingian::database::find_current_state_event_id(store, room_id, "m.room.member", alice); + REQUIRE(join_id.has_value()); + auto const stored = store.events.find_in_room(room_id, *join_id); + REQUIRE(stored != store.events.end()); + auto const parsed = merovingian::canonicaljson::parse_lossless(stored->json); + REQUIRE(parsed.error == merovingian::canonicaljson::ParseError::none); + auto const& event = std::get(parsed.value.storage()); + auto const signatures = std::ranges::find_if(event, [](auto const& member) { + return member.key == "signatures"; + }); + REQUIRE(signatures != event.end()); + auto const& by_server = std::get(signatures->value->storage()); + REQUIRE(object_member_count(*signatures->value, local_server) == 1U); + REQUIRE(object_member_count(*signatures->value, resident_server) == 1U); + REQUIRE(by_server.size() == 2U); + } + } + + WHEN("the resident answers without the event, with the join as it was sent, or co-signed with another key") + { + auto const answer = GENERATE(as{}, "omitted", "unsigned", "impostor"); + auto const result = join_answered(answer_with([&](merovingian::canonicaljson::Value const& received) { + if (answer == "omitted") + { + return std::string{}; + } + if (answer == "unsigned") + { + return serialized(received); + } + return co_signed_with("restricted-join-745-impostor-seed")(received); + })); + + THEN("the join is refused and the user is not joined") + { + CAPTURE(answer, result.reason); + REQUIRE_FALSE(result.ok); + REQUIRE(result.status == 502U); + REQUIRE_FALSE(alice_joined()); + } + } + } +} + // Spec: Matrix Server-Server API v1.19, "Checks performed on receipt of a PDU", and rooms/v10.md, authorisation rule // 2.4: "If there is no m.room.create event among the entries, reject." A room version comes from the room's create // event (client-server-api.md, m.room.create), so a PDU that arrived before the create was stored is judged again diff --git a/tests/integration/test_receipt_thread_admission_flow.cpp b/tests/integration/test_receipt_thread_admission_flow.cpp index 45c5e49cd..c645cd4f5 100644 --- a/tests/integration/test_receipt_thread_admission_flow.cpp +++ b/tests/integration/test_receipt_thread_admission_flow.cpp @@ -46,7 +46,7 @@ namespace hs = merovingian::homeserver; } // namespace -SCENARIO("a threaded receipt must name an event related to its thread", "[receipts][c23][threads]") +SCENARIO("a threaded receipt must name an event related to its thread", "[receipts][c23][threads][adr0746]") { GIVEN("a room with a thread root, a reply in the thread, and two unrelated messages") { @@ -78,6 +78,32 @@ SCENARIO("a threaded receipt must name an event related to its thread", "[receip } } + // ADR-0746: "related" is the relation chain, whatever its types, within the 3 hops of the thread rule. + // Complement TestThreadedReceipts marks a thread read through an annotation of a reference to its root. + WHEN("receipts are posted under the thread for events whose relation chain reaches the root in 3 hops or more") + { + auto const relation = [](std::string const& rel_type, std::string const& parent) { + return R"(,"m.relates_to":{"rel_type":")" + rel_type + R"(","event_id":")" + parent + R"("})"; + }; + auto const reference = send_event(runtime, token, room_id, "r1", relation("m.reference", root)); + auto const annotation = send_event(runtime, token, room_id, "r2", relation("m.annotation", reference)); + auto const third_hop = send_event(runtime, token, room_id, "r3", relation("m.reference", annotation)); + auto const fourth_hop = send_event(runtime, token, room_id, "r4", relation("m.reference", third_hop)); + + THEN("an event within 3 hops is accepted and one further away is refused") + { + REQUIRE(post_receipt(runtime, token, room_id, annotation, root).status == 200U); + REQUIRE(post_receipt(runtime, token, room_id, third_hop, root).status == 200U); + auto const too_far = post_receipt(runtime, token, room_id, fourth_hop, root); + REQUIRE(too_far.status == 400U); + REQUIRE(too_far.body.find("M_INVALID_PARAM") != std::string::npos); + auto const rows = db::receipts_of_user_in_room(store, room_id, "@alice:example.org"); + REQUIRE(rows.size() == 1U); + REQUIRE(rows.front()->event_id == third_hop); + REQUIRE(rows.front()->thread_id == root); + } + } + WHEN("receipts are posted under main, or unthreaded") { THEN("they are accepted for any event") diff --git a/tests/integration/test_room_alias_caps_flow.cpp b/tests/integration/test_room_alias_caps_flow.cpp index f2aa6bfc3..499c12e34 100644 --- a/tests/integration/test_room_alias_caps_flow.cpp +++ b/tests/integration/test_room_alias_caps_flow.cpp @@ -1,6 +1,7 @@ // SPDX-FileCopyrightText: 2026 James Chapman // SPDX-License-Identifier: GPL-3.0-or-later #include "../support/InMemoryDatabaseConfig.hpp" +#include "../support/TempDirectory.hpp" // Security audit run-4 F-03 (ADR-0581): a room alias is created only while its creator is under // max_room_aliases_per_user and its room under max_room_aliases_per_room, so the alias listing @@ -82,6 +83,8 @@ struct Reply final "rooms": [] } })"; + out.close(); + REQUIRE(out.good()); return path.string(); } @@ -349,7 +352,7 @@ SCENARIO("An application service is exempt from the per-user alias cap but not t { GIVEN("a bridge and a server that allows one alias per user and three per room") { - auto const path = std::filesystem::temp_directory_path() / "merovingian-f03-alias-bridge.json"; + auto const path = merovingian::tests::unique_temporary_path("merovingian-f03-alias-bridge", ".json"); auto const registration = write_bridge_registration(path); auto started = merovingian::homeserver::start_client_server(caps_config(1U, 3U, registration)); REQUIRE(started.started); @@ -448,7 +451,7 @@ SCENARIO("Only the owning application service deletes an alias in its exclusive { GIVEN("a bridge with an exclusive alias namespace and a user who is the recorded creator of an alias in it") { - auto const path = std::filesystem::temp_directory_path() / "merovingian-c40-alias-bridge.json"; + auto const path = merovingian::tests::unique_temporary_path("merovingian-c40-alias-bridge", ".json"); auto const registration = write_bridge_registration(path); auto started = merovingian::homeserver::start_client_server(caps_config(100U, 100U, registration)); REQUIRE(started.started); diff --git a/tests/integration/test_security_audit_worker_snapshot_flow.cpp b/tests/integration/test_security_audit_worker_snapshot_flow.cpp index 6d4cabfd3..2aa1de8d3 100644 --- a/tests/integration/test_security_audit_worker_snapshot_flow.cpp +++ b/tests/integration/test_security_audit_worker_snapshot_flow.cpp @@ -16,12 +16,13 @@ #include "merovingian/config/ServerConfig.hpp" #include "merovingian/federation/FederationRemoteRuntime.hpp" #include "merovingian/federation/InboundRequest.hpp" +#include "merovingian/homeserver/AuthService.hpp" #include "merovingian/homeserver/FederationProxy.hpp" #include "merovingian/homeserver/HomeserverRuntime.hpp" #include "merovingian/homeserver/LocalHttpRequest.hpp" -#include "merovingian/homeserver/RuntimeStartResult.hpp" -#include "merovingian/homeserver/AuthService.hpp" #include "merovingian/homeserver/LocalHttpRouter.hpp" +#include "merovingian/homeserver/RuntimeStartResult.hpp" +#include "merovingian/homeserver/WorkerPool.hpp" #include #include @@ -33,6 +34,7 @@ #include #include #include +#include #include #include #include @@ -236,9 +238,16 @@ auto write_worker_config(std::filesystem::path const& path, merovingian::config: R"(",sig=")" + signature + R"(",destination=")" + std::string{destination} + R"(")"; } +// A room create_room_and_set_acl made: its ID, the access token of the admin who created it, and its ACL event's ID. +struct AclRoom +{ + std::string room_id; + std::string admin_token; + std::string acl_event_id; +}; + [[nodiscard]] auto create_room_and_set_acl(merovingian::homeserver::HomeserverRuntime& runtime, - std::string_view local_server_name) - -> std::tuple + std::string_view local_server_name) -> AclRoom { auto const admin_user = std::string{"worker_snapshot_admin"}; auto const registration = merovingian::homeserver::bootstrap_admin_user(runtime, admin_user, "CorrectHorse7!"); @@ -275,6 +284,47 @@ auto write_worker_config(std::filesystem::path const& path, merovingian::config: return {std::move(room_id), login.body, acl_state->event_id}; } +// A remote server's signed GET /state_ids for a room at an event. +[[nodiscard]] auto signed_state_ids_read(merovingian::config::Config const& config, std::string_view room_id, + std::string_view event_id) -> merovingian::homeserver::LocalHttpRequest +{ + auto request = merovingian::homeserver::LocalHttpRequest{}; + request.method = "GET"; + request.target = "/_matrix/federation/v1/state_ids/" + std::string{room_id} + "?event_id=" + std::string{event_id}; + request.access_token = x_matrix_authorization(config.server().server_name, request.target); + request.remote_addr = "198.51.100.27"; + return request; +} + +// A remote server's signed PUT /send carrying `body`. +[[nodiscard]] auto signed_send(merovingian::config::Config const& config, std::string_view txn_id, std::string body) + -> merovingian::homeserver::LocalHttpRequest +{ + auto request = merovingian::homeserver::LocalHttpRequest{}; + request.method = "PUT"; + request.target = "/_matrix/federation/v1/send/" + std::string{txn_id}; + auto const destination = config.server().server_name; + auto const signature = merovingian::federation::make_federation_signature( + origin, destination, "PUT", request.target, body, + merovingian::federation::test::keypair_from_seed(remote_key_seed).secret_key); + request.access_token = std::string{R"(X-Matrix origin=")"} + std::string{origin} + R"(",key=")" + + std::string{remote_key_id} + R"(",sig=")" + signature + R"(",destination=")" + destination + + R"(")"; + request.body = std::move(body); + request.remote_addr = "198.51.100.27"; + return request; +} + +// A link to the worker binary for a pool to start from. Removing it before killing a worker makes every restart of +// that worker fail, so its shard stays down until the test ends, however slowly the test runs; the pool's teardown +// stops the supervisor's retries. +[[nodiscard]] auto link_worker_binary(std::filesystem::path const& directory) -> std::filesystem::path +{ + auto link = directory / "merovingian-fed-worker"; + std::filesystem::create_symlink(std::filesystem::path{test_worker_binary()}, link); + return link; +} + auto change_acl_to_deny(merovingian::homeserver::HomeserverRuntime& runtime, std::string_view token, std::string_view room_id) -> void { @@ -370,3 +420,590 @@ SCENARIO("A failed worker room reload cannot serve federation reads with stale A } } } + +// ADR-0748: Complement TestEventAuth sent a state event, saw it in /sync, and at once asked GET /event_auth for it. +// The worker that owns the room had not reloaded it yet and answered 404. While a reload is pending, main serves the +// room's reads. +SCENARIO("A federation read of an event sent a moment ago is answered while the worker reloads the room", + "[security_audit_worker_snapshot][federation][integration][adr0748]") +{ + GIVEN("a real federation worker that loaded a world-readable room") + { + if (test_worker_binary().empty()) + { + SKIP("MEROVINGIAN_TEST_FEDERATION_WORKER is not defined"); + } + REQUIRE(sodium_init() >= 0); + + auto const temporary = TemporaryDirectory{}; + auto const config = make_config(temporary.path()); + auto const worker_config_path = temporary.path() / "merovingian.conf"; + write_worker_config(worker_config_path, config); + + auto started = merovingian::homeserver::start_runtime(config); + REQUIRE(started.started); + auto& runtime = started.runtime; + merovingian::homeserver::wire_federation_callbacks(runtime); + runtime.federation.remote_key_resolver = make_remote_key_resolver(); + auto const room = create_room_and_set_acl(runtime, config.server().server_name); + auto const& room_id = room.room_id; + auto const& admin_token = room.admin_token; + + auto const worker_link = link_worker_binary(temporary.path()); + auto proxy = merovingian::homeserver::FederationProxy{config.federation_worker(), runtime, worker_link.string(), + worker_config_path.string()}; + REQUIRE(wait_for_proxy(proxy, 15s)); + auto const wait_until = [](auto const& done) { + auto const deadline = std::chrono::steady_clock::now() + 15s; + while (!done() && std::chrono::steady_clock::now() < deadline) + { + std::this_thread::sleep_for(20ms); + } + return done(); + }; + // One reload the worker completes, so the room is one it holds and only a pending reload sends it to main. + proxy.notify_room_changed(room_id); + REQUIRE(wait_until([&proxy, &room_id]() { + return proxy.room_sync_status(room_id).state == "ok"; + })); + REQUIRE(proxy.room_sync_status(room_id).held); + // The reload the next change starts must not finish before the read, or the read would pass without the + // pending fallback. Killing the room's worker with its binary's link removed keeps that reload pending. + auto const worker = proxy.worker_pid_for_room(room_id); + REQUIRE(worker > 0); + REQUIRE(std::filesystem::remove(worker_link)); + REQUIRE(::kill(worker, SIGKILL) == 0); + REQUIRE(wait_until([&proxy, &room_id]() { + return proxy.worker_pid_for_room(room_id) == -1 && !proxy.healthy(); + })); + + WHEN("a client sends a state event and a remote server asks for its auth events at once") + { + auto const topic = merovingian::homeserver::handle_local_http_request( + runtime, {"PUT", "/_matrix/client/v3/rooms/" + room_id + "/state/m.room.topic", admin_token, + R"({"topic":"just sent"})"}); + REQUIRE(topic.status == 200U); + auto const topic_state = + std::ranges::find_if(runtime.database.persistent_store.state, [&room_id](auto const& state) { + return state.room_id == room_id && state.event_type == "m.room.topic" && state.state_key.empty(); + }); + REQUIRE(topic_state != runtime.database.persistent_store.state.end()); + // What send_event_as_user does for the proxy main owns; this test's proxy is its own. + proxy.notify_room_changed(room_id); + auto const target = "/_matrix/federation/v1/event_auth/" + room_id + "/" + topic_state->event_id; + auto request = merovingian::homeserver::LocalHttpRequest{}; + request.method = "GET"; + request.target = target; + request.access_token = x_matrix_authorization(config.server().server_name, target); + request.remote_addr = "198.51.100.27"; + auto const response = proxy.handle(request); + // ADR-0759: a remote user rejecting an invite main sent a moment ago asks for a leave template at once. + auto const leave_target = "/_matrix/federation/v1/make_leave/" + room_id + "/@bob:" + std::string{origin}; + auto leave_request = request; + leave_request.target = leave_target; + leave_request.access_token = x_matrix_authorization(config.server().server_name, leave_target); + auto const leave_response = proxy.handle(leave_request); + auto const status = proxy.room_sync_status(room_id); + auto const still_down = proxy.worker_pid_for_room(room_id) == -1; + + THEN("the auth chain and a leave template are returned while the reload is pending, never 404") + { + REQUIRE(still_down); + CHECK(status.state == "pending"); + CHECK(status.held); + CHECK(response.status == 200U); + CHECK(response.body.find("auth_chain") != std::string::npos); + CAPTURE(leave_response.body); + CHECK(leave_response.status == 200U); + CHECK(leave_response.body.find(R"("membership":"leave")") != std::string::npos); + } + } + } +} + +// ADR-0756: Complement TestKnockingInMSC3787Room created a room and had a remote server join it at once. The worker had +// never loaded the room, took room version 12, computed another event ID than the join's and refused it. A room main +// notifies for the first time is not held by the worker until a reload of it succeeds, and main serves it until then. +SCENARIO("A room created after the worker started is held by the worker only once it has loaded it", + "[security_audit_worker_snapshot][federation][integration][adr0756]") +{ + GIVEN("a real federation worker started before a room is created") + { + if (test_worker_binary().empty()) + { + SKIP("MEROVINGIAN_TEST_FEDERATION_WORKER is not defined"); + } + REQUIRE(sodium_init() >= 0); + + auto const temporary = TemporaryDirectory{}; + auto const config = make_config(temporary.path()); + auto const worker_config_path = temporary.path() / "merovingian.conf"; + write_worker_config(worker_config_path, config); + + auto started = merovingian::homeserver::start_runtime(config); + REQUIRE(started.started); + auto& runtime = started.runtime; + merovingian::homeserver::wire_federation_callbacks(runtime); + runtime.federation.remote_key_resolver = make_remote_key_resolver(); + auto const [listed_room_id, admin_token, listed_acl_event_id] = + create_room_and_set_acl(runtime, config.server().server_name); + + auto proxy = merovingian::homeserver::FederationProxy{ + config.federation_worker(), runtime, std::string{test_worker_binary()}, worker_config_path.string()}; + REQUIRE(wait_for_proxy(proxy, 15s)); + // A healthy channel is not a finished start-up copy. Wait until the worker that owns the listed room serves + // it, so the table hidden below cannot fail that copy. + auto const listed_served = [&proxy, &config, &listed_room_id, &listed_acl_event_id]() { + auto const deadline = std::chrono::steady_clock::now() + 15s; + while (std::chrono::steady_clock::now() < deadline) + { + if (proxy.handle(signed_state_ids_read(config, listed_room_id, listed_acl_event_id)).status == 200U) + { + return true; + } + std::this_thread::sleep_for(20ms); + } + return false; + }(); + REQUIRE(listed_served); + + auto const create = merovingian::homeserver::handle_local_http_request( + runtime, {"POST", "/_matrix/client/v3/createRoom", admin_token, R"({"room_version":"10"})"}); + REQUIRE(create.status == 200U); + auto const new_room_id = create.body; + + auto const wait_for_state = [&proxy, &new_room_id](std::string_view wanted) { + auto const deadline = std::chrono::steady_clock::now() + 5s; + while (std::chrono::steady_clock::now() < deadline) + { + auto const status = proxy.room_sync_status(new_room_id); + if (status.state == wanted) + { + return status; + } + std::this_thread::sleep_for(20ms); + } + return proxy.room_sync_status(new_room_id); + }; + + WHEN("main notifies the worker of the new room and the worker's first load of it fails") + { + // Test-only fault injection, as in the failed-reload scenario: hide a table main's snapshot reads. + auto database_table = SqliteTableHider{config.database().sqlite_path}; + REQUIRE(database_table.hide()); + proxy.notify_room_changed(new_room_id); + auto const first = wait_for_state("failed"); + REQUIRE(database_table.restore()); + + AND_WHEN("a later change is loaded") + { + proxy.notify_room_changed(new_room_id); + auto const second = wait_for_state("ok"); + + THEN("the room was not held before its first load and is held after it") + { + CHECK(proxy.room_sync_status(listed_room_id).held); + CHECK(first.state == "failed"); + CHECK_FALSE(first.held); + CHECK(second.state == "ok"); + CHECK(second.held); + CHECK(second.generation > first.generation); + } + } + } + } +} + +// ADR-0756: a room the worker's start-up copy left out was reported as held, because main had never notified it, so +// every request for it went to a worker that had no copy of it. The worker reports each room it leaves out, and main +// serves that room until a reload of it succeeds. +SCENARIO("A room the worker's start-up copy left out is served by main until the worker loads it", + "[security_audit_worker_snapshot][federation][integration][adr0756]") +{ + GIVEN("a room that exists before a worker starts, whose start-up copy fails") + { + if (test_worker_binary().empty()) + { + SKIP("MEROVINGIAN_TEST_FEDERATION_WORKER is not defined"); + } + REQUIRE(sodium_init() >= 0); + + auto const temporary = TemporaryDirectory{}; + auto const config = make_config(temporary.path()); + auto const worker_config_path = temporary.path() / "merovingian.conf"; + write_worker_config(worker_config_path, config); + + auto started = merovingian::homeserver::start_runtime(config); + REQUIRE(started.started); + auto& runtime = started.runtime; + merovingian::homeserver::wire_federation_callbacks(runtime); + runtime.federation.remote_key_resolver = make_remote_key_resolver(); + auto const room = create_room_and_set_acl(runtime, config.server().server_name); + auto const& room_id = room.room_id; + auto const& acl_event_id = room.acl_event_id; + + // Test-only fault injection: main lists its rooms from `rooms`, but each room's snapshot also reads `invites`. + auto database_table = SqliteTableHider{config.database().sqlite_path}; + REQUIRE(database_table.hide()); + auto proxy = merovingian::homeserver::FederationProxy{ + config.federation_worker(), runtime, std::string{test_worker_binary()}, worker_config_path.string()}; + REQUIRE(wait_for_proxy(proxy, 15s)); + + auto const wait_for_state = [&proxy, &room_id](std::string_view wanted) { + auto const deadline = std::chrono::steady_clock::now() + 10s; + while (std::chrono::steady_clock::now() < deadline) + { + auto const status = proxy.room_sync_status(room_id); + if (status.state == wanted) + { + return status; + } + std::this_thread::sleep_for(20ms); + } + return proxy.room_sync_status(room_id); + }; + + WHEN("the worker reports the room it left out") + { + auto const left_out = wait_for_state("failed"); + REQUIRE(database_table.restore()); + // The worker still has no copy of the room, so only main can answer this read before the reload. + auto const read_before_load = proxy.handle(signed_state_ids_read(config, room_id, acl_event_id)); + + AND_WHEN("a later change is loaded") + { + proxy.notify_room_changed(room_id); + auto const loaded = wait_for_state("ok"); + + THEN("main served the room until the worker loaded it, and the worker holds it after") + { + CHECK(left_out.state == "failed"); + CHECK_FALSE(left_out.held); + CHECK(left_out.generation == 0U); + CHECK(read_before_load.status == 200U); + CHECK(loaded.state == "ok"); + CHECK(loaded.held); + } + } + } + } +} + +// ADR-0756, ADR-0758: every worker's start-up copy lists every room. When the status was kept per room, a shard that +// failed to copy a room another shard owns marked that room as not held although its owner had loaded it. Status is +// kept per shard now: the failed copy is not held on the shard that failed it, and the owner's copy stays held. +SCENARIO("A shard's failed start-up copy of a room is not held on that shard only", + "[security_audit_worker_snapshot][federation][integration][adr0756][adr0758]") +{ + GIVEN("a pool of two workers, one room owned by each, and the listed room served by its owner") + { + if (test_worker_binary().empty()) + { + SKIP("MEROVINGIAN_TEST_FEDERATION_WORKER is not defined"); + } + REQUIRE(sodium_init() >= 0); + + auto const temporary = TemporaryDirectory{}; + auto const config = make_config(temporary.path()); + REQUIRE(config.federation_worker().shards == 2U); + auto const worker_config_path = temporary.path() / "merovingian.conf"; + write_worker_config(worker_config_path, config); + + auto started = merovingian::homeserver::start_runtime(config); + REQUIRE(started.started); + auto& runtime = started.runtime; + merovingian::homeserver::wire_federation_callbacks(runtime); + runtime.federation.remote_key_resolver = make_remote_key_resolver(); + auto const [listed_room_id, admin_token, listed_acl_event_id] = + create_room_and_set_acl(runtime, config.server().server_name); + auto const owner = merovingian::homeserver::federation_worker_shard_for(listed_room_id, 2U); + auto const other = owner == 0U ? std::size_t{1U} : std::size_t{0U}; + auto other_room_id = std::string{}; + for (auto attempt = 0; attempt < 64 && other_room_id.empty(); ++attempt) + { + auto const create = merovingian::homeserver::handle_local_http_request( + runtime, {"POST", "/_matrix/client/v3/createRoom", admin_token, R"({"room_version":"10"})"}); + REQUIRE(create.status == 200U); + if (merovingian::homeserver::federation_worker_shard_for(create.body, 2U) == other) + { + other_room_id = create.body; + } + } + REQUIRE_FALSE(other_room_id.empty()); + + auto proxy = merovingian::homeserver::FederationProxy{ + config.federation_worker(), runtime, std::string{test_worker_binary()}, worker_config_path.string()}; + auto const wait_until = [](auto const& done) { + auto const deadline = std::chrono::steady_clock::now() + 15s; + while (!done() && std::chrono::steady_clock::now() < deadline) + { + std::this_thread::sleep_for(20ms); + } + return done(); + }; + REQUIRE(wait_until([&proxy, &config, &listed_room_id, &listed_acl_event_id]() { + return proxy.healthy() && + proxy.handle(signed_state_ids_read(config, listed_room_id, listed_acl_event_id)).status == 200U; + })); + + WHEN("the other shard's worker restarts and its start-up copy fails both rooms") + { + // Test-only fault injection, as in the scenarios above: hide a table main's snapshot reads. + auto database_table = SqliteTableHider{config.database().sqlite_path}; + REQUIRE(database_table.hide()); + auto const killed = proxy.worker_pid_for_room(other_room_id); + REQUIRE(killed > 0); + REQUIRE(::kill(killed, SIGKILL) == 0); + // The restarted worker reports both rooms as left out, and main records both against its shard. + auto const both_dropped = wait_until([&proxy, other, &other_room_id, &listed_room_id]() { + return !proxy.room_sync_status(other, other_room_id).held && + !proxy.room_sync_status(other, listed_room_id).held; + }); + auto const owner_copy = proxy.room_sync_status(owner, listed_room_id); + auto const owner_view = proxy.room_sync_status(listed_room_id); + REQUIRE(database_table.restore()); + + THEN("both rooms stop being held on the restarted shard only") + { + CHECK(both_dropped); + CHECK(proxy.worker_pid_for_room(other_room_id) != killed); + CHECK(owner_copy.held); + CHECK(owner_view.held); + } + } + } +} + +// ADR-0758: only the shard that owns a room was told when it changed, so a /send whose first PDU picked another shard +// reached a worker holding a start-up copy of the room, or none. Every shard now reloads every changed room. +SCENARIO("A room created after the workers started is loaded and held by every shard", + "[security_audit_worker_snapshot][federation][integration][adr0758]") +{ + GIVEN("a pool of two workers started before a room is created") + { + if (test_worker_binary().empty()) + { + SKIP("MEROVINGIAN_TEST_FEDERATION_WORKER is not defined"); + } + REQUIRE(sodium_init() >= 0); + + auto const temporary = TemporaryDirectory{}; + auto const config = make_config(temporary.path()); + REQUIRE(config.federation_worker().shards == 2U); + auto const worker_config_path = temporary.path() / "merovingian.conf"; + write_worker_config(worker_config_path, config); + + auto started = merovingian::homeserver::start_runtime(config); + REQUIRE(started.started); + auto& runtime = started.runtime; + merovingian::homeserver::wire_federation_callbacks(runtime); + runtime.federation.remote_key_resolver = make_remote_key_resolver(); + // Only the admin's token is used: the helper's room exists before the workers start, and this scenario is about + // a room created after them. + auto const admin_token = create_room_and_set_acl(runtime, config.server().server_name).admin_token; + + auto proxy = merovingian::homeserver::FederationProxy{ + config.federation_worker(), runtime, std::string{test_worker_binary()}, worker_config_path.string()}; + REQUIRE(wait_for_proxy(proxy, 15s)); + + auto const create = merovingian::homeserver::handle_local_http_request( + runtime, {"POST", "/_matrix/client/v3/createRoom", admin_token, R"({"room_version":"10"})"}); + REQUIRE(create.status == 200U); + auto const new_room_id = create.body; + auto const owner = proxy.shard_for_room(new_room_id); + auto const other = owner == 0U ? std::size_t{1U} : std::size_t{0U}; + + WHEN("main notifies the workers of the new room") + { + proxy.notify_room_changed(new_room_id); + auto const pending_on_other = proxy.room_sync_status(other, new_room_id); + auto const loaded_everywhere = [&proxy, &new_room_id]() { + auto const deadline = std::chrono::steady_clock::now() + 10s; + auto const loaded = [&proxy, &new_room_id](std::size_t shard) { + auto const status = proxy.room_sync_status(shard, new_room_id); + return status.state == "ok" && status.held; + }; + while (std::chrono::steady_clock::now() < deadline && !(loaded(0U) && loaded(1U))) + { + std::this_thread::sleep_for(20ms); + } + return loaded(0U) && loaded(1U); + }(); + auto const owner_copy = proxy.room_sync_status(owner, new_room_id); + auto const other_copy = proxy.room_sync_status(other, new_room_id); + + THEN("the shard that does not own the room loads it too, at the same generation") + { + CHECK(pending_on_other.generation == 1U); + CHECK(loaded_everywhere); + CHECK(owner_copy.generation == other_copy.generation); + CHECK(other_copy.held); + CHECK_FALSE(proxy.room_sync_status(2U, new_room_id).held); + } + } + } +} + +// ADR-0758 (CodeRabbit, PR #592): a /send whose first PDU has no room_id goes to shard 0 (ADR-0131), and the proxy +// judged a later PDU's room on the copy of the shard that owns it. When that owner held the room and shard 0 did not, +// the transaction reached a worker with no copy of the room. Every room is now judged on the selected shard's copy. +SCENARIO("A /send whose first PDU has no room is judged on shard 0's copy of every room it names", + "[security_audit_worker_snapshot][federation][integration][adr0758][adr0131]") +{ + GIVEN("two shards, a room owned by shard 1 that shard 1 holds and shard 0, whose worker is down, does not") + { + if (test_worker_binary().empty()) + { + SKIP("MEROVINGIAN_TEST_FEDERATION_WORKER is not defined"); + } + REQUIRE(sodium_init() >= 0); + + auto const temporary = TemporaryDirectory{}; + auto const config = make_config(temporary.path()); + REQUIRE(config.federation_worker().shards == 2U); + auto const worker_config_path = temporary.path() / "merovingian.conf"; + write_worker_config(worker_config_path, config); + + auto started = merovingian::homeserver::start_runtime(config); + REQUIRE(started.started); + auto& runtime = started.runtime; + merovingian::homeserver::wire_federation_callbacks(runtime); + runtime.federation.remote_key_resolver = make_remote_key_resolver(); + auto const first_room = create_room_and_set_acl(runtime, config.server().server_name); + auto const& first_room_id = first_room.room_id; + auto const& admin_token = first_room.admin_token; + auto room_id = merovingian::homeserver::federation_worker_shard_for(first_room_id, 2U) == 1U + ? std::string{first_room_id} + : std::string{}; + for (auto attempt = 0; attempt < 64 && room_id.empty(); ++attempt) + { + auto const create = merovingian::homeserver::handle_local_http_request( + runtime, {"POST", "/_matrix/client/v3/createRoom", admin_token, R"({"room_version":"10"})"}); + REQUIRE(create.status == 200U); + if (merovingian::homeserver::federation_worker_shard_for(create.body, 2U) == 1U) + { + room_id = create.body; + } + } + REQUIRE_FALSE(room_id.empty()); + + auto const worker_link = link_worker_binary(temporary.path()); + auto proxy = merovingian::homeserver::FederationProxy{config.federation_worker(), runtime, worker_link.string(), + worker_config_path.string()}; + REQUIRE(wait_for_proxy(proxy, 15s)); + auto const wait_until = [](auto const& done) { + auto const deadline = std::chrono::steady_clock::now() + 15s; + while (!done() && std::chrono::steady_clock::now() < deadline) + { + std::this_thread::sleep_for(20ms); + } + return done(); + }; + // Shard 0's worker goes down for good (its binary's link removed), so a request sent to it is answered 503. + auto const shard_zero_worker = proxy.worker_pid_for_room(""); + REQUIRE(shard_zero_worker > 0); + REQUIRE(std::filesystem::remove(worker_link)); + REQUIRE(::kill(shard_zero_worker, SIGKILL) == 0); + REQUIRE(wait_until([&proxy]() { + return proxy.worker_pid_for_room("") == -1 && !proxy.healthy(); + })); + std::this_thread::sleep_for(200ms); + // The room's first change marks it not held on every shard; only shard 1 is up to load it again. + proxy.notify_room_changed(room_id); + REQUIRE(wait_until([&proxy, &room_id]() { + auto const status = proxy.room_sync_status(1U, room_id); + return status.state == "ok" && status.held; + })); + + WHEN("a remote server sends a transaction whose first PDU has no room_id and whose second is in the room") + { + auto const body = std::string{R"({"origin":")"} + std::string{origin} + + R"(","origin_server_ts":1700000000000,"pdus":[{"type":"m.room.message"},{"room_id":")" + + room_id + R"(","type":"m.room.message","sender":"@remote:)" + std::string{origin} + + R"(","content":{"body":"hi"}}],"edus":[]})"; + auto const shard_zero_copy = proxy.room_sync_status(0U, room_id); + auto const owner_copy = proxy.room_sync_status(room_id); + auto const response = proxy.handle(signed_send(config, "adr0758-malformed-first", body)); + auto const still_down = proxy.worker_pid_for_room("") == -1; + + THEN("main answers it, because shard 0 holds no copy of the room, though the room's owner does") + { + REQUIRE(still_down); + CHECK(proxy.shard_for_room("") == 0U); + CHECK(owner_copy.held); + CHECK_FALSE(shard_zero_copy.held); + CHECK(response.status == 200U); + } + } + } +} + +// ADR-0756: notify_room_changed returned before recording anything when the worker's channel was down, so a room +// first changed then read as held by a worker that may never have loaded it. +SCENARIO("A room first notified while the worker's channel is down is not held by the worker", + "[security_audit_worker_snapshot][federation][integration][adr0756]") +{ + GIVEN("a worker pool whose worker for the room has just been killed") + { + // The first spawn exchanges keys before the pool is built, so a worker that exits at once makes the pool + // throw. Killing a started worker instead leaves its shard down for the supervisor's restart backoff. + if (test_worker_binary().empty()) + { + SKIP("MEROVINGIAN_TEST_FEDERATION_WORKER is not defined"); + } + REQUIRE(sodium_init() >= 0); + + auto const temporary = TemporaryDirectory{}; + auto const config = make_config(temporary.path()); + auto const worker_config_path = temporary.path() / "merovingian.conf"; + write_worker_config(worker_config_path, config); + + auto started = merovingian::homeserver::start_runtime(config); + REQUIRE(started.started); + auto& runtime = started.runtime; + auto const worker_link = link_worker_binary(temporary.path()); + auto pool = merovingian::homeserver::WorkerPool{config.federation_worker(), runtime, worker_link.string(), + worker_config_path.string()}; + auto const wait_until = [](auto const& done) { + auto const deadline = std::chrono::steady_clock::now() + 15s; + while (!done() && std::chrono::steady_clock::now() < deadline) + { + std::this_thread::sleep_for(20ms); + } + return done(); + }; + REQUIRE(wait_until([&pool]() { + return pool.healthy(); + })); + + auto const room_id = std::string{"!created-while-down:example.org"}; + auto const shard = pool.shard_for(room_id); + auto const worker = pool.worker_pid(shard); + REQUIRE(worker > 0); + // With the link removed the worker is never restarted, so its shard stays down. + REQUIRE(std::filesystem::remove(worker_link)); + REQUIRE(::kill(worker, SIGKILL) == 0); + REQUIRE(wait_until([&pool, shard]() { + return pool.worker_pid(shard) == -1 && !pool.healthy(); + })); + // The supervisor drops the dead channel just after marking itself unhealthy. Waiting longer is always safe. + std::this_thread::sleep_for(200ms); + + WHEN("main notifies a room the worker has never loaded") + { + auto const before = pool.room_sync_status(room_id); + pool.notify_room_changed(room_id); + auto const after = pool.room_sync_status(room_id); + auto const still_down = pool.worker_pid(shard) == -1; + + THEN("the room was held only by default before, and is pending and not held after") + { + REQUIRE(still_down); + CHECK(before.held); + CHECK(after.state == "pending"); + CHECK(after.generation == 1U); + CHECK_FALSE(after.held); + } + } + } +} diff --git a/tests/integration/test_server_startup_hardening_flow.cpp b/tests/integration/test_server_startup_hardening_flow.cpp index 88de60d6e..1b1203562 100644 --- a/tests/integration/test_server_startup_hardening_flow.cpp +++ b/tests/integration/test_server_startup_hardening_flow.cpp @@ -176,6 +176,14 @@ auto write_minimal_config(std::filesystem::path const& config_path, std::filesys // still applied for real -- this opt-out only changes behaviour // when Landlock is genuinely unavailable. << "federation.worker.allow_without_landlock=true\n"; +#if !defined(__linux__) + // ADR-0041: the worker's in-process sandbox exists only on Linux, so + // elsewhere the worker refuses to start until the config accepts that, + // as scripts/check-startup-hardening-gate.sh does for the packaged + // server. The main process still applies its own platform hardening + // (Capsicum or pledge), which is what this test exercises there. + f << "federation.worker.apply_hardening=false\n"; +#endif } // The server rejects config files with group/other write permission. ::chmod(config_path.c_str(), 0644); // NOLINT(google-runtime-int) diff --git a/tests/integration/test_space_hierarchy_federation_flow.cpp b/tests/integration/test_space_hierarchy_federation_flow.cpp index 303d8ecc0..f48eeaf61 100644 --- a/tests/integration/test_space_hierarchy_federation_flow.cpp +++ b/tests/integration/test_space_hierarchy_federation_flow.cpp @@ -25,11 +25,11 @@ // (`via`: "The servers to attempt to request the summary from when the local server cannot // generate it") -#include "../support/TlsTestCertificate.hpp" #include "../support/JsonTestSupport.hpp" #include "../support/MasterKey.hpp" #include "../support/RegistrationToken.hpp" #include "../support/TlsMockServer.hpp" +#include "../support/TlsTestCertificate.hpp" #include "merovingian/config/ClientRateLimitsConfig.hpp" #include "merovingian/config/Config.hpp" #include "merovingian/config/ListenersConfig.hpp" @@ -37,14 +37,14 @@ #include "merovingian/config/SecurityConfig.hpp" #include "merovingian/config/ServerConfig.hpp" #include "merovingian/core/QueryParams.hpp" +#include "merovingian/homeserver/ClientOutboundProxy.hpp" #include "merovingian/homeserver/ClientServerRuntime.hpp" #include "merovingian/homeserver/ClientServerStartResult.hpp" #include "merovingian/homeserver/LocalHttpRequest.hpp" #include "merovingian/homeserver/LocalHttpResponse.hpp" #include "merovingian/homeserver/TestOnlyForcedOutboundResolution.hpp" -#include "merovingian/homeserver/TlsServerContextResult.hpp" -#include "merovingian/homeserver/ClientOutboundProxy.hpp" #include "merovingian/homeserver/Tls.hpp" +#include "merovingian/homeserver/TlsServerContextResult.hpp" #include "merovingian/net/TcpAcceptor.hpp" #include @@ -452,6 +452,8 @@ SCENARIO("A remote server that cannot or will not answer leaves the rest of the auto const alice = register_user(runtime, "alice"); auto const space = create_space(runtime, alice); link_child(runtime, alice, space, "!remoteroom:remote.example.org"); + // One call per client, so the slot held here is the client's whole share (ADR-0751's default is three). + runtime.homeserver.client_outbound_proxy_policy.per_client_cap = 1U; auto const held = merovingian::homeserver::admit_client_outbound_proxy(runtime.homeserver, std::string{client_address}); REQUIRE(held.has_value()); @@ -738,6 +740,8 @@ SCENARIO("A room summary request that calls another server is bounded", "[integr REQUIRE(started.started); auto& runtime = started.runtime; auto const alice = register_user(runtime, "alice"); + // One call per client, so the slot held here is the client's whole share (ADR-0751's default is three). + runtime.homeserver.client_outbound_proxy_policy.per_client_cap = 1U; auto const held = merovingian::homeserver::admit_client_outbound_proxy(runtime.homeserver, std::string{client_address}); REQUIRE(held.has_value()); diff --git a/tests/integration/test_worker_policy_authority_flow.cpp b/tests/integration/test_worker_policy_authority_flow.cpp index eeaa2c15e..44f1adac5 100644 --- a/tests/integration/test_worker_policy_authority_flow.cpp +++ b/tests/integration/test_worker_policy_authority_flow.cpp @@ -18,17 +18,18 @@ #include "../FederationSigningTestSupport.hpp" #include "../support/RemoteRoomFixture.hpp" #include "merovingian/canonicaljson/CanonicalJsonError.hpp" -#include "merovingian/canonicaljson/Value.hpp" #include "merovingian/canonicaljson/Serializer.hpp" +#include "merovingian/canonicaljson/Value.hpp" #include "merovingian/database/ExaminedRows.hpp" +#include "merovingian/database/PersistentStore.hpp" +#include "merovingian/federation/InboundIngestion.hpp" #include "merovingian/federation/InboundPduEnvelope.hpp" #include "merovingian/federation/PduIngestionStatus.hpp" -#include "merovingian/federation/InboundIngestion.hpp" #include "merovingian/homeserver/HomeserverRuntime.hpp" #include "merovingian/homeserver/KeyServerCache.hpp" +#include "merovingian/homeserver/LocalHttpRouter.hpp" #include "merovingian/homeserver/RuntimeStartResult.hpp" #include "merovingian/homeserver/WorkerPool.hpp" -#include "merovingian/homeserver/LocalHttpRouter.hpp" #include @@ -612,33 +613,60 @@ SCENARIO("main applies the room ACL to the verified inviter of a relayed invite, // F-12 (ADR-0591): main applies the acceptor's admission to a relayed send_join, send_leave or send_knock itself, // whatever the worker decided: a room in which no user of this server is joined, invited or knocking is not one it -// answers for, so nothing is stored. +// answers for, so nothing is stored. The one exception (ADR-0747) is a leave whose sender, a remote user, still holds +// a join, invite or knock row there: that row bounds what is stored, so a sender with no row is refused as before. SCENARIO("main refuses a relayed membership event for a room with no local member and stores nothing", - "[integration][federation-worker][worker_policy_authority][f12]") + "[integration][federation-worker][worker_policy_authority][f12][adr0747]") { GIVEN("a room whose only local member has left, in main's store") { - WHEN("the worker relays a valid, signed membership event of the remote user") + WHEN("the worker relays a valid, signed leave of a remote user who holds no membership to end there") { - THEN("main does not take it in and the store is unchanged") + THEN("main refuses it at the admission gate and the store is unchanged") { auto relay_case = RelayCase{RelayPath::membership, "no-interest", 110}; auto& runtime = relay_case.runtime(); for (auto& membership : runtime.database.persistent_store.memberships) { - if (membership.user_id == local_admin) + if (membership.user_id == local_admin || membership.user_id == remote_member) { membership.membership = "leave"; } } auto const before = store_shape(runtime); - CHECK_FALSE(relay_once(runtime, RelayPath::membership, relay_case.event(), remote_server)); + auto const frame = relay_frame(RelayPath::membership, relay_case.event().envelope, remote_server); + auto const response = homeserver::handle_membership_ingest_request(runtime, frame); + CHECK(response.find(R"("accepted":true)") == std::string::npos); + CHECK(response.find("this server has no local member in that room") != std::string::npos); CHECK(store_shape(runtime) == before); CHECK_FALSE(event_stored(runtime, relay_case.event().envelope.event_id)); } } + WHEN("the worker relays the leave of the remote member, who is still joined there") + { + THEN("main takes it in once, and the member's row becomes leave, so it admits no second leave") + { + auto relay_case = RelayCase{RelayPath::membership, "remote-leave", 112}; + auto& runtime = relay_case.runtime(); + for (auto& membership : runtime.database.persistent_store.memberships) + { + if (membership.user_id == local_admin) + { + membership.membership = "leave"; + } + } + + CHECK(relay_once(runtime, RelayPath::membership, relay_case.event(), remote_server)); + CHECK(event_stored(runtime, relay_case.event().envelope.event_id)); + auto const* const row = database::find_membership_row(runtime.database.persistent_store, + relay_case.event().room_id, remote_member); + REQUIRE(row != nullptr); + CHECK(row->membership == "leave"); + } + } + WHEN("the local member is still joined") { THEN("the same relay is taken in, so the refusal is the admission's and nothing else's") diff --git a/tests/meson.build b/tests/meson.build index 96baa2b53..d5485ce80 100644 --- a/tests/meson.build +++ b/tests/meson.build @@ -165,6 +165,12 @@ shell = find_program('sh') timeout: 90, ) + test( + 'check-elf-hardening-tooling', + python, + args: [meson.project_source_root() / 'tests' / 'tooling' / 'test_check_elf_hardening.py'], + ) + test( 'complement-image-tooling', python, @@ -215,6 +221,12 @@ shell = find_program('sh') args: [meson.project_source_root() / 'tests' / 'tooling' / 'test_dependency_wraps.py'], ) + test( + 'build-vm-tooling', + python, + args: [meson.project_source_root() / 'tests' / 'tooling' / 'test_build_vm.py'], + ) + test( 'security-workflow-tooling', python, diff --git a/tests/smoke/meson.build b/tests/smoke/meson.build index a8dfd886d..074941803 100644 --- a/tests/smoke/meson.build +++ b/tests/smoke/meson.build @@ -15,6 +15,11 @@ secure_example_config = ( + ' && sed "s|^security.secrets.master_key_file=.*|security.secrets.master_key_file=$master_key|" ' + example_config + ' > "$example_config" && chmod 0644 "$example_config" && ' ) +# Without --config the server reads the installed system config when there is +# one, so a --dry-run probe that is not about the defaults names the example +# config: on a host with merovingian installed it still tests the shipped +# config, not whatever the host has (ADR-0752). +example_dry_run = tmp_smoke_prefix + secure_example_config + server_executable + ' --dry-run --config "$example_config"' test( 'server-help-smoke-test', @@ -158,7 +163,7 @@ test( smoke_test, args: [ '-c', - server_executable + ' --dry-run >/dev/null 2>&1', + example_dry_run + ' >/dev/null 2>&1', ], ) @@ -167,7 +172,7 @@ test( smoke_test, args: [ '-c', - server_executable + ' --dry-run 2>&1 | grep -q "Starting merovingian-server ' + meson.project_version() + '"', + example_dry_run + ' 2>&1 | grep -q "Starting merovingian-server ' + meson.project_version() + '"', ], ) @@ -184,7 +189,8 @@ test( smoke_test, args: [ '-c', - 'output=$(' + server_executable + ' --dry-run 2>&1) && ' + tmp_smoke_prefix + secure_example_config + 'output=$(' + server_executable + + ' --dry-run --config "$example_config" 2>&1) && ' + ' && '.join(server_library_log_checks) + ' && echo "$output" | grep -qE "Thumbnail worker (library|libraries not reported)"', ], @@ -228,16 +234,21 @@ test( smoke_test, args: [ '-c', - server_executable + ' --dry-run 2>&1 | grep -q "Configuration validation passed"', + example_dry_run + ' 2>&1 | grep -q "Configuration validation passed"', ], ) +# The one probe of the defaults themselves: without --config the server reads +# the packaged system config when one exists, so on a host with merovingian +# installed there are no defaults to report, and it skips (exit 77) (ADR-0752). test( 'server-default-summary-smoke-test', smoke_test, args: [ '-c', - server_executable + ' --dry-run 2>&1 | grep -q "Configuration source: defaults"', + 'if [ -e "' + _sysconfdir + '/merovingian/merovingian.conf" ]; then' + + ' echo "skipped: an installed system config is read instead of the defaults"; exit 77; fi; ' + + server_executable + ' --dry-run 2>&1 | grep -q "Configuration source: defaults"', ], ) @@ -246,9 +257,11 @@ test( smoke_test, args: [ '-c', - server_executable + ' --dry-run 2>&1 | grep -q "Startup hardening checks:" && ' - + server_executable + ' --dry-run 2>&1 | grep -qE "Hardening self-check: seccomp=(enabled|unknown)" && ' - + server_executable + ' --dry-run 2>&1 | grep -qE "Hardening readiness: ready=false blockers=[0-9]+"', + tmp_smoke_prefix + secure_example_config + 'output=$(' + server_executable + + ' --dry-run --config "$example_config" 2>&1) && ' + + 'echo "$output" | grep -q "Startup hardening checks:" && ' + + 'echo "$output" | grep -qE "Hardening self-check: seccomp=(enabled|unknown)" && ' + + 'echo "$output" | grep -qE "Hardening readiness: ready=false blockers=[0-9]+"', ], ) diff --git a/tests/support/AGENTS.md b/tests/support/AGENTS.md index bc2858a93..c505eb69e 100644 --- a/tests/support/AGENTS.md +++ b/tests/support/AGENTS.md @@ -22,7 +22,7 @@ Shared test utilities used across unit, conformance, and integration tests. | `ForgedUnsigned.hpp` | `with_forged_unsigned()` / `forged_unsigned_json`: a signed PDU with the `unsigned` a hostile relaying server would write (`m.relations`, `redacted_because`, `prev_content`, `age`, `transaction_id`, `membership`). The signatures and event ID still hold, so it ingests like the genuine PDU (ADR-0402) | | `RegistrationToken.hpp` | Generates registration tokens for test users without going through the full UIAA flow | | `FakeClamdServer.hpp` | `FakeClamdServer`: a clamd stand-in on a temporary AF_UNIX socket. Reads INSTREAM exchanges, records the streamed bytes and answers with a fixed reply (or stays silent for the timeout case). Another I/O exception, shared by the unit and integration suites (G15, ADR-0565); assert on its recorded data from the test thread only | -| `TempDirectory.hpp` | Fallback-aware temporary-directory helper for test scratch files | +| `TempDirectory.hpp` | `temporary_directory()`: the fallback-aware temporary directory for test scratch files; `unique_temporary_path(stem, extension)`: a name under it unique to the process and call (pid, counter, random), not created. Every scratch file or directory a test creates is named with it, never a fixed name (ADR-0753) | | `TlsMockServer.hpp` | Test certificate generation (`write_test_tls_certificate()`: a server certificate issued by a throwaway test CA made per call; `certificate_pem` is the CA, which clients trust, never the server certificate, because libcurl's upki check on Ubuntu 26.10 and later refuses a self-signed server certificate as its own anchor, ADR-0650) plus one-shot and path-dispatching local TLS servers, for tests that need a real HTTPS peer. Also `MockIdentityServer`: a trusted loopback identity server serving any number of requests (responses chosen by request-line substring, requests captured, optional stall), with `cooperative_responses()` for tests that just need a 3PID validated and `install(runtime)` to trust it | | `CapturedAppserviceRequest.hpp` | `CapturedAppserviceRequest`: one request recorded by `MockAppservice` (method, target, `Authorization`, body, answered status) | | `TestAppservice.hpp` | `TestAppservice`: the fields of one appservice registration written by `appservice_test_config()` | diff --git a/tests/support/TempDirectory.hpp b/tests/support/TempDirectory.hpp index fb26fe426..eae0a9094 100644 --- a/tests/support/TempDirectory.hpp +++ b/tests/support/TempDirectory.hpp @@ -1,9 +1,17 @@ // SPDX-License-Identifier: GPL-3.0-or-later #pragma once +#include + +#include +#include +#include +#include #include +#include #include #include +#include namespace merovingian::tests { @@ -68,4 +76,27 @@ namespace merovingian::tests throw std::runtime_error{"merovingian::tests::temporary_directory: no usable temporary directory found"}; } +// Return a path under temporary_directory() that no other test, process or +// account uses: ---. The file is not +// created. A fixed name in the shared temporary directory is not safe: a file +// another account left there cannot be overwritten (sticky /tmp), so the write +// fails and the test reads that run's stale file, and two suites running at +// once overwrite each other's files (ADR-0753). +[[nodiscard]] inline auto unique_temporary_path(std::string_view stem, std::string_view extension) + -> std::filesystem::path +{ + static auto counter = std::atomic{0U}; + auto device = std::random_device{}; + auto const random_bits = (static_cast(device()) << 32U) | static_cast(device()); + auto const sequence = counter.fetch_add(1U, std::memory_order_relaxed); + // std::to_chars, not std::format: the libc++ of the NetBSD build has no . + auto random_hex = std::array{}; + auto const converted = std::to_chars(random_hex.data(), random_hex.data() + random_hex.size(), random_bits, 16); + auto name = std::string{stem}; + name += "-" + std::to_string(::getpid()) + "-" + std::to_string(sequence) + "-"; + name.append(random_hex.data(), converted.ptr); + name += extension; + return temporary_directory() / name; +} + } // namespace merovingian::tests diff --git a/tests/tooling/test_build_vm.py b/tests/tooling/test_build_vm.py new file mode 100644 index 000000000..c23797180 --- /dev/null +++ b/tests/tooling/test_build_vm.py @@ -0,0 +1,769 @@ +#!/usr/bin/env python3 +# SPDX-License-Identifier: GPL-3.0-or-later +"""Tests for `build.py vm` (ADR-0752): settings, named VMs, paths, the sync plan +and archive, the script the remote host runs on Linux and the BSDs, the +`--jobs` and `--hardening` options of `build-linux.sh` and `build-bsd.sh`, and +`remote-host-setup.sh`.""" +from __future__ import annotations + +import argparse +import importlib.util +import io +import os +import shutil +import subprocess +import sys +import tarfile +import tempfile +import time +import unittest +from pathlib import Path + + +REPO_ROOT = Path(__file__).resolve().parents[2] +LINUX_BUILD_WRAPPER = REPO_ROOT / "scripts" / "build-linux.sh" +BSD_BUILD_WRAPPER = REPO_ROOT / "scripts" / "build-bsd.sh" +REMOTE_HOST_SETUP = REPO_ROOT / "scripts" / "remote-host-setup.sh" + +_spec = importlib.util.spec_from_file_location("merovingian_build", REPO_ROOT / "build.py") +build = importlib.util.module_from_spec(_spec) +_spec.loader.exec_module(build) + + +def vm_args(**overrides) -> argparse.Namespace: + values = {"host": None, "user": None, "port": None, "identity": None, "jobs": None, + "vm": None} + values.update(overrides) + return argparse.Namespace(**values) + + +def git(repo: Path, *args: str) -> None: + subprocess.run(["git", "-C", str(repo), *args], check=True, capture_output=True) + + +class VmSettingsTests(unittest.TestCase): + def setUp(self) -> None: + self.temp = tempfile.TemporaryDirectory() + self.dir = Path(self.temp.name) + self.config = self.dir / "vm.ini" + self.key = self.dir / "key" + self.key.write_text("not a real key\n", encoding="utf-8") + + def tearDown(self) -> None: + self.temp.cleanup() + + def test_command_line_beats_environment_which_beats_the_settings_file(self) -> None: + # GIVEN a settings file, environment variables and command-line values for the host. + self.config.write_text(f"[vm]\nhost = file-host\nuser = fileuser\nport = 2222\n" + f"identity = {self.key}\njobs = 3\n", encoding="utf-8") + env = {"MEROVINGIAN_VM_HOST": "env-host", "MEROVINGIAN_VM_USER": "envuser"} + + # WHEN the settings are resolved with and without a command-line host. + from_cli = build.load_vm_settings(vm_args(host="cli-host"), env, self.config) + from_env = build.load_vm_settings(vm_args(), env, self.config) + from_file = build.load_vm_settings(vm_args(), {}, self.config) + + # THEN each source wins only where the ones before it say nothing. + self.assertEqual(("cli-host", "envuser"), (from_cli.host, from_cli.user)) + self.assertEqual(("env-host", "envuser"), (from_env.host, from_env.user)) + self.assertEqual(build.VmSettings("file-host", "fileuser", 2222, str(self.key), 3), + from_file) + + def test_missing_host_and_option_like_values_are_refused(self) -> None: + # GIVEN no host anywhere, and values ssh would read as options or that are malformed. + cases = [ + vm_args(), + vm_args(host="-oProxyCommand=touch /tmp/x"), + vm_args(host="host name"), + vm_args(host="vm", user="-luser"), + vm_args(host="vm", port="0"), + vm_args(host="vm", port="65536"), + vm_args(host="vm", port="22x"), + vm_args(host="vm", identity=str(self.dir / "absent")), + ] + for args in cases: + # WHEN each is resolved. + # THEN it is refused before ssh ever runs. + with self.subTest(args=args), self.assertRaises(build.VmError): + build.load_vm_settings(args, {}, self.config) + + def test_unknown_setting_and_bad_jobs_in_the_file_are_refused(self) -> None: + for body in ("[vm]\nhost = vm\nhots = typo\n", "[vm]\nhost = vm\njobs = 0\n", + "[vm]\nhost = vm\njobs = two\n"): + # GIVEN a settings file with a misspelt key or an unusable jobs value. + self.config.write_text(body, encoding="utf-8") + # WHEN it is read. + # THEN build.py refuses it rather than guessing. + with self.subTest(body=body), self.assertRaises(build.VmError): + build.load_vm_settings(vm_args(), {}, self.config) + + def test_a_named_vm_reads_its_own_section_and_builds_into_its_own_directory(self) -> None: + # GIVEN a default VM and a named one in the settings file. + self.config.write_text("[vm]\nhost = linux-vm\nuser = claude\n\n" + "[vm.freebsd]\nhost = bsd-vm\nuser = builder\njobs = 4\n", + encoding="utf-8") + + # WHEN each is chosen, by --vm and by MEROVINGIAN_VM_NAME, and without a name. + named = build.load_vm_settings(vm_args(vm="freebsd"), {}, self.config) + from_env = build.load_vm_settings(vm_args(), {"MEROVINGIAN_VM_NAME": "freebsd"}, self.config) + default = build.load_vm_settings(vm_args(), {}, self.config) + + # THEN a named VM takes only its own section, and its logs land in build-vm-. + self.assertEqual(("bsd-vm", "builder", 4, "freebsd"), + (named.host, named.user, named.jobs, named.name)) + self.assertEqual(named, from_env) + self.assertEqual(("linux-vm", "claude", None, ""), + (default.host, default.user, default.jobs, default.name)) + self.assertEqual("build-vm-freebsd", build.vm_default_builddir(named)) + self.assertEqual("build-vm", build.vm_default_builddir(default)) + + def test_an_unknown_or_malformed_vm_name_is_refused(self) -> None: + # GIVEN a settings file with one named VM. + self.config.write_text("[vm.freebsd]\nhost = bsd-vm\n", encoding="utf-8") + for name in ("openbsd", "../x", "Free BSD", "-x"): + # WHEN another name, or one that is not a plain lower-case name, is chosen. + # THEN it is refused rather than falling back to another VM. + with self.subTest(name=name), self.assertRaises(build.VmError): + build.load_vm_settings(vm_args(vm=name), {}, self.config) + # WHEN the file is absent THEN a named VM is refused too. + with self.assertRaises(build.VmError): + build.load_vm_settings(vm_args(vm="freebsd"), {}, self.dir / "absent.ini") + + def test_build_options_reach_the_remote_build_script(self) -> None: + # GIVEN build.py vm options, including --hardening, which both build scripts accept. + args = argparse.Namespace(cc="clang", cxx="clang++", wrap_mode="forcefallback", profile="", + buildtype="", sanitize="", coverage=False, build_fuzz=False, + hardening=True, no_tests=False, setup_only=False, + compile_only=False) + + # WHEN the remote build command is formed, with and without a fixed job count. + sized = build._vm_build_line(args, "build-vm-freebsd", None) + fixed = build._vm_build_line(args, "build-vm", 3) + + # THEN it runs the host's own build script with the options and the right job count. + self.assertIn('sh "./scripts/$build_script"', sized) + self.assertIn("'--hardening'", sized) + self.assertIn("'build-vm-freebsd'", sized) + self.assertTrue(sized.endswith('--jobs "$jobs"')) + self.assertTrue(fixed.endswith("--jobs '3'") or fixed.endswith("--jobs 3")) + + def test_ssh_never_prompts_and_refuses_unknown_host_keys(self) -> None: + # GIVEN settings with a key file. + settings = build.VmSettings("vm.example", "builder", 2200, str(self.key), None) + + # WHEN the ssh command is built. + command = build.vm_ssh_command(settings) + + # THEN it is batch-only, pins host keys, uses only that key, and ends options before the host. + self.assertIn("BatchMode=yes", command) + self.assertIn("StrictHostKeyChecking=yes", command) + self.assertIn("IdentitiesOnly=yes", command) + self.assertEqual(["-p", "2200"], command[command.index("-p"):command.index("-p") + 2]) + self.assertEqual(["--", "builder@vm.example"], command[-2:]) + + +class VmPathTests(unittest.TestCase): + def test_builddir_must_stay_under_a_build_prefixed_relative_path(self) -> None: + # GIVEN accepted and refused build directories. + for good in ("build-vm", "build", "build-vm/asan", "build_tsan"): + # WHEN validated THEN accepted unchanged. + self.assertEqual(good, build.validate_vm_builddir(good)) + for bad in ("out", "/build-vm", "../build-vm", "build-vm/../src", "build-vm/.", + "build vm", "build-vm/", ""): + with self.subTest(bad=bad), self.assertRaises(build.VmError): + build.validate_vm_builddir(bad) + + def test_checkout_name_is_stable_per_path_and_distinct_between_paths(self) -> None: + # GIVEN two checkouts with the same directory name in different places. + with tempfile.TemporaryDirectory() as first, tempfile.TemporaryDirectory() as second: + one = Path(first) / "Merovingian" + two = Path(second) / "Merovingian" + one.mkdir() + two.mkdir() + + # WHEN each is named twice. + # THEN the name repeats for one path, differs between paths, and is shell-safe. + self.assertEqual(build.vm_checkout_name(one), build.vm_checkout_name(one)) + self.assertNotEqual(build.vm_checkout_name(one), build.vm_checkout_name(two)) + self.assertRegex(build.vm_checkout_name(one), r"^Merovingian-[0-9a-f]{12}$") + + +@unittest.skipUnless(shutil.which("git"), "needs git") +class VmLocalFilesTests(unittest.TestCase): + def setUp(self) -> None: + self.temp = tempfile.TemporaryDirectory() + self.repo = Path(self.temp.name) + git(self.repo, "init", "-q") + (self.repo / ".gitignore").write_text("ignored.txt\n", encoding="utf-8") + (self.repo / "src").mkdir() + (self.repo / "src" / "a.cpp").write_bytes(b"int a;\n") + (self.repo / "run.sh").write_text("#!/bin/sh\n", encoding="utf-8") + (self.repo / "gone.txt").write_text("x\n", encoding="utf-8") + git(self.repo, "add", ".gitignore", "src/a.cpp", "run.sh", "gone.txt") + git(self.repo, "update-index", "--chmod=+x", "run.sh") + + def tearDown(self) -> None: + self.temp.cleanup() + + def test_tracked_and_new_files_are_sent_but_build_and_dot_directories_are_not(self) -> None: + # GIVEN a new source file, build output, a dot directory, an ignored file and a deleted file. + (self.repo / "src" / "new.cpp").write_text("int b;\n", encoding="utf-8") + for path in ("build-vm/meson-logs/testlog.txt", "build-wsl/x.o", ".cache/c", "ignored.txt"): + (self.repo / path).parent.mkdir(parents=True, exist_ok=True) + (self.repo / path).write_text("skip\n", encoding="utf-8") + (self.repo / "gone.txt").unlink() + + # WHEN the files to sync are listed. + files = build.vm_local_files(self.repo) + + # THEN only the tracked files on disk and the new source file are sent, with git's modes. + self.assertEqual({".gitignore", "run.sh", "src/a.cpp", "src/new.cpp"}, set(files)) + self.assertEqual(0o755, files["run.sh"][0]) + self.assertEqual(0o644, files["src/a.cpp"][0]) + self.assertEqual(len(b"int a;\n"), files["src/a.cpp"][1]) + + def test_a_tracked_file_replaced_on_disk_by_a_symbolic_link_is_refused(self) -> None: + # GIVEN a tracked regular file replaced, uncommitted, by a link to a file outside the checkout. + outside_dir = tempfile.TemporaryDirectory() + self.addCleanup(outside_dir.cleanup) + outside = Path(outside_dir.name) / "secret.txt" + outside.write_text("secret\n", encoding="utf-8") + (self.repo / "src" / "a.cpp").unlink() + try: + os.symlink(outside, self.repo / "src" / "a.cpp") + except (OSError, NotImplementedError) as error: + self.skipTest(f"cannot create a symbolic link here: {error}") + + # WHEN the files to send are listed. + # THEN the link is refused rather than followed to its target. + with self.assertRaisesRegex(build.VmError, "src/a.cpp"): + build.vm_local_files(self.repo) + + def test_a_tracked_symbolic_link_is_refused(self) -> None: + # GIVEN a symbolic link recorded in the index. + blob = subprocess.run(["git", "-C", str(self.repo), "hash-object", "-w", "--stdin"], + input=b"src/a.cpp", capture_output=True, check=True).stdout.decode().strip() + git(self.repo, "update-index", "--add", "--cacheinfo", f"120000,{blob},link") + + # WHEN the files to sync are listed. + # THEN build.py refuses rather than sending a link the VM would follow. + with self.assertRaises(build.VmError): + build.vm_local_files(self.repo) + + +class VmManifestAndArchiveTests(unittest.TestCase): + def test_only_changed_files_are_planned_and_bad_manifest_lines_are_resent(self) -> None: + # GIVEN local files and what the VM's manifest says it last received. + local = {"a": (0o644, 1, 10), "b": (0o644, 2, 20), "c": (0o755, 3, 30), "d": (0o644, 4, 40)} + remote_text = build.format_vm_manifest({"a": (0o644, 1, 10), "b": (0o644, 2, 99), + "c": (0o644, 3, 30)}) + "garbage line\n644\tx\t1\td\n" + + # WHEN the sync is planned. + send = build.plan_vm_sync(local, build.parse_vm_manifest(remote_text)) + + # THEN unchanged files are skipped; a changed mtime or mode, or an unreadable entry, is resent. + self.assertEqual(["b", "c", "d"], send) + + def test_archive_carries_the_sent_files_then_the_full_manifest(self) -> None: + # GIVEN two files on disk, of which one is to be sent. + with tempfile.TemporaryDirectory() as temp: + root = Path(temp) + (root / "dir").mkdir() + (root / "dir" / "x.sh").write_bytes(b"echo x\n") + (root / "y.txt").write_bytes(b"y\n") + local = {"dir/x.sh": (0o755, 7, 1_700_000_000), "y.txt": (0o644, 2, 1_700_000_001)} + stream = io.BytesIO() + + # WHEN the archive is written. + build.write_vm_sync_archive(stream, root, local, ["dir/x.sh"]) + + # THEN it holds that file with its mode and mtime, and the manifest of both files last. + stream.seek(0) + with tarfile.open(fileobj=stream, mode="r") as archive: + members = archive.getmembers() + self.assertEqual(["dir/x.sh", build.VM_MANIFEST_NEXT], [m.name for m in members]) + self.assertEqual((0o755, 1_700_000_000), (members[0].mode, members[0].mtime)) + self.assertEqual(b"echo x\n", archive.extractfile(members[0]).read()) + manifest = archive.extractfile(members[1]).read().decode("utf-8") + self.assertEqual(local, build.parse_vm_manifest(manifest)) + + +@unittest.skipUnless(os.name == "posix" and all(shutil.which(tool) for tool in + ("sh", "tar", "python3", "comm", "cut", "sort")), + "needs a POSIX host with tar and python3") +class VmRemoteScriptTests(unittest.TestCase): + def setUp(self) -> None: + self.temp = tempfile.TemporaryDirectory() + self.home = Path(self.temp.name) / "home" + self.source = Path(self.temp.name) / "source" + self.stubs = Path(self.temp.name) / "stubs" + self.home.mkdir() + self.source.mkdir() + self.stubs.mkdir() + self.checkout = self.home / build.VM_REMOTE_ROOT / "co" + self.path = os.environ.get("PATH", "") + + def tearDown(self) -> None: + self.temp.cleanup() + + def stub(self, name: str, body: str) -> None: + """Put a command on PATH ahead of the real one, as another system would have it.""" + script = self.stubs / name + script.write_text(f"#!/bin/sh\n{body}\n", encoding="utf-8") + script.chmod(0o755) + self.path = f"{self.stubs}{os.pathsep}{os.environ.get('PATH', '')}" + + TOKEN = "0123456789abcdef" + + def prepare(self, files: dict[str, bytes], run_line: str, **options) -> tuple[str, bytes]: + """The script and stream build_vm would send, planned against the VM's manifest now.""" + local = {} + for path, data in files.items(): + (self.source / path).parent.mkdir(parents=True, exist_ok=True) + (self.source / path).write_bytes(data) + local[path] = (0o644, len(data), 1_700_000_000 + len(data)) + manifest = self.checkout / build.VM_MANIFEST + baseline = manifest.read_bytes() if manifest.exists() else b"" + remote = build.parse_vm_manifest(baseline.decode("utf-8")) + archive = io.BytesIO() + build.write_vm_sync_stream(archive, self.source, local, build.plan_vm_sync(local, remote)) + script = build.vm_remote_script("co", run_line, token=self.TOKEN, + baseline_sha256=build.vm_manifest_digest(baseline), + stale=build.plan_vm_stale(local, remote), **options) + return script, archive.getvalue() + + @property + def run_dir(self) -> Path: + return self.checkout / build.vm_run_dir(self.TOKEN) + + def environment(self) -> dict[str, str]: + return {**os.environ, "HOME": str(self.home), "PATH": self.path} + + def sync(self, files: dict[str, bytes], run_line: str, **options) -> subprocess.CompletedProcess: + script, archive = self.prepare(files, run_line, **options) + return subprocess.run(["sh", "-c", script], input=archive, capture_output=True, + env=self.environment()) + + def test_files_dropped_locally_are_deleted_and_nothing_outside_is_touched(self) -> None: + # GIVEN a first sync of two files, a build output file, and a file beside the checkout. + first = self.sync({"src/a.cpp": b"a1", "src/b.cpp": b"b"}, "true") + self.assertEqual(0, first.returncode, first.stderr) + (self.checkout / "build-vm").mkdir() + (self.checkout / "build-vm" / "obj.o").write_bytes(b"o") + outside = self.home / build.VM_REMOTE_ROOT / "outside.txt" + outside.write_bytes(b"keep") + manifest = self.checkout / build.VM_MANIFEST + manifest.write_text(manifest.read_text(encoding="utf-8") + "644\t1\t1\t../outside.txt\n", + encoding="utf-8") + + # WHEN the second sync drops b.cpp and changes a.cpp, and runs a command. + second = self.sync({"src/a.cpp": b"a2-changed"}, "echo ran > ran.txt") + + # THEN b.cpp is gone, a.cpp is current, build output and the path outside survive, + # and the command ran in the checkout. + self.assertEqual(0, second.returncode, second.stderr) + self.assertFalse((self.checkout / "src" / "b.cpp").exists()) + self.assertEqual(b"a2-changed", (self.checkout / "src" / "a.cpp").read_bytes()) + self.assertTrue((self.checkout / "build-vm" / "obj.o").exists()) + self.assertEqual(b"keep", outside.read_bytes()) + self.assertEqual(b"ran\n", (self.checkout / "ran.txt").read_bytes()) + self.assertEqual({"src/a.cpp"}, set(build.parse_vm_manifest(manifest.read_text(encoding="utf-8")))) + + def test_a_build_run_removes_stale_test_logs_and_clean_removes_the_build(self) -> None: + # GIVEN a checkout with a previous build's test log and object file. + self.assertEqual(0, self.sync({"a": b"a"}, "true").returncode) + logs = self.checkout / "build-vm" / "meson-logs" + logs.mkdir(parents=True) + (logs / "testlog.txt").write_text("Ok: 99\n", encoding="utf-8") + (self.checkout / "build-vm" / "obj.o").write_bytes(b"o") + + # WHEN a build run fails before writing a log. + failed = self.sync({"a": b"a"}, "exit 3", builddir="build-vm") + + # THEN its exit status comes back and the old log cannot be mistaken for its result. + self.assertEqual(3, failed.returncode) + self.assertFalse((logs / "testlog.txt").exists()) + self.assertTrue((self.checkout / "build-vm" / "obj.o").exists()) + + # WHEN a clean run follows THEN the build directory is gone before the command runs. + cleaned = self.sync({"a": b"a"}, "test ! -e build-vm", builddir="build-vm", clean=True) + self.assertEqual(0, cleaned.returncode, cleaned.stderr) + + def test_old_logs_are_gone_even_when_unpacking_fails(self) -> None: + # GIVEN a checkout with a previous build's test log. + self.assertEqual(0, self.sync({"a": b"a"}, "true").returncode) + logs = self.checkout / "build-vm" / "meson-logs" + logs.mkdir(parents=True) + (logs / "testlog.txt").write_text("Ok: 99\n", encoding="utf-8") + script, _ = self.prepare({"a": b"a"}, "true", builddir="build-vm") + + # WHEN the archive that arrives is not a tar archive, as when the stream breaks. + result = subprocess.run(["sh", "-c", script], input=b"not a tar archive" * 64, + capture_output=True, env=self.environment()) + + # THEN the run fails, and neither the build's log nor a snapshot of it is left to fetch. + self.assertNotEqual(0, result.returncode) + self.assertFalse((logs / "testlog.txt").exists()) + self.assertFalse((self.run_dir / "testlog.txt").exists()) + + def test_a_file_that_becomes_a_directory_and_back_syncs(self) -> None: + # GIVEN a synced file x. + self.assertEqual(0, self.sync({"x": b"file"}, "true").returncode) + shutil.rmtree(self.source) + self.source.mkdir() + + # WHEN x becomes a directory holding x/y, and then a file again. + to_directory = self.sync({"x/y": b"nested"}, "true") + self.assertEqual(0, to_directory.returncode, to_directory.stderr) + self.assertEqual(b"nested", (self.checkout / "x" / "y").read_bytes()) + shutil.rmtree(self.source) + self.source.mkdir() + to_file = self.sync({"x": b"file again"}, "true") + + # THEN each sync unpacks, because the stale path is removed before unpacking. + self.assertEqual(0, to_file.returncode, to_file.stderr) + self.assertEqual(b"file again", (self.checkout / "x").read_bytes()) + + def test_a_file_removed_by_a_failed_sync_is_resent_when_restored_unchanged(self) -> None: + # GIVEN two synced files. + self.assertEqual(0, self.sync({"a": b"a", "b": b"bb"}, "true").returncode) + + # WHEN b is dropped locally and the sync that deletes it on the VM fails to unpack. + script, _ = self.prepare({"a": b"a"}, "true") + (self.source / "b").unlink() + failed = subprocess.run(["sh", "-c", script], input=b"not a tar archive" * 64, + capture_output=True, env=self.environment()) + self.assertNotEqual(0, failed.returncode) + self.assertFalse((self.checkout / "b").exists()) + + # AND b comes back with the size, mode and mtime it had, and the next sync runs. + restored = self.sync({"a": b"a", "b": b"bb"}, "true") + + # THEN that sync sends b again, because the failed one left no manifest claiming the VM has it. + self.assertEqual(0, restored.returncode, restored.stderr) + self.assertEqual(b"bb", (self.checkout / "b").read_bytes()) + + def test_a_retry_after_an_unpack_interrupted_inside_a_new_directory_succeeds(self) -> None: + # GIVEN a synced file x that has become a directory holding x/y locally. + self.assertEqual(0, self.sync({"x": b"file"}, "true").returncode) + shutil.rmtree(self.source) + self.source.mkdir() + script, _ = self.prepare({"x/y": b"nested"}, "true") + + # WHEN the sync is cut off just after unpacking x/y, before the manifest arrives. + partial = io.BytesIO() + with tarfile.open(fileobj=partial, mode="w|", format=tarfile.PAX_FORMAT) as archive: + info = tarfile.TarInfo("x/y") + info.size = len(b"nested") + info.mode = 0o644 + archive.addfile(info, io.BytesIO(b"nested")) + cut_off = subprocess.run(["sh", "-c", script], + input=build.vm_incoming_list(["x/y"]) + partial.getvalue(), + capture_output=True, env=self.environment()) + self.assertNotEqual(0, cut_off.returncode) + self.assertTrue((self.checkout / "x").is_dir()) + + # THEN the retry, which still sees x as a stale file, leaves the directory and completes. + retry = self.sync({"x/y": b"nested"}, "true") + self.assertEqual(0, retry.returncode, retry.stderr) + self.assertEqual(b"nested", (self.checkout / "x" / "y").read_bytes()) + + def cut_off_after_unpacking_x_y(self) -> None: + """Sync file x, make it the directory x/y locally, and cut that sync off just after + x/y was unpacked, before the manifest arrived.""" + self.assertEqual(0, self.sync({"x": b"file"}, "true").returncode) + shutil.rmtree(self.source) + self.source.mkdir() + script, _ = self.prepare({"x/y": b"nested"}, "true") + partial = io.BytesIO() + with tarfile.open(fileobj=partial, mode="w|", format=tarfile.PAX_FORMAT) as archive: + info = tarfile.TarInfo("x/y") + info.size = len(b"nested") + info.mode = 0o644 + archive.addfile(info, io.BytesIO(b"nested")) + cut_off = subprocess.run(["sh", "-c", script], + input=build.vm_incoming_list(["x/y"]) + partial.getvalue(), + capture_output=True, env=self.environment()) + self.assertNotEqual(0, cut_off.returncode) + self.assertTrue((self.checkout / "x").is_dir()) + shutil.rmtree(self.source) + self.source.mkdir() + + def test_a_file_restored_after_an_unpack_interrupted_inside_its_directory_syncs(self) -> None: + # GIVEN a sync of x/y over the file x, cut off once x/y was unpacked. + self.cut_off_after_unpacking_x_y() + + # WHEN x is a file again locally and the next sync runs. + retry = self.sync({"x": b"file again"}, "true") + + # THEN x/y, which only the cut-off sync brought, is removed, so the file x unpacks. + self.assertEqual(0, retry.returncode, retry.stderr) + self.assertEqual(b"file again", (self.checkout / "x").read_bytes()) + manifest = build.parse_vm_manifest((self.checkout / build.VM_MANIFEST).read_text(encoding="utf-8")) + self.assertEqual({"x"}, set(manifest)) + + def test_a_path_dropped_after_an_unpack_interrupted_is_deleted_on_the_retry(self) -> None: + # GIVEN a sync of x/y over the file x, cut off once x/y was unpacked. + self.cut_off_after_unpacking_x_y() + + # WHEN x/y becomes x/z locally and the next sync runs. + retry = self.sync({"x/z": b"other"}, "true") + + # THEN x/z arrives and x/y, which no manifest had named, is gone. + self.assertEqual(0, retry.returncode, retry.stderr) + self.assertEqual(b"other", (self.checkout / "x" / "z").read_bytes()) + self.assertFalse((self.checkout / "x" / "y").exists()) + + def test_a_manifest_changed_since_planning_stops_the_run_untouched(self) -> None: + # GIVEN a run planned against the VM's manifest. + self.assertEqual(0, self.sync({"a": b"a"}, "true").returncode) + script, archive = self.prepare({"a": b"a-changed", "b": b"b"}, "touch ran") + + # WHEN another run replaces that manifest before this one takes the lock. + manifest = self.checkout / build.VM_MANIFEST + manifest.write_text(manifest.read_text(encoding="utf-8") + "644\t1\t1\tother\n", + encoding="utf-8") + result = subprocess.run(["sh", "-c", script], input=archive, capture_output=True, + env=self.environment()) + + # THEN it exits for build.py to retry, marks why, and neither unpacks nor runs anything. + self.assertEqual(build.VM_BASELINE_CHANGED, result.returncode, result.stderr) + self.assertTrue((self.run_dir / build.VM_BASELINE_MARKER).exists()) + self.assertEqual(b"a", (self.checkout / "a").read_bytes()) + self.assertFalse((self.checkout / "b").exists()) + self.assertFalse((self.checkout / "ran").exists()) + + def test_a_run_keeps_its_own_log_snapshot_and_exit_status(self) -> None: + # GIVEN a build run whose command writes a test log and fails. + run_line = "mkdir -p build-vm/meson-logs && echo 'Ok: 7' > build-vm/meson-logs/testlog.txt && exit 4" + + # WHEN it runs. + result = self.sync({"a": b"a"}, run_line, builddir="build-vm") + + # THEN its status comes back and its log is copied where only this run's token reaches it. + self.assertEqual(4, result.returncode, result.stderr) + self.assertEqual(b"Ok: 7\n", (self.run_dir / "testlog.txt").read_bytes()) + self.assertFalse((self.run_dir / "testlog.json").exists()) + + def test_jobs_are_sized_from_memory_and_capped_at_the_cpu_count(self) -> None: + # GIVEN a build run that sizes its jobs. + # WHEN it prints the value it chose. + result = self.sync({"a": b"a"}, 'echo "jobs=$jobs"', builddir="build-vm", size_jobs=True) + + # THEN it is at least one and no more than the CPUs. + self.assertEqual(0, result.returncode, result.stderr) + jobs = int(result.stdout.decode().strip().split("=")[1]) + self.assertGreaterEqual(jobs, 1) + self.assertLessEqual(jobs, os.cpu_count()) + + def test_a_bsd_host_runs_build_bsd_sh_and_sizes_jobs_from_sysctl(self) -> None: + # GIVEN a host whose uname says FreeBSD, with 8 GiB of memory and 8 CPUs by sysctl. + self.stub("uname", "echo FreeBSD") + self.stub("sysctl", 'case "$2" in hw.physmem) echo 8589934592 ;; hw.ncpu) echo 8 ;;' + ' *) exit 1 ;; esac') + + # WHEN a build run reports what it chose. + result = self.sync({"a": b"a"}, 'echo "$build_script jobs=$jobs"', builddir="build-vm", + size_jobs=True) + + # THEN it picks the BSD build script and one job per 1.5 GiB: five, under the eight CPUs. + self.assertEqual(0, result.returncode, result.stderr) + self.assertEqual("build-bsd.sh jobs=5", result.stdout.decode().strip()) + + def test_a_linux_host_runs_build_linux_sh(self) -> None: + # GIVEN a host whose uname says Linux. + self.stub("uname", "echo Linux") + # WHEN a run reports its build script. + result = self.sync({"a": b"a"}, 'echo "$build_script"') + # THEN it is the Linux one. + self.assertEqual(0, result.returncode, result.stderr) + self.assertEqual("build-linux.sh", result.stdout.decode().strip()) + + def test_an_unsupported_system_is_refused_before_anything_is_unpacked(self) -> None: + # GIVEN a host whose uname names a system build.py vm has no build script for. + self.stub("uname", "echo Darwin") + + # WHEN a run is sent to it. + result = self.sync({"a": b"a"}, "touch ran") + + # THEN it fails with exit 2, naming the system, and neither unpacks nor runs anything. + self.assertEqual(2, result.returncode) + self.assertIn(b"unsupported remote system Darwin", result.stderr) + self.assertFalse(self.checkout.exists()) + + def test_a_second_run_waits_for_the_lock_and_then_runs(self) -> None: + # GIVEN another process holding the ssh user's lock, as a running suite does. + root = self.home / build.VM_REMOTE_ROOT + root.mkdir(parents=True) + holder = subprocess.Popen( + [sys.executable, "-c", + "import fcntl, sys\n" + f"lock = open({str(root / '.lock')!r}, 'a')\n" + "fcntl.flock(lock, fcntl.LOCK_EX)\n" + "print('held', flush=True)\n" + "sys.stdin.read()\n"], + stdin=subprocess.PIPE, stdout=subprocess.PIPE, text=True) + try: + self.assertEqual("held", holder.stdout.readline().strip()) + script, archive = self.prepare({"a": b"a"}, "echo ran") + # The run reads its archive only once it holds the lock, so the archive + # comes from a file: written into a pipe, it can exceed the pipe's buffer + # (16 KiB on FreeBSD) and block this test before the holder lets go. + archive_file = Path(self.temp.name) / "archive.tar" + archive_file.write_bytes(archive) + + # WHEN a run starts while it is held. + with archive_file.open("rb") as stdin: + waiting = subprocess.Popen(["sh", "-c", script], stdin=stdin, + stdout=subprocess.PIPE, stderr=subprocess.PIPE, + env=self.environment()) + time.sleep(1) + + # THEN it has not run yet. + self.assertIsNone(waiting.poll()) + finally: + holder.communicate(timeout=30) + + # WHEN the holder exits THEN the waiting run takes the lock, says it waited, and runs. + stdout, stderr = waiting.communicate(timeout=30) + self.assertEqual(0, waiting.returncode, stderr) + self.assertIn(b"another run holds the VM lock; waiting for it", stderr) + self.assertEqual(b"ran\n", stdout) + + +@unittest.skipUnless(os.name == "posix" and shutil.which("sh"), "needs a POSIX sh") +class VmFetchTests(unittest.TestCase): + """vm_fetch appends one shell command line to the ssh command, so ["sh", "-c"] + stands in for ssh here and runs that line locally.""" + + def setUp(self) -> None: + self.temp = tempfile.TemporaryDirectory() + self.home = Path(self.temp.name) + self.environ = os.environ.copy() + os.environ["HOME"] = str(self.home) + + def tearDown(self) -> None: + os.environ.clear() + os.environ.update(self.environ) + self.temp.cleanup() + + def test_a_present_file_is_read_and_a_missing_one_is_none(self) -> None: + # GIVEN a log under the remote user's home and a path with no file. + (self.home / "co" / "meson-logs").mkdir(parents=True) + (self.home / "co" / "meson-logs" / "testlog.txt").write_bytes(b"Ok: 84\n") + + # WHEN both are fetched. + # THEN the log's bytes come back, and the missing file is None. + self.assertEqual(b"Ok: 84\n", build.vm_fetch(["sh", "-c"], "co/meson-logs/testlog.txt")) + self.assertIsNone(build.vm_fetch(["sh", "-c"], "co/meson-logs/testlog.json")) + + def test_an_ssh_failure_is_an_error_not_a_missing_file(self) -> None: + # GIVEN an ssh that fails as it does when the VM refuses the key (exit 255). + refused = ["sh", "-c", "echo 'Permission denied (publickey)' >&2; exit 255", "ssh"] + + # WHEN a log is fetched through it. + # THEN the failure is raised with its exit status, never reported as "no log". + with self.assertRaisesRegex(build.VmError, r"exit 255.*Permission denied"): + build.vm_fetch(refused, "co/meson-logs/testlog.txt") + + +@unittest.skipUnless(shutil.which("sh"), "needs sh") +class BuildScriptJobsAndHardeningTests(unittest.TestCase): + """build.py vm runs build-linux.sh or build-bsd.sh with --jobs, and build.py + linux and bsd pass --hardening to them, so both scripts take both options.""" + + def run_wrapper(self, wrapper: Path, *args: str) -> subprocess.CompletedProcess: + return subprocess.run(["sh", str(wrapper), *args], capture_output=True, text=True) + + def test_jobs_takes_a_positive_whole_number(self) -> None: + for wrapper in (LINUX_BUILD_WRAPPER, BSD_BUILD_WRAPPER): + # GIVEN unusable --jobs values. + for bad in ("0", "-1", "two", "", "07"): + # WHEN the script parses them THEN it fails before configuring anything. + result = self.run_wrapper(wrapper, "--jobs", bad) + with self.subTest(wrapper=wrapper.name, bad=bad): + self.assertNotEqual(0, result.returncode) + self.assertIn("--jobs", result.stderr) + # GIVEN a valid value WHEN followed by --help THEN parsing accepts it and documents it. + accepted = self.run_wrapper(wrapper, "--jobs", "3", "--help") + with self.subTest(wrapper=wrapper.name): + self.assertEqual(0, accepted.returncode, accepted.stderr) + self.assertIn("--jobs ", accepted.stdout) + + def test_jobs_and_hardening_reach_meson(self) -> None: + for wrapper in (LINUX_BUILD_WRAPPER, BSD_BUILD_WRAPPER): + # GIVEN --jobs 3 and --hardening. + # WHEN the script previews its commands. + result = self.run_wrapper(wrapper, "--dry-run", "--jobs", "3", "--hardening", + "--builddir", "build-x") + + # THEN Meson configures hardening, compiles with three jobs and tests three at once. + with self.subTest(wrapper=wrapper.name): + self.assertEqual(0, result.returncode, result.stderr) + self.assertIn("-Dhardening=true", result.stdout) + self.assertIn("meson compile -C build-x -j 3", result.stdout) + self.assertIn("--num-processes 3", result.stdout) + + +@unittest.skipUnless(os.name == "posix" and shutil.which("sh"), "needs a POSIX sh") +class RemoteHostSetupTests(unittest.TestCase): + def dry_run(self, os_release: str) -> subprocess.CompletedProcess: + with tempfile.TemporaryDirectory() as temp: + release = Path(temp) / "os-release" + release.write_text(os_release, encoding="utf-8") + return subprocess.run( + ["sh", str(REMOTE_HOST_SETUP), "--dry-run", "--os-release", str(release), + "--tools-venv", "/opt/venv", "--bin-dir", "/opt/bin"], + capture_output=True, text=True) + + def test_a_debian_family_host_gets_every_library_build_linux_checks_and_a_current_meson(self) -> None: + for os_release in ('ID=ubuntu\n', 'ID=debian\n', 'ID=linuxmint\nID_LIKE="ubuntu debian"\n'): + # GIVEN Ubuntu, Debian, or a derivative that names them in ID_LIKE. + # WHEN the setup is previewed. + result = self.dry_run(os_release) + + # THEN it installs the libraries build-linux.sh requires, and Meson 1.11+ linked into the bin dir. + with self.subTest(os_release=os_release): + self.assertEqual(0, result.returncode, result.stderr) + for package in ("libsodium-dev", "libssl-dev", "libpq-dev", "libcurl4-openssl-dev", + "libpng-dev", "libturbojpeg0-dev", "python3-venv", "clang"): + self.assertIn(f" {package}", result.stdout) + self.assertIn("'meson>=1.11' ninja", result.stdout) + self.assertIn("ln -sf /opt/venv/bin/meson /opt/bin/meson", result.stdout) + + def test_a_freebsd_host_gets_the_ci_packages_from_pkg_and_no_virtual_environment(self) -> None: + # GIVEN FreeBSD. + # WHEN the setup is previewed. + result = self.dry_run('NAME=FreeBSD\nVERSION_ID="15.1"\nID=freebsd\n') + + # THEN pkg installs Meson, the libraries build-bsd.sh needs and python3 for build.py vm's lock, + # and no apt-get or virtual environment is involved. + self.assertEqual(0, result.returncode, result.stderr) + self.assertIn("pkg install -y meson ninja llvm", result.stdout) + for package in ("libsodium", "openssl", "postgresql17-client", "curl", "python3", "gmake", + "catch2", "png", "libjpeg-turbo"): + self.assertIn(f" {package}", result.stdout) + self.assertNotIn("apt-get", result.stdout) + self.assertNotIn("venv", result.stdout) + + def test_another_distribution_is_refused_before_anything_runs(self) -> None: + # GIVEN a distribution outside the Debian family. + # WHEN the setup is previewed. + result = self.dry_run('ID=fedora\nID_LIKE="rhel centos"\n') + + # THEN it fails naming the distribution and prints no command. + self.assertNotEqual(0, result.returncode) + self.assertIn("ID=fedora", result.stderr) + self.assertNotIn("+ ", result.stdout) + + def test_it_is_not_the_wsl_setup_script(self) -> None: + # GIVEN the remote-host setup script. + # WHEN its text is read. + script = REMOTE_HOST_SETUP.read_text(encoding="utf-8") + + # THEN it neither runs nor tells the user to run the WSL wrapper. + self.assertNotIn("build-wsl.sh", script) + + +if __name__ == "__main__": + unittest.main() diff --git a/tests/tooling/test_check_elf_hardening.py b/tests/tooling/test_check_elf_hardening.py new file mode 100644 index 000000000..6382af5cd --- /dev/null +++ b/tests/tooling/test_check_elf_hardening.py @@ -0,0 +1,151 @@ +#!/usr/bin/env python3 +# SPDX-License-Identifier: GPL-3.0-or-later +"""scripts/check-elf-hardening.sh against the readelf of each platform (ADR-0757). + +The FreeBSD release evidence reported BIND_NOW missing from a binary that had it: +the BSDs' elftoolchain readelf prints the dynamic flags as "(FLAGS) BIND_NOW" and +"(FLAGS_1) NOW", and the gate knew only GNU binutils' "(FLAGS_1) Flags: NOW". These +tests run the gate with a fake readelf on PATH that prints each tool's output. +""" +from __future__ import annotations + +import os +import shutil +import stat +import subprocess +import tempfile +import unittest +from pathlib import Path + + +REPO_ROOT = Path(__file__).resolve().parents[2] +GATE = REPO_ROOT / "scripts" / "check-elf-hardening.sh" + +GNU_HEADER_PIE = """ELF Header: + Class: ELF64 + Type: DYN (Position-Independent Executable file) + Machine: Advanced Micro Devices X86-64 +""" + +ELFTOOLCHAIN_HEADER_PIE = """ELF Header: + Class: ELF64 + Type: DYN (Shared object file) + Machine: AMD x86-64 +""" + +ELFTOOLCHAIN_HEADER_EXEC = """ELF Header: + Class: ELF64 + Type: EXEC (Executable file) + Machine: AMD x86-64 +""" + +PROGRAM_HEADERS = """Program Headers: + Type Offset VirtAddr PhysAddr FileSiz MemSiz Flg Align + LOAD 0x000000 0x0000000000000000 0x0000000000000000 0x001000 0x001000 R 0x1000 + GNU_RELRO 0x002000 0x0000000000002000 0x0000000000002000 0x000100 0x001000 R 0x1 + GNU_STACK 0x000000 0x0000000000000000 0x0000000000000000 0x000000 0x000000 RW 0 +""" + +GNU_DYNAMIC_NOW = """Dynamic section at offset 0x2000 contains 4 entries: + Tag Type Name/Value + 0x0000000000000001 (NEEDED) Shared library: [libc.so.6] + 0x000000000000001e (FLAGS) BIND_NOW + 0x000000006ffffffb (FLAGS_1) Flags: NOW PIE + 0x0000000000000000 (NULL) 0x0 +""" + +# What /usr/bin/readelf printed on FreeBSD 15.1 for a build linked with -Wl,-z,now. +ELFTOOLCHAIN_DYNAMIC_NOW = """Dynamic section at offset 0x2000 contains 4 entries: + Tag Type Name/Value + 0x0000000000000001 (NEEDED) Shared library: [libc.so.7] + 0x000000000000001e (FLAGS) BIND_NOW + 0x000000006ffffffb (FLAGS_1) NOW + 0x0000000000000000 (NULL) 0x0 +""" + +ELFTOOLCHAIN_DYNAMIC_LAZY = """Dynamic section at offset 0x2000 contains 3 entries: + Tag Type Name/Value + 0x0000000000000001 (NEEDED) Shared library: [libc.so.7] + 0x000000006ffffffb (FLAGS_1) PIE + 0x0000000000000000 (NULL) 0x0 +""" + + +class CheckElfHardeningTests(unittest.TestCase): + def run_gate(self, header: str, dynamic: str) -> subprocess.CompletedProcess[str]: + sh = shutil.which("sh") + if sh is None: + self.skipTest("sh is not available") + with tempfile.TemporaryDirectory(prefix="check-elf-hardening-") as scratch: + root = Path(scratch) + outputs = {"header": header, "program": PROGRAM_HEADERS, "dynamic": dynamic} + for name, text in outputs.items(): + (root / f"{name}.txt").write_text(text, encoding="utf-8", newline="\n") + fake_bin = root / "bin" + fake_bin.mkdir() + readelf = fake_bin / "readelf" + readelf.write_text( + "#!/bin/sh\n" + 'case "$1" in\n' + f' -h) cat "{root}/header.txt" ;;\n' + f' -lW) cat "{root}/program.txt" ;;\n' + f' -d) cat "{root}/dynamic.txt" ;;\n' + " *) exit 2 ;;\n" + "esac\n", + encoding="utf-8", + newline="\n", + ) + readelf.chmod(readelf.stat().st_mode | stat.S_IXUSR | stat.S_IXGRP | stat.S_IXOTH) + binary = root / "merovingian-server" + binary.write_bytes(b"\x7fELF") + environment = dict(os.environ) + environment["PATH"] = str(fake_bin) + os.pathsep + environment.get("PATH", "") + return subprocess.run( + [sh, str(GATE), str(binary)], + check=False, + text=True, + capture_output=True, + env=environment, + ) + + def test_a_hardened_binary_passes_with_gnu_readelf(self) -> None: + # GIVEN GNU binutils' output for a PIE binary with BIND_NOW + # WHEN the gate reads it + result = self.run_gate(GNU_HEADER_PIE, GNU_DYNAMIC_NOW) + + # THEN every check passes + self.assertEqual(0, result.returncode, result.stdout + result.stderr) + self.assertIn("OK: BIND_NOW", result.stdout) + + def test_a_hardened_binary_passes_with_the_bsd_elftoolchain_readelf(self) -> None: + # GIVEN FreeBSD's elftoolchain output for a PIE binary with BIND_NOW + # WHEN the gate reads it + result = self.run_gate(ELFTOOLCHAIN_HEADER_PIE, ELFTOOLCHAIN_DYNAMIC_NOW) + + # THEN BIND_NOW is recognised and every check passes + self.assertEqual(0, result.returncode, result.stdout + result.stderr) + self.assertIn("OK: BIND_NOW", result.stdout) + self.assertIn("OK: PIE", result.stdout) + + def test_lazy_binding_still_fails_with_the_bsd_elftoolchain_readelf(self) -> None: + # GIVEN elftoolchain output whose FLAGS_1 holds PIE but not NOW, and no DT_FLAGS + # WHEN the gate reads it + result = self.run_gate(ELFTOOLCHAIN_HEADER_PIE, ELFTOOLCHAIN_DYNAMIC_LAZY) + + # THEN BIND_NOW fails, so the wider match did not let lazy binding through + self.assertEqual(1, result.returncode, result.stdout + result.stderr) + self.assertIn("FAIL: BIND_NOW", result.stderr) + + def test_an_executable_that_is_not_pie_fails(self) -> None: + # GIVEN elftoolchain output for an ET_EXEC binary, as FreeBSD's clang links without -pie + # WHEN the gate reads it + result = self.run_gate(ELFTOOLCHAIN_HEADER_EXEC, ELFTOOLCHAIN_DYNAMIC_NOW) + + # THEN PIE fails and nothing else does + self.assertEqual(1, result.returncode, result.stdout + result.stderr) + self.assertIn("FAIL: PIE", result.stderr) + self.assertIn("1 check(s) failed", result.stderr) + + +if __name__ == "__main__": + unittest.main() diff --git a/tests/tooling/test_complement_image.py b/tests/tooling/test_complement_image.py index 6a07e82ee..4341511cc 100644 --- a/tests/tooling/test_complement_image.py +++ b/tests/tooling/test_complement_image.py @@ -298,13 +298,23 @@ def test_every_rate_limit_the_config_sets_is_out_of_reach(self) -> None: # THEN none is low enough for a Complement run to meet. self.assert_out_of_reach(key) - def test_appservice_cleartext_http_is_allowed_to_the_docker_host_alone(self) -> None: + def test_appservice_cleartext_http_is_allowed_to_the_complement_hosts_alone(self) -> None: # GIVEN the rendered config. # WHEN the appservice cleartext host list is read. hosts = self.config.get("appservice.allow_cleartext_hosts") # THEN it names host.docker.internal, where Complement registers its appservices, - # and nothing else (ADR-0515). - self.assertEqual(hosts, "host.docker.internal") + # and localhost, where the hs_with_application_service blueprint registers one, + # and nothing else (ADR-0515, ADR-0750). + self.assertEqual(hosts, "host.docker.internal,localhost") + + def test_parallel_remote_joins_from_one_address_are_not_refused(self) -> None: + # GIVEN the rendered config. + # WHEN the per-client outbound call cap is read. + cap = self.config.get("server.http.client_outbound_max_per_client") + # THEN it is the global cap of the default request pool: Complement sends every + # request from one address and does not retry a 429, and at the default of three + # its parallel subtests' remote joins refused each other (ADR-0751). + self.assertEqual(cap, "16") def test_password_hashing_admits_complements_parallel_logins(self) -> None: # GIVEN the rendered config. diff --git a/tests/tooling/test_packages_workflow.py b/tests/tooling/test_packages_workflow.py index 1593f1a09..a6d119fce 100644 --- a/tests/tooling/test_packages_workflow.py +++ b/tests/tooling/test_packages_workflow.py @@ -283,15 +283,20 @@ def test_freebsd_packages_workflow_uses_the_supported_dependency_set(self) -> No self.assertIn("gmake openssl libsodium postgresql17-client python3", workflow) self.assertNotIn("postgresql17-client curl sqlite3 catch2", workflow) - def test_bsds_the_self_check_can_pass_request_full_relro(self): - # The self-check requires BIND_NOW, and the packaging scripts pass only - # -pie, so without this a FreeBSD or OpenBSD server refuses to start. + def test_bsds_the_self_check_can_pass_request_full_relro_and_pie(self): + # GIVEN the self-check requires BIND_NOW and PIE, and FreeBSD's clang + # links neither without being asked, so a FreeBSD or OpenBSD server + # built without them refuses to start (ADR-0757). meson_build = (REPO_ROOT / "meson.build").read_text(encoding="utf-8") - self.assertIn( - "if host_machine.system() in ['linux', 'freebsd', 'openbsd']\n" - " hardening_link_flags += relro_now_flags", - meson_build, - ) + opening = "if host_machine.system() in ['linux', 'freebsd', 'openbsd']\n" + + # WHEN the hardened link flags for those platforms are read + self.assertIn(opening, meson_build) + block = meson_build.split(opening, 1)[1].split("\nendif", 1)[0] + + # THEN they request both -pie and full RELRO + self.assertIn("hardening_link_flags += ['-pie']", block) + self.assertIn("hardening_link_flags += relro_now_flags", block) def test_freebsd_package_lists_the_files_it_ships(self) -> None: # GIVEN pkg-create(8) packages only what a plist or manifest lists, so a package diff --git a/tests/unit/test_appservice_registration.cpp b/tests/unit/test_appservice_registration.cpp index d5c88ff44..4dcfda765 100644 --- a/tests/unit/test_appservice_registration.cpp +++ b/tests/unit/test_appservice_registration.cpp @@ -13,6 +13,7 @@ #include "../support/InMemoryDatabaseConfig.hpp" #include "../support/MasterKey.hpp" #include "../support/RegistrationToken.hpp" +#include "../support/TempDirectory.hpp" #include "merovingian/appservice/AppserviceRegistration.hpp" #include "merovingian/appservice/AppserviceRegistry.hpp" #include "merovingian/appservice/EventInterestFacts.hpp" @@ -214,11 +215,13 @@ SCENARIO("loading a registration file from disk", "[appservice][registration]") { GIVEN("a registration file written to a temp path") { - auto const dir = std::filesystem::temp_directory_path(); - auto const path = dir / "merovingian-test-appservice-registration.json"; + auto const path = + merovingian::tests::unique_temporary_path("merovingian-test-appservice-registration", ".json"); { auto out = std::ofstream{path, std::ios::binary}; out << irc_bridge_registration_json(); + out.close(); + REQUIRE(out.good()); } WHEN("load_registration_file reads it") @@ -543,11 +546,12 @@ SCENARIO("AppserviceRegistry lookup and exclusivity checks", "[appservice][regis { GIVEN("a registry loaded from a single registration file") { - auto const dir = std::filesystem::temp_directory_path(); - auto const path = dir / "merovingian-test-appservice-registry.json"; + auto const path = merovingian::tests::unique_temporary_path("merovingian-test-appservice-registry", ".json"); { auto out = std::ofstream{path, std::ios::binary}; out << irc_bridge_registration_json(); + out.close(); + REQUIRE(out.good()); } auto const loaded = load_registrations({path.string()}); REQUIRE(loaded.findings.empty()); @@ -1155,7 +1159,7 @@ SCENARIO("A registration in the form Complement writes is loaded", "[appservice] { GIVEN("a registration file written exactly as Complement writes one") { - auto const path = std::filesystem::temp_directory_path() / "merovingian-test-complement-appservice.yaml"; + auto const path = merovingian::tests::unique_temporary_path("merovingian-test-complement-appservice", ".yaml"); { auto out = std::ofstream{path, std::ios::binary}; out << "id: my_as_id\n" @@ -1173,6 +1177,8 @@ SCENARIO("A registration in the form Complement writes is loaded", "[appservice] " regex: .*\n" " rooms: []\n" " aliases: []\n"; + out.close(); + REQUIRE(out.good()); } WHEN("it is loaded") @@ -1207,7 +1213,7 @@ SCENARIO("A registration with an unknown plain key is still refused", "[appservi { GIVEN("an otherwise valid registration with a mistyped top-level key") { - auto const path = std::filesystem::temp_directory_path() / "merovingian-test-unknown-key-appservice.yaml"; + auto const path = merovingian::tests::unique_temporary_path("merovingian-test-unknown-key-appservice", ".yaml"); { auto out = std::ofstream{path, std::ios::binary}; out << "id: typo_bridge\n" @@ -1220,6 +1226,8 @@ SCENARIO("A registration with an unknown plain key is still refused", "[appservi " users: []\n" " rooms: []\n" " aliases: []\n"; + out.close(); + REQUIRE(out.good()); } WHEN("it is loaded") @@ -1247,13 +1255,15 @@ namespace [[nodiscard]] auto load_with_url(std::string const& url_json, std::vector const& allowed_hosts = {}) -> merovingian::appservice::LoadRegistrationsResult { - auto const path = std::filesystem::temp_directory_path() / "merovingian-test-appservice-cleartext.json"; + auto const path = merovingian::tests::unique_temporary_path("merovingian-test-appservice-cleartext", ".json"); { auto out = std::ofstream{path, std::ios::binary}; out << R"({"id": "cleartext-bridge", "url": )" << url_json << R"(, "as_token": "as-token-cleartext-secret", "hs_token": "hs-token-cleartext-secret", "sender_localpart": "_cleartext_bot", "namespaces": {"users": [{"exclusive": true, "regex": "@_cleartext_.*"}], "aliases": [], "rooms": []}})"; + out.close(); + REQUIRE(out.good()); } auto loaded = load_registrations({path.string()}, "example.org", allowed_hosts); std::filesystem::remove(path); diff --git a/tests/unit/test_client_outbound_proxy.cpp b/tests/unit/test_client_outbound_proxy.cpp index e6f3ffa6d..d3bd3a0a1 100644 --- a/tests/unit/test_client_outbound_proxy.cpp +++ b/tests/unit/test_client_outbound_proxy.cpp @@ -7,10 +7,10 @@ #include "../support/InMemoryDatabaseConfig.hpp" #include "../support/MasterKey.hpp" #include "merovingian/config/Config.hpp" +#include "merovingian/homeserver/ClientOutboundProxy.hpp" #include "merovingian/homeserver/HomeserverRuntime.hpp" #include "merovingian/homeserver/OutboundDeadline.hpp" #include "merovingian/homeserver/RuntimeStartResult.hpp" -#include "merovingian/homeserver/ClientOutboundProxy.hpp" #include @@ -23,20 +23,23 @@ using namespace merovingian::homeserver; -SCENARIO("The client outbound proxy caps are derived from the main request pool size", "[http-2][out-7][homeserver]") +SCENARIO("The client outbound proxy caps are derived from the main request pool size", + "[http-2][out-7][homeserver][adr0751]") { GIVEN("the shipped pool size") { auto const policy = default_client_outbound_proxy_policy(); - THEN("the global cap is half the default pool of sixteen and each client may hold one call") + THEN("the global cap is half the default pool of sixteen and each client may hold three calls") { // HTTP-1 made the pool configurable (server.http.request_threads, // default 16); the default policy follows the default pool. REQUIRE(default_main_request_pool_threads == 16U); REQUIRE(policy.global_cap == default_main_request_pool_threads / 2U); REQUIRE(policy.global_cap == 8U); - REQUIRE(policy.per_client_cap == 1U); + // ADR-0751: three, the default of server.http.client_outbound_max_per_client. + REQUIRE(default_client_outbound_max_per_client == 3U); + REQUIRE(policy.per_client_cap == 3U); } THEN("directory lookups get at most ten seconds and remote media at most thirty") @@ -59,6 +62,16 @@ SCENARIO("The client outbound proxy caps are derived from the main request pool REQUIRE(client_outbound_proxy_policy_for_pool(1U).global_cap == 1U); REQUIRE(client_outbound_proxy_policy_for_pool(0U).global_cap == 1U); } + + THEN("the per-client cap is the configured one, never zero and never above the global cap (ADR-0751)") + { + REQUIRE(client_outbound_proxy_policy_for_pool(32U, 5U).per_client_cap == 5U); + REQUIRE(client_outbound_proxy_policy_for_pool(32U, 16U).per_client_cap == 16U); + REQUIRE(client_outbound_proxy_policy_for_pool(32U, 200U).per_client_cap == 16U); + REQUIRE(client_outbound_proxy_policy_for_pool(4U).per_client_cap == 2U); + REQUIRE(client_outbound_proxy_policy_for_pool(32U, 0U).per_client_cap == 1U); + REQUIRE(client_outbound_proxy_policy_for_pool(0U, 3U).per_client_cap == 1U); + } } } @@ -79,18 +92,25 @@ SCENARIO("A runtime starts with the default client outbound proxy policy and an } } -SCENARIO("Admission to client outbound proxying honours the runtime policy", "[http-2][out-7][homeserver]") +SCENARIO("Admission to client outbound proxying honours the runtime policy", "[http-2][out-7][homeserver][adr0751]") { GIVEN("a runtime with the default policy") { auto runtime = HomeserverRuntime{}; - WHEN("one client holds a slot") + WHEN("one client holds as many slots as the per-client cap, three by default (ADR-0751)") { - auto const held = admit_client_outbound_proxy(runtime, "192.0.2.1"); - REQUIRE(held.has_value()); + REQUIRE(runtime.client_outbound_proxy_policy.per_client_cap == 3U); + auto held = std::vector>{}; + auto all_admitted = true; + for (auto index = 0U; index < runtime.client_outbound_proxy_policy.per_client_cap; ++index) + { + held.push_back(admit_client_outbound_proxy(runtime, "192.0.2.1")); + all_admitted = all_admitted && held.back().has_value(); + } + REQUIRE(all_admitted); - THEN("the same client is refused and another is admitted") + THEN("the same client is refused one more and another is admitted") { REQUIRE_FALSE(admit_client_outbound_proxy(runtime, "192.0.2.1").has_value()); REQUIRE(admit_client_outbound_proxy(runtime, "192.0.2.2").has_value()); diff --git a/tests/unit/test_config_operational_limits.cpp b/tests/unit/test_config_operational_limits.cpp index 5f06e3a17..8fcc9991f 100644 --- a/tests/unit/test_config_operational_limits.cpp +++ b/tests/unit/test_config_operational_limits.cpp @@ -2,8 +2,8 @@ // SPDX-License-Identifier: GPL-3.0-or-later #include "merovingian/config/Config.hpp" -#include "merovingian/config/ReloadPlan.hpp" #include "merovingian/config/ConfigParser.hpp" +#include "merovingian/config/ReloadPlan.hpp" #include #include @@ -526,3 +526,68 @@ SCENARIO("The joint cap on the tips of senders without power is parsed, bounded } } } + +// ADR-0751: how many client-triggered outbound calls one client address may have in flight. Three by default; the +// policy never lets it exceed the global cap of request_threads / 2. +SCENARIO("The per-client outbound call cap defaults to three and can be tuned", + "[config][operational-limits][client-outbound][adr0751]") +{ + GIVEN("a configuration that does not mention the per-client outbound cap") + { + auto const parsed = merovingian::config::parse_key_value_config(""); + + THEN("each client address may have three outbound calls in flight") + { + REQUIRE(parsed.config.server().http.client_outbound_max_per_client == 3U); + REQUIRE(merovingian::config::validate(parsed.config).empty()); + } + } + + GIVEN("a configuration that sets it") + { + auto const parsed = + merovingian::config::parse_key_value_config("server.http.client_outbound_max_per_client=16\n"); + + WHEN("it is parsed and validated") + { + auto const findings = merovingian::config::validate(parsed.config); + + THEN("the value is applied without findings") + { + REQUIRE(parsed.findings.empty()); + REQUIRE(findings.empty()); + REQUIRE(parsed.config.server().http.client_outbound_max_per_client == 16U); + } + } + + WHEN("a reload plan is built against the defaults") + { + auto const plan = merovingian::config::build_reload_plan(merovingian::config::Config{}, parsed.config); + + THEN("it is a restart-required change, because the policy is built at startup") + { + REQUIRE(plan.changes().size() == 1U); + REQUIRE(plan.reloadable_change_count() == 0U); + REQUIRE(plan.restart_required_change_count() == 1U); + } + } + } + + GIVEN("an invalid per-client outbound cap") + { + auto const input = GENERATE( + "server.http.client_outbound_max_per_client=0\n", "server.http.client_outbound_max_per_client=129\n", + "server.http.client_outbound_max_per_client=-1\n", "server.http.client_outbound_max_per_client=many\n"); + + WHEN("the configuration is parsed and validated") + { + auto const parsed = merovingian::config::parse_key_value_config(input); + auto const findings = merovingian::config::validate(parsed.config); + + THEN("the operator gets an error instead of a cap that refuses every call or has no bound") + { + REQUIRE((!parsed.findings.empty() || !findings.empty())); + } + } + } +} diff --git a/tests/unit/test_database_persistence.cpp b/tests/unit/test_database_persistence.cpp index 273ebdd14..b0c8eed71 100644 --- a/tests/unit/test_database_persistence.cpp +++ b/tests/unit/test_database_persistence.cpp @@ -2292,7 +2292,7 @@ SCENARIO("Physical migration files load into validated migration plans", "[datab { GIVEN("a temporary migration file with explicit statement names") { - auto const directory = merovingian::tests::temporary_directory() / "merovingian-migration-file-test"; + auto const directory = merovingian::tests::unique_temporary_path("merovingian-migration-file-test", ""); std::filesystem::create_directories(directory); auto const file = directory / "003_policy_rules.sql"; { @@ -2300,6 +2300,8 @@ SCENARIO("Physical migration files load into validated migration plans", "[datab output << "-- merovingian-migration version=3 name=policy_rules direction=upgrade\n"; output << "-- statement create_policy_rules_extra\n"; output << "CREATE TABLE policy_rules_extra (rule_id TEXT PRIMARY KEY)\n"; + output.close(); + REQUIRE(output.good()); } WHEN("the migration directory is loaded") diff --git a/tests/unit/test_federation_request_routing.cpp b/tests/unit/test_federation_request_routing.cpp index 6cbaf4c86..71aa10fa1 100644 --- a/tests/unit/test_federation_request_routing.cpp +++ b/tests/unit/test_federation_request_routing.cpp @@ -12,6 +12,7 @@ namespace { +using merovingian::homeserver::federation_request_room_ids; using merovingian::homeserver::federation_request_should_bypass_worker; using merovingian::homeserver::federation_worker_room_id_from_request; using merovingian::homeserver::is_federation_key_server_endpoint; @@ -815,3 +816,213 @@ SCENARIO("A space hierarchy request is answered by main, not by the shard 0 work } } } + +// ADR-0748: Complement TestEventAuth asked GET /event_auth for an event a client had just sent and got 404 from a +// worker whose reload of the room was still pending. A pending reload sends a read main can serve to main; writes stay +// with the worker, and a failed reload sends every request for the room to main. +SCENARIO("A room read is served by main while the worker's reload of the room is pending", + "[federation][routing][room-sync][adr0748]") +{ + using merovingian::homeserver::federation_room_request_served_by_main; + using merovingian::homeserver::RoomSyncStatus; + + GIVEN("a read of a room's event auth, its state IDs, and a transaction for the room") + { + auto const event_auth = make_request("/_matrix/federation/v1/event_auth/%21room%3Aexample.org/%24event"); + auto const state_ids = make_request("/_matrix/federation/v1/state_ids/%21room%3Aexample.org?event_id=%24e"); + auto transaction = make_request("/_matrix/federation/v1/send/txn1"); + transaction.method = "PUT"; + // v1.19 defines get_missing_events as POST; it reads, so it falls back with the GET reads. + auto missing_events = make_request("/_matrix/federation/v1/get_missing_events/%21room%3Aexample.org"); + missing_events.method = "POST"; + // A GET of it is no route the spec defines, and send_join is a write, so neither falls back while pending. + auto const missing_events_get = make_request("/_matrix/federation/v1/get_missing_events/%21room%3Aexample.org"); + auto send_join = make_request("/_matrix/federation/v2/send_join/%21room%3Aexample.org/%24join"); + send_join.method = "PUT"; + // ADR-0759: the leave and knock templates name the user's current membership, so they are reads of the room. + auto const make_leave = make_request("/_matrix/federation/v1/make_leave/%21room%3Aexample.org/%40bob%3Ar.org"); + auto const make_knock = make_request("/_matrix/federation/v1/make_knock/%21room%3Aexample.org/%40bob%3Ar.org"); + // A PUT of make_leave is no route the spec defines, so it does not fall back. + auto make_leave_put = make_leave; + make_leave_put.method = "PUT"; + + WHEN("the worker's copy of the room is current") + { + auto const ok = RoomSyncStatus{"ok", 3U}; + + THEN("the worker serves all of them") + { + REQUIRE_FALSE(federation_room_request_served_by_main(event_auth, ok)); + REQUIRE_FALSE(federation_room_request_served_by_main(state_ids, ok)); + REQUIRE_FALSE(federation_room_request_served_by_main(transaction, ok)); + REQUIRE_FALSE(federation_room_request_served_by_main(make_leave, ok)); + REQUIRE_FALSE(federation_room_request_served_by_main(make_knock, ok)); + } + } + + WHEN("main has changed the room and the worker has not reloaded it yet") + { + auto const pending = RoomSyncStatus{"pending", 4U}; + + THEN("main serves the reads and the worker still takes the transaction") + { + REQUIRE(federation_room_request_served_by_main(event_auth, pending)); + REQUIRE(federation_room_request_served_by_main(state_ids, pending)); + REQUIRE(federation_room_request_served_by_main(missing_events, pending)); + REQUIRE_FALSE(federation_room_request_served_by_main(missing_events_get, pending)); + REQUIRE_FALSE(federation_room_request_served_by_main(send_join, pending)); + REQUIRE_FALSE(federation_room_request_served_by_main(transaction, pending)); + REQUIRE(federation_room_request_served_by_main(make_leave, pending)); + REQUIRE(federation_room_request_served_by_main(make_knock, pending)); + REQUIRE_FALSE(federation_room_request_served_by_main(make_leave_put, pending)); + } + } + + WHEN("the worker failed to reload the room") + { + auto const failed = RoomSyncStatus{"failed", 4U}; + + THEN("main serves every request for the room") + { + REQUIRE(federation_room_request_served_by_main(event_auth, failed)); + REQUIRE(federation_room_request_served_by_main(state_ids, failed)); + REQUIRE(federation_room_request_served_by_main(transaction, failed)); + } + } + } +} + +// ADR-0756: Complement TestKnockingInMSC3787Room created a room and had a remote server join it at once. The worker had +// not loaded the room yet, so it took room version 12, computed another event ID than the join's and answered 400 +// "membership event does not match endpoint, path or origin". A room the worker holds no copy of is served by main. +SCENARIO("Every request for a room the worker has never loaded is served by main", + "[federation][routing][room-sync][adr0756]") +{ + using merovingian::homeserver::federation_room_request_served_by_main; + using merovingian::homeserver::RoomSyncStatus; + + GIVEN("a send_join, a transaction and a read for a room") + { + auto send_join = make_request("/_matrix/federation/v2/send_join/%21room%3Aexample.org/%24join"); + send_join.method = "PUT"; + auto transaction = make_request("/_matrix/federation/v1/send/txn1"); + transaction.method = "PUT"; + auto const event_auth = make_request("/_matrix/federation/v1/event_auth/%21room%3Aexample.org/%24event"); + + WHEN("main has just created the room and the worker's first load of it is pending") + { + auto const first_load = RoomSyncStatus{"pending", 1U, false}; + + THEN("main serves the writes as well as the read") + { + REQUIRE(federation_room_request_served_by_main(send_join, first_load)); + REQUIRE(federation_room_request_served_by_main(transaction, first_load)); + REQUIRE(federation_room_request_served_by_main(event_auth, first_load)); + } + } + + WHEN("the worker holds the room and a later reload of it is pending") + { + auto const reload = RoomSyncStatus{"pending", 2U, true}; + + THEN("the writes still go to the worker, as ADR-0748 has them") + { + REQUIRE_FALSE(federation_room_request_served_by_main(send_join, reload)); + REQUIRE_FALSE(federation_room_request_served_by_main(transaction, reload)); + REQUIRE(federation_room_request_served_by_main(event_auth, reload)); + } + } + } +} + +// ADR-0756: a /send goes whole to the shard its first PDU picks, so main checks every room it names. A later PDU for a +// room the worker has never loaded would otherwise reach a worker that takes room version 12 for it. +SCENARIO("Every room a federation request concerns is listed for the sync status check", + "[federation][routing][room-id][adr0756]") +{ + GIVEN("a /send transaction whose PDUs name two rooms, one of them twice, and one PDU without a room_id") + { + auto const body = std::string{R"({"origin":"remote.example","origin_server_ts":1,"pdus":[)" + R"({"room_id":"!first:example.com","type":"m.room.message"},)" + R"({"room_id":"!second:example.com","type":"m.room.message"},)" + R"({"type":"m.room.message"},)" + R"({"room_id":"!first:example.com","type":"m.room.message"}]})"}; + auto request = make_request("/_matrix/federation/v1/send/txn-rooms", body); + request.method = "PUT"; + + WHEN("the rooms it concerns are listed") + { + auto const room_ids = federation_request_room_ids(request); + + THEN("each named room appears once, in the order it first appears, and the shard key is the first") + { + REQUIRE(room_ids == std::vector{"!first:example.com", "!second:example.com"}); + REQUIRE(federation_worker_room_id_from_request(request) == room_ids.front()); + } + } + } + + GIVEN("a /send transaction naming more distinct rooms than a transaction may carry PDUs") + { + auto body = std::string{R"({"origin":"remote.example","origin_server_ts":1,"pdus":[)"}; + auto const count = merovingian::federation::max_federation_transaction_pdus + 5U; + for (auto index = 0U; index < count; ++index) + { + body += (index == 0U ? "" : ","); + body += R"({"room_id":"!room)" + std::to_string(index) + R"(:example.com"})"; + } + body += "]}"; + auto request = make_request("/_matrix/federation/v1/send/txn-many", body); + request.method = "PUT"; + + WHEN("the rooms it concerns are listed") + { + auto const room_ids = federation_request_room_ids(request); + + THEN("no more are listed than a transaction the worker accepts can carry") + { + REQUIRE(room_ids.size() == merovingian::federation::max_federation_transaction_pdus); + } + } + } + + GIVEN("/send bodies that name no room: not JSON, an array, an object without pdus, and pdus that are not objects") + { + auto const bodies = std::vector{ + R"({"pdus":[)", + R"([{"room_id":"!array:example.com"}])", + R"({"origin":"remote.example","edus":[]})", + R"({"pdus":[1,"!text:example.com",[{"room_id":"!nested:example.com"}],{"room_id":7}]})", + }; + + WHEN("the rooms each concerns are listed") + { + THEN("none is listed, and the request has no shard key") + { + for (auto const& body : bodies) + { + auto request = make_request("/_matrix/federation/v1/send/txn-none", body); + request.method = "PUT"; + INFO(body); + REQUIRE(federation_request_room_ids(request).empty()); + REQUIRE(federation_worker_room_id_from_request(request).empty()); + } + } + } + } + + GIVEN("a room-scoped path request and a request that names no room") + { + auto const state = make_request("/_matrix/federation/v1/state/%21pathroom%3Aexample.com?event_id=%24e"); + auto const version = make_request("/_matrix/federation/v1/version"); + + WHEN("the rooms each concerns are listed") + { + THEN("the path request lists its decoded room, and the other lists none") + { + REQUIRE(federation_request_room_ids(state) == std::vector{"!pathroom:example.com"}); + REQUIRE(federation_request_room_ids(version).empty()); + } + } + } +} diff --git a/tests/unit/test_homeserver_vertical_slice.cpp b/tests/unit/test_homeserver_vertical_slice.cpp index 50e331809..c902d3ef3 100644 --- a/tests/unit/test_homeserver_vertical_slice.cpp +++ b/tests/unit/test_homeserver_vertical_slice.cpp @@ -244,6 +244,30 @@ SCENARIO("The joining server puts its own origin and timestamp on a membership t } } + // ADR-0745: signing adds to the `signatures` an event holds, so a template's own would travel in this server's + // join and be stored with it, one filed under this server's name included. + GIVEN("a validated make_join template carrying hashes and signatures the resident chose") + { + auto const body = std::string{ + R"({"room_version":"10","event":{"type":"m.room.member","room_id":"!room:remote.example.org","sender":"@alice:local.example.org","state_key":"@alice:local.example.org","content":{"membership":"join"},"hashes":{"sha256":"cmVzaWRlbnQ"},"signatures":{"local.example.org":{"ed25519:forged":"Zm9yZ2Vk"},"remote.example.org":{"ed25519:a":"cmVzaWRlbnQ"}}}})"}; + auto parsed = merovingian::homeserver::validate_make_join_response("!room:remote.example.org", + "@alice:local.example.org", body); + REQUIRE(parsed.ok); + REQUIRE(member_of(parsed.event, "signatures") != nullptr); + + WHEN("this server claims the template") + { + merovingian::homeserver::claim_membership_event_template(parsed.event, "local.example.org", 42); + + THEN("neither the template's hashes nor its signatures are left to be signed over or sent") + { + REQUIRE(member_of(parsed.event, "hashes") == nullptr); + REQUIRE(member_of(parsed.event, "signatures") == nullptr); + REQUIRE(string_of(parsed.event, "sender") == "@alice:local.example.org"); + } + } + } + GIVEN("a validated make_join template without origin or origin_server_ts") { auto const body = std::string{ diff --git a/tests/unit/test_logger_writers.cpp b/tests/unit/test_logger_writers.cpp index fadd7b632..f8aac6618 100644 --- a/tests/unit/test_logger_writers.cpp +++ b/tests/unit/test_logger_writers.cpp @@ -225,7 +225,7 @@ SCENARIO("A message logged before the writers start is not lost", "[observabilit { GIVEN("a logger with console and file sinks and no writers yet") { - auto const directory = merovingian::tests::temporary_directory() / "merovingian-logger-writers-test"; + auto const directory = merovingian::tests::unique_temporary_path("merovingian-logger-writers-test", ""); std::filesystem::create_directories(directory); auto const log_path = directory / "before-start.log"; @@ -310,7 +310,7 @@ SCENARIO("A CRITICAL log is written synchronously even after writer threads have { GIVEN("a logger whose writers are running and a file sink") { - auto const directory = merovingian::tests::temporary_directory() / "merovingian-logger-critical-test"; + auto const directory = merovingian::tests::unique_temporary_path("merovingian-logger-critical-test", ""); std::filesystem::create_directories(directory); auto const log_path = directory / "critical.log"; diff --git a/tests/unit/test_outbound_discovery_cache.cpp b/tests/unit/test_outbound_discovery_cache.cpp index 731834144..d8d8509e2 100644 --- a/tests/unit/test_outbound_discovery_cache.cpp +++ b/tests/unit/test_outbound_discovery_cache.cpp @@ -21,11 +21,12 @@ #include "merovingian/config/ServerConfig.hpp" #include "merovingian/federation/CachedServerDiscovery.hpp" #include "merovingian/federation/DiscoveryCachePolicy.hpp" +#include "merovingian/federation/DispatchWorker.hpp" #include "merovingian/federation/OutboundTransaction.hpp" #include "merovingian/homeserver/HomeserverRuntime.hpp" #include "merovingian/homeserver/KeyServerCache.hpp" -#include "merovingian/homeserver/RuntimeStartResult.hpp" #include "merovingian/homeserver/RoomService.hpp" +#include "merovingian/homeserver/RuntimeStartResult.hpp" #include #include @@ -145,3 +146,64 @@ SCENARIO("A cache serving only successes never serves a remembered failure", } } } + +// ADR-0749: Complement TestToDeviceMessagesOverFederation/stopped_server. A sender restarted while the receiver was +// down replayed its queued to-device EDU, and discovery failed and was remembered for 60 s. The retries after the +// receiver came back were then refused from the cache, so the backoff grew past the test's 30 s window. Outbound +// delivery is paced by its own backoff, so its resolver asks the network again instead of serving a remembered failure. +SCENARIO("The dispatch worker's resolver never serves a remembered discovery failure", + "[federation][discovery-cache][outbound][dispatch][adr0749]") +{ + GIVEN("a destination whose discovery failed and was remembered by a caller that accepts failures") + { + auto network = ScriptedDiscoveryNetwork{}; + network.reachable = false; + auto cached = merovingian::federation::CachedServerDiscovery{network, 60'000U, [] { + return 1'000U; + }}; + REQUIRE_FALSE(cached.discover("remote.example.org:8448", 1U).discovery_allowed); + auto const resolver = merovingian::federation::make_dispatch_resolver(&network, &cached, 1U); + + WHEN("the dispatch worker resolves it while it is still down") + { + auto const before = network.calls.load(); + auto const down = resolver("remote.example.org:8448"); + + THEN("the network is asked again and the attempt fails") + { + REQUIRE_FALSE(down.has_value()); + REQUIRE(network.calls.load() > before); + } + } + + WHEN("the destination comes back within the cache lifetime and the worker retries") + { + network.reachable = true; + auto const up = resolver("remote.example.org:8448"); + + THEN("the retry is resolved from the live answer") + { + REQUIRE(up.has_value()); + REQUIRE(up->discovery_allowed); + REQUIRE_FALSE(up->pinned_addresses.empty()); + } + } + } + + GIVEN("no cache, only a network") + { + auto network = ScriptedDiscoveryNetwork{}; + auto const resolver = merovingian::federation::make_dispatch_resolver(&network, nullptr, 1U); + + WHEN("a destination is resolved") + { + auto const result = resolver("remote.example.org:8448"); + + THEN("the network answers it") + { + REQUIRE(result.has_value()); + REQUIRE(network.calls.load() > 0U); + } + } + } +} diff --git a/tests/unit/test_read_markers.cpp b/tests/unit/test_read_markers.cpp index f41d168b2..b35d45526 100644 --- a/tests/unit/test_read_markers.cpp +++ b/tests/unit/test_read_markers.cpp @@ -6,9 +6,9 @@ #include "merovingian/database/PersistentReceipt.hpp" #include "merovingian/database/PersistentStore.hpp" #include "merovingian/sync/ReadMarkers.hpp" +#include "merovingian/sync/SlidingSyncRoomBuilder.hpp" #include "merovingian/sync/UnreadEventClassifier.hpp" #include "merovingian/sync/UnreadEventOutcome.hpp" -#include "merovingian/sync/SlidingSyncRoomBuilder.hpp" #include @@ -300,3 +300,63 @@ SCENARIO("Unread counts and read markers read only the room they are asked about } } } + +// Spec: Client-Server API, POST /rooms/{roomId}/receipt/{receiptType}/{eventId}, 400: "The event_id is not related to +// the thread_id". The chain is followed for at most 3 hops, as for event_thread_id (ADR-0746). +SCENARIO("event_relates_to follows an event's relation chain to the target within three hops", + "[sync][read-markers][adr0746]") +{ + GIVEN("a root, a reply to it, an edit of the reply, an edit of that edit, a parent that is not JSON, and a parent " + "held in another room") + { + auto store = merovingian::database::PersistentStore{}; + add_event(store, "$root", R"({"type":"m.room.message","content":{"body":"root"}})", 1U); + add_event(store, "$reply", relates_to("m.thread", "$root"), 2U); + add_event(store, "$edit", relates_to("m.replace", "$reply"), 3U); + add_event(store, "$edit2", relates_to("m.replace", "$edit"), 4U); + add_event(store, "$broken", "not json", 5U); + auto elsewhere = merovingian::database::PersistentEvent{}; + elsewhere.event_id = "$elsewhere"; + elsewhere.room_id = "!other:example.org"; + elsewhere.sender_user_id = "@bob:example.org"; + elsewhere.json = relates_to("m.thread", "$root"); + elsewhere.stream_ordering = 6U; + store.events.push_back(std::move(elsewhere)); + + auto const relates = [&](std::string const& json, std::string_view target) { + auto const parsed = merovingian::canonicaljson::parse_lossless(json); + auto const* object = std::get_if(&parsed.value.storage()); + REQUIRE(object != nullptr); + return merovingian::sync::event_relates_to(store, "!room:example.org", *object, target); + }; + + WHEN("events are judged against the root") + { + THEN("an event reaching it in one, two or three hops relates to it") + { + REQUIRE(relates(relates_to("m.thread", "$root"), "$root")); + REQUIRE(relates(relates_to("m.replace", "$reply"), "$root")); + REQUIRE(relates(relates_to("m.replace", "$edit"), "$root")); + } + THEN("one that needs a fourth hop does not") + { + REQUIRE_FALSE(relates(relates_to("m.replace", "$edit2"), "$root")); + } + THEN("one with no relation, an empty related event ID, or a related event ID that is not a string does " + "not") + { + REQUIRE_FALSE(relates(R"({"type":"m.room.message","content":{"body":"x"}})", "$root")); + REQUIRE_FALSE(relates(relates_to("m.thread", ""), "$root")); + REQUIRE_FALSE( + relates(R"({"type":"m.room.message","content":{"m.relates_to":{"event_id":7}}})", "$root")); + } + THEN("a chain through a parent the room does not hold, one that is not JSON, or one held only in another " + "room ends without relating") + { + REQUIRE_FALSE(relates(relates_to("m.replace", "$missing"), "$root")); + REQUIRE_FALSE(relates(relates_to("m.replace", "$broken"), "$root")); + REQUIRE_FALSE(relates(relates_to("m.replace", "$elsewhere"), "$root")); + } + } + } +} diff --git a/tests/unit/test_worker_hardening_threads.cpp b/tests/unit/test_worker_hardening_threads.cpp index d50238593..6ba15c9c0 100644 --- a/tests/unit/test_worker_hardening_threads.cpp +++ b/tests/unit/test_worker_hardening_threads.cpp @@ -248,13 +248,18 @@ SCENARIO("Logging before hardening starts no thread, so Landlock and seccomp cov { GIVEN("a directory the worker may read and a master key file outside it") { - auto const base = merovingian::tests::temporary_directory() / "merovingian-iso1-hardening-test"; + auto const base = merovingian::tests::unique_temporary_path("merovingian-iso1-hardening-test", ""); auto const allowed_dir = base / "allowed"; auto const secret_dir = base / "secret"; std::filesystem::create_directories(allowed_dir); std::filesystem::create_directories(secret_dir); auto const master_key = secret_dir / "master.key"; - std::ofstream{master_key} << "not a real key"; + { + auto out = std::ofstream{master_key}; + out << "not a real key"; + out.close(); + REQUIRE(out.good()); + } WHEN("a process logs, applies Landlock and worker seccomp, and only then starts the logger's writers") { diff --git a/tests/unit/test_worker_landlock.cpp b/tests/unit/test_worker_landlock.cpp index 9283122f6..5ca5a396e 100644 --- a/tests/unit/test_worker_landlock.cpp +++ b/tests/unit/test_worker_landlock.cpp @@ -619,7 +619,7 @@ SCENARIO("A real Landlock ruleset denies access outside its allowlist and permit { GIVEN("an allowed directory and a separate, unrelated directory holding a secret file") { - auto const base = merovingian::tests::temporary_directory() / "merovingian-landlock-test"; + auto const base = merovingian::tests::unique_temporary_path("merovingian-landlock-test", ""); auto const allowed_dir = base / "allowed"; auto const denied_dir = base / "denied"; std::filesystem::create_directories(allowed_dir); @@ -630,10 +630,14 @@ SCENARIO("A real Landlock ruleset denies access outside its allowlist and permit { auto out = std::ofstream{allowed_file}; out << "not secret"; + out.close(); + REQUIRE(out.good()); } { auto out = std::ofstream{denied_file}; out << "secret"; + out.close(); + REQUIRE(out.good()); } WHEN("a forked child restricts itself to only the allowed directory and probes both files")