Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -307,7 +307,7 @@ as legal advice or a promise of a court outcome.
**Scalability** and **elasticity** — sync is peer to peer with no central bottleneck; a relay, if used,
forwards ciphertext and scales to zero between sessions. **Timeliness** — capture, hashing, and sealing
complete within a perceptible moment with no network in the loop, and the RFC 3161 token is fetched
asynchronously once the device is online; latency budgets for the local path are asserted in CI.
asynchronously once the device is online; [latency budgets for the local path](docs/performance-budget.md) are asserted in CI.
**Affordability** — the tool is free, runs on a tenant's existing phone, uses free public timestamp
authorities, and needs no paid infrastructure, because the people using it have none to spare. **Process capabilities** and
**producibility** — `make verify` reproduces the full gate; a release is one tagged, signed command.
Expand Down
11 changes: 8 additions & 3 deletions ROADMAP.md
Original file line number Diff line number Diff line change
Expand Up @@ -177,9 +177,14 @@ The courtroom rests on this; it gets the most scrutiny.
i18n parity test already guards this).
- **Plain-language & cognitive review.** *Objective:* usable under stress and across
reading levels. *Exit:* a reviewed plain-language pass of UI copy and the setup guide.
- **Low-end-device performance.** *Objective:* capture/seal/hash feel instant on an old
phone. *Exit:* a documented latency budget for the local path, checked on a low-end
target.
- *Shipped:* **Low-end-device performance budget.** A documented latency budget for the
local path — per-operation targets for content hashing, seal/store, custody append,
CRDT merge, and packet assembly — tied to a reference low-end device modeled as ~10×
slower than the CI runner, with network TSA latency explicitly excluded (it is
deferred, off the capture path). `tests/test_perf_budget.py` asserts the budget on
every CI run (`make test`), and `docs/performance-budget.md` records the model and the
tolerance band. *Remaining:* replace the 10× model with a measurement on named
reference hardware once mobile packaging lands (see workstream C).

### C. Apps, sync & platform

Expand Down
107 changes: 107 additions & 0 deletions docs/performance-budget.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,107 @@
<!-- SPDX-License-Identifier: AGPL-3.0-or-later -->
# Low-end-device performance budget

The people this tool is for document a habitability problem on **the only device they
have** — often an old, slow phone — and they do it while under stress. Capture, hashing,
and sealing therefore have to feel instant, with no network in the loop. This document
states the latency budget for that **local path**, ties it to a **reference low-end
device**, and explains how CI asserts the budget on every run so a regression is caught
before it ships.

The budget is enforced by [`tests/test_perf_budget.py`](../tests/test_perf_budget.py),
which runs under `make test` (`pytest -m "not integration"`) and hence in CI. The
constants in that module mirror the table below; keep the two in sync.

## What is (and is not) on the local path

The **local path** is everything that happens between the tenant pressing *capture* and
having an evidence-grade, verifiable record on the device, plus the two other operations
a tenant triggers by hand that must stay responsive:

1. **Content hash** — SHA-256 of a multi-megabyte capture (the fixity anchor).
2. **Seal / store** — `Vault.store_original_bytes` (and `Vault.seal_original`): encrypt
the original under the data key and write it immutably to disk, re-hashing to bind the
ciphertext to its content hash.
3. **Custody append** — `CustodyLog.append`: hash-link and Ed25519-sign one chain-of-custody
entry.
4. **CRDT merge** — `CaseDocument.merge`: join another replica's state during offline-first
sync.
5. **Packet assembly** — `build_packet`: render the signed bundle, the accessible
`packet.html`, and the PDF for a case.

**Explicitly excluded: RFC 3161 timestamp-authority network latency.** Fetching a trusted
timestamp is **deliberately off the capture path** — it is asynchronous and deferred. When
the device is offline (or the authority is slow) the capture is queued and shown as
*awaiting-timestamp*; the token is fetched later by `resolve_deferred` once the device is
online (see [`docs/evidence-method.md`](evidence-method.md) and `src/habitable/capture.py`).
Network round-trips to a public TSA are governed by that authority and the network, not by
this tool, so they are **not** part of this budget. Those paths are exercised by the
`integration`-marked tests, which do not run in the default gate.

## The reference low-end device

We do not have a lab of old phones in CI, so we model one. The reference target is an
**older low-end smartphone, assumed ~10× slower than the CI runner** for this workload
(single-threaded hashing, symmetric encryption, small-object JSON, and local file I/O).
That multiplier is intentionally conservative for CPU-and-disk-bound work like SHA-256 and
AEAD; a real budget on hardware should replace the model with a measurement (see *Revisiting
the model* below).

The relationship the budget rests on:

```
device_latency ≈ LOW_END_SLOWDOWN × ci_latency (LOW_END_SLOWDOWN = 10)

so CI asserts: ci_latency < device_budget / LOW_END_SLOWDOWN
```

If the CI-measured latency stays under `device_budget / 10`, then the reference device —
modeled as 10× slower — stays under its human-facing `device_budget`.

## The budget

`Device budget` is the human-facing target on the reference low-end phone. `CI-asserted`
is `device_budget ÷ 10`, the ceiling the test enforces on CI hardware.

| Operation | What is measured | Payload | Device budget | CI-asserted |
|-----------|------------------|---------|--------------:|------------:|
| Content hash | `sha256_bytes` of a capture | 4 MB | 500 ms | 50 ms |
| Seal / store | `store_original_bytes`: hash + AEAD-encrypt + write | 4 MB | 1000 ms | 100 ms |
| Custody append | `CustodyLog.append`: hash-link + Ed25519 sign | one entry | 200 ms | 20 ms |
| CRDT merge | `CaseDocument.merge` of another replica | ~20-issue case | 300 ms | 30 ms |
| Packet assembly | `build_packet`: bundle + `packet.html` + PDF | 1-item case | 2000 ms | 200 ms |

A capture as the tenant experiences it is the sum of hash + seal + two custody appends
(one at capture, one after the fixity re-check) plus the local model write — comfortably
inside a **perceptible moment** (well under one second) on the reference device, with the
trusted timestamp arriving later off the critical path.

## Tolerance band and why the test is not flaky

Timing tests are notorious for flaking. Two choices keep this one stable:

- **Best-of-N, not average.** Each operation is warmed up, then run N times and the
**minimum** elapsed time (`time.perf_counter`) is taken. Noise — GC pauses, scheduler
preemption, a busy CI box — can only make a run *slower*, so the minimum is a robust
lower bound on "how fast can this go here." A slow neighbor never fails the test.
- **≥5× headroom locally.** The CI-asserted ceilings above sit roughly 15–30× above the
measured local latency of each operation, so ordinary machine-to-machine variation
(a CI runner a few times slower than a dev laptop) still leaves comfortable margin. The
budget catches an *order-of-magnitude* regression — an accidental re-hash, an O(n²)
merge, re-encrypting bulk data on a passphrase change — not a few percent of jitter.

## Revisiting the model

The `LOW_END_SLOWDOWN = 10` figure is a **stated assumption**, not a measurement. When the
project can run the budget on real reference hardware (a named old Android phone, per the
mobile-packaging work in [`ROADMAP.md`](../ROADMAP.md)), replace the model with a measured
device latency and, if warranted, adjust the multiplier or move to a device-in-the-loop
check. Until then, the CI assertion guards against regressions on the *shape* of the local
path, which is where the risk of an accidental slowdown actually lives.

## Reproducing locally

```
uv run pytest tests/test_perf_budget.py -q # just the budget
make test # the full default gate (includes it)
```
171 changes: 171 additions & 0 deletions tests/test_perf_budget.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,171 @@
# SPDX-License-Identifier: AGPL-3.0-or-later
# Copyright 2026 Chelsea Kelly-Reif
"""Latency budget for the local (offline) path — asserted in CI.

Makes the README's "latency budgets for the local path are asserted in CI" true and
closes the roadmap's *Low-end-device performance* item. The full rationale — the
reference low-end device, the slowdown model, the tolerance band, and why network TSA
latency is deliberately excluded — lives in ``docs/performance-budget.md``. The budget
constants here MIRROR that document; keep the two in sync.

These tests are intentionally NOT marked ``integration``, so they run under
``make test`` (``pytest -m "not integration"``) and hence in CI. They use best-of-N with
``time.perf_counter`` and generous ceilings (>=5x local headroom), so noise can only make
a run slower and the minimum stays a robust lower bound — the budget catches an
order-of-magnitude regression, never a few percent of jitter.
"""

from __future__ import annotations

import math
import time
from collections.abc import Callable
from pathlib import Path

from habitable.canonical import sha256_bytes
from habitable.capture import capture
from habitable.evidence import CustodyAction
from habitable.packet import build_packet
from habitable.tsa import LocalRfc3161TSA
from habitable.vault import Vault

# The reference low-end phone is modeled as this many times slower than the CI runner
# for this (CPU + local-I/O bound) workload. See docs/performance-budget.md.
LOW_END_SLOWDOWN = 10.0

# Per-operation budgets on the *reference low-end device*, in milliseconds. These mirror
# the table in docs/performance-budget.md. CI runs on faster hardware, so each op must
# finish within ``budget / LOW_END_SLOWDOWN`` for the device budget to hold.
DEVICE_BUDGET_MS: dict[str, float] = {
"content_hash": 500.0,
"seal_store": 1000.0,
"custody_append": 200.0,
"crdt_merge": 300.0,
"packet_assembly": 2000.0,
}

# A "multi-MB capture": large enough to be a realistic photo/short-clip payload, small
# enough that best-of-N stays well under a second per op.
PAYLOAD_MB = 4
PAYLOAD_SIZE = PAYLOAD_MB * 1024 * 1024


def _ci_budget_ms(op: str) -> float:
"""The ceiling the test enforces on CI hardware: device budget / slowdown."""
return DEVICE_BUDGET_MS[op] / LOW_END_SLOWDOWN


def _best_ms(fn: Callable[[], object], *, repeats: int = 7, warmup: int = 2) -> float:
"""Best-of-N wall time for ``fn`` in milliseconds (minimum, after warmup)."""
for _ in range(warmup):
fn()
best = math.inf
for _ in range(repeats):
start = time.perf_counter()
fn()
best = min(best, time.perf_counter() - start)
return best * 1000.0


def _payload() -> bytes:
# A fixed non-trivial byte pattern; SHA-256 and AEAD both touch every byte, so the
# exact contents do not matter to the cost — only the size does.
return b"\xa5" * PAYLOAD_SIZE


def test_content_hash_within_budget() -> None:
"""SHA-256 of a multi-MB capture — the fixity anchor — is well under budget."""
payload = _payload()
elapsed = _best_ms(lambda: sha256_bytes(payload))
assert elapsed < _ci_budget_ms("content_hash"), (
f"content hash of {PAYLOAD_MB} MB took {elapsed:.2f} ms "
f"(CI budget {_ci_budget_ms('content_hash'):.0f} ms)"
)


def test_seal_store_within_budget(make_vault: Callable[..., Vault]) -> None:
"""Sealing a multi-MB original (hash + AEAD-encrypt + write) is within budget."""
vault = make_vault()
payload = _payload()
content_hash = sha256_bytes(payload)
counter = {"n": 0}

def seal() -> None:
counter["n"] += 1
vault.store_original_bytes(f"cap-{counter['n']}", payload, content_hash)

elapsed = _best_ms(seal)
assert elapsed < _ci_budget_ms("seal_store"), (
f"seal/store of {PAYLOAD_MB} MB took {elapsed:.2f} ms "
f"(CI budget {_ci_budget_ms('seal_store'):.0f} ms)"
)


def test_custody_append_within_budget(make_vault: Callable[..., Vault]) -> None:
"""Appending one signed chain-of-custody entry is within budget."""
vault = make_vault()
counter = {"n": 0}

def append() -> None:
counter["n"] += 1
vault.custody.append(
CustodyAction.CAPTURED,
f"cap-{counter['n']}",
actor="tester",
hlc=str(counter["n"]),
details={"media_type": "image/jpeg"},
identity=vault.identity,
)

elapsed = _best_ms(append)
assert elapsed < _ci_budget_ms("custody_append"), (
f"custody append took {elapsed:.3f} ms (CI budget {_ci_budget_ms('custody_append'):.0f} ms)"
)


def test_crdt_merge_within_budget(make_vault: Callable[..., Vault]) -> None:
"""Merging another replica's case state (CRDT join) is within budget."""
local = make_vault("local")
peer = make_vault("peer", passphrase="pw-peer")
for i in range(20):
issue = peer.document.add_issue(category="mold", issue_id=f"i{i}")
peer.document.add_timeline_entry(issue, "observed", "spreading")
peer.document.add_capture(
issue_id=issue,
content_hash=sha256_bytes(f"payload-{i}".encode()),
media_type="image/jpeg",
sealed_name=f"cap{i}.enc",
captured_at="2026-01-02T00:00:00Z",
capture_id=f"cap{i}",
)
state = peer.document.to_state()

elapsed = _best_ms(lambda: local.document.merge(state))
assert elapsed < _ci_budget_ms("crdt_merge"), (
f"CRDT merge took {elapsed:.3f} ms (CI budget {_ci_budget_ms('crdt_merge'):.0f} ms)"
)


def test_packet_assembly_within_budget(
make_vault: Callable[..., Vault],
make_jpeg: Callable[..., Path],
local_tsa: LocalRfc3161TSA,
tmp_path: Path,
) -> None:
"""Assembling an export packet (bundle + HTML + PDF) is within budget."""
vault = make_vault()
issue = vault.document.add_issue(category="mold", room="bath", title="Mold", issue_id="i1")
vault.document.add_timeline_entry(issue, "observed", "spreading")
capture(vault, make_jpeg(with_location=True), issue_id=issue, tsa=local_tsa)
counter = {"n": 0}

def assemble() -> None:
counter["n"] += 1
build_packet(vault, tmp_path / f"pkt-{counter['n']}", generated_at="2026-01-02T00:10:00Z")

# Packet assembly is the heaviest op (PDF rendering); fewer repeats keep runtime low.
elapsed = _best_ms(assemble, repeats=5, warmup=1)
assert elapsed < _ci_budget_ms("packet_assembly"), (
f"packet assembly took {elapsed:.2f} ms "
f"(CI budget {_ci_budget_ms('packet_assembly'):.0f} ms)"
)
Loading