feat(search): index mailing list traffic

[salmonumbrella]

What changed

• Persist RFC 2919 List-Id values for new email imports and repair existing archives with msgvault repair-list-ids.
• Add list: and list-id: filtering across local, semantic, hybrid, API, and daemon-backed search, with exact scoped filtering for list drill-down and deletion.
• Export List-Id to Parquet and add Lists views to the TUI and web analytics, including compatibility guards for older caches and daemons.

Why

Mailing-list traffic was searchable only through sender, subject, or body text. That makes a busy discussion list difficult to find, group, and safely act on because its messages can come from many senders. Indexing the canonical List-Id gives every supported local surface one stable mailing-list identity without re-fetching provider data.

Usage

msgvault repair-list-ids
msgvault repair-list-ids --apply
msgvault search 'list:announce.example.org'
msgvault search 'list-id:"Example Announcements"'

repair-list-ids is dry-run by default. Use --apply to persist the backfill, then rebuild the analytics cache through the normal cache workflow.

Fixes #674

[roborev-ci]

roborev: Combined Review (ac66672)

Verdict: Medium-severity issues remain in List-ID mappings, compatibility handling, command routing, Unicode matching, aggregate queries, and terminal rendering.

Medium

• Missing ViewLists mappings: Remote list aggregations become senders, and direct view_type=lists requests are rejected. Add "lists" to all view-type conversions, allowlists, and validation messages.
  Locations: internal/daemonclient/engine_adapter.go:124-142; internal/api/handlers.go:2073-2121

• Incomplete List-ID capability checks: search.Query.ListIDs is not checked, so list: operators may be sent to older daemons and interpreted as ordinary text. Apply the capability check whenever q.ListIDs is nonempty.
  Locations: internal/daemonclient/engine_adapter.go:866-925,947-969,1010-1022

• repair-list-ids rejected through the daemon: The CLI routes the command through the daemon, but the daemon allowlist omits it. Add it to the allowlist and cover the admission path.
  Locations: cmd/msgvault/cmd/repair_list_ids.go:21-25; internal/api/cli_handlers.go:1482-1534

• ASCII-only Unicode List-ID matching: SQLite LOWER causes Unicode List-Ids to fail case-insensitive matching and split into separate groups. Use the registered Unicode-lower expression for predicates and grouping.
  Locations: internal/store/api.go:621-630; internal/query/sqlite.go:180-188,584-592

• DuckDB ignores List-ID filters: Aggregate, subaggregate, and related statistics queries omit q.ListIDs. Add escaped, ANDed List-ID predicates to the shared aggregate search-condition builder.
  Location: internal/query/duckdb.go:858-872

• Terminal escape injection through List-Id values: Crafted List-Id values can contain ANSI/C1 control sequences and are used as aggregate keys in the Lists view. Sanitize aggregate keys and breadcrumb values with textutil.SanitizeTerminal before rendering.
  Locations: internal/query/sqlite.go:185; internal/tui/view.go:418

---

Reviewers: 2 done | Synthesis: codex, 10s | Total: 26m41s

[salmonumbrella]

this one was hard. unsure if this is the best way to build it.

[roborev-ci]

roborev: Combined Review (f358860)

Verdict: Medium-severity API gap identified; no security issues found.

Medium

• internal/api/deletions.go:43-86, 575-594 — StageDeletionFilter does not accept or forward ListID, so POST /api/v1/deletions cannot stage a deletion directly by mailing-list ID. Add ListID through the request type, validation, conversion, manifest mapping, OpenAPI schema, and generated client types.

---

Reviewers: 2 done | Synthesis: codex, 6s | Total: 34m36s

[roborev-ci]

roborev: Combined Review (ec5b592)

Verdict: No Critical, High, or Medium severity findings.

Low-severity findings were omitted per the requested criteria.

---

Reviewers: 2 done | Synthesis: codex, 6s | Total: 22m2s

[roborev-ci]

roborev: Combined Review (337848c)

Verdict: Two medium-severity issues require attention; no critical or high-severity findings were reported.

Medium

• Change-feed mappings omit list_id — internal/store/api.go:1291-1298
list_id changes emit change-feed events, but ChangedMessage and its SQL/API mappings omit the value. Incremental consumers may remain stale. Include nullable ListID in the change-feed model, query, HTTP response, OpenAPI schema, and generated clients.

• Legacy email messages are skipped during List-ID repair — internal/store/list_id_repair.go:255
  Repair only selects message_type = 'email', excluding rows with empty or NULL message types that are treated as legacy email messages elsewhere. Apply the same legacy-email predicate in both the repair and writer-lock queries.

---

Reviewers: 2 done | Synthesis: codex, 7s | Total: 31m0s

[roborev-ci]

roborev: Combined Review (c891790)

Verdict: One medium-severity issue found; low-severity findings omitted.

Medium

• internal/daemonclient/cli.go:857-883 — GetCLIHybridSearch checks req.Filter.ListID but does not serialize it into the generated search request. On current daemons, hybrid search can silently return results outside the requested mailing list. Forward list_id through the hybrid API and vector filter, or route structured list filtering through an endpoint that supports exact scoping.

---

Reviewers: 2 done | Synthesis: codex, 6s | Total: 22m17s