fix(search): preserve tokenizer migration compatibility

[samzong]

Summary

• add an append-only trigram FTS migration while retaining unicode61 for short and ASCII queries
• preserve full trigram coverage for legacy or downgraded writes while indexing only new CJK messages in the trigram table
• retry deferred migrations safely and keep writes compatible before the new column is available
• cover short, mixed, ahead-version, rebuild, delete-trigger, downgrade-write, and lock-deferral behavior

Why

Short lexical terms were discarded by the trigram path, and a deferred migration could send trigram grammar to the active unicode61 index. Maintaining two full indexes also caused a real write and search regression. This keeps query grammar aligned with the available index, preserves upgrade and downgrade compatibility, and avoids the steady-state performance cost for new non-CJK messages.

Verification

• cargo test -p recall search_keeps_short_mixed_and_deferred_legacy_terms_without_embeddings
• cargo test -p recall migrate_v12_retries_after_worker_lock_deferral
• cargo test -p recall init_recreates_missing_trigram_fts_and_reindexes_existing_messages
• same-machine Criterion comparison against the parent commit: write and search medians returned to baseline range
• make check

[codspeed-hq]

Merging this PR will not alter performance

✅ 22 untouched benchmarks
⏩ 2 skipped benchmarks¹

---

_{Comparing fix/search-index-compatibility (8a98467) with main (3dd6496)}

Footnotes

1. 2 benchmarks were skipped, so the baseline results were used instead. If they were deleted from the codebase, click here and archive them to remove them from the performance reports. ↩