docs(observability): cache evictions carry a reason label and are exported from startup

[lukekim]

Summary

Two merged runtime PRs changed what the results-cache metrics report, and neither was reflected in the docs:

• *_cache_evictions gained a reason label and now counts invalidation (spiceai/spiceai#12791). moka's RemovalCause::was_evicted covers only Expired and Size, which left out the cause that dominates an accelerated deployment: invalidate_entries_if — how a refresh or a DML write drops a table's entries — delivers Explicit. The counter is now split size / expired / invalidated. The same PR made every cache counter publish at zero during CacheMetrics::init, because a LazyLock instrument registers only on first deref, so a healthy runtime exported no eviction series at all — indistinguishable from a broken exporter or a bad scrape config.
• results_cache_stale_rejections is a new counter (spiceai/spiceai#12703) for lookups that found an entry but refused to serve it because a table the result read had since been invalidated. These are also counted as misses.

The docs said *_cache_evictions was "Total number of cache evictions due to size or TTL limits" with no dimensions — which now under-counts by exactly the dominant cause and omits the label an operator needs to tell cache pressure from routine refresh invalidation.

Changes

website/docs/features/observability/index.md:

• All three *_cache_evictions rows (results_, search_results_, embeddings_) get the reason breakdown and a reason dimension in place of —.
• New results_cache_stale_rejections row.
• Preamble note that cache counters are published at zero from startup, so an absent series means a scrape/exporter problem rather than an idle cache.

website/docs/features/caching/index.md (the cross-page site — it carries its own metrics table and a sample /metrics scrape):

• *_cache_evictions description updated, plus a reason value table and a note on why invalidated is split out.
• results_cache_stale_rejections documented alongside its relationship to results_cache_misses.
• Sample scrape output updated to show the labelled eviction series and the new counter.

Verified against origin/trunk, not the merged diffs: EvictionReason::{Size,Expired,Invalidated} and its KeyValue::new("reason", …) plus publish_counters_at_zero() in crates/cache/src/metrics.rs; the RemovalCause → EvictionReason mapping in crates/cache/src/lru_cache.rs; and the single CachedQueryResult::record_stale_rejection() call site in crates/cache/src/lib.rs.

Note for a reviewer (upstream, not fixed here)

Two things worth an upstream issue rather than a docs change:

1. The EVICTIONS .with_description() string on trunk reads "Entries dropped by table invalidation are counted by the invalidations metric instead" — but no *_cache_invalidations metric exists anywhere in crates/, and eviction_reason() still maps RemovalCause::Explicit to EvictionReason::Invalidated. The description (exported as Prometheus HELP text) contradicts the code; the docs here follow the code. The sample scrape in caching/index.md therefore shows a shortened HELP line rather than reproducing that sentence.
2. search_results_cache_stale_rejections and embeddings_cache_stale_rejections are registered and published at zero, but nothing increments them — only the SQL results cache has the validated-read path. They are intentionally left out of the tables rather than documented with a meaning they never carry.

Source PRs

• fix(cache): count an invalidation as an eviction, and export the counters before one fires (fixes #12687) spiceai#12791 — fix(cache): count an invalidation as an eviction, and export the counters before one fires
• fix(cache): never serve a result whose tables changed after it read them spiceai#12703 — fix(cache): never serve a result whose tables changed after it read them

Versioned-docs propagation

Addition — vNext only. Both commits post-date v2.1.4 (tagged 2026-08-05) and git tag --contains is empty for each, so versioned_docs/version-2.1.x/ correctly documents the unlabelled counter.

Test plan

• cd website && npm run build passes (Docusaurus throws on broken links / undefined tags)
• Versioned-docs propagation checked — addition, vNext only
• Files updated: 2

[github-actions]

✅ Pull with Spice Passed

Passing checks:

• ✅ Title meets minimum length requirement (10 characters)
• ✅ Has at least one of the required labels: area/blog, area/docs, area/cookbook, dependencies
• ✅ No banned labels detected
• ✅ Has at least one assignee: lukekim

[github-actions]

🚀 deployed to https://82842210.spiceai-org-website.pages.dev

[github-actions]

🚀 deployed to https://6788227c.spiceai-org-website.pages.dev