Key EPContext cache by compile identity

[Qiong Wu (qiowu) (DingmaomaoBJTU)]

Summary

• replace direct WinMLSession mtime-only EPContext reuse with a strict compile identity
• compile cache misses into immutable UUID generations selected by an atomic marker
• serialize same-identity compilation across threads/processes while isolating different identities
• hash source ONNX/external-data bytes, EP DLL, hardware, effective options, embed mode, ORT version, and generated artifacts
• publish stable CLI outputs atomically with content-addressed binary references while retaining the legacy stable binary alias

Problem

WinMLSession.compile() previously reused <model>_<device>_ctx.onnx whenever it was newer than the source ONNX. The cache key omitted the EP/source/version, hardware, provider/session options, embed mode, ORT version, external-data sidecars, and context binary integrity. Different configurations could therefore reuse or overwrite one shared EPContext artifact.

Behavior

• legacy or malformed caches without an identity marker rebuild once
• exact identities reuse an immutable generation
• source/external data, EP DLL, hardware, provider/session options, embed mode, or ORT changes produce a different cache namespace
• source changes observed while waiting or compiling are retried and never published under a stale identity
• custom opaque SessionOptions and uncertain source identity use unique non-cacheable generations
• marker/artifact failures fail closed without discarding a successfully compiled generation
• missing, modified, or timestamp-preserving substituted context binaries invalidate reuse
• compiler finalization consumes the exact session generation and never scans stale siblings after raw fallback
• public ONNX output remains stable, references an immutable content-addressed binary, and still publishes the documented stable binary alias

Validation

• affected unit suites: 125 passed, 6 existing skips
• focused cache/concurrency suite: 16 passed
• Ruff check and format check on all changed Python files
• mypy on all changed production modules
• real QNN/NPU keen_hominy compile:
  • first compile produced one identity marker and immutable generation
  • second compile logged Using cached EPContext
  • second compile reduced total time from about 13.0s to 1.75s
  • public model_qnn_ctx.onnx references model_qnn_ctx_qnn.<sha256>.bin
  • legacy model_qnn_ctx_qnn.bin alias remains present with matching size

uv lock --offline --check could not resolve uncached Linux onnx metadata on this ARM64 host. The lockfile TOML parses and the added unit test verifies that every direct project dependency is represented in both the editable package dependencies and requires-dist metadata; CI runs on the supported AMD64 environment.

[xieofxie]

The strict compile identity and per-identity locking solve a real correctness problem. However, the PR currently grows that fix into an unbounded immutable-generation/content-addressed publication system. I think the core direction is right, but the storage lifecycle should be bounded and the identity contract completed before taking on this maintenance surface.

[xieofxie]

The follow-up fixes cleanup after compile_to_file() failure and adds coverage for absolute file-backed options. Retention is still incomplete, though, and the new pruning path introduces a race with runtime session construction. I would not consider the cache lifecycle resolved until these cases are addressed.

[xieofxie]

The cache-hit lease now covers InferenceSession construction, which resolves the original prune-before-open race. However, the markerless lifecycle currently deletes the active model path immediately, and the new canonicalization/pruning logic has correctness gaps. These should be fixed before considering the lifecycle complete.

[xieofxie]

Quick re-review: no new high-confidence findings in this delta. The markerless EPContext is now retained through perf() reopen and finalized before reset, unpublished generations are cleaned on final-identity failure, pruning revalidates the marker while holding its lease, and file canonicalization is limited to explicitly declared provider-option keys. The previously raised lifecycle concerns appear resolved.

[xieofxie]

The previously raised cache lifecycle, pruning, markerless artifact, and file-backed option concerns are resolved. No remaining high-confidence blockers from my review.