Extend problem cache with hardware provenance metadata#4835
Extend problem cache with hardware provenance metadata#4835danieyan-amd wants to merge 9 commits into
Conversation
Two changes to problem_cache.cpp:
1. load(): Project deserialized keys to only {name, problem} so that
extra metadata fields in the JSON don't break cache key matching.
Previously, the full JSON object (all fields) was used as the map
key, causing 100% cache misses when metadata was present.
2. save(): Enrich each key with hardware provenance before writing:
gpu_arch, cu_count, graphics_clock_mhz, memory_clock_mhz,
memory_bus_bits, vram_bytes, wavefront_size, regs_per_block,
max_threads_per_cu. Queried once via hipGetDeviceProperties at
session end — negligible performance cost.
The in-memory map always uses {name, problem} keys for O(1) lookups.
The on-disk JSON carries additional hardware context for traceability.
On load, the extra fields are projected away, preserving fast matching.
|
Sorry Chris, I didnt mean to hit ready for review. |
There was a problem hiding this comment.
Pull request overview
This PR updates the GPU problem cache persistence format to remain resilient to extra on-disk metadata while also recording hardware provenance for traceability.
Changes:
- In
load(), deserialize into a temporary map and project keys down to{name, problem}to prevent metadata fields from breaking cache-key matching. - In
save(), enrich persisted keys with HIP device properties (e.g., arch, CU count, clocks, VRAM) before writing the JSON file.
| // Enrich keys with hardware provenance metadata on write. | ||
| // This runs once at session end — negligible cost. | ||
| hipDeviceProp_t props{}; | ||
| auto status = hipGetDeviceProperties(&props, get_device_id()); |
| std::unordered_map<value, value> raw; | ||
| from_value(from_json_string(read_string(pc_path)), raw); | ||
| for(auto& [k, v] : raw) | ||
| { | ||
| auto projected = create_key(k.at("name").to<std::string>(), k.at("problem")); | ||
| cache[projected] = v; | ||
| } |
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## develop #4835 +/- ##
===========================================
- Coverage 92.86% 92.66% -0.20%
===========================================
Files 586 588 +2
Lines 30287 30412 +125
===========================================
+ Hits 28126 28180 +54
- Misses 2161 2232 +71 🚀 New features to boost your workflow:
|
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
| { | ||
| auto projected = create_key(k.at("name").to<std::string>(), k.at("problem")); | ||
| cache[projected] = v; | ||
| } |
There was a problem hiding this comment.
Make an extra copy can get slow with larger problem caches.
|
I think the metadata should be managed externally. In the future, we may use sqlite dbs to manage problem caches which may not be efficient to insert metadata like this. |
Do you mind if we move forward with the code as it is and optimize if it becomes a problem? We just want to incrementally improve the code and build up the database and caching logic. If it becomes a performance issue, we will optimize it since we are focusing on time-to-first-inference right now. |
I dont understand why this is needed. I would like to keep the code as simple as possible so it easier to update with newer features in the future such as multi-targets, multi-file, sqlite, etc. |
It's needed so that we can create problem cache databases that span different GPU's and GPU configurations. It will start off simple by selecting a single matching problem if there are multiple different matching hardware configurations but we plan to make the algorithm have better selection login in the future as our data improves. This is so that we can start collecting the data and work on building the database. Our next update will actually be adding multi-targets and sqlite support. Daniel has also been profiling different backend support and will be providing an abstraction interface to allow interchangeable backends for logging problem cache data. We need this change so we don't need to redo all the data collection work in the future when we want to select better solution when the hardware configuration changes but has the same gfx arch. |
Well this is not the way we would approach this. We would make the device a key. So instead of it being
Make sure to use type erasure instead of inheritance for this. |
I can look into this, and make the necessary changes |
Addresses PR review feedback: - Device (gpu_arch|cu_count|wavefront_size) used as composite cache key - Type-erased problem_cache_backend wrapper (no virtual inheritance) - JSON backend as default implementation - load()/save() in problem_cache rewritten to use backend abstraction
danieyan-amd
force-pushed
the
feature/problem-cache-schema-extension
branch
from
4b65154 to
3a47ddc
Compare
There was a problem hiding this comment.
Any Feed back on the changes to use device key? @pfultz2
Fixes licensing CI failure detected by tools/check_stamped.py. Supersedes PR ROCm#4915 (one-line year bump) by folding the fix into this PR.
- problem_cache_backend.hpp: rename trailing-underscore virtual methods to do_* prefix (readability-identifier-naming); replace && / ! with and / not (UseNamedLogicOperator); cast bitwise shift literals to unsigned (hicpp-signed-bitwise); add rule-of-five defaults to concept_t - problem_cache.cpp: replace ! with not on flagged lines; declare local backend as const auto& in has() and get() (cppcheck readability) - json_cache_backend.cpp: replace && / ! with and / not on flagged lines; declare loop variable e as const auto& in load_entries() - All three files re-run through clang-format (ROCm clang-format 21)
- problem_cache_backend.cpp: replace ! and && with not / and in fallback factory - json_cache_backend.hpp: cast bitwise shift literals to unsigned (hicpp-signed-bitwise) - All 3 files re-run through clang-format (ROCm clang-format 21) - Verified all 6 PR4835 files pass clang-format --dry-run -Werror
…k FPs
- src/targets/gpu/CMakeLists.txt: register problem_cache_backend.cpp and json_cache_backend.cpp in add_library(migraphx_gpu ...). Without this, the linker fails with undefined references to make_default_cache_backend / make_cache_backend / make_cache_backend_with_fallback, which is the root cause of the 5 failing Jenkins build gates (All Targets Release, HIP Clang Release, Navi4x, RTC Debug, MLIR Debug).
- src/targets/gpu/problem_cache_backend.cpp: add 'cppcheck-suppress returnDanglingLifetime' before the three return statements in make_cache_backend / make_default_cache_backend / make_cache_backend_with_fallback. These are inconclusive false positives: the temporary json_cache_backend{} is sunk into a std::unique_ptr<concept_t> by problem_cache_backend's templated constructor (canonical type-erasure pattern), so there is no dangling reference. The suppressions silence the cppcheck CI gate.
Locally verified with clang-format 21 dry-run -Werror (exit 0) and cppcheck 2.20.0 using the project's CMakeLists suppress list (exit 0) on all 6 PR4835-introduced cache files.
Two changes to problem_cache.cpp:
load(): Project deserialized keys to only {name, problem} so that extra metadata fields in the JSON don't break cache key matching. Previously, the full JSON object (all fields) was used as the map key, causing 100% cache misses when metadata was present.
save(): Enrich each key with hardware provenance before writing: gpu_arch, cu_count, graphics_clock_mhz, memory_clock_mhz, memory_bus_bits, vram_bytes, wavefront_size, regs_per_block, max_threads_per_cu. Queried once via hipGetDeviceProperties at session end — negligible performance cost.
The in-memory map always uses {name, problem} keys for O(1) lookups. The on-disk JSON carries additional hardware context for traceability. On load, the extra fields are projected away, preserving fast matching.
Motivation
Adding hardware info to the problem cache, and added handling of the hardware data when doing cache lookups for solutions.
Technical Details
Changelog Category
Add a
CHANGELOG.mdentry for any option other thanNot Applicable