Reduce dynamic-shape compile cost and select_module dispatch overhead

[chun-wan]

For models with a single non-fixed dynamic dimension, today's split_single_dyn_dim pass enumerates every integer in [min, max] and stamps out one static-shape submodule per integer, regardless of dynamic_dimension::optimals. Combined with a per-inference linear scan inside select_module::compute(), this makes wide [min, max] ranges (e.g. {1, 5000000, opt:[400000]}) unusable: ~16 days of compile, ~40 GB engine and +80 - 100 us of per-inference host overhead vs a hand-static engine on MI308X.

This change introduces three layered, opt-in / auto-guarded optimizations:

1. MIGRAPHX_DYN_DIM_BUCKET_BY_OPTIMALS=1 makes split_single_dyn_dim honour dynamic_dimension::optimals (one submodule per optimal plus min/max endpoints) and adds a smallest-compatible-bucket fallback in select_module::compute() (zero-pads on the ref backend). Collapses compile time and .mxr size to O(|optimals|).

2. MIGRAPHX_DYN_DIM_FREEZE_TO= adds a new freeze_dyn_dim pass that runs before split_single_dyn_dim and rewrites every non-fixed dynamic parameter to a static shape at size N. Downstream sees a fully-static program -- no submodules, no select_module, no per-inference dispatch. Matches hand-written-static inference latency.

3. select_module::compute() gets a per-instance dispatch cache. Hot loops with stable input shape skip the linear scan and the per-call get_input_parameter_names / get_parameter_shapes allocations. Auto-activates when select_module has 4+ submodules; kill-switch MIGRAPHX_DISABLE_SELECT_MODULE_CACHE=1 for A/B benchmarks and emergency rollback.

Validated on MI308X (gfx942, ROCm 7.2.0):

• Pointwise compile: 40.5 s (legacy max=256) -> 0.48 s (bucket max=5M).
• Pointwise inference: 215 us legacy -> 121 us bucket + cache ->
  115 us freeze (= true-static control at 115 us).
• All existing unit + ref tests pass, plus 16 new test cases: 4 split_single_dyn_dim, 4 freeze_dyn_dim, 2 ref/freeze_dyn_dim, 6 ref/select_module (incl. cache hot-loop and alternating-shape regression). Validation ran with cache enabled and with the kill-switch on; bitwise-identical outputs verified across frozen / bucket / static engines.

Addresses #1023 (Dynamic Shape Support umbrella) and #4618 (ONNX-Runtime MIGraphX EP recompiling per shape).

Motivation

Technical Details

Changelog Category

Add a CHANGELOG.md entry for any option other than Not Applicable

• • Added: New functionality.
• • Changed: Changes to existing functionality.
• • Removed: Functionality or support that has been removed. (Compared to a previous release)
• • Optimized: Component performance that has been optimized or improved.
• • Resolved Issues: Known issues from a previous version that have been resolved.
• • Not Applicable: This PR is not to be included in the changelog.

[codecov]

Codecov Report

❌ Patch coverage is 96.00000% with 3 lines in your changes missing coverage. Please review.

Files with missing lines	Patch %	Lines
src/include/migraphx/op/select_module.hpp	95.38%	3 Missing ⚠️

Additional details and impacted files

@@             Coverage Diff             @@
##           develop    #4911      +/-   ##
===========================================
+ Coverage    92.66%   92.68%   +0.02%     
===========================================
  Files          588      588              
  Lines        30412    30472      +60     
===========================================
+ Hits         28180    28242      +62     
+ Misses        2232     2230       -2     

Files with missing lines	Coverage Δ
src/include/migraphx/split_single_dyn_dim.hpp	100.00% <ø> (ø)
src/split_single_dyn_dim.cpp	98.65% <100.00%> (+0.19%)	⬆️
src/targets/ref/target.cpp	100.00% <ø> (ø)
src/include/migraphx/op/select_module.hpp	95.15% <95.38%> (+8.61%)	⬆️

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

[pfultz2]

MIGRAPHX_DYN_DIM_FREEZE_TO= adds a new freeze_dyn_dim pass that runs before split_single_dyn_dim and rewrites every non-fixed dynamic parameter to a static shape at size N. Downstream sees a fully-static program -- no submodules, no select_module, no per-inference dispatch. Matches hand-written-static inference latency.

Is this padding the dimension to the frozen dim? What happens when the dim is larger than the static size?

[pfultz2]

Dont suppress this cppcheck warning.

[pfultz2]

Dont use ::value.

[pfultz2]

This can already be achieved by setting the dimensions to a static size when parsing onnx. This pass should be removed.

[CharlieL7]

Agreed, that method is also more customizable such that multiple different static dimensions can be set for different dynamic dimensions. Also remove the env variable.

[pfultz2]

Delete all this. Dont unit tests with env variables(or duplicate functionality we already have). These should be parameters to the pass and then passed in when running the tests: run_pass({.dyn_dim_bucket_by_optimals=true})

Suggested change

	// Cross-platform env helpers (Windows has _putenv_s instead of setenv).
	namespace {
	inline int set_env(const char* name, const char* value)
	{
	#ifdef _WIN32
	return _putenv_s(name, value);
	#else
	return ::setenv(name, value, /*overwrite=*/1);
	#endif
	}
	inline int unset_env(const char* name)
	{
	#ifdef _WIN32
	return _putenv_s(name, "");
	#else
	return ::unsetenv(name);
	#endif
	}
	} // namespace
	namespace {
	// RAII guard that turns MIGRAPHX_DYN_DIM_BUCKET_BY_OPTIMALS on for the
	// lifetime of the guard and restores the prior value on destruction. Lets a
	// single test binary mix bucket-mode and default-mode test cases without
	// polluting the environment across tests.
	struct bucket_env_guard
	{
	static constexpr const char* name = "MIGRAPHX_DYN_DIM_BUCKET_BY_OPTIMALS";
	std::string prev;
	bool had_prev = false;
	explicit bucket_env_guard(const char* value)
	{
	if(auto* p = std::getenv(name))
	{
	prev = p;
	had_prev = true;
	}
	set_env(name, value);
	}
	~bucket_env_guard()
	{
	if(had_prev)
	set_env(name, prev.c_str());
	else
	unset_env(name);
	}
	};
	} // namespace

[CharlieL7]

Please make the docstrings more terse and/or use some type of bullet point format. There are way too many comment paragraphs to read.

[CharlieL7]

You will need to update this file with respect to the GPU pipeline changes rather than just overwriting back to the previous.