Crypto support : App supply I/O callbacks to EP + callback and fallback helpers#28624
Merged
Conversation
GopalakrishnanN
force-pushed
the
gokrishnan/EpContextHelperUtilities
branch
from
97cfedb to
ffbbb4f
Compare
![github-actions[bot]](https://avatars.githubusercontent.com/in/15368?s=60&v=4){kind=small}
Contributor
There was a problem hiding this comment.
Pull request overview
This PR adds sample-only EPContext data I/O helper utilities (callback-or-file fallback) and wires them through the EP plugin APIs so example/test EPs can read/write EPContext payloads without forcing policy into ORT core/public ABI.
Changes:
- Add new EPContext read/write callback types and APIs (C, C++ wrappers, EP API extraction via
OrtEpContextConfig). - Add sample
ep_context_data_utilshelpers for path handling and callback/file fallback read/write. - Update AutoEP example plugin EP + tests to use the helpers for external EPContext read/write paths.
Reviewed changes
Copilot reviewed 23 out of 23 changed files in this pull request and generated 6 comments.
Show a summary per file
| File | Description |
|---|---|
| onnxruntime/test/framework/ep_plugin_provider_test.cc | Adds unit tests for new EPContext callback APIs and C++ wrappers. |
| onnxruntime/test/autoep/test_execution.cc | Adds AutoEP tests covering callback/file fallback behavior for EPContext data. |
| onnxruntime/test/autoep/library/example_plugin_ep/ep.h | Extends example EP config and plumbs Ort::EpContextConfig into the EP instance. |
| onnxruntime/test/autoep/library/example_plugin_ep/ep.cc | Uses the new helper to read/write EPContext external data with callback/file fallback. |
| onnxruntime/test/autoep/library/example_plugin_ep/ep_factory.cc | Extracts/stores EPContext callbacks from session options and reads embed/path config. |
| onnxruntime/test/autoep/library/ep_context_data_utils.h | Adds sample-only helper utilities for path conversion + callback/file fallback I/O. |
| onnxruntime/core/session/plugin_ep/ep_api.h | Declares new EP plugin API entrypoints for EPContext config + callbacks. |
| onnxruntime/core/session/plugin_ep/ep_api.cc | Implements EPContext config handle + accessor functions and appends to OrtEpApi. |
| onnxruntime/core/session/ort_apis.h | Declares new public C API entrypoint for EPContext read callback registration. |
| onnxruntime/core/session/onnxruntime_c_api.cc | Exposes SessionOptions_SetEpContextDataReadFunc in the versioned OrtApi table. |
| onnxruntime/core/session/model_compilation_options.h | Adds ModelCompilationOptions::SetEpContextDataWriteFunc API. |
| onnxruntime/core/session/model_compilation_options.cc | Stores EPContext write callback into model-gen options. |
| onnxruntime/core/session/compile_api.h | Declares new compile API function to set EPContext write callback. |
| onnxruntime/core/session/compile_api.cc | Implements compile API function and appends to OrtCompileApi. |
| onnxruntime/core/session/abi_session_options.cc | Implements SessionOptions_SetEpContextDataReadFunc. |
| onnxruntime/core/framework/session_options.h | Adds fields to store EPContext read callback + state in SessionOptions. |
| onnxruntime/core/framework/ep_context_options.h | Adds holder for EPContext write callback in model-gen options. |
| onnxruntime/core/framework/ep_context_options.cc | Adds accessor for EPContext write callback holder. |
| include/onnxruntime/core/session/onnxruntime_ep_c_api.h | Adds EP ABI surface for OrtEpContextConfig and callback accessors. |
| include/onnxruntime/core/session/onnxruntime_cxx_inline.h | Adds inline C++ wrappers for EpContextConfig + new setter wrappers. |
| include/onnxruntime/core/session/onnxruntime_cxx_api.h | Adds owning C++ wrapper type Ort::EpContextConfig. |
| include/onnxruntime/core/session/onnxruntime_c_api.h | Adds callback typedefs + new OrtApi/OrtCompileApi entrypoints. |
| cmake/onnxruntime_unittests.cmake | Adds the new helper header to the AutoEP example plugin build sources. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
ericcraw
reviewed
Jun 2, 2026
ericcraw
left a comment
ericcraw
left a comment
Contributor
There was a problem hiding this comment.
The ep apis look good to me.
MayureshV1
approved these changes
Jun 2, 2026
MayureshV1
left a comment
MayureshV1
left a comment
Contributor
There was a problem hiding this comment.
EP interfaces look good !
GopalakrishnanN
force-pushed
the
gokrishnan/EpContextHelperUtilities
branch
from
ada27a9 to
efa190d
Compare
edgchen1
reviewed
Jun 3, 2026
GopalakrishnanN
changed the title
GopalakrishnanN
changed the title
GopalakrishnanN
force-pushed
the
gokrishnan/EpContextHelperUtilities
branch
from
18124ff to
c0bb9f2
Compare
- Fix doc to reference OrtCompileApi (not OrtApi) for ModelCompilationOptions_SetEpContextBinaryInformation. - EpContextConfig: drop the default and raw OrtSessionOptions* constructors; keep only the SessionOptions/ConstSessionOptions overloads (wrap at the ep_factory call site). - example EP: comment that the load-side EPContext read during compile is intentionally exercised then discarded. - ep_context_data_utils: low-level *WithFileFallback overloads now take the callback + state directly; high-level overloads extract them from OrtEpContextConfig. - Allow a NULL EPContext write callback to clear a previously set one (symmetric with the read setter); update doc/SAL and tests. - Move the public EPContext write-callback tests out of the framework provider test into test/shared_lib/test_ep_context_data_api.cc.
edgchen1
previously approved these changes
Jun 25, 2026
edgchen1
reviewed
Jun 25, 2026
Replaces the manual Ort::Exception try/catch in ExampleEpFactory::CreateEpImpl with the EXCEPTION_TO_RETURNED_STATUS_BEGIN/END macros (consistent with the example EP kernels), which also map std::exception and unknown exceptions to a returned OrtStatus to match the noexcept contract.
added 2 commits
June 25, 2026 16:30
…te a directory ValidateEpContextDataName and the model-derived (untrusted) branch of ResolveEpContextDataPath now reject a name whose final component is empty (a trailing separator) or is a dot, which would otherwise resolve to a directory and only surface later as a confusing file I/O failure. Adds an IsDirectoryOrEmptyName helper and covering tests.
Reword the OrtEpContextConfig definition comment (it is not opaque at the definition site); make ReadEpContextDataWithFileFallback exception-free by using the C allocator API (GetAllocatorWithDefaultOptions / AllocatorFree) instead of Ort::AllocatorWithDefaultOptions; alphabetize includes in ep_plugin_provider_test.cc; and use the Ort::Experimental::EpContextConfig RAII wrapper in the read-back tests in test_ep_context_data_api.cc.
edgchen1
approved these changes
Jun 26, 2026
GopalakrishnanN
pushed a commit
that referenced
this pull request
Jun 27, 2026
…h resolution Addresses #28624 review feedback. Now that IsResolvedPathWithinBase() does real model-directory containment for untrusted model-relative names, the lexical ContainsPathTraversal() guard is no longer applied on the trusted graph==nullptr branch of ResolveEpContextDataPath(): trusted callers already may pass absolute paths and own their paths, so there is no model directory to contain against. ContainsPathTraversal() is kept solely for ValidateEpContextDataName(), which validates the logical callback-namespace name written into the model ep_cache_context attribute (never resolved against a filesystem base). Updates the two trusted-branch tests; logical-name and model-directory containment coverage is unchanged.
GopalakrishnanN
pushed a commit
that referenced
this pull request
Jun 27, 2026
…h resolution Addresses #28624 review feedback. Now that IsResolvedPathWithinBase() does real model-directory containment for untrusted model-relative names, the lexical ContainsPathTraversal() guard is no longer applied on the trusted graph==nullptr branch of ResolveEpContextDataPath(): trusted callers already may pass absolute paths and own their paths, so there is no model directory to contain against. ContainsPathTraversal() is kept solely for ValidateEpContextDataName(), which validates the logical callback-namespace name written into the model ep_cache_context attribute (never resolved against a filesystem base). Updates the two trusted-branch tests; logical-name and model-directory containment coverage is unchanged.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
This PR adds an opt-in mechanism that lets an application supply its own I/O callbacks for an execution provider's EPContext binary data, so the data can live somewhere other than a plain file on disk (for example, an encrypted store or an in-memory buffer). It introduces the callback APIs end-to-end and demonstrates their use with a sample helper in the AutoEP example plugin EP.
When an EP compiles a model into an EPContext model, it may emit the compiled blob either embedded in the ONNX model or as a separate external payload. For the external case, ORT previously assumed the payload is a file. These callbacks let the application own that read/write instead, while ORT core stays policy-neutral and never imposes a storage format.
What this PR adds
OrtWriteNamedBufferFunc) + setterOrtCompileApi::ModelCompilationOptions_SetEpContextDataWriteFunc. Set onOrtModelCompilationOptions, because writing EPContext binary data happens only during compilation. Passing a NULL callback clears a previously set one.OrtReadNamedBufferFunc) + setterOrtApi::SessionOptions_SetEpContextDataReadFunc. Set onOrtSessionOptions, because reading external EPContext binary data happens during session load / inference. Passing a NULL callback clears a previously set one.OrtEpContextConfig. Both callbacks are surfaced to execution providers through a single unified handle,OrtEpContextConfig, obtained viaOrtEpApi::SessionOptions_GetEpContextConfig(gettersEpContextConfig_GetEpContextDataReadFunc/EpContextConfig_GetEpContextDataWriteFunc, released withReleaseEpContextConfig). This keeps the application-facing setters scoped to the correct lifecycle while giving EPs one consistent place to retrieve both callbacks. Each setter's doc comment cross-references the other so the split is discoverable.include/onnxruntime/core/session/onnxruntime_experimental_c_api.inc), so they are reached via the generatedOrt::Experimental::Get_<name>_SinceV28_Fn(...)/...FnOrThrow(...)accessors rather than fixedOrtApislots. A move-only RAII wrapper,Ort::Experimental::EpContextConfig(inonnxruntime_experimental_cxx_api.h), owns anOrtEpContextConfigand exposesGetReadFunc()/GetWriteFunc(); it can be constructed directly from a C++SessionOptions/ConstSessionOptions.onnxruntime/test/autoep/library/ep_context_data_utils.h) implementing callback-or-file fallback behavior: if a callback is supplied it is used, otherwise the helper falls back to direct file I/O. The AutoEP example plugin EP uses this helper for its external EPContext read/write paths. Because the names read on the load side originate from the untrusted EPContext model (ep_cache_contextattribute), the helper validates them: it rejects absolute/rooted paths,..traversal, and directory-like names (.or a trailing separator), and confines model-relative names to the model directory (resolving./..and symlinks viastd::filesystem::weakly_canonical). It reports all failures viaOrtStatus*(no exceptions) and lives outside the public C API / EP ABI, so it is purely illustrative and imposes no policy on ORT core; its doc comments note that production EPs should still apply their own sandboxing and payload size limits.The callback typedef names (
OrtReadNamedBufferFunc/OrtWriteNamedBufferFunc) are intentionally generic. They are currently used for EPContext binary data, but the contract is deliberately storage-agnostic so future APIs can reuse the same callback shape for other named data payloads.Note on the Android workflow change
.github/workflows/android.ymlbumps the minimal-build binary-size threshold (1436672->1438720bytes) to accommodate the small size increase from compiling the new experimental API into the Android minimal build.Testing
python tools/ci_build/build.py --config RelWithDebInfo --build --parallel --test --build_dir build\Windows.onnxruntime_shared_lib_test.exe --gtest_filter=EpContextDataApiTest.*-> 9 passed.onnxruntime_autoep_test.exe --gtest_filter=*EpContext*-> 17 passed, 1 skipped (EpContextDataUtils_ResolvePathRejectsSymlinkEscaperequires the Windows "create symbolic link" privilege).clang-formatclean on touched C++ files;git diff --check: clean.Test layout: public EPContext API tests in
onnxruntime/test/shared_lib/test_ep_context_data_api.cc; sample-helper unit tests inonnxruntime/test/autoep/ep_context_data_utils_test.cc; compile/load end-to-end tests inonnxruntime/test/autoep/test_execution.cc.