perf: defer eager type/streaming imports at import openai - #3843
perf: defer eager type/streaming imports at import openai#3843harsheet-shah wants to merge 1 commit into
import openai#3843Conversation
`import openai` eagerly imported the entire Assistants `beta` type tree and the streaming helpers, building hundreds of pydantic models that most callers never use. This dominates cold-start / first-import latency in serverless and hosted-agent environments. Make the top-level `openai` package expose `types`, `pydantic_function_tool`, `AssistantEventHandler`, `AsyncAssistantEventHandler` and the `ReconnectingEvent`/`ReconnectingOverrides` websocket types lazily via a module-level `__getattr__` (PEP 562) plus a `types` LazyProxy, and defer `openai.lib` helper imports. Public attribute access, `from openai import *`, and static type checking (via `TYPE_CHECKING` re-exports) are unchanged. Measured (openai 3.11.0, warm best-of-3): `import openai` drops from ~1,944 ms to ~1,101 ms (~43%), and eagerly-loaded `openai.*` submodules drop from 835 to 497 -- the Assistants `beta` subtree (309 modules) is no longer built at import. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 93f793d2-1b86-4677-908c-3722d4fef290
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: bd68d7aad5
ℹ️ About Codex in GitHub
Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".
| from .types.websocket_reconnection import ReconnectingEvent, ReconnectingOverrides | ||
| else: | ||
| from ._utils._resources_proxy import resources as resources | ||
| from ._utils._types_proxy import types as types |
There was a problem hiding this comment.
Do not replace the already-cached types module
During every import openai, _exceptions.py:10 has already imported openai.types.shared.oauth_error_code, which executes and caches the real openai.types package. This assignment therefore replaces the parent attribute after the module is cached rather than deferring its import; subsequent import openai.types as types returns the TypesProxy, and operations requiring a genuine module, such as importlib.reload(types), fail because the proxy is not the object in sys.modules. Keep the cached module here, or remove the earlier package import before installing a proxy, so the exported module API is not changed without any import-time benefit.
AGENTS.md reference: AGENTS.md:L5-L8
Useful? React with 👍 / 👎.
eavanvalkenburg
left a comment
There was a problem hiding this comment.
This would definitely improve import performance across the board, and complements a fix I made in our package: microsoft/agent-framework#8219
Summary
import openaieagerly imports the entire Assistantsbetatype tree and the streaming helpers, building hundreds of pydantic models that most callers never use. This dominates cold-start / first-import latency in serverless and hosted-agent environments (e.g. containers that boot, import the SDK, then serve a single Responses/Chat request).This PR makes the top-level
openaipackage expose the rarely-needed surfaces lazily while keeping the public API and static typing identical:openai.typesis now aLazyProxy(mirrors the existingopenai.resourcesproxy) instead offrom . import types.pydantic_function_tool,AssistantEventHandler,AsyncAssistantEventHandler, and theReconnectingEvent/ReconnectingOverrideswebsocket types resolve through a module-level__getattr__(PEP 562).openai.libdefers_tools/_parsingthe same way.TYPE_CHECKINGre-exports, so IDEs / pyright are unaffected, andfrom openai import *is unchanged.Azure / Bedrock module-client support and every existing public object are untouched.
Measurements
openai
3.11.0, warm best-of-3,import openaion top of an already-importedpydantic+httpx2:import openaiopenai.*submodulesmain(eager)~43% faster first import; the Assistants
betasubtree (309 modules) is no longer built at import time. It loads transparently on first access (openai.beta,openai.types.beta, etc.).Tests
Adds
tests/test_lazy_imports.py:import openaino longer eagerly loadsopenai.types.betaoropenai.lib.streaming.typesproxy loads the real module on first attribute access.Existing
tests/test_module_client.pyandtests/lib/test_old_api.pycontinue to pass, and a full import smoke over everyopenai.types.*subtree +from openai import *verifies no exported name regressed.