Skip to content

perf: defer eager type/streaming imports at import openai - #3843

Open
harsheet-shah wants to merge 1 commit into
openai:mainfrom
harsheet-shah:harsheet/lazy-top-level-imports
Open

perf: defer eager type/streaming imports at import openai#3843
harsheet-shah wants to merge 1 commit into
openai:mainfrom
harsheet-shah:harsheet/lazy-top-level-imports

Conversation

@harsheet-shah

Copy link
Copy Markdown

Summary

import openai eagerly imports the entire Assistants beta type tree and the streaming helpers, building hundreds of pydantic models that most callers never use. This dominates cold-start / first-import latency in serverless and hosted-agent environments (e.g. containers that boot, import the SDK, then serve a single Responses/Chat request).

This PR makes the top-level openai package expose the rarely-needed surfaces lazily while keeping the public API and static typing identical:

  • openai.types is now a LazyProxy (mirrors the existing openai.resources proxy) instead of from . import types.
  • pydantic_function_tool, AssistantEventHandler, AsyncAssistantEventHandler, and the ReconnectingEvent / ReconnectingOverrides websocket types resolve through a module-level __getattr__ (PEP 562).
  • openai.lib defers _tools / _parsing the same way.
  • All names remain statically importable via TYPE_CHECKING re-exports, so IDEs / pyright are unaffected, and from openai import * is unchanged.

Azure / Bedrock module-client support and every existing public object are untouched.

Measurements

openai 3.11.0, warm best-of-3, import openai on top of an already-imported pydantic + httpx2:

import openai eager openai.* submodules
main (eager) ~1,944 ms 835
this PR (lazy) ~1,101 ms 497

~43% faster first import; the Assistants beta subtree (309 modules) is no longer built at import time. It loads transparently on first access (openai.beta, openai.types.beta, etc.).

Tests

Adds tests/test_lazy_imports.py:

  • import openai no longer eagerly loads openai.types.beta or openai.lib.streaming.
  • The lazy top-level exports resolve to the same objects as their canonical modules.
  • The types proxy loads the real module on first attribute access.

Existing tests/test_module_client.py and tests/lib/test_old_api.py continue to pass, and a full import smoke over every openai.types.* subtree + from openai import * verifies no exported name regressed.

`import openai` eagerly imported the entire Assistants `beta` type tree and
the streaming helpers, building hundreds of pydantic models that most callers
never use. This dominates cold-start / first-import latency in serverless and
hosted-agent environments.

Make the top-level `openai` package expose `types`, `pydantic_function_tool`,
`AssistantEventHandler`, `AsyncAssistantEventHandler` and the
`ReconnectingEvent`/`ReconnectingOverrides` websocket types lazily via a
module-level `__getattr__` (PEP 562) plus a `types` LazyProxy, and defer
`openai.lib` helper imports. Public attribute access, `from openai import *`,
and static type checking (via `TYPE_CHECKING` re-exports) are unchanged.

Measured (openai 3.11.0, warm best-of-3): `import openai` drops from ~1,944 ms
to ~1,101 ms (~43%), and eagerly-loaded `openai.*` submodules drop from 835 to
497 -- the Assistants `beta` subtree (309 modules) is no longer built at import.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot-Session: 93f793d2-1b86-4677-908c-3722d4fef290
@harsheet-shah
harsheet-shah requested a review from a team as a code owner September 10, 2026 09:48

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: bd68d7aad5

ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".

Comment thread src/openai/__init__.py
from .types.websocket_reconnection import ReconnectingEvent, ReconnectingOverrides
else:
from ._utils._resources_proxy import resources as resources
from ._utils._types_proxy import types as types

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Do not replace the already-cached types module

During every import openai, _exceptions.py:10 has already imported openai.types.shared.oauth_error_code, which executes and caches the real openai.types package. This assignment therefore replaces the parent attribute after the module is cached rather than deferring its import; subsequent import openai.types as types returns the TypesProxy, and operations requiring a genuine module, such as importlib.reload(types), fail because the proxy is not the object in sys.modules. Keep the cached module here, or remove the earlier package import before installing a proxy, so the exported module API is not changed without any import-time benefit.

AGENTS.md reference: AGENTS.md:L5-L8

Useful? React with 👍 / 👎.

@eavanvalkenburg eavanvalkenburg left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This would definitely improve import performance across the board, and complements a fix I made in our package: microsoft/agent-framework#8219

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants