Conversation
|
Important Review skippedDraft detected. Please check the settings in the CodeRabbit UI or the You can disable this status message by setting the 📝 WalkthroughWalkthroughThis PR introduces a new dynamic model generation feature that enables runtime creation of Python model classes from schemas using Pydantic v2. It includes a public API function, type resolver, constraint handler, and comprehensive model creator infrastructure. Changes
Sequence DiagramsequenceDiagram
participant User
participant generate_dynamic_models as generate_dynamic_models()
participant Parser
participant DynamicModelCreator
participant TypeResolver
participant constraints_to_field_kwargs
participant Pydantic as Pydantic.create_model()
User->>generate_dynamic_models: input, input_file_type
generate_dynamic_models->>Parser: instantiate & parse
Parser-->>generate_dynamic_models: parsed results
generate_dynamic_models->>DynamicModelCreator: init(parser)
generate_dynamic_models->>DynamicModelCreator: create_models()
rect rgb(200, 220, 255)
Note over DynamicModelCreator: Model Creation Phase
loop for each DataModel
DynamicModelCreator->>DynamicModelCreator: sort & categorize
alt is enum
DynamicModelCreator->>Pydantic: Enum.__init__
else is datamodel
DynamicModelCreator->>TypeResolver: resolve_with_constraints(field.type)
TypeResolver-->>DynamicModelCreator: (resolved_type, constraints)
DynamicModelCreator->>constraints_to_field_kwargs: constraints_to_field_kwargs(constraints)
constraints_to_field_kwargs-->>DynamicModelCreator: field_kwargs
DynamicModelCreator->>Pydantic: create_model(class_name, **fields)
Pydantic-->>DynamicModelCreator: model_class
end
end
end
rect rgb(200, 240, 200)
Note over DynamicModelCreator: Rebuild Phase (Forward References)
DynamicModelCreator->>DynamicModelCreator: _rebuild_models()
loop for each model
DynamicModelCreator->>Pydantic: model_rebuild()
end
end
DynamicModelCreator-->>generate_dynamic_models: {class_name: model_type}
generate_dynamic_models-->>User: dynamic_models dict
Estimated Code Review Effort🎯 4 (Complex) | ⏱️ ~45 minutes Possibly Related PRs
Suggested Labels
Poem
Pre-merge checks and finishing touches❌ Failed checks (1 warning)
✅ Passed checks (2 passed)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
|
📚 Docs Preview: https://pr-2898.datamodel-code-generator.pages.dev |
![github-advanced-security[bot]](https://avatars.githubusercontent.com/in/57789?s=60&v=4){kind=small}
Codecov Report❌ Patch coverage is Additional details and impacted files@@ Coverage Diff @@
## main #2898 +/- ##
==========================================
- Coverage 99.37% 99.16% -0.22%
==========================================
Files 92 98 +6
Lines 16192 16667 +475
Branches 1915 1978 +63
==========================================
+ Hits 16091 16528 +437
- Misses 52 69 +17
- Partials 49 70 +21
Flags with carried forward coverage won't be shown. Click here to find out more. ☔ View full report in Codecov by Sentry. 🚀 New features to boost your workflow:
|
CodSpeed Performance ReportMerging #2898 will not alter performanceComparing
|
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 0
🧹 Nitpick comments (22)
src/datamodel_code_generator/dynamic/constraints.py (2)
83-89: Potential constraint overwrite when bothgtandexclusive_minimumare present.If a constraint object has both
gtandexclusive_minimumset (orltandexclusive_maximum), the exclusive bounds will silently overwrite the already-setgt/ltvalues. While JSON Schema semantics make this unlikely in practice, consider adding a check or using the stricter bound.🔎 Proposed defensive handling
exclusive_minimum = getattr(constraints, "exclusive_minimum", None) if exclusive_minimum is not None: - kwargs["gt"] = exclusive_minimum + if "gt" not in kwargs: + kwargs["gt"] = exclusive_minimum + else: + # Use stricter bound + kwargs["gt"] = max(kwargs["gt"], exclusive_minimum) exclusive_maximum = getattr(constraints, "exclusive_maximum", None) if exclusive_maximum is not None: - kwargs["lt"] = exclusive_maximum + if "lt" not in kwargs: + kwargs["lt"] = exclusive_maximum + else: + # Use stricter bound + kwargs["lt"] = min(kwargs["lt"], exclusive_maximum)
11-11: Consider removing the unusednoqadirective.Static analysis indicates the
noqa: PLR0912, PLR0914, PLR0915directive is unused since these rules are not enabled. If the project's linter configuration doesn't enable these rules, the directive can be safely removed.🔎 Proposed fix
-def constraints_to_field_kwargs( # noqa: PLR0912, PLR0914, PLR0915 +def constraints_to_field_kwargs( constraints: ConstraintsBase | None, ) -> dict[str, Any]:src/datamodel_code_generator/parser/base.py (2)
3136-3170: Consider adding a runtime check for Pydantic v2.The docstring explicitly states "Only Pydantic v2 models are supported," but there's no runtime enforcement. Users on Pydantic v1 will get unclear errors when attempting to use this feature.
🔎 Proposed Pydantic v2 check
def create_dynamic_models(self) -> dict[str, type]: """Create actual Python model classes from parsed schema. ... """ from datamodel_code_generator.dynamic.creator import DynamicModelCreator # noqa: PLC0415 + from datamodel_code_generator.util import is_pydantic_v2 # noqa: PLC0415 + + if not is_pydantic_v2(): + msg = "Dynamic model generation requires Pydantic v2" + raise NotImplementedError(msg) if not self.results: return {} creator = DynamicModelCreator(self) return creator.create_models()
3164-3164: Consider removing the unusednoqadirective.Static analysis indicates the
noqa: PLC0415directive is unused. IfPLC0415(import not at top of file) is not enabled in your linter configuration, the directive can be removed. However, if you plan to enable this rule in the future, keeping it is fine.tests/dynamic/test_creator.py (1)
1-1: Consider removing the unusednoqadirective if D101/D102 rules are not enabled.Static analysis indicates this directive may be unused. If docstring rules for classes (D101) and methods (D102) are not enabled in your linter configuration, this can be removed.
src/datamodel_code_generator/__init__.py (3)
959-959: Consider specifying encoding when reading file content.The
source.read_text()call doesn't specify an encoding, which will use the system default. For consistency with the rest of the codebase (which usesconfig.encoding), consider specifyingencoding="utf-8"or making it configurable.🔎 Proposed fix
elif isinstance(source, Path): - input_file_type = infer_input_type(source.read_text()) + input_file_type = infer_input_type(source.read_text(encoding="utf-8"))
899-975: Consider adding a runtime check for Pydantic v2.The docstring states "Only Pydantic v2 models are supported," but this isn't enforced at runtime. Users on Pydantic v1 will encounter unclear errors.
🔎 Proposed check at function entry
def generate_dynamic_models( input_: Path | str | ParseResult | Mapping[str, Any], *, input_file_type: InputFileType = InputFileType.Auto, ) -> dict[str, type]: """Generate actual Python model classes from schema at runtime. ... """ + if not is_pydantic_v2(): + msg = "Dynamic model generation requires Pydantic v2" + raise NotImplementedError(msg) + from datamodel_code_generator.parser.jsonschema import JsonSchemaParser # noqa: PLC0415
938-939: Consider removing unusednoqadirectives.Static analysis indicates the
noqa: PLC0415directives on lines 938-939 are unused. If thePLC0415rule is not enabled, these can be removed.🔎 Proposed fix
- from datamodel_code_generator.parser.jsonschema import JsonSchemaParser # noqa: PLC0415 - from datamodel_code_generator.parser.openapi import OpenAPIParser # noqa: PLC0415 + from datamodel_code_generator.parser.jsonschema import JsonSchemaParser + from datamodel_code_generator.parser.openapi import OpenAPIParsersrc/datamodel_code_generator/dynamic/type_resolver.py (5)
77-77: Remove unusednoqadirective.Static analysis indicates
PLR0911is not enabled in the project's linting configuration.🔎 Proposed fix
- def resolve_with_constraints( # noqa: PLR0911 + def resolve_with_constraints( self, data_type: DataType ) -> tuple[Any, dict[str, Any]]:
118-118: Remove unusednoqadirective.Static analysis indicates
UP007is not enabled.🔎 Proposed fix
- return Union[inner_types], constraints # type: ignore[valid-type] # noqa: UP007 + return Union[inner_types], constraints # type: ignore[valid-type]
149-149: Remove unusednoqadirective.Static analysis indicates
PLR6301is not enabled.🔎 Proposed fix
- def _extract_constraints(self, data_type: DataType, constraints: dict[str, Any]) -> None: # noqa: PLR6301 + def _extract_constraints(self, data_type: DataType, constraints: dict[str, Any]) -> None:
154-158: Consider handling double-quoted raw string patterns.The normalization only handles single-quoted raw strings (
r'...'). If DataType.kwargs can contain double-quoted raw strings (r"..."), those would not be normalized.🔎 Proposed fix to handle both quote styles
if key == "regex": pattern_value = kwarg_value if isinstance(pattern_value, str) and pattern_value.startswith("r'") and pattern_value.endswith("'"): pattern_value = pattern_value[2:-1] + elif isinstance(pattern_value, str) and pattern_value.startswith('r"') and pattern_value.endswith('"'): + pattern_value = pattern_value[2:-1] constraints["pattern"] = pattern_value
169-169: Remove unusednoqadirective.Static analysis indicates
UP007is not enabled.🔎 Proposed fix
- return Union[inner_types] # type: ignore[valid-type] # noqa: UP007 + return Union[inner_types] # type: ignore[valid-type]src/datamodel_code_generator/dynamic/creator.py (9)
52-53: Remove unusednoqadirectives.Static analysis indicates
PLC0415is not enabled. The local imports are appropriate for avoiding circular dependencies.🔎 Proposed fix
- from datamodel_code_generator.model.enum import Enum as EnumModel # noqa: PLC0415 - from datamodel_code_generator.parser.base import sort_data_models # noqa: PLC0415 + from datamodel_code_generator.model.enum import Enum as EnumModel + from datamodel_code_generator.parser.base import sort_data_models
90-90: Remove unusednoqadirective.🔎 Proposed fix
- from pydantic import BaseModel as BaseModelCls # noqa: PLC0415 + from pydantic import BaseModel as BaseModelCls
132-132: Remove unusednoqadirective.🔎 Proposed fix
- def _create_field_info( # noqa: PLR6301 + def _create_field_info( self, field: DataModelFieldBase, type_constraints: dict[str, Any] | None = None ) -> FieldInfo:
161-161: Remove unusednoqadirective.🔎 Proposed fix
- from pydantic import BaseModel # noqa: PLC0415 + from pydantic import BaseModel
187-190: Consider logging when falling back to BaseModel for unresolved bases.When a base class cannot be resolved from
_short_name_lookup, the code silently falls back toBaseModel. Consider adding debug logging here to aid troubleshooting inheritance issues.🔎 Proposed fix
if type_name in self._short_name_lookup: bases.append(self._short_name_lookup[type_name]) else: + logger.debug("Base class '%s' not found, falling back to BaseModel", type_name) bases.append(BaseModel) else: + logger.debug("Base class has no reference or type, falling back to BaseModel") bases.append(BaseModel)
194-194: Remove unusednoqadirective.🔎 Proposed fix
- def _build_model_config(self, data_model: DataModel) -> ConfigDict | None: # noqa: PLR6301 + def _build_model_config(self, data_model: DataModel) -> ConfigDict | None:
219-219: Remove unusednoqadirective.🔎 Proposed fix
- def _get_module_name(self, data_model: DataModel) -> str: # noqa: PLR6301 + def _get_module_name(self, data_model: DataModel) -> str:
249-249: Remove unusednoqadirective.🔎 Proposed fix
- from pydantic import BaseModel # noqa: PLC0415 + from pydantic import BaseModel
257-258: Consider adding debug logging for unresolved annotations.Silently passing on
PydanticUndefinedAnnotationcould hide issues with forward references that genuinely cannot be resolved. Consider adding debug logging to aid troubleshooting.🔎 Proposed fix
try: model.model_rebuild(_types_namespace=namespace) - except PydanticUndefinedAnnotation: - pass + except PydanticUndefinedAnnotation as e: + logger.debug("Model %s has unresolved annotation: %s", model_key, e)
📜 Review details
Configuration used: Path: .coderabbit.yaml
Review profile: CHILL
Plan: Pro
📒 Files selected for processing (9)
src/datamodel_code_generator/__init__.pysrc/datamodel_code_generator/dynamic/__init__.pysrc/datamodel_code_generator/dynamic/constraints.pysrc/datamodel_code_generator/dynamic/creator.pysrc/datamodel_code_generator/dynamic/exceptions.pysrc/datamodel_code_generator/dynamic/type_resolver.pysrc/datamodel_code_generator/parser/base.pytests/dynamic/__init__.pytests/dynamic/test_creator.py
🧰 Additional context used
🧬 Code graph analysis (7)
src/datamodel_code_generator/parser/base.py (1)
src/datamodel_code_generator/dynamic/creator.py (2)
DynamicModelCreator(32-263)create_models(42-68)
src/datamodel_code_generator/dynamic/exceptions.py (1)
src/datamodel_code_generator/types.py (1)
DataType(400-941)
src/datamodel_code_generator/__init__.py (2)
src/datamodel_code_generator/enums.py (1)
InputFileType(35-45)src/datamodel_code_generator/parser/jsonschema.py (2)
JsonSchemaParser(552-3765)parse_raw(3637-3668)
src/datamodel_code_generator/dynamic/__init__.py (1)
src/datamodel_code_generator/dynamic/exceptions.py (5)
CircularReferenceError(26-32)ConstraintConversionError(35-43)DynamicModelError(11-12)TypeResolutionError(15-23)UnsupportedModelTypeError(46-52)
tests/dynamic/test_creator.py (3)
src/datamodel_code_generator/enums.py (1)
InputFileType(35-45)src/datamodel_code_generator/__init__.py (1)
generate_dynamic_models(899-975)src/datamodel_code_generator/model/base.py (1)
name(827-829)
src/datamodel_code_generator/dynamic/constraints.py (1)
src/datamodel_code_generator/model/base.py (1)
ConstraintsBase(101-148)
src/datamodel_code_generator/dynamic/type_resolver.py (2)
src/datamodel_code_generator/types.py (1)
DataType(400-941)src/datamodel_code_generator/reference.py (2)
reference(77-79)short_name(187-189)
🪛 Ruff (0.14.10)
src/datamodel_code_generator/parser/base.py
3164-3164: Unused noqa directive (non-enabled: PLC0415)
Remove unused noqa directive
(RUF100)
src/datamodel_code_generator/__init__.py
938-938: Unused noqa directive (non-enabled: PLC0415)
Remove unused noqa directive
(RUF100)
939-939: Unused noqa directive (non-enabled: PLC0415)
Remove unused noqa directive
(RUF100)
tests/dynamic/test_creator.py
1-1: Unused noqa directive (non-enabled: D101, D102)
Remove unused noqa directive
(RUF100)
src/datamodel_code_generator/dynamic/constraints.py
11-11: Unused noqa directive (non-enabled: PLR0912, PLR0914, PLR0915)
Remove unused noqa directive
(RUF100)
src/datamodel_code_generator/dynamic/type_resolver.py
77-77: Unused noqa directive (non-enabled: PLR0911)
Remove unused noqa directive
(RUF100)
118-118: Unused noqa directive (non-enabled: UP007)
Remove unused noqa directive
(RUF100)
149-149: Unused noqa directive (non-enabled: PLR6301)
Remove unused noqa directive
(RUF100)
169-169: Unused noqa directive (non-enabled: UP007)
Remove unused noqa directive
(RUF100)
src/datamodel_code_generator/dynamic/creator.py
52-52: Unused noqa directive (non-enabled: PLC0415)
Remove unused noqa directive
(RUF100)
53-53: Unused noqa directive (non-enabled: PLC0415)
Remove unused noqa directive
(RUF100)
90-90: Unused noqa directive (non-enabled: PLC0415)
Remove unused noqa directive
(RUF100)
132-132: Unused noqa directive (non-enabled: PLR6301)
Remove unused noqa directive
(RUF100)
161-161: Unused noqa directive (non-enabled: PLC0415)
Remove unused noqa directive
(RUF100)
194-194: Unused noqa directive (non-enabled: PLR6301)
Remove unused noqa directive
(RUF100)
219-219: Unused noqa directive (non-enabled: PLR6301)
Remove unused noqa directive
(RUF100)
249-249: Unused noqa directive (non-enabled: PLC0415)
Remove unused noqa directive
(RUF100)
⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (13)
- GitHub Check: 3.10 on Ubuntu
- GitHub Check: 3.10 on Windows
- GitHub Check: py312-black22 on Ubuntu
- GitHub Check: py312-isort5 on Ubuntu
- GitHub Check: py312-isort6 on Ubuntu
- GitHub Check: benchmarks
- GitHub Check: py312-black24 on Ubuntu
- GitHub Check: 3.12 on Ubuntu
- GitHub Check: 3.13 on Windows
- GitHub Check: 3.11 on Windows
- GitHub Check: 3.12 on Windows
- GitHub Check: 3.14 on Windows
- GitHub Check: Analyze (python)
🔇 Additional comments (14)
tests/dynamic/__init__.py (1)
1-1: LGTM!Standard test package initializer with an appropriate docstring.
src/datamodel_code_generator/dynamic/exceptions.py (1)
1-52: LGTM!Well-structured exception hierarchy with informative error messages and proper storage of contextual information for debugging. The use of
TYPE_CHECKINGfor theDataTypeforward reference correctly avoids circular imports.tests/dynamic/test_creator.py (1)
1-16: LGTM! Comprehensive test coverage.The test suite provides excellent coverage of dynamic model generation features including constraints, nested models, circular references, various data types, and different input formats. The Pydantic v2 skip condition correctly gates the tests.
src/datamodel_code_generator/__init__.py (1)
1037-1037: LGTM!The
__all__export list is correctly updated to include the newgenerate_dynamic_modelsfunction.src/datamodel_code_generator/dynamic/__init__.py (1)
1-25: LGTM!Clean package initialization with appropriate re-exports. The
__all__list is well-organized and includes all the necessary public components for dynamic model generation.src/datamodel_code_generator/dynamic/type_resolver.py (5)
1-57: Well-structured type mapping constants.The three type mapping dictionaries provide clear separation of concerns for format-based types, primitive types, and constrained types. The coverage of standard formats (date-time, uuid variants, etc.) and primitives is comprehensive.
61-68: Clean class initialization with forward reference tracking.The TypeResolver is properly initialized with a models lookup dictionary and forward reference tracking, which integrates well with the dynamic model creation flow.
116-122: Verify ordering of union vs optional type resolution.The check for
len(data_type.data_types) > 1on line 116 runs before theis_optionalcheck on line 120. If a DataType is markedis_optional=Trueand also has multiple inner types (e.g.,Optional[Union[A, B]]), the union branch handles it first without wrapping in| None. Confirm this is the intended behavior, or whetheris_optionalshould be checked first.
126-147: Appropriate fallback chain for type resolution.The resolution priority (constrained types → format-based types → primitive types → model lookup → Any fallback) is well-ordered. Using
getattrfor theformatattribute safely handles cases where it may not be present.
171-174: LGTM!Clean property implementation exposing forward references for model rebuilding.
src/datamodel_code_generator/dynamic/creator.py (4)
12-15: Good compatibility handling for Pydantic versions.The fallback to
NameErrorwhenPydanticUndefinedAnnotationis unavailable provides graceful degradation for different Pydantic versions.
141-148: Default handling logic is correct.The condition on line 141 properly handles the case where
field.defaultis explicitlyNoneby first checkinghas_default. The fallback chain (has_default → default_factory → required → None) covers all cases appropriately.
226-245: LGTM!Clean enum creation using the functional API with proper quote stripping and consistent model registration.
104-105: Verify__config__usage with Pydantic v2'screate_model.In Pydantic v2,
create_modelmay expectmodel_configparameter or handle__config__differently. Verify this is the correct approach for the target Pydantic version.Pydantic v2 create_model __config__ parameter
- Remove line comments from generate_dynamic_models function - Remove unused exclusive_minimum/exclusive_maximum from CONSTRAINT_FIELD_MAP - Fix description access: use field.extras.get() instead of getattr()
- Move all inline schemas to tests/data/dynamic/ - Rewrite tests as functions instead of classes - Use inline-snapshot for assertions - Consolidate similar tests into grouped test functions
- Replace PRIMITIVE_TYPE_MAP with builtins lookup - Use TYPE_ALIASES for schema type names (string->str, etc.) - Rename CONSTRAINED_TYPE_MAP to CONSTRAINED_TYPE_BASE
| from typing import TYPE_CHECKING, Any, ForwardRef, Literal | ||
|
|
||
| if TYPE_CHECKING: | ||
| from datamodel_code_generator.types import DataType |
Check notice
Code scanning / CodeQL
Cyclic import Note
- Add handling for list, set, and dict container types - Fix circular reference resolution (now returns correct list[Model] type) - Refactor resolve_with_constraints to reduce return statements
2 participants
Fixes: #1160
Summary by CodeRabbit
Release Notes
New Features
Tests
✏️ Tip: You can customize this high-level summary in your review settings.