Skip to content

Fix required field default rendering and --use-default nullable types#3054

Open
butvinm wants to merge 6 commits intokoxudaxi:mainfrom
butvinm:fix/required-field-defaults
Open

Fix required field default rendering and --use-default nullable types#3054
butvinm wants to merge 6 commits intokoxudaxi:mainfrom
butvinm:fix/required-field-defaults

Conversation

@butvinm
Copy link
Copy Markdown
Contributor

@butvinm butvinm commented Mar 16, 2026

Fixes #3048

Two changes:

1. Consistent required field defaults (fixes the reported inconsistency)

__set_validate_default_on_fields now skips required fields, so model-ref fields no longer get defaults while scalar fields don't. All required fields behave the same — no defaults rendered without --use-default.

2. --use-default no longer makes fields nullable (breaking change)

Previously --use-default turned status: str | None = 'active' — making required fields optional and nullable. Now it produces status: str = 'active' — default is rendered but the type stays non-nullable. I consider the previous behavior a bug: the schema says the type is string, not string | null, and --use-default shouldn't change that.

Before (bug)

Using the schema from #3048, without --use-default:

class Order(BaseModel):
    order_id: str
    status: Status
    priority: Priority
    quantity: int
    note: str
    tags: list[str]
    metadata: dict[str, str]
    shipping_address: Address = Field(
        {'street': '123 Main St', 'city': 'Springfield'}, validate_default=True
    )
    all_addresses: list[Address] = Field(
        [{'street': '1 First St', 'city': 'Shelbyville'}], validate_default=True
    )
    addresses: dict[str, Address] = Field(
        {'home': {'street': '10 Oak Ave', 'city': 'Ogdenville'}}, validate_default=True
    )

Scalar required fields (status, quantity, etc.) have defaults dropped, but model-ref required fields (shipping_address, etc.) render defaults — inconsistent.

After (fix)

Without --use-default — consistent, all required fields have no defaults:

class Order(BaseModel):
    order_id: str
    status: Status
    priority: Priority
    quantity: int
    note: str
    tags: list[str]
    metadata: dict[str, str]
    shipping_address: Address
    all_addresses: list[Address]
    addresses: dict[str, Address]

With --use-default — all defaults rendered, types stay non-nullable:

class Order(BaseModel):
    order_id: str
    status: Status = 'pending'
    priority: Priority = 2
    quantity: int = 1
    note: str = 'your note here'
    tags: list[str] = ['new']
    metadata: dict[str, str] = {'source': 'web'}
    shipping_address: Address = Field(
        {'street': '123 Main St', 'city': 'Springfield'}, validate_default=True
    )
    all_addresses: list[Address] = Field(
        [{'street': '1 First St', 'city': 'Shelbyville'}], validate_default=True
    )
    addresses: dict[str, Address] = Field(
        {'home': {'street': '10 Oak Ave', 'city': 'Ogdenville'}}, validate_default=True
    )

Summary by CodeRabbit

  • New Features

    • Introduced an option to apply defaults to fields that remain marked as required, preserving non-nullable types while keeping defaults.
  • Bug Fixes

    • Prevented removal of default/default_factory for required fields when the new option is enabled, fixing unintended optionalization and placeholder emissions.
  • Tests

    • Added end-to-end tests and expected fixtures validating required-with-default behavior across JSON Schema, OpenAPI and GraphQL generation.

butvinm and others added 2 commits March 16, 2026 05:49
__set_validate_default_on_fields set validate_default=True on fields
with model references without checking field.required, causing required
model-ref fields to render defaults while required scalar fields didn't.

Skip required fields (unless use_default_with_required) so all required
fields consistently have no defaults rendered.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
--use-default previously skipped setting field.required=True, which made
fields non-required and therefore nullable (e.g. str | None = 'foo').

Now fields stay required=True with a new use_default_with_required flag
that allows defaults to render without changing the type. Produces
`status: str = 'foo'` instead of `status: str | None = 'foo'`.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
@coderabbitai
Copy link
Copy Markdown

coderabbitai Bot commented Mar 16, 2026

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro

Run ID: f562b9ae-8441-42a8-8a93-b8ee5149f896

📥 Commits

Reviewing files that changed from the base of the PR and between bd38bd1 and 4297c8a.

⛔ Files ignored due to path filters (2)
  • tests/data/jsonschema/use_default_required_null_default.json is excluded by !tests/data/**/*.json and included by none
  • tests/data/jsonschema/use_default_strict_nullable_required.json is excluded by !tests/data/**/*.json and included by none
📒 Files selected for processing (7)
  • src/datamodel_code_generator/model/dataclass.py
  • src/datamodel_code_generator/model/msgspec.py
  • src/datamodel_code_generator/model/pydantic_base.py
  • tests/data/expected/main/jsonschema/use_default_msgspec_field_ordering.py
  • tests/data/expected/main/jsonschema/use_default_required_null_default.py
  • tests/data/expected/main/jsonschema/use_default_strict_nullable_required.py
  • tests/main/jsonschema/test_main_jsonschema.py
✅ Files skipped from review due to trivial changes (3)
  • tests/data/expected/main/jsonschema/use_default_msgspec_field_ordering.py
  • tests/data/expected/main/jsonschema/use_default_strict_nullable_required.py
  • tests/data/expected/main/jsonschema/use_default_required_null_default.py

📝 Walkthrough

Walkthrough

Adds a use_default_with_required: bool flag to DataModelFieldBase and propagates it from GraphQL, JSON Schema, and OpenAPI parsers; field stringification and parser logic now preserve default/default_factory for required fields only when this flag is true, instead of unconditionally stripping defaults for required fields.

Changes

Cohort / File(s) Summary
Core Field Model
src/datamodel_code_generator/model/base.py
Added use_default_with_required: bool = False to DataModelFieldBase.
Field Serialization
src/datamodel_code_generator/model/dataclass.py, src/datamodel_code_generator/model/msgspec.py, src/datamodel_code_generator/model/pydantic_base.py
Adjusted stringification and has_field_assignment/Field(...) logic to preserve default/default_factory and avoid emitting "..." for required fields when use_default_with_required is true.
Parser Base
src/datamodel_code_generator/parser/base.py
__set_validate_default_on_fields skips setting validate_default for required fields that do not use default-with-required.
Schema Parsers
src/datamodel_code_generator/parser/jsonschema.py, src/datamodel_code_generator/parser/graphql.py, src/datamodel_code_generator/parser/openapi.py
Compute use_default_with_required (from required, has_default, and config) and pass it into data model field construction; removed previous behavior of toggling required→optional when defaults existed. Signatures and call sites updated accordingly.
Generated Tests / Expected Outputs
tests/data/expected/main/graphql/*, tests/data/expected/main/jsonschema/*, tests/data/expected/main/openapi/*
Updated expected outputs so fields with defaults remain required (types drop `
Test Cases
tests/main/jsonschema/test_main_jsonschema.py
Added E2E tests covering allOf/required/default interactions and framework-specific regressions for the new behavior.

Sequence Diagram

sequenceDiagram
    participant Parser as Parser (jsonschema/graphql/openapi)
    participant Flag as FlagComputation
    participant Field as DataModelFieldBase
    participant Serializer as FieldSerializer
    participant Output as GeneratedCode

    Parser->>Flag: provide required, effective_has_default, config
    Flag-->>Parser: use_default_with_required = required && effective_has_default && config.apply_default_values_for_required_fields
    Parser->>Field: instantiate(name, required, default, use_default_with_required)
    Field->>Serializer: request string representation
    alt use_default_with_required == true
        Serializer->>Output: include default / default_factory / validate_default
    else
        Serializer->>Output: omit default keys for required fields (emit "..." or Field(...))
    end
Loading

Estimated code review effort

🎯 3 (Moderate) | ⏱️ ~20 minutes

Possibly related PRs

Suggested labels

breaking-change-analyzed

Suggested reviewers

  • ilovelinux

Poem

🐰
A tiny flag hopped into the stream,
Kept defaults snug like a coder's dream.
Required but steady, no None in sight,
Fields stay complete — snug and right.
Hooray — the generator bounces with delight.

🚥 Pre-merge checks | ✅ 5
✅ Passed checks (5 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title clearly and concisely summarizes the main changes: fixing required field default rendering and preventing --use-default from making fields nullable.
Linked Issues check ✅ Passed The PR successfully addresses both objectives from issue #3048: it ensures required fields don't render defaults by default and prevents --use-default from making fields nullable/optional.
Out of Scope Changes check ✅ Passed All changes are directly related to the two stated objectives: fixing consistent required field defaults and preventing --use-default from making fields nullable. No extraneous changes detected.
Docstring Coverage ✅ Passed Docstring coverage is 85.71% which is sufficient. The required threshold is 80.00%.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.

✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands and usage tips.

@codspeed-hq
Copy link
Copy Markdown

codspeed-hq Bot commented Mar 16, 2026

Merging this PR will not alter performance

⚠️ Unknown Walltime execution environment detected

Using the Walltime instrument on standard Hosted Runners will lead to inconsistent data.

For the most accurate results, we recommend using CodSpeed Macro Runners: bare-metal machines fine-tuned for performance measurement consistency.

✅ 11 untouched benchmarks
⏩ 98 skipped benchmarks1


Comparing butvinm:fix/required-field-defaults (4297c8a) with main (92bdc27)

Open in CodSpeed

Footnotes

  1. 98 benchmarks were skipped, so the baseline results were used instead. If they were deleted from the codebase, click here and archive them to remove them from the performance reports.

@codecov
Copy link
Copy Markdown

codecov Bot commented Mar 16, 2026

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 100.00%. Comparing base (92bdc27) to head (4297c8a).

Additional details and impacted files
@@            Coverage Diff            @@
##              main     #3054   +/-   ##
=========================================
  Coverage   100.00%   100.00%           
=========================================
  Files           87        87           
  Lines        18306     18321   +15     
  Branches      2091      2091           
=========================================
+ Hits         18306     18321   +15     
Flag Coverage Δ
unittests 100.00% <100.00%> (ø)

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

butvinm and others added 2 commits March 16, 2026 09:29
Cover the previously uncovered code paths:
- allOf with $ref + sub-schema required fields + --use-default
- allOf with outer required fields + --force-optional

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
@butvinm butvinm force-pushed the fix/required-field-defaults branch from 3f51eb9 to bcef9e0 Compare March 16, 2026 08:54
@butvinm butvinm marked this pull request as ready for review March 16, 2026 10:05
@koxudaxi
Copy link
Copy Markdown
Owner

@butvinm

Thanks for the PR!
This is an issue that has been on the agenda for a while. It looks like the PR is trying to fix multiple problems at once, so I want to check the related issues for each of them before reviewing. Please bear with me. I woke up early today and I'm a bit tired, so I'll set aside time tomorrow to take a proper look.
As a project policy, I'd like to avoid breaking changes, so I'm hoping we can have some kind of migration period.

@koxudaxi
Copy link
Copy Markdown
Owner

@ilovelinux If you have time, could you share your thoughts on this PR, the related issues, and the approach? I feel like the PR needs to be split since it addresses two separate problems.

@ilovelinux ilovelinux self-requested a review March 19, 2026 22:28
@ilovelinux
Copy link
Copy Markdown
Collaborator

@koxudaxi I'll give a look ASAP 🙂

@koxudaxi
Copy link
Copy Markdown
Owner

@ilovelinux Thanks! I haven't gone through all the related issues yet and this looks like it could have a wide impact, so I want to take my time. I'll be back tonight, no rush at all.

Copy link
Copy Markdown
Collaborator

@ilovelinux ilovelinux left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hi @koxudaxi, @butvinm. Sorry for the late answer. Last week has been tough.

Note

I splitted this comment in multiple paragraph. I though I wouldn't have much to say, but I was wrong. I hope they are easier to read.

About the changes

Note

@koxudaxi

If you have time, could you share your thoughts on this PR, the related issues, and the approach?

I admit reviewing changes hasn't been easy. I feel like we should reduce code complexity to increase code maintainability. However it kinda makes sense to me. I'll try to do a more in-depth review after this comment to double-check the logic.

I appreciate @butvinm work and agree with the fixes introduced, because:

  • The first change allow a coherent usage of default values.
  • The second change fix None type hint for required properties with default value defined (which can't be None because they are required).

IMHO both changes make sense and reflect the expected user experience.

I also found an useful description in the OpenAPI docs1 (emphasis mine):

The default value to use for substitution, which SHALL be sent if an alternate value is not supplied.

E.g. We have a Python client and a Python server which both uses datamodel-code-generator. The server shouldn't populate ANY property with default values, instead the client should use default values and send them explicitly.

More generally, this adhere to the DRY principle: one side populate the data, the other side validates the data. Double-validation and, more important, double-population of default values, is error-prone.

As now, it isn't possible to adhere to the DRY principle using models generated by datamodel-code-generator. This PR make default values coherent avoiding double-population, and removes wrong None type hint improving data validation, allowing to apply DRY principle.

About the multiple-responsibilities

@koxudaxi

It looks like the PR is trying to fix multiple problems at once, so I want to check the related issues for each of them before reviewing.

I agree the two changes solve two different problems. However:

  1. I'm not sure if they are also independent of each other.
    • If they are interdependent, splitting would make things more complex since one of them would depend on the other.
  2. They are both breaking changes about the same topic and I feel like they should be released together to reduce the end-user impact.

Despite single-responsibility PRs are cleaner and easier to review, I think we could avoid splitting this PR because the two changes are small and share the same surface impact (= default values).

About the breaking changes

I don't know how could a migration period for the users looks like for a change like this. I am not sure it would be useful because there's no way to avoid the breaking change. E.g.: changing the default formatter is a breaking change, but the migration period allow the users to acknowledge that and pin the old ones in the configuration file; this, instead, changes the way default values are rendered. There's no way to avoid that for the user and there's nothing the user can do to mitigate the breaking change. This looks more like an "incompatible version migration".

What if, instead, we communicate to the user what did change as soon as the user run the new version? A warning about breaking changes would:

  • Reduce the cognitive effort to keep track of the breaking changes.
  • Increases the chances the user understand and acknowledge what and why that happened.
  • Encourage the user to open an issue if something is broken because of the breaking change (e.g. a corner case that is no longer handled).

In this case, I think the users would appreciate this PR since it's a bugfix and I don't think many people were relying on either the wrong generated type hints or the default values only for model-ref fields. I think, instead, many other people may be having some hard time because of this.


I'm not approving this PR because I'm taking some time to think about the impact, what @koxudaxi said, and all I have written. In the meanwhile, I'd like to hear what do you all think! 🙂

Also, a hot pizza is waiting for me! 🍕 Can't ignore Italian priorities 😆

Footnotes

  1. https://swagger.io/specification/#server-variable-object

@butvinm
Copy link
Copy Markdown
Contributor Author

butvinm commented Mar 26, 2026

@ilovelinux thanks for the thorough review! I agree on all points - especially about not splitting and treating this as a bugfix rather than a breaking change.

Building on that, I think we could go even further. Since this PR already introduces breaking changes, we could address the broader nullable/required fields problem rather than patching it incrementally.

Required fields have their defaults dropped by default. The sole purpose of --use-default is to change this. But looking at the git history, this was never an intentional design decision - it was an artifact of template ordering (PR #99, March 2020). When a user reported it (issue #224), --use-default was added as a quick opt-in. The maintainer's own
truth table from issue #242 shows name: int = 100 (non-nullable with default) as the intended output - which is exactly what this PR now produces.

If there's no evidence of use cases that depend on dropping defaults from required fields, we could always render defaults - removing the special case entirely and deprecating --use-default. This would simplify both the API and the code.

I understand this is a bigger breaking change, so I'm fine keeping this PR as-is and exploring that direction separately.

@ilovelinux
Copy link
Copy Markdown
Collaborator

@butvinm

If there's no evidence of use cases that depend on dropping defaults from required fields, we could always render defaults - removing the special case entirely and deprecating --use-default. This would simplify both the API and the code.

Deprecating --use-default isn't something we want because there are some use-cases where a given component shall not have the responsibility of populating fields with default values. I mentioned that in my previous comment (DRY principle).

However, since I think the end user expects that default values are used to populate fields "by default", we may consider making --use-default opt-out (e.g. --no-use-default or --ignore-default). 👀

I understand this is a bigger breaking change, so I'm fine keeping this PR as-is and exploring that direction separately.

Yes. Feel free to open an issue about that so we can explore this proposal deeply without polluting this PR. 🙂

@butvinm
Copy link
Copy Markdown
Contributor Author

butvinm commented Mar 26, 2026

@ilovelinux agreed, will file a separate issue. Worth noting that currently --use-default doesn't cover this use case either — it only controls required fields. Non-required fields with non-None defaults always render them, with no way to opt out. A broader --strip-default flag (superset of --strip-default-none) might be the right solution for that.

@koxudaxi
Copy link
Copy Markdown
Owner

koxudaxi commented Apr 4, 2026

@butvinm @ilovelinux

Thanks for the detailed discussion.
I think this PR is currently touching on two different topics:

  1. fixing the inconsistency reported in Default values for required fields are sometimes rendered and sometimes not #3048
  2. discussing the broader design of default handling

For this PR, I would prefer to limit the scope to the first one.
In other words, I think we should focus on:

  • making required fields with defaults behave consistently
  • preventing --use-default from making required fields nullable

From the behavior/correctness point of view, I think these changes are closer to bug fixes.
At the same time, since they change generated output, I also agree that they may feel like breaking changes for users depending on the previous behavior.

So I think the safest way to handle this is:

  • treat this as a bug fix with compatibility impact
  • document the generated output changes clearly in the release notes / changelog

For the broader design questions around --use-default, such as:

  • whether it should remain opt-in
  • whether it should become opt-out in the future
  • whether we need a more general option like --strip-default

I think it would be better to open a separate issue and discuss them there.
That would make this PR easier to review and help us avoid mixing the immediate fix with a wider API / design discussion.

Does this direction sound good to both of you?
If we agree on this direction, I think the next steps should be:

  • @butvinm, please update the PR description to make the scoped goal explicit, and add a short note about the compatibility impact
  • @butvinm, please open a separate issue for the broader default-handling discussion if you still want to continue that proposal
  • after that, we can continue reviewing this PR only within the limited scope above

butvinm and others added 2 commits April 26, 2026 01:05
Three regressions where the new use_default_with_required flag wasn't
propagated to all the rendering sites that decide whether a field has
an assignment. Each produces broken output on this branch but the
correct output on baseline (92bdc27).

1. pydantic_base.py:205 — early-return for required+nullable fields
   with no other Field args returned Field(...) without checking
   use_default_with_required, dropping the user's default.
   Repro: --use-default --strict-nullable, required nullable str with
   default. Got: x: str | None = Field(...). Want: x: str | None = 'hello'.

2. dataclass.py:has_field_assignment — sort-key helper didn't account
   for use_default_with_required, so a required field with default null
   (use_default_with_required=True, field.field empty) was sorted as
   no-assignment alongside truly required fields. Schema order was then
   preserved, placing the defaulted field before the non-defaulted one.
   Repro: --use-default --output-model-type dataclasses.dataclass, required
   nullable a (default null) before required b. Got: a before b
   (TypeError on import). Want: b before a. Same fix covers
   pydantic_v2.dataclass since it shares the helper.

3. msgspec.py:_has_field_assignment — same class of bug. Helper checked
   neither bool(field.field) nor use_default_with_required, breaking
   ordering for both string-defaulted and null-defaulted required fields.
   Repro: --use-default --output-model-type msgspec.Struct on the existing
   allof_required_use_default.json fixture. Got: name='unnamed' before tag
   (TypeError on import). Want: tag before name='unnamed'.

Adds three regression tests in tests/main/jsonschema/test_main_jsonschema.py
covering the three repros above.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
@butvinm
Copy link
Copy Markdown
Contributor Author

butvinm commented Apr 25, 2026

Hi! Sorry for disappearing — it was a tough month. I agree with limiting the scope of this PR to the two fixes, and moving the broader default-handling discussion to a separate issue.

Merged latest main, caught and fixed a few implementation bugs. PR is ready to merge whenever you'd like.

I'll open the follow-up issue for the broader default-handling discussion soon.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Default values for required fields are sometimes rendered and sometimes not

3 participants