Add namespace-based types file splitting by nac5504 · Pull Request #925 · apple/swift-openapi-generator

nac5504 · 2026-07-14T17:21:39Z

Motivation

This is the first stage of a recently proposed effort to add output file splitting to the generator. Rather than landing the full dependency-sharding design all at once, this PR introduces a simple splitting strategy that is easy to reason about and yields substantial results already.

The immediate strategy is titlednamespace splitting: keep root declarations in Types.swift, move Components to Types+Components.swift, and move Operations to Types+Operations.swift. While this current splitting configuration is trivial, the broader goal of this PR is to make the output pipeline and config model ready for more advanced splitting strategies later, such as count-based slices or dependency-based layering.

Commit Outline (5)

The commit messages contain more specific technical decisions that were made during this process.

1. Add multi-file generator output plumbing (a2013a8)

Adds the multi-output generator path so one generation run can produce more than one Swift file.
Initially kept existing single-file behavior intact while introducing the multi-output path; the final API shape is simplified in Review Patch 2 below.

2. Add file splitting output config (d2949a8)

Adds the output.types.fileSplitting configuration model and initial namespace strategy option.
Exposes the opt-in through both YAML config and the direct CLI flag.

3. Add namespace-based types file splitting (eb817f7)

Emits Types.swift, Types+Components.swift, and Types+Operations.swift when namespace splitting is enabled.
Preserves the root API surface while moving Components and Operations into their own generated files.

4. [Review Patch 1] Simplify multi-file output model (b002759)

Addresses initial review feedback by flattening rendered output to an array instead of adding a wrapper type.
Removes brittle single-file conveniences now that structured output can contain multiple files.
Removes empty namespace-specific config options and the dead output-file-name conditional in the tool write path.

5. [Review Patch 2] Address multi-output generator API review feedback (3979529)

Removes the single-file core generator entry point so runGenerator returns the pipeline final multi-file output directly.
Updates the tool and tests to use the simplified multi-output API.
Keeps the output-file-name composition helper internal and renames its variadic argument to avoid confusion with file extensions.

Usability

This is opt-in and exposed through the same places users already configure generation.

YAML config:

output:
  types:
    fileSplitting:
      strategy: namespace

CLI flag:

swift-openapi-generator generate openapi.yaml \
  --mode types \
  --types-file-splitting namespace

Programmatic callers that need every generated file can use the multi-output API:

let files = try runGenerator(input: input, config: config, diagnostics: diagnostics)

Compatibility

Existing users are unaffected unless they explicitly opt in:

Existing configs still generate the same single Types.swift by default.
Programmatic runGenerator callers now receive the generator pipeline output array directly.
Existing configured output file names continue to apply to the mode's primary output file.
Build-tool plugin invocations reject file splitting with a validation error instead of silently producing an undeclared output set.

Future Build-Tool Plugin Support

The SwiftPM/Xcode build-tool plugin must declare generated output files before invoking the generator executable. Supporting splitting there needs a separate design so the plugin can determine the generated output set without duplicating the generator's YAML parsing logic.

I explored whether the plugin could reuse the existing Yams/OpenAPI generator dependency graph for this, but SwiftPM plugin targets do not get access to library dependencies transitively through executable dependencies, and adding Yams directly as a plugin dependency is rejected because it is a library product. That makes manual config parsing in the plugin an unattractive direction for this first PR.

For this reason, this PR keeps the first slice focused on the CLI/command-plugin path and rejects build-tool plugin usage when output.types.fileSplitting is configured. I opened a draft follow-up PR targeting this branch to show one concrete build-tool plugin implementation using a lightweight manual YAML lookup, so reviewers can evaluate that direction separately from this initial CLI-focused slice.

Future Direction

This PR intentionally adds only the smallest useful splitting strategy. The underlying shape is meant to make later sharding work easier to review and merge in smaller pieces, instead of coupling every output pipeline change to the final splitting algorithm. Future strategies can build on the same config and multi-output path, for example:

flowchart TD
    A["types file splitting"] --> B["namespace"]
    A --> C["slices"]
    A --> D["schema dependency layering"]

The important part is that the generator pipeline, renderer, CLI, config model, and tool output path now have a real multi-output path to build on. Further improvements to this effort will be more understandable and reviewable.

Test Plan

Unit Tests

Checklist links point to the commit diff that introduced each test, so reviewers can see the added lines in context.

Benchmark Testing

I ran an extensive benchmark suite on against representative OpenAPI specs of different sizes and shapes, seen in the charts below. Each chart row compares the current single-file output against the new namespace splitting mode, over 5 attempts excluding outliers due to VM compute inconsistencies. Whiskers on the build and generator time charts show +/-1 standard deviation across the attempts. An observability layer was patched locally on top of the generator to observe durations spent in each stage, shown in the last chart.

Benchmark Results

The results show the main target metric improves: post-generator Swift build time goes down with namespace splitting. This first chart shows the swift build times after generator output has been written, interfaced through the generator CLI. Splitting the generated Types.swift surface into root, Components, and Operations files gives Swift's compiler smaller source files to process, and that shows up most clearly on the larger specs where generated types dominate the output.

These benefits come at little cost, as the generator time does not increase with statistical significance. Namespace splitting is very close to the single-file baseline across the suite, and is even slightly faster in several rows after outlier filtering.

The phase breakdown within the generator supports the same read. Namespace splitting adds some bookkeeping to route declarations into multiple output files and preserve imports, but that work is not large enough to materially move end-to-end generator time.

Overall, the benchmark story matches the design goal: namespace splitting produces a source layout that is friendlier to downstream Swift compilation without adding a meaningful generator-time cost or substantial added complexity.

### Motivation Prepare the generator pipeline for file-splitting features by allowing a single generator run to carry more than one rendered Swift output, while preserving the existing single-file API and behavior for current users. ### Modifications Introduces new models/endpoints needed for multi-file plumbing: - `StructuredSwiftRepresentation.files`: lets the structured Swift stage contain one or more named Swift files before rendering. - `StructuredSwiftRepresentation.file`: keeps the existing single-file access pattern for pipeline stages that still expect exactly one structured file. - `RenderedSwiftRepresentation`: remains the representation of one rendered Swift file. - `RenderedSwiftOutputs`: represents all rendered Swift files produced by one generator pipeline run. Updates the generator pipeline so rendering still happens one file at a time. Each structured file is rendered through its own renderer instance, then the resulting files are collected into `RenderedSwiftOutputs`. Adds `runGeneratorOutputs`, which returns all generated `InMemoryOutputFile`s from a generator run. Keeps `runGenerator` as the compatibility wrapper for existing callers. It still returns a single `InMemoryOutputFile` and now asserts that the pipeline produced exactly one file before returning it. Updates the tool output path to write every file returned by `runGeneratorOutputs`. File names flow from the translator-provided `NamedFileDescription.name` through rendering to `InMemoryOutputFile.baseName`, with the existing configured `outputFileName` override still applied to the mode’s primary output file. ### Result Existing users are not regressed: - The public `runGenerator` behavior remains single-file. - Existing generator modes still produce one output file by default. - Existing configured output filenames are preserved. - Multi-file output is only exposed through the new `runGeneratorOutputs` API. This gives later file-splitting branches a dedicated multi-output path without changing the behavior of current single-output callers.

### Motivation Add configuration plumbing for splitting `Types.swift` output across multiple files. This prepares the generator for file-splitting strategies without changing the default generated output. ### Modifications - Add `OutputOptions` to `Config` as the top-level home for generated-output settings. - Add `TypesOutputOptions` and `TypesFileSplittingConfig` to model `types`\-specific file splitting configuration. - Add the initial `TypesFileSplittingStrategy.namespace` strategy and `NamespaceTypesFileSplittingOptions`. - Decode the new `output.types.fileSplitting` section from YAML configuration files. - Add a --types-file-splitting command-line option so direct CLI invocations can select the file-splitting strategy. - Keep CLI option resolution structured by constructing a full `TypesFileSplittingConfig` from command-line options before merging it into resolved output options. (Not utilized yet since `namespace` splitting has no parameters. More advanced splitting configs will have knob) - Include the resolved types file splitting strategy in verbose generator output. - Document the new YAML configuration shape and matching command-line option. ### Consumer Interfaces #### Programmatic/Core API Callers that construct `_OpenAPIGeneratorCore.Config` directly can pass output options through the new `output` parameter: ```swift let config = Config( mode: .types, access: .public, namingStrategy: .defensive, output: .init( types: .init( fileSplitting: .init(strategy: .namespace) ) ) ) ``` The default remains no file splitting: ```swift let config = Config( mode: .types, access: .public, namingStrategy: .defensive ) ``` #### YAML Configuration All tool integrations that use `openapi-generator-config.yaml` can opt in through the new `output` section: ```yaml generate: - types output: types: fileSplitting: strategy: namespace ``` #### Direct CLI and Custom Build Systems Direct command-line users and build rules that shell out to the generator can use the new strategy flag: ```sh swift-openapi-generator generate path/to/openapi.yaml \ --config path/to/openapi-generator-config.yaml \ --types-file-splitting namespace \ --output-directory "$DERIVED_SOURCES_DIR" ``` The `namespace` strategy does not need additional CLI flags. Strategies that require extra inputs can add strategy-specific flags in their implementation branches and resolve them into the same `TypesFileSplittingConfig` model. ### Result Users and integration points now have a stable way to request types file splitting: - Programmatic callers use `Config(output:)`. - Config-file users use `output.types.fileSplitting`. - Direct CLI users can use `--types-file-splitting`. Existing configurations continue to default to no file splitting.

nac5504 · 2026-07-15T14:52:58Z

    validator: @escaping (ParsedOpenAPIRepresentation, Config) throws -> [Diagnostic] = validateDoc,
    translator: any TranslatorProtocol = MultiplexTranslator(),
-    renderer: any RendererProtocol = TextBasedRenderer.default,
+    renderer: @escaping () -> any RendererProtocol = { TextBasedRenderer.default },


Uses a fresh renderer for each file because TextBasedRenderer owns a stateful StringCodeWriter and would accumulate rendered swift for multiple file outputs

nac5504 · 2026-07-15T15:04:53Z

    public static var allOutputFileNames: [String] { GeneratorMode.allCases.map(\.outputFileName) }

+    /// Returns a Swift output file name composed from the provided name components.
+    public static func outputFileName(_ name: String, _ extensionNames: String...) -> String {


Helpful to extract to create split output filenames, such as Types+Components.swift, Types+Operations.swift, or Types+Slice1.swift.

Fine to have. Does it need to be public? And the use of the term "extension" in the name is confusing when considering file names. IIUC this is the Swift extension (e.g. Components) and the file extension will always be .swift. Open to ideas on how you make that clearer.

### Motivation Add the first concrete types file-splitting strategy using the multi-output generator plumbing. The namespace strategy splits `Types.swift` into stable top-level files without parsing rendered Swift source. ### Modifications - Teach `TypesFileTranslator` to emit multiple structured Swift files when `output.types.fileSplitting.strategy` is `namespace`. - Keep the root declarations in `Types.swift`, including `APIProtocol`, the API protocol extension, and server declarations. - Move the generated `Components` namespace into `Types+Components.swift`. - Move the generated `Operations` namespace into `Types+Operations.swift`. - Add a shared `GeneratorMode` helper for constructing generated Swift file names. - Add typed output-name planning on `TypesFileSplittingConfig` for the generated namespace files. - Reuse the existing renderer multi-output path so each split file is rendered independently from its translator-provided name. - Reject file splitting for build-tool plugin invocations with a clear validation error. - Document that build-tool plugin support is not included in this first slice. - Add coverage for namespace splitting, the default unsplit behavior, and build-tool plugin rejection. ### Build-Tool Plugin Follow-Up The SwiftPM/Xcode build-tool plugin must declare generated output files before invoking the generator executable. Supporting splitting there needs a separate design so the plugin can determine the generated output set without duplicating the generator's YAML parsing logic. SwiftPM plugin targets do not get access to library dependencies transitively through executable dependencies, and adding `Yams` directly as a plugin dependency is rejected because it is a library product. For that reason, this PR intentionally keeps support focused on direct generator/command-plugin usage and leaves build-tool plugin support to a follow-up. ### Result Users can opt in to namespace-based types splitting with: ```yaml output: types: fileSplitting: strategy: namespace ``` or with: ```sh swift-openapi-generator generate openapi.yaml --mode types --types-file-splitting namespace ``` When enabled, types generation emits: ```text Types.swift Types+Components.swift Types+Operations.swift ``` When the setting is absent, generation continues to emit the existing single `Types.swift` file.

simonjbeaumont

Thanks for getting the ball rolling here. I'm broadly in support of us moving in this direction and have left some initial comments to get us going.

As a courtesy note: I'm about to head OOO for a couple of weeks. It's possible @czechboy0 will be able to keep the ball rolling but I cannot promise that.

simonjbeaumont · 2026-07-15T07:25:41Z

+    /// Options that only affect `Types.swift` generation.
+    public var types: TypesOutputOptions?


Do we foresee a need to have this level of distinction in the config? Are we anticipating output options that only affect types that we wouldn't want to also apply to the client and server outputs?

I do think splitting client/server could be useful later for very large specs, but I see it as a separate strategy rather than this namespace strategy. Client/Server outputs are mostly flat operation methods that reference Operations types, while Types.swift owns the large generated declaration graph. So for this first slice I scoped the config under output.types to reflect the strategy we actually support today. I could also foresee some other type of output option in the future that doesn't fall within the realm of output file splitting for types.

simonjbeaumont · 2026-07-15T15:31:23Z

+        for output in outputs {
+            try replaceFileContents(
+                inDirectory: outputDirectory,
+                fileName: output.baseName == config.mode.outputFileName ? outputFileName : output.baseName,


Does this condition ever do anything any more?

nac5504 · 2026-07-15T17:13:06Z

    /// - Throws: An error if an issue occurs during rendering.
-    func render(structured code: StructuredSwiftRepresentation, config: Config, diagnostics: any DiagnosticCollector)
-        throws -> InMemoryOutputFile
+    func render(file: NamedFileDescription, config: Config, diagnostics: any DiagnosticCollector) throws -> InMemoryOutputFile


now renders one file at a time, instead of StructuredSwiftRepresentation which contains multiple files now

nac5504 · 2026-07-15T17:32:35Z

 public func runGenerator(input: InMemoryInputFile, config: Config, diagnostics: any DiagnosticCollector) throws
    -> InMemoryOutputFile
-{ try makeGeneratorPipeline(config: config, diagnostics: diagnostics).run(input) }
+{


Would you also be in favor of removing the existing runGenerator endpoint which returns one file, and instead just have the multi-output version? The reason I did this was to preserve existing callsites of current users.

Yes, I'd expect this to be the case. The pipeline runs as a series of transformations over inputs and outputs. It's possible for one stage to fan out or in, and the output of the whole pipeline should, therefore, be the output type of the final stage.

Resolve Simon's initial review comments: - Remove RenderedSwiftOutputs and make RenderedSwiftRepresentation the rendered file array directly. - Keep rendered-output plumbing internal instead of adding new public wrapper API. - Remove StructuredSwiftRepresentation.file and the explicit init(files:) convenience; callers now use the files storage/memberwise initializer directly. - Render one NamedFileDescription at a time via RendererProtocol.render(namedFile:), with GeneratorPipeline mapping over structured files. - Remove NamespaceTypesFileSplittingOptions and the namespace: {} YAML/test/documentation surface until there are real namespace options. - Remove the dead runGenerator outputFileName parameter and write generated outputs by output.baseName directly.

nac5504 · 2026-07-16T17:59:26Z

@czechboy0 Would love to get your thoughts on this first proposal also! Happy to talk through design decisions, as I will be putting substantial time towards this effort over the next month. Thank you!

simonjbeaumont

I think this is really starting to take shape. Thanks for the hard work @nac5504!

I'd like to give @czechboy0 a chance to look at this before we land it, but I've marked it as approved meaning approved-in-principle, notwithstanding any feedback he may have.

simonjbeaumont · 2026-07-17T14:07:44Z

    public static var allOutputFileNames: [String] { GeneratorMode.allCases.map(\.outputFileName) }

+    /// Returns a Swift output file name composed from the provided name components.
+    public static func outputFileName(_ name: String, _ extensionNames: String...) -> String {


Fine to have. Does it need to be public? And the use of the term "extension" in the name is confusing when considering file names. IIUC this is the Swift extension (e.g. Components) and the file extension will always be .swift. Open to ideas on how you make that clearer.

simonjbeaumont · 2026-07-17T14:09:13Z

 public func runGenerator(input: InMemoryInputFile, config: Config, diagnostics: any DiagnosticCollector) throws
    -> InMemoryOutputFile
-{ try makeGeneratorPipeline(config: config, diagnostics: diagnostics).run(input) }
+{


Yes, I'd expect this to be the case. The pipeline runs as a series of transformations over inputs and outputs. It's possible for one stage to fan out or in, and the output of the whole pipeline should, therefore, be the output type of the final stage.

Remove the single-file core generator entry point so runGenerator returns the pipeline's final multi-file output directly. Also keep the filename composition helper internal and rename its variadic argument to avoid confusion with file extensions.

nac5504 · 2026-07-17T15:31:07Z

Thanks for the support @simonjbeaumont! Just addressed some of your comments, let me know if you see anything else worth cleaning up :)

Additionally, am I expected to update documentation for this contribution?

Nick Candello added 2 commits July 14, 2026 13:20

nac5504 force-pushed the nick/oss-types-file-splitting branch 3 times, most recently from 93c06a0 to a905d34 Compare July 14, 2026 21:42

nac5504 commented Jul 15, 2026

View reviewed changes

Comment thread Sources/_OpenAPIGeneratorCore/Layers/StructuredSwiftRepresentation.swift Outdated

nac5504 commented Jul 15, 2026

View reviewed changes

nac5504 force-pushed the nick/oss-types-file-splitting branch from a905d34 to eb817f7 Compare July 15, 2026 15:08

nac5504 marked this pull request as ready for review July 15, 2026 15:10

simonjbeaumont reviewed Jul 15, 2026

View reviewed changes

nac5504 mentioned this pull request Jul 15, 2026

[Generator][Experimental] Add opt-in dependency-aware sharded Types generation #866

Open

nac5504 force-pushed the nick/oss-types-file-splitting branch from 4ec525d to 0bf8f4e Compare July 15, 2026 17:11

nac5504 commented Jul 15, 2026

View reviewed changes

nac5504 force-pushed the nick/oss-types-file-splitting branch 2 times, most recently from 47ef39b to d430cf6 Compare July 15, 2026 17:20

nac5504 commented Jul 15, 2026

View reviewed changes

nac5504 force-pushed the nick/oss-types-file-splitting branch from d430cf6 to 5893941 Compare July 15, 2026 17:41

nac5504 force-pushed the nick/oss-types-file-splitting branch from 5893941 to b002759 Compare July 15, 2026 17:43

nac5504 requested a review from simonjbeaumont July 15, 2026 17:46

simonjbeaumont approved these changes Jul 17, 2026

View reviewed changes

		/// Options that only affect `Types.swift` generation.
		public var types: TypesOutputOptions?

Uh oh!

Conversation

nac5504 commented Jul 14, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Motivation

Commit Outline (5)

1. Add multi-file generator output plumbing (a2013a8)

2. Add file splitting output config (d2949a8)

3. Add namespace-based types file splitting (eb817f7)

4. [Review Patch 1] Simplify multi-file output model (b002759)

5. [Review Patch 2] Address multi-output generator API review feedback (3979529)

Usability

Compatibility

Future Build-Tool Plugin Support

Future Direction

Test Plan

Unit Tests

Benchmark Testing

Benchmark Results

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

simonjbeaumont left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

nac5504 commented Jul 16, 2026

Uh oh!

simonjbeaumont left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

nac5504 commented Jul 17, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

nac5504 commented Jul 14, 2026 •

edited

Loading

nac5504 commented Jul 17, 2026 •

edited

Loading