Pin protobuf field and enum value numbers in a sidecar artifact

[jwils]

Why

With #1080's sequential numbering, reordering or removing a field renumbers everything after it — breaking wire compatibility with previously serialized data. Protobuf field numbers must stay stable across schema evolution.

What

• schema_artifacts:dump now reads and writes a proto_field_numbers.yaml sidecar artifact: existing numbers stay fixed even if field order changes, and new fields get the next available numbers
• schema.proto keeps the public GraphQL field names while the sidecar stores private name_in_index overrides
• a field renamed with field.renamed_from reuses its existing number under the new public name
• enum value numbers are pinned in an enums section; removed values keep their numbers reserved so they are never reused (0 remains the generated *_UNSPECIFIED value)
• mappings are validated: collisions raise clear errors, and non-integer numbers are rejected instead of letting Integer() silently truncate
• configure_proto_field_number_mappings lets tests and advanced callers seed mappings without an artifact file

Risk Assessment

Low — only affects the unreleased elasticgraph-proto_ingestion extension from #1080.

References

• Stacked on Generate a protobuf schema artifact from ElasticGraph schemas #1080

---

Update — 2026-07-15

• Merged the latest Generate a protobuf schema artifact from ElasticGraph schemas #1080 generator refactor so reachable types are selected before rendering.
• Interface and union oneof alternatives now use the same stable message-field map: existing and removed subtype tags remain reserved, while new subtypes receive the next available tag.
• Verification: all 68 elasticgraph-proto_ingestion examples pass with 100% line and branch coverage; GraphQL-schema validation, lint, spelling, and Steep are green.

[myronmarston]

Suggested change

	changes, and new fields get the next available numbers. Field numbers follow protobuf's
	rules: they must be between 1 and 536,870,911, and the protobuf-reserved 19000-19999
	range is never allocated and is rejected in mappings.
	changes, and new fields get the next available numbers.

/nit I think that's a level of detail that readers won't need to know. Being informed of that doesn't impact how anyone will use this library.

[myronmarston]

I'm concerned about this--it has the potential to re-use an old field number that used to be used by another field that has been deleted.

If the logic grabs the lowest unused field number, what is the mechanism to prevent "recycling" old field numbers (which causes problems when deserializing old messages)?

Related to this, one thing that's unclear to me is what happens to deleted fields. Do they stick around in the artifact forever?

[myronmarston]

This is making me realize something: the proto_field_numbers.yaml artifact has pretty different semantics than the other artifacts and needs to be treated with more care than the others.

All the other artifacts are a pure function of the schema definition. It's safe to delete and re-recreate them at any time. This is actually a technique I've used frequently: if I had a PR which had merge conflicts on schema artifacts, the simplest approach to resolving the conflict (assuming no conflict on the Ruby schema definition) was just to delete the schema artifacts and re-generate them. It was guaranteed to be correct.

But this file can't safely be deleted: if you delete it, then you lose information about what the original field numbers were. And unlike the other schema artifacts, it's not simply an output: it's also an input into the schema generation process. The schema.proto file is a function of the Ruby definition + this file.

Another difference: the proto_field_numbers.yaml file is only used at schema artifact generation time. In contrast, the other artifacts are used by the other parts of EG.

In theory, it would be nice for all schema artifacts to share the "pure function of schema definition" property but having to maintain field numbers in the Ruby definition would be annoying so I like the idea of the proto_field_numbers.yaml file.

Here's one idea to consider: instead of treating it as a schema artifact, treat it as part of the schema definition--it just happens to be a part that EG helps you maintain (rather than needing to maintain it by hand). Concretely, here's what how I'm imagining we'd treat it differently:

• Rather than it going in the schema artifacts directory, store it alongside the schema definition files. Specifically, as a sibling to path_to_schema. This emphasizes that it's part of the schema, not really an artifact.
• Have the comment premable (at the top of the file) explain that it's an input to schema generation, can be updated by hand, EG helps you maintain it, and that (unlike the proper schema artifacts) you must not delete it and regenerate it or you will lose information.

Thoughts?