Add register_indexer_extension schema definition API#1299
Merged
Conversation
jwils
requested review from
BrianSigafoos-SQ,
bsorbo,
ellisandrews-toast,
jwondrusch,
marcdaniels-toast,
myronmarston and
rossroberts-toast
as code owners
July 7, 2026 21:10
This was referenced Jul 7, 2026
jwils
force-pushed
the
joshuaw/register-indexer-extension
branch
from
July 7, 2026 21:20
c441084 to
2cd643a
Compare
This was referenced Jul 7, 2026
jwils
force-pushed
the
joshuaw/register-indexer-extension
branch
from
July 8, 2026 02:10
2cd643a to
9845487
Compare
jwils
force-pushed
the
joshuaw/indexer-extensions-runtime-metadata
branch
from
July 8, 2026 02:10
7565b32 to
a1e6674
Compare
myronmarston
approved these changes
Jul 8, 2026
jwils
added a commit
that referenced
this pull request
Jul 8, 2026
## Why To support ingestion in any data format while keeping format-specific logic in its own gem (`elasticgraph-json_ingestion`, the upcoming `elasticgraph-proto_ingestion`), the indexer needs an extension mechanism analogous to the GraphQL one: schema definition extensions register runtime extensions in the schema artifacts, and the runtime component applies them at boot. This PR adds the runtime metadata storage for that. ## What - Add `indexer_extension_modules` to `RuntimeMetadata::Schema`, mirroring `graphql_extension_modules` - Add a `RuntimeMetadata::IndexerExtension` wrapper (twin of `GraphQLExtension`) - Nothing registers or consumes these yet, and empty lists are pruned from dumped YAML, so no schema artifacts change ## Verification - `script/lint`, `script/type_check` - `script/run_gem_specs elasticgraph-schema_artifacts` (100% coverage) - `script/run_gem_specs elasticgraph-schema_definition` - `bundle exec rake schema_artifacts:check` (no artifact changes) ## Stack Current PR is marked with `->`. - -> [#1298 Add indexer_extension_modules to runtime metadata](#1298) - [#1299 Add register_indexer_extension schema definition API](#1299) - [#1300 Apply indexer extension modules when the Indexer boots](#1300) - [#1301 Extract an ingestion adapter seam inside elasticgraph-indexer](#1301) - [#1302 Move JSON ingestion into elasticgraph-json_ingestion via an indexer extension](#1302)
Base automatically changed from
joshuaw/indexer-extensions-runtime-metadata
to
main
July 8, 2026 15:42
Mirrors register_graphql_extension: lets schema definition extensions (such as ingestion format gems) register indexer extension modules that get dumped into runtime_metadata.yaml for elasticgraph-indexer to apply at boot.
jwils
force-pushed
the
joshuaw/register-indexer-extension
branch
from
July 8, 2026 15:47
9845487 to
95c2292
Compare
jwils
added a commit
that referenced
this pull request
Jul 8, 2026
## Why With runtime metadata storage (#1298) and the registration API (#1299) in place, `elasticgraph-indexer` needs to actually apply registered extensions--introducing the indexer extension concept, mirroring how `elasticgraph-graphql` applies GraphQL extension modules from both settings YAML and runtime metadata. ## What - `Indexer#initialize` extends configured extension modules onto the instance: first from the new `indexer.extension_modules` setting (mirroring `graphql.extension_modules`), then from the runtime metadata's `indexer_extension_modules` - Regenerated config schema artifacts for the new setting - Nothing registers an indexer extension yet, so this is a no-op for all existing schemas ## Verification - `script/lint`, `script/type_check` - `script/run_gem_specs elasticgraph-indexer` (100% coverage) - `script/run_gem_specs elasticgraph-local` ## Stack Current PR is marked with `->`. - [#1298 Add indexer_extension_modules to runtime metadata](#1298) - [#1299 Add register_indexer_extension schema definition API](#1299) - -> [#1300 Apply indexer extension modules when the Indexer boots](#1300) - [#1301 Extract an ingestion adapter seam inside elasticgraph-indexer](#1301) - [#1302 Move JSON ingestion into elasticgraph-json_ingestion via an indexer extension](#1302)
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Why
Ingestion format gems need a way to ensure their indexer-side behavior is active whenever a schema is defined with their schema definition extension--the same guarantee
register_graphql_extensionprovides for GraphQL extensions (e.g. howelasticgraph-apolloregisters its engine extension).What
register_indexer_extensionto the schema definition API, mirroringregister_graphql_extensionindexer_extension_modulesruntime metadata added in Add indexer_extension_modules to runtime metadata #1298Verification
script/lint,script/type_checkscript/run_gem_specs elasticgraph-schema_definition(100% coverage)Stack
Current PR is marked with
->.