Pipeline execution search data cannot be repaired: make it a first-class entity time series

[manerow]

Is your feature request related to a problem? Please describe.

pipeline_status_search_index is the only one of 67 registered indexes with no reindex coverage. Every other index can be rebuilt from the database when it drifts. This one cannot, so a wrong document stays wrong permanently.

It does drift. On a local stack the database holds endTime = null for an execution while the index holds endTime = 1787829046072 and a derived runtime = 60000, a value computed from an endTime that does not exist in the source of truth.

This is not confined to unread data. The index is reachable two ways:

• dataInsight/custom/lineChart.json declares searchIndex as a free-form string with no enum or allowlist, and DataInsightSystemChartRepository.getLiveSearchIndex(index) passes the value straight through. A Data Insights custom chart configured against this index queries it directly.
• GET /v1/search/query?index=pipelineStatus resolves, because the index is registered in indexMapping.json.

So a stale document is a wrong number on someone's dashboard, with no way to repair it short of deleting and re-ingesting history.

Why it has no coverage. Execution history is time-series data, but it is written ad hoc from PipelineRepository instead of through EntityTimeSeriesRepository, so it never enters the path PartitionWorker already provides for time-series types. Four structural facts block a simple registration:

1. Not registered as a time-series entity, so SearchIndexEntityTypes.isTimeSeriesEntity returns false and the reindex job never sources it.
2. Stored in the shared entity_extension_time_series table under extension pipeline.pipelineStatus. Every registered time-series entity uses a dedicated table (testCaseResult → data_quality_data_time_series, entityReportData → report_data_time_series) because PaginatedEntityTimeSeriesSource.getFilter() returns an unfiltered ListFilter for anything that is not a data-insight index: scope comes from the table, not a predicate. Registering a shared-table type would enumerate every extension in that table and try to deserialize each as a PipelineStatus.
3. Two unrelated document shapes share one physical index. PipelineExecutionIndex writes entityType: pipelineExecution; TableRepository.indexPipelineStatus (TableRepository.java:3536) writes a different shape with entityType: pipelineStatus and id pipelineFqn_tableId. A staged rebuild driven from executions alone would delete every table-observability document at alias swap.
4. PipelineStatus has no id. EntityTimeSeriesInterface requires getId() and setId(UUID).

Introduced in #23341 (2025-11-25) under #23073, whose delivered read path goes to the database via entityExtensionDAO, leaving these search writes unconsumed by product code.

Surfaced while verifying #31782 / #32181. That fix removes an alert storm which had been incidentally rewriting every execution document on each ingestion cycle, masking the drift. Nothing in the alerting path reads this index, so #32181 regresses nothing; it only removes the accidental self-healing.

Describe the solution you'd like

Make pipeline execution history a first-class entity time series so it is rebuilt by machinery the platform already has. Six changes, in dependency order.

1. Give PipelineStatus a derived id. Add id to the pipelineStatus definition in pipeline.json, computed from the same tuple that keys the database row:

UUID.nameUUIDFromBytes((fqn + "|" + timestamp).getBytes(UTF_8))

This is the existing idiom. ColumnSearchIndex.java:154 uses it for the column child index, as do PersonaResource, WorkflowInstanceStageListener, and WorkflowFailureListener. It must be derived rather than random because:

• The ES document id for a time-series entity is the entity id (ElasticSearchBulkSink.java:771), and rows are upserted on (entityFQNHash, extension, timestamp). Keying the id on anything outside that tuple lets one row map to several documents. executionId is optional in the schema (only timestamp and executionStatus are required), so a value that appears or changes on a later ingestion would orphan the previous document. This also corrects PipelineExecutionIndex.getDocumentId, which keys on fqn_executionId_timestamp today and can fork for exactly that reason.
• A regenerated id would leave the previous document behind on every re-send, forking one row into many documents.
• A rebuild reproduces identical ids, so reindexed documents collide correctly with live writes and the rebuild is idempotent.
• Existing rows derive their id from content on read, so no id backfill is required.

EntityTimeSeriesRepository.java:78 assigns UUID.randomUUID(), so this overrides the base behaviour rather than inheriting it.

2. Move execution history to a dedicated time-series table. Add pipeline_status_time_series with a PipelineStatusTimeSeriesDAO extends EntityTimeSeriesDAO, and migrate rows out of entity_extension_time_series where extension = 'pipeline.pipelineStatus'. This matches every existing time-series entity and makes enumeration correct by construction, which is what point 2 above requires. The DAO already exposes update(entityFQNHash, extension, json, timestamp), so the upsert-by-timestamp semantics executions need are preserved.

3. Add PipelineExecutionRepository extends EntityTimeSeriesRepository. Registers pipelineExecution in ENTITY_TS_REPOSITORY_MAP through the super constructor, owns the write path, and supplies the derived id. TestCaseResultRepository is the model: its constructor takes only a collection path, its time-series DAO, its class, and its entity type. No new REST collection is needed: collectionPath is stored on the base class and never read.

The write path being moved is PipelineRepository.addPipelineStatus and addBulkPipelineStatus, which carry alerting behaviour from #32181 that must be preserved verbatim:

• One change event per new or changed status. The bulk endpoint emits them itself via buildChangeEventJsonForBulkOperation + insertChangeEventsBatch and returns ENTITY_NO_CHANGE so the response filter adds no N+1th. EntityTimeSeriesRepository.createNewRecord has a different lifecycle (postCreate), so this does not carry over for free. Losing it stops failures alerting; duplicating it restores the notification storm.
• The unchanged-status gate. resolveStatusChange returns UNCHANGED/CREATED/UPDATED, and UNCHANGED writes nothing, search included. That is deliberate: it is what stops the storm, and it is why a reindex is the intended repair rather than a side effect of ingestion. Do not restore unconditional search writes to keep the index fresh.

4. Register pipelineExecution in SearchIndexEntityTypes.TIME_SERIES_ENTITIES. This is the change that grants coverage: PartitionWorker.java:653 already constructs a PaginatedEntityTimeSeriesSource for any type in that set, bringing keyset paging and timeSeriesEntityDays window bounding with it.

5. Split the index and preserve the alias. Move executions to pipeline_execution_search_index with mapping files for en, jp, ru, zh; leave table observability in pipeline_status_search_index untouched. Wire compatibility through the existing alias mechanism:

• pipelineExecution declares parentAliases: ["pipelineStatus"]
• pipelineStatus declares childAliases: ["pipelineExecution"]

This is the shape table already uses, declaring childAliases: ["testSuite", "testCase", "testCaseResolutionStatus", "testCaseResult", "tableColumn"] so a query on table spans its own index plus its children. Existing Data Insights charts and direct index=pipelineStatus queries keep returning both shapes, while a rebuild becomes physically incapable of touching observability documents because they live in a different index.

6. Delete PipelineRepository.reindexPipelineExecutions(). Zero callers; loads a pipeline's entire history into heap via getResultsFromAndToTimestamps with null bounds and issues one search round trip per document.

Additional context

Sequencing. Build on top of #32181, which rewrites both methods step 3 moves and edits the same pipelineStatus definition step 1 extends. Steps 1 and 2 are storage-only and change no behaviour. Step 3 moves the write path. Steps 4 and 5 turn on coverage. Each is independently shippable.

Verification.

Check	Expectation
Corrupt or delete a document, then POST /v1/apps/trigger/SearchIndexingApplication	rebuilt to match the database
Table-observability documents after alias swap	survive untouched
GET /v1/search/query?index=pipelineStatus	still returns both document shapes
Run bounded by timeSeriesEntityDays	only in-window documents rebuilt
Re-ingest an unchanged execution	one document, not two
Bulk PUT of N new statuses	N change events; re-sending the same batch emits none
Whole suite against OpenSearch	same results

Outcome. Execution history becomes repairable by the same mechanism as every other index, documents that are wrong today are corrected by the first reindex, and the two document shapes stop sharing a physical index.