Merge timeseries into develop - #3732
Open
satween wants to merge 61 commits into
Open
Conversation
### What does this PR do? Adds JSON schemas for the CPU and memory timeseries event types (`timeseries-cpu-schema.json`, `timeseries-memory-schema.json`), registers them in the model-generation Gradle task, and updates the generated `api/apiSurface` and `api/dd-sdk-android-rum.api` files to reflect the new `RumTimeseriesCpuEvent`, `RumTimeseriesMemoryEvent`, and `TimeseriesConfiguration` public surface. ### Motivation The timeseries feature needs a well-defined wire format (JSON schemas) and generated Kotlin model classes before any collector or serializer code can reference them. ### Additional Notes The generated Kotlin model classes (`RumTimeseriesCpuEvent`, `RumTimeseriesMemoryEvent`) are produced at build time from these schemas via the existing `generateRumModelsFromJson` task and are not included in this commit.
Aligns timeseries model class names with the existing convention used
by ActionEvent, ErrorEvent, ViewEvent etc. (the inputNameMapping in
generate_rum_models.gradle.kts already strips the Rum prefix that
appears in the JSON schema titles).
- generate_rum_models.gradle.kts: map both schemas to
Timeseries{Memory,Cpu}Event.
- features/dd-sdk-android-rum/api: regenerate apiSurface and api dump.
…eries-json-schemas-api-surface `timeseries` RUM-13949: Add timeseries JSON schemas and regenerate API surface
Merge `develop` into `timeseries` Co-authored-by: satween <1608994+satween@users.noreply.github.com> Co-authored-by: jonathanmos <48201295+jonathanmos@users.noreply.github.com> Co-authored-by: hamorillo <hector.morilloprieto@datadoghq.com> Co-authored-by: 0xnm <4046447+0xnm@users.noreply.github.com> Co-authored-by: datadog-datadog-prod-us1-2[bot] <261164178+datadog-datadog-prod-us1-2[bot]@users.noreply.github.com> Co-authored-by: kikoveiga <francisco.veiga@datadoghq.com> Co-authored-by: sbarrio <sergio.barrio.slocker@gmail.com> Co-authored-by: timur.valeev <timur.valeev@datadoghq.com>
Merge origin/develop into feature/timeseries
- Add Pipeline, Buffer, DataPoint, Timeseries, TimeseriesFactory types - Add RumSessionScopeTimeseries and RumSessionScopeTimeseriesFactory - Wire timeseries start/stop into RumSessionScope (renewSession/stopSession) - Add NoOpTimeseries and NoOpTimeseriesFactory - Add unit tests for all new types
…eries-pipeline-infrastructure `timeseries` [2/6] RUM-13949: Implement timeseries pipeline infrastructure
…rom the existing `CpuVitalReader` vital and wraps them as `DataPoint` values; and `CpuEventSerializer`, which converts a batch of CPU `DataPoint`s into a `RumTimeseriesCpuEvent` JSON object ready for ingestion.
…eries-cpu-support `timeseries` [3/6] RUM-13949: Add CPU timeseries data-point reader and serializer
Adds `MemoryEventSerializer`, which converts a batch of memory `DataPoint`s (heap + native RSS) into a `RumTimeseriesMemoryEvent` JSON object. Introduces `TimeseriesConfiguration` as the public `@ExperimentalRumApi` configuration class (sampling intervals, batch size, background collection flag). Completes the pipeline with `RumSessionScopeTimeseriesFactory`, which wires CPU and memory readers, serializers, and the `TimeseriesConfiguration` together to create fully configured `RumSessionScopeTimeseries` instances for each new session. Completes the two metric types and provides the factory needed to tie the pipeline to the RUM session lifecycle. `MemoryEventSerializer` reports both heap and native RSS in bytes, normalised to a 0–100 percent scale using device `maxMemory`. `RumSessionScopeTimeseriesFactory` creates one pipeline per metric type and dispatches them through the same `Timeseries.Factory` interface. `TimeseriesConfiguration` exposes sampling interval, batch size, and a background-collection flag with sensible defaults.
- Remove executorFactory from TimeseriesConfiguration; executor is provided by RumFeature.vitalExecutorService at wiring time, not owned by the config class - Promote collectInBackground to a class-body property so the apiSurface generator picks it up (generator only reads propertyDeclaration nodes) - Regenerate apiSurface with val collectInBackground: Boolean included - Apply safe-cast fix in MemoryEventSerializer (as? JsonObject + null-safe chaining) - Remove executor tests from TimeseriesConfigurationTest Ref: RUM-13949
Pass the required `scale` argument to `roundToLongSafely` in the delta encoding path, matching the pattern already used in CpuEventSerializer.
- Make bufferSize/intervalMs/collectInBackground internal to avoid
- UndocumentedPublicProperty detekt violations. Reorder import in
- MemoryEventSerializerTest (ktlint). Regenerate dd-sdk-android-rum.api.
- Replace the primary constructor with an internal constructor and a
- public Builder class (setBufferSize/setIntervalMs/setCollectInBackground/build).
- Update tests and regenerate API surface files.
- Replace `?: error()` in // When blocks with `checkNotNull()` in // Then
- Wrap bare UUID.fromString() call in assertDoesNotThrow { }
- Remove unnecessary newSingleThreadScheduledExecutor entry from detekt_custom_safe_calls.yml
Ref: RUM-13949
…eries-memory-session-factory `timeseries` [4/6] RUM-13949 Add memory timeseries serializer and session factory
Adds `enableTimeseries()`/`disableTimeseries()` builder methods on `RumConfiguration.Builder`. Wires the `Timeseries.Factory` through `RumFeature` → `DatadogRumMonitor` → `RumApplicationScope` → `RumSessionScope`, so that a new `RumSessionScopeTimeseries` collector is created and started for each sampled session and stopped on session end or SDK teardown. Also enables timeseries in the Kotlin sample app. Connects the standalone pipeline and public configuration API to the existing RUM session lifecycle, completing the end-to-end data flow from configuration through to event emission. Existing scope and monitor test scaffolding is updated to pass `timeseriesFactory = null` / a mock factory where required. The `RumFeature.onStop()` ordering test verifies the writer is still live when the final flush fires.
- Restore closing `}` for `createDataWriter` accidentally dropped when `createTimeseriesCollectingFactory` was inserted after it - Add `@OptIn(ExperimentalRumApi::class)` to `createTimeseries` test helper to satisfy -Werror on the experimental API usage - Rename `RumSessionScopeFactoryTest.kt` → `RumSessionScopeTimeseriesFactoryTest.kt` to match contained class (ktlint filename rule) Ref: RUM-13949
Adds a self-contained end-to-end test suite for the timeseries pipeline using CSV-driven input fixtures and golden JSON files, without any real Android API dependencies. Includes `CSVReader`, `CsvTimeseries` (a synchronous, executor-free test double mirroring `RumSessionScopeTimeseriesFactory`), `TimeseriesEndToEndTest`, and the accompanying CSV input and expected JSON fixture files. Provides regression coverage that the full memory and CPU batch outputs match the expected wire format exactly, catching any serialization regressions independently of mock-heavy unit tests. Fixtures use a fixed 10-sample / 2-batch scenario. Numeric comparisons use a small relative tolerance for floating-point memory_percent values.
Add missing "count" field to all four expected JSON fixtures so they match the serializer output that now includes a sample count per batch. Ref: RUM-13949
…eries-rum-session-wiring `timeseries` [5/6] RUM-13949 Wire timeseries collection into RUM session lifecycle
…eries-e2e-csv-fixtures `timeseries` [6/6] RUM-13949 Add end-to-end timeseries verification tests with CSV fixtures
…meseries-RUM-16325-dev-merge # Conflicts: # tools/benchmark/src/main/java/com/datadog/benchmark/EndPoint.kt
…16325-dev-merge RUM-16325 [1/3]: Merge `develop` into `feature/timeseries`
The oneOf-primitive option generator kept its own private knownTypes set that was never cleared between files, unlike every other generator which shares (and clears per file) the FileGenerator set. As a result, when two top-level models contained a structurally identical oneOf of primitives (e.g. the Path = oneOf[string,integer] from the shared _graphql schema, referenced by both ResourceEvent and ErrorEvent), the second model's option subclasses extended the first model's nested sealed class (ErrorEvent.String : ResourceEvent.Path()), which fails to compile. Wire the shared knownTypes set into OneOfPrimitiveOptionGenerator so each model resolves its own nested type. Add a dedicated regression test that generates two models sharing an identical oneOf-primitive and asserts the second does not leak a reference to the first's nested type.
Regenerate RUM models and API surface from the updated JSON schemas: new graphql/stream/trace/transition/vital-duration/view_update schemas and updated action/error/resource/view definitions.
…16325-schema-update RUM-16325 `timeseries`: Update RUM JSON schemas and regenerate models
…16325-post-review RUM-16325 `timeseries`: Post-review fixes for timeseries
RUM-17613: Merging `develop` into `feature/timeseries`
…meseries-upd-develop
…develop Merge `develop` into `feature/timeseries` Co-authored-by: satween <timur.valeev@datadoghq.com> Co-authored-by: aleksandr-gringauz <aleksandr.gringauz@datadoghq.com> Co-authored-by: abrooksv <abrooksv@users.noreply.github.com> Co-authored-by: leoromanovsky <leo.romanovsky@datadoghq.com> Co-authored-by: ambushwork <luyi1022@outlook.com>
Rename the internal Timeseries interface to TimeseriesCollector and rename its implementations to DefaultTimeseriesCollector and DefaultTimeseriesCollectorFactory, propagating the new names through RumFeature, the RUM scope tree and the tests. No behaviour change. Ref: RUM-17613
Co-authored-by: Francisco Veiga <francisco.veiga@datadoghq.com>
…lector-rename RUM-17613: `timeseries` [1/5] Rename Timeseries abstractions to TimeseriesCollector
Derive the CPU and memory timeseries schemas from _common-schema.json and replace the hand-written JSON serializers with typed event factories that populate the full common event shape (os, device, dd, ddtags). Restrict collection through TimeseriesConfiguration.collectOnly. Ref: RUM-17613
Array.toSet() over an enum's values() is always safe, so whitelist it globally instead of suppressing UnsafeThirdPartyFunctionCall at each call site.
toTimeseriesCpuConnectivity() and toTimeseriesMemoryConnectivity() were never called: the timeseries event factories don't populate connectivity at all. Remove them along with their tests and the helper table that only those tests used.
Pipeline used to read the timeseries name back out of the hand-built JSON event; it now takes it from EventFactory.eventName, so nothing in production references these keys anymore. The only remaining caller was a PipelineTest helper whose JSON shape is never asserted on.
…eseries-schema RUM-17613: `timeseries` [2/5] Using updated timeseries schema from rum_event_schema
Timeseries batches carried the RumContext captured when RumSessionScope created the collector, so every event was attributed to whatever view was active at session start. The collector now tracks the context and hands it to Pipeline.execute()/flush() per call, and RumSessionScope feeds it the active context on each handled event. Pipeline takes over its own synchronization as part of that: reader.read() stays outside the lock so a concurrent flush() never waits for /proc I/O. Ref: RUM-17613
…kground-flush RUM-17613: `timeseries` [3/5] Pass the live RUM context to timeseries pipelines
Sampling kept running while the app sat in background, and the buffered batch was only written on session stop, so points collected before a backgrounding could stay unsent for the rest of the session. The collector now suspends the sampling chain when the active view leaves the foreground and flushes the buffers at that point, attributing the batch to the last foreground context. The suspension is delayed by 200 ms to match ActivityViewTrackingStrategy.STOP_VIEW_DELAY_MS, so an Activity-to-Activity transition is not mistaken for a backgrounding. Sampling state carries a generation counter so a suspension pending on an older generation cannot stop a chain that has since been resumed. Drops TimeseriesConfiguration.collectInBackground: background suspension is now unconditional, so the flag no longer has a meaning. Ref: RUM-17613
The pending suspend read lastForegroundRumContext at fire time, so a foreground re-entry landing between stopGeneration() and the flush made it attribute the previous view's batch to the new view. Snapshot the context when the stop is scheduled instead.
…ckground-suspend RUM-17613: `timeseries` [4/5] Flush the batch in background
Add a playground activity and an instrumented test that assert CPU and memory timeseries are collected, flushed on backgrounding and resumed on return to foreground.
…egration-tests RUM-17613: `timeseries` [5/5] Add integration tests for timeseries collection
🎉 All green!🧪 All tests passed 🎯 Code Coverage (details) 🔗 Commit SHA: 4e1a746 | Docs | View more details | Give us feedback! |
satween
marked this pull request as draft
August 19, 2026 13:43
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: ff495915ff
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
…eries-telemetry RUM-18101: Supporting `timeseries` apiUsage telemetry.
…v-2-timeseries # Conflicts: # features/dd-sdk-android-rum/src/test/kotlin/com/datadog/android/rum/internal/domain/RumDataWriterTest.kt
Merge `develop` into `timeseries`
satween
marked this pull request as ready for review
August 21, 2026 14:29
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What does this PR do?
Merges
timeseriesintodevelopAdditional Notes
All of the comments here was reviewed with the corresponding separate pull requests.
Review checklist (to be filled by reviewers)