You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
doc: document accumulator lifecycle and ownership in worker section
Adds an 'Accumulator lifecycle and ownership' subsection to §3.4 (Worker)
explaining the three-level lazy initialisation: configs copied to all
workers at startup, AggregationState created on first sample per series,
AccumulatorUpdater created on first sample per pane. Includes the full
ownership hierarchy diagram and the deterministic-hash ownership guarantee.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Copy file name to clipboardExpand all lines: asap-query-engine/src/precompute_engine/precompute_engine_design_doc.md
+43Lines changed: 43 additions & 0 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -150,6 +150,49 @@ struct AggregationState {
150
150
}
151
151
```
152
152
153
+
#### Accumulator lifecycle and ownership
154
+
155
+
Accumulators are not pre-assigned — they are created **lazily** at three nested levels:
156
+
157
+
**1. At engine startup** (`engine.rs`): every worker receives a full copy of all `AggregationConfig`s. All workers are symmetric; none is pre-assigned to any series or config.
**2. On first sample for a series** (`get_or_create_series_state`): the worker calls `matching_agg_configs(series_key)` to filter the config map by metric name, then creates one `AggregationState` per match (a `WindowManager` + empty pane map). No accumulators exist yet.
active_panes:BTreeMap::new(), // ← empty; no memory allocated for sketches yet
173
+
}).collect();
174
+
```
175
+
176
+
**3. On first sample in a pane** (`process_samples`): the accumulator is created the moment a sample falls into a pane that does not yet exist in `active_panes`.
└── series_map[series_key] one entry per series this worker owns
189
+
└── aggregations[i] one AggregationState per matching config
190
+
└── active_panes[pane_start] one AccumulatorUpdater per open pane
191
+
└── Box<dyn AggregateCore> the actual sketch / sum / minmax / etc.
192
+
```
193
+
194
+
Because `xxhash64(series_key) % N` is deterministic, a series always lands on the same worker. Its accumulators live in exactly one worker with no sharing and no locking. Workers that never receive a series never allocate any state for it.
195
+
153
196
#### Pane-Based Sliding Window Optimization
154
197
155
198
The worker uses **pane-based incremental computation** to reduce per-sample
0 commit comments