Skip to content

Bloom-filter fast-path for the dense tag store (O(1) insertion)#11900

Draft
dougqh wants to merge 1 commit into
dougqh/dense-storefrom
dougqh/dense-bloom-filter
Draft

Bloom-filter fast-path for the dense tag store (O(1) insertion)#11900
dougqh wants to merge 1 commit into
dougqh/dense-storefrom
dougqh/dense-bloom-filter

Conversation

@dougqh

@dougqh dougqh commented Jul 9, 2026

Copy link
Copy Markdown
Contributor

What

Adds a per-map bloom-style presence filter (one long, knownBloom) to OptimizedTagMap's dense known-tag store. A clear bit means a known tag is definitely absent, so an append skips the O(n) knownIndexOf linear scan — the common per-span-build case (each tag set once) becomes O(1).

Correctness — superset invariant

The filter is a superset of the present ids' bits: set on every add, never cleared on remove (a stale-set bit only costs an unnecessary scan, never a wrong answer). The linear scan stays authoritative — a set bit means "maybe present, scan to confirm." So correctness is independent of the position→bit collision rate; only the fast-path hit rate depends on it. All 339 TagMap + dense fuzz tests pass.

Position→bit mapping

Crude 1L << (fieldPos & 63) to start. fieldPos (not globalSerial) is deliberate: it's the axis a future per-type graph-coloring optimizes — coloring co-occurring tags to distinct positions raises the fast-path hit rate without touching correctness.

Why (evidence)

On the stacked id-comparison benchmark, the bloom brings dense id-insertion to HashMap parity at typical tag counts (0.99× at 7 tags, up from 0.91× without it), while dense still allocates ~half of HashMap. Cost: +8 B/op (the single long field). The demonstrating benchmark lands in the stacked id PR.

Stacking

Dense-store id line: #11814 (Entry-less storage) → this (bloom fast-path) → id API + comparison benchmark. Based on dougqh/dense-store.

Follow-ups

Per-type graph coloring (raises hit rate); multi-level transitive prune for read-through (skip parents whose bit is clear).

🤖 Generated with Claude Code

@dougqh dougqh added comp: core Tracer core tag: ai generated Largely based on code generated by an AI or LLM tag: no release notes Changes to exclude from release notes type: refactoring labels Jul 9, 2026
@dd-octo-sts

dd-octo-sts Bot commented Jul 9, 2026

Copy link
Copy Markdown
Contributor

🟢 Java Benchmark SLOs — All performance SLOs passed

Suite Status
Startup 🟢 pass

SLO thresholds are defined here based on automatically generated metrics. A warning is raised when results are within 5% of the threshold.

PR vs. master results
Scenario Candidate master Δ (95% CI of mean)
startup:insecure-bank:iast:Agent 13.96 s 13.98 s [-0.8%; +0.6%] (no difference)
startup:insecure-bank:tracing:Agent 12.95 s 13.01 s [-1.3%; +0.4%] (no difference)
startup:petclinic:appsec:Agent 16.84 s 16.68 s [-0.1%; +2.0%] (no difference)
startup:petclinic:iast:Agent 17.08 s 17.00 s [-0.4%; +1.3%] (no difference)
startup:petclinic:profiling:Agent 16.84 s 16.90 s [-1.4%; +0.7%] (no difference)
startup:petclinic:sca:Agent 16.96 s 16.83 s [-0.4%; +2.0%] (no difference)
startup:petclinic:tracing:Agent 16.14 s 16.03 s [-0.5%; +1.8%] (no difference)

Commit: 56550cb4 · CI Pipeline · Benchmarking Platform UI


Load and DaCapo benchmarks can be triggered manually in the GitLab pipeline. Results will appear in the Benchmarking Platform UI after completion.

@datadog-prod-us1-4

datadog-prod-us1-4 Bot commented Jul 15, 2026

Copy link
Copy Markdown

🎯 Code Coverage (details)
Patch Coverage: 92.86%
Overall Coverage: 57.32% (-0.00%)

This comment will be updated automatically if new data arrives.
🔗 Commit SHA: 56550cb | Docs | Datadog PR Page | Give us feedback!

@dougqh dougqh force-pushed the dougqh/dense-store branch from a5e12e4 to 6c70848 Compare July 15, 2026 21:23
…scan)

A per-map long bitmask (knownBloom) over the dense store: a set bit means a
tagId MAY be present (scan to confirm), a clear bit means DEFINITELY absent —
so the common per-build insert skips the linear knownIndexOf scan and appends
in O(1). Crude position->bit map (fieldPos & 63); a collision-minimizing
per-type coloring later only raises the hit rate — correctness never depends
on it because the scan stays authoritative. Superset semantics: set on add,
never cleared on remove (a stale bit costs a scan, never a wrong answer).

Alloc-neutral (one long field, no extra allocation); the win is insertion CPU,
moving the dense store toward HashMap insertion parity without the scan.

Reconciled onto the folded-class dense store (#11814).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
@dougqh dougqh force-pushed the dougqh/dense-bloom-filter branch from 267ebad to 56550cb Compare July 15, 2026 21:23
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

comp: core Tracer core tag: ai generated Largely based on code generated by an AI or LLM tag: no release notes Changes to exclude from release notes type: refactoring

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant