Skip to content

fix(flags): stop losing queued feature flag reload callbacks - #710

Merged
turnipdabeets merged 4 commits into
mainfrom
fix/queued-flags-reload-callbacks
Aug 20, 2026
Merged

fix(flags): stop losing queued feature flag reload callbacks#710
turnipdabeets merged 4 commits into
mainfrom
fix/queued-flags-reload-callbacks

Conversation

@turnipdabeets

@turnipdabeets turnipdabeets commented Aug 19, 2026

Copy link
Copy Markdown
Contributor

💡 Motivation and Context

executeFeatureFlags keeps one pending reload behind the request in flight. Two separate paths could lose a queued reload's callback entirely — the caller waits forever.

1. A displaced reload's callbacks were discarded. A third reload overwrote pendingFeatureFlagsRequest wholesale, so the displaced request's internalOnFeatureFlags and onFeatureFlags were never invoked.

2. A queued reload could be stranded with nothing to execute it. isLoadingFeatureFlags was claimed and released outside pendingFeatureFlagsLock, while the pending slot was written and drained inside it — two mechanisms guarding one piece of logical state. A reload could observe "already loading", the in-flight request could then drain an empty queue and clear the flag, and only afterwards would the reload write itself into a slot nothing would ever drain.

Both are latent, not user-visible. Every executor the SDK constructs is single-threaded (PostHog.kt, PostHogStateless.kt) and api.flags blocks on that thread, so two executeFeatureFlags calls cannot overlap. The branch was already unreachable when #407 introduced it. A host can reach it by supplying its own pooled executor through the public remoteConfigProvider hook, which is why this is worth closing rather than deleting.

The pending slot now carries every displaced caller's callbacks — newest parameters still win, but nothing queued loses its completion — and the in-flight claim, the queue write and the release all happen under one lock. Each callback also runs behind its own try/catch, so one throwing host callback can no longer skip the rest.

This is the Android counterpart to PostHog/posthog-ios#737, and matches the shape the cross-SDK spec describes for this mechanism ("queue one pending reload … so identity-sensitive requests are not lost").

💚 How did you test it?

queued reloads displaced from the pending slot still run their callbacks in PostHogRemoteConfigTest. It injects a multi-threaded executor, because that is the only way to reach the branch — it pins the contract the machinery claims, not a scenario the SDK ships. Written before the fix, and it fails on main:

java.lang.AssertionError: every queued reload must run its callback, 8 never did

It counts invocations per callback and asserts exactly one, so a double-invocation would also fail. 5 consecutive runs green.

  • :posthog:test (102 tests in PostHogRemoteConfigTest, full module suite) — passing
  • spotlessCheck and :posthog:apiCheck — clean, public API unchanged
  • Sample app on an emulator against a real project, 3 runs: person-property overrides reach /flags and the flag reads correctly, ~1s. This is a regression check only — it exercises the normal single-threaded path, so it cannot demonstrate the fix.

Not run: instrumented/Robolectric tests, or anything on a physical device.

📝 Checklist

  • I reviewed the submitted code.
  • I added tests to verify the changes.
  • I updated the docs if needed.
  • No breaking change or entry added to the changelog.

If releasing new changes

  • Ran pnpm changeset to generate a changeset file

🤖 Agent context

Autonomy: Human-driven (agent-assisted)

Written with Claude Code, directed by @turnipdabeets while reviewing the iOS counterpart (PostHog/posthog-ios#737), where @marandaneto asked whether Android needed the same fix.

executeFeatureFlags holds one pending reload behind the request in flight. A
third reload overwrote that slot wholesale, so the displaced request's
internalOnFeatureFlags and onFeatureFlags were dropped and never invoked — a
caller waiting on that callback would wait forever.

This is latent, not user-visible. Every executor the SDK constructs is single
threaded (PostHog.kt, PostHogStateless.kt) and api.flags blocks on that thread,
so two executeFeatureFlags calls cannot overlap on any stock configuration; the
queuing branch is unreachable. It was already unreachable when #407 introduced
it. Forcing overlap with a multi-threaded executor is the only way to observe
the drop, and it loses 8 of 10 callbacks.

The pending slot now carries every displaced caller's callbacks: newest
parameters still win, but nothing queued loses its completion. That is the
shape the cross-SDK spec describes for this mechanism, and matches the fix
landing in posthog-ios.

The accompanying test injects a multi-threaded executor because that is the
only way to reach the branch. It pins the contract the machinery claims, not a
scenario the SDK ships.
Review of the first commit found another way to lose a queued reload, and
reproduced it with that fix already applied.

isLoadingFeatureFlags was claimed and released outside pendingFeatureFlagsLock
while the pending slot was written and drained inside it. Two mechanisms, one
piece of logical state: a reload could observe "already loading", the in-flight
request could then drain an empty queue and clear the flag, and only afterwards
would the reload write itself into a slot nothing would ever execute. Its
request never went out and its callback never fired — the same failure the
queuing machinery exists to prevent, reached by a different door. The claim and
the queue write now happen under one lock, and the flag is released inside the
same block as the drain.

The fan-out also had no error isolation: collapsing N carried callbacks into one
meant the first host callback that threw skipped the rest, and a throwing
internal callback skipped every user callback. Each now runs behind its own
try/catch, which is what "queuing a reload never loses a callback" has to mean.

The test asserted at-least-once, so it could not have caught a double
invocation; it now counts per callback and asserts exactly one. Its latch drops
from 30s to 10s so a regression fails fast instead of stalling CI.

Changeset reworded: the reachability claim is "no executor the SDK constructs",
not "not user-visible" — remoteConfigProvider is a public hook and a host can
supply a pooled executor.
@turnipdabeets turnipdabeets self-assigned this Aug 19, 2026
The changeset body is published verbatim into CHANGELOG.md, so it is read by a
developer skimming release notes to decide whether to upgrade — not by this PR's
reviewer. It carried a paragraph of root cause, internal function names and
concurrency rationale, all of which belongs in the PR body and is already there.
Reduced to the one user-observable line.

Also trims the code comments to the parts that are not visible from the code:
the reason the lock has to cover both the claim and the queue write, the
idempotency requirement on the repeated internal callback, and why the test
needs an executor no production path uses. Dropped the ones restating what the
next line does.
@turnipdabeets
turnipdabeets marked this pull request as ready for review August 19, 2026 18:56
@turnipdabeets
turnipdabeets requested a review from a team as a code owner August 19, 2026 18:56
@github-actions

github-actions Bot commented Aug 19, 2026

Copy link
Copy Markdown
Contributor

posthog-android Compliance Report

Date: 2026-08-19 19:31:53 UTC
Duration: 118485ms

✅ All Tests Passed!

46/46 tests passed


Capture Tests

29/29 tests passed

View Details
Test Status Duration
Format Validation.Event Has Required Fields 411ms
Format Validation.Event Has Uuid 35ms
Format Validation.Event Has Lib Properties 42ms
Format Validation.Distinct Id Is String 29ms
Format Validation.Token Is Present 31ms
Format Validation.Custom Properties Preserved 30ms
Format Validation.Event Has Timestamp 33ms
Retry Behavior.Retries On 503 7031ms
Retry Behavior.Does Not Retry On 400 4027ms
Retry Behavior.Does Not Retry On 401 4033ms
Retry Behavior.Respects Retry After Header 7025ms
Retry Behavior.Implements Backoff 17035ms
Retry Behavior.Retries On 500 7019ms
Retry Behavior.Retries On 502 7017ms
Retry Behavior.Retries On 504 7019ms
Retry Behavior.Max Retries Respected 17035ms
Deduplication.Generates Unique Uuids 39ms
Deduplication.Preserves Uuid On Retry 7016ms
Deduplication.Preserves Uuid And Timestamp On Retry 12033ms
Deduplication.Preserves Uuid And Timestamp On Batch Retry 7017ms
Deduplication.No Duplicate Events In Batch 43ms
Deduplication.Different Events Have Different Uuids 31ms
Compression.Sends Gzip When Enabled 24ms
Batch Format.Uses Proper Batch Structure 22ms
Batch Format.Flush With No Events Sends Nothing 14ms
Batch Format.Multiple Events Batched Together 42ms
Error Handling.Does Not Retry On 403 4024ms
Error Handling.Does Not Retry On 413 4025ms
Error Handling.Retries On 408 5022ms

Feature_Flags Tests

17/17 tests passed

View Details
Test Status Duration
Request Payload.Request With Person Properties Device Id 34ms
Request Payload.Flags Request Uses V2 Query Param 23ms
Request Payload.Flags Request Hits Flags Path Not Decide 18ms
Request Payload.Flags Request Omits Authorization Header 23ms
Request Payload.Token In Flags Body Matches Init 34ms
Request Payload.Groups Round Trip 24ms
Request Payload.Groups Default To Empty Object 23ms
Request Payload.Disable Geoip False Propagates As Geoip Disable False 20ms
Request Payload.Disable Geoip Omitted Defaults To False 17ms
Request Payload.Flag Keys To Evaluate Contains Only Requested Key 19ms
Request Lifecycle.No Flags Request On Init Alone 10ms
Request Lifecycle.No Flags Request On Normal Capture 24ms
Request Lifecycle.Two Flag Calls Produce Two Remote Requests 41ms
Request Lifecycle.Mock Response Value Is Returned To Caller 21ms
Retry Behavior.Retries Flags On 502 324ms
Retry Behavior.Retries Flags On 504 323ms
Side Effect Events.Get Feature Flag Captures Feature Flag Called Event 21ms

@greptile-apps

greptile-apps Bot commented Aug 19, 2026

Copy link
Copy Markdown
Contributor

Reviews (1): Last reviewed commit: "chore(flags): write the changeset for re..." | Re-trigger Greptile

…g logger

Cycle 2 of review found a regression this branch introduced and a pre-existing
way to disable flag loading permanently.

The pending slot appended callbacks without dedup, but the SDK reuses single
instances - internalOnFeatureFlagsLoaded and config.onFeatureFlags are passed on
every automatic reload - so a coalesced reload fired them once per queued reload
rather than once per response. Measured 10 invocations for 2 responses, against 2
on main. distinct() restores one-per-response; the lists still carry every
distinct caller, so nothing is dropped.

runSafely logged from inside its own catch, unguarded. A PostHogLogger that
throws escaped the finally before the in-flight claim was released, leaving it
set forever: flags never reload again for the process lifetime, and every later
identify/group/reset appends to a slot nothing will drain. Logging is now
guarded and the callback fan-out cannot escape the finally at all. This was
reachable on the shipped single-threaded executor, unlike the rest of this
branch.

Both are pinned by tests that fail without the fix ("shared listener fired 10
times for 2 responses", "a later reload never ran - the in-flight claim was
stranded"), plus one for the isolation change below. The queued-reload test now
also asserts the reloads actually coalesced, so it cannot pass vacuously if the
scheduler happens to serialise it.

isLoadingFeatureFlags and pendingFeatureFlagsReload were only ever touched under
pendingFeatureFlagsLock, but AtomicBoolean advertises lock-free access - the
signal that produced the strand bug in cycle 1. The first is now a plain
lock-guarded Boolean; the second was exactly redundant with
pendingFeatureFlagsRequest != null and is gone.

Changeset rewritten. The queuing fixes are unreachable on any executor the SDK
constructs, but isolating the two callbacks means a throwing internal callback
no longer suppresses the caller's onFeatureFlags, and that happens on ordinary
single-threaded flag loads. That is the user-visible change, so it is what the
release note describes.
@posthog

posthog Bot commented Aug 19, 2026

Copy link
Copy Markdown
Contributor

🦔 ReviewHog reviewed this pull request

Nothing worth raising this time, so here's a calming picture instead:

A happy dog on a sunny path

@turnipdabeets
turnipdabeets merged commit e37fd6a into main Aug 20, 2026
17 checks passed
@turnipdabeets
turnipdabeets deleted the fix/queued-flags-reload-callbacks branch August 20, 2026 10:09
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants