Carry a correlation's recording decision on its span, and number fork buckets per callsite

[maverox]

Carry a correlation's recording decision on its span

A request's correlation and its recording decision have had different
lifetimes. The correlation rides the tracing span, so it survives for as long as
anything still claims to belong to the request; the decision lived only in the
correlation-keyed registry, and the host drops that entry when the response is
built. Work detached from the request outlives that moment. Entering its span on
a later poll re-resolved the decision from a registry that no longer held it, so
the correlation arrived on the thread with its authorization stripped, every
boundary the work crossed answered SkipNoDecision, and nothing was written —
including nothing to say that anything had been dropped. A tape simply ended
mid-correlation. On the reference recording that is sixteen of the blocking
divergences, all of them outgoing-webhook work spawned after the response.

The answer was already being computed and then discarded. on_new_span resolves
the decision when a span carrying its own correlation is created, which is while
the registry entry is still live, and cached it as a bool used for one purpose:
deciding whether to engage the correlation scope. Keep the resolved decision
itself alongside it and install both when the span is entered, and the decision
inherits the span's lifetime, which is the lifetime it should have had.

Engagement and capture stay separate predicates, because they need opposite
defaults. Engagement is permissive when no decision exists: on replay none is
ever pushed, and both the host's per-correlation database routing and the
span-path address depend on the scope being engaged there. Capture is opt-in, so
an absent decision skips; that rule is load-bearing and has a regression behind
it. One field answering both would have to choose a default and would break
whichever it did not choose.

This is late-binding on a miss rather than early binding. A span created before
the host pushes the decision carries none, as does every span on replay, and
freezing that into a skip would trade this bug for a worse one — the whole
request would stop recording. It does not, because snapshot_recording_decision_for
already falls through to the registry whenever the thread-local holds no decision
for the correlation being asked about. That rule is now depended upon rather than
duplicated, and a test pins it.

Restoring on exit puts back the saved pair rather than re-resolving, for the same
reason: re-resolving on the way out would strip the authorization off a
correlation that still has one.

Number fork buckets per callsite and discard them with the span

A fork's bucket id was its arrival order within the correlation. That made it
positional, and positional identity amplifies: a candidate that spawned one
extra task early shifted the number of every fork after it, moved each one's
occurrence partition, and re-addressed every call beneath them. One behavioural
difference arrived as a cascade of divergences that looked unrelated to it and
to each other. A candidate differs from its recording by construction — that
difference is the replay — so this is the normal case rather than a corner, and
re-recording cannot help, because the renumbering happens inside a single
recording-and-candidate pair.

The information needed to fix it was already there and being discarded. The
correlation layer computes a fork span's own logical path twelve lines before it
builds the lineage, then numbered the fork by its parent's bucket instead. Number
it within the path and the sequence becomes local to the callsite that produced
it: a fork somewhere else in the request no longer touches this one, and a loop
that spawns three tasks instead of two shifts only the third.

Two spawn sites under one span with nothing instrumented between them still
share a path and still number against each other. That residue is left as it
is, because its blast radius is one function rather than the whole correlation,
which is the difference between a local divergence and a cascade.

The counters also leaked, in two ways that both come from the response finalizer
being the wrong clock. It cleared exactly one key, so a nested fork's entry had
no owner and was never removed; and it exists only on the recording path, so a
sampled-out request left its entries behind forever. Nearly every request is
sampled out, and the process does not restart. Group the sequences by
correlation so a correlation's whole set goes in one removal, and do the removal
when the span that owns the correlation closes. A span closes once every task
holding it has finished, which is exactly the moment no further fork can appear
under that correlation, and it happens whether or not the request was recorded.

That also leaves one owner. The layer is the only thing that creates these
sequences, and it is now the only thing that discards them.

Verification

just verify on the branch: fmt, clippy with warnings denied, workspace tests, all green. The two runtime crates this touches (deja-context, deja-runtime) were checked separately on Rust 1.85.0 with --all-targets, and the override was proven to take by confirming a post-1.85 API fails under it. Four new tests, each mutation-verified. The acceptance signal was measured end to end rather than argued from the diff: hyperswitch's deja_tail_capture tripwire, run through the real actix middleware against main plus these two commits, inverts from SkipNoDecision to Capture.

Landing order

Nothing in hyperswitch needs to change for the sixteen: the webhook site already carries .in_current_span(). The router picks this up through a pin bump to the deja main sha after this merges, not the branch tip, since a merge commit changes it. That pin-bump PR must also flip the deja_tail_capture assertion in hyperswitch, which asserts SkipNoDecision today and inverts the moment the pin carries this change. The bump stays a plain pin bump while #95's revert of the diesel 2.3 bump holds; if the diesel bump re-lands first (#96, the re-raise of #91), the same PR would also move the router from diesel 2.2.10 to 2.3, with async-bb8-diesel 0.3.0, bb8 0.9.1 and async-trait 0.1.89 moving alongside it.