[FFI] Fix headers-provider use-after-free via ownership transfer#529
Open
zlata-stefanovic-db wants to merge 9 commits into
Open
[FFI] Fix headers-provider use-after-free via ownership transfer#529zlata-stefanovic-db wants to merge 9 commits into
zlata-stefanovic-db wants to merge 9 commits into
Conversation
The FFI now owns the headers provider user_data and frees it via a destroy callback only after any in-flight get_headers() returns, closing the recovery-vs-teardown use-after-free. Updates Go and C++ wrappers to hand ownership across, adds tests, changelogs, and lifetime docs. Signed-off-by: Zlata Stefanovic <zlata.stefanovic@databricks.com>
…ract After merging main, headers_callback_test (added there) passed a raw HeadersProvider* to zerobus_cpp_headers_trampoline, which this branch changed to expect a heap std::shared_ptr<HeadersProvider>* â�� causing a segfault. Wrap each provider in a heap shared_ptr (as Sdk::create_* does) and release it via zerobus_cpp_headers_free, matching the FFI ownership contract. Signed-off-by: Zlata Stefanovic <zlata.stefanovic@databricks.com>
zlata-stefanovic-db
marked this pull request as ready for review
July 17, 2026 17:31
Resolve conflicts in rust/ffi/src/common.rs and rust/ffi/src/stream.rs. - common.rs: keep both the new HeadersProviderFreeCallback type (this branch) and the async-callback types (main), side by side. - stream.rs: adopt main's build_stream_from_parts / StreamCreateAuth refactor, but carry a pre-built Arc<dyn HeadersProvider> through it so the provider is still constructed before any fallible work � preserving this branch's UAF fix (Drop frees user_data on every error path). The async entry point builds the provider with free_user_data = None, since it exposes no free callback. Signed-off-by: Zlata Stefanovic <zlata.stefanovic@databricks.com>
Signed-off-by: Zlata Stefanovic <zlata.stefanovic@databricks.com>
…ship gap - goFreeHeadersProvider: nil-check before cgo.Handle.Delete() so a zero handle can't panic across the C boundary (matches the C++ null tolerance). - Document that the async create-with-provider variant does not take ownership (free_user_data = None) and must gain one before any wrapper adopts it, to avoid reintroducing the recovery-vs-teardown UAF. - Regenerate zerobus.h with the async doc comment (cbindgen; no ABI change). Signed-off-by: Zlata Stefanovic <zlata.stefanovic@databricks.com>
The FFI now takes ownership of the headers-provider user_data via a free_user_data destroy callback. Update the .NET sync binding to match (the async FFI signature is unchanged) and hand the provider's GCHandle ownership to the FFI: it is released by NativeFree after any in-flight GetHeaders returns, instead of the stream freeing it on dispose. Fixes the cross-sdk-dotnet test-host crash from the stale signature and closes the recovery-vs-teardown use-after-free for the sync path. Re-scope now-async-only lifetime comments, document the provider lifetime/threading contract on IHeadersProvider, and add unit tests for the NativeFree destroy callback. Signed-off-by: Zlata Stefanovic <zlata.stefanovic@databricks.com>
GCHandle is a struct wrapping an IntPtr, so NativeFree frees the runtime handle through its own GCHandle.FromIntPtr copy and cannot zero the test's local handle field � asserting on handle.IsAllocated was wrong and always saw True. Observe the actual release with a WeakReference instead, from a non-inlined helper so no stray root keeps the bridge alive. Signed-off-by: Zlata Stefanovic <zlata.stefanovic@databricks.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Fixes a use-after-free where a custom
HeadersProvidercould be freed by the wrapper (Go/C++) while a Rust worker thread was still inside a synchronousget_headers()callback during connection recovery. Cooperativeabort()cannot preempt a synchronous FFI call, andclose()waited onlySHUTDOWN_TIMEOUT_SECS = 1before aborting, so a slow provider (e.g. one fetching/rotating a token) racing teardown could be invoked on freed memory.Approach: ownership transfer at the FFI boundary
The C FFI now takes ownership of the provider
user_data.zerobus_sdk_create_stream_with_headers_provider/zerobus_sdk_create_arrow_stream_with_headers_providergain a nullablefree_user_datadestroy callback.CallbackHeadersProviderinvokes it fromDrop, which runs only after the lastArcreferencing the provider drops — and the supervisor task holds its ownArcclone across the in-flightget_headers()call, so the destroy is guaranteed to run only after any synchronous callback has returned. The providerArcis constructed before any fallible work, sofree_user_datafires exactly once on every path (success after the last reference drops; failed create before returning). Wrappers hand ownership across and never freeuser_datathemselves.Changes by layer
rust/ffi):free_user_dataparam +Drop-based destroy (panic-contained viacatch_unwind); regeneratedzerobus.h; ownership/thread-safety docs.go/ffi.go,go/arrow_ffi.go): hands thecgo.Handleto the FFI viagoFreeHeadersProvider; removes the per-stream handle registry.cpp/): passes a heapstd::shared_ptr<HeadersProvider>+zerobus_cpp_headers_freedestroy trampoline;Stream/ArrowStreamno longer keep the provider.Scope: why only Go and C++
The bug class is specific to passing a borrowed
user_dataacross the C FFI while the core holds a raw pointer. The other consumers are not affected:HeadersProviderWrapperowns aPyObject, handed as an ownedArcStaticHeadersProviderTests
free_user_datafires once onDropand on failed create; null destroy is a no-op; and a teardown test reproducing the recovery-vs-teardown race (a blocking in-flightget_headerson oneArcclone while another drops) asserts the free is deferred until the callback returns.goFreeHeadersProviderreleases thecgo.Handle.All 82 Rust FFI tests and 6 C++ tests pass; changelogs updated for ffi/go/cpp.
Breaking change (FFI)
The two create-with-provider C signatures gain a
free_user_dataparameter — a non-additivezerobus.hchange. Filed under Breaking Changes inrust/ffi/NEXT_CHANGELOG.md.Follow-up required before Go picks this up
This changes the FFI signature, so an FFI release (
ffi/v*) must land and regenerate all five platformlibzerobus_ffi.aarchives ingo/lib/before the Go SDK can consume it. The committed archives are intentionally left at their pre-PR builds (the locally-rebuilt one exceeded GitHub's 100 MB limit and would have been an inconsistent single-platform artifact). Go CI's cross-SDK job builds the FFI from local Rust source, so it validates this PR without the prebuilt libs.Draft — open items
cgo.NewHandle(C++ already throws on null) — pre-existing asymmetry, non-blocking.