Skip to content
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -133,7 +133,7 @@ Instead of writing `.proto` files by hand, each SDK ships a tool to generate pro

### Arrow Flight ingestion (Beta)

Available in the Rust, Python, Go, TypeScript, and Java SDKs starting from version 2.0.0, and in the C++ SDK from its initial `0.1.0` release. Currently in Beta — the API is stabilising but may still change before reaching GA. A third record format option alongside JSON and Protocol Buffers: send Apache Arrow `RecordBatch` data directly to Zerobus over the Arrow Flight protocol, on the same gRPC connection. Best fit when:
Available in the Rust, Python, Go, TypeScript, and Java SDKs starting from their 2.0.0 releases, and in the C++ SDK from its initial `0.1.0` release. Currently in Beta — the API is stabilising but may still change before reaching GA. A third record format option alongside JSON and Protocol Buffers: send Apache Arrow `RecordBatch` data directly to Zerobus over the Arrow Flight protocol, on the same gRPC connection. Best fit when:

- Your workload is naturally columnar or batched — analytics pipelines, gateways aggregating short windows of rows, wide/numeric schemas where row-by-row serialization adds noticeable CPU overhead.
- Your application already produces Arrow data — pyarrow, the [arrow-rs](https://github.com/apache/arrow-rs) crates, DataFusion, Polars, or other libraries built on Arrow.
Expand Down
8 changes: 4 additions & 4 deletions cpp/include/zerobus/arrow_stream.hpp
Original file line number Diff line number Diff line change
Expand Up @@ -64,10 +64,10 @@ class ArrowStream {
/// Flush all pending batches and wait for their acknowledgment.
void flush();

/// Return all unacknowledged batches from a closed or failed stream, each as
/// a self-contained Arrow IPC stream (schema + one batch). Remains callable
/// after a failed `close()`, which keeps the handle alive so recovery is
/// possible.
/// Return all unacknowledged batches from a failed stream, each as a
/// self-contained Arrow IPC stream (schema + one batch). Callable after a
/// failed `close()` (the handle stays alive for recovery); after a successful
/// `close()` the handle is freed, so this throws instead.
std::vector<std::vector<std::uint8_t>> get_unacked_batches();

/// Gracefully close the stream, flushing pending batches first. Idempotent.
Expand Down
11 changes: 8 additions & 3 deletions purego/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -20,9 +20,14 @@ Implemented packages:
`RecordType`, descriptor requirement for `PROTO`) and sets auth metadata from
`StreamParams.Token` (`"Bearer <token>"` for bare tokens, verbatim when a
known scheme — `Bearer`, `Basic`, or `DPoP` — is already present).

Planned layers: OAuth/auth, ingest/ack state management, recovery, and the
public API.
- `internal/auth` — token providers for stream authentication. `OAuthTokenProvider`
implements the Unity Catalog OAuth 2.0 client-credentials flow with per-table
token caching and proactive refresh; `StaticTokenProvider` wraps a fixed token
for tests or externally managed lifecycles. The package also exposes
`HeadersProvider` variants (`OAuthHeadersProvider`, `StaticHeadersProvider`)
so transport open can authenticate via provider callbacks like other SDKs.

Planned layers: ingest/ack state management, recovery, and the public API.

## Regenerating the protobuf bindings

Expand Down
Loading