Skip to content

[C++][Parquet] Speed up DELTA_BINARY_PACKED decoding #51268

Description

@prtkgaur

Describe the enhancement requested

DeltaBitPackDecoder decodes noticeably slower than the other integer encodings, for two
reasons that are both in the decoder rather than the format:

  1. The prefix-sum loop reloads the frame and stores the running value on every value, because
    both members may alias the caller-owned output buffer.
  2. The bit unpacker is called once per miniblock -- one call per 32 values with the default
    geometry -- even though miniblocks sharing a bit width are packed contiguously and can be
    unpacked in a single call.

Together these are 1.5x-1.7x on the DELTA_BINARY_PACKED decode benchmarks already in the tree,
on the arms where the running sum and per-call setup are a meaningful share of the work.

Component(s)

C++

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Type

No type

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions