Skip to content

feat(table): support zstd metadata compression codec #1005

@laskoviymishka

Description

@laskoviymishka

Parent: #589

The v3 spec allows zstd-compressed metadata.json. iceberg-go's metadata-location codec layer handles gzip and none and returns unsupported write metadata compression codec for anything else. Add zstd on both read (decompress when reading a .metadata.json.zstd) and write (compress when write.metadata.compression-codec=zstd).

klauspost/compress/zstd is already a transitive dependency via parquet, so no new go.mod entry. Add a new MetadataCompressionCodecZstd constant alongside the existing gzip / none, and thread it through both directions of the codec helper.

Tests: round-trip a zstd-compressed metadata.json end-to-end; golden-fixture test against a Java-produced zstd metadata.json pins the exact byte format.

Metadata

Metadata

Assignees

No one assigned

    Labels

    Type

    No type
    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions