Conversation
The zstd feature binds the C library through zstd-sys, which needs a C toolchain at build time and links C code into every consumer. The new zstd-rust feature provides the same CompressionMethod::Zstd and the same `with zstd` syntax through structured-zstd, a pure-Rust implementation of the format. Frames are interchangeable between the two backends; when both features are enabled, zstd-rust is used. - compress: backend selected by feature, same public API. - codegen, root crate: feature passed through. - Tests run under zstd-rust as well; a new interop test decodes C-zstd frames with the Rust backend and the reverse. - CI: test matrix entries for zstd-rust, and a check that it pulls in neither zstd, zstd-sys nor libflate.
structured-zstd keeps the coder state inline, which made the Zstd variants of FlateEncoder and FlateDecoder far larger than the others (clippy::large_enum_variant). Boxing them keeps the enums compact; the C backend already holds its state behind a pointer.
SOF3
reviewed
Sep 27, 2026
| type ZstdDecoder<R> = zstd::Decoder<'static, std::io::BufReader<R>>; | ||
|
|
||
| #[cfg(feature = "zstd-rust")] | ||
| fn zstd_encoder<W: Write>(write: W) -> io::Result<ZstdEncoder<W>> { |
Owner
There was a problem hiding this comment.
not a fan of exclusive features. i think they should be exposed as if separate decoding formats rather than replacing one another.
SOF3
reviewed
Sep 27, 2026
| @@ -0,0 +1,53 @@ | |||
| // include-flate | |||
Owner
There was a problem hiding this comment.
should this have been a unit test instead of an integration test?
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The
zstdfeature binds the C library throughzstd-sys, so every crate that embeds datawith zstdneeds a C toolchain at build time and links C code. This adds an opt-inzstd-rustfeature that provides the sameCompressionMethod::Zstdand the samewith zstdsyntax throughstructured-zstd, a pure-Rust implementation of the format (no FFI, no cmake).Nothing changes for existing users: the default features are untouched.
Behaviour
zstd-rustalone: Zstandard without any C dependency (cargo treeshows neitherzstd,zstd-sysnorlibflate).zstdandzstd-rusttogether:zstd-rustis used. A crate that asks for it is opting out of the C toolchain, and a transitive default elsewhere in the graph should not override that.Changes
compress: the backend is selected by feature behind the existingFlateEncoder/FlateDecoderAPI. The pure-Rust coders keep their state inline, so they are boxed to keep the enums compact (clippy::large_enum_variant).codegen, root crate: the feature is passed through;with zstdaccepts either feature.zstd-rust; a newcompress/tests/zstd_interop.rsdecodes C-zstd frames with the Rust backend and the reverse (built with both features).--no-default-features --features zstd-rustand--features zstd-rust, plus a check thatzstd-rustpulls in no C bindings.# Algorithmsection offlate!now describes both Zstandard backends.Measurements
Decompressing jieba-rs's embedded dictionary (5,071,843 bytes), 200 decodes each, Apple M-series:
zstd(C, level 0)zstd-rust(CompressionLevel::Default)End to end, the first
Jieba::newin a process moves from 65.7 ms to 68.5 ms (min of 30 interleaved runs), about +4%; the firstTfIdf::defaultfrom 33.1 ms to 34.5 ms. Opting in trades that for a build with no C toolchain.Test plan
cargo test --workspacewith default features,--no-default-features --features deflate,--no-default-features --features zstd,--no-default-features --features zstd-rust, and--features zstd-rustcargo clippy --workspace --all-targetsfor the same sets (no new warnings)cargo fmt --all -- --checkcargo tree --no-default-features --features zstd-rust --invert zstd-sysfinds nothing