Support zstd compression #109
Conversation
Add initial support for compressing tarballs with zstd compression. This provides comparable or in some cases better compression, while substantially improving decompression performance. This doesn't change any of the defaults (which still use gz and xz), just introduces initial off-by-default support for zst to allow us to experiment.
|
I'm wondering, what's your plan for experimenting? If you want to do test releases with zstd in addition to gz and xz the best place would be If you want to have test releases with ztd I'd add recompression to Still, I didn't do a full review but from a cursory glance this looks correct! We'll definitely want to land this once we start producing zstd tarballs consistently. |
|
@pietroalbini My original intention was to add support here, then experiment with doing some CI builds that added zstd. But if promote-release can do this instead, that seems like a great plan. Also, if compression happens in promote-release rather than here, we could experiment with higher compression levels there as well. |
Based on [1], but using compression level 3 for speed rather than 19 for competitiveness with xz. [1] rust-lang#109
|
Yeah, I think we should push to avoid further expansion of compression format support in rust-installer, which is harder to update (needs a submodule bump in rust-lang/rust). (In fact, I'd be OK dropping xz support too, but that's potentially a harder conversation). We can probably do this in promote-release pretty easily though, if someone wanted to file a similarish PR there. |
|
Rustup supports zstd as of 1.24.0 though with only local test cases in the codebase I can't be certain it'll work exactly as you hope, so perhaps making it happen in nightly only to begin with makes sense. |
Based on [1], but using compression level 3 for speed rather than 19 for competitiveness with xz. [1] rust-lang#109
Based on [1], but using compression level 3 for speed rather than 19 for competitiveness with xz. [1] rust-lang#109
Based on [1], but using compression level 3 for speed rather than 19 for competitiveness with xz. [1] rust-lang#109
Based on [1], but using compression level 3 for speed rather than 19 for competitiveness with xz. [1] rust-lang#109
Add initial support for compressing tarballs with zstd compression. This
provides comparable or in some cases better compression, while
substantially improving decompression performance.
This doesn't change any of the defaults (which still use gz and xz),
just introduces initial off-by-default support for zst to allow us to
experiment.