Zstandard v1.4.1
Maintenance
This release is primarily a maintenance release.
It includes a few bug fixes, including a fix for a rare data corruption bug, which could only be triggered in a niche use case, when doing all of the following: using multithreading mode, with an overlap size >= 512 MB, using a strategy >= ZSTD_btlazy, and compressing more than 4 GB. None of the default compression levels meet these requirements (not even --ultra ones).
Performance
This release also includes some performance improvements, among which the primary improvement is that Zstd decompression is ~7% faster, thanks to @mgrice.
See this comparison of decompression speeds at different compression levels, measured on the Silesia Corpus, on an Intel i9-9900K with GCC 9.1.0.
| Level | v1.4.0 | v1.4.1 | Delta |
|---|---|---|---|
| 1 | 1390 MB/s | 1453 MB/s | +4.5% |
| 3 | 1208 MB/s | 1301 MB/s | +7.6% |
| 5 | 1129 MB/s | 1233 MB/s | +9.2% |
| 7 | 1224 MB/s | 1347 MB/s | +10.0% |
| 16 | 1278 MB/s | 1430 MB/s | +11.8% |
Detailed list of changes
- bug: Fix data corruption in niche use cases (huge inputs + multithreading + large custom window sizes + other conditions) by @terrelln (#1659)
- bug: Fuzz legacy modes, fix uncovered bugs by @terrelln (#1593, #1594, #1595)
- bug: Fix out of bounds read by @terrelln (#1590)
- perf: Improved decoding speed by ~7% @mgrice (#1668)
- perf: Large compression ratio improvement for small
windowLogby @Cyan4973 (#1624) - perf: Faster compression speed in high compression mode for repetitive data by @terrelln (#1635)
- perf: Slightly improved compression ratio of level 3 and 4 (
ZSTD_dfast) by @Cyan4973 (#1681) - perf: Slightly faster compression speed when re-using a context by @Cyan4973 (#1658)
- api: Add parameter to generate smaller dictionaries by @Tyler-Tran (#1656)
- cli: Recognize symlinks when built in C99 mode by @felixhandte (#1640)
- cli: Expose cpu load indicator for each file on -vv mode by @ephiepark (#1631)
- cli: Restrict read permissions on destination files by @chungy (#1644)
- cli: zstdgrep: handle -f flag by @felixhandte (#1618)
- cli: zstdcat: follow symlinks by @vejnar (#1604)
- doc: Remove extra size limit on compressed blocks by @felixhandte (#1689)
- doc: Improve documentation on streaming buffer sizes by @Cyan4973 (#1629)
- build: CMake: support building with LZ4 @LeeYoung624 (#1626)
- build: CMake: install zstdless and zstdgrep by @LeeYoung624 (#1647)
- build: CMake: respect existing uninstall target by @j301scott (#1619)
- build: Make: skip multithread tests when built without support by @michaelforney (#1620)
- build: Make: Fix examples/ test target by @sjnam (#1603)
- build: Meson: rename options out of deprecated namespace by @lzutao (#1665)
- build: Meson: fix build by @lzutao (#1602)
- build: Visual Studio: don't export symbols in static lib by @scharan (#1650)
- build: Visual Studio: fix linking by @Absotively (#1639)
- build: Fix MinGW-W64 build by @myzhang1029 (#1600)
- misc: Expand decodecorpus coverage by @ephiepark (#1664)