Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix+reduce allocations #668

Merged
merged 1 commit into from
Sep 25, 2022
Merged

Fix+reduce allocations #668

merged 1 commit into from
Sep 25, 2022

Conversation

klauspost
Copy link
Owner

Adds WithDecodeBuffersBelow to tweak buffer switch-over.

Fixes #666 and generally reduces Reader allocations.

Adds `WithDecodeBuffersBelow` to tweak buffer switch-over.

Fixes #666 and generally reduces Reader allocations.
@klauspost klauspost merged commit 3690e90 into master Sep 25, 2022
@klauspost klauspost deleted the fix-reader-short-circuit branch September 25, 2022 08:37
owen-d pushed a commit to grafana/loki that referenced this pull request Sep 27, 2022
**What this PR does / why we need it**:

Bring in various improvements from upstream, particularly
klauspost/compress#668

**Checklist**
- [x] Reviewed the `CONTRIBUTING.md` guide
- NA Documentation added
- NA Tests updated
- [x] `CHANGELOG.md` updated
- NA Changes that require user attention or interaction to upgrade are
documented in `docs/sources/upgrading/_index.md`

**Benchmarks**

(using changes from #7246, ignoring lines that didn't change much)

```
name                           old time/op      new time/op      delta
Write/ordered-flate-4              81.5ms ± 5%      78.7ms ± 2%    -3.39%  (p=0.016 n=5+5)
Read/none_66_kB-4                  4.16ms ± 6%      3.93ms ± 4%    -5.51%  (p=0.032 n=5+5)
Read/flate_66_kB-4                  168ms ± 4%       179ms ±11%    +6.50%  (p=0.032 n=5+5)
Read/zstd_66_kB-4                   334ms ±16%       152ms ± 3%   -54.57%  (p=0.008 n=5+5)
Read/none_262_kB-4                 3.93ms ± 1%      3.81ms ± 2%    -2.91%  (p=0.008 n=5+5)
Read/zstd_262_kB-4                  369ms ± 8%       131ms ± 1%   -64.62%  (p=0.008 n=5+5)
Read/zstd_524_kB-4                  466ms ±17%       146ms ± 2%   -68.68%  (p=0.008 n=5+5)
Read/sample_zstd_66_kB-4            420ms ±18%       224ms ± 1%   -46.69%  (p=0.016 n=5+4)
Read/sample_gzip_262_kB-4           214ms ± 1%       218ms ± 1%    +1.90%  (p=0.016 n=5+5)
Read/sample_zstd_262_kB-4           418ms ±21%       200ms ± 2%   -52.15%  (p=0.016 n=5+4)
Read/sample_zstd_524_kB-4           576ms ±35%       234ms ± 1%   -59.42%  (p=0.008 n=5+5)

name                           old speed        new speed        delta
Write/ordered-gzip-4              267MB/s ±10%     284MB/s ± 2%    +6.51%  (p=0.008 n=5+5)
Write/ordered-flate-4             274MB/s ± 4%     291MB/s ± 2%    +6.49%  (p=0.008 n=5+5)
Read/none_66_kB-4                1.82GB/s ± 6%    1.92GB/s ± 4%    +5.78%  (p=0.032 n=5+5)
Read/gzip_66_kB-4                 669MB/s ± 4%     698MB/s ± 1%    +4.30%  (p=0.008 n=5+5)
Read/zstd_66_kB-4                 425MB/s ±14%     930MB/s ± 3%  +119.12%  (p=0.008 n=5+5)
Read/none_262_kB-4               1.93GB/s ± 1%    1.99GB/s ± 2%    +3.00%  (p=0.008 n=5+5)
Read/gzip_262_kB-4                701MB/s ± 5%     721MB/s ± 2%    +2.96%  (p=0.032 n=5+5)
Read/zstd_262_kB-4                357MB/s ± 8%    1009MB/s ± 1%  +182.89%  (p=0.008 n=5+5)
Read/flate_524_kB-4               750MB/s ± 1%     763MB/s ± 1%    +1.74%  (p=0.008 n=5+5)
Read/zstd_524_kB-4                365MB/s ±18%    1149MB/s ± 2%  +215.34%  (p=0.008 n=5+5)
Read/sample_zstd_66_kB-4          341MB/s ±17%     616MB/s ± 9%   +80.67%  (p=0.008 n=5+5)
Read/sample_zstd_262_kB-4         318MB/s ±18%     658MB/s ± 2%  +107.08%  (p=0.016 n=5+4)
Read/sample_gzip_524_kB-4         521MB/s ± 1%     530MB/s ± 1%    +1.72%  (p=0.032 n=5+5)
Read/sample_zstd_524_kB-4         299MB/s ±28%     718MB/s ± 1%  +140.23%  (p=0.008 n=5+5)

name                           old %compressed  new %compressed  delta
Write/ordered-gzip-4                 6.89 ± 0%        6.73 ± 0%    -2.32%  (p=0.000 n=5+4)
Write/ordered-flate-4                6.89 ± 0%        6.69 ± 0%    -2.84%  (p=0.008 n=5+5)
Write/ordered-zstd-4                 6.17 ± 0%        6.16 ± 0%    -0.02%  (p=0.008 n=5+5)
Write/unordered-gzip-4               6.89 ± 0%        6.73 ± 0%    -2.32%  (p=0.008 n=5+5)
Write/unordered-flate-4              6.89 ± 0%        6.70 ± 0%    -2.81%  (p=0.000 n=4+5)
Write/unordered-zstd-4               6.17 ± 0%        6.16 ± 0%    -0.02%  (p=0.008 n=5+5)

name                           old alloc/op     new alloc/op     delta
Write/unordered-lz4-4              18.8MB ± 4%      17.8MB ± 1%    -5.28%  (p=0.008 n=5+5)
Read/gzip_66_kB-4                   713kB ± 0%       735kB ± 1%    +2.99%  (p=0.008 n=5+5)
Read/flate_66_kB-4                  715kB ± 0%       733kB ± 0%    +2.57%  (p=0.008 n=5+5)
Read/zstd_66_kB-4                   423MB ± 0%         1MB ± 0%   -99.65%  (p=0.016 n=5+4)
Read/gzip_262_kB-4                  198kB ± 0%       204kB ± 0%    +3.07%  (p=0.016 n=4+5)
Read/lz4-64k_262_kB-4               169kB ± 0%       169kB ± 0%    -0.07%  (p=0.000 n=5+4)
Read/flate_262_kB-4                 197kB ± 0%       205kB ± 0%    +3.77%  (p=0.008 n=5+5)
Read/zstd_262_kB-4                  553MB ± 0%         0MB ± 0%   -99.92%  (p=0.016 n=5+4)
Read/gzip_524_kB-4                 96.2kB ± 0%      98.8kB ± 0%    +2.71%  (p=0.016 n=4+5)
Read/flate_524_kB-4                95.4kB ± 0%      98.7kB ± 0%    +3.46%  (p=0.008 n=5+5)
Read/zstd_524_kB-4                  786MB ± 0%         1MB ± 6%   -99.93%  (p=0.008 n=5+5)
Read/sample_gzip_66_kB-4            686kB ± 0%       703kB ± 0%    +2.48%  (p=0.029 n=4+4)
Read/sample_flate_66_kB-4           688kB ± 0%       710kB ± 1%    +3.08%  (p=0.016 n=4+5)
Read/sample_zstd_66_kB-4            423MB ± 0%         2MB ± 0%   -99.59%  (p=0.016 n=5+4)
Read/sample_gzip_262_kB-4           194kB ± 0%       200kB ± 0%    +3.10%  (p=0.016 n=4+5)
Read/sample_flate_262_kB-4          194kB ± 0%       201kB ± 0%    +3.77%  (p=0.016 n=4+5)
Read/sample_zstd_262_kB-4           553MB ± 0%         0MB ± 0%   -99.91%  (p=0.016 n=5+4)
Read/sample_gzip_524_kB-4          95.4kB ± 0%      97.9kB ± 0%    +2.65%  (p=0.008 n=5+5)
Read/sample_flate_524_kB-4         95.2kB ± 0%      98.8kB ± 0%    +3.71%  (p=0.016 n=5+4)
Read/sample_zstd_524_kB-4           786MB ± 0%         1MB ± 0%   -99.90%  (p=0.008 n=5+5)

name                           old allocs/op    new allocs/op    delta
Write/ordered-gzip-4                  701 ± 0%         718 ± 1%    +2.40%  (p=0.008 n=5+5)
Write/ordered-lz4-256k-4              656 ± 0%         657 ± 0%    +0.21%  (p=0.048 n=5+5)
Write/ordered-flate-4                 626 ± 0%         634 ± 1%    +1.37%  (p=0.008 n=5+5)
Write/ordered-zstd-4                2.07k ± 0%       1.44k ± 0%   -30.49%  (p=0.008 n=5+5)
Write/unordered-gzip-4               418k ± 0%        428k ± 0%    +2.39%  (p=0.008 n=5+5)
Write/unordered-flate-4              418k ± 0%        430k ± 0%    +2.89%  (p=0.008 n=5+5)
Write/unordered-zstd-4               469k ± 0%        468k ± 0%    -0.12%  (p=0.008 n=5+5)
Read/gzip_66_kB-4                   13.2k ± 0%       13.5k ± 0%    +2.54%  (p=0.016 n=5+4)
Read/lz4-256k_66_kB-4               13.7k ± 0%       13.7k ± 0%    -0.01%  (p=0.029 n=4+4)
Read/flate_66_kB-4                  13.2k ± 0%       13.6k ± 0%    +2.72%  (p=0.008 n=5+5)
Read/zstd_66_kB-4                   18.0k ± 1%       10.8k ± 0%   -40.19%  (p=0.008 n=5+5)
Read/gzip_262_kB-4                  4.61k ± 0%       4.76k ± 0%    +3.17%  (p=0.016 n=4+5)
Read/flate_262_kB-4                 4.61k ± 0%       4.78k ± 0%    +3.62%  (p=0.008 n=5+5)
Read/zstd_262_kB-4                  7.10k ± 2%       2.54k ± 0%   -64.22%  (p=0.016 n=5+4)
Read/gzip_524_kB-4                  2.62k ± 0%       2.69k ± 0%    +2.36%  (p=0.016 n=4+5)
Read/flate_524_kB-4                 2.62k ± 0%       2.70k ± 0%    +3.09%  (p=0.008 n=5+5)
Read/zstd_524_kB-4                  6.13k ± 2%       1.63k ± 0%   -73.40%  (p=0.008 n=5+5)
Read/sample_gzip_66_kB-4            13.2k ± 0%       13.5k ± 0%    +2.54%  (p=0.029 n=4+4)
Read/sample_flate_66_kB-4           13.2k ± 0%       13.6k ± 0%    +2.88%  (p=0.016 n=4+5)
Read/sample_zstd_66_kB-4            18.0k ± 1%       10.8k ± 0%   -40.00%  (p=0.008 n=5+5)
Read/sample_gzip_262_kB-4           4.63k ± 0%       4.77k ± 0%    +3.16%  (p=0.016 n=4+5)
Read/sample_flate_262_kB-4          4.62k ± 0%       4.79k ± 0%    +3.61%  (p=0.016 n=4+5)
Read/sample_zstd_262_kB-4           7.12k ± 1%       2.54k ± 0%   -64.28%  (p=0.008 n=5+5)
Read/sample_gzip_524_kB-4           2.63k ± 0%       2.69k ± 0%    +2.36%  (p=0.008 n=5+5)
Read/sample_flate_524_kB-4          2.63k ± 0%       2.72k ± 0%    +3.31%  (p=0.016 n=5+4)
Read/sample_zstd_524_kB-4           6.11k ± 2%       1.64k ± 0%   -73.23%  (p=0.008 n=5+5)
```
lxwzy pushed a commit to lxwzy/loki that referenced this pull request Nov 7, 2022
**What this PR does / why we need it**:

Bring in various improvements from upstream, particularly
klauspost/compress#668

**Checklist**
- [x] Reviewed the `CONTRIBUTING.md` guide
- NA Documentation added
- NA Tests updated
- [x] `CHANGELOG.md` updated
- NA Changes that require user attention or interaction to upgrade are
documented in `docs/sources/upgrading/_index.md`

**Benchmarks**

(using changes from grafana#7246, ignoring lines that didn't change much)

```
name                           old time/op      new time/op      delta
Write/ordered-flate-4              81.5ms ± 5%      78.7ms ± 2%    -3.39%  (p=0.016 n=5+5)
Read/none_66_kB-4                  4.16ms ± 6%      3.93ms ± 4%    -5.51%  (p=0.032 n=5+5)
Read/flate_66_kB-4                  168ms ± 4%       179ms ±11%    +6.50%  (p=0.032 n=5+5)
Read/zstd_66_kB-4                   334ms ±16%       152ms ± 3%   -54.57%  (p=0.008 n=5+5)
Read/none_262_kB-4                 3.93ms ± 1%      3.81ms ± 2%    -2.91%  (p=0.008 n=5+5)
Read/zstd_262_kB-4                  369ms ± 8%       131ms ± 1%   -64.62%  (p=0.008 n=5+5)
Read/zstd_524_kB-4                  466ms ±17%       146ms ± 2%   -68.68%  (p=0.008 n=5+5)
Read/sample_zstd_66_kB-4            420ms ±18%       224ms ± 1%   -46.69%  (p=0.016 n=5+4)
Read/sample_gzip_262_kB-4           214ms ± 1%       218ms ± 1%    +1.90%  (p=0.016 n=5+5)
Read/sample_zstd_262_kB-4           418ms ±21%       200ms ± 2%   -52.15%  (p=0.016 n=5+4)
Read/sample_zstd_524_kB-4           576ms ±35%       234ms ± 1%   -59.42%  (p=0.008 n=5+5)

name                           old speed        new speed        delta
Write/ordered-gzip-4              267MB/s ±10%     284MB/s ± 2%    +6.51%  (p=0.008 n=5+5)
Write/ordered-flate-4             274MB/s ± 4%     291MB/s ± 2%    +6.49%  (p=0.008 n=5+5)
Read/none_66_kB-4                1.82GB/s ± 6%    1.92GB/s ± 4%    +5.78%  (p=0.032 n=5+5)
Read/gzip_66_kB-4                 669MB/s ± 4%     698MB/s ± 1%    +4.30%  (p=0.008 n=5+5)
Read/zstd_66_kB-4                 425MB/s ±14%     930MB/s ± 3%  +119.12%  (p=0.008 n=5+5)
Read/none_262_kB-4               1.93GB/s ± 1%    1.99GB/s ± 2%    +3.00%  (p=0.008 n=5+5)
Read/gzip_262_kB-4                701MB/s ± 5%     721MB/s ± 2%    +2.96%  (p=0.032 n=5+5)
Read/zstd_262_kB-4                357MB/s ± 8%    1009MB/s ± 1%  +182.89%  (p=0.008 n=5+5)
Read/flate_524_kB-4               750MB/s ± 1%     763MB/s ± 1%    +1.74%  (p=0.008 n=5+5)
Read/zstd_524_kB-4                365MB/s ±18%    1149MB/s ± 2%  +215.34%  (p=0.008 n=5+5)
Read/sample_zstd_66_kB-4          341MB/s ±17%     616MB/s ± 9%   +80.67%  (p=0.008 n=5+5)
Read/sample_zstd_262_kB-4         318MB/s ±18%     658MB/s ± 2%  +107.08%  (p=0.016 n=5+4)
Read/sample_gzip_524_kB-4         521MB/s ± 1%     530MB/s ± 1%    +1.72%  (p=0.032 n=5+5)
Read/sample_zstd_524_kB-4         299MB/s ±28%     718MB/s ± 1%  +140.23%  (p=0.008 n=5+5)

name                           old %compressed  new %compressed  delta
Write/ordered-gzip-4                 6.89 ± 0%        6.73 ± 0%    -2.32%  (p=0.000 n=5+4)
Write/ordered-flate-4                6.89 ± 0%        6.69 ± 0%    -2.84%  (p=0.008 n=5+5)
Write/ordered-zstd-4                 6.17 ± 0%        6.16 ± 0%    -0.02%  (p=0.008 n=5+5)
Write/unordered-gzip-4               6.89 ± 0%        6.73 ± 0%    -2.32%  (p=0.008 n=5+5)
Write/unordered-flate-4              6.89 ± 0%        6.70 ± 0%    -2.81%  (p=0.000 n=4+5)
Write/unordered-zstd-4               6.17 ± 0%        6.16 ± 0%    -0.02%  (p=0.008 n=5+5)

name                           old alloc/op     new alloc/op     delta
Write/unordered-lz4-4              18.8MB ± 4%      17.8MB ± 1%    -5.28%  (p=0.008 n=5+5)
Read/gzip_66_kB-4                   713kB ± 0%       735kB ± 1%    +2.99%  (p=0.008 n=5+5)
Read/flate_66_kB-4                  715kB ± 0%       733kB ± 0%    +2.57%  (p=0.008 n=5+5)
Read/zstd_66_kB-4                   423MB ± 0%         1MB ± 0%   -99.65%  (p=0.016 n=5+4)
Read/gzip_262_kB-4                  198kB ± 0%       204kB ± 0%    +3.07%  (p=0.016 n=4+5)
Read/lz4-64k_262_kB-4               169kB ± 0%       169kB ± 0%    -0.07%  (p=0.000 n=5+4)
Read/flate_262_kB-4                 197kB ± 0%       205kB ± 0%    +3.77%  (p=0.008 n=5+5)
Read/zstd_262_kB-4                  553MB ± 0%         0MB ± 0%   -99.92%  (p=0.016 n=5+4)
Read/gzip_524_kB-4                 96.2kB ± 0%      98.8kB ± 0%    +2.71%  (p=0.016 n=4+5)
Read/flate_524_kB-4                95.4kB ± 0%      98.7kB ± 0%    +3.46%  (p=0.008 n=5+5)
Read/zstd_524_kB-4                  786MB ± 0%         1MB ± 6%   -99.93%  (p=0.008 n=5+5)
Read/sample_gzip_66_kB-4            686kB ± 0%       703kB ± 0%    +2.48%  (p=0.029 n=4+4)
Read/sample_flate_66_kB-4           688kB ± 0%       710kB ± 1%    +3.08%  (p=0.016 n=4+5)
Read/sample_zstd_66_kB-4            423MB ± 0%         2MB ± 0%   -99.59%  (p=0.016 n=5+4)
Read/sample_gzip_262_kB-4           194kB ± 0%       200kB ± 0%    +3.10%  (p=0.016 n=4+5)
Read/sample_flate_262_kB-4          194kB ± 0%       201kB ± 0%    +3.77%  (p=0.016 n=4+5)
Read/sample_zstd_262_kB-4           553MB ± 0%         0MB ± 0%   -99.91%  (p=0.016 n=5+4)
Read/sample_gzip_524_kB-4          95.4kB ± 0%      97.9kB ± 0%    +2.65%  (p=0.008 n=5+5)
Read/sample_flate_524_kB-4         95.2kB ± 0%      98.8kB ± 0%    +3.71%  (p=0.016 n=5+4)
Read/sample_zstd_524_kB-4           786MB ± 0%         1MB ± 0%   -99.90%  (p=0.008 n=5+5)

name                           old allocs/op    new allocs/op    delta
Write/ordered-gzip-4                  701 ± 0%         718 ± 1%    +2.40%  (p=0.008 n=5+5)
Write/ordered-lz4-256k-4              656 ± 0%         657 ± 0%    +0.21%  (p=0.048 n=5+5)
Write/ordered-flate-4                 626 ± 0%         634 ± 1%    +1.37%  (p=0.008 n=5+5)
Write/ordered-zstd-4                2.07k ± 0%       1.44k ± 0%   -30.49%  (p=0.008 n=5+5)
Write/unordered-gzip-4               418k ± 0%        428k ± 0%    +2.39%  (p=0.008 n=5+5)
Write/unordered-flate-4              418k ± 0%        430k ± 0%    +2.89%  (p=0.008 n=5+5)
Write/unordered-zstd-4               469k ± 0%        468k ± 0%    -0.12%  (p=0.008 n=5+5)
Read/gzip_66_kB-4                   13.2k ± 0%       13.5k ± 0%    +2.54%  (p=0.016 n=5+4)
Read/lz4-256k_66_kB-4               13.7k ± 0%       13.7k ± 0%    -0.01%  (p=0.029 n=4+4)
Read/flate_66_kB-4                  13.2k ± 0%       13.6k ± 0%    +2.72%  (p=0.008 n=5+5)
Read/zstd_66_kB-4                   18.0k ± 1%       10.8k ± 0%   -40.19%  (p=0.008 n=5+5)
Read/gzip_262_kB-4                  4.61k ± 0%       4.76k ± 0%    +3.17%  (p=0.016 n=4+5)
Read/flate_262_kB-4                 4.61k ± 0%       4.78k ± 0%    +3.62%  (p=0.008 n=5+5)
Read/zstd_262_kB-4                  7.10k ± 2%       2.54k ± 0%   -64.22%  (p=0.016 n=5+4)
Read/gzip_524_kB-4                  2.62k ± 0%       2.69k ± 0%    +2.36%  (p=0.016 n=4+5)
Read/flate_524_kB-4                 2.62k ± 0%       2.70k ± 0%    +3.09%  (p=0.008 n=5+5)
Read/zstd_524_kB-4                  6.13k ± 2%       1.63k ± 0%   -73.40%  (p=0.008 n=5+5)
Read/sample_gzip_66_kB-4            13.2k ± 0%       13.5k ± 0%    +2.54%  (p=0.029 n=4+4)
Read/sample_flate_66_kB-4           13.2k ± 0%       13.6k ± 0%    +2.88%  (p=0.016 n=4+5)
Read/sample_zstd_66_kB-4            18.0k ± 1%       10.8k ± 0%   -40.00%  (p=0.008 n=5+5)
Read/sample_gzip_262_kB-4           4.63k ± 0%       4.77k ± 0%    +3.16%  (p=0.016 n=4+5)
Read/sample_flate_262_kB-4          4.62k ± 0%       4.79k ± 0%    +3.61%  (p=0.016 n=4+5)
Read/sample_zstd_262_kB-4           7.12k ± 1%       2.54k ± 0%   -64.28%  (p=0.008 n=5+5)
Read/sample_gzip_524_kB-4           2.63k ± 0%       2.69k ± 0%    +2.36%  (p=0.008 n=5+5)
Read/sample_flate_524_kB-4          2.63k ± 0%       2.72k ± 0%    +3.31%  (p=0.016 n=5+4)
Read/sample_zstd_524_kB-4           6.11k ± 2%       1.64k ± 0%   -73.23%  (p=0.008 n=5+5)
```
changhyuni pushed a commit to changhyuni/loki that referenced this pull request Nov 8, 2022
**What this PR does / why we need it**:

Bring in various improvements from upstream, particularly
klauspost/compress#668

**Checklist**
- [x] Reviewed the `CONTRIBUTING.md` guide
- NA Documentation added
- NA Tests updated
- [x] `CHANGELOG.md` updated
- NA Changes that require user attention or interaction to upgrade are
documented in `docs/sources/upgrading/_index.md`

**Benchmarks**

(using changes from grafana#7246, ignoring lines that didn't change much)

```
name                           old time/op      new time/op      delta
Write/ordered-flate-4              81.5ms ± 5%      78.7ms ± 2%    -3.39%  (p=0.016 n=5+5)
Read/none_66_kB-4                  4.16ms ± 6%      3.93ms ± 4%    -5.51%  (p=0.032 n=5+5)
Read/flate_66_kB-4                  168ms ± 4%       179ms ±11%    +6.50%  (p=0.032 n=5+5)
Read/zstd_66_kB-4                   334ms ±16%       152ms ± 3%   -54.57%  (p=0.008 n=5+5)
Read/none_262_kB-4                 3.93ms ± 1%      3.81ms ± 2%    -2.91%  (p=0.008 n=5+5)
Read/zstd_262_kB-4                  369ms ± 8%       131ms ± 1%   -64.62%  (p=0.008 n=5+5)
Read/zstd_524_kB-4                  466ms ±17%       146ms ± 2%   -68.68%  (p=0.008 n=5+5)
Read/sample_zstd_66_kB-4            420ms ±18%       224ms ± 1%   -46.69%  (p=0.016 n=5+4)
Read/sample_gzip_262_kB-4           214ms ± 1%       218ms ± 1%    +1.90%  (p=0.016 n=5+5)
Read/sample_zstd_262_kB-4           418ms ±21%       200ms ± 2%   -52.15%  (p=0.016 n=5+4)
Read/sample_zstd_524_kB-4           576ms ±35%       234ms ± 1%   -59.42%  (p=0.008 n=5+5)

name                           old speed        new speed        delta
Write/ordered-gzip-4              267MB/s ±10%     284MB/s ± 2%    +6.51%  (p=0.008 n=5+5)
Write/ordered-flate-4             274MB/s ± 4%     291MB/s ± 2%    +6.49%  (p=0.008 n=5+5)
Read/none_66_kB-4                1.82GB/s ± 6%    1.92GB/s ± 4%    +5.78%  (p=0.032 n=5+5)
Read/gzip_66_kB-4                 669MB/s ± 4%     698MB/s ± 1%    +4.30%  (p=0.008 n=5+5)
Read/zstd_66_kB-4                 425MB/s ±14%     930MB/s ± 3%  +119.12%  (p=0.008 n=5+5)
Read/none_262_kB-4               1.93GB/s ± 1%    1.99GB/s ± 2%    +3.00%  (p=0.008 n=5+5)
Read/gzip_262_kB-4                701MB/s ± 5%     721MB/s ± 2%    +2.96%  (p=0.032 n=5+5)
Read/zstd_262_kB-4                357MB/s ± 8%    1009MB/s ± 1%  +182.89%  (p=0.008 n=5+5)
Read/flate_524_kB-4               750MB/s ± 1%     763MB/s ± 1%    +1.74%  (p=0.008 n=5+5)
Read/zstd_524_kB-4                365MB/s ±18%    1149MB/s ± 2%  +215.34%  (p=0.008 n=5+5)
Read/sample_zstd_66_kB-4          341MB/s ±17%     616MB/s ± 9%   +80.67%  (p=0.008 n=5+5)
Read/sample_zstd_262_kB-4         318MB/s ±18%     658MB/s ± 2%  +107.08%  (p=0.016 n=5+4)
Read/sample_gzip_524_kB-4         521MB/s ± 1%     530MB/s ± 1%    +1.72%  (p=0.032 n=5+5)
Read/sample_zstd_524_kB-4         299MB/s ±28%     718MB/s ± 1%  +140.23%  (p=0.008 n=5+5)

name                           old %compressed  new %compressed  delta
Write/ordered-gzip-4                 6.89 ± 0%        6.73 ± 0%    -2.32%  (p=0.000 n=5+4)
Write/ordered-flate-4                6.89 ± 0%        6.69 ± 0%    -2.84%  (p=0.008 n=5+5)
Write/ordered-zstd-4                 6.17 ± 0%        6.16 ± 0%    -0.02%  (p=0.008 n=5+5)
Write/unordered-gzip-4               6.89 ± 0%        6.73 ± 0%    -2.32%  (p=0.008 n=5+5)
Write/unordered-flate-4              6.89 ± 0%        6.70 ± 0%    -2.81%  (p=0.000 n=4+5)
Write/unordered-zstd-4               6.17 ± 0%        6.16 ± 0%    -0.02%  (p=0.008 n=5+5)

name                           old alloc/op     new alloc/op     delta
Write/unordered-lz4-4              18.8MB ± 4%      17.8MB ± 1%    -5.28%  (p=0.008 n=5+5)
Read/gzip_66_kB-4                   713kB ± 0%       735kB ± 1%    +2.99%  (p=0.008 n=5+5)
Read/flate_66_kB-4                  715kB ± 0%       733kB ± 0%    +2.57%  (p=0.008 n=5+5)
Read/zstd_66_kB-4                   423MB ± 0%         1MB ± 0%   -99.65%  (p=0.016 n=5+4)
Read/gzip_262_kB-4                  198kB ± 0%       204kB ± 0%    +3.07%  (p=0.016 n=4+5)
Read/lz4-64k_262_kB-4               169kB ± 0%       169kB ± 0%    -0.07%  (p=0.000 n=5+4)
Read/flate_262_kB-4                 197kB ± 0%       205kB ± 0%    +3.77%  (p=0.008 n=5+5)
Read/zstd_262_kB-4                  553MB ± 0%         0MB ± 0%   -99.92%  (p=0.016 n=5+4)
Read/gzip_524_kB-4                 96.2kB ± 0%      98.8kB ± 0%    +2.71%  (p=0.016 n=4+5)
Read/flate_524_kB-4                95.4kB ± 0%      98.7kB ± 0%    +3.46%  (p=0.008 n=5+5)
Read/zstd_524_kB-4                  786MB ± 0%         1MB ± 6%   -99.93%  (p=0.008 n=5+5)
Read/sample_gzip_66_kB-4            686kB ± 0%       703kB ± 0%    +2.48%  (p=0.029 n=4+4)
Read/sample_flate_66_kB-4           688kB ± 0%       710kB ± 1%    +3.08%  (p=0.016 n=4+5)
Read/sample_zstd_66_kB-4            423MB ± 0%         2MB ± 0%   -99.59%  (p=0.016 n=5+4)
Read/sample_gzip_262_kB-4           194kB ± 0%       200kB ± 0%    +3.10%  (p=0.016 n=4+5)
Read/sample_flate_262_kB-4          194kB ± 0%       201kB ± 0%    +3.77%  (p=0.016 n=4+5)
Read/sample_zstd_262_kB-4           553MB ± 0%         0MB ± 0%   -99.91%  (p=0.016 n=5+4)
Read/sample_gzip_524_kB-4          95.4kB ± 0%      97.9kB ± 0%    +2.65%  (p=0.008 n=5+5)
Read/sample_flate_524_kB-4         95.2kB ± 0%      98.8kB ± 0%    +3.71%  (p=0.016 n=5+4)
Read/sample_zstd_524_kB-4           786MB ± 0%         1MB ± 0%   -99.90%  (p=0.008 n=5+5)

name                           old allocs/op    new allocs/op    delta
Write/ordered-gzip-4                  701 ± 0%         718 ± 1%    +2.40%  (p=0.008 n=5+5)
Write/ordered-lz4-256k-4              656 ± 0%         657 ± 0%    +0.21%  (p=0.048 n=5+5)
Write/ordered-flate-4                 626 ± 0%         634 ± 1%    +1.37%  (p=0.008 n=5+5)
Write/ordered-zstd-4                2.07k ± 0%       1.44k ± 0%   -30.49%  (p=0.008 n=5+5)
Write/unordered-gzip-4               418k ± 0%        428k ± 0%    +2.39%  (p=0.008 n=5+5)
Write/unordered-flate-4              418k ± 0%        430k ± 0%    +2.89%  (p=0.008 n=5+5)
Write/unordered-zstd-4               469k ± 0%        468k ± 0%    -0.12%  (p=0.008 n=5+5)
Read/gzip_66_kB-4                   13.2k ± 0%       13.5k ± 0%    +2.54%  (p=0.016 n=5+4)
Read/lz4-256k_66_kB-4               13.7k ± 0%       13.7k ± 0%    -0.01%  (p=0.029 n=4+4)
Read/flate_66_kB-4                  13.2k ± 0%       13.6k ± 0%    +2.72%  (p=0.008 n=5+5)
Read/zstd_66_kB-4                   18.0k ± 1%       10.8k ± 0%   -40.19%  (p=0.008 n=5+5)
Read/gzip_262_kB-4                  4.61k ± 0%       4.76k ± 0%    +3.17%  (p=0.016 n=4+5)
Read/flate_262_kB-4                 4.61k ± 0%       4.78k ± 0%    +3.62%  (p=0.008 n=5+5)
Read/zstd_262_kB-4                  7.10k ± 2%       2.54k ± 0%   -64.22%  (p=0.016 n=5+4)
Read/gzip_524_kB-4                  2.62k ± 0%       2.69k ± 0%    +2.36%  (p=0.016 n=4+5)
Read/flate_524_kB-4                 2.62k ± 0%       2.70k ± 0%    +3.09%  (p=0.008 n=5+5)
Read/zstd_524_kB-4                  6.13k ± 2%       1.63k ± 0%   -73.40%  (p=0.008 n=5+5)
Read/sample_gzip_66_kB-4            13.2k ± 0%       13.5k ± 0%    +2.54%  (p=0.029 n=4+4)
Read/sample_flate_66_kB-4           13.2k ± 0%       13.6k ± 0%    +2.88%  (p=0.016 n=4+5)
Read/sample_zstd_66_kB-4            18.0k ± 1%       10.8k ± 0%   -40.00%  (p=0.008 n=5+5)
Read/sample_gzip_262_kB-4           4.63k ± 0%       4.77k ± 0%    +3.16%  (p=0.016 n=4+5)
Read/sample_flate_262_kB-4          4.62k ± 0%       4.79k ± 0%    +3.61%  (p=0.016 n=4+5)
Read/sample_zstd_262_kB-4           7.12k ± 1%       2.54k ± 0%   -64.28%  (p=0.008 n=5+5)
Read/sample_gzip_524_kB-4           2.63k ± 0%       2.69k ± 0%    +2.36%  (p=0.008 n=5+5)
Read/sample_flate_524_kB-4          2.63k ± 0%       2.72k ± 0%    +3.31%  (p=0.016 n=5+4)
Read/sample_zstd_524_kB-4           6.11k ± 2%       1.64k ± 0%   -73.23%  (p=0.008 n=5+5)
```
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

zstd decoder not re-using memory effectively between Reset() and Read()
1 participant