Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

deflate: Faster/efficient low level encoders #105

Merged
merged 62 commits into from
Oct 10, 2019
Merged

deflate: Faster/efficient low level encoders #105

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
62 commits
Select commit Hold shift + click to select a range
b1cb937
deflate: Gap hashing and skip encoding
klauspost May 22, 2019
844f232
Skip hash for the last part of blocks.
klauspost May 22, 2019
0e829b6
Tweak further.
klauspost May 23, 2019
7728ed2
Optimize deflate bit writing.
klauspost May 23, 2019
721e794
Merge branch 'deflate-optimize-bitwriting' into deflate-skip-lower-le…
klauspost May 23, 2019
f1182b1
Avoid a lot of bounds checks for byte buffer.
klauspost May 23, 2019
df7a94e
Don't check error value unless there are sideeffects.
klauspost May 23, 2019
a13e71f
Merge remote-tracking branch 'remotes/klauspost/deflate-optimize-bitw…
klauspost May 23, 2019
06a10a9
Avoid more bounds checks.
klauspost May 23, 2019
c9f0cde
Remove more bounds checks
klauspost May 24, 2019
1e6f5a8
Remove final bounds checks from indexer+writer.
klauspost May 24, 2019
d81e961
fmt it
klauspost May 24, 2019
8be34c6
Merge branch 'deflate-optimize-bitwriting' into deflate-skip-lower-le…
klauspost May 24, 2019
0fd006b
Merge branch 'master' into deflate-skip-lower-levels
klauspost May 26, 2019
aec1bd8
Check 8 bytes/loop for matches.
klauspost May 27, 2019
80fae85
Try 4 byte match before doing 8 bytes.
klauspost May 28, 2019
87fe04d
fmt
klauspost May 28, 2019
e5a97db
Remove unused code.
klauspost May 28, 2019
ac656c7
Simplify bounds checks
klauspost Jun 2, 2019
3d74e17
Rename to fastEnc
klauspost Jun 10, 2019
7f5ca4f
Merge branch 'master' into deflate-skip-lower-levels
klauspost Jun 10, 2019
4c2da43
Use flat history
klauspost Jun 10, 2019
bc78eb2
Move Level 2 -> 1, add new level 2.
klauspost Jun 10, 2019
970835a
Add new level 4 with dual hashing.
klauspost Jun 10, 2019
0a41eb6
Add dedicated level 5 compression.
klauspost Jun 10, 2019
a2a0ea3
Add a decent level 6
klauspost Jun 10, 2019
6a6c230
Move encoders to separate files.
klauspost Jun 12, 2019
b34654e
Support dictionaries for levels 1-6.
klauspost Jun 12, 2019
05d211c
Do token indexing as they are added.
klauspost Jun 13, 2019
2736b3e
Fix size estimation function.
klauspost Jun 14, 2019
c325151
Move bit flushing to separate function. At least 5-8 faster due to in…
klauspost Jun 14, 2019
d7af83e
Tweak AddMatch for inlining.
klauspost Jun 14, 2019
f3292d2
Re-add bounds check avoidance. Credit to @egonelbre :)
klauspost Jun 14, 2019
9f407f5
Use check without shift for offsetCode - @egonelbre strikes again!
klauspost Jun 14, 2019
3cc1639
Completely remove unneeded check. offset length can only be up to 15 …
klauspost Jun 14, 2019
1786eb3
reorder fields and use extra tables (#125)
egonelbre Jun 15, 2019
7455284
Add test/benchmark for EstimatedBits
klauspost Jun 15, 2019
2caf33c
Add huffman table reuse.
klauspost Jun 20, 2019
1259b8e
Manually inline code writing.
klauspost Jun 20, 2019
ae21d9c
Fix huffman code generation.
klauspost Jun 20, 2019
09fbddc
Don't add maxbits for no-present values.
klauspost Jun 21, 2019
3a3bce5
* Add bufferReset as constant
klauspost Jun 25, 2019
818e7a3
Fix token writing.
klauspost Aug 5, 2019
40e8dba
Fix typo in level 6.
klauspost Aug 5, 2019
a29415d
Remove table masks.
klauspost Aug 5, 2019
6d9dac0
Update bit writer tests files.
klauspost Aug 5, 2019
9875bfb
Remove assembler completely.
klauspost Aug 5, 2019
af86d6c
Fix buffer reset size.
klauspost Aug 5, 2019
e15b88c
Fix level 4 table reset.
klauspost Aug 5, 2019
9bb3e90
Minor compression tweaks.
klauspost Aug 6, 2019
2843ac6
Tweak level 3+4.
klauspost Aug 6, 2019
fe5f2ab
Cleanup.
klauspost Aug 15, 2019
9992353
Emit EOB when asked for sync.
klauspost Aug 15, 2019
682a962
Don't limit match length searches.
klauspost Aug 18, 2019
f3de74d
Tweak skiplog.
klauspost Aug 18, 2019
e390b13
Don't create historgram on no compression.
klauspost Aug 18, 2019
fce35b4
Merge branch 'master' into deflate-skip-lower-levels
klauspost Aug 26, 2019
7125ab1
Tweaks.
klauspost Aug 26, 2019
5b74cc8
Merge branch 'master' into deflate-skip-lower-levels
klauspost Aug 29, 2019
e038cca
Documentation.
klauspost Aug 29, 2019
931f7f3
Merge branch 'master' into deflate-skip-lower-levels
klauspost Oct 10, 2019
d110fb0
Update README.md
klauspost Oct 10, 2019
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -12,6 +12,7 @@ This package provides various compression algorithms.
[![fuzzit](https://app.fuzzit.dev/badge?org_id=klauspost)](https://fuzzit.dev)

# changelog
* Oct 10, 2019: Big deflate rewrite, 30-40% faster with better compression [#105](https://github.com/klauspost/compress/pull/105)
* Oct 10, 2019: (v1.8.6) zstd: Allow partial reads to get flushed data. [#169](https://github.com/klauspost/compress/pull/169)
* Oct 3, 2019: Fix inconsistent results on broken zstd streams.
* Sep 25, 2019: Added `-rm` (remove source files) and `-q` (no output except errors) to `s2c` and `s2d` [commands](https://github.com/klauspost/compress/tree/master/s2#commandline-tools)
Expand Down
193 changes: 0 additions & 193 deletions flate/asm_test.go
View file
Open in desktop

This file was deleted.

42 changes: 0 additions & 42 deletions flate/crc32_amd64.go
View file
Open in desktop

This file was deleted.

Loading