flate: Investigate smaller tables levels1-6 #223
Comments
klauspost
changed the title
klauspost
added a commit
that referenced
this issue
Feb 19, 2020
Fixes #223 After: ``` BenchmarkCompressAllocations/level(-2)/flate-12 15889 75134 ns/op 342272 B/op 11 allocs/op BenchmarkCompressAllocations/level(-2)/gzip-12 15424 76300 ns/op 342448 B/op 12 allocs/op BenchmarkCompressAllocations/level(-1)/flate-12 2673 378711 ns/op 2448774 B/op 14 allocs/op BenchmarkCompressAllocations/level(-1)/gzip-12 3342 377507 ns/op 2448949 B/op 15 allocs/op BenchmarkCompressAllocations/level(0)/flate-12 17437 76986 ns/op 339968 B/op 9 allocs/op BenchmarkCompressAllocations/level(0)/gzip-12 15076 82031 ns/op 340144 B/op 10 allocs/op BenchmarkCompressAllocations/level(1)/flate-12 3382 377466 ns/op 1924486 B/op 14 allocs/op BenchmarkCompressAllocations/level(1)/gzip-12 3436 387788 ns/op 1924662 B/op 15 allocs/op BenchmarkCompressAllocations/level(2)/flate-12 1971 591518 ns/op 2710923 B/op 14 allocs/op BenchmarkCompressAllocations/level(2)/gzip-12 2073 516709 ns/op 2711102 B/op 15 allocs/op BenchmarkCompressAllocations/level(3)/flate-12 3250 426246 ns/op 2186626 B/op 14 allocs/op BenchmarkCompressAllocations/level(3)/gzip-12 3084 420084 ns/op 2186802 B/op 15 allocs/op BenchmarkCompressAllocations/level(4)/flate-12 2733 390467 ns/op 2186626 B/op 14 allocs/op BenchmarkCompressAllocations/level(4)/gzip-12 3165 400509 ns/op 2186802 B/op 15 allocs/op BenchmarkCompressAllocations/level(5)/flate-12 2797 417904 ns/op 2448774 B/op 14 allocs/op BenchmarkCompressAllocations/level(5)/gzip-12 2455 456214 ns/op 2448948 B/op 15 allocs/op BenchmarkCompressAllocations/level(6)/flate-12 2733 471116 ns/op 2448773 B/op 14 allocs/op BenchmarkCompressAllocations/level(6)/gzip-12 2673 443633 ns/op 2448949 B/op 15 allocs/op BenchmarkCompressAllocations/level(7)/flate-12 6015 198306 ns/op 1006979 B/op 13 allocs/op BenchmarkCompressAllocations/level(7)/gzip-12 5728 188045 ns/op 1007155 B/op 14 allocs/op BenchmarkCompressAllocations/level(8)/flate-12 6684 195617 ns/op 1006979 B/op 13 allocs/op BenchmarkCompressAllocations/level(8)/gzip-12 6331 193922 ns/op 1007155 B/op 14 allocs/op BenchmarkCompressAllocations/level(9)/flate-12 6015 193829 ns/op 1006980 B/op 13 allocs/op BenchmarkCompressAllocations/level(9)/gzip-12 5728 197447 ns/op 1007155 B/op 14 allocs/op ```
klauspost
added a commit
that referenced
this issue
Feb 23, 2020
* Reduce deflate table sizes Fixes #223 After: ``` BenchmarkCompressAllocations/level(-2)/flate-12 15889 75134 ns/op 342272 B/op 11 allocs/op BenchmarkCompressAllocations/level(-2)/gzip-12 15424 76300 ns/op 342448 B/op 12 allocs/op BenchmarkCompressAllocations/level(-1)/flate-12 2673 378711 ns/op 2448774 B/op 14 allocs/op BenchmarkCompressAllocations/level(-1)/gzip-12 3342 377507 ns/op 2448949 B/op 15 allocs/op BenchmarkCompressAllocations/level(0)/flate-12 17437 76986 ns/op 339968 B/op 9 allocs/op BenchmarkCompressAllocations/level(0)/gzip-12 15076 82031 ns/op 340144 B/op 10 allocs/op BenchmarkCompressAllocations/level(1)/flate-12 3382 377466 ns/op 1924486 B/op 14 allocs/op BenchmarkCompressAllocations/level(1)/gzip-12 3436 387788 ns/op 1924662 B/op 15 allocs/op BenchmarkCompressAllocations/level(2)/flate-12 1971 591518 ns/op 2710923 B/op 14 allocs/op BenchmarkCompressAllocations/level(2)/gzip-12 2073 516709 ns/op 2711102 B/op 15 allocs/op BenchmarkCompressAllocations/level(3)/flate-12 3250 426246 ns/op 2186626 B/op 14 allocs/op BenchmarkCompressAllocations/level(3)/gzip-12 3084 420084 ns/op 2186802 B/op 15 allocs/op BenchmarkCompressAllocations/level(4)/flate-12 2733 390467 ns/op 2186626 B/op 14 allocs/op BenchmarkCompressAllocations/level(4)/gzip-12 3165 400509 ns/op 2186802 B/op 15 allocs/op BenchmarkCompressAllocations/level(5)/flate-12 2797 417904 ns/op 2448774 B/op 14 allocs/op BenchmarkCompressAllocations/level(5)/gzip-12 2455 456214 ns/op 2448948 B/op 15 allocs/op BenchmarkCompressAllocations/level(6)/flate-12 2733 471116 ns/op 2448773 B/op 14 allocs/op BenchmarkCompressAllocations/level(6)/gzip-12 2673 443633 ns/op 2448949 B/op 15 allocs/op BenchmarkCompressAllocations/level(7)/flate-12 6015 198306 ns/op 1006979 B/op 13 allocs/op BenchmarkCompressAllocations/level(7)/gzip-12 5728 188045 ns/op 1007155 B/op 14 allocs/op BenchmarkCompressAllocations/level(8)/flate-12 6684 195617 ns/op 1006979 B/op 13 allocs/op BenchmarkCompressAllocations/level(8)/gzip-12 6331 193922 ns/op 1007155 B/op 14 allocs/op BenchmarkCompressAllocations/level(9)/flate-12 6015 193829 ns/op 1006980 B/op 13 allocs/op BenchmarkCompressAllocations/level(9)/gzip-12 5728 197447 ns/op 1007155 B/op 14 allocs/op ``` * Reduce bits and history buffer.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Tables for levels 1-6 are quite big.
Investigate impact of 1) reducing precision and 2) don't store values along with the offset.
Reducing the precision will lose compression, but probably not too much when reduced by 1 bit.
Loading values may be offset by smaller tables, but it will add several more offset checks and a bounds check.
The impact of each change will reduce the size by half.
Impact of looking up values looks positive:
Seems like it could be an overall win, though higher levels may suffer a bit more.
Speed seems rather unaffected by less precision, making it simply less effective by reducing the size.
The text was updated successfully, but these errors were encountered: