- A collection of 120+ RLE and related codecs optimized for all kinds of different inputs and scenarios.
- Usually the fastest run length en/decoder by far. Single Core Decompression Speeds > 34 GB/s and Compression Speeds > 28 GB/s have been observed with large files. (small files can exceed 120 GB/s decode, 60 GB/s encode)
- Written in C.
- SIMD Variants for AVX-512F, AVX2, AVX, SSE4.1, SSSE3 and SSE2 variants are available for various decoders and encoders. Automatically picked at runtime based on the extensions available on the current platform.
- Variants include: Single RLE Symbol, Short Strings of RLE Symbols, Byte Alignmed, Symbol Aligned, 8 Bit, 16 Bit, 24 Bit, 32 Bit, 48 Bit, 64 Bit, 128 Bit, Different probabilities of reoccuring symbols, ...
OpenCL
variant available for some of the decoders.- Licensed under Two Clause BSD.
See Full Benchmark with Graphs, the tables below only contain a tiny selection of the 100+ codecs.
- Single-Threaded
- Running on an
AMD Ryzen 9 7950X
,32 GB DDR5-6000 CL30
on Windows 11. - Compiled with
Visual Studio 2022
. - Compared to TurboRLE and MRLE (with AVX2 enabled; benchmarking results have been converted from MB/s to MiB/s) (7f6e486, April 2023)
- Contained Codecs w/ Compression / Decompression Speed vs. Ratio Pareto + Notable Entropy Highlighted
video-frame.raw (heavily quantized video frame DCTs, 88,473,600 Bytes)
Type | Ratio | Encoding Speed | Decoding Speed | Entropy Compressible To |
---|---|---|---|---|
8 Bit | 19.34 % | 3,319.6 MiB/s | 25,830.4 MiB/s | 9.50 % |
8 Bit Packed | 17.95 % | 2,907.5 MiB/s | 19,783.6 MiB/s | 9.69 % |
8 Bit 1LUT Short | 16.93 % | 2,365.1 MiB/s | 11,069.1 MiB/s | 9.87 % |
8 Bit 3LUT | 17.41 % | 2,146.7 MiB/s | 15,819.2 MiB/s | 9.80 % |
8 Bit 3LUT Short | 16.49 % | 2,085.6 MiB/s | 8,100.4 MiB/s | 9.93 % |
8 Bit 7LUT | 17.46 % | 2,094.6 MiB/s | 16,100.9 MiB/s | 9.94 % |
8 Bit Single | 18.31 % | 1,654.7 MiB/s | 23,750.2 MiB/s | 9.71 % |
8 Bit Single Short | 17.05 % | 1,414.4 MiB/s | 14,143.6 MiB/s | 9.81 % |
16 Bit 1LUT Short (Symbol) | 18.23 % | 3,783.5 MiB/s | 13,705.5 MiB/s | 9.72 % |
16 Bit 1LUT Short (Byte) | 18.11 % | 3,745.3 MiB/s | 12,951.5 MiB/s | 9.90 % |
16 Bit 3LUT (Byte) | 17.74 % | 3,708.2 MiB/s | 15,577.4 MiB/s | 9.82 % |
16 Bit 7LUT (Byte) | 17.53 % | 3,526.7 MiB/s | 13,249.8 MiB/s | 9.96 % |
16 Bit 7LUT Short Greedy (Byte) | 15.37 % | 691.3 MiB/s | 3,075.2 MiB/s | 10.59 % |
24 Bit Packed (Byte) | 19.19 % | 6,890.9 MiB/s | 21,560.0 MiB/s | 9.57 % |
24 Bit 3LUT (Byte) | 18.52 % | 5,367.1 MiB/s | 18,636.5 MiB/s | 9.49 % |
24 Bit 7LUT (Byte) | 18.44 % | 5,196.5 MiB/s | 17,679.4 MiB/s | 9.59 % |
24 Bit 7LUT Short Greedy (Byte) | 15.46 % | 639.3 MiB/s | 3,375.4 MiB/s | 10.67 % |
32 Bit Packed (Byte) | 19.82 % | 8,946.4 MiB/s | 23,569.8 MiB/s | 9.46 % |
32 Bit 3LUT (Byte) | 19.25 % | 7,149.3 MiB/s | 23,580.3 MiB/s | 9.34 % |
32 Bit 7LUT (Byte) | 19.22 % | 7,066.1 MiB/s | 23,033.8 MiB/s | 9.45 % |
32 Bit 7LUT Short Greedy (Byte) | 15.09 % | 663.1 MiB/s | 3,298.1 MiB/s | 10.59 % |
48 Bit Packed (Byte) | 20.84 % | 10,911.9 MiB/s | 26,397.7 MiB/s | 9.40 % |
48 Bit 3LUT (Byte) | 20.41 % | 9,304.9 MiB/s | 25,603.9 MiB/s | 9.25 % |
48 Bit 3LUT Short Greedy (Byte) | 15.88 % | 931.3 MiB/s | 6,064.8 MiB/s | 10.04 % |
48 Bit 7LUT (Byte) | 20.42 % | 9,457.1 MiB/s | 25,088.5 MiB/s | 9.34 % |
64 Bit Packed (Symbol) | 22.34 % | 12,697.9 MiB/s | 26,375.4 MiB/s | 9.23 % |
64 Bit (Byte) | 23.66 % | 13,514.1 MiB/s | 25,773.6 MiB/s | 9.63 % |
64 Bit Packed (Byte) | 21.63 % | 13,306.5 MiB/s | 26,183.1 MiB/s | 9.40 % |
64 Bit 3LUT (Byte) | 21.30 % | 11,442.9 MiB/s | 27,864.0 MiB/s | 9.25 % |
64 Bit 3LUT Short Greedy (Byte) | 15.66 % | 905.2 MiB/s | 6,293.0 MiB/s | 9.95 % |
64 Bit 7LUT (Byte) | 21.32 % | 11,479.3 MiB/s | 26,879.6 MiB/s | 9.32 % |
8 Bit RLE + Huffman-esque | 12.51 % | 332.0 MiB/s | 1,676.7 MiB/s | 9.82 % |
Low Entropy | 19.93 % | 800.2 MiB/s | 2,690.6 MiB/s | 9.05 % |
Low Entropy Single | 19.96 % | 802.4 MiB/s | 4,389.0 MiB/s | 9.10 % |
- | - | - | - | - |
memcpy | 100.00 % | 28,667.8 MiB/s | 28,590.1 MiB/s | 14.03 % |
trle | 16.0 % | 1,177.7 MiB/s | 5,356.0 MiB/s | - |
srle 0 | 17.8 % | 1,182.3 MiB/s | 11,214.7 MiB/s | - |
srle 8 | 18.7 % | 2,044.3 MiB/s | 12,300.2 MiB/s | - |
srle 16 | 21.3 % | 1,997.0 MiB/s | 15,479.0 MiB/s | - |
srle 32 | 24.2 % | 3,782.2 MiB/s | 18,975.8 MiB/s | - |
srle 64 | 27.5 % | 6,581.4 MiB/s | 23,128.3 MiB/s | - |
mrle | 19.7 % | 450.3 MiB/s | 3,168.4 MiB/s | - |
1034.db (Checkers program "End Game Table Base", 419,225,625 Bytes)
Type | Ratio | Encoding Speed | Decoding Speed | Entropy Compressible To |
---|---|---|---|---|
8 Bit | 23.02 % | 2,443.8 MiB/s | 21,198.0 MiB/s | 12.08 % |
8 Bit Packed | 20.10 % | 2,199.4 MiB/s | 15,151.8 MiB/s | 12.74 % |
8 Bit 1LUT Short | 18.91 % | 1,767.7 MiB/s | 7,905.5 MiB/s | 12.46 % |
8 Bit 3LUT | 19.77 % | 1,841.7 MiB/s | 14,211.6 MiB/s | 12.31 % |
8 Bit 7LUT | 19.75 % | 1,768.7 MiB/s | 14,629.5 MiB/s | 12.50 % |
8 Bit Single | 20.59 % | 1,325.0 MiB/s | 18,469.7 MiB/s | 12.06 % |
8 Bit Single Short | 18.55 % | 1,157.3 MiB/s | 9,756.9 MiB/s | 12.09 % |
16 Bit Packed (Byte) | 20.65 % | 3,378.7 MiB/s | 15,516.6 MiB/s | 12.59 % |
16 Bit 3LUT (Byte) | 20.20 % | 2,641.6 MiB/s | 15,500.6 MiB/s | 12.30 % |
16 Bit 7LUT (Byte) | 20.14 % | 2,708.0 MiB/s | 13,971.5 MiB/s | 12.35 % |
24 Bit Packed (Byte) | 22.05 % | 4,233.2 MiB/s | 17,670.5 MiB/s | 12.34 % |
24 Bit 3LUT (Byte) | 21.52 % | 3,448.9 MiB/s | 16,464.5 MiB/s | 11.96 % |
32 Bit Packed (Byte) | 23.44 % | 5,240.1 MiB/s | 17,864.4 MiB/s | 12.15 % |
32 Bit 3LUT (Byte) | 22.86 % | 4,289.1 MiB/s | 20,829.1 MiB/s | 11.79 % |
32 Bit 7LUT (Byte) | 22.89 % | 4,294.6 MiB/s | 20,065.2 MiB/s | 11.86 % |
48 Bit Packed (Byte) | 25.87 % | 6,516.7 MiB/s | 22,210.4 MiB/s | 11.95 % |
48 Bit 3LUT (Byte) | 25.41 % | 5,712.4 MiB/s | 20,904.4 MiB/s | 11.66 % |
64 Bit 3LUT (Symbol) | 28.66 % | 6,464.5 MiB/s | 23,401.9 MiB/s | 11.56 % |
64 Bit (Byte) | 31.66 % | 8,034.7 MiB/s | 21,153.2 MiB/s | 12.16 % |
64 Bit Packed (Byte) | 27.29 % | 7,836.3 MiB/s | 22,219.0 MiB/s | 11.90 % |
64 Bit 3LUT (Byte) | 26.90 % | 6,780.3 MiB/s | 23,777.1 MiB/s | 11.63 % |
64 Bit 7LUT (Byte) | 27.05 % | 6,839.2 MiB/s | 22,970.8 MiB/s | 11.76 % |
128 Bit (Byte) | 37.25 % | 6,763.4 MiB/s | 24,011.4 MiB/s | 12.28 % |
128 Bit Packed (Byte) | 31.21 % | 6,582.9 MiB/s | 23,994.6 MiB/s | 11.98 % |
8 Bit RLE + Huffman-esque | 16.76 % | 326.5 MiB/s | 1,456.9 MiB/s | 12.36 % |
Low Entropy | 21.15 % | 799.7 MiB/s | 4,282.1 MiB/s | 10.37 % |
Low Entropy Single | 21.15 % | 765.9 MiB/s | 4,292.2 MiB/s | 10.37 % |
- | - | - | - | - |
memcpy | 100.00 % | 26,689.1 MiB/s | 27,304.6 MiB/s | 16.76 % |
trle | 17.4 % | 1,112.8 MiB/s | 4,097.4 MiB/s | - |
srle 0 | 20.2 % | 1,080.3 MiB/s | 8,027.7 MiB/s | - |
srle 8 | 22.0 % | 1,816.5 MiB/s | 8,802.3 MiB/s | - |
srle 16 | 27.1 % | 1,644.2 MiB/s | 9,915.1 MiB/s | - |
srle 32 | 32.7 % | 2,870.4 MiB/s | 13,289.7 MiB/s | - |
srle 64 | 39.5 % | 4,763.9 MiB/s | 18,966.4 MiB/s | - |
mrle | 21.0 % | 446.9 MiB/s | 3,115.4 MiB/s | - |
enwik9.bwt
(Wikipedia extract enwiki9 encoded using libdivsufsort, 1,000,000,124 Bytes)
Type | Ratio | Encoding Speed | Decoding Speed | Entropy Compressible To |
---|---|---|---|---|
8 Bit | 48.80 % | 1,225.6 MiB/s | 16,097.6 MiB/s | 34.13 % |
8 Bit Short | 44.42 % | 899.3 MiB/s | 6,975.0 MiB/s | 32.26 % |
8 Bit Packed | 44.86 % | 1,038.7 MiB/s | 7,514.9 MiB/s | 32.95 % |
8 Bit 1LUT Short | 41.87 % | 828.2 MiB/s | 4,088.0 MiB/s | 31.38 % |
8 Bit 3LUT | 43.29 % | 775.1 MiB/s | 6,116.6 MiB/s | 32.27 % |
8 Bit 3LUT Short | 40.34 % | 765.3 MiB/s | 3,046.5 MiB/s | 30.63 % |
8 Bit 7LUT | 42.98 % | 720.4 MiB/s | 4,845.2 MiB/s | 32.42 % |
16 Bit Packed (Symbol) | 47.89 % | 1,871.4 MiB/s | 9,417.5 MiB/s | 34.25 % |
16 Bit 1LUT Short (Symbol) | 46.43 % | 1,425.4 MiB/s | 5,926.8 MiB/s | 33.47 % |
16 Bit Packed (Byte) | 46.99 % | 1,975.6 MiB/s | 8,322.4 MiB/s | 33.96 % |
16 Bit 1LUT Short (Byte) | 46.15 % | 1,404.6 MiB/s | 5,410.5 MiB/s | 33.47 % |
16 Bit 3LUT (Byte) | 44.91 % | 1,332.9 MiB/s | 5,786.9 MiB/s | 33.36 % |
16 Bit 7LUT (Byte) | 44.17 % | 1,269.3 MiB/s | 4,792.9 MiB/s | 33.00 % |
24 Bit Packed (Byte) | 49.39 % | 2,846.0 MiB/s | 10,096.3 MiB/s | 35.00 % |
24 Bit 3LUT (Byte) | 47.63 % | 2,154.2 MiB/s | 8,316.6 MiB/s | 34.23 % |
24 Bit 7LUT (Byte) | 47.15 % | 2,021.5 MiB/s | 7,081.2 MiB/s | 33.86 % |
32 Bit Packed (Byte) | 51.63 % | 3,991.3 MiB/s | 12,316.8 MiB/s | 35.94 % |
32 Bit 3LUT (Byte) | 50.02 % | 3,049.7 MiB/s | 11,014.7 MiB/s | 35.15 % |
32 Bit 7LUT (Byte) | 49.72 % | 2,929.3 MiB/s | 9,734.1 MiB/s | 34.97 % |
48 Bit (Byte) | 57.58 % | 5,568.3 MiB/s | 17,036.4 MiB/s | 38.86 % |
48 Bit Packed (Byte) | 55.01 % | 5,463.9 MiB/s | 16,106.6 MiB/s | 37.48 % |
48 Bit 3LUT (Byte) | 53.66 % | 4,511.6 MiB/s | 14,552.7 MiB/s | 36.78 % |
48 Bit 7LUT (Byte) | 53.48 % | 4,362.4 MiB/s | 13,340.3 MiB/s | 36.64 % |
64 Bit Packed (Symbol) | 58.87 % | 6,995.2 MiB/s | 17,252.6 MiB/s | 39.42 % |
64 Bit (Byte) | 59.94 % | 7,594.6 MiB/s | 17,821.4 MiB/s | 40.05 % |
64 Bit Packed (Byte) | 57.33 % | 7,285.0 MiB/s | 16,557.0 MiB/s | 38.65 % |
64 Bit 1LUT Short (Byte) | 57.24 % | 6,176.4 MiB/s | 16,263.8 MiB/s | 38.60 % |
64 Bit 3LUT (Byte) | 56.28 % | 6,050.7 MiB/s | 17,019.3 MiB/s | 38.04 % |
64 Bit 7LUT (Byte) | 56.10 % | 5,933.0 MiB/s | 16,061.7 MiB/s | 37.92 % |
128 Bit Packed (Symbol) | 64.32 % | 5,640.8 MiB/s | 20,237.7 MiB/s | 42.35 % |
128 Bit Packed (Byte) | 62.94 % | 5,650.8 MiB/s | 20,158.4 MiB/s | 41.61 % |
8 Bit MMTF 128 | 63.49 % | 1,101.2 MiB/s | 1,299.9 MiB/s | 29.84 % |
Low Entropy | 64.19 % | 383.3 MiB/s | 913.9 MiB/s | 34.06 % |
Low Entropy Single | 89.22 % | 561.6 MiB/s | 7,491.0 MiB/s | 59.50 % |
Low Entropy Short | 66.17 % | 394.1 MiB/s | 1,412.4 MiB/s | 36.11 % |
Low Entropy Short Single | 89.65 % | 576.5 MiB/s | 9,841.2 MiB/s | 61.05 % |
Multi MTF 128 Bit (Transform) | 100.00 % | 1,206.2 MiB/s | 1,289.8 MiB/s | 33.92 % |
Multi MTF 256 Bit (Transform) | 100.00 % | 2,012.2 MiB/s | 1,966.1 MiB/s | 35.65 % |
Bit MMTF 8 Bit (Transform) | 100.00 % | 1,843.2 MiB/s | 1,973.5 MiB/s | 35.87 % |
Bit MMTF 16 Bit (Transform) | 100.00 % | 2,309.8 MiB/s | 2,272.9 MiB/s | 38.34 % |
- | - | - | - | - |
memcpy | 100.00 % | 26,973.3 MiB/s | 27,041.2 MiB/s | 65.94 % |
trle | 42.0 % | 697.6 MiB/s | 2,062.9 MiB/s | - |
srle 0 | 46.4 % | 680.6 MiB/s | 5,004.5 MiB/s | - |
srle 8 | 46.9 % | 858.6 MiB/s | 6,095.1 MiB/s | - |
srle 16 | 54.4 % | 906.4 MiB/s | 8,676.0 MiB/s | - |
srle 32 | 61.0 % | 1,766.0 MiB/s | 13,019.7 MiB/s | - |
srle 64 | 66.8 % | 3,289.0 MiB/s | 18,491.6 MiB/s | - |
mrle | 64.1 % | 320.2 MiB/s | 1,104.0 MiB/s | - |
The 24 Bit and 48 Bit Variants allow for run length encoding of common data layouts that are usually not covered by RLE implementations:
Pixel Art Bitmap Image (PNG converted to BMP, 123,710,454 Bytes)
Type | Ratio | Encoding Speed | Decoding Speed | Entropy Compressible To |
---|---|---|---|---|
8 Bit | 99.99 % | 6,995.3 MiB/s | 18,357.0 MiB/s | 76.10 % |
8 Bit 1LUT Short | 99.98 % | 6,974.3 MiB/s | 18,023.2 MiB/s | 75.63 % |
8 Bit 3LUT | 99.99 % | 6,936.1 MiB/s | 18,595.7 MiB/s | 75.94 % |
8 Bit Single | 100.00 % | 3,512.8 MiB/s | 18,470.1 MiB/s | 77.03 % |
16 Bit Short (Byte) | 99.99 % | 18,127.2 MiB/s | 18,953.0 MiB/s | 75.56 % |
16 Bit 3LUT Short (Byte) | 99.99 % | 17,991.9 MiB/s | 19,252.9 MiB/s | 75.71 % |
24 Bit (Symbol) | 1.84 % | 24,537.1 MiB/s | 33,067.5 MiB/s | 1.32 % |
24 Bit Short (Symbol) | 2.08 % | 21,904.2 MiB/s | 32,713.6 MiB/s | 1.52 % |
24 Bit Packed (Symbol) | 2.00 % | 23,590.4 MiB/s | 32,955.6 MiB/s | 1.39 % |
24 Bit 1LUT Short (Symbol) | 2.08 % | 19,894.5 MiB/s | 32,751.7 MiB/s | 1.51 % |
24 Bit 3LUT (Symbol) | 1.31 % | 21,232.5 MiB/s | 33,073.5 MiB/s | 0.93 % |
24 Bit 3LUT Short (Symbol) | 1.53 % | 19,147.7 MiB/s | 32,890.5 MiB/s | 1.04 % |
24 Bit 7LUT (Symbol) | 1.20 % | 20,599.0 MiB/s | 32,846.9 MiB/s | 0.83 % |
24 Bit 7LUT Short (Symbol) | 1.41 % | 18,916.5 MiB/s | 31,750.2 MiB/s | 0.92 % |
24 Bit (Byte) | 2.13 % | 25,939.7 MiB/s | 33,113.4 MiB/s | 1.44 % |
24 Bit Short (Byte) | 2.17 % | 23,579.9 MiB/s | 32,825.6 MiB/s | 1.58 % |
24 Bit Packed (Byte) | 2.32 % | 26,738.2 MiB/s | 32,812.0 MiB/s | 1.51 % |
24 Bit 1LUT Short (Byte) | 2.17 % | 21,550.6 MiB/s | 32,835.9 MiB/s | 1.59 % |
24 Bit 1LUT Short Greedy (Byte) | 2.17 % | 5,534.5 MiB/s | 32,679.0 MiB/s | 1.59 % |
24 Bit 3LUT (Byte) | 1.49 % | 23,408.0 MiB/s | 33,010.6 MiB/s | 1.03 % |
24 Bit 3LUT Short (Byte) | 1.63 % | 21,063.2 MiB/s | 32,713.6 MiB/s | 1.14 % |
24 Bit 3LUT Short Greedy (Byte) | 1.63 % | 5,403.1 MiB/s | 32,578.9 MiB/s | 1.14 % |
24 Bit 7LUT (Byte) | 1.37 % | 22,589.6 MiB/s | 32,629.3 MiB/s | 0.98 % |
24 Bit 7LUT Short (Byte) | 1.52 % | 20,560.6 MiB/s | 31,915.9 MiB/s | 0.97 % |
24 Bit 7LUT Short Greedy (Byte) | 1.52 % | 5,436.8 MiB/s | 32,009.5 MiB/s | 0.97 % |
32 Bit Packed (Byte) | 99.99 % | 16,961.2 MiB/s | 19,048.9 MiB/s | 75.71 % |
32 Bit 7LUT Short (Byte) | 99.99 % | 16,887.8 MiB/s | 19,240.9 MiB/s | 75.79 % |
48 Bit (Symbol) | 2.78 % | 23,645.9 MiB/s | 32,870.0 MiB/s | 2.12 % |
48 Bit Short (Symbol) | 2.79 % | 21,386.8 MiB/s | 32,833.3 MiB/s | 2.26 % |
48 Bit Packed (Symbol) | 2.87 % | 22,575.5 MiB/s | 32,913.6 MiB/s | 2.17 % |
48 Bit 1LUT Short (Symbol) | 3.09 % | 19,362.4 MiB/s | 32,750.9 MiB/s | 2.41 % |
48 Bit 3LUT (Symbol) | 1.71 % | 20,826.5 MiB/s | 32,843.5 MiB/s | 1.33 % |
48 Bit 3LUT Short (Symbol) | 1.99 % | 18,834.0 MiB/s | 32,717.0 MiB/s | 1.47 % |
48 Bit 7LUT (Symbol) | 1.49 % | 20,185.7 MiB/s | 32,619.2 MiB/s | 1.15 % |
48 Bit 7LUT Short (Symbol) | 1.77 % | 18,184.3 MiB/s | 31,846.0 MiB/s | 1.26 % |
48 Bit (Byte) | 3.16 % | 25,473.5 MiB/s | 32,911.9 MiB/s | 2.31 % |
48 Bit Short (Byte) | 3.20 % | 23,439.2 MiB/s | 32,689.1 MiB/s | 2.48 % |
48 Bit Packed (Byte) | 3.35 % | 25,371.9 MiB/s | 32,548.7 MiB/s | 2.40 % |
48 Bit 1LUT Short (Byte) | 3.20 % | 20,935.0 MiB/s | 32,603.2 MiB/s | 2.48 % |
48 Bit 1LUT Short Greedy (Byte) | 3.20 % | 9,723.8 MiB/s | 32,490.3 MiB/s | 2.48 % |
48 Bit 3LUT (Byte) | 1.98 % | 22,790.5 MiB/s | 32,861.4 MiB/s | 1.49 % |
48 Bit 3LUT Short (Byte) | 2.13 % | 20,273.7 MiB/s | 32,717.9 MiB/s | 1.62 % |
48 Bit 3LUT Short Greedy (Byte) | 2.13 % | 8,178.3 MiB/s | 32,537.9 MiB/s | 1.62 % |
48 Bit 7LUT (Byte) | 1.76 % | 21,776.5 MiB/s | 32,615.9 MiB/s | 1.34 % |
48 Bit 7LUT Short (Byte) | 1.90 % | 20,145.3 MiB/s | 31,847.6 MiB/s | 1.35 % |
48 Bit 7LUT Short Greedy (Byte) | 1.90 % | 7,895.9 MiB/s | 31,782.1 MiB/s | 1.35 % |
64 Bit (Symbol) | 99.99 % | 14,820.8 MiB/s | 19,040.6 MiB/s | 75.87 % |
64 Bit 7LUT (Byte) | 99.99 % | 14,692.8 MiB/s | 19,217.2 MiB/s | 75.78 % |
8 Bit MMTF 128 | 17.70 % | 5,287.0 MiB/s | 2,978.1 MiB/s | 12.31 % |
Multi MTF 128 Bit (Transform) | 100.00 % | 3,474.5 MiB/s | 3,482.1 MiB/s | 19.94 % |
Multi MTF 256 Bit (Transform) | 100.00 % | 5,030.6 MiB/s | 5,028.6 MiB/s | 27.52 % |
Bit MMTF 8 Bit (Transform) | 100.00 % | 1,861.6 MiB/s | 1,984.2 MiB/s | 73.85 % |
Bit MMTF 16 Bit (Transform) | 100.00 % | 2,340.1 MiB/s | 2,293.9 MiB/s | 72.48 % |
- | - | - | - | - |
memcpy | 100.00 % | 28,288.5 MiB/s | 28,261.3 MiB/s | 77.03 % |
trle | 100.0 % | 420.41 MiB/s | 5,473.00 MiB/s | - |
srle 0 | 100.0 % | 329.09 MiB/s | 18,106.88 MiB/s | - |
srle 8 | 100.0 % | 468.74 MiB/s | 17,951.84 MiB/s | - |
srle 16 | 100.0 % | 575.39 MiB/s | 20,198.96 MiB/s | - |
srle 32 | 100.0 % | 1,195.77 MiB/s | 18,437.51 MiB/s | - |
srle 64 | 100.0 % | 2,294.94 MiB/s | 19,407.80 MiB/s | - |
mrle | 100.0 % | 383.18 MiB/s | 2,551.12 MiB/s | - |
- Extremely Fast.
- Variants for always aligning with the symbol width or allowing byte-wide repeats even for > 8 bit symbols.
- Decoder interprets blocks of data to boil down to a highly optimized
memcpy
,memset
,memcpy
,memset
(with various different byte-lengths). - Encoder searches for repeats and their respective lengths using movemask instructions.
- Optional variant for 8 bit single symbol encoding.
- Similar to the base variant, but keeps around the previously used RLE symbol which is usually very beneficial to the compression ratio and tries to pack lengths a bit more optimistically.
- Also Extremely Fast, whilst providing better compression ratio for many inputs
- Also has those variants for always aligning with the symbol width or allowing byte-wide repeats even for > 8 bit symbols.
- Optional variant for 8 bit single symbol encoding (only has the optimistic packing, as it already knows what the next symbol is going to be anyways).
- Similar to the base variant, but keeps around one / three / seven of the previously used RLE symbols, usually further improving compression ratios.
- Short Variant: Packs Range & Count Bits to fit the entire RLE command into just one byte for short ranges.
- Similar to n-Sym LUT Short (actually uses the exact same decoder), but tries to even compress partial occurences of a symbol if that would save space
- Only available when byte-aligned for 16, 24, 32, 48, 64 Bit Variants
- Comparably slow to en/decode
- Tries to keep symbol general symbol frequency to improve compression ratio of an entropy encoder that could go after the Run Length Encoding like ANS, Arithmetic Coding or Huffman.
- Parses the output for run-length-encodable symbols, which are specified in the header.
- Has a single-symbol variant, that only encodes the most run-length-encodable symbol (useful for some image codecs).
- Short Variant: Same as Low Entropy, but optimized for shorter strings of run-length-encodable symbols, usually faster, also has a single-symbol variant.
- Runs a block-wide vectorized MTF transform on the input and depending on how many bits this needs to represent a given block (if it's not entirely representable by a variant of
memset
) uses only the required amount of bits to encode the block. - Performs well on a wide variety of inputs, but usually doesn't produce the best compression ratios or (de-) compression speeds, as all blocks need to be decoded and cannot simply be
memcpy
d if they don't contain an encodable symbol.
- Uses a separate header, that contains a huffman-esque instructions to place recent high-prevalence symbols, copy or set a specific symbol to a block.
- Usually very high compression ratios, but comparably slow to en- & decode.
- Simply runs the block-wide vectorized MTF transform on the input to improve compressability for some scenarios.
- Doesn't compress itself.
git clone https://github.com/rainerzufalldererste/hypersonic-rle-kit.git
cd hypersonic-rle-kit
git submodule update --init --recursive
create_project.bat
Choose your preferred compiler toolset
MSBuild /p:Configuration=Release /nologo /v:m
premake/premake5 gmake2
config=release_x64 make
#include "rle.h"
uint8_t *pUncompressedData; // Some Data.
uint32_t fileSize; // Some Size.
// Get Compress Bounds.
const uint32_t compressedBufferSize = rle_compress_bounds(fileSize);
uint8_t *pCompressedData = (uint8_t *)malloc(compressedBufferSize);
// Compress.
const uint32_t compressedSize = rle8_multi_compress(pUncompressedData, fileSize, pCompressedData, compressedBufferSize);
// Allocate Output Buffer.
uint8_t *pDecompressedData = (uint8_t *)malloc(fileSize + rle_decompress_additional_size());
// Decompress.
const uint32_t decompressedSize = rle8_decompress(pCompressedData, compressedSize, pDecompressedData, fileSize);
// Cleanup.
free(pCompressedData);
free(pDecompressedData);
Two Clause BSD