Improved hashing algorithm

[syrusakbary]

Based on this feedback:
https://twitter.com/jedisct1/status/1100200484764303361

Speed analysis

These are the speeds when hashing a 1Mb file (nginx):

• meowhash hashes in 50ns (1Mb file) - 1x
• blake2bp (SIMD version) hashes in 500ns - 10x
• blake2 (non-SIMD version) hashes in 1ms - 20x
• sha256 (previous implementation) hashes in 5ms - 100x

blake2

nginx HASH              time:   [1.0330 ms 1.0471 ms 1.0589 ms]
                        change: [-3.3836% -0.6012% +2.1520%] (p = 0.69 > 0.05)

blake2bp

nginx HASH              time:   [510.72 us 526.18 us 541.60 us]
                        change: [-6.8755% -1.4050% +3.7167%] (p = 0.66 > 0.05)

meowhash

nginx HASH              time:   [47.033 us 48.020 us 49.028 us]
                        change: [-95.542% -95.334% -95.097%] (p = 0.00 < 0.05)

Other resources

Collision analysis: cmuratori/meow_hash#7 (no collisions in a 156Gb dataset for 2M files, in the non-truncated version).

Hacker News article: https://news.ycombinator.com/item?id=18262627

[jedisct1]

You're using blake2, not blake2bp here. See https://docs.rs/blake2b_simd/0.4.1/blake2b_simd/blake2bp/index.html

[syrusakbary]

@jedisct1 thanks for the ping! I'll update the code and benchmark soon.

[syrusakbary]

After researching more we decided to merge this since the timings with blake2bp (500ns for a 1Mb file) are good enough for our use case.

Thanks @jedisct1 and @briansmith for bringing this to our attention.