-
Notifications
You must be signed in to change notification settings - Fork 51
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Slowdown when inserting elements in the iteration order of another FxHashMap #45
Comments
The problem is that the new hash is completely linear with the seed, which still runs into the accidentally-quadratic probing issue. |
Because of the deterministic property of `FxHasher` in rustc-hash crate, the performance will be extremely poor when there are iteration + reinsertion operations. A more thorough explanation can be found in the articles below: - https://accidentallyquadratic.tumblr.com/post/153545455987/rust-hash-iteration-reinsertion - https://morestina.net/blog/1843/the-stable-hashmap-trap And related issue in rustc-hash: - rust-lang/rustc-hash#45
Is there any hope that this issue will be fixed in the near term? If not then I think a warning should be added to the readme and/or documentation. This issue makes the current hashing algorithm a potential source of extreme slowdown in any use case where hahsmaps are serialized and deserialized. |
It seems that in 2.0.0 there has been a regression in the fix to #14.
In 1.2.0 it was possible to avoid the issue by using different seeds for different FxHashMaps (by manually setting the seed or using a randomized seed). This no longer works in 2.0.0. Even if you use two FxHashMaps/FxHashSets with different seeds you still get extremely bad performance when values are inserted in the iteration order of another FxHashMap/FxHashSet.
For example, try this:
On my computer this takes 0.3 seconds for the inital build, but 2.6 seconds for the rebuild with 2.0.0. With 1.2.0 this takes 0.3 seconds for both build and rebuild.
Note that FxHashSetRand and FxHashMapRand have the same problem, since no matter what seeds they generate you get more or less the same 8-9x slowdown.
The text was updated successfully, but these errors were encountered: