Serialization and deserialization of hash-based collections should not re-use hashCode

[scabug]

As explained on ticket #1387, the fact that the hashCode for tuples and case classes depends on the VM (they use getClass().hashCode()) means that the serialization and deserialization of immutable.HashSet (and probably other hash-based collections, but I have not verified) breaks.

Although I think it's worth fixing tuples and case classes, this issue is about changing hash-based collections so that serialization and deserialization work correctly even if the hashCode relies on some VM-specific computation.

The easiest way to do that is similar to how java.util.HashMap does it by serialising the objects into an array and then inserting them one by one into the Map on deserialisation. This means that the hashCodes are recomputed on deserialisation so there are no issues anymore.

[scabug]

Imported From: https://issues.scala-lang.org/browse/SI-1600?orig=1
Reporter: @ijuma
Attachments:

• t1600.diff (created on Nov 30, 2009 7:04:50 PM UTC, 20566 bytes)

[scabug]

@odersky said:
I don't want to fix this myself. Any other takers?

[scabug]

@ijuma said:
I could supply a patch, but there are two questions:

• What are the backwards compatibility requirements?
• Is it worth doing this now, or better to do it for the new collections in 2.8.0?

[scabug]

@odersky said:
I think it's better to wait for 2.8.0, which will have looser backwards requirements as well. I'd happily integrate a patch then!

[scabug]

@ijuma said:
I committed this to a github branch:

http://github.com/ijuma/scala/commit/b7b0459ae872701a9486d7692b76a353b02cbfbc

I will also attach a patch.

Commit comment follows for convenience:

Fix ticket #1600: Serialization and deserialization of hash-based collections should not re-use hashCode.

Implementation and tests are included for immutable.HashMap, mutable.HashMap, mutable.LinkedHashMap, immutable.HashSet, mutable.HashSet, immutable.LinkedHashSet.

The basic idea is that we store the size, load factor and elements during serialization and we rebuild the collection on deserialization. Note that this is not compatible with the previous serialization format. All @SerialVersionUIDs have been reset to 1.

I'm not particularly happy about the way _loadFactor is used in some cases, but that seemed like the lesser evil given the current interface.

WeakHashMap is not Serializable and it would not make sense to make it so. TreeHashMap has not been reintegrated yet. OpenHashMap has not been updated. I think that collection is flawed in many ways and it should either be removed or it should be reimplemented. I'll file a separate ticket for that.

Some tests in serialization.check were updated as the toString order after serialization can be different in some cases.

ant all.clean && ant dist finished successfully.

[scabug]

@ijuma said:
Sorry, the github link should be:

http://github.com/ijuma/scala/commit/5627e4b40a6cc1f086c6a43c38a6bb2da2f86ddd

[scabug]

@ijuma said:
Implementation and tests for this issue.

[scabug]

@ijuma said:
Rebased the patch and github branch and improved documentation:

http://github.com/ijuma/scala/commit/733eec09d25c0580ef65a32945b053da6c2d87e8

Assigning to scala_reviewer as suggested by Toni.

[scabug]

@adriaanm said:
Thanks for the patch!

[scabug]

@adriaanm said:
Paul, could you please shepherd this?

[scabug]

@ijuma said:
Paul committed this in r19964. Thanks, closing.