std::unordered_map serializer causing huge memory explosion with empty container

[lifflander]

This code seems like it has a bug that is causing a huge number of unordered_map nodes to be allocated even though the target container is actually empty the entire time. It's causing a massive growth of hundreds of megabytes:

template <typename SerializerT, typename ContainerT>
inline void serializeUnorderedAssociativeContainer(
  SerializerT& s, ContainerT& cont
) {
  if (not s.isFootprinting()) {
    auto max_load_factor = cont.max_load_factor();
    s | max_load_factor;
    cont.max_load_factor(max_load_factor);

    auto bucket_count = cont.bucket_count();
    s | bucket_count;
    cont.rehash(bucket_count);
  }

  serializeMapLikeContainer(s, cont);
}

At the very least we should not be rehashing everything during sizing and packing! That has linear complexity.

[PhilMiller]

Does it work to stick a if (s.isUnpacking()) before the rehash call?

[lifflander]

We can probably start by writing a unit test (@cz4rs) to try to reproduce this problem locally. I think we pushed the latest checkpoint code out to EMPIRE so we need to fix that ASAP.

[lifflander]

Does it work to stick a if (s.isUnpacking()) before the rehash call?

I haven't tried that... I'm just removing the code altogether to ensure all the memory increase is due to this.

[cz4rs]

looking into it right now, I'll try isUnpacking after setting up the test

[lifflander]

With those lines completely commented out the memory usage looks like this:

[cz4rs]

@lifflander @PhilMiller I have creted a PR trying to call rehash only when necessary (s.isUnpacking() only cuts the calls number roughly in half).