Fix mutex lock error in logging utils

[jparismorgan]

What

Right now when we run C++ unit tests we are getting this error in logs: libc++abi: terminating due to uncaught exception of type std::__1::system_error: mutex lock failed: Invalid argument, even though the test still passes.

This is coming from std::lock_guard<std::mutex> lock(mtx_); in memory_data

src/include/utils/logging.h

class count_data {
  ...
  void insert_entry(const std::string& name, const count_type& use) {
    std::cout << "[count_data@insert_entry]" << std::endl;
    std::lock_guard<std::mutex> lock(mtx_);
    count_usages_.insert(std::make_pair(name, use));
  }
...

The root cause is because of the _count_data.insert_entry(msg_ + " num_ss_comps", num_comps_); call in the destructor of logging_sum_of_squares_distance object.

src/include/scoring.h

struct logging_sum_of_squares_distance {
  size_t num_comps_{0};
  std::string msg_;

  logging_sum_of_squares_distance() = default;
  explicit logging_sum_of_squares_distance(const std::string& msg)
      : msg_(msg) {
        std::cout << "[logging_sum_of_squares_distance@ctor]" << std::endl;
  }

  template <class V, class U>
  constexpr auto operator()(const V& a, const U& b) {
    ++num_comps_;
    return unroll4_sum_of_squares(a, b);
  }

  void reset() {
    num_comps_ = 0;
  }

  ~logging_sum_of_squares_distance() {
    std::cout << "[logging_sum_of_squares_distance@dtor]" << std::endl;
    _count_data.insert_entry(msg_ + " num_ss_comps", num_comps_);
  }
};

}  // namespace _l2_distance

using logging_sum_of_squares_distance =
    _l2_distance::logging_sum_of_squares_distance;
inline auto logging_l2_distance =
    _l2_distance::logging_sum_of_squares_distance{};

I believe the root cause is because of the static initialization order fiasco. Though I'm not exactly sure of the reasoning, I believe the count_data object is being initialized before the mtx_ mutex object.

One particularly strange thing is that when I log out all the calls, I don't see [logging_sum_of_squares_distance@dtor] or [count_data@insert_entry] being printed. But if I comment out _count_data.insert_entry(msg_ + " num_ss_comps", num_comps_); or std::lock_guard<std::mutex> lock(mtx_); then the error goes away.

Regardless, the fix we make in this PR is to use another approach to initialize the get_instance() function. Before we used the Meyers Singleton pattern, but now we change to a new thread-safe lazy initialization pattern, which fixes this error and still passes tests.

Testing

• Existing tests pass.
• I don't see libc++abi: terminating due to uncaught exception of type std::__1::system_error: mutex lock failed: Invalid argument anymore.