[Core] Ray fails to build due to spdlog char8_t keyword issue

[peytondmurray]

What happened + What you expected to happen

The most recent commit on master (cc3fa33) fails to build with the following error:

[2,451 / 4,089] Compiling src/spdlog.cpp; 5s linux-sandbox ... (24 actions, 5 running)
    ERROR: /home/pdmurray/Desktop/workspace/ray/BUILD.bazel:2397:11: Compiling src/ray/util/logging.cc failed: (Exit 1): gcc failed: error executing command
      (cd /home/pdmurray/.cache/bazel/_bazel_pdmurray/fd1a49e9b70d1b9b9f23ee71905db765/sandbox/linux-sandbox/1505/execroot/com_github_ray_project_ray && \
      exec env - \
        PATH=/home/pdmurray/.cache/bazelisk/downloads/bazelbuild/bazel-5.4.1-linux-x86_64/bin:/home/pdmurray/.pyenv/versions/ray/bin:/usr/share/pyenv/libexec:/usr/share/pyenv/plugins/python-build/bin:/usr/share/pyenv/plugins/pyenv-virtualenv/bin:/home/pdmurray/perl5/bin:/home/pdmurray/.pyenv/shims:/home/pdmurray/.pyenv/bin:/home/pdmurray/.luarocks/bin:/usr/lib/emscripten:/home/pdmurray/.config/scripts:/home/pdmurray/.cargo/bin:/home/pdmurray/bin:/home/pdmurray/go/bin:/home/pdmurray/.volta/bin:/usr/local/sbin:/usr/local/bin:/usr/bin:/usr/lib/jvm/default/bin:/usr/bin/site_perl:/usr/bin/vendor_perl:/usr/bin/core_perl:/usr/lib/rustup/bin \
        PWD=/proc/self/cwd \
      /usr/bin/gcc -U_FORTIFY_SOURCE -fstack-protector -Wall -Wunused-but-set-parameter -Wno-free-nonheap-object -fno-omit-frame-pointer -g0 -O2 '-D_FORTIFY_SOURCE=1' -DNDEBUG -ffunction-sections -fdata-sections '-std=c++0x' -MD -MF bazel-out/k8-opt/bin/_objs/ray_util/logging.pic.d '-frandom-seed=bazel-out/k8-opt/bin/_objs/ray_util/logging.pic.o' -fPIC -DBOOST_NO_CXX98_FUNCTION_BASE -DBOOST_ASIO_SEPARATE_COMPILATION -iquote . -iquote bazel-out/k8-opt/bin -iquote external/com_google_protobuf -iquote bazel-out/k8-opt/bin/external/com_google_protobuf -iquote external/zlib -iquote bazel-out/k8-opt/bin/external/zlib -iquote external/boost -iquote bazel-out/k8-opt/bin/external/boost -iquote external/com_github_spdlog -iquote bazel-out/k8-opt/bin/external/com_github_spdlog -iquote external/com_google_absl -iquote bazel-out/k8-opt/bin/external/com_google_absl -iquote external/com_google_googletest -iquote bazel-out/k8-opt/bin/external/com_google_googletest -iquote external/nlohmann_json -iquote bazel-out/k8-opt/bin/external/nlohmann_json -Ibazel-out/k8-opt/bin/_virtual_includes/ray_util -Ibazel-out/k8-opt/bin/_virtual_includes/aligned_alloc -Ibazel-out/k8-opt/bin/_virtual_includes/sha256 -Ibazel-out/k8-opt/bin/external/com_github_spdlog/_virtual_includes/spdlog -isystem external/com_google_protobuf/src -isystem bazel-out/k8-opt/bin/external/com_google_protobuf/src -isystem external/zlib -isystem bazel-out/k8-opt/bin/external/zlib -isystem external/boost -isystem bazel-out/k8-opt/bin/external/boost -isystem external/boost/libs/math/include_private -isystem bazel-out/k8-opt/bin/external/boost/libs/math/include_private -isystem external/com_github_spdlog -isystem bazel-out/k8-opt/bin/external/com_github_spdlog -isystem external/com_github_spdlog/include -isystem bazel-out/k8-opt/bin/external/com_github_spdlog/include -isystem external/com_google_googletest/googlemock -isystem bazel-out/k8-opt/bin/external/com_google_googletest/googlemock -isystem external/com_google_googletest/googlemock/include -isystem bazel-out/k8-opt/bin/external/com_google_googletest/googlemock/include -isystem external/com_google_googletest/googletest -isystem bazel-out/k8-opt/bin/external/com_google_googletest/googletest -isystem external/com_google_googletest/googletest/include -isystem bazel-out/k8-opt/bin/external/com_google_googletest/googletest/include -isystem external/nlohmann_json/single_include -isystem bazel-out/k8-opt/bin/external/nlohmann_json/single_include '-std=c++17' -DBAZEL_OPT -Werror -fno-canonical-system-headers -Wno-builtin-macro-redefined '-D__DATE__="redacted"' '-D__TIMESTAMP__="redacted"' '-D__TIME__="redacted"' -c src/ray/util/logging.cc -o bazel-out/k8-opt/bin/_objs/ray_util/logging.pic.o)
    # Configuration: b8ccd03f99515bc83e61dcc1d6d7dae19d73297e07e9a7e7a9e5a922a187fae8
    # Execution platform: @local_config_platform//:host

    Use --sandbox_debug to see verbose messages from the sandbox and retain the sandbox build root for debugging
    In file included from bazel-out/k8-opt/bin/external/com_github_spdlog/_virtual_includes/spdlog/spdlog/fmt/fmt.h:20,
                     from bazel-out/k8-opt/bin/external/com_github_spdlog/_virtual_includes/spdlog/spdlog/common.h:36,
                     from bazel-out/k8-opt/bin/external/com_github_spdlog/_virtual_includes/spdlog/spdlog/details/file_helper.h:6,
                     from bazel-out/k8-opt/bin/external/com_github_spdlog/_virtual_includes/spdlog/spdlog/sinks/basic_file_sink.h:6,
                     from src/ray/util/logging.cc:39:
    bazel-out/k8-opt/bin/external/com_github_spdlog/_virtual_includes/spdlog/spdlog/fmt/bundled/core.h:443:7: error: identifier 'char8_t' is a keyword in C++20 [-Werror=c++20-compat]
      443 | using char8_t FMT_DEPRECATED_ALIAS = internal::char8_type;
          |       ^~~~~~~

Looks like spdlog is using a keyword incompatible with the C++20 standard.

Versions / Dependencies

Ray: cc3fa33
Python: 3.11.2; also tested on 3.10.8
OS: Linux 6.3.1-arch1-1 x86_64

Reproduction script

cd ray/python
pip install -ve .[all]

Issue Severity

High: It blocks me from completing my task.

[richardliaw]

cc @vitsai can you take a quick look?

[peytondmurray]

@vitsai seems like this could be related: gabime/spdlog#1634. I'll try looking into how to set the build variables correctly here. From the error it looks like it is trying to use the bundled fmt library. But I can't tell if it should be...

[rkooo567]

We don't support cpp 20 now. Maybe try again with cpp 17?

[peytondmurray]

@rkooo567 This is with the default settings on two separate machines. I think we probably need to enforce the version of the cpp standard we are using. I'll try doing this in bazel/BUILD.spdlog.

As an aside, it looks like #35287 solves my issue, so maybe merging that is enough.

[vitsai]

Right now we advise users to build with G++ 9 (2019) in the documentation. Using that, I was able to successfully build on Ubuntu 22.04 LTS upgraded to the 6.3.0 kernel. Were you using a more recent version of G++?

sudo add-apt-repository -y ppa:ubuntu-toolchain-r/test
sudo apt-get update
sudo apt-get install -y build-essential curl gcc-9 g++-9 pkg-config psmisc unzip
sudo update-alternatives --install /usr/bin/gcc gcc /usr/bin/gcc-9 90 \
              --slave /usr/bin/g++ g++ /usr/bin/g++-9 \
              --slave /usr/bin/gcov gcov /usr/bin/gcov-9

[peytondmurray]

Yeah, this seems like it could be the culprit - I'm running g++ 13.1.1. I'll try installing an old version of g++ and setting CC, then report back here.

[peytondmurray]

@vitsai Okay, you were totally right, the build failure was caused by my system upgrading to gcc-13. Even setting CC=gcc-12 still works. Should I submit a different issue or rename this one to track this?

[vitsai]

We can discuss on a separate issue. I vaguely remember gcc-10 not working, but perhaps that's fixed now.