add swap for windows, add debug for macos, rerender

[mattip]

Checklist

• Used a personal fork of the feedstock to propose changes
• Bumped the build number (if the version is unchanged)
• Reset the build number to 0 (if the version changed)
• Re-rendered with the latest conda-smithy (Use the phrase @conda-forge-admin, please rerender in a comment in this PR for automated rerendering)
• Ensured the license file is being packaged.

Debug build issues

[conda-forge-admin]

Hi! This is the friendly automated conda-forge-linting service.

I just wanted to let you know that I linted all conda-recipes in your PR (recipe/meta.yaml) and found it was in an excellent condition.

[h-vetinari]

Could you try removing

ray-packages-feedstock/recipe/build-core.sh

Line 11 in 548b853

export CXXFLAGS=${CXXFLAGS/-stdlib=libc++ /}

?

[mattip]

cc @h-vetinari

This all seems so fragile. The windows builds are newly failing with

external/com_google_protobuf/upb/message/internal/message.c(22): error C2099: initializer is not a constant

and macOS arm64 is newly failing with

In file included from external/zlib/zutil.c:10:
In file included from external/zlib/gzguts.h:20:
In file included from /Users/runner/miniforge3/conda-bld/ray-packages_1730357054105/_build_env/include/c++/v1/stdio.h:108:
/Applications/Xcode_15.2.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX11.0.sdk/usr/include/stdio.h:210:7: error: expected identifier or '('
  210 | FILE    *fdopen(int, const char *) __DARWIN_ALIAS_STARTING(__MAC_10_6, __IPHONE_2_0, __DARWIN_ALIAS(fdopen));
      |          ^
external/zlib/zutil.h:147:33: note: expanded from macro 'fdopen'
  147 | #        define fdopen(fd,mode) NULL /* No fdopen() */

It would be nice if we could refactor bazel build to use the conda-provided system libraries instead of building everything from source.

[mattip]

Anyhow, to the point of the PR: the macOS x64_86 builds show CXXFLAGS has -stdlib=libc++ just before the build:

CXXFLAGS=-march=core2 -mtune=haswell -mssse3 -ftree-vectorize -fPIC -fstack-protector-strong -O2 -pipe -stdlib=libc++ -fvisibility-inlines-hidden -fmessage-length=0 -isystem /Users/runner/miniforge3/conda-bld/ray-packages_1730357074667/_h_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placeh/include -fdebug-prefix-map=/Users/runner/miniforge3/conda-bld/ray-packages_1730357074667/work=/usr/local/src/conda/ray-core-2.38.0 -fdebug-prefix-map=/Users/runner/miniforge3/conda-bld/ray-packages_1730357074667/_h_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placeh=/usr/local/src/conda-prefix

[mattip]

Could you try removing ...

I removed all the bug-fixes I put in over what is needed for a cross-platform build. Let's see which are still needed

[h-vetinari]

This all seems so fragile. The windows builds are newly failing with

Given that this only happened in one job, I was wondering if this was a flake...

Anyhow, to the point of the PR: the macOS x64_86 builds show CXXFLAGS has -stdlib=libc++ just before the build:

Yeah, I think the issue is that bazel does all its hermetic build isolation, and will generally ignore all environment variables unless specifically instructed to do so (at least, IIRC). Looking at the failed call (added linebreaks for legibility), I don't see -stdlib=libc++ anywhere.

external/local_config_cc/wrapped_clang_pp \
    '-D_FORTIFY_SOURCE=1' \
    -fstack-protector \
    -fcolor-diagnostics \
    -Wall \
    -Wthread-safety \
    -Wself-assign \
    -fno-omit\
    -frame-pointer \
    -g0 \
    -O2 \
    -DNDEBUG \
    '-DNS_BLOCK_ASSERTIONS=1' \
    '-std=c++11' \
    'DEBUG_PREFIX_MAP_PWD=.' \
    -iquote external/com_google_absl \
    -iquote bazel-out/darwin-opt-exec-2B5CBBC6/bin/external/com_google_absl \
    -MD \
    -MF bazel-out/darwin-opt-exec-2B5CBBC6/bin/external/com_google_absl/absl/numeric/_objs/int128/int128.d \
    '-DBAZEL_CURRENT_REPOSITORY="com_google_absl"' \
    '-frandom-seed=bazel-out/darwin-opt-exec-2B5CBBC6/bin/external/com_google_absl/absl/numeric/_objs/int128/int128.o' \
    -isysroot __BAZEL_XCODE_SDKROOT__ \
    -f__BAZEL_XCODE_SDKROOT__/System/Library/Frameworks \
    -f__BAZEL_XCODE_DEVELOPER_DIR__/Platforms/MacOSX.platform/Developer/Library/Frameworks \
    -no-canonical-prefixes \
    -pthread \
    -g0 \
    -W \
    -g0 \
    '-std=c++17' \
    -Wall \
    -Wextra \
    -Wc++98-compat-extra-semi \
    -Wcast-qual \
    -Wconversion \
    -Wdeprecated-pragma \
    -Wfloat-overflow-conversion \
    -Wfloat-zero-conversion \
    -Wfor-loop-analysis \
    -Wformat-security \
    -Wgnu-redeclared-enum \
    -Winfinite-recursion \
    -Winvalid-constexpr \
    -Wliteral-conversion \
    -Wmissing-declarations \
    -Woverlength-strings \
    -Wpointer-arith \
    -Wself-assign \
    -Wshadow-all \
    -Wshorten-64-to-32 \
    -Wsign-conversion \
    -Wstring-conversion \
    -Wtautological-overlap-compare \
    -Wtautological-unsigned-zero-compare \
    -Wundef \
    -Wuninitialized \
    -Wunreachable-code \
    -Wunused-comparison \
    -Wunused-local-typedefs \
    -Wunused-result \
    -Wvla \
    -Wwrite-strings \
    -Wno\
    -float-conversion \
    -Wno-implicit\
    -float-conversion \
    -Wno-implicit-int\
    -float-conversion \
    -Wno-unknown\
    -Warning-option \
    -DNOMINMAX \
    -no-canonical-prefixes \
    -Wno-builtin-macro-redefined \
    '-D__DATE__="redacted"' \
    '-D__TIMESTAMP__="redacted"' \
    '-D__TIME__="redacted"' \
    -target x86_64-apple-macosx10.15 \
    -c external/com_google_absl/absl/numeric/int128.cc \
    -o bazel-out/darwin-opt-exec-2B5CBBC6/bin/external/com_google_absl/absl/numeric/_objs/int128/int128.o

It would be nice if we could refactor bazel build to use the conda-provided system libraries instead of building everything from source.

💯

[h-vetinari]

If you want to persist this across rerenders, you can do

azure:
  settings_osx:
    pool:
      vmImage: macos-14

in conda-forge.yml.

[mattip]

No, it was just an experiment

[mattip]

This blog post provides a recipe for using system libraries in a bazel build.

[mattip]

I will try to get this to pass CI with windows SET_PAGEFILE: 'True' and leave the temporary fix to macOS-12. I know the macOS image will be overwritten in the next render, maybe something magically will have fixed the problem by then.

[mattip]

@h-vetinari looking again at the exploded compilation arguments, I see

-isysroot __BAZEL_XCODE_SDKROOT__ \
    -f__BAZEL_XCODE_SDKROOT__/System/Library/Frameworks \
    -f__BAZEL_XCODE_DEVELOPER_DIR__/Platforms/MacOSX.platform/Developer/Library/Frameworks \

That seems wrong. I think the __BAZEL_XCODE_SDKROOT__ is meant to be replaced with an actual path, maybe from the $SDKROOT env variable?

[h-vetinari]

Yeah that looks suspect. You can try and check if you find something on the bazel-toolchain feedstock related to the SDK that bazel uses. If there's nothing there, then it's likely coming from bazel itself.

[mattip]

My bad, I need to use the bazel-toolchain recipe on both arm64 and x86_64 builds. If I recall correctly, on the macOS-12 image, using it on x86_64 broke the build, but on later images it is required.

[mattip]

This PR is part of #188, closing.