Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

ray-packages v2.39.0 #188

Open
wants to merge 27 commits into
base: main
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
27 commits
Select commit Hold shift + click to select a range
ff089e9
updated v2.39.0
regro-cf-autotick-bot Nov 14, 2024
9bbea35
MNT: Re-rendered with conda-build 24.9.0, conda-smithy 3.44.3, and co…
regro-cf-autotick-bot Nov 14, 2024
c5598d3
update setup.py patch and dependencies, use swapfile on windows
mattip Nov 14, 2024
94c9c5a
temporarily pin azure vmImage to macOS-12
mattip Nov 14, 2024
4479b0f
linter fix
mattip Nov 14, 2024
726e8f1
patch zlib<1.3.1 to remove fdopen define that breaks macOS build
mattip Nov 18, 2024
5e725ea
try macOS-13 image again
mattip Nov 18, 2024
02652d3
add debug cruft, remove cross-building removal of '-stdlib=libc++'
mattip Nov 18, 2024
592769f
no ~/.bazelrc
mattip Nov 18, 2024
f7e64e1
patch protobuf for windows
mattip Nov 18, 2024
20e29e7
try harder to add 'stdlib=libc++' to the macos options
mattip Nov 18, 2024
0ff2f3a
blindly follow the instructions to use bazel apple support
mattip Nov 18, 2024
6dd945b
probe which SDK is being downloaded
mattip Nov 18, 2024
0b5c1a2
try to use the bazel-toolchain on bot macos builds
mattip Nov 18, 2024
8a86a11
add another zlib patch, always remove '-stdlib=libc++' flag
mattip Nov 19, 2024
af72db2
add more quotes
mattip Nov 19, 2024
3d98c7f
quote differently
mattip Nov 19, 2024
0c5f78d
patch order of headers in file_system_monitor.h to avoid macOS c++ de…
mattip Nov 19, 2024
ec642ac
add file
mattip Nov 19, 2024
74abe36
update spdlog version
mattip Nov 19, 2024
96131e0
tweak build flags
mattip Nov 19, 2024
d65d3b3
add '/utf-8' to cxxopt on windows
mattip Nov 19, 2024
4e21763
use spdlog 1.14 since 1.15 does not compile for windows
mattip Nov 20, 2024
370736f
windows still runs out of heap space, even with 'SET_PAGEFILE: 'True''
mattip Nov 20, 2024
c37893d
patch spdlog rather than change version
mattip Nov 20, 2024
f32c276
redo patch
mattip Nov 20, 2024
db8c039
max stdouterr larger to maybe capture an error message on macOS-x86_64
mattip Nov 21, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion .azure-pipelines/azure-pipelines-osx.yml

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

2 changes: 0 additions & 2 deletions .ci_support/linux_64_python3.10.____cpython.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -36,5 +36,3 @@ target_platform:
zip_keys:
- - c_compiler_version
- cxx_compiler_version
- - c_stdlib_version
- cdt_name
2 changes: 0 additions & 2 deletions .ci_support/linux_64_python3.11.____cpython.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -36,5 +36,3 @@ target_platform:
zip_keys:
- - c_compiler_version
- cxx_compiler_version
- - c_stdlib_version
- cdt_name
2 changes: 0 additions & 2 deletions .ci_support/linux_64_python3.12.____cpython.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -36,5 +36,3 @@ target_platform:
zip_keys:
- - c_compiler_version
- cxx_compiler_version
- - c_stdlib_version
- cdt_name
2 changes: 0 additions & 2 deletions .ci_support/linux_64_python3.9.____cpython.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -36,5 +36,3 @@ target_platform:
zip_keys:
- - c_compiler_version
- cxx_compiler_version
- - c_stdlib_version
- cdt_name

This file was deleted.

4 changes: 2 additions & 2 deletions .ci_support/osx_64_python3.10.____cpython.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@ MACOSX_SDK_VERSION:
c_compiler:
- clang
c_compiler_version:
- '17'
- '18'
c_stdlib:
- macosx_deployment_target
c_stdlib_version:
Expand All @@ -19,7 +19,7 @@ curl:
cxx_compiler:
- clangxx
cxx_compiler_version:
- '17'
- '18'
libgrpc:
- '1.65'
macos_machine:
Expand Down
4 changes: 2 additions & 2 deletions .ci_support/osx_64_python3.11.____cpython.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@ MACOSX_SDK_VERSION:
c_compiler:
- clang
c_compiler_version:
- '17'
- '18'
c_stdlib:
- macosx_deployment_target
c_stdlib_version:
Expand All @@ -19,7 +19,7 @@ curl:
cxx_compiler:
- clangxx
cxx_compiler_version:
- '17'
- '18'
libgrpc:
- '1.65'
macos_machine:
Expand Down
4 changes: 2 additions & 2 deletions .ci_support/osx_64_python3.12.____cpython.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@ MACOSX_SDK_VERSION:
c_compiler:
- clang
c_compiler_version:
- '17'
- '18'
c_stdlib:
- macosx_deployment_target
c_stdlib_version:
Expand All @@ -19,7 +19,7 @@ curl:
cxx_compiler:
- clangxx
cxx_compiler_version:
- '17'
- '18'
libgrpc:
- '1.65'
macos_machine:
Expand Down
4 changes: 2 additions & 2 deletions .ci_support/osx_64_python3.9.____cpython.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@ MACOSX_SDK_VERSION:
c_compiler:
- clang
c_compiler_version:
- '17'
- '18'
c_stdlib:
- macosx_deployment_target
c_stdlib_version:
Expand All @@ -19,7 +19,7 @@ curl:
cxx_compiler:
- clangxx
cxx_compiler_version:
- '17'
- '18'
libgrpc:
- '1.65'
macos_machine:
Expand Down
4 changes: 2 additions & 2 deletions .ci_support/osx_arm64_python3.10.____cpython.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@ MACOSX_SDK_VERSION:
c_compiler:
- clang
c_compiler_version:
- '17'
- '18'
c_stdlib:
- macosx_deployment_target
c_stdlib_version:
Expand All @@ -19,7 +19,7 @@ curl:
cxx_compiler:
- clangxx
cxx_compiler_version:
- '17'
- '18'
libgrpc:
- '1.65'
macos_machine:
Expand Down
4 changes: 2 additions & 2 deletions .ci_support/osx_arm64_python3.11.____cpython.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@ MACOSX_SDK_VERSION:
c_compiler:
- clang
c_compiler_version:
- '17'
- '18'
c_stdlib:
- macosx_deployment_target
c_stdlib_version:
Expand All @@ -19,7 +19,7 @@ curl:
cxx_compiler:
- clangxx
cxx_compiler_version:
- '17'
- '18'
libgrpc:
- '1.65'
macos_machine:
Expand Down
4 changes: 2 additions & 2 deletions .ci_support/osx_arm64_python3.12.____cpython.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@ MACOSX_SDK_VERSION:
c_compiler:
- clang
c_compiler_version:
- '17'
- '18'
c_stdlib:
- macosx_deployment_target
c_stdlib_version:
Expand All @@ -19,7 +19,7 @@ curl:
cxx_compiler:
- clangxx
cxx_compiler_version:
- '17'
- '18'
libgrpc:
- '1.65'
macos_machine:
Expand Down
4 changes: 2 additions & 2 deletions .ci_support/osx_arm64_python3.9.____cpython.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@ MACOSX_SDK_VERSION:
c_compiler:
- clang
c_compiler_version:
- '17'
- '18'
c_stdlib:
- macosx_deployment_target
c_stdlib_version:
Expand All @@ -19,7 +19,7 @@ curl:
cxx_compiler:
- clangxx
cxx_compiler_version:
- '17'
- '18'
libgrpc:
- '1.65'
macos_machine:
Expand Down
2 changes: 1 addition & 1 deletion .scripts/run_osx_build.sh

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

10 changes: 4 additions & 6 deletions .scripts/run_win_build.bat

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

2 changes: 1 addition & 1 deletion azure-pipelines.yml

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

5 changes: 5 additions & 0 deletions conda-forge.yml
Original file line number Diff line number Diff line change
Expand Up @@ -7,3 +7,8 @@ github:
branch_name: main
tooling_branch_name: main
test: native_and_emulated

azure:
settings_win:
variables:
SET_PAGEFILE: 'True'
1 change: 0 additions & 1 deletion recipe/build-core.bat
Original file line number Diff line number Diff line change
Expand Up @@ -31,7 +31,6 @@ echo ==========================================================
echo calling pip to install
echo ==========================================================
cd python
rem This requires patch 0006
echo startup --output_user_root=D:/tmp >> ..\.bazelrc
echo build --jobs=1 >> ..\.bazelrc
"%PYTHON%" -m pip install . -vv
Expand Down
23 changes: 7 additions & 16 deletions recipe/build-core.sh
Original file line number Diff line number Diff line change
Expand Up @@ -6,36 +6,27 @@ if [[ "$target_platform" == osx* ]]; then
# also passed as we are using at least OSX 10.15 which moves the include directory out
# of /usr/include to ${SDKROOT}/MacOSX.sdk/usr/include
if [[ "$target_platform" == osx-arm64 ]]; then
# https://github.com/conda-forge/bazel-toolchain-feedstock/issues/18
# delete the line from the template and the CXXFLAGS
export CXXFLAGS=${CXXFLAGS/-stdlib=libc++ /}
export LDFLAGS="$LDFLAGS -undefined dynamic_lookup -Wl,-framework,Foundation"
sed -e"/stdlib=libc/d" -i'' $CONDA_PREFIX/share/bazel_toolchain/CROSSTOOL.template
source gen-bazel-toolchain
fi
# https://github.com/conda-forge/bazel-toolchain-feedstock/issues/18
# delete the line from the template and the CXXFLAGS
export CXXFLAGS="${CXXFLAGS/-stdlib=libc++ /} -Wno-vla-cxx-extension"
sed -i"''" -e'/stdlib=libc/d' $CONDA_PREFIX/share/bazel_toolchain/CROSSTOOL.template
source gen-bazel-toolchain
cat >> .bazelrc <<EOF
build --define CONDA_CC=${CC}
build --define CONDA_CFLAGS="${CFLAGS}"
build --define CONDA_AR=${AR}
build --define CONDA_NM=${NM}
build --define CONDA_RANLIB=${RANLIB}
build --define CONDA_SDKROOT=${SDKROOT}
EOF
if [[ "$target_platform" == osx-arm64 ]]; then
cat >> .bazelrc <<EOF
# build --subcommands
build --crosstool_top=//bazel_toolchain:toolchain
build --cpu=darwin_arm64
build --cpu=${TARGET_CPU}
build --platforms=//bazel_toolchain:target_platform
build --host_platform=//bazel_toolchain:build_platform
build --experimental_ui_max_stdouterr_bytes=16000000
EOF
fi
fi

if [[ -e $CONDA_PREFIX/include/crypt.h ]]; then
# fix for python3.8 which depends on system includes for crypt.h
# but the bazel sandbox does not add it
cp $CONDA_PREFIX/include/crypt.h $PREFIX/include/python*
fi

cd python/
Expand Down
18 changes: 13 additions & 5 deletions recipe/meta.yaml
Original file line number Diff line number Diff line change
@@ -1,12 +1,12 @@
{% set version = "2.38.0" %}
{% set version = "2.39.0" %}

package:
name: ray-packages
version: {{ version }}

source:
url: https://github.com/ray-project/ray/archive/ray-{{ version }}.tar.gz
sha256: 1579a974ef80eba0796be0fc95e4e7546cb1885cd682c50bf952f498618b8f04
sha256: fef2bb41d4ce2c9004612cbed5c2c6c4a2446782c58b557818279d97728d88dc
patches:
- patches/0001-patch-redis-for-ar-ranlib.patch
- patches/0002-Disable-making-entry-scripts.patch
Expand All @@ -17,9 +17,17 @@ source:
# Keep in sync with current or active migration of libgrpc to avoid
# ABI breakage
- patches/0005-Vendor-grpc-1.65.5.patch
# Patch out old zlib-1.3.0 to remove fdopen, part of zlib-1.3.1
- patches/0007-patch-zlib-to-remove-define-fdopen-on-macos.patch
# Patch around non-const NAN on windows in protobuf 27.5
- patches/0008-patch-protobuf-use-of-nan-on-windows.patch
# Patch out zlib-1.2.13 to remove fdopen, part of zlib-1.3.1
- patches/0009-patch-zlib-in-prometheus-cpp-and-boost-as-well.patch
# Patch bundled fmtlib/fmt in spdlog for char8_type which is invalid on macOS
- patches/0011-patch-bundled-fmt-in-spdlog-for-invalid-char8_type.patch

build:
number: 2
number: 0

# Need these up here for conda-smithy to handle them properly.
requirements:
Expand Down Expand Up @@ -137,6 +145,7 @@ outputs:
- {{ pin_subpackage('ray-core', exact=True) }}
- pandas >=1.3
- pyarrow >=6.0.1
- pyarrow <18 # [osx and x86_64]
- fsspec
test:
imports:
Expand Down Expand Up @@ -208,8 +217,7 @@ outputs:
- python
- {{ pin_subpackage('ray-tune', exact=True) }}
- dm-tree
- gymnasium ==0.28.1 # [py<312]
- gymnasium ==0.29.1 # [py>=312]
- gymnasium ==1.0.0
- lz4
- pyyaml
- scikit-image
Expand Down
Loading