Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Add CUDA 11.8 cross-compilation support #261

Merged
merged 8 commits into from
Aug 31, 2023
Merged

Add CUDA 11.8 cross-compilation support #261

Show file tree
Hide file tree
Changes from 3 commits
Commits
Show all changes
8 commits
Select commit Hold shift + click to select a range
c451d0a
Add CUDA 11.8 cross-compilation support
jakirkham Aug 23, 2023
7678c16
Add CUDA 11.8 migrator
jakirkham Aug 23, 2023
86b9449
Enable using local CUDA 11.8 migrator
jakirkham Aug 23, 2023
8714682
MNT: Re-rendered with conda-build 3.24.0, conda-smithy 3.24.1, and co…
jakirkham Aug 23, 2023
ed1113a
Tweak RPM path to use `$CUDA_COMPILER_VERSION`
jakirkham Aug 24, 2023
e1165f1
Merge branch 'main' into add_cuda118_arch_cross
jakirkham Aug 31, 2023
0d8bb1c
free up disk space
h-vetinari Aug 31, 2023
c65ca87
MNT: Re-rendered with conda-build 3.26.1, conda-smithy 3.24.1, and co…
h-vetinari Aug 31, 2023
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
92 changes: 92 additions & 0 deletions .ci_support/migrations/cuda118.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,92 @@
migrator_ts: 1692828152
__migrator:
kind:
version
use_local: true # TODO: Remove this line after completing testing
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not sure that we want to have the cross-compiled jobs for merging. We don't need them from the POV of the CI setup itself, just for testing the branches here in CI. I suggested adding them in #236, but never got a response.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yep no strong feelings here. Happy to keep or drop as preferred

migration_number:
1
build_number:
1
paused: false
override_cbc_keys:
- cuda_compiler_stub
operation: key_add
check_solvable: false
primary_key: cuda_compiler_version
ordering:
cxx_compiler_version:
- 9
- 8
- 7
c_compiler_version:
- 9
- 8
- 7
fortran_compiler_version:
- 9
- 8
- 7
docker_image:
- quay.io/condaforge/linux-anvil-comp7 # [os.environ.get("BUILD_PLATFORM") == "linux-64"]
- quay.io/condaforge/linux-anvil-aarch64 # [os.environ.get("BUILD_PLATFORM") == "linux-aarch64"]
- quay.io/condaforge/linux-anvil-ppc64le # [os.environ.get("BUILD_PLATFORM") == "linux-ppc64le"]
- quay.io/condaforge/linux-anvil-armv7l # [os.environ.get("BUILD_PLATFORM") == "linux-armv7l"]
- quay.io/condaforge/linux-anvil-cuda:9.2 # [linux64 and os.environ.get("BUILD_PLATFORM") == "linux-64"]
- quay.io/condaforge/linux-anvil-cuda:10.0 # [linux64 and os.environ.get("BUILD_PLATFORM") == "linux-64"]
- quay.io/condaforge/linux-anvil-cuda:10.1 # [linux64 and os.environ.get("BUILD_PLATFORM") == "linux-64"]
- quay.io/condaforge/linux-anvil-cuda:10.2 # [linux64 and os.environ.get("BUILD_PLATFORM") == "linux-64"]
- quay.io/condaforge/linux-anvil-cuda:11.0 # [linux64 and os.environ.get("BUILD_PLATFORM") == "linux-64"]
- quay.io/condaforge/linux-anvil-cuda:11.1 # [linux64 and os.environ.get("BUILD_PLATFORM") == "linux-64"]
- quay.io/condaforge/linux-anvil-cuda:11.2 # [linux64 and os.environ.get("BUILD_PLATFORM") == "linux-64"]
# case: native compilation (build == target)
- quay.io/condaforge/linux-anvil-ppc64le-cuda:11.2 # [ppc64le and os.environ.get("BUILD_PLATFORM") == "linux-ppc64le"]
- quay.io/condaforge/linux-anvil-aarch64-cuda:11.2 # [aarch64 and os.environ.get("BUILD_PLATFORM") == "linux-aarch64"]
# case: cross-compilation (build != target)
- quay.io/condaforge/linux-anvil-cuda:11.2 # [ppc64le and os.environ.get("BUILD_PLATFORM") == "linux-64"]
- quay.io/condaforge/linux-anvil-cuda:11.2 # [aarch64 and os.environ.get("BUILD_PLATFORM") == "linux-64"]
- quay.io/condaforge/linux-anvil-cuda:11.8 # [linux64 and os.environ.get("BUILD_PLATFORM") == "linux-64"]
# case: native compilation (build == target)
- quay.io/condaforge/linux-anvil-ppc64le-cuda:11.8 # [ppc64le and os.environ.get("BUILD_PLATFORM") == "linux-ppc64le"]
- quay.io/condaforge/linux-anvil-aarch64-cuda:11.8 # [aarch64 and os.environ.get("BUILD_PLATFORM") == "linux-aarch64"]
# case: cross-compilation (build != target)
- quay.io/condaforge/linux-anvil-cuda:11.8 # [ppc64le and os.environ.get("BUILD_PLATFORM") == "linux-64"]
- quay.io/condaforge/linux-anvil-cuda:11.8 # [aarch64 and os.environ.get("BUILD_PLATFORM") == "linux-64"]
# case: non-CUDA builds
- quay.io/condaforge/linux-anvil-cos7-x86_64 # [linux64 and os.environ.get("BUILD_PLATFORM") == "linux-64"]
cuda_compiler_version:
- None
- 10.2 # [(linux64 or win) and os.environ.get("CF_CUDA_ENABLED", "False") == "True"]
- 11.0 # [(linux64 or win) and os.environ.get("CF_CUDA_ENABLED", "False") == "True"]
- 11.1 # [(linux64 or win) and os.environ.get("CF_CUDA_ENABLED", "False") == "True"]
- 11.2 # [(linux or win) and os.environ.get("CF_CUDA_ENABLED", "False") == "True"]
- 11.8 # [(linux or win) and os.environ.get("CF_CUDA_ENABLED", "False") == "True"]
- 12.0 # [(linux or win) and os.environ.get("CF_CUDA_ENABLED", "False") == "True"]
commit_message: |
Rebuild for CUDA 11.8 w/arch support

cuda_compiler: # [(linux or win64) and os.environ.get("CF_CUDA_ENABLED", "False") == "True"]
- nvcc # [(linux or win64) and os.environ.get("CF_CUDA_ENABLED", "False") == "True"]

cuda_compiler_version: # [(linux or win64) and os.environ.get("CF_CUDA_ENABLED", "False") == "True"]
- 11.8 # [(linux or win64) and os.environ.get("CF_CUDA_ENABLED", "False") == "True"]

c_compiler_version: # [linux and os.environ.get("CF_CUDA_ENABLED", "False") == "True"]
- 11 # [linux and os.environ.get("CF_CUDA_ENABLED", "False") == "True"]

cxx_compiler_version: # [linux and os.environ.get("CF_CUDA_ENABLED", "False") == "True"]
- 11 # [linux and os.environ.get("CF_CUDA_ENABLED", "False") == "True"]

fortran_compiler_version: # [linux and os.environ.get("CF_CUDA_ENABLED", "False") == "True"]
- 11 # [linux and os.environ.get("CF_CUDA_ENABLED", "False") == "True"]

cdt_name: # [linux and os.environ.get("CF_CUDA_ENABLED", "False") == "True"]
- cos7 # [linux and os.environ.get("CF_CUDA_ENABLED", "False") == "True"]

docker_image: # [os.environ.get("BUILD_PLATFORM", "").startswith("linux-") and os.environ.get("CF_CUDA_ENABLED", "False") == "True"]
- quay.io/condaforge/linux-anvil-cuda:11.8 # [linux64 and os.environ.get("BUILD_PLATFORM") == "linux-64"]
# case: native compilation (build == target)
- quay.io/condaforge/linux-anvil-ppc64le-cuda:11.8 # [ppc64le and os.environ.get("BUILD_PLATFORM") == "linux-ppc64le"]
- quay.io/condaforge/linux-anvil-aarch64-cuda:11.8 # [aarch64 and os.environ.get("BUILD_PLATFORM") == "linux-aarch64"]
# case: cross-compilation (build != target)
- quay.io/condaforge/linux-anvil-cuda:11.8 # [ppc64le and os.environ.get("BUILD_PLATFORM") == "linux-64"]
- quay.io/condaforge/linux-anvil-cuda:11.8 # [aarch64 and os.environ.get("BUILD_PLATFORM") == "linux-64"]
21 changes: 17 additions & 4 deletions recipe/cross_compile_support.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -38,7 +38,7 @@ if [[ "${HOST_PLATFORM}" != "${BUILD_PLATFORM}" ]]; then
fi


if [[ "${CUDA_COMPILER_VERSION}" == "11.2" ]] && [[ "${CDT_NAME}" == "cos8" || "${CDT_NAME}" == "cos7" ]]; then
if [[ "${CUDA_COMPILER_VERSION}" == "11.2" || "${CUDA_COMPILER_VERSION}" == "11.8" ]] && [[ "${CDT_NAME}" == "cos8" || "${CDT_NAME}" == "cos7" ]]; then
# We use cdt_name=cos7 for rhel8 based nvcc till we figure out
# a stable cos8 replacement.
EXTRACT_DIR=$(mktemp -d)
Expand All @@ -48,8 +48,18 @@ if [[ "${HOST_PLATFORM}" != "${BUILD_PLATFORM}" ]]; then
else
CUDA_HOST_PLATFORM_ARCH=${HOST_PLATFORM_ARCH}
fi
# download manifest for latest 11.2.x patch version
curl -L https://developer.download.nvidia.com/compute/cuda/repos/rhel8/${CUDA_HOST_PLATFORM_ARCH}/version_11.2.2.json > manifest.json
# download manifest for latest CUDA patch version
CUDA_MANIFEST_VERSION=$(
case "${CUDA_COMPILER_VERSION}" in
("11.2") echo "11.2.2" ;;
("11.8") echo "11.8.0" ;;
(*) echo "" ;;
esac)
if [[ "${CUDA_MANIFEST_VERSION}" == "" ]]; then
echo "cross compiling with cuda not in (11.2, 11.8, 12.0) not supported yet"
exit 1
fi
curl -L https://developer.download.nvidia.com/compute/cuda/repos/rhel8/${CUDA_HOST_PLATFORM_ARCH}/version_${CUDA_MANIFEST_VERSION}.json > manifest.json
# packages for which we also need to install the -devel version
# (names need "_" not "-" to match spelling in manifest);
# some packages don't have a key in the manifest, so we
Expand Down Expand Up @@ -119,6 +129,9 @@ if [[ "${HOST_PLATFORM}" != "${BUILD_PLATFORM}" ]]; then
elif [[ "${CUDA_COMPILER_VERSION}" == "11.2" ]]; then
echo "cross compiling with cuda == 11.2 and cdt != cos7/8 not supported yet"
exit 1
elif [[ "${CUDA_COMPILER_VERSION}" == "11.8" ]]; then
echo "cross compiling with cuda == 11.8 and cdt != cos7/8 not supported yet"
exit 1
elif [[ "${CUDA_COMPILER_VERSION}" == "12.0" ]] && [[ "${CDT_NAME}" == "cos7" ]]; then
# No extra steps necessary for CUDA 12, handled through new packages
true
Expand All @@ -128,7 +141,7 @@ if [[ "${HOST_PLATFORM}" != "${BUILD_PLATFORM}" ]]; then
elif [[ "${CUDA_COMPILER_VERSION}" != "None" ]]; then
# FIXME: can use anaconda.org/nvidia packages to get the includes and libs
# for cuda >=11.3.
echo "cross compiling with cuda not in (11.2, 12.0) not supported yet"
echo "cross compiling with cuda not in (11.2, 11.8, 12.0) not supported yet"
exit 1
fi
fi
Expand Down