Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Rebuild for CUDA 12 w/arch support #51

Merged
merged 11 commits into from
Oct 6, 2023
Merged

Rebuild for CUDA 12 w/arch support #51

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
11 commits
Select commit Hold shift + click to select a range
2c8a259
Rebuild for CUDA 12 w/arch support
regro-cf-autotick-bot Aug 30, 2023
08c4062
MNT: Re-rendered with conda-build 3.26.1, conda-smithy 3.24.1, and co…
regro-cf-autotick-bot Aug 30, 2023
1881a86
Sort `conda-forge.yml` keys
jakirkham Aug 30, 2023
a2ef91d
Free up space on Azure images
jakirkham Aug 30, 2023
3e33401
MNT: Re-rendered with conda-build 3.26.1, conda-smithy 3.24.1, and co…
Aug 30, 2023
a092ff9
Add 10GiB swap file on Azure Linux
jakirkham Aug 31, 2023
1a69a90
MNT: Re-rendered with conda-build 3.26.1, conda-smithy 3.25.0, and co…
Aug 31, 2023
776cad7
Fix swap file setting syntax
jakirkham Aug 31, 2023
494d280
MNT: Re-rendered with conda-build 3.26.1, conda-smithy 3.25.0, and co…
Aug 31, 2023
d93f613
Remove unknown `swapfile_size` parameter
jakirkham Aug 31, 2023
dc3acec
Only build tests on linux-64
kkraus14 Oct 5, 2023
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
36 changes: 36 additions & 0 deletions .azure-pipelines/azure-pipelines-linux.yml
View file
Open in desktop

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

37 changes: 37 additions & 0 deletions ...piler_version12cuda_compilercuda-nvcccuda_compiler_version12.0cxx_compiler_version12.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,37 @@
BUILD:
- aarch64-conda_cos7-linux-gnu
c_compiler:
- gcc
c_compiler_version:
- '12'
cdt_arch:
- aarch64
cdt_name:
- cos7
channel_sources:
- conda-forge
channel_targets:
- conda-forge main
cuda_compiler:
- cuda-nvcc
cuda_compiler_version:
- '12.0'
cxx_compiler:
- gxx
cxx_compiler_version:
- '12'
docker_image:
- quay.io/condaforge/linux-anvil-aarch64
fmt:
- '9'
spdlog:
- '1.11'
target_platform:
- linux-aarch64
zip_keys:
- - c_compiler_version
- cxx_compiler_version
- cuda_compiler
- cuda_compiler_version
- cdt_name
- docker_image
33 changes: 33 additions & 0 deletions ...piler_version12cuda_compilercuda-nvcccuda_compiler_version12.0cxx_compiler_version12.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,33 @@
c_compiler:
- gcc
c_compiler_version:
- '12'
cdt_name:
- cos7
channel_sources:
- conda-forge
channel_targets:
- conda-forge main
cuda_compiler:
- cuda-nvcc
cuda_compiler_version:
- '12.0'
cxx_compiler:
- gxx
cxx_compiler_version:
- '12'
docker_image:
- quay.io/condaforge/linux-anvil-ppc64le
fmt:
- '9'
spdlog:
- '1.11'
target_platform:
- linux-ppc64le
zip_keys:
- - c_compiler_version
- cxx_compiler_version
- cuda_compiler
- cuda_compiler_version
- cdt_name
- docker_image
50 changes: 30 additions & 20 deletions .ci_support/migrations/cuda120.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@ __migrator:
kind:
version
migration_number:
1
2
build_number:
1
paused: false
Expand Down Expand Up @@ -37,44 +37,54 @@ __migrator:
- quay.io/condaforge/linux-anvil-cuda:11.0 # [linux64 and os.environ.get("BUILD_PLATFORM") == "linux-64"]
- quay.io/condaforge/linux-anvil-cuda:11.1 # [linux64 and os.environ.get("BUILD_PLATFORM") == "linux-64"]
- quay.io/condaforge/linux-anvil-cuda:11.2 # [linux64 and os.environ.get("BUILD_PLATFORM") == "linux-64"]
# case: native compilation (build == target)
- quay.io/condaforge/linux-anvil-ppc64le-cuda:11.2 # [ppc64le and os.environ.get("BUILD_PLATFORM") == "linux-ppc64le"]
- quay.io/condaforge/linux-anvil-aarch64-cuda:11.2 # [aarch64 and os.environ.get("BUILD_PLATFORM") == "linux-aarch64"]
# case: cross-compilation (build != target)
- quay.io/condaforge/linux-anvil-cuda:11.2 # [ppc64le and os.environ.get("BUILD_PLATFORM") == "linux-64"]
- quay.io/condaforge/linux-anvil-cuda:11.2 # [aarch64 and os.environ.get("BUILD_PLATFORM") == "linux-64"]
# case: non-CUDA builds
- quay.io/condaforge/linux-anvil-cos7-x86_64 # [linux64 and os.environ.get("BUILD_PLATFORM") == "linux-64"]
cuda_compiler_version:
- None
- 10.2 # [(linux64 or win) and os.environ.get("CF_CUDA_ENABLED", "False") == "True"]
- 11.0 # [(linux64 or win) and os.environ.get("CF_CUDA_ENABLED", "False") == "True"]
- 11.1 # [(linux64 or win) and os.environ.get("CF_CUDA_ENABLED", "False") == "True"]
- 11.2 # [(linux64 or win) and os.environ.get("CF_CUDA_ENABLED", "False") == "True"]
- 12.0 # [(linux64 or win) and os.environ.get("CF_CUDA_ENABLED", "False") == "True"]
- 11.2 # [(linux or win) and os.environ.get("CF_CUDA_ENABLED", "False") == "True"]
- 12.0 # [(linux or win) and os.environ.get("CF_CUDA_ENABLED", "False") == "True"]
commit_message: |
Rebuild for CUDA 12
Rebuild for CUDA 12 w/arch support

The transition to CUDA 12 SDK includes new packages for all CUDA libraries and
build tools. Notably, the cudatoolkit package no longer exists, and packages
should depend directly on the specific CUDA libraries (libblas, libcusolver,
should depend directly on the specific CUDA libraries (libcublas, libcusolver,
etc) as needed. For an in-depth overview of the changes and to report problems
[see this issue]( https://github.com/conda-forge/conda-forge.github.io/issues/1963 ).
Please feel free to raise any issues encountered there. Thank you! :pray:

cuda_compiler: # [linux64 and os.environ.get("CF_CUDA_ENABLED", "False") == "True"]
- cuda-nvcc # [linux64 and os.environ.get("CF_CUDA_ENABLED", "False") == "True"]

cuda_compiler_version: # [linux64 and os.environ.get("CF_CUDA_ENABLED", "False") == "True"]
- 12.0 # [linux64 and os.environ.get("CF_CUDA_ENABLED", "False") == "True"]
cuda_compiler: # [linux and os.environ.get("CF_CUDA_ENABLED", "False") == "True"]
- cuda-nvcc # [linux and os.environ.get("CF_CUDA_ENABLED", "False") == "True"]

c_compiler_version: # [linux64 and os.environ.get("CF_CUDA_ENABLED", "False") == "True"]
- 12 # [linux64 and os.environ.get("CF_CUDA_ENABLED", "False") == "True"]
cuda_compiler_version: # [linux and os.environ.get("CF_CUDA_ENABLED", "False") == "True"]
- 12.0 # [linux and os.environ.get("CF_CUDA_ENABLED", "False") == "True"]

cxx_compiler_version: # [linux64 and os.environ.get("CF_CUDA_ENABLED", "False") == "True"]
- 12 # [linux64 and os.environ.get("CF_CUDA_ENABLED", "False") == "True"]
c_compiler_version: # [linux and os.environ.get("CF_CUDA_ENABLED", "False") == "True"]
- 12 # [linux and os.environ.get("CF_CUDA_ENABLED", "False") == "True"]

fortran_compiler_version: # [linux64 and os.environ.get("CF_CUDA_ENABLED", "False") == "True"]
- 12 # [linux64 and os.environ.get("CF_CUDA_ENABLED", "False") == "True"]
cxx_compiler_version: # [linux and os.environ.get("CF_CUDA_ENABLED", "False") == "True"]
- 12 # [linux and os.environ.get("CF_CUDA_ENABLED", "False") == "True"]

cudnn: # [linux64 and os.environ.get("CF_CUDA_ENABLED", "False") == "True"]
- 8 # [linux64 and os.environ.get("CF_CUDA_ENABLED", "False") == "True"]
fortran_compiler_version: # [linux and os.environ.get("CF_CUDA_ENABLED", "False") == "True"]
- 12 # [linux and os.environ.get("CF_CUDA_ENABLED", "False") == "True"]

cdt_name: # [linux64 and os.environ.get("CF_CUDA_ENABLED", "False") == "True"]
- cos7 # [linux64 and os.environ.get("CF_CUDA_ENABLED", "False") == "True"]
cdt_name: # [linux and os.environ.get("CF_CUDA_ENABLED", "False") == "True"]
- cos7 # [linux and os.environ.get("CF_CUDA_ENABLED", "False") == "True"]

docker_image: # [os.environ.get("BUILD_PLATFORM", "").startswith("linux-") and os.environ.get("CF_CUDA_ENABLED", "False") == "True"]
- quay.io/condaforge/linux-anvil-cos7-x86_64 # [linux64 and os.environ.get("BUILD_PLATFORM") == "linux-64" and os.environ.get("CF_CUDA_ENABLED", "False") == "True"]
# case: native compilation (build == target)
- quay.io/condaforge/linux-anvil-ppc64le # [ppc64le and os.environ.get("BUILD_PLATFORM") == "linux-ppc64le"]
- quay.io/condaforge/linux-anvil-aarch64 # [aarch64 and os.environ.get("BUILD_PLATFORM") == "linux-aarch64"]
# case: cross-compilation (build != target)
- quay.io/condaforge/linux-anvil-cos7-x86_64 # [ppc64le and os.environ.get("BUILD_PLATFORM") == "linux-64"]
- quay.io/condaforge/linux-anvil-cos7-x86_64 # [aarch64 and os.environ.get("BUILD_PLATFORM") == "linux-64"]
14 changes: 14 additions & 0 deletions README.md
View file
Open in desktop

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

8 changes: 6 additions & 2 deletions conda-forge.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,3 +1,9 @@
azure:
free_disk_space: true
settings_linux:
swapfile_size: 10GiB
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Still seeing the same error. Not sure what we should bump this too. Any thoughts?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm really unsure where there's a lot of "weird environment" in the loop here. The fact that different components are failing with signal 11 points to environment from my perspective.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

IIUC the feedstock is using emulation for arch builds. Could that be an issue? Should we use cross-compilation or native builds?

librmm-feedstock/conda-forge.yml

Lines 9 to 11 in f0423fb

provider:
linux_aarch64: azure
linux_ppc64le: azure

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is only building tests to ensure dependency version changes don't break downstream usage of RMM headers. I would prefer to keep the setup as simple as possible where the easiest path forward would possibly be to just only build the tests on linux-64?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That sounds ok to me

conda_build:
pkg_format: '2'
conda_forge_output_validation: true
github:
branch_name: main
Expand All @@ -9,5 +15,3 @@ os_version:
provider:
linux_aarch64: azure
linux_ppc64le: azure
conda_build:
pkg_format: '2'
7 changes: 6 additions & 1 deletion recipe/build.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -11,7 +11,12 @@ export LIBRMM_BUILD_DIR=${LIBRMM_BUILD_DIR:=${RMM_SRC_DIR}/build}
export INSTALL_PREFIX=${PREFIX:=${CONDA_PREFIX}}
export CUDA_STATIC_RUNTIME=OFF
export PER_THREAD_DEFAULT_STREAM=OFF
export BUILD_TESTS=ON

export BUILD_TESTS=OFF
if [[ "$build_platform" == "linux-64" ]]; then
export BUILD_TESTS=ON
fi

export BUILD_BENCHMARKS=OFF
export BUILD_TYPE=Release
export VERBOSE_FLAG="-v"
Expand Down
2 changes: 1 addition & 1 deletion recipe/meta.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -11,7 +11,7 @@ source:
folder: rmm

build:
number: 1
number: 2
skip: true # [(not linux) or (cuda_compiler_version == "None")]
run_exports:
- {{ pin_subpackage(name, max_pin="x.x") }}
Expand Down