Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add CUDA 11.8 migrator #4834

Merged
merged 5 commits into from
Sep 1, 2023

Conversation

jakirkham
Copy link
Member

@jakirkham jakirkham commented Aug 23, 2023

Adds a migrator to include CUDA 11.8 as discussed in issue ( conda-forge/conda-forge.github.io#1981 ).


Checklist

  • Used a personal fork of the feedstock to propose changes
  • Bumped the build number (if the version is unchanged)
  • Reset the build number to 0 (if the version changed)
  • Re-rendered with the latest conda-smithy (Use the phrase @conda-forge-admin, please rerender in a comment in this PR for automated rerendering)
  • Ensured the license file is being packaged.

Fixes conda-forge/conda-forge.github.io#1981

Closes conda-forge/ucx-split-feedstock#133
Closes conda-forge/nccl-feedstock#96
Closes conda-forge/pycolmap-feedstock#7

@jakirkham jakirkham requested a review from a team as a code owner August 23, 2023 22:07
@conda-forge-webservices
Copy link
Contributor

Hi! This is the friendly automated conda-forge-linting service.

I just wanted to let you know that I linted all conda-recipes in your PR (recipe) and found it was in an excellent condition.

@jakirkham
Copy link
Member Author

jakirkham commented Aug 23, 2023

Started a couple test PRs using this migration (updated below).

These both work:

This PR was created and builds part way ok. Then runs into a tooling issue outside the scope of this change ( some bug in cross-compiling linking: conda-forge/pycolmap-feedstock#7 (comment) )

@jakirkham
Copy link
Member Author

jakirkham commented Aug 23, 2023

It appears that native builds work ok. However CUDA 11.8 cross-compilation support is absent atm. Started PR ( conda-forge/conda-forge-ci-setup-feedstock#261 ) to add CUDA 11.8 cross-compilation support

Edit: Marking as draft until CUDA 11.8 cross-compilation support is added

@jakirkham
Copy link
Member Author

Thoughts on this @conda-forge/core ?

Copy link
Member

@h-vetinari h-vetinari left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Basically LGTM!

recipe/migrations/cuda118.yaml Outdated Show resolved Hide resolved
Comment on lines +36 to +38
- quay.io/condaforge/linux-anvil-cuda:9.2 # [linux64 and os.environ.get("BUILD_PLATFORM") == "linux-64"]
- quay.io/condaforge/linux-anvil-cuda:10.0 # [linux64 and os.environ.get("BUILD_PLATFORM") == "linux-64"]
- quay.io/condaforge/linux-anvil-cuda:10.1 # [linux64 and os.environ.get("BUILD_PLATFORM") == "linux-64"]
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Q: Do we still need these? I thought they are not zippable because cuda_compiler_version does not list them?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

They didn't seem to cause any issues when applying the migration to different feedstocks

Zipping isn't relevant here as this is the ordering section and is only used to preserve the order of these when they do appear

Comment on lines 61 to 62
# non-CUDA arch: cross-compilation (build != target)
- quay.io/condaforge/linux-anvil-cos7-x86_64 # [linux64 and os.environ.get("BUILD_PLATFORM") == "linux-64"]
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Q: Wouldn't this last item be zipped with cuda_compiler_version == 12.0? (I guess I never understood the meaning of ordering in the migrator, even after the CUDA 12 migration. Would be nice if someone could shed some light 🙂)

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

(My very same question from the old days: #4400 (comment))

Copy link
Member Author

@jakirkham jakirkham Sep 1, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Applied this to feedstocks that completed the CUDA 12 migration (so they had both migrators like NCCL). AFAICT it worked without issues

This is due to conda-build reordering values of keys (so losing the original ordering). As a result we have to reorder things after the fact in conda-smithy

In any event as the long term goal is to eliminate the need for different Docker images (as is already happening with CUDA 12.0). Think the docker_image ordering can eventually go away. So don't think it is worth thinking to hard about

Copy link
Member

@h-vetinari h-vetinari left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM thx!

@jakirkham jakirkham merged commit 91edef5 into conda-forge:main Sep 1, 2023
2 checks passed
@jakirkham jakirkham deleted the add_cuda_118_migr branch September 1, 2023 21:43
@jakirkham
Copy link
Member Author

Thanks all! 🙏

Going to go ahead and get this started then. We can follow up on anything else separately

@leofang
Copy link
Member

leofang commented Sep 2, 2023

I noticed that cusparselt is marked migrated, but it isn't, likely because it has this skip condition

skip: true  # [ppc64le or cuda_compiler_version not in ("11.2", "12.0")]

and the bot can't see any diff before/after applying the migrator. This would likely happen to other feedstocks like cupy too.

@jakirkham
Copy link
Member Author

Are there differences in the cusparselt binary needed for the CUDA 11.8 case?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Bump minimal CUDA version to CUDA 11.8?
3 participants