Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Cannot upgrade to CUDA.jl 5 #1500

Closed
simonbyrne opened this issue Oct 13, 2023 · 1 comment
Closed

Cannot upgrade to CUDA.jl 5 #1500

simonbyrne opened this issue Oct 13, 2023 · 1 comment
Labels
bug Something isn't working
Milestone
O1.1.2: 6 SYPD fo...

Comments

@simonbyrne
Copy link
Member

Describe the bug

When we upgraded to CUDA.jl 5, non-init jobs started triggering precompilation (which then triggere a race condition in deleting a scratch directory, which threw an error due to a bug fixed by JuliaLang/julia#50842, which is not yet in a release version of Julia).

This was caused by adding the CUDA driver library as an "include_dependency" to CUDA_Runtime_jll (JuliaGPU/CUDA.jl#1877), which meant that if we loaded CUDA on a node without the driver available, it would trigger recompilation.

I proposed a fix in JuliaPackaging/Yggdrasil#7523, but it needs more work.
@simonbyrne simonbyrne added the bug Something isn't working label Oct 13, 2023
This was referenced Oct 17, 2023
Open
Merged
bors bot added a commit that referenced this issue Oct 18, 2023
@bors @simonbyrne
Merge #1501
b9a357b 
1501: Upgrade to CUDA.jl 5 r=simonbyrne a=simonbyrne

#1500 should now be fixed.

Co-authored-by: Simon Byrne <simonbyrne@gmail.com>
@simonbyrne
Copy link
Member Author

Closed by #1501

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
bug Something isn't working
Projects
None yet
Milestone
O1.1.2: 6 SYPD for moist Held Suarez
Development

No branches or pull requests
1 participant
@simonbyrne