Upgrade taichi to use LLVM15 #6447

ailzhang · 2022-10-26T13:45:59Z

Thanks to @python3kgae we have pretty nice LLVM15 compatible code across the codebase now. And using an old version of LLVM (LLVM10) have limited us from applying advanced optimizations as well as adapting to newer hardware. So we're planning to upgrade taichi to depend on LLVM15. The overall process is kinda simple, fix the failed tests and enable TI_LLVM_15 in every CI job we have, and finally remove all the ifdef TI_LLVM_15 in the codebase.

A rough list of tasks:

linux cpu ([ci] Swtich CPU jobs to use prebuilt LLVM15 #6435)
linux cuda ([ci] Switch linux gpu job to use LLVM 15 #6465)
mac cpu （[ci] Move macos10.15 job to build with LLVM 15 #6634, [ci] Move m1 job to use build with LLVM 15 #6483）
windows cpu
windows cuda (@python3kgae already did this, thanks!)
do we want to upgrade the compiler to compile .bc files? ([build] Support any clang<=15 as CLANG_EXECUTABLE #6678)
remove all the ifdef TI_LLVM_15! ([build] Delete all llvm10 code and TI_LLVM_15 macro #6685)
release prebuilt llvm binaries across different platforms & update docs (or shall we add llvm as a submodule?) ([doc] Update prebuilt llvm15 links in dev install doc #6648)

The text was updated successfully, but these errors were encountered:

python3kgae · 2022-10-27T01:17:44Z

windows cpu
windows cpu is enabled too.

do we want to upgrade the compiler to compile .bc files?
It would be nice to upgrade clang to 15, that will generate more test cases for opaque ptr.

release prebuilt llvm binaries across different platforms & update docs (or shall we add llvm as a submodule?)
rust add llvm as a submodule. But that will make clean build very slow. Also clone llvm repo will take some time.

ailzhang · 2022-10-27T01:48:33Z

But that will make clean build very slow. Also clone llvm repo will take some time.
+1 that's the main reason we ship prebuilt llvm binaries on different platform IIRC. Building llvm from source does take a long time so we might have to stay on the current prebuilt approach ...

…6458) Issue: #6447 ### Brief Summary Previously we inject some cuda kernel metadata after linking, that was problematic since when you modify metadata for a llvm::Function you'll have to use the exactly same LLVMContext it was created in. Thus we failed verifyModule check from LLVM15. And we happened to only TI_WARN there so didn't notice this issue. ![Screenshot from 2022-10-26 17-10-43](https://user-images.githubusercontent.com/5248122/198422336-fd4ed5e8-aa3c-4baf-841a-95bba5c148e0.png) This PR fixes it by moving the mark as CUDA step up to parallel compilation so that it was guaranteed to be in the same LLVMContext.

Issue: #6447 ### Brief Summary

Issue: #6447 ### Brief Summary Just trying my luck in CI since my local mac is on macOS12.

Issue: #6447 ### Brief Summary Deleting this job first since we're almost there, deleting this one can make the windows queue shorter.

Issue: #6447 ### Brief Summary Getting closer...

Issue: #6447 ### Brief Summary Updating links in dev install doc. Nit: we still depend on clang-10/clang-11 to produce runtime.bc files and that part will be updated once those are cleaned up.

For taichi devs, if you hit any issues with this option, please turn it off temporarily and open an issue. Issue: #6447 ### Brief Summary There're a few followup cleanups after PR but let's turn it on by default to shake out all the problems.

ailzhang · 2022-11-18T04:56:45Z

FYI #6649 turned on TI_LLVM_15 by default, we'll make sure nightly runs well and then remove all the #ifdef TI_LLVM_15 in the codebase.

Other tasks I'm looking into atm:

Can we use llvm15 from brew on macs? (for linux it's a bit tricky since llvm15 isn't available for all unix distributions yet)
Can we upgrade compiler for runtim.cpp and handle it better? (yes we can, [build] Support any clang<=15 as CLANG_EXECUTABLE #6678, [build] Support any 13<clang<=15 as CLANG_EXECUTABLE on macOS #6682)

Issue: #6447 ### Brief Summary Got the following error message when building manylinux2014 wheel since the base docker image doesn't have `unzip`: ``` 2022-11-18T06:14:27.6476325Z + unzip taichi-llvm-15-linux.zip 2022-11-18T06:14:27.6485595Z /home/dev/taichi/.github/workflows/scripts/unix-build.sh: line 13: unzip: command not found ``` reference log: https://pipelines.actions.githubusercontent.com/serviceHosts/ebf0abad-1193-4ff9-9c60-da990d1eacd9/_apis/pipelines/1/runs/56327/signedlogcontent/20?urlExpires=2022-11-18T06%3A42%3A59.6939664Z&urlSigningMethod=HMACV1&urlSignature=pRp%2BtWL7jg6DXBr5bfgq0r5KqUwjrJ6bAM4WAXwP5S0%3D

ailzhang · 2022-11-21T23:55:00Z

FYI I'd consider LLVM15 upgrade is done at this point. There're two followups that might be helpful but with lower priority:

Can we use llvm15 from brew on macs? (for linux it's a bit tricky since llvm15 isn't available for all unix distributions yet)
Revamp developer install doc since it can be much simpler due to [build] Support any clang<=15 as CLANG_EXECUTABLE #6678 and [build] Support any 13<clang<=15 as CLANG_EXECUTABLE on macOS #6682.

I'll open separate issues to track these followups.

Issue: taichi-dev#6447 ### Brief Summary Just trying my luck in CI since my local mac is on macOS12.

Issue: taichi-dev#6447 ### Brief Summary Deleting this job first since we're almost there, deleting this one can make the windows queue shorter.

Issue: taichi-dev#6447 ### Brief Summary Getting closer...

Issue: taichi-dev#6447 ### Brief Summary Updating links in dev install doc. Nit: we still depend on clang-10/clang-11 to produce runtime.bc files and that part will be updated once those are cleaned up.

For taichi devs, if you hit any issues with this option, please turn it off temporarily and open an issue. Issue: taichi-dev#6447 ### Brief Summary There're a few followup cleanups after PR but let's turn it on by default to shake out all the problems.

Issue: taichi-dev#6447 ### Brief Summary Got the following error message when building manylinux2014 wheel since the base docker image doesn't have `unzip`: ``` 2022-11-18T06:14:27.6476325Z + unzip taichi-llvm-15-linux.zip 2022-11-18T06:14:27.6485595Z /home/dev/taichi/.github/workflows/scripts/unix-build.sh: line 13: unzip: command not found ``` reference log: https://pipelines.actions.githubusercontent.com/serviceHosts/ebf0abad-1193-4ff9-9c60-da990d1eacd9/_apis/pipelines/1/runs/56327/signedlogcontent/20?urlExpires=2022-11-18T06%3A42%3A59.6939664Z&urlSigningMethod=HMACV1&urlSignature=pRp%2BtWL7jg6DXBr5bfgq0r5KqUwjrJ6bAM4WAXwP5S0%3D

ailzhang added the potential bug Something that looks like a bug but not yet confirmed label Oct 26, 2022

ailzhang changed the title ~~LLVM upgrad~~ Upgrade taichi to use LLVM15 Oct 26, 2022

ailzhang mentioned this issue Oct 27, 2022

[llvm] Fix broken llvm module on CUDA backend when TI_LLVM_15 is on #6458

Merged

ailzhang mentioned this issue Oct 28, 2022

[ci] Switch linux gpu job to use LLVM 15 #6465

Merged

neozhaoliang assigned ailzhang Oct 28, 2022

ailzhang added a commit that referenced this issue Oct 31, 2022

[ci] Switch linux gpu job to use LLVM 15 (#6465)

b1b7fc8

Issue: #6447 ### Brief Summary

ailzhang mentioned this issue Oct 31, 2022

[ci] Move m1 job to use build with LLVM 15 #6483

Merged

ailzhang added a commit that referenced this issue Nov 17, 2022

[ci] Move m1 job to use build with LLVM 15 (#6483)

ede76d2

Issue: #6447 ### Brief Summary Just trying my luck in CI since my local mac is on macOS12.

This was referenced Nov 17, 2022

[ci] Move macos10.15 job to build with LLVM 15 #6634

Merged

[ci] Delete windows llvm10 ci job #6639

Merged

ailzhang added a commit that referenced this issue Nov 17, 2022

[ci] Delete windows llvm10 ci job (#6639)

1fba52f

Issue: #6447 ### Brief Summary Deleting this job first since we're almost there, deleting this one can make the windows queue shorter.

ailzhang added a commit that referenced this issue Nov 17, 2022

[ci] Move macos10.15 job to build with LLVM 15 (#6634)

9c320cf

Issue: #6447 ### Brief Summary Getting closer...

This was referenced Nov 17, 2022

[doc] Update prebuilt llvm15 links in dev install doc #6648

Merged

[build] Turn TI_LLVM_15 on by default #6649

Merged

ailzhang mentioned this issue Nov 18, 2022

[ci] Install unzip in manylinux build #6658

Merged

This was referenced Nov 21, 2022

[build] Support any clang<=15 as CLANG_EXECUTABLE #6678

Merged

[build] Support any 13<clang<=15 as CLANG_EXECUTABLE on macOS #6682

Merged

[build] Delete all llvm10 code and TI_LLVM_15 macro #6685

Merged

ailzhang closed this as completed Nov 21, 2022

This was referenced Nov 21, 2022

Use LLVM15 from brew on Macs #6695

Open

[doc] Revamp developer install documentation #6696

Open

[build] Use a separate prebuilt llvm15 binary for manylinux environment #6732

Merged

This was referenced Nov 25, 2022

[build] Cherrypicking #6732 into v1.3.0 release branch #6735

Merged

[doc] Update dev install about clang version #6759

Merged

quadpixels pushed a commit to quadpixels/taichi that referenced this issue May 13, 2023

[ci] Move m1 job to use build with LLVM 15 (taichi-dev#6483)

924e6d6

Issue: taichi-dev#6447 ### Brief Summary Just trying my luck in CI since my local mac is on macOS12.

quadpixels pushed a commit to quadpixels/taichi that referenced this issue May 13, 2023

[ci] Delete windows llvm10 ci job (taichi-dev#6639)

869b88b

Issue: taichi-dev#6447 ### Brief Summary Deleting this job first since we're almost there, deleting this one can make the windows queue shorter.

quadpixels pushed a commit to quadpixels/taichi that referenced this issue May 13, 2023

[ci] Move macos10.15 job to build with LLVM 15 (taichi-dev#6634)

1a3e898

Issue: taichi-dev#6447 ### Brief Summary Getting closer...

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Upgrade taichi to use LLVM15 #6447

Upgrade taichi to use LLVM15 #6447

ailzhang commented Oct 26, 2022 •

edited

Loading

python3kgae commented Oct 27, 2022 •

edited

Loading

ailzhang commented Oct 27, 2022

ailzhang commented Nov 18, 2022 •

edited

Loading

ailzhang commented Nov 21, 2022

Upgrade taichi to use LLVM15 #6447

Upgrade taichi to use LLVM15 #6447

Comments

ailzhang commented Oct 26, 2022 • edited Loading

python3kgae commented Oct 27, 2022 • edited Loading

ailzhang commented Oct 27, 2022

ailzhang commented Nov 18, 2022 • edited Loading

ailzhang commented Nov 21, 2022

ailzhang commented Oct 26, 2022 •

edited

Loading

python3kgae commented Oct 27, 2022 •

edited

Loading

ailzhang commented Nov 18, 2022 •

edited

Loading