-
Notifications
You must be signed in to change notification settings - Fork 530
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Cannot cross-compile Gazelle extension #1913
Comments
Yeah, I am planning to revert our internal diff that upgraded the Gazelle extension. Sorry for not catching this earlier. I thought it was a "pure golang helper", but it turned out to depend heavily on C |
I don't know much about cross-compilation, but I tried hermetic_cc_toolchain, and it seems to work normally (compiling linux_amd64 target on my m1 mac).
(If the target platform is darwin_amd64, we may need to make some changes to gazelle_binary. like https://github.com/uber/hermetic_cc_toolchain/blob/13b27b4e1fa461c57ef12d4e84e0ef34d0e697fb/examples/bzlmod/BUILD.bazel#L23C1-L29C8) |
Thanks @hunshcn for pointing out... Due to the limitation of hermetic_cc_toolchain, cross-compiling C libraries into any darwin platform won't work. Let me update the issue. |
@aignas Updated the description. So the main question here is:
|
The deepest motivation for me to raise that PR is that py_binary is not hermetic. Maybe darwin build can be supported by uber/hermetic_cc_toolchain#10 (comment) ? |
So I guess there are a few alternatives here:
The dependence on the system python is only to launch the interpreter that is bundled with the gazelle plugin, but #1599 needs to be addressed before it can work in I am inclined to go with 3. unless there is a better way to avoid the added dependencies here. @hunshcn, do you know if there are other ways to work around this issue? |
Just retell the current situation. Can't cross-compile to darwin-any, other systems are ok. Because hermetic_cc_toolchain does not support sysroot (even on macos machines). This may be a future direction, but no one provides support. I'm interested, but it's beyond my knowledge. It would be best if someone could achieve this. I think rules_python decides how much cross-compilation should be supported. If the answer is that we need complete cross-compilation ability, then we may need revert. 1 is also an option, but it can be a user operation (I mean patch by the user himself, this part of the code is relatively simple, and I think it is feasible. If revert, I will do it.) |
A related issue from the upstream bindings package is here: smacker/go-tree-sitter#120 This issue is a tough one, the fact that the cross-compilation does not work means that it is not entirely hermetic on all platforms. So I think we should revert the #1895 especially since #1929 is in the works. |
This reverts commit 7fc7962. Fixes bazelbuild#1913
As discussed in #1931, the main idea to address this would be to use https://github.com/go-python/gpython to try and use a pure go implementation to fix this issue. Within the maintainers meeting today we discussed that this issue is affecting a small subset of So to summarize:
I'll pin this issue as it is a known issue. PR's are welcome for this. |
FYI that the Aspect CLI has a Here's a demo for a python project, setup from scratch: |
🐞 bug report
Affected Rule
The issue is caused by the rule: Gazelle extension
Is this a regression?
Yes, before #1895, we were able to cross-compile Gazelle extension by disabling cc toolchain resolution with
--noincompatible_enable_cc_toolchain_resolution
(#1825).Description
The cross-compilation of Gazelle extension is broken after #1895, due to the introduction of dependency on a C library .
The issue can be partly mitigated by using hermetic_cc_toolchain, which supports cross-compiling into Linux, but we still cannot cross-compile from Linux into any macOS, or from darwin_arm64 into darwin_amd64.
This is technically not a problem specific to the Gazelle extension, but a problem of the lack of C toolchain support. However, are we better off by replacing a Python dependency with a C dependency?
🔬 Minimal Reproduction
From the
gazelle
directory of this rules_python repo on a Linux or Apple M1 machine:🔥 Exception or Error
🌍 Your Environment
Operating System:
Output of
bazel version
:7.1.2
Rules_python version:
730a2e3
Anything else relevant?
We can register a hermetic_cc_toolchain by adding this to MODULE.bazel file:
Then from a Linux machine:
The text was updated successfully, but these errors were encountered: