make the main zig executable no longer depend on LLVM, LLD, and Clang libraries #16270
Comments
andrewrk
added
the
proposal
|
I see the 1.0.0 milestone for this, is it meant to be a blocker for 1.0? or simply the goal |
|
The milestone, on an issue labeled "proposal", means that a decision must be made to accept or reject that proposal before tagging the release corresponding to that milestone. For an issue labeled "accepted", the milestone means that it must be implemented by then. So, if this proposal is accepted, then I will evaluate at that time which milestone to move it to. |
IMO, this is the biggest question. One of the most compelling reasons to use Zig is runtime performance of software written in Zig. Without LLVM's optimization passes, what will that look like? |
Zig will continue to implement optimization passes of its own over time and get faster. |
|
So here the projects that depend on the ability to compile C++ that I currently developing:
And also some NDA game platform that have C++ only API that will require some C++-to-C glue code to be compiled, but obviously not implemented. To me, the ability to seamlessly build any C, C++ and Obj-C is a big selling point of the Zig toolchain even if it is behind a optional flag to enable LLVM and Clang when compiling Zig. A part of the hype momentum around Zig is due to that fact. If this happens, I think I will remove my donation to the Zig Software Foundation. TL;DR: Lots of libraries in the game development world (closed or open source) require the ability to compile C++. |
|
I think this proposal would hurt the Zig ecosystem more than it would help it, due to several reasons:
Imho, this proposal strongly violates the
idea, as the current direction we're heading is a really good unified native build environment based on a single static executable that can serve projects in arbitrar, sizes, shipping compilers for several major languages, a huge ass support for targets and a build system, making work in systems programming fun, even if one doesnt use Zig as a language. We are on a good way to replace a huge list of tools with a single executable, making contributions to projects build on Zig fun, easy and platform independent. When this proposal is accepted, in addition to Zig one will need to have the following tools installed:
We can potentially replace all of those tools wit a single, equally powerful executable, making the live easier for all native devs out there personal projects that would be affected:
|
|
I'm still a beginner I believe, but if it is at all worth it for me to give my point of view, Zig's ability to replace all of the build tools mentioned above is a big reason I was interested in Zig in the first place. I struggled a lot with all the different build systems and Zig is a really refreshing breath of fresh air. I have two personal projects that use zgui heavily and I had planned on continuing. |
|
With Mach engine we have two dependencies that would be very, very painful to remove (would set us back years):
It will be a long time before Zig's SPIRV backend is capable enough to generate non-GPGPU shaders for graphics APIs (if ever, since it would likely require major language changes) - so I don't see a way for us to escape these aside from replicating what these two projects -- LLVM forks -- do on our side. Every other C++ dependency I believe we could safely escape from. |
|
A positive long-term effect of this change is that it would push us as a community away from wrapping C++ code and towards more pure-Zig solutions. Many of the comments in this thread are about people using zgui, wwise, zig-gamedev, assimp, and other C++ libraries wrapped with Zig. It gives you a leg up in the short term, but I worry in the long-term that people's gamedev experiences coming to Zig will be 'initially I saw a nice language... then I encountered the guts of the libraries I was told to use were large, clunky C++ codebases' |
|
Also a beginner, but I think being able to use Zig as a C++ build system (which is what I use for all my private C++ projects) is an invaluable feature to me and I believe many other people. The simplicity of Zig, having a compiler and a build system contained within a single executable (with the added bonus of being easily cross platform), is really cool, and it would be kinda unfortunate to see that feature be removed as an effect of removing clang and friends. However, if this does go through, the ability to fallback to generating |
|
One of my friends pointed out that nothing stops you from invoking clang in a build.zig to compile C++ dependencies, even if Zig stops including clang. I wonder how much of a problem that would really be for these projects? |
|
doing that would be an additional dependency without the ease of zigs cross compilation |
|
Hmm, the more I think about this proposal the less I like it. I'm feeling it would be better to make the LLVM backend non-default (that is, switch |
|
I think I can speak for all gamedevs by saying that removing C++ compilation would be a disaster. Too many amazing existing gamedev libraries and tools are built on C++ that disallowing easy use would strangle adoption in that field, as well as complicate existing projects. dear imgui is the obvious example but it's not the only one. |
|
It's easy to imagine that in the long term, this change will push us towards "rewrite it in zig" with all the benefits that would entail, but the downside is that the existing corpus becomes inaccessible; limiting our options heavily even once zigs ecosystem matures |
|
I think llvm is needed until ecosystem of pure-zig library is very very mature and rich. Yeah we want faster compiling speed and smaller tarball, but not at the risk of losing one-zig-to-rule-all. |
|
I love the ambition of this proposal, but to reiterate what has already been stated, losing c++ compilation would be losing one of the main selling points of zig. I was drawn to zig in part because it reduces the hellishness of depending on c/c++ projects. Zig having a c++ compiler inside it also has the benefit of there being less c++ to have to deal with, less python to have to deal with, less cmake to have to deal with. On the other hand, the core of this proposal has too many benefits for it to be rejected entirely. I think reducing the scope would be beneficial. How about:
|
|
I totally get the desire to get rid of huge third-party dependencies that bring a lot of baggage. There's also something to be said for avoiding an LLVM monoculture in the programming language space. Even so, I see this as a net negative for users. What drew me to Zig in the first place was the pragmatic approach of acknowledging that there is a world outside of Zig that needs to be interoperated with for the foreseeable future and even providing a best-in-class cross-compilation experience along with that. I built my project integrating Zig build support with the .NET/MSBuild ecosystem on that selling point. On the whole, I think this proposal would be an unfortunate (if well-intentioned) bait-and-switch, considering the Zig website for a while has advertised this:
In addition, these blog posts drove a lot of attention to Zig in the past: Just to be super clear: I don't mean to insinuate bad faith or anything of the sort here. But I think it's fair to say that you have to contend with the fact that this proposal would pull features that are not only usable today, but are also prominently advertised. All that said... assuming this is even remotely practical, maybe there's a potential middle ground: Would it be possible for Zig to continue to use the Clang frontend to provide C-family support, but rip out the LLVM IR lowering and replace it with lowering to ZIR/AIR? (I guess this is more or less how Aro would be integrated too?) If this could be done, the codegen dependency on LLVM could be killed, achieving at least some of the goals of this proposal. There's probably also no reason to keep LLD support around as long as zld can catch up, so that eventually goes too. And users remain happy. Some of the build and distro woes would remain, of course, but, that's compromise. |
|
I’m fully in favor of making it possible to use Zig without any LLVM components, but I agree with many of the comments here that it’s important for Zig to maintain the capabilities that it currently has in terms of cross-compilation, compiling C/C++ code seamlessly, and generating maximally performant binaries. These factors are big drivers of Zig’s adoption, and I fear that damaging them (even temporarily as in the case of code generation quality) would seriously hurt Zig’s future. Personally, I work in the robotics space, where C++ is the dominant language for many libraries and frameworks. I think Zig has a lot of potential in this space, but being able to integrate with existing libraries is absolutely essential for adoption. |
This comment was marked as resolved.
This comment was marked as resolved.
|
I'll start by stating my opinion: the C language frontends are super important to me for Zig to maintain. I agree with others on the point that Zig being a C/C++ compiler is a big point of attraction that brought me to the language. I started with loving that idea, and ultimately fell in the love with the language, and now I use both. I don't have anything more to add to that that the others above haven't. I'll add my own personal experience. I have many personal Zig projects, but my biggest one that people tend to know about in the community is my terminal emulator. There are two important dependencies that would be impacted by this:
Andrew, your dislike of C++ is well known! I don't love it either (to put it kindly). If, as an audacious goal, you wanted Zig to lower C++ usage, I think Zig being able to compile existing projects and enable iterative migration away from C++ to Zig is the way to do it. I think if Zig doesn't support C++, the C++ "people" would just avoid Zig altogether. |
|
Main gripe seems to be the need/desire to be able to compile C++ due to the mountain of pre-existing libraries that are already being used in the Zig ecosystem. For this proposal to satisfy people and not to break a huge selling point for Zig, a C++ (and ObjC?) compiler in Zig would have to be up-streamed. Dunno what madman will be the one to write that tho. At the very least, any kind of serious effort towards removing LLVM should be done after reaching 1.0. That said, I don't think divorcing from LLVM itself is a bad idea, if the Zig toolchain itself can slowly grow to replace its functionality, just so long as it sticks around until opting out of it doesn't result in Zig losing out on its existing features. |
|
While I can see this being a good thing in theory I like others have concerns over what it will do more short term. Zig to me is expected to be a highly performant language competitive with languages like C, if it is not performant then it impacts my ability to use it for writing the projects I am making in it because it simply will be a worse option in practice for such things. Perhaps this could be mitigated if the C backend becomes fully functional to allow projects to compile to C code and have those compile with typical C compilers for when performance is needed, but otherwise I do not think this would be well-advised until some sort of reasonable performance guarantee can be made. Even with the Zig->C->Machine Code process I feel like you'd be losing some optimization potential as no longer would Zig be able to annotate LLVM IR directly and would instead be confined by whatever C can express language wise so that might not even be a foolproof option either (though at the very least it'd hopefully put it on the level of C for most things). Edit: Apparently I've been told this is what the bitcode support could be used for, I've never used LLVM bitcode stuff myself before but yeah if that's a supported target and still gets focus knowing people will be using it to generate higher quality optimized code until Zig can compete maybe this is less of an issue, albeit a bit more convoluted in how a project would have to be compiled. Frankly with how insanely complex x86 is and how much work has gone into LLVM over the years I am doubtful Zig would ever reach the same standard of performance. Other larger languages like Rust which aren't even as entangled with C++ haven't tackled this sort of challenge yet either for instance despite the some similar motivation to and more developer resources at their disposal, and to me that is not a very promising sign for its feasibility (though of course Zig could always be the first thing to prove this long-held mindset of LLVM being impossible to replace wrong...). Also as an aside while I am not as invested in the C++ compilation support as others may be I do think that it'd hurt a lot of projects. Being a gamedev myself losing the ability to use ImGui would be unfortunate as others have mentioned, and personally I also use Tracy in some of my Zig projects which is also C++-based. It wouldn't be too hard to just compile these libraries and link to their binaries (or use a system library I suppose) but still that just makes Zig a bit more pain to interface with this stuff. |
|
Another thing that I don't think has been mentioned/considered yet, if this proposal were to go through, and that were to happen around or after the time that 1.0 is released, it could cause a split in the zig community. |
|
I agree that apart from the mentioned problems with LLVM, it also struggles with the a baggage of legacy code, e.g. in TableGen modules, in how Clang is tied to LLVM compared to the newer compilers that use middle-level intermediate languages, how story of migration from FastISel and SelectionDAG to GlobalISel stalled for many years and isn't really progressing for all supported architectures, and so on and so forth. Using C++ language for writing such a complex piece of software doesn't help either. But even Rust didn't dare to get rid of it just yet. Thus, I think it would be prematurely to do that for Zig either. Long-term it might be a worthy goal, but definitely not in the upcoming 5 years or so, in my opinion. Just my 2c. I second @yujiri8 here, having optional LLVM target for many years while working on the Zig backends would be a perfect strategy, reducing the maintenance of LLVM parts and providing room for experimentation and optimization of the mainstream targets directly in the Zig code: ARM64 and x86_64, probably RISC-V in the future, if it really takes off. GHC (Haskell) uses the similar approach for at least a decade already, it seems to work for them. P.S. Why PE format is not in the list for linker? |
|
Maybe the best long-term solution would be to offer some kind of plugin system for the zig compiler. Pros:
Cons:
|
|
At least create a new external project to mantain the goodness of "zig cc c++ bpf..." and builtin platform SDKs .h and cross compilation!! |
|
I just wanted to share a little "build.zig" hack showing you can already today mix in Zig and system LLVM to make optimized binaries. Here Zig compiles in Debug mode to LLVM bytecode, then system "llvm-opt" produce an optimized object file. That way LLVM doesn't need to be distributed through Zig, and can be an external dependency managed through the package manager, and it will probably only require a small build.zig change for users. |
|
I have no idea about compilers. And I'm not sure I get the whole discussion here. But are we losing anything of todays features? Especially the great cross-compilation (incl. WASM as target). Or the smooth C integration and replacement of pure C compilation. This is important to me because I use Zig also together with Go for cross-compilation. 🤔 |
|
@sebastianbauer20 No, no features will be lost. See #16270 (comment) |
|
I think that lot of the fears are misguided, I guess they underline the importance of the use case of compiling everything with zig, but mid-term installing optional components to achieve that is not so cumbersome (clang can cross compile, and if zig build uses it correctly things should work. Long term some things might become an issue if the community does not care about that enough, and the progress of the infrastructure and backends is such that it becomes incompatible with llvm, and difficult to maintain the LLVM compatibility. Short term zig becomes more focused, less reliant on other things, and can attract people interested in working on an efficient and fast compiler infrastructure. |
|
At my company, we build database backed web applications, system software and embedded software (for a variety of architectures) and Zig represent(ed) a really exciting future for me. We generally write C, JavaScript & Python. Embedded targets often include C/C++ libraries that we need to compile. I've been watching and hoping to see Zig cross that v1.0 threshold which would allow us to finally dig in and use it everywhere we can. I've been putting off some major projects (for some years now), based on this hope. If we lose the ability to (a) easily cross compile into embedded architectures, and (b) compile existing C/C++ libraries, Zig is unlikely going to be something I'm able to invest in. I'm not even a little bit interested in fighting with toolchain incompatibilities across our dev and build environments, so BYO-LLVM isn't an option that I would choose. I've also seen some comments floating around (maybe incorrect?) that the planned, optional LLVM backend may not be supported into the long term as the format doesn't support the Zig plans for async/await. Lastly, this move would be incredible as a long term, big hairy audacious side-project - after v1.0. But as a v1.0 blocker, it makes me super sad. IMO, dropping LLVM is very likely to be a 3-10 year side-quest that drastically reduces the utility and performance of an otherwise absolutely incredible and transformational technology. I have so much feels and admiration for everything you are doing, please reconsider this one. |
@lukebayes You might wanna read #16270 (comment) |
|
@lukebayes LLVM isn't being removed as an option, it's being removed as the default. At least if I understand correctly, Zig will no longer ship with LLVM, but you can easily import a package in your build.zig if your project needs it (for compiling C++, building for less common architectures, etc). Most people will not need LLVM for their projects, which is why it's logical to keep Zig from being reliant on it. |
|
Hey @cryptocode and @mgord9518 thank you for the helpful responses. Unfortunately, I can no longer find the source (maybe I misunderstood the original), but I saw a comment somewhere that indicated one of the reasons for this change, had to do with LLVM not supporting Zig's plans for async/await. This got me concerned that the BYO-LLVM (or LLVM as a package) plan may not be something that will continue working after async/await is brought back. I may not be reading the linked comment correctly, but I see language like, "could be" and I'm not sure if this is just used because it's a proposal, or if this is meant to say that, "someone else could at some point...", and then I see, "independently maintained," and it's really not clear to me if this LLVM package is something the Zig project would introduce and manage or if it's something I would need to create and carry. I understand that LLVM represents an enormous amount of pain and limitation, but it seems to me that externalizing it would make that pain and those problems even more challenging, especially if it's managed by someone who's not on the core Zig team. To be honest, I'm feeling really bad because I love what Zig represents, I'm blown away by the incredible outcomes this project has been able to accomplish and I fully believe that this group will make this work for the common use cases. My concern is for the breadth of less common targets, and embedded in particular, but I'm also saddened by how much calendar time this is likely to add to a v1.0 release. |
|
I will be crystal clear about this: LLVM support is not going away. If we end up reintroducing async/await to the language in a form which LLVM cannot support, then perhaps you will not be able to use async/await when utilizing the LLVM backend; but that backend will remain, and will certainly remain the default backend for release builds for a long time. |
|
Regarding the planned external LLVM package, I can't say for sure, but I would anticipate that this is implemented and supported by the core Zig team. Regardless of the path our self-hosted backends take, LLVM support will remain important for years to come, and we will not want it to regress -- for instance, we will still be using it to create release builds of the compiler for many years. The point about independent maintainability is more about separation of concerns: upgrades to LLVM can happen independently of compiler versions, and contributors do not need to worry about LLVM when working on the compiler. It is still intended that LLVM be well-supported, easy to use, and always functional. |
|
This is really helpful clarification @mlugg! The only issue remaining, is how this could impact the timeline to v1.0. I'll try to work on my own patience for that. Thank you for the helpful responses folks. |
The self-hosted backend work would happen regardless of whether or not LLVM is removed as a dependency. Self-hosted backends are required for fast debug builds, incremental compilation, hot reloading, and all that good stuff :) Also, there are many people who work on Zig, and not all of them work on the backends! Frontend work will still happen quickly, so I wouldn't worry about them impacting the timeline too much. Zig will still release at some point within the next 37 years, probably ^-^ |
Upgrades the LLVM, Clang, and LLD dependencies to LLVM 18.x Related to #16270
Just a quick comment: it should not be zig or LLVM's responsibility to fix issues in linux distribution and package managers. All that these should provide is a working way to compile / bootstrap. If that works then EVERY downstream problem is not the responsibility of zig. I have seen projects that try to care for distributions and package managers, and I understand their rationale, but I really think it should be solely the responsibility of these distributions and package managers to fix their OWN problems, provided that e. g. zig can be compiled as-is. |
Upgrades the LLVM, Clang, and LLD dependencies to LLVM 19.x Related to #16270 Big thanks to Alex Rønne Petersen for doing the bulk of the upgrade work in this branch.
Upgrades the LLVM, Clang, and LLD dependencies to LLVM 19.x Related to ziglang#16270 Big thanks to Alex Rønne Petersen for doing the bulk of the upgrade work in this branch.
and others
This issue is to fully eliminate LLVM, Clang, and LLD libraries from the Zig project. The remaining ties to these projects are as follows:
optimization passesnot actually a prerequisite for this proposal. The clang package mentioned below will beable to provide LLVM optimizations as well.
upstream Aro and use it for translate-c instead of clang #16268not actually a prerequisite for this proposalWhen the compiler is built without LLVM, use Aro to compile C code instead of Clang #16269not actually a prerequisite for this proposalCompiling C++, Objective-C, Objective-C++, etc. files.Solve with a clang package that can be depended on via the zig build system.zig cc,zig c++,zig translate-cand other subcommands without a clang/llvm dependency in the compiler binary #20875@cImportto the build system #20630Note that there would still be an LLVM backend for outputting
.bcfiles (#13265), but the Zig compiler would lack the capability to compile .bc files into object files. LLVM or Clang would need to be installed and invoked separately for that use case.In exchange, Zig gains these benefits:
Please read my other comments in this issue before replying:
The text was updated successfully, but these errors were encountered: