-
Notifications
You must be signed in to change notification settings - Fork 30.1k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Proposal: gypkg for GYP dependency management #7240
Comments
Oh, and also it will reduce future size of the Node.js repo tree, since we won't be adding way too much changes on each dep update anymore. |
I like the work vey much, but as posted on twitter I am very strongly opposed to introducing another js-foreign dependency or something that increasing organizational dependency to one. I think it's rather time to build our own tooling since we are big enough and can agree that most stuff out there isn't really what we are looking for out of the box and we are just piggy-backing on the Google stuff. Since node is proliferated really much nowadays I think we would also do the software world a favor and build more cross platform tooling that anyone could use for other purposes too without having things like python installed correctly. Gyp seems orthogonal to that goal. I am working on something wrapping shelling out to |
Ohai! Here are my 2 cents, FWIW. This project is nice in that it reduces bolireplate and makes it easier to use GYP for Node and other projects. Alas, it still requires Google's GYP, which doesn't change much of the status quo. If we backtrack a little, it seems that the dependency on Python was one of the things people don't seem to like. Not to mention the fact that GYP doesn't work on Python 3, which adds more to the confusion. Google itself it transitioning to GN, which just generates Ninja buildfiles, and lets Ninja do the work. Personally, I think this a good approach. My little contribution to help solve this problem was GYN a while ago. It's basically Google's GYP, with a single generator (Ninja) and ported to Python 3. I could compile Node itself. It was met with some resistance since it's still Python, which I do understand. That said, I think a good way forward would be to port GYP to JS but make it just generate only Ninja files. We'd still be using GYP files, which Node is kinda heavily invested on it by now, and Ninja is super small and self-contained, so bundling it in shouldn't be much of a problem. I briefly mentioned this last part to @eljefedelrodeodeljefe on Twitter. The idea being that Ninja could be replaced later, saving that stage and going directly from GYP files to binaries if need be. Personally, I'd take the Unix approach of one tool one job and let Ninja do the compilation. One thing I wonder though, is that if this looks like a bigger problem than it is because we don't have a good story for binary packages. If we did then only the people who build the package would need the build tools, and (specially those on Windows) others wouldn't have much trouble. |
I don't think this is a good thing. Currently, Node.js repo is self-contained. This guarantees that even 10 years from now you can easily check out an ancient commit and build it. If it had dependencies in other repos, then it would depend on whether these repos still exist. |
I generally share @saghul's point of view. Adding on top: I now and then attempt to disconnect gn from the rest of the google stack which is perfectly possible but very hard in reality without hard forking; it's just moving too quick. It's relatively simple to build a standalone version of it; and I really like the idea of using gn-files/ninja as the "future" native compilation stack but I have a hard time seeing it happen if we stick with python. |
Some of this discussion can move to nodejs/CTC#2 which is attempting to take a broader perspective on what to do with GYP in core and the ecosystem. |
It's a chicken and egg problem. I think people should be able to build |
Very interesting idea. The issue is that Duktape doesn't support most node standard modules like |
Yes, it doesn't support any modules out of the box. It does have a |
FWIW, I'm working on JS port of GYP right now. Will keep you updated. |
@kzc chicken-and-egg problem does not apply in this case. It's a common misconception. That is what cross compilation and downloading binaries is for. Go can build itself, so can gcc. We are no different. |
@eljefedelrodeodeljefe Comparisons with Go don't apply as node can't be built with node alone. A C++ compiler is really the key prerequisite of building node. That's what builds v8, libuv, openssl and node itself. That's what I like about @saghul's proposal bootstrapping via Duktape - it eliminates the python dependency in favor of the C++ dependency that node can't avoid. |
@kzc that's wrong. Any build tool needs a C/C++ compiler. In the end, no matter what tool you are using, you are calling out to your platforms compiler binary. So why using a dependency for making a call to |
@kzc fun fact: the reason why you want to use some toolchain is to avoid writing something like the below just to compile an "C:\Program Files (x86)\Microsoft Visual Studio 14.0\VC\bin\amd64\CL.exe" /c /I"C:\Program Files (x86)\Windows Kits\10\Include\10.0.10240.0\ucrt" /I"C:\Program Files (x86)\Microsoft Visual Studio 14.0\VC\atlmfc\include" /I"C:\Program Files (x86)\Microsoft Visual Studio 14.0\VC\include" /I"C:\Users\jefe-win\.node-gyp\6.2.1\include\node" /I"C:\Program Files (x86)\Microsoft Visual Studio 14.0\VC\include" /I"C:\Users\jefe-win\.node-gyp\6.2.1\src" /I"C:\Program Files (x86)\MSBuild\Microsoft.Cpp\v4.0\V140" /I"C:\Program Files (x86)\MSBuild\Microsoft.Cpp\v4.0" /I"C:\Program Files (x86)\Windows Kits\8.1\Include\um" /I"C:\Program Files (x86)\Windows Kits\8.1\Include\shared" /I"C:\Program Files (x86)\Windows Kits\8.1\Include\winrt" /I"C:\Users\jefe-win\.node-gyp\6.2.1\deps\uv\include" /I"C:\Users\jefe-win\.node-gyp\6.2.1\deps\v8\include" /I..\node_modules\nan /Zi /nologo /W3 /WX- /Ox /Ob2 /Oi /Ot /Oy /GL /D NODE_GYP_MODULE_NAME=addon /D WIN32 /D _CRT_SECURE_NO_DEPRECATE /D _CRT_NONSTDC_NO_DEPRECATE /D _HAS_EXCEPTIONS=0 /D BUILDING_V8_SHARED=1 /D BUILDING_UV_SHARED=1 /D BUILDING_NODE_EXTENSION /D _WINDLL /GF /Gm- /MT /GS /Gy /fp:precise /Zc:wchar_t /Zc:forScope /Zc:inline /GR- /Fo"Release\obj\addon\\" /Fd"Release\obj\addon\vc140.pdb" /Gd /TP /wd4351 /wd4355 /wd4800 /wd4251 /errorReport:queue /MP ..\addon.cc
|
Great. Then we're in agreement on that point. Using node to bootstrap node is a needless complication. It can be done, but it's not worth the trouble when other ways are simpler. |
-.- I bail the conversation here. |
@kzc the relevant question here isn't bootstrapping node itself, rather being able to compile node native modules on a system where node is installed. |
@jbergstroem Ah, okay. My mistake. But perhaps this JS port of GYP initiative could have applications to bootstrapping node itself if successful. |
Port of GYP (WIP): https://github.com/indutny/gyp.js |
This issue has been inactive for sufficiently long that it seems like perhaps it should be closed. Feel free to re-open (or leave a comment requesting that it be re-opened) if you disagree. I'm just tidying up and not acting on a super-strong opinion or anything like that. |
Hello @nodejs/build @nodejs/collaborators @nodejs/ctc !
I have been recently working on gypkg, which is something that I have craved for a very long time. What do you think about trying it out for Node.js core, and maybe libuv (cc @saghul) ?
This will allow us to put every GYP dependency to a separate repo, and manage them with a semver in a
node.gyp
file. Should be pretty neat, and should make the upgrade process easier and cleaner.Thank you,
Fedor.
The text was updated successfully, but these errors were encountered: