Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Building nightly node 12 on alpine is broken since 2018-12-06 (alpine linux) #25095

Closed
lsmoura opened this issue Dec 17, 2018 · 12 comments
Closed
Labels
build Issues and PRs related to build files or the CI. v8 engine Issues and PRs related to the V8 dependency.

Comments

@lsmoura
Copy link

lsmoura commented Dec 17, 2018

I've been building node12 nightly to create alpine linux docker images since December 1st using the source packages available at the nodejs website.

Ever since release nightly20181207cbf0e5a1f4, the build fails with the same error:

  g++ -o /build/node-nightly/src/node-v12.0.0-nightly20181217b54d4a68e3/out/Release/obj.host/v8_libbase/deps/v8/src/base/platform/platform-posix-time.o ../deps/v8/src/base/platform/platform-posix-time.cc '-DV8_DEPRECATION_WARNINGS' '-DV8_IMMINENT_DEPRECATION_WARNINGS' '-DV8_GYP_BUILD' '-DV8_TYPED_ARRAY_MAX_SIZE_IN_HEAP=0' '-DOPENSSL_THREADS' '-DV8_TARGET_ARCH_X64' '-DV8_EMBEDDER_STRING="-node.5"' '-DENABLE_DISASSEMBLER' '-DV8_PROMISE_INTERNAL_FIELD_COUNT=1' '-DV8_INTL_SUPPORT' '-DV8_CONCURRENT_MARKING' '-DDISABLE_UNTRUSTED_CODE_MITIGATIONS' '-DV8_EMBEDDED_BUILTINS' '-DV8_EMBEDDED_BYTECODE_HANDLERS' -I../deps/v8  -pthread -Wall -Wextra -Wno-unused-parameter -m64 -fno-strict-aliasing -m64 -O3 -fno-omit-frame-pointer -fdata-sections -ffunction-sections -O3 -fno-rtti -fno-exceptions -std=gnu++1y -MMD -MF /build/node-nightly/src/node-v12.0.0-nightly20181217b54d4a68e3/out/Release/.deps//build/node-nightly/src/node-v12.0.0-nightly20181217b54d4a68e3/out/Release/obj.host/v8_libbase/deps/v8/src/base/platform/platform-posix-time.o.d.raw   -c
  LD_LIBRARY_PATH=/build/node-nightly/src/node-v12.0.0-nightly20181217b54d4a68e3/out/Release/lib.host:/build/node-nightly/src/node-v12.0.0-nightly20181217b54d4a68e3/out/Release/lib.target:$LD_LIBRARY_PATH; export LD_LIBRARY_PATH; cd ../deps/v8/gypfiles; mkdir -p /build/node-nightly/src/node-v12.0.0-nightly20181217b54d4a68e3/out/Release/obj/gen; python ../tools/gen-postmortem-metadata.py "/build/node-nightly/src/node-v12.0.0-nightly20181217b54d4a68e3/out/Release/obj/gen/debug-support.cc" ../src/objects.h ../src/objects-inl.h ../src/objects/allocation-site-inl.h ../src/objects/allocation-site.h ../src/objects/code-inl.h ../src/objects/code.h ../src/objects/data-handler.h ../src/objects/data-handler-inl.h ../src/objects/fixed-array-inl.h ../src/objects/fixed-array.h ../src/objects/js-array-inl.h ../src/objects/js-array.h ../src/objects/js-array-buffer-inl.h ../src/objects/js-array-buffer.h ../src/objects/js-objects-inl.h ../src/objects/js-objects.h ../src/objects/js-regexp-inl.h ../src/objects/js-regexp.h ../src/objects/js-regexp-string-iterator-inl.h ../src/objects/js-regexp-string-iterator.h ../src/objects/map.h ../src/objects/map-inl.h ../src/objects/name.h ../src/objects/name-inl.h ../src/objects/scope-info.h ../src/objects/script.h ../src/objects/script-inl.h ../src/objects/shared-function-info.h ../src/objects/shared-function-info-inl.h ../src/objects/string.h ../src/objects/string-inl.h
  touch /build/node-nightly/src/node-v12.0.0-nightly20181217b54d4a68e3/out/Release/obj.target/tools/icu/icu_implementation.stamp
  touch /build/node-nightly/src/node-v12.0.0-nightly20181217b54d4a68e3/out/Release/obj.target/tools/icu/icu_uconfig.stamp
  touch /build/node-nightly/src/node-v12.0.0-nightly20181217b54d4a68e3/out/Release/obj.target/tools/icu/icu_uconfig_target.stamp
  touch /build/node-nightly/src/node-v12.0.0-nightly20181217b54d4a68e3/out/Release/obj.host/tools/icu/icu_implementation.stamp
  touch /build/node-nightly/src/node-v12.0.0-nightly20181217b54d4a68e3/out/Release/obj.host/tools/icu/icu_uconfig.stamp
make[1]: *** No rule to make target '../deps/v8/test/torque/test-torque.tq', needed by '2f2d73a3ab02a00709f3dbbaf3638568d55301e2.intermediate'.  Stop.

I'm building the code using the base alpine linux docker image for version 3.8.1, with no custom packages (only official ones required to build the image).

Here's how I'm building from the source:

	./configure --prefix=/usr --fully-static --without-npm
	make -j$(getconf _NPROCESSORS_ONLN)

Every nightly on december before that is build properly.

You can check lsmoura/node:12nightly if you want to see other nightly builds working.

Thanks for all the work from the nodejs developers!

@richardlau
Copy link
Member

I think deps/v8/test/torque/test-torque.tq came with the V8 7.1 update.

deps/v8/test is excluded from the source tarball:

node/Makefile

Line 984 in a6f69eb

$(RM) -r $(TARNAME)/deps/v8/test

cc @nodejs/build-files @nodejs/v8-update

@richardlau richardlau added build Issues and PRs related to build files or the CI. v8 engine Issues and PRs related to the V8 dependency. labels Dec 17, 2018
@lsmoura
Copy link
Author

lsmoura commented Dec 17, 2018

Should I be taking extra steps to prepare the source to build, then? maybe removing the requirement to build deps/v8/test/torque/test-torque.tq from the makefile?

Or maybe the nightly source file is not being built properly?

@richardlau
Copy link
Member

Since it's in test, does the gyp file need to reference it? Maybe try removing

"../test/torque/test-torque.tq",

@richardlau
Copy link
Member

@lsmoura It's a gap in our process -- The nightly builds are built from a git checkout, not the source tarball. I'm not aware that we have any builds that build from the source tarball -- cc @nodejs/build ?

@lsmoura
Copy link
Author

lsmoura commented Dec 17, 2018

well, 50% of my motivation to make these builds are to help the nodejs team with something. Even if it is "I've found these errors and I don't know if you're aware of them". Mission accomplished, somehow (=

I'll automate this process soon enough and make logfiles available...

Thanks for the tips. I'll give it a try.

@targos
Copy link
Member

targos commented Dec 17, 2018

I'm preparing a PR to fix this

@lsmoura
Copy link
Author

lsmoura commented Dec 17, 2018

This fixed temporarily on my build script, after unpacking the file:

  cp "deps/v8/gypfiles/v8.gyp" "deps/v8/gypfiles/v8.gyp.tmp"
  grep -v test-torque deps/v8/gypfiles/v8.gyp.tmp > deps/v8/gypfiles/v8.gyp

targos added a commit to targos/node that referenced this issue Dec 17, 2018
- Less files to maintain.
- We don't use them.
- We build with GN to run V8 tests.

Fixes: nodejs#25095
Refs: nodejs#25010
@lsmoura
Copy link
Author

lsmoura commented Dec 17, 2018

By the end of the build process, I got this one other missing piece:

  g++ -o /build/node-nightly/src/node-v12.0.0-nightly20181217b54d4a68e3/out/Release/obj.target/v8_initializers/gen/torque-generated/builtins-test-from-dsl-gen.o /build/node-nightly/src/node-v12.0.0-nightly20181217b54d4a68e3/out/Release/obj/gen/torque-generated/builtins-test-from-dsl-gen.cc '-DV8_DEPRECATION_WARNINGS' '-DV8_IMMINENT_DEPRECATION_WARNINGS' '-DV8_GYP_BUILD' '-DV8_TYPED_ARRAY_MAX_SIZE_IN_HEAP=0' '-DOPENSSL_THREADS' '-DV8_TARGET_ARCH_X64' '-DV8_EMBEDDER_STRING="-node.5"' '-DENABLE_DISASSEMBLER' '-DV8_PROMISE_INTERNAL_FIELD_COUNT=1' '-DV8_INTL_SUPPORT' '-DV8_CONCURRENT_MARKING' '-DDISABLE_UNTRUSTED_CODE_MITIGATIONS' '-DV8_EMBEDDED_BUILTINS' '-DV8_EMBEDDED_BYTECODE_HANDLERS' '-DUCONFIG_NO_SERVICE=1' '-DU_ENABLE_DYLOAD=0' '-DU_STATIC_IMPLEMENTATION=1' '-DU_HAVE_STD_STRING=1' '-DUCONFIG_NO_BREAK_ITERATION=0' -I../deps/v8 -I../deps/v8/include -I/build/node-nightly/src/node-v12.0.0-nightly20181217b54d4a68e3/out/Release/obj/gen -I../deps/icu-small/source/i18n -I../deps/icu-small/source/common  -pthread -Wall -Wextra -Wno-unused-parameter -m64 -fno-strict-aliasing -m64 -O3 -fno-omit-frame-pointer -fdata-sections -ffunction-sections -O3 -fno-rtti -fno-exceptions -std=gnu++1y -MMD -MF /build/node-nightly/src/node-v12.0.0-nightly20181217b54d4a68e3/out/Release/.deps//build/node-nightly/src/node-v12.0.0-nightly20181217b54d4a68e3/out/Release/obj.target/v8_initializers/gen/torque-generated/builtins-test-from-dsl-gen.o.d.raw -Os -fomit-frame-pointer -Os -fomit-frame-pointer -c
g++: error: /build/node-nightly/src/node-v12.0.0-nightly20181217b54d4a68e3/out/Release/obj/gen/torque-generated/builtins-test-from-dsl-gen.cc: No such file or directory
g++: fatal error: no input files
compilation terminated.
make[1]: *** [deps/v8/gypfiles/v8_initializers.target.mk:196: /build/node-nightly/src/node-v12.0.0-nightly20181217b54d4a68e3/out/Release/obj.target/v8_initializers/gen/torque-generated/builtins-test-from-dsl-gen.o] Error 1

@targos
Copy link
Member

targos commented Dec 17, 2018

@lsmoura see https://github.com/nodejs/node/pull/25097/files#diff-011f2f230002e291fd17e99c2012fcab

@rvagg
Copy link
Member

rvagg commented Dec 18, 2018

So, what might a test worker for src tarballs look like? We can probably only afford to do it on a single platform, and that'll likely be one of the Ubuntu 16.04 or 18.04 Docker containers we have in the mix for these kinds of miscellaneous things.

make tar
tar zxvf <tarball> -C ${NODE_TEST_DIR}
cd ${NODE_TEST_DIR}/<expected path>
./configure <with normal test options>
make <with normal test options>
.... ?

Would we copy in the tests and run a full test suite on the compiled build? Is that necessary or just confirming that it compiles is enough?

@richardlau
Copy link
Member

So, what might a test worker for src tarballs look like? We can probably only afford to do it on a single platform, and that'll likely be one of the Ubuntu 16.04 or 18.04 Docker containers we have in the mix for these kinds of miscellaneous things.

make tar
tar zxvf <tarball> -C ${NODE_TEST_DIR}
cd ${NODE_TEST_DIR}/<expected path>
./configure <with normal test options>
make <with normal test options>
.... ?

Would we copy in the tests and run a full test suite on the compiled build? Is that necessary or just confirming that it compiles is enough?

We shouldn't need to copy in any tests (Node.js tests are included in the source tarball). The things that don't get into the source tarball are tests for deps (which we don't run on the regular CI anyway) and things from tools which I think prevents linting.

@targos targos closed this as completed in f0e4609 Dec 20, 2018
refack pushed a commit to refack/node that referenced this issue Jan 14, 2019
- Less files to maintain.
- We don't use them.
- We build with GN to run V8 tests.

Fixes: nodejs#25095
Refs: nodejs#25010

PR-URL: nodejs#25097
Reviewed-By: Richard Lau <riclau@uk.ibm.com>
Reviewed-By: Daniel Bevenius <daniel.bevenius@gmail.com>
Reviewed-By: James M Snell <jasnell@gmail.com>
Reviewed-By: Daijiro Wachi <daijiro.wachi@gmail.com>
Reviewed-By: Ujjwal Sharma <usharma1998@gmail.com>
@mhart
Copy link
Contributor

mhart commented Sep 26, 2019

I've just encountered this with the v12.11.0 tarball. Almost identical error:

INFO: Using floating patch "tools/icu/patches/64/source/common/putil.cpp" from "tools/icu"
INFO: Using floating patch "tools/icu/patches/64/source/i18n/dtptngen.cpp" from "tools/icu"
INFO: configure completed successfully
make -C out BUILDTYPE=Release V=0
  touch /node-v12.11.0/out/Release/obj.target/tools/v8_gypfiles/v8_version.stamp
make[1]: *** No rule to make target '../deps/v8/test/torque/test-torque.tq', needed by 'df39af2ee1844610cfa693dbfb109555906de7ec.intermediate'.  Stop.
make[1]: *** Waiting for unfinished jobs....
make: *** [Makefile:101: node] Error 2

Regression maybe?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
build Issues and PRs related to build files or the CI. v8 engine Issues and PRs related to the V8 dependency.
Projects
None yet
Development

No branches or pull requests

5 participants