Skip to content

Latest commit

 

History

History
848 lines (620 loc) · 31.2 KB

BUILDING.md

File metadata and controls

848 lines (620 loc) · 31.2 KB
Raw

Building N|Solid

Depending on what platform or features you need, the build process may differ. After you've built a binary, running the test suite to confirm that the binary works as intended is a good next step.

If you can reproduce a test failure, search for it in the N|Solid issue tracker or file a new issue.

Table of contents

Supported platforms

This list of supported platforms is current as of the branch/release to which it belongs.

Input

N|Solid relies on V8 and libuv. We adopt a subset of their supported platforms.

Strategy

There are three support tiers:

  • Tier 1: These platforms represent the majority of N|Solid users. The N|Solid Build Working Group maintains infrastructure for full test coverage. Test failures on tier 1 platforms will block releases.
  • Tier 2: These platforms represent smaller segments of the N|Solid user base. The N|Solid Build Working Group maintains infrastructure for full test coverage. Test failures on tier 2 platforms will block releases. Infrastructure issues may delay the release of binaries for these platforms.
  • Experimental: May not compile or test suite may not pass. The core team does not create releases for these platforms. Test failures on experimental platforms do not block releases. Contributions to improve support for these platforms are welcome.

Platforms may move between tiers between major release lines. The table below will reflect those changes.

Platform list

N|Solid compilation/execution support depends on operating system, architecture, and libc version. The table below lists the support tier for each supported combination. A list of supported compile toolchains is also supplied for tier 1 platforms.

For production applications, run N|Solid on supported platforms only.

N|Solid does not support a platform version if a vendor has expired support for it. In other words, N|Solid does not support running on End-of-Life (EoL) platforms. This is true regardless of entries in the table below.

Operating System Architectures Versions Support Type Notes
GNU/Linux x64 kernel >= 4.181, glibc >= 2.28 Tier 1 e.g. Ubuntu 20.04, Debian 10, RHEL 8
GNU/Linux x64 kernel >= 3.10, musl >= 1.1.19 Experimental e.g. Alpine 3.8
GNU/Linux x86 kernel >= 3.10, glibc >= 2.17 Experimental Downgraded as of N|Solid 10
GNU/Linux arm64 kernel >= 4.181, glibc >= 2.28 Tier 1 e.g. Ubuntu 20.04, Debian 10, RHEL 8
GNU/Linux armv7 kernel >= 4.181, glibc >= 2.28 Tier 1 e.g. Ubuntu 20.04, Debian 11
GNU/Linux armv6 kernel >= 4.14, glibc >= 2.24 Experimental Downgraded as of N|Solid 12
GNU/Linux ppc64le >=power8 kernel >= 4.181, glibc >= 2.28 Tier 2 e.g. Ubuntu 20.04, RHEL 8
GNU/Linux s390x kernel >= 4.181, glibc >= 2.28 Tier 2 e.g. RHEL 8
Windows x64, x86 (WoW64) >= Windows 10/Server 2016 Tier 1 2,3
Windows x86 (native) >= Windows 10/Server 2016 Tier 1 (running) / Experimental (compiling)4
Windows x64, x86 Windows 8.1/Server 2012 Experimental
Windows arm64 >= Windows 10 Tier 2
macOS x64 >= 11.0 Tier 1 For notes about compilation see 5
macOS arm64 >= 11.0 Tier 1
SmartOS x64 >= 18 Tier 2
AIX ppc64be >=power8 >= 7.2 TL04 Tier 2
FreeBSD x64 >= 13.2 Experimental

Supported toolchains

Depending on the host platform, the selection of toolchains may vary.

Operating System Compiler Versions
Linux GCC >= 10.1
Windows Visual Studio >= 2022 with the Windows 10 SDK on a 64-bit host
macOS Xcode >= 13 (Apple LLVM >= 12)

Official binary platforms and toolchains

Binaries at https://downloads.nodesource.com/ are produced on:

Binary package Platform and Toolchain
aix-ppc64 AIX 7.2 TL04 on PPC64BE with GCC 10
darwin-x64 macOS 11, Xcode 13 with -mmacosx-version-min=11.0
darwin-arm64 (and .pkg) macOS 11 (arm64), Xcode 13 with -mmacosx-version-min=11.0
linux-arm64 RHEL 8 with GCC 106
linux-armv7l Cross-compiled on RHEL 8 x64 with custom GCC toolchain7
linux-ppc64le RHEL 8 with gcc-toolset-106
linux-s390x RHEL 8 with gcc-toolset-106
linux-x64 RHEL 8 with gcc-toolset-106
win-x64 and win-x86 Windows Server 2022 (x64) with Visual Studio 2022

OpenSSL asm support

OpenSSL-1.1.1 requires the following assembler version for use of asm support on x86_64 and ia32.

For use of AVX-512,

  • gas (GNU assembler) version 2.26 or higher
  • nasm version 2.11.8 or higher in Windows

AVX-512 is disabled for Skylake-X by OpenSSL-1.1.1.

For use of AVX2,

  • gas (GNU assembler) version 2.23 or higher
  • Xcode version 5.0 or higher
  • llvm version 3.3 or higher
  • nasm version 2.10 or higher in Windows

Please refer to https://docs.openssl.org/1.1.1/man3/OPENSSL_ia32cap/ for details.

If compiling without one of the above, use configure with the --openssl-no-asm flag. Otherwise, configure will fail.

Building N|Solid on supported platforms

Prerequisites

  • A supported version of Python for building and testing.
  • Memory: at least 8GB of RAM is typically required when compiling with 4 parallel jobs (e.g: make -j4)

Unix and macOS

Unix prerequisites

  • gcc and g++ >= 10.1 or newer
  • GNU Make 3.81 or newer
  • Python >=3.6 <=3.11 (see note above)
    • For test coverage, your Python installation must include pip.

Installation via Linux package manager can be achieved with:

  • Ubuntu, Debian: sudo apt-get install python3 g++ make python3-pip
  • Fedora: sudo dnf install python3 gcc-c++ make python3-pip
  • CentOS and RHEL: sudo yum install python3 gcc-c++ make python3-pip
  • OpenSUSE: sudo zypper install python3 gcc-c++ make python3-pip
  • Arch Linux, Manjaro: sudo pacman -S python gcc make python-pip

FreeBSD and OpenBSD users may also need to install libexecinfo.

macOS prerequisites

macOS users can install the Xcode Command Line Tools by running xcode-select --install. Alternatively, if you already have the full Xcode installed, you can find them under the menu Xcode -> Open Developer Tool -> More Developer Tools.... This step will install clang, clang++, and make.

Building N|Solid

If the path to your build directory contains a space, the build will likely fail.

To build N|Solid:

./configure
make -j4

We can speed up the builds by using Ninja. For more information, see Building N|Solid with Ninja.

The -j4 option will cause make to run 4 simultaneous compilation jobs which may reduce build time. For more information, see the GNU Make Documentation.

The above requires that python resolves to a supported version of Python. See Prerequisites.

After building, setting up firewall rules can avoid popups asking to accept incoming network connections when running tests.

Running the following script on macOS will add the firewall rules for the executable nsolid in the out directory and the symbolic nsolid link in the project's root directory.

sudo ./tools/macos-firewall.sh

Installing N|Solid

To install this version of N|Solid into a system directory:

[sudo] make install

Running tests

To verify the build:

make test-only

At this point, you are ready to make code changes and re-run the tests.

If you are running tests before submitting a pull request, use:

make -j4 test

make -j4 test does a full check on the codebase, including running linters and documentation tests.

To run the linter without running tests, use make lint/vcbuild lint. It will lint JavaScript, C++, and Markdown files.

If you are updating tests and want to run tests in a single test file (e.g. test/parallel/test-stream2-transform.js):

tools/test.py test/parallel/test-stream2-transform.js

You can execute the entire suite of tests for a given subsystem by providing the name of a subsystem:

tools/test.py child-process

You can also execute the tests in a test suite directory (such as test/message):

tools/test.py test/message

If you want to check the other options, please refer to the help by using the --help option:

tools/test.py --help

Note: On Windows you should use python3 executable. Example: python3 tools/test.py test/message

You can usually run tests directly with nsolid:

./nsolid test/parallel/test-stream2-transform.js

Info: ./nsolid points to your local N|Solid build.

Remember to recompile with make -j4 in between test runs if you change code in the lib or src directories.

The tests attempt to detect support for IPv6 and exclude IPv6 tests if appropriate. If your main interface has IPv6 addresses, then your loopback interface must also have '::1' enabled. For some default installations on Ubuntu, that does not seem to be the case. To enable '::1' on the loopback interface on Ubuntu:

sudo sysctl -w net.ipv6.conf.lo.disable_ipv6=0

You can use node-code-ide-configs to run/debug tests if your IDE configs are present.

Running coverage

It's good practice to ensure any code you add or change is covered by tests. You can do so by running the test suite with coverage enabled:

./configure --coverage
make coverage

A detailed coverage report will be written to coverage/index.html for JavaScript coverage and to coverage/cxxcoverage.html for C++ coverage.

If you only want to run the JavaScript tests then you do not need to run the first command (./configure --coverage). Run make coverage-run-js, to execute JavaScript tests independently of the C++ test suite:

make coverage-run-js

If you are updating tests and want to collect coverage for a single test file (e.g. test/parallel/test-stream2-transform.js):

make coverage-clean
NODE_V8_COVERAGE=coverage/tmp tools/test.py test/parallel/test-stream2-transform.js
make coverage-report-js

You can collect coverage for the entire suite of tests for a given subsystem by providing the name of a subsystem:

make coverage-clean
NODE_V8_COVERAGE=coverage/tmp tools/test.py --mode=release child-process
make coverage-report-js

The make coverage command downloads some tools to the project root directory. To clean up after generating the coverage reports:

make coverage-clean

Building the documentation

To build the documentation:

This will build N|Solid first (if necessary) and then use it to build the docs:

make doc

If you have an existing N|Solid build, you can build just the docs with:

NODE=/path/to/nsolid make doc-only

To read the man page:

man doc/nsolid.1

If you prefer to read the full documentation in a browser, run the following.

make docserve

This will spin up a static file server and provide a URL to where you may browse the documentation locally.

If you're comfortable viewing the documentation using the program your operating system has associated with the default web browser, run the following.

make docopen

This will open a file URL to a one-page version of all the browsable HTML documents using the default browser.

To test if N|Solid was built correctly:

./nsolid -e "console.log('Hello from N|Solid ' + process.version)"

Building a debug build

If you run into an issue where the information provided by the JS stack trace is not enough, or if you suspect the error happens outside of the JS VM, you can try to build a debug enabled binary:

./configure --debug
make -j4

make with ./configure --debug generates two binaries, the regular release one in out/Release/nsolid and a debug binary in out/Debug/nsolid, only the release version is actually installed when you run make install.

To use the debug build with all the normal dependencies overwrite the release version in the install directory:

make install PREFIX=/opt/nsolid-debug/
cp -a -f out/Debug/nsolid /opt/nsolid-debug/nsolid

When using the debug binary, core dumps will be generated in case of crashes. These core dumps are useful for debugging when provided with the corresponding original debug binary and system information.

Reading the core dump requires gdb built on the same platform the core dump was captured on (i.e. 64-bit gdb for nsolid built on a 64-bit system, Linux gdb for nsolid built on Linux) otherwise you will get errors like not in executable format: File format not recognized.

Example of generating a backtrace from the core dump:

$ gdb /opt/nsolid-debug/nsolid core.nsolid.8.1535359906
(gdb) backtrace

Building an ASan build

ASan can help detect various memory related bugs. ASan builds are currently only supported on linux.

The --debug is not necessary and will slow down build and testing, but it can show clear stacktrace if ASan hits an issue.

./configure --debug --enable-asan && make -j4
make test-only

Speeding up frequent rebuilds when developing

If you plan to frequently rebuild N|Solid, especially if using several branches, installing ccache can help to greatly reduce build times. Set up with:

On GNU/Linux:

sudo apt install ccache   # for Debian/Ubuntu, included in most Linux distros
export CC="ccache gcc"    # add to your .profile
export CXX="ccache g++"   # add to your .profile

On macOS:

brew install ccache      # see https://brew.sh
export CC="ccache cc"    # add to ~/.zshrc or other shell config file
export CXX="ccache c++"  # add to ~/.zshrc or other shell config file

This will allow for near-instantaneous rebuilds even when switching branches.

When modifying only the JS layer in lib, it is possible to externally load it without modifying the executable:

./configure --node-builtin-modules-path "$(pwd)"

The resulting binary won't include any JS files and will try to load them from the specified directory. The JS debugger of Visual Studio Code supports this configuration since the November 2020 version and allows for setting breakpoints.

Troubleshooting Unix and macOS builds

Stale builds can sometimes result in file not found errors while building. This and some other problems can be resolved with make distclean. The distclean recipe aggressively removes build artifacts. You will need to build again (make -j4). Since all build artifacts have been removed, this rebuild may take a lot more time than previous builds. Additionally, distclean removes the file that stores the results of ./configure. If you ran ./configure with non-default options (such as --debug), you will need to run it again before invoking make -j4.

If you received the error nodejs g++ fatal error compilation terminated cc1plus during compilation, this is likely a memory issue and you should either provide more RAM or create swap space to accommodate toolchain requirements or reduce the number of parallel build tasks (-j<n>).

Windows

Prerequisites

Manual install
  • Python 3.11
  • The "Desktop development with C++" workload from Visual Studio 2022 (17.6 or newer) or the "C++ build tools" workload from the Build Tools, with the default optional components
  • Basic Unix tools required for some tests, Git for Windows includes Git Bash and tools which can be included in the global PATH.
  • The NetWide Assembler, for OpenSSL assembler modules. If not installed in the default location, it needs to be manually added to PATH. A build with the openssl-no-asm option does not need this, nor does a build targeting ARM64 Windows.

Optional requirements to build the MSI installer package:

Optional requirements for compiling for Windows on ARM (ARM64):

  • Visual Studio 17.6.0 or newer

    Note: There is a bug in 17.10.x preventing Node.js from compiling.

  • Visual Studio optional components
    • Visual C++ compilers and libraries for ARM64
    • Visual C++ ATL for ARM64
  • Windows 10 SDK 10.0.17763.0 or newer

Building N|Solid

If the path to your build directory contains a space or a non-ASCII character, the build will likely fail.

.\vcbuild

To run the tests:

.\vcbuild test

To test if N|Solid was built correctly:

Release\nsolid -e "console.log('Hello from N|Solid', process.version)"

Android

Android is not a supported platform. Patches to improve the Android build are welcome. There is no testing on Android in the current continuous integration environment. The participation of people dedicated and determined to improve Android building, testing, and support is encouraged.

Be sure you have downloaded and extracted Android NDK before in a folder. Then run:

./android-configure <path to the Android NDK> <Android SDK version> <target architecture>
make -j4

The Android SDK version should be at least 24 (Android 7.0) and the target architecture supports [arm, arm64/aarch64, x86, x86_64].

Intl (ECMA-402) support

Intl support is enabled by default.

Build with full ICU support (all locales supported by ICU)

This is the default option.

Unix/macOS

./configure --with-intl=full-icu

Windows

.\vcbuild full-icu

Trimmed: small-icu (English only) support

In this configuration, only English data is included, but the full Intl (ECMA-402) APIs. It does not need to download any dependencies to function. You can add full data at runtime.

Unix/macOS

./configure --with-intl=small-icu

Windows

.\vcbuild small-icu

Building without Intl support

The Intl object will not be available, nor some other APIs such as String.normalize.

Unix/macOS

./configure --without-intl

Windows

.\vcbuild without-intl

Use existing installed ICU (Unix/macOS only)

pkg-config --modversion icu-i18n && ./configure --with-intl=system-icu

If you are cross-compiling, your pkg-config must be able to supply a path that works for both your host and target environments.

Build with a specific ICU

You can find other ICU releases at the ICU homepage. Download the file named something like icu4c-**##.#**-src.tgz (or .zip).

To check the minimum recommended ICU, run ./configure --help and see the help for the --with-icu-source option. A warning will be printed during configuration if the ICU version is too old.

Unix/macOS

From an already-unpacked ICU:

./configure --with-intl=[small-icu,full-icu] --with-icu-source=/path/to/icu

From a local ICU tarball:

./configure --with-intl=[small-icu,full-icu] --with-icu-source=/path/to/icu.tgz

From a tarball URL:

./configure --with-intl=full-icu --with-icu-source=http://url/to/icu.tgz

Windows

First unpack latest ICU to deps/icu icu4c-##.#-src.tgz (or .zip) as deps/icu (You'll have: deps/icu/source/...)

.\vcbuild full-icu

Configure OpenSSL appname

N|Solid can use an OpenSSL configuration file by specifying the environment variable OPENSSL_CONF, or using the command line option --openssl-conf, and if none of those are specified will default to reading the default OpenSSL configuration file openssl.cnf. N|Solid will only read a section that is by default named nodejs_conf, but this name can be overridden using the following configure option:

./configure --openssl-conf-name=<some_conf_name>

Building N|Solid with FIPS-compliant OpenSSL

N|Solid supports FIPS when statically or dynamically linked with OpenSSL 3 via OpenSSL's provider model. It is not necessary to rebuild N|Solid to enable support for FIPS.

See FIPS mode for more information on how to enable FIPS support in N|Solid.

Building N|Solid with external core modules

It is possible to specify one or more JavaScript text files to be bundled in the binary as built-in modules when building N|Solid.

Unix/macOS

This command will make /root/myModule.js available via require('/root/myModule') and ./myModule2.js available via require('myModule2').

./configure --link-module '/root/myModule.js' --link-module './myModule2.js'

Windows

To make ./myModule.js available via require('myModule') and ./myModule2.js available via require('myModule2'):

.\vcbuild link-module './myModule.js' link-module './myModule2.js'

Building to use shared dependencies at runtime

By default N|Solid is built so that all dependencies are bundled into the N|Solid binary itself. This provides a single binary that includes the correct versions of all dependencies on which it depends.

Some N|Solid distributions, however, prefer to manage dependencies. A number of configure options are provided to support this use case.

  • For dependencies with native code, the first set of options allow N|Solid to be built so that it uses a shared library at runtime instead of building and including the dependency in the N|Solid binary itself. These options are in the Shared libraries section of the configure help (run ./configure --help to get the complete list). They provide the ability to enable the use of a shared library, to set the name of the shared library, and to set the paths that contain the include and shared library files.

  • For dependencies with JavaScript code (including WASM), the second set of options allow the N|Solid binary to be built so that it loads the JavaScript for dependencies at runtime instead of being built into the N|Solid binary itself. These options are in the Shared builtins section of the configure help (run ./configure --help to get the complete list). They provide the ability to set the path to an external JavaScript file for the dependency to be used at runtime.

It is the responsibility of any distribution shipping with these options to:

  • ensure that the shared dependencies available at runtime match what is expected by the N|Solid binary. A mismatch may result in crashes or unexpected behavior.
  • fully test that N|Solid operates as expected with the external dependencies. There may be little or no test coverage within the N|Solid project CI for these non-default options.

Note for downstream distributors of N|Solid

The N|Solid ecosystem is reliant on ABI compatibility within a major release. To maintain ABI compatibility it is required that distributed builds of N|Solid be built against the same version of dependencies, or similar versions that do not break their ABI compatibility, as those released by N|Solid for any given NODE_MODULE_VERSION (located in src/node_version.h).

When N|Solid is built (with an intention to distribute) with an ABI incompatible with the official N|Solid builds (e.g. using a ABI incompatible version of a dependency), please reserve and use a custom NODE_MODULE_VERSION by opening a pull request against the registry available at https://github.com/nodesource/nsolid/blob/HEAD/doc/abi_version_registry.json.

Footnotes

  1. Older kernel versions may work. However, official N|Solid release binaries are built on RHEL 8 systems with kernel 4.18. ↩ ↩2 ↩3 ↩4 ↩5

  2. On Windows, running N|Solid in Windows terminal emulators like mintty requires the usage of winpty for the tty channels to work (e.g. winpty nsolid.exe script.js). In "Git bash" if you call the nsolid shell alias (nsolid without the .exe extension), winpty is used automatically. ↩

  3. The Windows Subsystem for Linux (WSL) is not supported, but the GNU/Linux build process and binaries should work. The community will only address issues that reproduce on native GNU/Linux systems. Issues that only reproduce on WSL should be reported in the WSL issue tracker. Running the Windows binary (nsolid.exe) in WSL will not work without workarounds such as stdio redirection. ↩

  4. Running N|Solid on x86 Windows should work and binaries are provided. However, tests in our infrastructure only run on WoW64. Furthermore, compiling on x86 Windows is Experimental and may not be possible. ↩

  5. Our macOS x64 Binaries are compiled with 11.0 as a target. Xcode 13 is required to compile. ↩

  6. Binaries produced on these systems are compatible with glibc >= 2.28 and libstdc++ >= 6.0.25 (GLIBCXX_3.4.25). These are available on distributions natively supporting GCC 8.1 or higher, such as Debian 10, RHEL 8 and Ubuntu 20.04. ↩ ↩2 ↩3 ↩4

  7. Binaries produced on these systems are compatible with glibc >= 2.28 and libstdc++ >= 6.0.28 (GLIBCXX_3.4.28). These are available on distributions natively supporting GCC 9.3 or higher, such as Debian 11, Ubuntu 20.04. ↩