-
Notifications
You must be signed in to change notification settings - Fork 523
Home
rules_nodejs 3.6.0 introduced exports_directories_only
attributes on npm_install
and yarn_install
that reduce the number of files Bazel has to work with, making builds a lot faster. See https://docs.aspect.dev/bazelbuild/rules_nodejs/3.6.0/docs/builtins.html#npm_install-exports_directories_only
ts_library doesn't work with this install mode at all.
rules_sass doesn't really support third-party dependencies directly. Many people have worked around this by passing files from node_modules into a sass_library rule as discussed in https://github.com/bazelbuild/rules_sass/issues/98 However that approach won't work with directory artifacts.
We suggest switching to the vanilla sass
rule instead, see this example or the following macros which should align with the rules_sass
rules:
def sass_library(name, srcs, deps = [], **args):
native.filegroup(
name = name,
srcs = srcs + deps,
**args
)
def sass_binary(name, src, output_style = "compressed", include_paths = [], output_name = None, deps = [], **args):
output_name = output_name if output_name else src.replace(".scss", "").replace(":", "") + ".css"
sass(
name = name,
outs = [output_name, "%s.map" % output_name],
args = [
"$(execpath %s)" % src,
"$(execpath %s)" % output_name,
"--style=%s" % output_style,
"--no-error-css", # don't output any files on error
"--quiet", # don't output warnings
"--load-path=external/npm/node_modules/", # for node modules
"--load-path=$(BINDIR)", # for generated files
] + [
"--load-path=%s" % p # custom include_paths
for p in include_paths
] + [
"--load-path=$(BINDIR)/%s" % p # custom include_paths for generated files
for p in include_paths
],
data = [src] + deps,
**args
)
--bazel_patch_module_resolver now defaults to false (#2324)
This option to nodejs_binary
causes us to monkey-patch the implementation of require()
in NodeJS, so that it becomes "Bazel-aware" and
knows how to resolve between the source folder and the output folder. However it has always had a compatibility problem. Subprocesses and code that uses alternate resolvers like https://www.npmjs.com/package/resolve don't get the patches, and patching require()
is a risky undertaking that can cause bugs.
If this breaks you, the quickest fix is to flip the flag back on a nodejs_binary/nodejs_test/npm_package_bin with templated_args = ["--bazel_patch_module_resolver"]
, see #2344 as an example.
One effect of the patched require() is to make the source tree and the bazel-out
output tree appear to be joined by reading from Bazel's "runfiles" symlink forest. This can be achieved instead in programs you control (including tests) by using our Runfiles helper library. See #2341 as an example. Note that since the Runfiles helper library is distributed as part of our "built-in" package installed in your WORKSPACE file, you reference its path with an Environment variable like require(process.env['BAZEL_NODE_RUNFILES_HELPER'])
unlike an npm-installed package.
Removal of the @bazel/karma
package (#2313)
The @bazel/karma
package has now been deprecated and will no longer be published. The existing karma_web_test
and karma_web_test_suite
rules have moved to @bazel/concatjs
(see Introducing @bazel/concatjs
for more information on this new package).
Remove references to npm_bazel_karma_dependencies
in the WORKSPACE
file:
load("@npm//@bazel/karma:package.bzl", "npm_bazel_karma_dependencies")
npm_bazel_karma_dependencies()
Buildozer can be used to update the load statements in existing BUILD[.bazel]
files.
npx @bazel/buildozer 'replace_load @npm//@bazel/concatjs:index.bzl karma_web_test_suite' //...:__pkg__
npx @bazel/buildozer 'replace_load @npm//@bazel/concatjs:index.bzl karma_web_test' //...:__pkg__
Finally, remove the package.json
dependency on @bazel/karma
and replace it with @bazel/concatjs
In some cases, it's likely possible to use the generated karma_test
rules directly by installing karma
and using load("@npm//karma:index.bzl", "karma_test")
.
If you also had a peer dependency on jasmine-core
for @bazel/karma
this can also be removed as it is no longer required (see 2253)
Rename ts_devserver
-> concatjs_devserver
(#2239)
ts_devserver
has been renamed to concactjs_devserver
and moved into the @bazel/concatjs
package (see Introducing @bazel/concatjs
for more information on this new package).
Buildozer can be used to update the load statements in existing BUILD[.bazel]
files.
npx @bazel/buildozer 'new_load @npm//@bazel/concatjs:index.bzl concatjs_devserver' //...:__pkg__
npx @bazel/buildozer 'set kind concatjs_devserver' //...:%ts_devserver
npx @bazel/buildozer 'fix unusedLoads' //...:__pkg__
3.0 introduces a new NPM package, @bazel/concatjs
. This is now the home for all concatjs
based rules, such as karma_web_test
and ts_devserver
. This package closely mirrors the internal toolchain used in Google. Users may decide to use this toolchain, but there are tradeoffs in complexity (for example, ts_devserver
requires all sources as named AMD bundles), it's often recommended to use alternatives to this package where possible.
Removal of install_bazel_dependencies
(#1877)
In a previous release, install_bazel_dependencies
was deprecated but still needed in some situations (eg when loading from @angular/bazel
). As @angular/bazel
is now deprecated, this is no longer required, and has now been removed.
Remove the following from WORKSPACE
if it is still present:
load("@npm//:install_bazel_dependencies.bzl", "install_bazel_dependencies")
install_bazel_dependencies(suppress_warning = True)
Removal of pkg_npm#replace_with_version
(#2312)
replace_with_version
on pkg_npm
was deprecated and has now been removed. Users should use substitutions
instead.
pkg_npm(
name = "publish",
...
replace_with_version = "my_version_placeholder",
substitutions = {
"replace_me": "replaced",
},
...
)
becomes:
pkg_npm(
name = "publish",
...
substitutions = {
"my_version_placeholder": "{BUILD_SCM_VERSION}",
"replace_me": "replaced",
},
...
)
Rollup config_file
no longer substitutes bazel_stamp_file
(#2312)
The config_file
used for rollup_bundle
no longer substitutes bazel_stamp_file
, only bazel_version_file
and bazel_info_file
are now substituted. Rename bazel_stamp_file
to bazel_version_file
as they were both substituted with the same value.
pkg_web/pkg_web.bzl
no longer exports move_files
as public API (#2159)
The move_files
function from pkg_web.bzl
is no longer public API and its usages should be removed.
ts_project#extends
no longer takes a list (#2266)
Previously, the extends
attr on ts_project
took a list, it now takes a single label from a ts_config
rule, or a tsconfig.json
file.
strict_visibility
on yarn_install
and npm_install
now defaults True
(#2199)
In 2.x, the npm_install
and yarn_install
rules introduced a strict_visibility
flag which applied limited visibility to transitive dependencies from resolved from the package.json
file. This flag has now been flipped to default to True
. All dependencies that are now referenced from external npm / yarn install rules must now be present in the given package.json
file.
To opt out of this behaviour, set strict_visibility
to False
on the npm_install
or yarn_install
repo rule:
npm_install(
name = "npm",
strict_visibility = False,
...
)
yarn_install(
name = "npm",
strict_visibility = False,
...
)
tsetse, our third-party strictness plugin for TypeScript under ts_library
, now checks for more patterns, for example you might get an error like
error TS21231: [tsetse] type assert `JSON.parse() as SomeExplicitType` for type & optimization safety.
See http://tsetse.info/must-type-assert-json-parse.
you can either repair the code in the way it suggests, or disable the check. To do the latter, in your tsconfig.json do this:
{
"compilerOptions": {
"plugins": [
{"name": "@bazel/tsetse", "disabledRules": ["must-type-assert-json-parse"]}
]
}
}
We removed the node_modules attribute from nodejs_binary
, nodejs_test
, jasmine_node_test
& ts_library
.
If you are using the node_modules attribute, you can simply add the target specified there to the data or deps attribute of the rule instead.
nodejs_test(
name = "test",
data = [
"test.js",
"@npm//:node_modules",
],
entry_point = "test.js",
)
or
ts_library(
name = "lib",
srcs = glob(["*.ts"]),
tsconfig = ":tsconfig.json",
deps = ["@npm//:node_modules"],
)
We also dropped support for filegroup based node_modules target and removed node_modules_filegroup
from index.bzl
.
If you are using this feature for user-managed deps, you must now a js_library target with external_npm_package
set to True instead.
js_library(
name = "node_modules",
srcs = glob(
include = [
"node_modules/**/*.js",
"node_modules/**/*.d.ts",
"node_modules/**/*.json",
"node_modules/.bin/*",
],
exclude = [
# Files under test & docs may contain file names that
# are not legal Bazel labels (e.g.,
# node_modules/ecstatic/test/public/中文/檔案.html)
"node_modules/**/test/**",
"node_modules/**/docs/**",
# Files with spaces in the name are not legal Bazel labels
"node_modules/**/* */**",
"node_modules/**/* *",
],
),
# Provide ExternalNpmPackageInfo which is used by downstream rules
# that use these npm dependencies
external_npm_package = True,
)
nodejs_test(
name = "test",
data = [
"test.js",
":node_modules",
],
entry_point = "test.js",
)
See examples/user_managed_deps for a working example of user-managed npm dependencies
Buildozer can be used to update the load statements in existing BUILD[.bazel]
files. The following will migrate all load statements within the current workspace from the @npm_bazel_
format, to @npm//@bazel/
format
npx @bazel/buildozer 'substitute_load ^@npm_bazel_(.*?)//:index\.bzl$ @npm//@bazel/${1}:index.bzl' //...:__pkg__
The install_source_map_support attribute is removed from nodejs_binary. source-map-support is vendored in at /third_party/github.com/source-map-support so it can always be installed. Simply remove any usage of that attribute. The following buildozer command will remove this off all nodejs_binary
rules in the workspace
npx @bazel/buildozer 'remove install_source_map_support' //...:%nodejs_binary
If your WORKSPACE
file has workspace(name="foo")
and your code expects to import("foo/thing")
, you need to make this explicit. Alternately you can keep the old behavior with this flag: https://github.com/bazelbuild/rules_nodejs/pull/2175
(Note, ts_library
converts relative imports to absolute, so even import("./thing")
will become import("foo/thing")
in the JS files in bazel-out)
In your root BUILD.bazel
file, add a pkg_npm
between the declaration site and the use site of these JS files. For example:
load("@build_bazel_rules_nodejs//:index.bzl", "pkg_npm")
pkg_npm(
name = "foo_pkg",
# Makes this code importable as "foo"
package_name = "foo",
deps = [":lib_that_declares_foo_thing"],
)
my_test(
data = ["foo_pkg"],
)
If you have upgraded to Angular Ivy, we no longer recommend depending on the deprecated @angular/bazel
package. The only rule needed from it was ng_module
which you can replace with the faster equivalent ts_library(use_angular_plugin=True)
Buildozer can be used to edit multiple BUILD
files at once, with multiple commands.
Create a text with the following commands below. The first line can be changed to provide more focused list of packages to add the ts_library
rule too, however ones that aren't then used will be removed.
new_load @npm//@bazel/typescript:index.bzl ts_library|//...:__pkg__
set use_angular_plugin True|//...:%ng_module
rename assets angular_assets|//...:%ng_module
set kind ts_library|//...:%ng_module
fix unusedLoads|//...:__pkg__
Optional worker support with the following, added below the previous set ...
line
set supports_workers True|//...:%ng_module
Then run the following to execute (assuming the file is named commands.txt
)
npx @bazel/buildozer -f commands.txt
If you still use Angular's old compiler ("View Engine") then consult our examples/angular_view_engine directory. We consider View Engine plus Bazel to be deprecated, but are doing our best to keep it working.
- you'll still need to use the
@angular/bazel
package, which requires you keep theinstall_bazel_dependencies
call in your/WORKSPACE
- a patch is required, see the patches/ directory under the example
To use Bazel, it obviously needs to be downloaded to your computer. It also needs to be the right version to be compatible with the code you're building.
This is the same problem we have with Node, which you need to fetch and correctly version. The community-accepted solution for this is to use nvm and a .nvmrc file. .nvmrc
says what version(s) of node work with the project, and nvm downloads a node.js version to your machine if needed.
Bazel has a nearly identical solution. The @bazel/bazelisk
package is a Bazel wrapper that downloads the correct version of Bazel based on your .bazelversion
file.
Now that this is available, we plan to deprecate our old mechanism for downloading and versioning Bazel in npm-based projects, the @bazel/bazel
package.
The bazel binary is pretty big (compared with typical JS tooling) at 40MB. But it's a native binary, so we actually had three optional dependencies from the @bazel/bazel
package to platform-specific ones. This is a nice solution, except that we found that in practice, both yarn and npm have behavior (bug?) where they download the tarballs for all three packages even though it was known beforehand that two of them are not compatible with the local platform. So you end up with 120MB download. We didn't succeed in escalating this issue with the package managers.
Second, by having Bazel versioned in the package.json
file, we were not compatible with the rest of Bazel environment. For example, if you downloaded Bazel outside of the npm mirror, you wouldn't get any warning that you have the wrong version. Bazel itself knows to look for the .bazelversion
file so we want that to work for JS developers too.
Next, we had a governance issue. rules_nodejs maintainers were mirroring Bazel to npm, but we didn't always mirror all the versions (like release candidates). The release process should be hosted entirely by the tool itself so every version is available and you can rely on the release engineer doing all the steps. Since Bazelisk now releases to npm (thanks @philwo!), we get out of the mirroring business and you can use any version of Bazel.
Finally, bazelisk has better support as a Bazel wrapper than the npm mirror had. It knows to execute the tools/bazel
wrapper if it exists, and can be hooked up with command-line completion. It even has features to help you migrate past Bazel breaking changes (akin to the ng update
command in Angular ecosystem)
- Remove your dependency on
@bazel/bazel
which is deprecated - Add a devDependency on
@bazel/bazelisk
instead. The version doesn't matter much since it's just a wrapper. (you could even use a global install ofbazelisk
if you'd rather have the tool on your$PATH
) - Add a
.bazelversion
file in your workspace root containing the version you want to pin to. See the bazelisk readme - In your
package.json#scripts
, callbazelisk
rather thanbazel
-
@bazel/bazel
had a transitive dependency on a workaround package,@bazel/hide-build-files
, whose purpose was to prevent Bazel from seeing thenode_modules/**/BUILD
files under external repositories. You can either- update Bazel to >= 2.1 and rules_nodejs >= 1.3.0 since the workaround is no longer needed, or
- add a devDependency on
@bazel/hide-build-files
to keep the workaround in place
If you had been relying on a
bazel
command in your$PATH
, you can still do this by aliasing it to thebazelisk
command. You can also add a"bazel": "bazelisk"
alias in thepackage.json#scripts
.
Note, Bazelisk is adding more robust handling of network constraints. Check this issue if you need to download via a corporate proxy: https://github.com/bazelbuild/bazelisk/issues/115.
For consistency, rules_nodejs always loads from a file called index.bzl
since this matches the node ecosystem, and Bazel itself doesn't have a single standard (defs.bzl, build_defs.bzl and others are common)
If you depend on another ruleset that still depends on defs.bzl, you must update. For rules_sass, the minimum is:
http_archive(
name = "io_bazel_rules_sass",
sha256 = "617e444f47a1f3e25eb1b6f8e88a2451d54a2afdc7c50518861d9f706fc8baaa",
urls = [
"https://github.com/bazelbuild/rules_sass/archive/1.23.7.zip",
"https://mirror.bazel.build/github.com/bazelbuild/rules_sass/archive/1.23.7.zip",
],
strip_prefix = "rules_sass-1.23.7",
)
For rules_docker, the minimum release is https://github.com/bazelbuild/rules_docker/releases/tag/v0.13.0
If you have trouble updating to the latest rules_docker, you can also apply a simple local patch to the version you're currently on. For example, within rules_nodejs we added one "patches" attribute to our rules_docker fetch:
http_archive(
name = "io_bazel_rules_docker",
patches = ["//:rules_docker.patch"], # <-- add this to apply a patch
sha256 = "7d453450e1eb70e238eea6b31f4115607ec1200e91afea01c25f9804f37e39c8",
strip_prefix = "rules_docker-0.10.0",
urls = ["https://github.com/bazelbuild/rules_docker/archive/v0.10.0.tar.gz"],
)
and add a file rules_docker.patch
next to WORKSPACE
like this:
diff --git nodejs/image.bzl nodejs/image.bzl
index a01ea3e..617aa06 100644
--- nodejs/image.bzl
+++ nodejs/image.bzl
@@ -17,7 +17,7 @@ The signature of this rule is compatible with nodejs_binary.
"""
load("@bazel_skylib//lib:dicts.bzl", "dicts")
-load("@build_bazel_rules_nodejs//:defs.bzl", "nodejs_binary")
+load("@build_bazel_rules_nodejs//:index.bzl", "nodejs_binary")
load(
"//container:container.bzl",
"container_pull",
The rollup_bundle
rule from the //internal
package has a bunch of problems, and has been replaced by a new implementation. The new implementation is much simpler, does just one thing, and is meant to run nearly identically to how you would use Rollup outside of Bazel. The //internal/rollup
package will be removed before 1.0. Some work is required to move to the new rule.
High-level notes:
- Terser minification was run in the old rule when you requested a
.min.js
output.rollup_bundle
andterser_minified
are now separate rules. - New rules use a peerDependency, so you need to install
rollup
/terser
if you haven't already - We don't have a downleveling step built-in, so if you need es5 output you can use a rollup plugin to do it. Note that this means differential loading scenarios need two rollup builds, which is slow - we'll have a better way to downlevel rollup outputs before we remove the old rollup_bundle rule.
- Works with
ng_module
with a minimum@angular/bazel
version of9.0.0-next.11
.
Rough steps to follow:
- Replace
load("@build_bazel_rules_nodejs//:defs.bzl", "rollup_bundle")
withload("@npm_bazel_rollup//:index.bzl", "rollup_bundle")
- If you reference a
min.js
output, also addload("@npm_bazel_terser//:index.bzl", "terser_minified")
- Generally, you can get the same
.min.js
output as before just by addingterser_minified
after the existingrollup_bundle
:# this will output bundle.min.js, but not a predeclared label terser_minified( name = "bundle.min", # assuming your rollup_bundle rule has name="bundle" src = "bundle.js", )
- The new terser rule cannot provide a label for
bundle.min.js
(because it doesn't know if the input is a directory until after the predeclared outputs are determined), so you can't reference that in your other targets. You'll have to usebundle.min
. If a downstream target expects a label which produces a single file, you can disable thesourcemap
feature interser_minified
so that the .js output is the only one. - If you used
enable_code_splitting = False
, you can just remove it. The new rule only produces chunked output when given aoutput_dir = True
attribute. -
additional_entry_points
are now combined with the primary entry_point in a singleentry_points
attribute. You can give TypeScript files as the entry point, likeindex.ts
and the rule will resolve that to the.js
output file.
Thanks to @jbedard for the design doc that motivated the changes.
These are detailed migration notes for https://github.com/bazelbuild/rules_nodejs/releases/tag/0.30.0
In prior releases, Bazel installed a copy of the node_modules
tree in its private external directory. This meant that you typically had to install dependencies twice, and if you wanted to do some println debugging in the node_modules
directory, you had to know where this other copy is installed.
As of Bazel 0.26, there is a new feature called "managed directories" which allows Bazel to reference the same node_modules
directory that your package manager creates and that the editor uses to resolve tools like TypeScript. This feature is now enabled by default.
You must update your Bazel dependency and/or your locally installed Bazel to 0.26 or greater before updating rules_nodejs to 0.30.
To use managed directories, make these changes to your app:
- if you are using the
@bazel/bazel
npm package, update it to"@bazel/bazel": "0.26.0-rc13"
or newer - update
WORKSPACE
to rules_nodejs 0.30.0 release - if you are using the
@bazel/typescript
npm package this needs to be updated to 0.30.0 as well - tell Bazel to opt-in to this feature by adding to
.bazelrc
:build --experimental_allow_incremental_repository_updates query --experimental_allow_incremental_repository_updates
- make sure you don’t have multiple
yarn_install
/npm_install
rules referencing the same package.json - add to
WORKSPACE
themanaged_directories
of all youryarn_install
/npm_install
rules. This typically looks like:workspace( name = "my_wksp", managed_directories = {"@npm": ["node_modules"]} )
- add the same
node_modules
directories as lines in.bazelignore
- these contain BUILD files that only work when Bazel references them externally
You can turn managed directories off for individual yarn_install
or npm_install
rules with the symlink_node_modules = False
attribute. If you you have multiple yarn_install
and/or npm_install
rules, any that have symlink_node_modules = False
do not need their node_modules folders be added to managed_directories
in your WORKSPACE or in .bazelignore
. If you add them anyway it won't break anything.
If the managed_directories feature breaks you, please file an issue on this repo to let us know. Then you can proceed to 0.30 release without the feature by adding symlink_node_modules = False
to any affected npm_install
/yarn_install
rules.
Due to an issue on Windows with managed directories and the node_modules/.bin
folder, no BUILD file is generated under node_mdoules/.bin
. If you were relying on targets in this BUILD file prior to 0.30 this will be an additional breaking change for you. You can still reference files in .bin
with labels such as @npm//:node_modules/.bin/foobar
or if you require a filegroup group with a multiple files in the .bin
folder you can use manual_build_file_contents
of your yarn_install
or npm_install
rule to add the appropriate filegroup to the root @npm
build file .
For example,
yarn_install(
name = "npm",
package_json = "//:package.json",
yarn_lock = "//:yarn.lock",
manual_build_file_contents = """
filegroup(
name = "bin_files",
srcs = glob(["node_modules/.bin/*"]),
)"""
)
would create target @npm//:bin_files
which would contain all files in the node_modules/.bin
folder. If this doesn't work for you please file an issue and we'll help you find an appropriate fix.
We are making a breaking change for version 0.13, related to loading npm dependencies.
nodejs_binary
and other rules that depend on it typically accept a node_modules
attribute. This attribute was optional and defaulted to @//:node_modules
, which means "find the node_modules
target in the root of the workspace where the build is happening."
However, we now are moving to a model where npm dependencies can be expressed as deps
(or data
) of rules. That means that a nodejs_binary
that requests specific npm modules as dependencies wants to get no additional modules through the node_modules
attribute.
To make this possible, node_modules
will now default to an empty filegroup. To unbreak your code, you can typically just set node_modules = "@//:node_modules"
- that is, inline the previous default into an explicit attribute.
As of 0.13.2, this also applies to rollup_bundle
which has a node_modules
attribute that now defaults to an empty filegroup.
Bazel version 0.17.1 is required as of version 0.13.0 in order to support fined grained npm dependency labels. Bazel 0.17.1 adds support for @
in label names which is required for labels such as @npm//:@types/jasmine
.
We used to take all transitive JS files and hand them to Jasmine as specs.
This causes a problem when some files have global side-effects (e.g. calling process.exit
)
We now filter only files ending with spec.js
or test.js
and ask Jasmine to execute those.