Skip to content

Commit

Permalink
feat: support for multi-linked first party dependencies
Browse files Browse the repository at this point in the history
Adds the ability to link first party deps to subdirectories and adds a new `links` to `yarn_install` and `npm_install` which simplifies linking first party deps into `yarn_install` and `npm_install` managed `node_modules` trees.

The first release of the multi-linker, which landed in 3.2.0, supported linking only 3rd party deps to subdirectories.

------------------------------

This new feature is enabled by new `npm_link` which can add a `LinkablePackageInfo` to any target.

If a target already provides a `LinkablePackageInfo`, it provides a new `LinkablePackageInfo` with a new provided values for `package_path` and `package_name` which informs the linker into which `${package_path}/node_modules/${package_name}` folder to link the package into.

For example, given

```
lib_a/BUILD.bazel:

js_library(
  name = "lib_a",
  srcs = [
    "index.js",
    "package.json",
  ],
  package_name = "@somescope/lib-a"
)
```

which makes `//lib_a:lib_a` link to the root node_modules at `node_modules/@somescope/lib-a`

`npm_link` can be used to create a new `lib_a` target that links elsewhere:

```
sub/BUILD.bazel:

npm_link(
  name = "lib_a",
  target = "//lib_a",
  package_path = "sub",
  package_name = "@somescope/lib-a",
)
```

which makes `//sub:lib_a` linked to `sub/node_modules/@somescope/lib-a`.

Note: a target may depend on both `//lib_a:lib_a` and `//sub:lib_a` in its `deps` and the linker will link the library to both
`node_modules/@somescope/lib-a` and `sub/node_modules/@somescope/lib-a` respectively.

Alternately, `npm_link` can add a `LinkablePackageInfo` provider to a target that doesn't yet have a `package_name` associated with it,

For example,

```
lib_b/BUILD.bazel:

js_library(
  name = "lib_b",
  srcs = [
    "index.js",
    "package.json",
  ],
)
```

```
sub/BUILD.bazel:

npm_link(
  name = "lib_b",
  target = "//lib_b",
  package_path = "sub",
  package_name = "@somescope/lib-b",
)
```

which makes `//sub:lib_b` linked to `sub/node_modules/@somescope/lib-b` while `//lib_b:lib_b` is not linked at all.

The linked target can be of any type, including a simple filegroup. For example,

```
lib_c/BUILD.bazel:

filegroup(
  name = "lib_c",
  srcs = [
    "index.js",
    "package.json",
  ],
)
```

```
sub/BUILD.bazel:

npm_link(
  name = "lib_c",
  target = "//lib_c",
  package_path = "sub",
  package_name = "@somescope/lib-c",
)
```

which makes `//sub:lib_c` linked to `sub/node_modules/@somescope/lib-c` while `//lib_c:lib_c` is not linked at all.

------------------------------

With the above abilities, we've also added syntactical sugar to `yarn_install` and `npm_install` which uses `npm_link` under the hood.

For example, given a `sub/package.json` we can define its `yarn_install` as,

```
WORKSPACE:

yarn_install(
    name = “@sub_npm_deps”,
    package_json = "//sub:package.json",
    package_path = "sub",
    links = {
      "@somescope/lib-a": "//lib_a:lib_a",
      "@somescope/lib-b": "//lib_b:lib_b",
    }
)
```

which generates two `npm_link` under the hood:

```
@sub_npm_deps//@somescope/lib-a:BUILD.bazel:

npm_link(
  name = "lib-a",
  target = "//lib_a:lib_a",
  package_path = "sub",
  package_name = "@somescope/lib-a",
)
```

```
@sub_npm_deps//@somescope/lib-b:BUILD.bazel:

npm_link(
  name = "lib-b",
  target = "//lib_b:lib_b",
  package_path = "sub",
  package_name = "@somescope/lib-b",
)
```

which are both linked to `sub/node_modules` (`sub/node_modules/@somescope/lib-a` and `sub/node_modules/@somescope/lib-b` respectively).

This allows the downstream syntactical sugar of depending on first party deps by their package names from the external workspace they are "linked" to.

```
sub/BUILD.bazel:

nodejs_binary(
  name = "bin",
  entry_point = "bin.js",
  data = [
    "bin.js"
    "@sub_npm_deps//@somescope/lib-a",
    "@sub_npm_deps//@somescope/lib-b",
  ]
)
```

and those deps will be available to the application in `sub/node_modules` where you would expect them to be if they had been defined in the `sub/package.json` itself using yarn workspaces outside of bazel.

`sub/bin.js` can then require those libs with,

```
const liba = require('@somescope/lib-a')
const libb = require('@somescope/lib-b')
```

and those will be resolved to `sub/node_modules/@somescope/lib-a/index.js` and `sub/node_modules/@somescope/lib-b/index.js` with standard node_modules resolution.
  • Loading branch information
Greg Magolan authored and alexeagle committed Apr 13, 2021
1 parent be184c2 commit e90b4ae
Show file tree
Hide file tree
Showing 119 changed files with 12,786 additions and 446 deletions.
194 changes: 188 additions & 6 deletions WORKSPACE
Original file line number Diff line number Diff line change
Expand Up @@ -18,8 +18,8 @@ workspace(
"@angular_deps": ["packages/angular/node_modules"],
# cypress_deps must be a managed directory to ensure it is downloaded before cypress_repository is run.
"@cypress_deps": ["packages/cypress/test/node_modules"],
"@internal_test_multi_linker_sub_deps": ["internal/linker/test/multi_linker/sub/node_modules"],
"@npm": ["node_modules"],
"@npm_internal_linker_test_multi_linker": ["internal/linker/test/multi_linker/node_modules"],
"@npm_node_patches": ["packages/node-patches/node_modules"],
},
)
Expand Down Expand Up @@ -54,23 +54,145 @@ yarn_install(
environment = {
"SOME_USER_ENV": "yarn is great!",
},
links = {
"@test_multi_linker/lib-a": "//internal/linker/test/multi_linker/lib_a",
"@test_multi_linker/lib-a2": "//internal/linker/test/multi_linker/lib_a",
"@test_multi_linker/lib-b": "@//internal/linker/test/multi_linker/lib_b",
"@test_multi_linker/lib-b2": "@//internal/linker/test/multi_linker/lib_b",
"@test_multi_linker/lib-c": "@build_bazel_rules_nodejs//internal/linker/test/multi_linker/lib_c",
"@test_multi_linker/lib-c2": "@build_bazel_rules_nodejs//internal/linker/test/multi_linker/lib_c",
"@test_multi_linker/lib-d": "@build_bazel_rules_nodejs//internal/linker/test/multi_linker/lib_d",
"@test_multi_linker/lib-d2": "@build_bazel_rules_nodejs//internal/linker/test/multi_linker/lib_d",
},
package_json = "//:package.json",
yarn_lock = "//:yarn.lock",
)

yarn_install(
name = "npm_internal_linker_test_multi_linker",
name = "internal_test_multi_linker_deps",
links = {
"@test_multi_linker/lib-a": "@//internal/linker/test/multi_linker/lib_a",
"@test_multi_linker/lib-a2": "@//internal/linker/test/multi_linker/lib_a",
"@test_multi_linker/lib-b": "@//internal/linker/test/multi_linker/lib_b",
"@test_multi_linker/lib-b2": "@//internal/linker/test/multi_linker/lib_b",
"@test_multi_linker/lib-c": "@//internal/linker/test/multi_linker/lib_c",
"@test_multi_linker/lib-c2": "@//internal/linker/test/multi_linker/lib_c",
"@test_multi_linker/lib-d": "@//internal/linker/test/multi_linker/lib_d",
"@test_multi_linker/lib-d2": "@//internal/linker/test/multi_linker/lib_d",
},
package_json = "//internal/linker/test/multi_linker:package.json",
package_path = "internal/linker/test/multi_linker",
symlink_node_modules = False,
yarn_lock = "//internal/linker/test/multi_linker:yarn.lock",
)

yarn_install(
name = "onepa_npm_deps",
package_json = "//internal/linker/test/multi_linker/onepa:package.json",
package_path = "internal/linker/test/multi_linker/onepa",
name = "internal_test_multi_linker_test_a_deps",
links = {
"@test_multi_linker/lib-a": "@//internal/linker/test/multi_linker/lib_a",
"@test_multi_linker/lib-a2": "@//internal/linker/test/multi_linker/lib_a",
"@test_multi_linker/lib-b": "@//internal/linker/test/multi_linker/lib_b",
"@test_multi_linker/lib-b2": "@//internal/linker/test/multi_linker/lib_b",
"@test_multi_linker/lib-c": "@//internal/linker/test/multi_linker/lib_c",
"@test_multi_linker/lib-c2": "@//internal/linker/test/multi_linker/lib_c",
"@test_multi_linker/lib-d": "@//internal/linker/test/multi_linker/lib_d",
"@test_multi_linker/lib-d2": "@//internal/linker/test/multi_linker/lib_d",
},
package_json = "//internal/linker/test/multi_linker/test_a:package.json",
package_path = "internal/linker/test/multi_linker/test_a",
symlink_node_modules = False,
yarn_lock = "//internal/linker/test/multi_linker/test_a:yarn.lock",
)

yarn_install(
name = "internal_test_multi_linker_test_b_deps",
package_json = "//internal/linker/test/multi_linker/test_b:package.json",
package_path = "internal/linker/test/multi_linker/test_b",
symlink_node_modules = False,
yarn_lock = "//internal/linker/test/multi_linker/test_b:yarn.lock",
)

yarn_install(
name = "internal_test_multi_linker_test_c_deps",
package_json = "//internal/linker/test/multi_linker/test_c:package.json",
package_path = "internal/linker/test/multi_linker/test_c",
symlink_node_modules = False,
yarn_lock = "//internal/linker/test/multi_linker/test_c:yarn.lock",
)

yarn_install(
name = "internal_test_multi_linker_test_d_deps",
package_json = "//internal/linker/test/multi_linker/test_d:package.json",
package_path = "internal/linker/test/multi_linker/test_d",
symlink_node_modules = False,
yarn_lock = "//internal/linker/test/multi_linker/test_d:yarn.lock",
)

yarn_install(
name = "internal_test_multi_linker_lib_b_deps",
# transitive deps for this first party lib should not include dev dependencies
args = ["--production"],
package_json = "//internal/linker/test/multi_linker/lib_b:package.json",
package_path = "internal/linker/test/multi_linker/lib_b",
symlink_node_modules = False,
yarn_lock = "//internal/linker/test/multi_linker/onepa:yarn.lock",
yarn_lock = "//internal/linker/test/multi_linker/lib_b:yarn.lock",
)

yarn_install(
name = "internal_test_multi_linker_lib_c_deps",
# transitive deps for this first party lib should not include dev dependencies
args = ["--production"],
package_json = "//internal/linker/test/multi_linker/lib_c:lib/package.json",
package_path = "internal/linker/test/multi_linker/lib_c/lib",
symlink_node_modules = False,
yarn_lock = "//internal/linker/test/multi_linker/lib_c:lib/yarn.lock",
)

yarn_install(
name = "internal_test_multi_linker_sub_dev_deps",
links = {
"@test_multi_linker/lib-a": "//internal/linker/test/multi_linker/lib_a",
"@test_multi_linker/lib-a2": "//internal/linker/test/multi_linker/lib_a",
"@test_multi_linker/lib-b": "//internal/linker/test/multi_linker/lib_b",
"@test_multi_linker/lib-b2": "//internal/linker/test/multi_linker/lib_b",
"@test_multi_linker/lib-c": "//internal/linker/test/multi_linker/lib_c",
"@test_multi_linker/lib-c2": "//internal/linker/test/multi_linker/lib_c",
"@test_multi_linker/lib-d": "//internal/linker/test/multi_linker/lib_d",
"@test_multi_linker/lib-d2": "//internal/linker/test/multi_linker/lib_d",
},
package_json = "//internal/linker/test/multi_linker/sub:package.json",
package_path = "internal/linker/test/multi_linker/sub/dev",
symlink_node_modules = False,
yarn_lock = "//internal/linker/test/multi_linker/sub:yarn.lock",
)

yarn_install(
name = "internal_test_multi_linker_sub_deps",
# transitive deps for this first party lib should not include dev dependencies
args = ["--production"],
links = {
"@test_multi_linker/lib-a": "@build_bazel_rules_nodejs//internal/linker/test/multi_linker/lib_a",
"@test_multi_linker/lib-a2": "@build_bazel_rules_nodejs//internal/linker/test/multi_linker/lib_a",
"@test_multi_linker/lib-b": "@build_bazel_rules_nodejs//internal/linker/test/multi_linker/lib_b",
"@test_multi_linker/lib-b2": "@build_bazel_rules_nodejs//internal/linker/test/multi_linker/lib_b",
"@test_multi_linker/lib-c": "@build_bazel_rules_nodejs//internal/linker/test/multi_linker/lib_c",
"@test_multi_linker/lib-c2": "@build_bazel_rules_nodejs//internal/linker/test/multi_linker/lib_c",
"@test_multi_linker/lib-d": "@build_bazel_rules_nodejs//internal/linker/test/multi_linker/lib_d",
"@test_multi_linker/lib-d2": "@build_bazel_rules_nodejs//internal/linker/test/multi_linker/lib_d",
},
package_json = "//internal/linker/test/multi_linker/sub:package.json",
package_path = "internal/linker/test/multi_linker/sub",
yarn_lock = "//internal/linker/test/multi_linker/sub:yarn.lock",
)

yarn_install(
name = "internal_test_multi_linker_onep_a_deps",
# transitive deps for this first party lib should not include dev dependencies
args = ["--production"],
package_json = "//internal/linker/test/multi_linker/onep_a:package.json",
package_path = "internal/linker/test/multi_linker/onep_a",
symlink_node_modules = False,
yarn_lock = "//internal/linker/test/multi_linker/onep_a:yarn.lock",
)

npm_install(
Expand Down Expand Up @@ -273,6 +395,61 @@ yarn_install(
".json",
".proto",
],
links = {
"@some-scope/some-target-b": "@//some/target/b",
"@some-scope/some-target-b2": "@//some/target/b",
"some-target-a": "//some/target/a",
"some-target-a2": "//some/target/a",
},
manual_build_file_contents = """
filegroup(
name = "golden_files",
srcs = [
"//:BUILD.bazel",
"//:manual_build_file_contents",
"//:WORKSPACE",
"//@angular/core:BUILD.bazel",
"//@gregmagolan:BUILD.bazel",
"//@gregmagolan/test-a/bin:BUILD.bazel",
"//@gregmagolan/test-a:BUILD.bazel",
"//@gregmagolan/test-a:index.bzl",
"//@gregmagolan/test-b:BUILD.bazel",
"//ajv:BUILD.bazel",
"//jasmine/bin:BUILD.bazel",
"//jasmine:BUILD.bazel",
"//jasmine:index.bzl",
"//rxjs:BUILD.bazel",
"//unidiff:BUILD.bazel",
"//zone.js:BUILD.bazel",
"//some-target-a:BUILD.bazel",
"//some-target-a2:BUILD.bazel",
"//@some-scope/some-target-b:BUILD.bazel",
"//@some-scope/some-target-b2:BUILD.bazel",
],
)""",
package_json = "//:tools/fine_grained_goldens/package.json",
symlink_node_modules = False,
yarn_lock = "//:tools/fine_grained_goldens/yarn.lock",
)

yarn_install(
name = "fine_grained_goldens_multi_linked",
included_files = [
"",
".js",
".jst",
".ts",
".map",
".d.ts",
".json",
".proto",
],
links = {
"@some-scope/some-target-b": "@//some/target/b",
"@some-scope/some-target-b2": "@//some/target/b",
"some-target-a": "@build_bazel_rules_nodejs//some/target/a",
"some-target-a2": "@build_bazel_rules_nodejs//some/target/a",
},
manual_build_file_contents = """
filegroup(
name = "golden_files",
Expand All @@ -293,9 +470,14 @@ filegroup(
"//rxjs:BUILD.bazel",
"//unidiff:BUILD.bazel",
"//zone.js:BUILD.bazel",
"//some-target-a:BUILD.bazel",
"//some-target-a2:BUILD.bazel",
"//@some-scope/some-target-b:BUILD.bazel",
"//@some-scope/some-target-b2:BUILD.bazel",
],
)""",
package_json = "//:tools/fine_grained_goldens/package.json",
package_path = "tools/fine_grained_goldens",
symlink_node_modules = False,
yarn_lock = "//:tools/fine_grained_goldens/yarn.lock",
)
Expand Down
16 changes: 8 additions & 8 deletions examples/user_managed_deps/BUILD.bazel
Original file line number Diff line number Diff line change
@@ -1,12 +1,15 @@
load("@build_bazel_rules_nodejs//:index.bzl", "js_library", "nodejs_binary", "nodejs_test")

# This is "user-managed" dependencies - Bazel knows nothing about our package manager.
# We assume that developers will install the `node_modules` directory themselves and
# keep it up-to-date.
# This rule simply exposes files in the node_modules directory to Bazel using js_library
# with external_npm_package set to True so it can read them as inputs to processes it spawns.
# This is "user-managed" dependencies - Bazel knows nothing about our package
# manager. We assume that developers will install the `node_modules` directory
# themselves and keep it up-to-date. This rule simply exposes files in the
# node_modules directory to Bazel using js_library with package_name set to
# "node_modules" so it can read them as inputs to processes it spawns.
js_library(
name = "node_modules",
# Special value to provide ExternalNpmPackageInfo which is used by downstream
# rules that use these npm dependencies
package_name = "$node_modules$",
srcs = glob(
include = [
"node_modules/**/*.js",
Expand All @@ -25,9 +28,6 @@ js_library(
"node_modules/**/* *",
],
),
# Provide ExternalNpmPackageInfo which is used by downstream rules
# that use these npm dependencies
external_npm_package = True,
)

nodejs_binary(
Expand Down
Loading

0 comments on commit e90b4ae

Please sign in to comment.