Add NODE_PRESERVE_SYMLINKS environment variable #8749

mlucool · 2016-09-23T19:07:34Z

Checklist

make -j8 test (UNIX), or vcbuild test nosign (Windows) passes
documentation is changed or added
commit message follows commit guidelines

Affected core subsystem(s)

cli

Description of change

Add a way through environment variables to set the --preserve-symlinks
flag. A value of '1' for NODE_PRESERVE_SYMLINKS will enable symlinks.

Fixes: #8509

addaleax · 2016-09-23T19:09:05Z

doc/api/cli.md

@@ -277,6 +277,12 @@ added: v0.11.15
 Data path for ICU (Intl object) data. Will extend linked-in data when compiled
 with small-icu support.

+### `NODE_PRESERVE_SYMLINKS=true`
+<!-- YAML
+added: v6.7.0.


There should be no . here, and generally it’s easiest to simply use added: REPLACEME (this likely won’t end up in v6.7.0, which is going to be a security release on Tuesday).

addaleax · 2016-09-23T19:09:18Z

doc/api/cli.md

+<!-- YAML
+added: v6.7.0.
+-->
+Instructs the module loader to preserve symbolic links when resolving and


nit: Could you leave an empty line before this?

mlucool · 2016-09-23T19:13:56Z

Fixed

imyller

LGTM

bnoordhuis · 2016-09-23T21:49:50Z

src/node.cc

+  // Allow for environment set preserving symlinks
+  if (secure_getenv("NODE_PRESERVE_SYMLINKS")) {
+     config_preserve_symlinks = true;
+  }


Please punctuate the comment and can you add a blank line after the block?

The way it's currently written means env NODE_PRESERVE_SYMLINKS= node will turn on the flag. Most seasoned UNIX users probably won't expect that.

Syntax fixed.

What you suggest is the best way to parse the flag, any value except for empty string? Should it handle env NODE_PRESERVE_SYMLINKS=false ?

I'd write it as e.g.:

if (auto var = secure_getenv("NODE_PRESERVE_SYMLINKS")) config_preserve_symlinks = (*var != '\0');

An empty string is false, everything else is true. That includes blank strings but I think that is an acceptable functionality vs. complexity trade-off.

jasnell

LGTM

Fishrock123 · 2016-09-26T13:37:06Z

Uncertain we should make more new env variables for people to rely on here, not sure we will be supporting this with ES Modules... Don't need more to break than already will. 😐

mlucool · 2016-09-26T14:06:01Z

Ideally we should support a .noderc file to allow for setting like this, similar to .npmrc. This seems like the most reasonable way to support it given current constraints.

bnoordhuis · 2016-09-26T18:44:07Z

Config file support gets brought up from time to time but node has always been a self-contained binary and IMO it should stay that way.

mlucool · 2016-09-26T20:05:31Z

I understand and am not trying to push one way or another, just buying into what is already there. Any more changes before this can be merged? Can this be pulled into the 6.x series also?

evanlucas · 2016-09-27T10:30:04Z

src/node.cc

@@ -4188,6 +4188,11 @@ void Init(int* argc,
  V8::SetFlagsFromString(NODE_V8_OPTIONS, sizeof(NODE_V8_OPTIONS) - 1);
 #endif

+  // Allow for environment set preserving symlinks.
+  if (auto preserve_symlinks = secure_getenv("NODE_PRESERVE_SYMLINKS")) {


Does this work on Windows?

Which part of this commit are you concerned will not work on windows?

ah, disregard. I misread it. Thanks!

jasnell · 2016-09-27T20:38:17Z

@Fishrock123 ... I understand the concern but this really doesn't add a new thing that would be broken as much as it adds a new switch to enable an existing thing (that would be broken). Given that, it's likely less of a concern. If it helps, I would be +1 on leaving this undocumented.

mlucool · 2016-10-01T14:46:23Z

I would prefer we document all features then have hidden ones to use at your own risk.

@Fishrock123 are you ok with merging this as is?

jasnell · 2016-10-06T22:30:04Z

@nodejs/ctc ... any thoughts on this?

bnoordhuis · 2016-10-07T09:44:03Z

src/node.cc

@@ -4188,6 +4188,11 @@ void Init(int* argc,
  V8::SetFlagsFromString(NODE_V8_OPTIONS, sizeof(NODE_V8_OPTIONS) - 1);
 #endif

+  // Allow for environment set preserving symlinks.
+  if (auto preserve_symlinks = secure_getenv("NODE_PRESERVE_SYMLINKS")) {
+     config_preserve_symlinks = (*preserve_symlinks != '\0');


Indentation seems to be off here and I'm having second thoughts about interpreting NODE_PRESERVE_SYMLINKS=0 as "yes please, enable this feature." Perhaps a more robust atoi or strtol-based parser is in order.

Whatever you prefer I will gladly change to. Just let me know.

For consistency with other env variables, it should be enabled when set to 1 and disabled otherwise.

So you are suggesting:
config_preserve_symlinks = (*preserve_symlinks == '1');

NODE_PRESERVE_SYMLINKS=1

This what we do NODE_DISABLE_COLORS=1 and NODE_TTY_UNSAFE_ASYNC=1.

See https://nodejs.org/dist/latest-v6.x/docs/api/cli.html#cli_environment_variables

Tested and updated

bnoordhuis · 2016-10-07T09:45:41Z

any thoughts on this?

I somewhat agree with @Fishrock123 that more knobs are not necessarily a good thing. More ways to shoot yourself in the foot, more baggage for us to maintain.

mlucool · 2016-10-07T12:54:43Z

@bnoordhuis My reason for this commit is although the ability to set a flag is great, without a way to always enable this feature it becomes very error prone. The errors associated with a missing symlink flag are often non-obvious to a non-expert.

The reason we want to use symlinks is to provide a set of build tools in an enterprise environment without our large number of projects installing a huge redundant sets of dependencies (costs include extra time, dependency management, and inodes). In general, good symlink support is a huge benefit to our enterprise environments.

That being said, aside from wrapping node in script named node (which seems hacky and likely to cause issues when people want to try new versions of node) how else should we maintain certain flags to always be set?

deregtd · 2016-10-12T20:44:48Z

Oh god I can't wait until this goes in. Then it can work with react-native as well! So close to actually being able to develop with symlinks!

amakhrov · 2016-10-12T21:04:45Z

One more use case is the IED package manager, which makes heavy use of symlinks

mlucool · 2016-10-13T12:34:47Z

@bnoordhuis does this look good to you now? Any further changes?

bnoordhuis · 2016-10-13T14:27:17Z

Technical side LGTM but needs a few regression tests. You can look at test/parallel/test-require-symlink.js for examples.

mlucool · 2016-10-13T21:37:28Z

Thanks for the tip @bnoordhuis . Let me know if you have any further comments. It seemed to me like it should go in the same test file as it is testing another variant of the same thing. Copying and pasting the whole case seemed to not be DRY. I could make it two test cases in this file if that is better.

Trott · 2016-10-20T20:03:21Z

Raising visibility: @nodejs/collaborators

sam-github · 2016-10-21T16:41:10Z

I'm generally in favour of any node CLI option being able to be controlled via environment.

Rather than doing this on an ad-hoc, CLI argument by argument basis, I wish that the PR to add an env variable from which any CLI option had progressed. It apparently didn't complete, and I can't find it in https://github.com/nodejs/node-v0.x-archive or here.

@Fishrock123 @misterdjules Do you remember the PR I'm talking about? It proposed a NODE_OPTIONS flag equivalent to RUBYOPT or PERL5OPT.

EDIT: actually, found it, and it didn't, it did only the minimal CLI option thing: #881

Anyhow, I know its frustrating to not get a feature you need now, because of some unspecified maybe in the future feature would be better, so I guess I'd have to be +1 on this.

imyller · 2016-10-24T12:12:14Z

CI: https://ci.nodejs.org/job/node-test-pull-request/4635/

imyller · 2016-10-24T19:17:12Z

Landing:

Approvals (LGTM): 6
No objections
PR has been open for the minimum time of 48 or 72 hours
All of the requested changes have been made
CI tests passed (only known/unrelated failures)

Add a way through environment variables to set the --preserve-symlinks flag. Any non-null value of NODE_PRESERVE_SYMLINKS will enable symlinks. PR-URL: nodejs#8749 Fixes: nodejs#8509 Reviewed-By: Ilkka Myller <ilkka.myller@nodefield.com> Reviewed-By: James M Snell <jasnell@gmail.com> Reviewed-By: Ben Noordhuis <info@bnoordhuis.nl> Reviewed-By: Colin Ihrig <cjihrig@gmail.com> Reviewed-By: Anna Henningsen <anna@addaleax.net> Reviewed-By: Rich Trott <rtrott@gmail.com>

imyller · 2016-10-24T19:19:26Z

Landed in d3b1a2b

Thank you, @mlucool

mlucool · 2016-10-24T19:37:56Z

@imyller thanks for merging! Can this be pulled into the 6.x branch also?

MylesBorins · 2016-10-24T19:43:39Z

I've thrown an lts watch and lts-agenda tag on here. We will discuss it at the next meeting.

mlucool · 2016-10-24T19:45:32Z

Thanks!

Add a way through environment variables to set the --preserve-symlinks flag. Any non-null value of NODE_PRESERVE_SYMLINKS will enable symlinks. PR-URL: #8749 Fixes: #8509 Reviewed-By: Ilkka Myller <ilkka.myller@nodefield.com> Reviewed-By: James M Snell <jasnell@gmail.com> Reviewed-By: Ben Noordhuis <info@bnoordhuis.nl> Reviewed-By: Colin Ihrig <cjihrig@gmail.com> Reviewed-By: Anna Henningsen <anna@addaleax.net> Reviewed-By: Rich Trott <rtrott@gmail.com>

zkochan · 2016-11-04T12:55:59Z

@Fishrock123, you said

Uncertain we should make more new env variables for people to rely on here, not sure we will be supporting this with ES Modules... Don't need more to break than already will. 😐

Do you mean the preserve symlinks feature might get deprecated? I am one of the contributors at pnpm and the preserve symlinks option is the only reason pnpm can still compete with yarn. (We use a single storage for packages in the file system)

If there's a discussion somewhere about the possibility of --preserve-symlinks deprecation, could you please point me to it?

refack · 2016-11-24T18:18:10Z

What are the chances of rewriting the commit message (~~Any non-null value of~~ a value of "1"). Since it's reflected in the changelogs.

mlucool · 2017-02-23T16:21:30Z

@MylesBorins are there notes on why this was not going to land on 4/6?

MylesBorins · 2017-02-24T14:35:08Z

@mlucool too high a probability of behavior changes

refack · 2017-04-13T17:18:29Z

@mlucool I edited the description to match behaviour and docs
REF: #12366 (comment) my sad "war-story"

nodejs-github-bot added the c++ Issues and PRs that require attention from people who are familiar with C++. label Sep 23, 2016

addaleax reviewed Sep 23, 2016

View reviewed changes

addaleax added module Issues and PRs related to the module subsystem. semver-minor PRs that contain new features and should be released in the next minor version. labels Sep 23, 2016

mlucool force-pushed the symlink-env-var branch from b88ef6f to 3cf619e Compare September 23, 2016 19:13

imyller approved these changes Sep 23, 2016

View reviewed changes

bnoordhuis reviewed Sep 23, 2016

View reviewed changes

mlucool force-pushed the symlink-env-var branch from 3cf619e to 3c08bdb Compare September 23, 2016 21:52

jasnell approved these changes Sep 23, 2016

View reviewed changes

mlucool force-pushed the symlink-env-var branch from 3c08bdb to 3531431 Compare September 26, 2016 13:28

evanlucas reviewed Sep 27, 2016

View reviewed changes

bnoordhuis requested changes Oct 7, 2016

View reviewed changes

mlucool force-pushed the symlink-env-var branch 2 times, most recently from e69f324 to baaac09 Compare October 7, 2016 20:10

mlucool force-pushed the symlink-env-var branch from baaac09 to 98e7567 Compare October 13, 2016 21:34

imyller closed this Oct 24, 2016

imyller removed their assignment Oct 24, 2016

MylesBorins added lts-watch-v6.x labels Oct 24, 2016

evanlucas mentioned this pull request Nov 3, 2016

v7.1.0 proposal - 2016-11-08 #9438

Merged

zkochan mentioned this pull request Nov 4, 2016

Override the node command when using pnpm run pnpm/pnpm#463

Closed

alexanderGugel mentioned this pull request Nov 8, 2016

Hotfix release needed - Issue with latest rxjs release candidate alexanderGugel/ied#178

Closed

MylesBorins added the lts-agenda label Nov 18, 2016

MylesBorins added dont-land-on-v4.x and removed lts-watch-v6.x labels Jan 23, 2017

vjpr mentioned this pull request Feb 6, 2017

Support peer dependencies when symlinking in development pnpm/pnpm#604

Closed

mlucool mentioned this pull request Mar 8, 2017

RFC: file: specifier changes npm/npm#15900

Closed

sam-github mentioned this pull request Mar 23, 2017

Proposal: --config=file command line argument #11997

Closed

refack mentioned this pull request Apr 13, 2017

src: reduce number of exported symbols #12366

Closed

2 tasks

Add NODE_PRESERVE_SYMLINKS environment variable #8749

Add NODE_PRESERVE_SYMLINKS environment variable #8749

Conversation

mlucool commented Sep 23, 2016 • edited by refack Loading

Checklist

Affected core subsystem(s)

Description of change

Choose a reason for hiding this comment

addaleax Sep 23, 2016 • edited Loading

Choose a reason for hiding this comment

mlucool commented Sep 23, 2016

imyller left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

jasnell left a comment

Choose a reason for hiding this comment

Fishrock123 commented Sep 26, 2016

mlucool commented Sep 26, 2016

bnoordhuis commented Sep 26, 2016

mlucool commented Sep 26, 2016

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

jasnell commented Sep 27, 2016

mlucool commented Oct 1, 2016

jasnell commented Oct 6, 2016

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Fishrock123 Oct 7, 2016 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

bnoordhuis commented Oct 7, 2016

mlucool commented Oct 7, 2016

deregtd commented Oct 12, 2016

amakhrov commented Oct 12, 2016

mlucool commented Oct 13, 2016

bnoordhuis commented Oct 13, 2016

mlucool commented Oct 13, 2016

Trott commented Oct 20, 2016

sam-github commented Oct 21, 2016

imyller commented Oct 24, 2016

imyller commented Oct 24, 2016

imyller commented Oct 24, 2016

mlucool commented Oct 24, 2016

MylesBorins commented Oct 24, 2016

mlucool commented Oct 24, 2016

zkochan commented Nov 4, 2016 • edited Loading

refack commented Nov 24, 2016

mlucool commented Feb 23, 2017

MylesBorins commented Feb 24, 2017

refack commented Apr 13, 2017

mlucool commented Sep 23, 2016 •

edited by refack

Loading

addaleax Sep 23, 2016 •

edited

Loading

Fishrock123 Oct 7, 2016 •

edited

Loading

zkochan commented Nov 4, 2016 •

edited

Loading