Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Idea: support package-lock.json from npm 5 #3614

Closed
kof opened this issue Jun 9, 2017 · 29 comments
Closed

Idea: support package-lock.json from npm 5 #3614

kof opened this issue Jun 9, 2017 · 29 comments

Comments

@kof
Copy link

kof commented Jun 9, 2017

I think we will face the issue in the near future that we get 2 lock files in one project, especially because npm creates it by default. Is there any chance that yarn could use the package-lock.json from npm by default if it is detected?

It would be especially useful for libraries when maintainer uses package-lock, others can still use yarn when installing it.

@kof kof changed the title Idea: support package-lockfile.json from npm 5 Idea: support package-lock.json from npm 5 Jun 9, 2017
@paulirish
Copy link
Member

I expect this will change in the future, but... What is the current best practice for handling both package-lock.json and yarn.lock?

These are the options I'm seeing...

  1. Don't, and only support one of these in your project.
  2. Everyone manually keeps them in sync for every dependency change
  3. Use some tool to rebuild one of the locks when you change the other. (I'm hoping for this one :)
  4. Just eat the pain and wait?

@joggienl
Copy link

joggienl commented Jul 14, 2017

I would opt for option 1 or 4.. keeping multiple files for the same in sync is tedious and error prone.
On the other side, a downside of choosing to support one of both is that you somewhat force your users to use either yarn or npm, but I think that this would be less harmful than different packages for different package managers.

So our projects use either a package-lock.json or an yarn.lock file, the first option.

@karfau
Copy link

karfau commented Jul 14, 2017

At work, we are currently in the following weird situation:

  • using yarn for 95% of our CI, but one step is using npm, because it is the only way we could make it work inside docker.
  • as we knew no way of syncing the files we decided to .gitignore package-lock.json for now, because our minimal version of npm is LTS, which doesn't support ti anyway.
  • we will soon (days or weeks) upgrade to use node v8 and npm v5 as minimal version, then we might switch to only use/support package-lock.json and .gitignore yarn.lock, in case there is no option to sync them.

The following approaches would work for us:
A) yarn using the values from package-lock.json for installing
B) yarn using the values from package-lock.json to create/update yarn.lock
C) an extra tool to create+sync yarn.lock from package-lock.json

@ghost
Copy link

ghost commented Jul 19, 2017

In the general packager case Yarn could potentially be used for Packagist (there's a prototype floating around), and perhaps even CocoaPods, in addition to NPM. A PHP project, for example, might have a composer.lock and a package.json for managing NPM scripts. Might leaning too far into NPM locking behaviors limit some of these future possibilities for packager extensibility?

@nottrobin
Copy link

If yarn starts managing other types of dependencies, it is still always expected to generate the exact same dependency versions as any other managers of the same dependencies. (yarn install or npm install will give you identical dependencies).

So I would have thought it should always follow the prevailing standard for the type of dependency. I.e., for PHP dependencies it should use composer.lock, for NPM it should use package-lock.json.

@ghost
Copy link

ghost commented Jul 20, 2017

The way I see it Composer is to Pear as Yarn is to NPM. Difference being NPM uses NPM where Composer uses Packagist. If Yarn were to maintain its own standards we could prevent gridlock and tooling complexity which could hamstring Yarn and inhibit forward momentum in other packagers.

Is it naïve to think 80% case could boil down to one packager, with additional packagers to add value for 20% and ancillary use cases? It seems such an approach might serve the majority while limiting complexity, allowing innovation on the fringes which could then be generalized and pulled into Yarn as the value proposition becomes clear longer-term.

It has always seemed to me this is what NPM itself is doing right now. I could be wrong.

@nottrobin
Copy link

nottrobin commented Jul 20, 2017

Yes I agree, Yarn wants to maintain its independent space as far as possible, so it has the flexibility to provide a better experience. Which is why, for example, Yarn's CLI is deliberately not compatible with NPM's.

However, I think lock files are outside of the space where Yarn can reasonably maintain independence. package-lock.json and composer.lock are committed to the repository, alongside package.json and composer.json. These are not tool-specific files, they are instead project-specific files specifying the exact dependency versions that the project is guaranteed to work with.

When NPM wasn't offering the ability to lock dependencies, it made sense for Yarn to create its own yarn.lock. But now that NPM has defined a standard way for projects to specify the explicit dependency versions that they are known to work with, and furthermore it is named similarly to the package.json that Yarn already relies on, it doesn't make sense for Yarn to continue to go its own way.

For Yarn to be a useful tool, it has to allow developers to follow standard models in their projects, so the project can remain tool-agnostic. Which, after all, is why Yarn was built on top of package.json and not its own separate dependency file.

balaclark added a commit to balaclark/save that referenced this issue Jul 27, 2017
Mingo introduced a breaking change in v1.3.0 (removing the Mingo.setup
method). This causes failures when installing save via mongo when no
yarn.lock is present.

This project uses the new npm 5 package-lock.json, this is not supported
by yarn and so mingo gets silently upgraded from 1.1.2 to 1.3.0. It is
likely that this project will need to upgrade to the mingo 1.3.0 API,
but in the meantime this commit will ensure that 1.1.2 gets installed
when using yarn.

yarnpkg/yarn#3614
@willin
Copy link

willin commented Aug 29, 2017

mark

@johncrim
Copy link

johncrim commented Oct 5, 2017

Stack Overflow thread discussing one of the problems resulting from having 2 flavors of lock files:

https://stackoverflow.com/questions/44552348/should-i-commit-yarn-lock-and-package-lock-json-files

@ryasmi
Copy link

ryasmi commented Oct 6, 2017

Been doing some reading around this today, stumbled across this in the npm docs.

One key detail about package-lock.json is that it cannot be published, and it will be ignored if found in any place other than the toplevel package.

https://docs.npmjs.com/files/package-lock.json

Maybe this is common knowledge, but I certainly thought npm would use the lock file in dependencies. I thought that might be interesting to other people that land on this issue, since I was mostly concerned about people installing my packages with npm 5 because we're not using a package lock (we use Yarn instead).

From reading some documentation on Renovate, it seems to make sense that it doesn't use the package lock in dependencies to avoid bloating node_modules and issues with installing two separate versions of packages.

A second reason for using ranges applies to “libaries” that are published as npm packages with the intention that they are used/require()‘d by other packages. In this case, it is usually a bad idea to pin all your dependencies because it will introduce an unnecessarily narrow range (one release!) and cause most users of your package to bloat their node_modules with duplicates.

For example, you might have pinned foobar to version 1.1.0 and another author pinned his/her foobar to dependency to 1.2.2. Any user of both your packages will end up with npm attempting to install two separate versions of foobar, which might not even work.

https://renovateapp.com/docs/deep-dives/dependency-pinning

@nottrobin
Copy link

Maybe this is common knowledge, but I certainly thought npm would use the lock file in dependencies.

@ryansmith94 this behaviour in package-lock.json mimics the exact same behaviour in yarn.lock:

When you publish a package that contains a yarn.lock, any user of that library will not be affected by it. When you install dependencies in your application or library, only your own yarn.lock file is respected. Lockfiles within your dependencies will be ignored.

@ryasmi
Copy link

ryasmi commented Oct 6, 2017

Thanks @nottrobin, that's good to know 😄

@nicojs
Copy link

nicojs commented Oct 10, 2017

Can someone of yarns core contributors comment on this issue? I would be very interested to hear their opinion. ✌️

@ryasmi
Copy link

ryasmi commented Oct 10, 2017

@nicojs I mentioned it on Discord before I replied here, if I remember correctly, they hadn't had time to properly look into it, but did want to get to it. I'm assuming I was talking to a core contributor in the support channel.

@ghost
Copy link

ghost commented Oct 10, 2017

From Irish's comment above

  1. Use some tool to rebuild one of the locks when you change the other. (I'm hoping for this one :)

So who has a tool they can share?

@ahz
Copy link

ahz commented Nov 20, 2017

@nottrobin My scenario was to generate package-lock.json file for project where there is already yarn.lock file for longer period of time (and newer versions of dependencies exist). Therefore I've used npm-shrinkwrap to generate lock file for current node_modules tree (installed by yarn). npm install would generate lock file with possibly newer versions of packages. And the trick is that npm-shrinkwrap.json has same structure as package-lock.json (excepts some metadata that are added by npm install at the end).

@Triloworld
Copy link

Triloworld commented Jan 25, 2018

Now is 2018. This issue is very important in setting consistent version. Core team need investigation in this topic. More duplicated issue will be added and more problems it create than resolve.
Possible solutions from Yarn core team to choose:

  • Will synchronize two file in every action to give freedom of choose packages (good way)
  • Deprecate own lock file in favor of default package-lock.json (best in my opinion)
  • Will be only created on demand
  • NPM will add possibility to add custom changes in package-lock.json and won't be removed (merge - best for everyone)

NPM won't remove their lock file. Yarn can (pleas don't) do the sam and state will be preserved as is (two lock file) - This issue pleas then set to "wontfix" and add documentation about decision.
Need decision of core team what approach we take.

@JLarky
Copy link

JLarky commented Jan 29, 2018

NPM won't remove their lock file.

$ cat .npmrc
package-lock=false

So far this is best solution that works for me. I tried using package-lock file, but unless npm/npm#17722 is resolved I don't see how npm lock file can be considered even barely usable in any large team. So it makes sense for yarn to consider package-lock.json file and update it when changes to real lock file is made, but I don't see Deprecate own lock file in favor of default package-lock.json (best in my opinion) being possible solution any time soon

@lkraider
Copy link

lkraider commented Feb 8, 2018

On our CI we simply use:

if [ -e 'yarn.lock' ]; then
    yarn install
elif [ -e 'package-lock.json' ]; then
    npm install
fi

@nottrobin
Copy link

nottrobin commented Mar 22, 2018

This has to be one of the most popular issues ever to be (apparently) completely ignored by the project maintainers - getting on for a year now. For me this is quite a significant blemish on Yarn's otherwise pretty good reputation.

@jahed
Copy link

jahed commented May 1, 2018

For those using both npm and yarn together in the same project, how are you maintaining the CLI differences in your scripts?

Using multiple package managers in the same project sounds like using multiple version control systems in the same project, or multiple task runners for the same tasks. They do different things in the same use case which is bound to cause undocumented conflicts.

yarn and npm may look very similar in terms of features, but they are very different in implementation.
Giving yarn the freedom to add more features in their yarn.lock without going through npm sounds a lot more simple and future proof to me. Moving over to package-lock.json will cause synchronisation pains in the future when npm decides to change something for their own feature set.

Yes, npm is default right now in NodeJS, but that's a known trade-off when picking any alternative package manager. I think the competition is a good thing; as proven with npm's numerous yarn-inspired features after a long period of stagnation.

At most, if this hasn't been done already, I think yarn should detect other lock files in a project, warn about them when trying to run a command and provide a useful suggestion.

@imsnif
Copy link
Member

imsnif commented May 2, 2018

Hey @jahed - we're actually in the process of implementing the very same. Warning when another lockfile is found and giving the ability to import from it. Here's a PR of mine doing the latter as a first step:
#5745

I'm hoping we'll be able to merge it soon. You can follow more here if you like:
#5654

@BYK
Copy link
Member

BYK commented May 3, 2018

Merging into #5654.

@BYK BYK closed this as completed May 3, 2018
jmohan57 added a commit to jmohan57/crowd-source-model that referenced this issue May 27, 2024
Having both `yarn.lock` and `package-lock.json` (i.e. using both Yarn and NPM) could [easily](yarnpkg/yarn#5654) [be](yarnpkg/yarn#3614) [problematic](https://stackoverflow.com/questions/44552348/should-i-commit-yarn-lock-and-package-lock-json-files). In short, it creates a risk of out-of-sync where one author installs with NPM and forgot to update yarn.lock so another author, when using yarn, won't have that dependency.

In this PR I propose to drop the use of yarn in favor of NPM since a search in the codebase yields more NPM usages (especially in the doc and in package.json) than Yarn (only in Docker file to install).

If others have strong reasons for using Yarn over NPM then feel free to yield them. I don't have strong personal reason for one over another :D
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests