-
Notifications
You must be signed in to change notification settings - Fork 2.7k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[feature] improve resolution deduping #3778
Comments
Up For Grabs - comment here if you want to give it a try. |
I'm looking at this now, happy if someone beats me to the punch though :) |
This sounds ideal, and how I would expect it to work. Might be behind a flag, though, in case some transitive deps has to be in a specific version? If behind a flag, I'd expect a warning printed that I can dedupe if I want to. ("warning pass
Current behavior, right?
On |
It should still dedupe but less aggressively, could depend on the dependencies order. I agree with the other statements |
I know this is a feature request, but did deduping change between 26.1 and 27.5? After the upgrade, it seems my Webpack build is pulling in multiple versions of Angular and jQuery. I'm not entirely sure how to go about actually resolving this issue besides manually editing the lockfile. The angular example: angular@1.6.1:
version "1.6.1"
resolved "https://registry.yarnpkg.com/angular/-/angular-1.6.1.tgz#a7b6d763a74c157325692619e97257e69d9b6a27"
angular@>=1.2.0, angular@^1.5.8:
version "1.6.5"
resolved "https://registry.yarnpkg.com/angular/-/angular-1.6.5.tgz#37f788eebec5ce2e3fa02b17bbcb2a231576a0d6" The latter here is a transitive dependency pulled in from another library, which allows The jQuery dependency is similar. Former here is my own supplied version, latter being transitive: jquery@3.1.1:
version "3.1.1"
resolved "https://registry.yarnpkg.com/jquery/-/jquery-3.1.1.tgz#347c1c21c7e004115e0a4da32cece041fad3c8a3"
jquery@>=1.10, jquery@>=1.6, jquery@>=1.8.0, jquery@>=1.9.1:
version "3.2.1"
resolved "https://registry.yarnpkg.com/jquery/-/jquery-3.2.1.tgz#5c4d9de652af6cd0a770154a631bba12b015c787" On the topic of the feature request, can we just reuse |
@mtraynham I keep around an older installation of yarn in order to dedupe, as it's broken in newer versions of yarn. You can manually edit the lockfile to dedupe it as well. |
I'm no longer seeing the dedupe problem with 0.27.5. |
I'm on 0.27.5 and still have duplication problems. I have a dependency tree like this
And when I |
There is a currently failing test (skipped) in master that should serve as the source of truth for if this is still happening. |
I can confirm the behaviour described by @xdumaine still occurs on Yarn v1.0.2. The only workarounds I know of are:
|
@OliverJAsh #4488 makes that test in #3779 passing. Maybe you'd like to try the build for that PR: https://5262-49970642-gh.circle-artifacts.com/0/home/ubuntu/yarn/artifacts/yarn-legacy-1.0.2.js |
@BYK i checked it on my project and i still get duplicated deps
|
@Bnaya #4488 intentionally does not rewrite the lockfile, because of the concerns raised in #79 – but I would expect it to produce a more flat lockfile when generating one from scratch ( |
@OliverJAsh @Bnaya You might want to try https://www.npmjs.com/package/yarn-tools to automatically deduplicate the lockfile. |
Will give a look, thanks! |
Why is a third party tool needed for de-duplication anyway? Ie in which circumstances does yarn generate "correct" lock files that require de-duplicating? This bit has me confused. |
@rarkins, for example:
Gives you
And usually after yarn upgrade you can get this dups if you are upgrading a direct dependency that is also dependency of dependency |
@rarkins: Here is a pretty minimal repro:
Note that ^3.0.0 and >=1.0.0 are compatible ranges, but because of the installation order you end up with two versions on disk. |
I want to point out that yarn-tools from |
It's been a while since this issue has had any activity, but this is still a problem with the current (1.3.2) release, as seen here: dependabot/dependabot-core#243 |
My current workflow with yarn upgrade is: yarn upgrade-interactive ...
yarn install # this is, in a lot of cases changes the yarn.lock file after yarn upgrade.
yarn-tools fix-duplicates yarn.lock > yarn.lock.fixed && mv yarn.lock.fixed yarn.lock;
yarn install # this is actually adding more deduping after yarn-tools fix-duplicates To see the changes on my lock file after each change i'm git adding or even committing them. |
Remove when yarnpkg/yarn#3778 is fixed.
Motivation: yarn is actually missing automatic dedup of deps, Which leads to multiple-but-compatible versions of deps. It's recommended to run it after any dependencies add/remove/upgrade Related: yarnpkg/yarn#3778 (And there are more issues around yarn repo related to that)
Motivation: yarn is actually missing automatic dedup of deps, Which leads to multiple-but-compatible versions of deps. [yarn-deduplicate](https://github.com/atlassian/yarn-deduplicate) is a complementary tool to solve that. It's recommended to run it after any dependencies add/remove/upgrade Related: yarnpkg/yarn#3778 (And there are more issues around yarn repo related to that)
Motivation: yarn is actually missing automatic dedup of deps, Which leads to multiple-but-compatible versions of deps. [yarn-deduplicate](https://github.com/atlassian/yarn-deduplicate) is a complementary tool to solve that. It's recommended to run it after any dependencies add/remove/upgrade Related: yarnpkg/yarn#3778 (And there are more issues around yarn repo related to that)
Motivation: yarn is actually missing automatic dedup of deps, Which leads to multiple-but-compatible versions of deps. [yarn-deduplicate](https://github.com/atlassian/yarn-deduplicate) is a complementary tool to solve that. It's recommended to run it after any dependencies add/remove/upgrade Related: yarnpkg/yarn#3778 (And there are more issues around yarn repo related to that)
Do you want to request a feature or report a bug?
feature
Tentative about making it into 1.0 release because I think it is an important improvement over the existing experience.
What is the current behavior?
As described in #3775 whether Yarn will download 2 different versions of left-pad depends on the order dependencies are resolved (it's alphabetic BFS).
Instead we should have a post resolution step that would try to dedupe all versions of a single package to the minimum after the dependency tree was resolved.
I started this https://github.com/yarnpkg/yarn/pull/3563/files#diff-80a032b8112ef9aa0ea41ee304a6024aR453 but never have finished.
At the same time we probably could have the opposite option - force Yarn to use latest dependencies everywhere and not optimize resolution at all.
Another aspect - we don't want to dedupe dependencies if they are already in the lockfile because Yarn guarantees same node_modules for the same lockfile.
Please mention your node.js, yarn and operating system version.
Yarn 0.27
The text was updated successfully, but these errors were encountered: