Lerna support

[greysteil]

From @sobolevn on November 10, 2017 5:49

Seems like right now all files inside packages are ignored.

Each folder inside packages could contain its own package.json.
It would be really nice to have it updated.

For the reference: https://github.com/wemake-services/remark-lint-are-links-valid

Copied from original issue: dependabot/feedback#37

[greysteil]

From @bennyn on November 13, 2017 11:41

It would be great if dependabot would support a mono-repository structure! Basically, all you would have to do is to evaluate package.json files recursively.

[greysteil]

We should definitely be supporting mono-repos - in the backend we actually have support for adding multiple directories for the same project/language, but never added it to the frontend because we couldn't figure out the design. I'll have a think.

@sobolevn - I'll look at Lerna, but it might have to wait a while.

[greysteil]

In the meantime, if anyone has a monorepo that they want to try Dependabot on, let me know and I can add the directories in the repo manually in the backend.

[greysteil]

From @bennyn on November 15, 2017 11:6

@greysteil We are using a Lerna monorepository at: https://github.com/wireapp/wire-web-packages - Can you enable monorepo support for us (we already installed Dependabot for this repository)?

[greysteil]

Uh oh, it looks like I've over-reached a little here. I didn't fully understand how Lerna works with Yarn, but can see now that there's more work here to be done on the Dependabot side.

@bennyn - since your repo uses Yarn workspaces I can't get it up-and-running manually (since workspaces mean you've got a single yarn.lock shared across many package.json files). I'll have a look at adding support over the next week or so, but can't promise how quick it will be.

[greysteil]

From @bennyn on November 15, 2017 13:56

@greysteil Can you do the following?

1. Update all dependencies in the package.json files of every package inside the packages folder (and forget about their yarn.lock files)
2. Go to the root folder (containing the "packages" folder) and update dependencies in the package.json file
3. Execute yarn install and lerna bootstrap in the root directory
4. Create a PR with the updated package.json files and the yarn.lock file of the root

[greysteil]

Sadly not - Dependabot hacks around with the internals or Yarn (and Bundler, Composer and Pip) in order to avoid performing installs (and a bunch of other reasons). As a result, in situations like this there's normally quite a bit of work to get everything up and running. We also don't clone your repos, instead preferring to fetch only the files we need.

I've made a start on this in #129. Lots more to do, but it will happen there.

[greysteil]

From @simondel on November 17, 2017 10:45

This would be awesome for us as well! We don't use yarn or the package.lock.json or anything. Our use case would be:

• Update **/package.json
• Create a PR to run the CI

I tried using Greenkeeper earlier today but unfortunately they don't seem to support this setup yet.

Our repo for reference: https://github.com/stryker-mutator/stryker

[greysteil]

OK, quick update on this:

• We're nearly there on Yarn workspaces support. There's a bug in it right now, but I'm hoping to have that cleaned up today. If you're using Yarn and Lerna, that's probably what you want. (More detail.) Yarn workspaces support is now live! 🎉
• Once that's done, I'll look into Lerna support more broadly. This may take a little while.
• If you're not using lockfiles (@simondel) we're a little further off, and the appropriate issue to track is this one. We now handle JS repos that don't use lockfiles! 🎉

Hope that helps make everything clear. @simondel - I'd definitely chat to the Greenkeeper guys about Lerna support if I was you, they ought to be all over that :)

[greysteil]

Another update: Dependabot can now support mono-repo structures. If you've got a mono-repo with many directories, each of which have their own package.json and package-lock.json (or yarn.lock) you can now add the same language multiple times in the Dependabot dashboard, specifying a different directory each time. Dependabot will treat the updates completely independently, so you don't have to merge an update to all your dependencies at once.

We're actually using this flow on dependabot-core to keep the files in helpers/yarn and helpers/npm up-to-date.

In future, we'll also add Lerna support, which would allow you to add Dependabot just once and get PRs that update all your packages at once.

[greysteil]

From @exactlyaron on February 13, 2018 16:3

Hi all.

We have a monorepo and looking for something to keep everything up to date that supports Lerna, so we're very happy to see you're working on support for it! 😄

I had a little test of Dependabot and in the dashboard when I've added a couple of directories located in the 'packages' directory of our repo it is showing as 'Error bumping dependencies' yet PRs have actually happened. The 'plugins' directory is fine though.

Is this known @greysteil ?

Thanks

[greysteil]

From @bennyn on February 13, 2018 16:9

@AronM28: At Wire we are using Dependabot with a mono-repository managed by Lerna:

• https://github.com/wireapp/wire-web-packages

As you can see in our merged PRs, it works really well 🌞:

• https://github.com/wireapp/wire-web-packages/pulls?q=is%3Apr+is%3Aclosed+author%3Aapp%2Fdependabot

[greysteil]

If there’s an error bumping any of your dependencies you’ll see that status (and I’ll get an error report on my side). I’ve been away for the last week (I’ll be back from holiday on Friday), so am a little behind on digging into those errors - sorry!

On 13 Feb 2018, at 06:09, Benny Neugebauer notifications@github.com wrote:

@AronM28: At Wire we are using Dependabot with a mono-repository managed by Lerna:

https://github.com/wireapp/wire-web-packages
As you can see in our merged PRs, it works really well 🌞:

https://github.com/wireapp/wire-web-packages/pulls?q=is%3Apr+is%3Aclosed+author%3Aapp%2Fdependabot
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub, or mute the thread.

[greysteil]

And thanks @bennyn!

[greysteil]

From @exactlyaron on February 15, 2018 8:22

After overloading Travis initially, I redone everything in a more patient manner and it looks good! 😄

Great work @greysteil !

[greysteil]

@AronM28 - FYI, I got back from holiday today and hunted down that bug. It was caused by hl-pg-client being listed in the dependencies for address-matcher but not published on the registry. Presumably this works locally because Lerna takes care of it.

The fix (in eb414d0) I've put in place is to ignore dependencies that aren't listed on the registry for library package.json files.

[greysteil]

From @exactlyaron on February 15, 2018 20:20

@greysteil Brilliant thanks a lot for the help! 👍 I had a feeling it was probably down to an unpublished package. There's a few more which probably came about for that same reason.

It is something that has been noted on some other things, another pro for publishing all the things.

[greysteil]

From @exactlyaron on April 16, 2018 8:50

Probably a long term nice to have but the ability to update package.json settings en masse would be nice.

For example, so with our monorepo I think there is 28 packages, so you'd need to manually change every setting for each individual one. It would be nice to be able select all in the repo to set bump frequency or auto merge setting. 👍

[greysteil]

Closing in favour of the PR at #197.