Monorepository tools and recomendations

[wtrocki]

Having multiple packages in single repo is common problem in enterprise world.
We should be giving recommendation for that.

Solutions:

• yarn/npm workspace
• https://github.com/semantic-release/semantic-release
• https://github.com/atlassian/changesets
• https://github.com/pnpm/pnpm
• https://rushjs.io/

[roastlechon]

We use monorepo at TWC. We use yarn workspace. Would love to contribute to this topic, however I understand this can also be opinionated too.

[wtrocki]

@roastlechon +1 for putting something simple out there. You have my vote for yarn workspace. It is opiniated topics but I do not have opinions and simply looking to get something that works. I'm researching this right now for other things and see struggle - yarn workspaces is ok but do leave tons of things to setup yourself. For the moment I love semantic releases (because it could resolve also versioning problem) vs changesets.

The worst choice seem to be lerna - tons of dependencies, security issues and not maintained

[roastlechon]

I agree with your worst choice with lerna. We had a couple experiments with it but ultimately did yarn workspace with custom local tooling.

[mhdawson]

I'll add this to the agenda for the next meeting as well.

[wtrocki]

Another problem is - changelogs, single version vs multiple versions.
As builder of multi package repos for couple years this have been my main struggle (usually custom tooling were made for purpose)

[helio-frota]

Some workarounds in husky but apparently no official solution yet:
typicode/husky#677

To consider as a link of:
https://github.com/nodeshift/nodejs-reference-architecture/blob/main/docs/development/code-consistency.md#guidance

[RoystonS]

+1 for https://github.com/atlassian/changesets. A bit like semantic-release but with a bit more manual control over which packages are released when; very useful if you to batch changes up for chunkier releases rather than releasing with every commit.

We're using yarn workspaces with our own hand-crafted 'mini lerna' to run tasks across the workspace, but we'll likely shift to https://github.com/hfour/wsrun as that includes the ability to consider a build complete when it emits a certain piece of text, which is very useful for running parallel 'watch' builds.

[jimlindeman]

So with large mono-repositories, dynamically managing which portions of build/test jobs get invoked becomes important to avoid small changes requiring 30-60+ minute builds in the PR-validation pipeline. To manage that, Bazel (https://bazel.build/) has been used, where a BUILD file was defined at each package/microservice in the monorepo, and so the dependency list in package.json would be duplicated there like this example:

deps = [
    "//ts/packages/base-config",
    "//ts/packages/logger",
    "//ts/packages/utils",
    "@npm//@types/amqplib",
    "@npm//@types/convict",
    "@npm//@types/node",
    "@npm//amqplib",
    "@npm//convict",
    "@npm//get-port",
    "@npm//prom-client",
] 

where the //ts/packages/... were peer packages in the monorepo (yarn workspaces) and the others were resolved by npm install.

[roastlechon]

First pass draft #92