Make the @lerna-lite/version and @lerna-lite/publish dependencies optional

[cackerman-qumulo]

Clear and concise description of the problem

I'm working in GItHub Enterprise actions with protected branches. Versioning in the CI pipeline and pushing the version updates back to a protected branch is problematic. So, I'm not using lerna-lite's version or publish commands. The only commands I am using are the run and exec commands. I would like to effectively "disable" the version and publish commands, in case someone thinks they should be used.

Suggested solution

Remove @lerna-lite/version and @lerna-lite/publish from @lerna-lite/cli dependencies, requiring that they be added just like other non-included plugins.

Alternative

Allow disabling specific commands in lerna.json.

Additional context

No response

Validations

• Follow our Code of Conduct
• Read the docs.
• Check that there isn't already an issue that request the same feature to avoid creating a duplicate.

[ghiscoding]

hmm there's a lot to think about but here's a few things to first think of:

1. that would be a breaking change and require a major version
2. I created the packages/core which has a few publish and version related stuff, for example conventional-commit
3. removing publish and version from the CLI would basically equal to an empty shell

Some explanation regarding these points

1. Making a major version is not a big deal since I plan to do one around the time Node 12 reaches EOL which is in April
2. and 3. are pretty much related and the fact is that I'm pretty sure that at least 95% of the people are interested in keeping the publish and version, while run and exec are not as often used. We can calculate a little of that by looking at the year downloads: CLI, so publish/version (183k), run (73k) and exec (26k). We can see that exec is barely used and run is 39% of CLI download, I'm sure that you're not the only person in this situation but the fact that the CLI would become an empty shell is kind of not a good idea in my mind and that would require a lot of refactoring and problem split the core package into a bunch of small separate packages which I'm not keen in doing, Lerna itself has like 50 small packages but they're all required by the CLI so Lerna-Lite is still better than Lerna on that side of it.

In summary, I'm not sure I'm interested into making these 2 packages optional and thinking of making this as a "won't do" topic. I'm still very open to suggestions though, so we can still discuss it.

EDIT

Thinking about it a little more in the shower, great place to brainstorm by the way lol. The current topology is that the CLI is dependent on publish, version and core (there's also init and info but they are tiny), and for your suggestion to work the topology would have to be inversed (publish and version would have to change so that the CLI as a dependency instead). Also like I said, a few pieces (like conventional-commit) would have to be separate from the core into separate packages. Lastly version is already a dependency of publish so the breaking change for the users would be to install Lerna-Lite from @lerna-lite/publish instead of @lerna-lite/cli

BTW is there a reason why you're not using something like pnpm run yarn? Most people seem to be more interested into moving away from lerna run and use pnpm, yarn, TurboRepo or even Nx. Perhaps some day NPM would also be able to deal with this too (as per this RFC)

[cackerman-qumulo]

I prefer using NPM to avoid adding an extra dependency management tool. pnpm and yarn replace NPM and have their own special flavor of dependency management. Lerna(-lite) isn't a dependency manager, it's a task orchestrator. It's a philosophical choice, but one I've developed over a very long time and lots of experimentation with the different options.

As for why not Turborepo or Nx... They focus on parallelization and caching. Both of which I think are of dubious value. They complicate things in return for (theoretically) speeding up your tasks by either running them in parallel, or skipping them. However, caching requires extra infra or a SaaS layer to be effective anywhere except locally. Caching is also 50% of the, "it worked on my machine", type of problem. Parallelism is less problematic, but it complicates log output (interleaved and/or prefixed), and in many environments it's not that helpful (low power CI workers, windows which has FS concurrency issues). Using faster build/test tools (eg. vite, esbuild, etc.) helps a lot more for task speeds (IMO), and doesn't introduce the fragility/complication.

I didn't quite follow your edit. I think the topology would stay the same. publish and version would still depend on CLI. The fact that run and exec are separate packages is what surprises me. Because publish and version are essentially specialized run/exec actions (logically). I would assume that 90% of the code for run/exec must exist in core (workspace enumeration and filtering). However, since this is a fork of Lerna, that may not actually be the functional case.

The breaking change would be the installing @lerna/cli would be a nearly empty shell (init and info only), and users would need to also install @lerna-lite/version and/or @lerna-lite/publish (and/or depending on whether publish has a peer or regular dependency on version).

[ghiscoding]

As for why not Turborepo or Nx... They focus on parallelization and caching. Both of which I think are of dubious value. They complicate things in return for (theoretically) speeding up your tasks by either running them in parallel, or skipping them.

Yeah I kind of agree on some of these points, in Lerna-Lite itself I don't use any of these caching stuff because I want to make sure every packages is fully rebuilt and want to avoid the "works on my machine" as you said.

For the whole thing, I thought about it over the weekend and I decided to go ahead with the change. I found out that there a few things related to version and publish that are in the core package today but they should really only exist (move to) in the version package and since publish depends on version then it's ok to move all these version core stuff (conventional-commits and others) under the version package since that is where they are used anyway, so that would also make the core much smaller and really focus on it being a package that has common stuff for every other package without carrying too many baggage.

The breaking change would be the installing @lerna/cli would be a nearly empty shell (init and info only), and users would need to also install @lerna-lite/version and/or @lerna-lite/publish (and/or depending on whether publish has a peer or regular dependency on version).

Yes but with a slight variance that I would prefer to do, as per my previous post, I think it's better to just change the dependencies of the CLI (remove publish/version from its deps) and instead make the CLI a dependency of every optional packages (run/exec/.../publish/version), this way the user won't need to install the CLI directly, just the command itself (he could if he wants too, but it would be unnecessary) since every other command will require it anyway. Also worth to know that Publish depends on Version, so the breaking change could become a single line change for the ones who want to keep version/publish and it wouldn't be too bad for a breaking change

{ 
  "devDependencies": {
-    "@lerna-lite/cli": "1.14.0", # it could also be kept but it would unnecessary
+    "@lerna-lite/publish": "1.14.0",
     "@lerna-lite/run": "1.14.0",
   }
}

the following code below could also work (and would be equivalent to above code) but it's just more lines which won't be needed after I make the changes on the CLI so that becomes a dependency of all optional commands

{ 
  "devDependencies": {
     "@lerna-lite/cli": "1.14.0",
+    "@lerna-lite/publish": "1.14.0",
+    "@lerna-lite/version": "1.14.0",
     "@lerna-lite/run": "1.14.0",
   }
}

I decided to go ahead with the change mainly because of the new lerna watch command that I added just last week and it could also benefit to be a standalone command.

Roadmap

The only thing that I'm not sure of would be the "when" to make this new major version available, I was planning a major for the end of April since that is when Node 14 becomes EOL and I also have plans for a few other changes as per #262 when Node 14 gets out of the way.

I could maybe do an alpha version a month prior but even then that is 2 months away at the very least, so I don't plan to make it the way you want until that time. The minimum requirement would also bump to Node >=16.17.0

• move all version/publish related stuff from core into version package
  • task completed in PR feat(version): move all version related methods under version package #456
• apply other changes mentioned in (done) Next Major 2.0 Release Roadmap #262
• change Node minimum requirement to >=16.15.0
• publish major alpha version
• publish final major version (~ end of April)

[cackerman-qumulo]

No hurry. It's working as-is. Depending on only what we need is a nice-to-have.

[silverbucket]

I would also like to see this, as I only use publish.

Currently, when installing @lerna-lite/cli, many dependencies exist (my pnpm-lock.yml grows ~2000k from that single dependency). I would love to be able to add just the publishing functionality along with the necessary CLI code to enable me to run it from a package.json script command.

[ghiscoding]

@silverbucket
I actually started the work on the next major version 2.0, you can see the roadmap in #262 (PRs are being merged under the next branch) and it will turn these 2 commands as optional commands but please note that publish depends on version and that will not change. What will change is that the CLI will no longer install version and publish by default, they will become optional like all other commands but like I said publish depends on version and that will probably never change.

I have been working on the next 2.0 version (created the first PR #537 yesterday) and it took me a long time (the past 2 weeks) to get going because I'm now building it as ESM only and I had to switch to Vitest because Jest isn't supporting ESM that well yet. The Vitest switch was (and is still a bit) long to accomplish, there are still couple tests failing but for now I'm just skipping some of them. Anyway if all goes well, I'm expecting to do an Alpha release of 2.0 in a week or two and that will also make version & publish as optional package. I'm expecting a final release before the end of the month (if all goes well with alpha)

Currently, when installing @lerna-lite/cli, many dependencies exist (my pnpm-lock.yml grows ~2000k from that single dependency). I would love to be able to add just the publishing functionality along with the necessary CLI code to enable me to run it from a package.json script command.

I'm switching the project to be ESM only (transparent to the user, except the new Node 16 requirement), that should lower the download size, but I don't think that is going to change anything about the list of dependencies that are used by version and publish. I didn't write that code, it came from Lerna and that will probably never change. I only created this fork to keep it up to date with dependencies and add couple of features like workspace: protocol but, I don't intend to rewrite how the code is working (that is way outside my knowledge and understanding of the code). I made it a lot more modular though, so it's still a lot smaller compare to the original Lerna (theirs is actually getting bigger since they now require Nx and Lerna-Lite doesn't). This is an Open Source project and I only use Lerna-Lite on other Open Source projects and I already spent of lot of my evenings and weekends maintaining it. So, I am not intending to do any core rewrite (except the ESM change) and that is not the goal of this project and I already spent a lot of my free time on this already. I'm also trying to keep up with the original Lerna PRs but it is more and more becoming another Nx branded library with every months that pass and these new maintainer (Nrwl) are actually paid developers (and I'm not), so please keep that in mind too ;)

Contributions are always welcome

[silverbucket]

Hi @ghiscoding I understand, thanks for all the work you are doing to create and maintain lerna-lite. Just get a little clarification, I realize version and publish are tightly coupled and wont be separated (that's totally fine), what I'm unclear on is the goals for run and exec (which I also understand are tightly coupled) and the cli.

eg. can I just have the cli and publish (along with version) capabilities for my usecase? While the OP @cackerman-qumulo would want just cli along with exec and run capabilities? Is that the goal here, or will exec and run always be required for cli functionality?

[ghiscoding]

what I'm unclear on is the goals for run and exec (which I also understand are tightly coupled) and the cli.

No they're not, these 2 are totally separate from each other but they do use common dependencies though which are filter-packages and profiler and the CLI of course. The common filter-packages is because of the --scope=pkg1 functionality.

eg. can I just have the cli and publish (along with version) capabilities for my use case?

It's already like that today, if you install just the CLI today then you'll get version, publish and init out of the box. But that will change in next major release so keep reading...

While the OP @cackerman-qumulo would want just cli along with exec and run capabilities? Is that the goal here, or will exec and run always be required for cli functionality?

Not quite, the goal of this issue is that currently and as I mentioned in previous line, if you install the CLI today you always get publish and version out of the box. The OP in this case doesn't use neither of these 2 commands, he is only using run and exec and the ask here is to make publish and version as optional commands like all other commands because the OP doesn't use version and publish for his use case so he doesn't feel like he needs to download version/publish to take advantage of run/exec.

The summary is that if you install the CLI today you get:

• init
• version
• publish

But when you'll install the CLI in upcoming v2.0 then you'll get only:

• init

This is the ask here to make publish and version as optional commands and this is a breaking change because most people that only use these 2 commands are only adding @lerna-lite/cli in their package.json but after the v2.0 release they'll have to do this instead

- "@lerna-lite/cli": "2.0.0",      <-- you could also keep it, it will be installed behind the scene anyway
+ "@lerna-lite/publish": "2.0.0"
+ "@lerna-lite/version": "2.0.0",  <-- you could skip version since it's tightly coupled to the publish pkg

or even simply install @lerna-lite/publish since version is tightly coupled. I removed @lerna-lite/cli from this example but in theory the CLI will become a dependencies of all commands in the next release so when you'll install publish then the CLI will follow.

In summary All Commands will become optional in the next release (well except for the init which will always be included but it's tiny anyway).

[silverbucket]

Gotcha, thanks for the clarification!

[ghiscoding]

closed by #552

@cackerman-qumulo
The first 2.0.0-alpha2 major alpha release is out, version and publish are now optional and the lib is now ESM only (transparent to the user apart from the new Node 16.15.0 and higher requirement)

EDIT

now officially released under v2.0.0 📦 🚀