.yarn cache and pnpm?

[hanseltime]

Hey all,

For some context:
One of the things I've come to really enjoy (especially in large legacy dependency monorepos or monoliths) is the idea of committing the .yarn/cache with our repos. While it does involve us having to change our git PR diff review strategies, the upsides of having a very resilient deploy pipeline that does not bottleneck on network, is awesome.

Unfortunately, yarn pnp is not working from a development speed aspect. The opacity of the zip files and additional hoops for IDE support or even tooling make education and debugging a large tax to a growing development team. I did run across the pnpm nodeLinker as a good compromise though.

The Question:
With that being said, I wanted to see if there is an issue with committing .yarn/cache with the pnpm linker mode. From my own tests with Dockerfiles, it seems like yarn still uses the cache and rebuilds native dependency libraries, however, the documentation says to only commit .yarn/cache for pnp (at least that's how I read it). I am hoping to move forward with switching to pnpm nodelinker and using the .yarn/cache for avoid the node install problem. Is there a reason I should not do this?

Thanks in advance!

[clemyan]

Yarn stores all fetched packages in a global cache. So in a day-to-day development usecase, each developer should incur the network bottleneck only once.

In a CI usecase, using caching provided by the CI platform is probably more efficient, but there seem to be no consensus on how to best cache Yarn Modern on CI at the moment.

The main benefit of committing .yarn/cache for pnp is that you don't even need to run an install, which is not true for other linkers. It is doable for other linkers, just not as beneficial when compared to pnp.

[hanseltime]

Interesting.

So as soon as I have a package that does extra platform installations (like an opencv package), committing the .yarn/cache just allows me to keep my cached packages in version control but either way, I still need to run yarn install because I need building and linking of the appropriate binaries, etc.?

It doesn't sound like there's danger in storing the .yarn/cache or copying it across platforms (i.e. into a docker container) for any of the linker strategies as long as I do run yarn install. Ultimately, I would just love to be able to minimize package repo pulls (additionally, limit the number of private npm repo creds I have to distribute to CI pipelines, etc.)

[clemyan]

In that case, yes, you will need to run yarn install to run the postinstall scripts anyway.

Your use case sounds reasonable enough to me, though I still think CI cache is a better solution.

I assume your repo lives in a private environment (where only trusted developers can submit PRs). Otherwise, you would be vulnerable to cache and lockfile poisoning attacks. In such an untrusted environment, it is recommended to check the cached zips and lockfile against the registry anyway, negating any saving made.

[hanseltime]

Thanks for the clarification. And yes, this is private only repos. There have been some issues historically with packages getting out of sync, or different versions due to local node_modules caches. I appreciate the link to yarn audit as well. Thanks!