Improving performance when creating many mocks

[manzanit0]

Hi!

We've been using mockery for a while now, and the team absolutely loves it. Thank you for the open contribution! We decided to put the generation of mocks in a pre-commit hook to ensure tests would always pass. However, as adoption has grown though, we've noticed that generation grows slower due to the sequential nature of the generation.

Do you know if there are any easy ways to improve this? I'm aware that the v3 take attempts to tackle this, but I was wondering if you know of any potential soothers until that comes out.

Cheers!

[LandonTClipp]

Unfortunately there are not many. The main problem comes from packages.Load which is doing the actual AST parsing. There have been various hacks suggested that I didn't like for various reasons. We've done what the maintainers have suggested with caching the Loads to avoid re-parsing the same thing multiple times: https://github.com/vektra/mockery/blob/v2.15.0/pkg/parse.go#L53

yet it doesn't help much. This is also not a problem unique to mockery:

golang/go#31087

cogentcore/cogent#150

Here's an explanation of the problem: golang/go#31087

One of the other potential issues is that packages.Load is loading the entire dep tree of every single file in your package, which would mean parsing the same parts of the tree multiple times. This could be fixed in v2 by discovering the packages/files to load first, then calling packages.Load once, which should fix the problem. Since I'm a full-time developer, I don't have much time to dedicate to writing of actual code, but I feel strongly that this method could fix our woes.

Notice this call here:

mockery/pkg/parse.go

Line 56 in 847d988

pkgs, err := packages.Load(&p.conf, "file="+fpath)

This method is called once for every file in your repo!

https://github.com/vektra/mockery/blob/v2.15.0/pkg/parse.go#L100

No wonder it's slow!

The way I envision this for v3 is that you will specify which packages you want loaded in config, instead of having the code auto-discover (although maybe that could still be a feature). That way, we'd make one call to packages.Load which according to the devs would result in O(n) performance.

Until either of those code changes are done, there's not much we can do. But, I think in theory the fix for v2 is pretty simple and could be done without a lot of headache.

[manzanit0]

Gotcha, that makes a lot of sense, thanks for the thorough explanation! :)

[LandonTClipp]

Note that this issue should be fixed if you are using the packages configuration feature.