test: Test 250+ popular npm packages

[timfish]

Adds a script that tests 250+ of the most popular npm packages. It compares the exports without and with the loader hook and fails if they don't match. It also fails if errors are thrown with the loader hook in use.

[AbhiPrasad]

Nice!

We're not adding this to CI yet? We can just comment out the packages that don't work for now.

Aside from these popular npm packages, could we also test against some of the popular web frameworks? express, fastify, hapi, connect, sveltekit, nextjs, gatsby, remix. These are the most common places where users will use APM instrumentation, so I think it's important we catch any issues asap.

[timfish]

We're not adding this to CI yet? We can just comment out the packages that don't work for now.

This can be added to CI, but likely not for every Node version that gets tested because some of these libs won't support such a wide range. These tests with then potentially break often as the versions aren't pinned and they will update their supported node versions.

Maybe it's better the check in the package.json to make them more reliable?

could we also test against some of the popular web frameworks? express, fastify, hapi, connect, sveltekit, nextjs, gatsby, remix. These are the most common places where users will use APM instrumentation, so I think it's important we catch any issues asap.

Yeah it's certainly worth adding more to this list. Initially this was to find potential gaps in the existing unit tests.

If we pin the versions in a package.json, we will not get new failures when libraries break.

[timfish]

I've now added this to CI testing and you can see it running in my fork.

changes have been made