[Feature Request] Support separating bundling and executing of workers

[schickling]

Currently the worker entrypoint file(s) (e.g. src/worker.ts) are doing both bundling and execution.

For my application, I'd like to take the responsibility of the bundling (and e.g. use something like esbuild instead of Webpack) and have my worker entrypoint (e.g. invoked by nodemon) only trigger the actual code execution and not any kind of building/bundling when starting.

[bergundy]

@schickling it'd take some work to support bundling with anything other than webpack from my experience.
The bundling process isn't that straight forward, user code needs to be lazily loaded so the runtime can be properly prepared for workflow execution.
It should be straight forward to save the bundle to disk and load it on worker startup for cases like production deployment.

[bergundy]

@lorensr @schickling @sw-yx I'd like some feedback on the public API for this.

In order to support this we'd need to export the WorkflowCodeBundler class added in #317 for users to be able to create the bundle.
In addition we need to understand how to pass this into the WorkerOptions, a few options come to mind.

1. Remove the workflowsPath and nodeModulesPaths options in favor of a new workflowCodeBundle option

• Pro: much more explicit about what's going on
• Con: More verbose samples

2. Add the workflowCodeBundle option to the existing options.

workflowCodeBundle could either be a path (string), the bundle itself (string / buffer / stream), I'm not sure what would be most convenient.
I'm leaning towards the string because it's the most flexible option.

Example:

const workflowCodeBundle = await WorkflowCodeBundler({ workflowsPath });
const worker = await Worker.create({ workflowCodeBundle, ... });

[lorensr]

I'm leaning towards the string because it's the most flexible option.

By string do you mean path?

Awaiting a SentenceCase function feels weird to me. SentenceCase makes me want to do new WorkflowCodeBundler() or WorkflowCodeBundler.bundle(). What about:

// built-in bundling
const workflowCodeBundle = await bundleWorkflowCode({ workflowsPath });
const worker = await Worker.create({ workflowCodeBundle, ... });

// dev doing their own bundling:
const worker = await Worker.create({ workflowCodeBundle: './bundle.js', ... });

That thing is happening where the word "bundle" is starting to sound really strange to me 😆

[bergundy]

So the return value of bundleWorkflowCode is a string which means that we can't use pass a path using the same parameter name.
I usually like to call path params what they are for clarity, e.g. workflowCodeBundlePath.
Another option is to have workflowCodeBundle param type: { path: string } | { code: string } and have bundleWorkflowCode return { code: string }.

[lorensr]

We could have both bundleWorkflowCode and workflowCodeBundlePath:

// built-in bundling
const workflowCodeBundle = await bundleWorkflowCode({ workflowsPath });
const worker = await Worker.create({ workflowCodeBundle, ... });

// dev doing their own bundling:
const worker = await Worker.create({ workflowCodeBundle: 'js code', ... });
const worker = await Worker.create({ workflowCodeBundlePath: './bundle.js', ... });

[bergundy]

IDK, I kind of like the { path: string } | { code: string } solution.
Removes some of the clutter from the already overwhelming WorkerOptions interface.