RFC: Sequential support for writeBundle closeBundle plugin hooks

[antfu]

Implementing PRs:

• Allow using objects as hooks to change execution order rollup/rollup#4600 (comment)

Summary

Introduce capability for plugins to run writeBundle
and closeBundle hooks sequentially. RFC targeting to Vite plugins, with the hope to be backported by Rollup.

Basic example

writeBundle and closeBundle can be provided as functions, resulting in parallel execution to keep backward compatibility. With this RFC, it can optionally be an object to provide additional metadata, in this case, allowing execution to control the behavior of the hooks.

export default myPlugin() {
  return {
    name: 'my-plugin',
    writeBundle: {
      sequential: true,
      async handler(options, bundle) {
        // ...
      }
    },
    closeBundle: {
      sequential: true,
      async handler() {
        // ...
      }
    }
  }
}

Motivation

With the growth of Vite and its ecosystem, we have developed many more plugin usages which might go beyond Rollup/Vite's initial design scope (targeting Web, SSR, SSG, etc.). Inherit from Rollup, both writeBundle
and closeBundle are executed in parallel. This creates a racing condition where some plugins are not able to read the bundle processed by other plugins.

Related Issues:

• Add sequential before build hook rollup/rollup#2826 (Rollup: Add sequential before build hook)
• Replace $ vite build && vite build --ssr with $ vite build + enable plugins to add build step #5936
• feat: preCloseBundle hook #9326
• please suggest more

Detailed design

While this pattern may be able to develop into a common convention applying to other hooks in the future. This RFC will focus on supporting writeBundle and closeBundle hooks.

TypeScript Definitation

writeBundle as an example:

type ExecutionMode = 'sequential' | 'parallel'
type WriteBundleHookFn = (options: Options, bundle: Bundle) => void | Promise<void>

interface WriteBundleHookOptions {
  execution?: ExecutionMode
  handle: WriteBundleHookFn
}

interface Plugin {
  // ...
  writeBundle?: WriteBundleHookOptions | WriteBundleHookFn
}

Ordering

For simplicity, we might use the plugin level enforce option to control the order of execution. The question here is if Rollup team is willing to backport the enforce concept to Rollup.

Execution

Leave the order to be defined normally by plugin position in the array, and each time a sequential execution hook is hit, then it is awaited. So they act as limits on what gets done in parallel

p1 p2 p3 sA p4 p5 sB

Will do p1 p2 p3 in parallel, await that, then do sA, then p4 p5 in parallel, then sB

Ordering should be respected as the sync part of each hook could expect that.

let parallelHooks = []
for (const hook of hooks) {
  const h = normalizeHook(hook)
  if (h.excecution === 'sequential')) {
    await Promise.all(parallelHooks)
    parallelHooks = []
    await h.handle(...)
  }
  else {
    parallelHooks.push(h.handle(...))
  }
}
await Promise.all(parallelHooks)

Credit @patak-dev

Drawbacks

Until Rollup backport it, this RFC will make:

• The hooks to be Vite specific
• Needs to do some hack outside Rollup to make it work

Additional Context

In Vite, we do support a similar convention in transformIndexHtml which accepts an object as the hook to specify the enfore. If this RFC gets accepted, we might take the chance to unify the convention for transformIndexHtml in another RFC.

Alternatives

Introduce new seperate hooks like writeBundleSequential and closeBundleSequential.

Adoption strategy

This RFC should be additive and does not require migrations from plugin authors.

Unresolved questions

• General feedback from the Rollup team
• If the rollup team willing to introduce enforce concept

[patak-dev]

Thanks for putting together this RFC @antfu! IMO this is a good idea in general, and it sounds like a better direction than having two separate hooks with the only difference of one being parallel and the other sequential. Looping @lukastaegert, I think we should go in this direction if there is a chance to push this upstream to Rollup at one point in the future to avoid forking the two plugin ecosystems.
I also saw that Lukas commented on a rollup PR that they were thinking of adopting the enforce concept.

The API looks good to me. For the execution loop, I think we shouldn't run all parallel hooks firsts, and then all sequential. We should leave the order to be defined normally by plugin position in the array, and each time a sequential execution hook is hit, then it is awaited. So they act as limits on what gets done in parallel

p1 p2 p3 sA p4 p5 sB

Will do p1 p2 p3 in parallel, await that, then do sA, then p4 p5 in parallel, then sB

Ordering should be respected as the sync part of each hook could expect that.

let parallelHooks = []
for (const hook of hooks) {
  const h = normalizeHook(hook)
  if (h.excecution === 'sequential')) {
    await Promise.all(parallelHooks)
    parallelHooks = []
    await h.handle(...)
  }
  else {
    parallelHooks.push(h.handle(...))
  }
}
await Promise.all(parallelHooks)

Normally you should only place sequential executed hooks where need it. For SvelteKit case, these hooks should be at the end, so normally the pipeline will look like p1 p2 p3 p4 sA sB. So you don't really have a regression in performance, everything that can run in parallel, will continue to run in parallel if plugins are placed in the right order. And using enforce will allow us to create plugins like SvelteKit that are an array of plugins, some in the middle (no enforce), and some with enforce: 'post' to push them to be at the end of the array. So we can get the order we need, even with monolithic plugins like SvelteKit.

[antfu]

Oh I see, love the way you manage to execute those hooks!

[aleclarson]

Will do p1 p2 p3 in parallel, await that, then do sA, then p4 p5 in parallel, then sB

Your example shows something different. The sA hook is executed before the parallel hooks are resolved.

[patak-dev]

Thanks @aleclarson, corrected 👍🏼

[antfu]

Updated to the RFC 👍

[userquin]

If the proposal is about implementing it in Vite (any of the approachs), we need to delete or wrap the writeBundle and closeBundle hooks in all plugins before passing them to Rollup configuration. Otherwise, both hooks will be called twice, one from Rollup and one from Vite. We cannot remove them from Rollup config, since each plugin can also have other hooks.

[antfu]

Yeah, we could discuss about how we implement it later. In RFCs, we focus on the design first.

[bluwy]

If the rollup team willing to introduce enforce concept

I'm not the rollup team, but I'm not sure how this can be ported to Rollup. enforce is a Vite-only concept to insert plugins in between various core Vite plugins to build the final array of plugins. In Rollup, you have full control of plugins and its order so I'm not sure how enforce could be used there.

[patak-dev]

Each time a pre plugin is found it is moved to the begining, and a post plugin is moved to the end. This would allow to create plugins like SvelteKit that are an array of plugins, some in the middle (no enforce), and some with enforce: 'post' to push them to be at the end of the array. So we can get the order we need, even with monolithic plugins like SvelteKit. So, a single plugin can spread its hooks to diff places of the pipeline.

[bluwy]

Yeah I think it may make sense if Rollup supports nested plugins array. I'd still be a bit worried though of re-ordering plugins internally, when the user has already specified an explicit order in rollup.config.js, which may catch them off guard. And also the possibility of plugins fighting for two kinds of ordering: plugins: [] order and enforce order. 🤔

[patak-dev]

I think Vite has proved that arrays+enforce is useful. If not, for some plugins like legacy it would need to generate three plugins and ask the user to inject one in each position. The other option is to add enforce at the hook level { execution, enforce, handle }.

[lukastaegert]

I think we can go there, I am just wondering what a good use case is to actually include this. My argument here is that I want to prevent feature creep for things where I am not really convinced they actually add value, but I may stand corrected.
So the situation is this: You have two plugins that need to do something on the written files, but they need to do it in a particular order.

Question 1: Why can't they do it on the bundle object? generateBundle is a perfectly sequential hook that already allows you to do some things on the bundle, and with Rollup 3, you will have nearly the complete bundle information available in renderChunk so you can do even more things in that hook. Also, generateBundle already knows when it is a "write" operation so you can do special things just when writing stuff.

Question 2: Assume we have established that you can do the things you want to do only in writeBundle (e.g. a chmod of a chunk). If there is such a strong coupling between your plugins that one needs to be executed before the other and your plugins are maybe not even usable on their own, maybe they should actually be a single plugin?

Now what I am looking for is actual good real world examples that answer the questions. Otherwise, I would rather direct users to use different patterns that go along with the existing APIs and in the process, keep the number of available patterns smaller with better established best-practices.

[benmccann]

Why can't they do it on the bundle object? generateBundle

The assets need to be on disk. Shared a more detailed explanation here: sveltejs/kit#5709 (comment)

If there is such a strong coupling between your plugins that one needs to be executed before the other and your plugins are maybe not even usable on their own, maybe they should actually be a single plugin?

There really isn't any coupling between the plugins. vite-plugin-pwa is used by a wide variety of projects across the ecosystem (their docs have examples for Vue, React, Svelte, SvelteKit, SolidJS, Preact, VitePress, iles, and Astro). The only requirement that it has is that it have a list of all the assets. For that, it needs to wait for our other plugins to finish running.

[antfu]

As @benmccann mentioned, my case is vite-plugin-pwa, which uses workbox to generate a sw.js all the assets on the dist. While technically we could trick it into reading from the memory bundle in generateBundle, the problem it that if any of the other plugins write files in writeBundle, closeBundle, workbox won't be able to scan for them, causing some runtime file missing in the PWA.

While for classic Vite client SPA build, vite-plugin-pwa works great by using writeBundle. The problem is that when involved with SSR/SSG it fails to grab the files generated in parallel.

Our current workaround is to expose the api field in vite-plugin-pwa (as Rollup is advised for plugin communication), so other plugins can call it after they finish the writing, like:

{
  name: 'my-prerender-plugin',
  async writeBundle() {
    // await rerendering...
    const pwaPlugin = config.plugins.find(i => i.name === 'vite-plugin-pwa')
    await pwaPlugin.api.generateSW()
  }
}

This makes the plugin couple with vite-plugin-pwa and potentially wasteful to generate the service worker multiple times.

[bluwy]

I think the generic idea of sequential support is sound, but the motivation doesn't sit right with me. The main push for this feature is to run multiple Vite builds with a single build command, which is confusing because build should just build one bundle (same goes to Rollup).

Scope

we have developed many more plugin usages which might go beyond Rollup/Vite's initial design scope

In which case, I think we should be building meta-frameworks instead since it's clear where Rollup/Vite has drawn the line. They shouldn't be or allow plugins to "upgrade" them as a meta-framework.

Interopability

Extending Rollup/Vite API so that integrations are still plugins doesn't always guarantee interopability. #9326 and vite-pwa/vite-plugin-pwa#327 are an example that vite-plugin-pwa still needs special support for sveltekit in order for them to work. It's the same as vite-plugin-vercel is never going to just work with sveltekit. What this RFC proposes is a way to make this "interopability" less hacky, but this should be exposed as an intercom api from sveltekit in the first place.

Maybe there is a future that all this could just work, but we need an extended contract that plugins can follow, and I don't think it's something Vite should declare, but a meta-framework does first.

Summary

To kinda summarize, I don't think the usecase justifies this feature yet. We're sort-of giving way to a hack that we don't support.

^{Though I'm also from the Svelte team, I'm trying to think from Vite's perspective even if it sounds like we're conflicting ideas.}

[bluwy]

There's no reason this has to be the case. In reality, people are already using vite build to produce multiple bundles.

The build command can be reframed as producing the output for one application instead of one bundle.

The issue is that from the build() API perspective, currently plugins can hack to call another round of build in the writeBundle/closeBundle hook. Now build is being overloaded as building both a bundle and an application, blowing out its scope, and I personally find it harder to comprehend the flow if this is the case.

It would work like:

toggle

| build start 1
|
| | build start 2
| |
| | | build start 3
| | |
| | | build end 3
| |
| | build end 2
|
| build end 1

instead of

toggle

| build start 1
| build end 1
|
| build start 2
| build end 2
|
| build start 3
| build end 3

Using the intercom API sounds much more hacky to me than using a hook. E.g. every framework will need to add support for the same intercom API. The whole point of Vite is to standardize and abstract stuff like this so that each framework doesn't have to implement the same exact set of APIs outside the Vite/Rollup ecosystem.

We would still need to import vite-plugin-pwa as import { ViteSvelteKitPWA } from 'vite-plugin-pwa' no? Even after this PR is implemented. vite-plugin-pwa still needs to know about SvelteKit in some way: generate the files at the right place, know when to generate these files (e.g. before adapting), etc. Every Vite/Rollup ecosystem would likely need special treatment.

Using the intercom api is Rollup's recommended way for plugins to "talk" to each other, so I don't think it's hacky.

[userquin]

Using a custom cli should be like the second approach: client + SSR + SSG as separated builds, for example Astro, îles or vite-ssg use this approach. In this case the pwa plugin will work, you only need to configure it on the corresponding fw hook (vite-ssg will automatically lookup the pwa plugin and call the api exposed by it).

Since SvelteKit is not using a custom cli, it would work with the former approach, I like the idea to use the intercom api, but there is a lack here, maybe this can be also another RFC: there is no standard way to do it, and so plugin authors will need to expose a lot of hooks to work with A, B, C and so on.

You can check this PR on Kit repo, I sent it a few days ago as an alternative to this RFC and preCloseBundle hook here: sveltejs/kit#5694

[benmccann]

Having SvelteKit have it's own CLI doesn't solve any of these problems unless you also want SvelteKit to no longer use vite.config.js and have its own configuration format in which case we'd lose all compatibility with the Vite ecosystem, which would be terrible for users.

Even after this PR is implemented. vite-plugin-pwa still needs to know about SvelteKit in some way: generate the files at the right place, know when to generate these files (e.g. before adapting), etc. Every Vite/Rollup ecosystem would likely need special treatment.

No, vite-plugin-pwa will not need to know about SvelteKit at all. It generates files to the configured Vite output directory, which doesn't depend on knowing anything about SvelteKit. It's true that the order that plugins are installed matters, but that's true of all sequential hooks! It's already well established in the Rollup ecosystem that we can have sequential plugins where the relative ordering matters. For the PWA plugin to say it needs to come after any other plugins that generate files doesn't seem to break this mould to me in anyway nor is it at all SvelteKit-specific

Using the intercom api is Rollup's recommended way for plugins to "talk" to each other, so I don't think it's hacky.

But these plugins shouldn't have to talk to each other. There's absolutely no reason to couple the SvelteKit and PWA plugins. The fact that they would be talking to each other at all is not a clean design in my opinion. And then this coupling is multiplied 10x when the PWA plugin also has to talk to every other plugin in the ecosystem

[bluwy]

Having SvelteKit have it's own CLI doesn't solve any of these problems unless you also want SvelteKit to no longer use vite.config.js and have its own configuration format in which case we'd lose all compatibility with the Vite ecosystem, which would be terrible for users.

I don't think this has to be the case if we don't restrict ourselves to "vite.config.js means we allow running vite build". We can still have our own CLI and use vite.config.js to be compatible with Vitest, Cypress, etc. solid-start has been using this model.

No, vite-plugin-pwa will not need to know about SvelteKit at all. It generates files to the configured Vite output directory, which doesn't depend on knowing anything about SvelteKit.

Maybe @userquin can fill us in, but from what I've read in vite-pwa/vite-plugin-pwa#327, it still needs to know about SvelteKit somehow even after this proposal is implemented.

The fact that they would be talking to each other at all is not a clean design in my opinion. And then this coupling is multiplied 10x when the PWA plugin also has to talk to every other plugin in the ecosystem

That's pretty much the case for vite-plugin-pwa right now if I understand correctly so far. But it's hard to compare now given there's only two in the ecosystem - SvelteKit and vite-plugin-ssr, that are frameworks disguised as Vite plugins. Also I'm not sure if vite-plugin-pwa and vite-plugin-ssr had issues in the past, maybe now it's only with SvelteKit? I might need @userquin to fill us in here too.

[benmccann]

We can still have our own CLI and use vite.config.js to be compatible with Vitest, Cypress, etc. solid-start has been using this model.

Yes, we could do that (besides the fact that it'd give our users whiplash, make the documentation harder, not make anything new possible, etc.). But if we do that, I don't see how it solves the issue of not being able to run plugins sequentially at the end of the build. If we ran multiple builds from within the SvelteKit CLI it doesn't change the fact that we still want users to be able to run plugins sequentially at the end of the build after all files have been generated.

from what I've read in vite-pwa/vite-plugin-pwa#327, it still needs to know about SvelteKit somehow even after this proposal is implemented

It will in the short-term, but I'd like to clean that up. There's no inherent reason why it's necessary other than this missing API

That's pretty much the case for vite-plugin-pwa right now if I understand correctly so far

Sure, we have to workaround the missing API. But that doesn't mean we should be satisfied with the status quo or that it's inherently necessary

[lukastaegert]

Ah, so the general use case is "I want to run some command line tools that I cannot apply to the in-memory bundle, and they need to have an order". Maybe I can get behind that.

The other option is to add enforce at the hook level { execution, enforce, handle }

That is the thing that bugs me most about enforce: I think it is applied at the wrong level. Plugins are sorted where it is actually hooks that need to be sorted. And while it is not much of a difference for simple one-trick-pony plugins, it is a huge difference for complex plugins like commonjs.

What I like about the current proposal is the introduction of a "hook object form that changes run characteristics of a hook", and that is something I could actually see Rollup introduce. That might be branching off into a different RFC but I think it is still fundamental to this proposal as well: If we introduce a general hook object form, what is the shape?

There is some prior art with your transformIndexHtml hook, but unfortunately it calls the hook transform rather than handle (maybe there is some potential to align eventual by deprecating but not removing the old format?). Also I wonder if at this point, one could reconsider the name enforce as it really does not state what is enforced—people just need to know. enforceOrder´ or runOrder` would be a better choice IMO.

So maybe we can move this into a more general direction:

• We want to support replacing function hooks with objects
• In that case, the actual hook is ... handle? Something else?
• If it is a sequential hook, we allow an option to enforce the order
• If it is a parallel, we allow an option to run it sequentially, in which case it will run sequentially after all parallel ones and also supports the option to enforce order

I would be willing to support this if we can align on the details.

and ask the user to inject one in each position

It is a little understood fact by plugin authors that a plugin can easily use the options hook to inject additional plugins at given positions. The commonjs plugin does this for example. But above solution would of course provide a different approach for many situations.

an array of plugins

The reason Rollup does not support this is because I am not sure how to do error handling right. Currently, Rollup tells you either the plugin name or the plugin position so that a user can infer which plugin caused issues. For nested plugins, my thought would have gone rather to support a plugins option in plugins so that a plugin can have nested plugins. In that case, Rollup could generate errors that say plugin x that is contained in plugin y. With arrays, this is slightly unclear, though one could add a convention that the first plugin in an array gives the array a "name". But this is a very different RFC and very tangential to the current issue.

[patak-dev]

For the bottom line, both should be post and sequential, and the user should have pwa before plugin vercel. Again, this has worked well in rollup so far. It is not a good use case to justify magic numbers IMO.

[brillout]

While it's a subpar DX and I still think it's a mistake (because magic numbers are trivial to implement and only rarely used given a proper warning in the docs), I guess it's ok. At least for now since, so far, it would only be for vite-plugin-pwa and vite-plugin-vercel. Time will tell whether more use cases arise. I guess we can consider this discussion resolved. Thanks for the answers!

[userquin]

don't forget sitemap plugins ;)

I think the order of the plugins is enough, trying to orchestrate the correct order doesn't make much sense from a functional point of view, right now you can do a destructure of any plugin and put it in the order you want, change the apply...

ofc, plugins should configure their hooks properly.

[brillout]

don't forget sitemap plugins ;)

You're giving me reasons to fight for magic numbers :-)

[brillout]

Follow up discussion at vitejs/vite - vitejs/vite#9613 - Hook ordering. (Which is a follow up of rollup/rollup - #4600 - comment by @ElMassimo.)

[sapphi-red]

If this RFC were implemented, doesn't vite-plugin-pwa still need to communicate with svelte-kit?

AFAIK svelte-kit runs vite build inside in a hook. So if svelte-kit uses the same plugin array which is defined in the vite.config.js, vite-plugin-pwa will run twice.

If svelte-kit removes plugins that shouldn't run twice, svelte-kit needs to somehow know which plugin needs to be removed.

I think it is possible to run once by having a ran variable inside the pwa plugin. But is it possible to run in the last time it is called?

[userquin]

Using Kit with more generic plugins can have the problem you describe, but plugin authors should deal with it (maybe Kit should document it)...

[sapphi-red]

Thank you. So for svelte-kit and vite-plugin-pwa, it works by filtering with build type.

How about a framework which runs client build twice and a plugin that needs to run at the end of those multiple build? (I'm not sure if this is a valid case)

[userquin]

If there is 2 client builds, pwa plugin and any other configured plugin will run twice, what's the problem with it?

I talk with Kit ppl, suggesting to include a way to use custom Kit hooks, for example, I sent 2 PR to Astro and îles, both app fws have the concept of ssg hooks and so the plugin is configured internally to run in the corresponding hook:

• Astro integration (I don't know why this PR was closed 😅): https://github.com/withastro/astro/pull/3428/files#diff-bc3255664291e971b7930316b17e9a1e3337df7cbb180b80fb068fe6793679e1R155
• îles plugin/module: https://github.com/ElMassimo/iles/blob/main/packages/pwa/src/pwa.ts#L150

[userquin]

PWA plugin will work with Vite, Astro, îles, vite-ssg and SvelteKit, it has a lot of work in its internals to deal with a few app fws with SSR/SSG.

[benmccann]

If this RFC were implemented, doesn't vite-plugin-pwa still need to communicate with svelte-kit?

No

If svelte-kit removes plugins that shouldn't run twice, svelte-kit needs to somehow know which plugin needs to be removed.

SvelteKit largely doesn't do this. It only does it with its own plugin, which was kind of a hacky initial implementation, which I think should be cleaned up and just use the ssr flag instead to run different code on the client and server builds.

[antfu]

In general, I am against the goal of #5936. I think running multiple builds in vite build is anti-pattern. While this RFC might extend the capability of doing something against the pattern, I don't think it's a valid reason to block other possibilities. I'd consider it to be a bit out of the scope of this RFC.

What I am for is to improve the limitation of the current plugin API, where a plugin can't know what others are doing. Which ends up forcing plugins to use api, and create coupling that doesn't scale.

[bluwy]

I agree too, but I think what we need for this RFC is a valid real world usecase. So far the only usecase we have is #5936 (or similar), which we oppose. So now we're only talking about theoretical future usecases, which I think is dangerous when designing an API.

[antfu]

https://github.com/vitejs/vite/discussions/9442#discussioncomment-3289166

I am not sure if you consider vite-plugin-pwa a valid case (not only in SvelteKit, we also need to do the hack in VitePress, vite-ssg, etc.).

Maybe let's say a more general case, if I just want to check the integrity of my bundle, I might do it in closeBundle. While if any of the other plugins write files in that hook async, I will fail to grab that changes. I assume the initial design of writeBundle and closeBundle being parallel is considering performance, while with @patak-dev's idea on the execution order, I think we can have the good for both worlds.

[bluwy]

Thanks for the hints. It seems like then the hacks are:

• here for vitepress post-build because VitePress generates extra files after Vite build
• here for vite-ssg post-build because vite-ssg generates extra files after Vite build

So with this RFC implemented, VitePress and vite-ssg would still have to refactor their build process to move all the extra file generation to writeBundle/closeBundle as a sequential-executed hook? Because looking at the code, it's the only way to have vite-plugin-pwa run last to scan all the generated files. If that's so, it looks like the refactor would lead to the same thing as #5936.

Anyways I think I'm going out of scope too, just laying out some future concerns. Re the general usecase example, I think that's fair, but it can also be done as an external script too if it's analyzing the final build output 😬

[antfu]

vite-ssg is hard-coded for the support of vite-plugin-pwa, which makes it hard for others to create another vite-plugin-pwa alternative (or similar usage). While in VitePress you have to use a separate script to do that, making a pitfall for users to install a plugin and silently failing to work if they did not add the extra script.

The difference between VitePress or vite-ssg from #5936 is that they are not extending vite build but use their own CLI and call Vite programmatically. I think in programmatic usage, it's reasonable to do rerendering or other stuff, as the script takes control of the whole process and writeBundle/closeBundle are more like the hooks for vite/rollup's lifecycle. I imagine after the RFC, vite-ssg can first bundle a SSR build, and then with a client build but having prerendering inside writeBundle (I consider rerendering is still part of the build), so later plugins like vite-plugin-pwa can generate correct manifest without vite-ssg aware of it.

Re integrity example: Maybe you want the integrity sha to be present somewhere in the front end, you might better to know the context/options of Vite. So yes, theoretically it can still be done with an external script, actually so as many other stuff. Like you can always use a separate workbox cli to generate the PWA, but that losing the points of having the vite-plugin-pwa integrations.

[userquin]

I think we need to address the target of this RFC, we're talking about a lot of concepts:

1. parallel execution of write and close bundle plugin hooks that cannot be converted to be sync
2. app frameworks with SSR and SSG with custom cli or using Vite cli with multiple build as a plugin

This is my point of view, I don't want it to be misunderstood, I think we've reached the point where Vite can't do everything, application frameworks also have to provide support for certain capabilities (SSG):

• application frameworks should add integration hooks for SSG: vite-ssg, VitePress, Astro, îles, SvelteKit... We cannot force how an application framework with multiple builds via viteBuild or via Vite cli work (try moving the logic of vite-ssg, VitePress, Astro, îles to use the approach mentioned by @bluwy will be almost impossible, it should be done via integration)
• plugin or application framework authors may or may not provide application frameworks integrations (via custom integration hooks or via Rollup intercomunication plugin api)
• maybe we can add the concept of SSG to Vite but this is another history

There are plugins that will require an extra step to do its work, such plugins will work without any problem if the Vite build is a normal build, for example running the client build for an SPA. Plugins such as pwa, sitemap and the integrity (mentioned by Anthony) will just work.

On the other hand, we have app frameworks with SSR and SSG that runs its own cli, that's, it will run multiple builds using viteBuild. There are app fws that provide custom integration hooks (Astro and îles) and others without such hooks (vite-ssg: it provides the integration hook, but it will require additional args on onFinished hook).
Plugin or app fws authors should provide integration with such plugins. For example, I sent integration for Astro and íles for pwa plugin, for vite-ssg we just look for the pwa plugin after ssg process finish.
VitePress has now a hook after SSG finish, will be there in next release, I sent the PR, and so we can add the integration.

Finally, we have app frameworks with SSR and SSG that will be a plugin, for example SvelteKit. The nature of kit plugin will have a race condition in both hooks with the plugins; such race conditions will be there only and only if the corresponding plugins hooks are async: for example, sitemap can be converted to be sync and we can register it via writeBundle and enforce post or closeBundle with enforce pre.

For SvelteKit I suggest to add such custom integrations via the kit config file or kit plugin options, just like the adapter entry, and so, from the kit plugin call the corresponding integrations hooks.
In fact you can move the kit pwa module as another integration. It is just about adding new app fw capabilties. Why you can add a custom adapter but not your custom pwa impl or just a sitemap generator? Kit has hardcoded the kit pwa integration like vite-ssg with pwa plugin, just move it as another integration.

I guess Rollup should add a new preCloseBundle hook and/or the capability to run plugins hooks with kind async, parallel and async behavior sequentially (this is the problem that triggers this RFC). This will help for example to SvelteKit to work with pwa plugin without any integration, but will require pwa plugin update to move the build plugin logic to preCloseBundle hook or add a new custom build plugin for SvelteKit for sequential support.

About the vite-plugin-pwa: it is almost impossible to explain all concepts involved, since we have client and server stuff, the plugin in any case should be added in the Vite plugins entry, the problem with the integration using it in SvelteKit projects is about the write and close plugins hooks with async behavior.

[bluwy]

The reason I brought up plugin vs custom CLI is because SvelteKit's plugin approach was the initial motivation for this RFC. Astro and iles wont benefit from this generic proposal as they have a step-by-step build process, so some coupling is always needed due to vite-plugin-pwa's special requirements. So we're trying find what the real world usecase for this is.

However, this proposal does allow ootb integration if Astro and iles want to refactor their code into plugin-ish flows like @antfu suggested for vite-ssg (which I feel like is another code smell of this proposal).

A custom CLI won't fix this dilemma, which I agree, but the plugin approach is not the solution too. If I'd have to pick a right solution, it would still be frameworks having special knowledge of vite-plugin-pwa, or better that vite-plugin-pwa have special knowledge of frameworks due to it's special requirements for generating service workers.

[benmccann]

special requirements for generating service workers

What special requirements are you referring to? The main requirement is that it run after all files have been generated, which doesn't seem like a particularly special requirement

Another plugin I'd like to create is a vite-plugin-sitemap. It would also require that all files have been written to disk before it runs

[bluwy]

The main requirement is that it run after all files have been generated

For me, this is a special requirement because there are different definitions of it.

• In a Vite SPA, this is after writeBundle
• In vite-ssg, this is after the client has been built, server has been built, and finally prerendered (vite-plugin-pwa would've ran twice already)
• In SvelteKit, this is after the client has been built, server has been built, service worker has been built (planning to be removed), prerendered, and before adapt. And adapt just so happens to be in the closeBundle hook because SvelteKit knows about vite-plugin-pwa. Adapt would've stayed in writeBundle otherwise.

(Not familiar of Astro and iles build flow, but should be similar to vite-ssg)

The takeaway is that a Vite plugin can't know if a framework is finished doing it's thing unless they refactor themselves to SvelteKit's plugin approach (or similar).

[userquin]

• In SvelteKit, this is after the client has been built, server has been built, service worker has been built (planning to be removed), prerendered, and before adapt. And adapt just so happens to be in the closeBundle hook because SvelteKit knows about vite-plugin-pwa. Adapt would've stayed in writeBundle otherwise.

That's not 100% true - Ben was working on aligning Kit and Vite and the pwa plugin helped him in the process, for example, to set up the outDir, publicDir and /_app folders correctly (the pwa plugin is just a coincidence, but it could have been someone else). There are people on Discord with similar issues with their custom plugins.

The adapter process was moved originally to allow run some plugins between the prerendering and the adapter processes, any plugin after kit didn't run (process.exit). Any other plugin like sitemap cannot run also, pwa and sitemap need run between both processes since both will generate assets and also need those assets to be copied by the adapter to the corresponding output.

[userquin]

FYI: you can take a look at the configuration required to run the old kit version with the pwa plugin (for each project, if you change the kit adapter it may or may not work):

• kit configuration: https://github.com/antfu/vite-plugin-pwa/blob/main/examples/sveltekit-pwa/svelte.config.js#L22
• pwa plugin config: https://github.com/antfu/vite-plugin-pwa/blob/main/examples/sveltekit-pwa/pwa-configuration.js
• pwa script to make the pwa build via Vite resolveConfig: https://github.com/antfu/vite-plugin-pwa/blob/main/examples/sveltekit-pwa/pwa.js
• script after Kit build: https://github.com/antfu/vite-plugin-pwa/blob/main/examples/sveltekit-pwa/package.json#L24

I removed all previous stuff with just a few lines in the pwa plugin Kit integration PR, now it seems zero configuration.

[lukastaegert]

Also going off on a tangent here, I think Vite should definitely think about how to do SSG in a "good" way, but this would be a larger thing while the proposal at present could be a stop-gap measure until there is a better approach in place (e.g. there could also be Vite adding a virtual "buildOutput" structure that is above Rollup bundles containing all generated files with its own sequential generateBuildOutput to interact with etc.).

[yyx990803]

I'll try to summarize a bit here:

1. There are two related motives behind this RFC:

   1. The more generic case of "I want to run some command line tools that I cannot apply to the in-memory bundle, and they need to have an order" (as summarized by @lukastaegert)
   2. The more specific case of "being able to run another vite build inside a vite build without race conditions" (a la SvelteKit)

   I believe (i) is a valid use case, and it provides a way for higher-level projects (e.g. VitePress) to be able to become compatible with plugins that rely on fully-written-to-disk file information (e.g. vite-plugin-pwa). The value I see here is that it allows better compatibility between tools without the knowledge of each other as long as they follow the simple convention of "all files are written to disk inside a sequential writeBundle hook".

   There are disagreements on whether this RFC is the right solution to (ii) on the SvelteKit side - I will leave that for the Svelte team to decide, but from Vite's perspective I don't believe that should block this RFC since we already see value in (i). If it happens to solve Svelte Kit's problem, then that's a nice bonus.

2. @lukastaegert mentioned that this RFC is related to the more generic question of "how do we replace function hooks with objects that can alter the hook's execution model". This is out of the scope for this particular RFC, but is somewhat a pre-requisite since we want future direction to align with what we land here. So here's what I think we should do:

   • Settle on a tentative design for writeBundle and closeBundle in this RFC, with consideration for future application to other hooks
   • Land the feature in Vite as experimental and test it for a bit
   • Open a separate RFC (maybe in Rollup?) to discuss the full "hooks as object" details
   • If we end up changing the design in the separate RFC, we can still update Vite to match since the feature is shipped as experimental and probably only used by projects involved in this discussion.

[patak-dev]

Thanks for the summary, about 2.2 I wonder if this could be added as experimental to Rollup instead of Vite (even before rollup 3, so we can start testing). I think we'll need that with the current form of the RFC. But maybe Im missing a hack we could do on our side, if that is the case I agree it is easier to get the feedback ourselves directly.

Oh, nvm, I imagine we can overwrite writeBundle directly

[lukastaegert]

Actually, I am already looking into it, but it will take a few more days. I would definitely put this into Rollup 2, even as non-experimental as I do not see a breaking change involved. I would start with support for changing the hook order, though, as that is definitely less controversial and provides immediate value e.g. to the commonjs plugin.

[brillout]

Discussion for 1.ii: #9496 - Should $ vite build also build SSR?. There doesn't seem to be consensus whether that should be the case. We can then discuss how to achieve this.

[lukastaegert]

The Rollup RFC/PR to support object form hooks: rollup/rollup#4600

Comments welcome!