Incremental builds support for static adapter

[kaushalyap]

Describe the problem

Having to rebuild the static pages from scratch after minor change which takes lot of time if there lots of pages.

Describe the proposed solution

Only build the changed pages.

Alternatives considered

No response

Importance

would make my life easier

Additional Information

Gatsby v3 have incremental builds.

[felipeqq2]

That would be a game changer for big websites, currently I have to wait around 15 minutes to ship a simple fix.

[olimination]

We have currently the same use case:

• Site with more than 1000 pages to build
• Approach: Using the SSG approach with client-side code for personalization stuff by using the adapter-static with Netlify
• Content is served by some CMS
• For every content change the entire site gets built and after around 20 minutes the content update is live

Try of a solution propsoal:

• Extend the svelte-kit build command to accept a list of routes --routes [ROUTE_01] [ROUTE_02] ...
• Example of a call: svelte-kit build --routes index news/2021/my-news-01 about/our-values. This would build these 3 routes or pages and store it in the build folder
• Some CI/build pipeline then could publish the newly generated files to some publishing service, e.g. with Netlify you could use their Deploys API

Some open questions from my side:

• The svelte-kit build command is actually not tied to the adapter-static adapter package, maybe the passing of the routes argument needs to be solved differently?

What do you think :)?

Maybe @benmccann and @Rich-Harris you have some thoughts about this?

[dav245]

Just out of curiosity, couldn't you just build some pages by specifying entries option on the prerender configuration object? I suppose you will always need some mechanism for obtaining modified pages. So one could fetch list of modified routes, supply it to the entries option and build the application. Afterwards it is only question of merging created files with already built files.

Maybe my insight in the topic is too shallow, so please correct me if i am wrong.

[multipliedtwice]

This would be awesome to have this feature. If I understood correctly, Nextjs has similar feature getStaticPaths: https://nextjs.org/docs/basic-features/data-fetching/get-static-paths

[Rich-Harris]

As mentioned in #2369 (comment), config.kit.prerender.entries (alongside crawl: false) is the solution to this problem. You can put something like this in your svelte.config.js...

// svelte.config.js
import adapter from '@sveltejs/adapter-static';

export default {
  kit: {
    prerender: {
      entries: process.argv.length > 3 ? process.argv.slice(3) : ['*'],
      crawl: process.argv.length <= 3
    }
  }
};

...then do

npm run build

to prerender everything, or

npm run build -- /an-article-i-want-to-update /another-article-i-want-to-update

to update /an-article-i-want-to-update and /another-article-i-want-to-update. Or, if you have some programmatic way of determining which URLs have changed, you can put that logic in your config file instead. We could add a new argument to svelte-kit build, but it would only be marginally more convenient than what's already possible in userland, and it wouldn't help in the programmatic case.

Of course, if you're using a hosted CI service then you need to be able to access the previously built pages somehow — there's nothing SvelteKit can do to help with that.

A better solution to all this would be to have some form of incremental static regeneration (#661), but that's a post-1.0 TODO.

[prathamesh-gharat]

I think Next.js' getStaticPaths solves a major issue of not having to centrally manage the paths to be rendered (during ISR too if configured right?).

My use case is 20000+ pages spread out across different categories (types) of pages.
Publishing 1 article through a CMS should not need the whole website to be crawled and rebuilt. I love the thoughts put behind Svelte but I still need to try and figure out how incremental static re-generation will work in Svelte for my use case.

[stephane-vanraes]

Of course, if you're using a hosted CI service then you need to be able to access the previously built pages somehow — there's nothing SvelteKit can do to help with that.

The previously built pages are available on your published site, I think this one can easily be solved if you add a sitemap.xml to your deployment.

// svelte.config.js
import adapter from '@sveltejs/adapter-static';

async function readSiteMap() {
   /* fetch the sitemap and parse the routes out from it */
}

const entries = await readSiteMap();

export default {
  kit: {
    prerender: {
      entries: entries.length ? entries : ['*'],
      crawl: entries.length == 0
    }
  }
};

This doesn't even have to be a sitemap, you could create an endpoint or any other method to provide this information from the already deployed website.

[eecue]

I tried the suggestions from @stephane-vanraes and @Rich-Harris above and both failed for different reasons:

import adapter from '@sveltejs/adapter-static';
import { vitePreprocess } from '@sveltejs/kit/vite';

async function readSiteMap() {
	/* fetch the sitemap and parse the routes out from it */
	return ['/','/collections/tags_cats']
 }
 
 const entries = await readSiteMap();

/** @type {import('@sveltejs/kit').Config} */
const config = {
	preprocess: vitePreprocess(),

	kit: {
		adapter: adapter({
			fallback: null,
			precompress: false,
			strict: false
		}),
		prerender: {
			concurrency: 10,
			handleHttpError: 'ignore',
			handleMissingId: 'ignore',
			entries: entries.length ? entries : ['*'],
		    crawl: entries.length === 0
		}
	}
};

export default config;

Failed with this message:

Error: The following routes were marked as prerenderable, but were not prerendered because they were not found while crawling your app:
  - /all/[slug],  - /blogs/[slug],  - /blog,  - /blog/[slug],  - /d/[slug],  - /error,  - /indexes,  - /indexes/[slug],  - /photo/[slug],  - /robots.txt,  - /rss.xml

I can make it work if I disable prerender on those pages, but that's not really what I want here. Ideally I'd just be able to render a single dynamic blog page, or all blog pages, and no do everything else. Right now my website as millions of pages all statically generated, which takes about an hour. Ideally I'd only regenerate things that are changing. Like when I add a new blog post, or a new link or a new photo I'd just update the relevant pages. I'm happy to keep track of what those are, I just don't see a way to do a partial crawl etc.

Here's my site if you'd like to take a look: https://eecue.com ... basically I'v posted my entire photo achieve from 25+ years of shooting.

[devsou-com]

Also having exactly the same problem 😢
I'm using Svelte 5 release candidate, so I guess we need to wait.
#2369 (comment)