Deployed application fails with "500 internal error" on both netlify and vercel

[mcmxcdev]

Describe the bug

I have been trying for a good 2 months now to get my Sveltekit website deployed and up-to-date again, but can't seem to figure out what's wrong.

The application is working just fine in dev and build + preview. Netlify and Vercel deployments are succeeding as well. As soon as I launch the page though, I am running into a completely useless "500 internal error" without any further logs or information about what the problem is.

I encounter these errors on all pages which load blog posts with code like const allPosts = await fetch('/blog.json').

Reproduction

I can offer to set my project temporarily to be public so that the code can be inspected.

Logs

No response

System Info

System:
    OS: Linux 5.15 Manjaro Linux
    CPU: (8) x64 Intel(R) Core(TM) i7-2600K CPU @ 3.40GHz
    Memory: 6.96 GB / 15.61 GB
    Container: Yes
    Shell: 5.9 - /bin/zsh
  Binaries:
    Node: 18.12.1 - ~/.volta/tools/image/node/18.12.1/bin/node
    Yarn: 1.22.19 - ~/.volta/tools/image/yarn/1.22.19/bin/yarn
    npm: 8.16.0 - ~/.volta/tools/image/npm/8.16.0/bin/npm
  Browsers:
    Brave Browser: 108.1.46.140
    Firefox: 107.0.1
  npmPackages:
    @sveltejs/adapter-netlify: 1.0.0-next.85 => 1.0.0-next.85 
    @sveltejs/kit: 1.0.0-next.579 => 1.0.0-next.579 
    svelte: ^3.54.0 => 3.54.0 
    vite: ^4.0.0 => 4.0.0

Severity

blocking all usage of SvelteKit

Additional Information

No response

[Rich-Harris]

If you're seeing a 500 Internal Error then something should definitely appear in your logs (you will need to be looking at the logs as the error occurs, since they're not buffered).

Have you tried creating a minimal project that demonstrates the problem?

[mcmxcdev]

On Netlify, "Deploy logs" are all green, "Function logs" only show "waiting for logs..." forever and nothing happens.

The root page shows the error, same for /blog, and when hovering over "Blog" in the topbar, then the network request for blog.json errors out.

sitemap.xml and rss.xml also work fine locally when navigating to them, but are broken on the deployed app.

[mcmxcdev]

Okay, I was able to quickly throw together a minimal reproduction case:

Clone https://github.com/mcmxcdev/sveltekit-issue-8039 locally, run it and navigate to blog and then to the single blog post entry. It should work as expected.

The same project is deployed to https://main--stalwart-pika-07042f.netlify.app/, it will run into http 500 when preloading blog.json on hover and the single blog post entry will break (in the dev tools this time)

[Rich-Harris]

Haven't had a chance to run the repo yet, but the problem is almost certainly that you're trying to read content from the filesystem that doesn't exist in the serverless functions.

We might add a way to include files in Lambda-based adapters in future, but until then if you want to access the filesystem (outside a non-Lambda Node.js deployment) then it has to be done during prerendering.

Acknowledge that this could use a more helpful error.

[iva2k]

I've been using helper scripts in package.json to test locally what adapter-vercel and adapter-netlify builds do:

{
  scripts: {
    "build:netlify": "cross-env NETLIFY=true pnpm build",
    "build:vercel": "cross-env VERCEL=true pnpm build",
    ...
  }
}

Inspecting their outputs in ./build (for adapter-netlify) and ./.vercel/output (for adapter-vercel) and comparing to normal builds may shed some light. It may be even possible to serve them out (without svelte-kit built backend functions).

I got my template created from SvelteKit demo app to deploy cleanly on both, and the above was instrumental to figure out what wrong info I picked from outdated Internet pages, which has plenty.

Another thing I do regularly is make a clean SvelteKit demo app, and diff it to my code base. SvelteKit dev velocity is insanely high, and things change daily.

[mcmxcdev]

Haven't had a chance to run the repo yet, but the problem is almost certainly that you're trying to read content from the filesystem that doesn't exist in the serverless functions.

We might add a way to include files in Lambda-based adapters in future, but until then if you want to access the filesystem (outside a non-Lambda Node.js deployment) then it has to be done during prerendering.

That's a good hint and most likely the issue!
I have used @sveltejs/adapter-static for like 2 years until it started throwing Error: Encountered dynamic routes where I couldn't figure out how to get rid of it except for attempting to switch adapter (which leads to this issue now)

So how can I now have blog posts loaded? No adapter-static since I have dynamic routes like /blog/[slug] but also Netlify/Vercel don't support loading from file system.

Acknowledge that this could use a more helpful error.

I just tried to switch back to @sveltejs/adapter-static with strict: false and everything builds fine. I open the preview deployment on Netlify and it shows me a Netlify 404 page...
So yeah, it definitely needs more logging here in all the important parts, otherwise no clue what went wrong and its no fun.

[iva2k]

adapter-static can serve dynamic routes ok. Current demo app has /sverdle page which is dynamic and adapter-static is fine with it. I'm guessing "dynamic" means a node route with sub-routes, not the fact that it has server-side data (/sverdle route has both).

But if you enable prerender/ssr flags in a certain way globally (i.e. in /src/routes/+layout.svelte), the adapter complains. I had to create my own "map" of what these flags mean, since most docs online (including SvelteKit's own) are not accurate. E.g. 'ssr=false' does not create a SPA as stated, but "fallback=" in adapter-static does. I had to enable prerender='always' and ssr=true globally for SPA to make deep-links work (or there's probably a bug in client-side router that does not navigate to the route after fallback page). But I digress...

You can add //+page.ts/js, and reset the offending flag for that page. I did so in my template project, and I have it build adapter-static/vercel/netlify without complaining. When dynamic page throws an error in the adapter, make export const prerender = false; in that route's +page.ts/js.

[Rich-Harris]

Current demo app has /sverdle page which is dynamic and adapter-static is fine with it

No, it isn't. sverdle requires a server.

'Dynamic' routes in the sense of dynamic parameters — /blog/[slug] — are a-ok, however. (https://kit.svelte.dev/docs is an example — /docs/[slug] reads from the filesystem at prerender time.)

[infovore]

So how can I now have blog posts loaded? No adapter-static since I have dynamic routes like /blog/[slug] but also Netlify/Vercel don't support loading from file system.

They do support loading from the file system. The problem is that the file isn't there for them to load; it doesn't get moved into the relevant place for the functions, because there's no obvious need to do so.

If your posts really are JSON, is it possible for you to import them, rather than using fetch? eg: import allPosts from "/posts.json"? If you do this, posts.json now becomes a dependency of the function - it can be resolved, because it's imporpted - I believe this means it'll be copied over into the relevant function. (It's unclear if posts.json is a server-generated endpoint, or just a file in the root of your site).

[iva2k]

@Rich-Harris

No, it isn't. sverdle requires a server.

adapter-static provides a server, at least for vite dev and vite build && vite preview, I did not try deploying with adapter-static. What I meant is that there is no error from adapter-static if src/routes/sverdle/+page.ts has export const prerender = false;, but if export const prerender = true;, adapter-static throws an error about that route being dynamic.

[mcmxcdev]

Generally speaking, the adapters need more documentation on what they can and can't do. I really don't care which one to use but just want it to work.

If your posts really are JSON, is it possible for you to import them, rather than using fetch?

That was a good suggestion and I tried to import $lib/posts instead of fetch the posts. Still running into 500 internal error unfortunately, locally still works fine.

This is what my posts file looks like:

import fs from 'fs';
import frontMatter from 'front-matter';
import { marked } from 'marked';
import Prism from 'prismjs';
import 'prism-svelte';
import loadLanguages from 'prismjs/components/index.js';
import readingTime from 'reading-time';

loadLanguages(['shell', 'markdown', 'json']);

const posts = fs
  .readdirSync('./src/posts')
  .filter((elem) => elem.endsWith('.svx'))
  .map((postFilename) => {
    const postContent = fs.readFileSync(`./src/posts/${postFilename}`, {
      encoding: 'utf8',
    });

    const postFrontMatter = frontMatter(postContent);

    const renderer = new marked.Renderer();

    renderer.code = (source, lang: string) => {
      const html = Prism.highlight(source, Prism.languages[lang], lang);
      return `<pre class='language-${lang}'><code class='language-${lang}'>${html}</code></pre>`;
    };

    const html = marked.parse(postFrontMatter.body, { renderer });

    const readingTimeDuration = readingTime(postFrontMatter.body).text;

    return {
      // @ts-expect-error Spread types may only be created from object types.
      ...postFrontMatter.attributes,
      html: marked.parse(html),
      readingTime: readingTimeDuration,
    };
  });

const modifiedPosts = posts
  .filter((post) => !post.hidden)
  .sort((a, b) =>
    new Date(a.creationDate).getTime() > new Date(b.creationDate).getTime()
      ? -1
      : new Date(a.creationDate).getTime() < new Date(b.creationDate).getTime()
      ? 1
      : 0,
  );

export default modifiedPosts;

[infovore]

Alas, I meant really a JSON file. Your example is an endpoint on your server, returning JSON, so you won't be able to load it from import - which only works for files directly available.oj disk.

I have anecdotal success of doing what you're doing - anecdotal in that I no longer remember it all, but in a nutshell:

• I do what you're doing for loading posts in development; in production, I have a pre-build phase that bakes out a JSON file to disk
• you could then dynamically import that if you're not in a dev environment, I think

This means it would get imported at build time, and then get deployed - but you'd still develop against the local endpoint.

[dummdidumm]

The issue here is that the blog.json route is not prerendered, so that "read the blog content from the file system and transform it" is done at runtime on Netlify, where it can't work because those files aren't there and you don't have access to the file system like that. You would need to add export const prerender = true to blog.json/+server.js so that it's done while building the app, not while running it.