Vite is slow on large codebases on windows

[Jabher]

Describe the bug

While using Vite for bundling large codebase (around 700 files served) on Windows machine (reproducible both on windows 10 and 11 beta) Vite page load time takes around 5-30 seconds depending on drive type (NVME on-board SSD, NVME SSD, SATA SSD), comparing to <2 seconds on Mac (did not try on linux).

Core reason is tryFsResolve function

Which is using accessSync call, which is internally using multiple sync FS calls, and NTFS (specifically NTFS, looks like FAT is working better) is rather complex system with multiple checks that can take milliseconds; also this is causing the server act in sync mode (instead of async) and not utilize the benefits of parallel http requests.

Another reason is rather slow HTTP service which takes several seconds overhead; I was able to win few seconds with page with this script (https://gist.github.com/Jabher/c9e5f9bb905c8ec2ab6289a181c2b6ce). Probably switching from express to fastify or native code will make things work faster.

Reproduction

issue seems obvious as long as performance result like was provided and specific code part was targeted as a reason of bug

System Info

System:
    OS: Windows 10 10.0.22000
    CPU: (8) x64 Intel(R) Core(TM) i7-8650U CPU @ 1.90GHz
    Memory: 4.26 GB / 15.92 GB
  Binaries:
    Node: 16.4.2 - C:\Program Files\nodejs\node.EXE
    Yarn: 1.22.10 - ~\AppData\Roaming\npm\yarn.CMD
    npm: 7.20.3 - C:\Program Files\nodejs\npm.CMD
  Browsers:
    Edge: Spartan (44.22000.120.0), Chromium (92.0.902.73)
    Internet Explorer: 11.0.22000.120
  npmPackages:
    vite: ^2.5.0 => 2.5.0

Used Package Manager

npm

Logs

No response

Validations

• Follow our Code of Conduct
• Read the Contributing Guidelines.
• Read the docs.
• Check that there isn't already an issue that reports the same bug to avoid creating a duplicate.
• Make sure this is a Vite issue and not a framework-specific issue. For example, if it's a Vue SFC related bug, it should likely be reported to https://github.com/vuejs/vue-next instead.
• Check that this is a concrete bug. For Q&A open a GitHub Discussion or join our Discord Chat Server.
• The provided reproduction is a minimal reproducible example of the bug.

[y1d7ng]

may same issue #4865 , try https://github.com/antfu/vite-plugin-optimize-persist

[Jabher]

no, it is not related; dependencies are loaded correctly, I just have around 600 files in project itself.

[Yesterday17]

This problem also exists if you're using a mounted remote file system. Since the speed of existSync(and other sync methods) is slow(due to latency), resolver was blocked, causing other (incoming) requests to wait for the previous (ongoing) resolve process to finish. The server degraded from (async) parallel processing to serial processing.

The problem can be reproduced if you mount a NFS/CIFS partition from a remote server and run a project on that with Vite.

[patak-dev]

@Yesterday17 PR welcome if you have a good setup to test and remove some of these existSyncs.

[Yesterday17]

@Yesterday17 PR welcome if you have a good setup to test and remove some of these existSyncs.

I tried to make all fs calls async yesterday. But then I found that a custom resolver would be assigned to Module._resolveFilename, a sync function called by node, making it impossible to call any async function in resolve process. It seems that vite is reusing require system in node, which block migration to async.

[patak-dev]

Interesting, thanks for digging in. I hope we find a way to refactor it to async.

[Yesterday17]

import-meta-resolve might be a good async alternative of require.resolve, and require can be replaced by dynamic import. I'll try it later.

[mreduar]

Finally I think I found the right issue. I have a similar problem, maybe it is the same, maybe not. The truth is that my development compile now takes about 30 minutes to compile, this is something that only happens to me in the team, the rest of my colleagues compile in 2 seconds having the same code base, only they are in Linux with Docker and I am in Windows without docker (I am not using WSL2), and no, it is not because of computer resources, and if it were I do not think the difference would be so brutal. We are working on a somewhat medium-sized project, about 400 components and counting.

We are using Tailwind, and I have read that it can cause slow startup, but why only me if we have the same configuration?

This is how long it takes me.

This is how long it takes my teammates.

Does anyone have any suggestions, or at least where to start?

[Thyiad]

I have the same problem.
vite optimize is very slowly on windows.
mac is 1 minute
windows is 10 minute

[mreduar]

I have the same problem. vite optimize is very slowly on windows. mac is 1 minute windows is 10 minute

Check this @Thyiad #7608 (comment)

[bluwy]

We have been improving Vite's performance lately and I believe this has been fixed. Closing this as the work has been completed, but if there's more to work on, we'll track it at #12363 instead. Appreciate if anyone following would give Vite 4.3 beta a spin too.