fscache: teach fscache to use NtQueryDirectoryFile

[benpeart]

Using FindFirstFileExW() requires the OS to allocate a 64K buffer for each directory and then free it when we call FindClose(). Update fscache to call the underlying kernel API NtQueryDirectoryFile so that we can do the buffer management ourselves. That allows us to allocate a single buffer for the lifetime of the cache and reuse it for each directory.

This change improves performance of 'git status' by 18% in a repo with ~200K and 30k folders.

Documentation for NtQueryDirectoryFile can be found at:

https://docs.microsoft.com/en-us/windows-hardware/drivers/ddi/content/ntifs/nf-ntifs-ntquerydirectoryfile
https://docs.microsoft.com/en-us/windows/desktop/FileIO/file-attribute-constants
https://docs.microsoft.com/en-us/windows/desktop/fileio/reparse-point-tags

To determine if the specified directory is a mounted folder, inspect the FileAttributes member to see if the FILE_ATTRIBUTE_REPARSE_POINT flag is set. If so, EaSize will contain the reparse tag. To determine if the reparse point is a mounted folder (and not some other form of reparse point), test whether the tag value equals the value IO_REPARSE_TAG_MOUNT_POINT.

Signed-off-by: Ben Peart benpeart@microsoft.com

[whoisj]

This change improves performance of 'git status' by 18% in a repo with ~200K and 30k folders.

I'm curious the impact on smaller, more reasonably sized repositories; especially given the explicit CreateFileW call (which all know is a pig, and you can only put so much lipstick on it).

I'm also curious about the impact on very flat repositories, which are rather common in OSS projects. Effectively 90% of the files are in the root, with a few specialized files in sub-dirs.

[benpeart]

This change improves performance of 'git status' by 18% in a repo with ~200K and 30k folders.

I'm curious the impact on smaller, more reasonably sized repositories; especially given the explicit CreateFileW call (which all know is a pig, and you can only put so much lipstick on it).

I'm also curious about the impact on very flat repositories, which are rather common in OSS projects. Effectively 90% of the files are in the root, with a few specialized files in sub-dirs.

I tested this on the git repo itself and saw a 9.8% savings but with small repos, the difference is between .0305 seconds and .0275 seconds (5 runs averaged) so we're getting down to the noise level. In short, with small repos it just doesn't make make much difference - they are already very fast.

[dscho]

Only a couple minor changes I'd like to see, most of them some clarifications in the commit message (in particular a link to more information about the low-level API used by the new code).

Thank you!

[dscho]

Let's merge this! Thank you so much!