[fs] Path::new(r#"C:\hiberfil.sys"#).exists() returns false, despite the file existing

[strega-nil-ms]

I noticed this bug while researching the same bug in https://github.com/microsoft/STL.

// test.rs
use std::{env, fs};
use std::path::Path;

fn main() {
  for p in env::args().skip(1) {
    println!("{:?}.exists() = {}", p, Path::new(p).exists());
    println!("  => (metadata = {})", if fs::metadata(p).is_err() { "error" } else { "ok" });
  }
}

Given test.exe C:\hiberfil.sys, I expected:

"C:\\hiberfil.sys".exists() = true
  => (metadata = ok)

Instead, this happened:

"C:\\hiberfil.sys".exists() = false
  => (metadata = error)

Solution

The related issue in the STL is microsoft/STL#2370; the PR to fix this in the STL is microsoft/STL#2715.

The problem is that fs::metadata opens the file; this is not allowed for the following four files (not sure if there are additional files which act similarly):

• C:\DumpStack.log.tmp
• C:\hiberfil.sys
• C:\pagefile.sys
• C:\swapfile.sys

We can get around this by using FindFirstFileW instead of open(); is that something that's desired, or is this considered a weird enough use case that we aren't interested in fixing it?

[nagisa]

Documentation explicitly documents that exists is equivalent to accessing metadata of the file, doesn't it? Changing this behaviour would be a breaking change I'm afraid we wouldn't be able to afford.

[strega-nil-ms]

@nagisa true, the question was whether fs::metadata(Path) could use FindFirstFileW() instead of CreateFileW() followed by GetFileInformationByHandle(); that's how we're solving this in the C++ standard library.

This function currently corresponds to the stat function on Unix and the GetFileAttributesEx function on Windows. Note that, this may change in the future.

(I'll note that this documentation is now wrong, since it now uses GetFileInformationByHandle)

This would mean that the Metadata returned from fs::metadata(p) would now have None for

• volume_serial_number
• number_of_links
• file_index

but otherwise it'd be complete (although for reparse points, we'd still have to open the file).

We could additionally do FindFirstFileW() followed by CreateFileW() -> GetFileInformationByHandle(), to fill out those fields. I'll note that we already have this behavior on UWP, and when getting metadata from a DirEntry.

[ChrisDenton]

Hm, changing fs::metadata is not something I'd be minded to do lightly. At least not without further investigation. Some questions that spring to mind:

• What happens if the user does not have permission to read the directory but does have permission to read the file?
• Can we get all the metadata that's needed? Ah your edit answered that.
• Does this affect performance?

[strega-nil-ms]

@ChrisDenton given that it's reading the directory, it should make performance better (assuming one doesn't do the double call thing).

I'm personally not that invested, I just thought I'd share with y'all since I have to fix this bug for the STL 😛

[ChrisDenton]

It's great that we have this report! I just want to make sure we investigate fully before making changes that may affect users of the standard library.

I'll also note we do have the unstable std::fs::try_exists whose behaviour can change.

[ChrisDenton]

What happens if the user does not have permission to read the directory but does have permission to read the file?

Ok I tested with two users. For the sake of this example, I'll call them "Admin" and "User".

As Admin I created a directory and denied User all access. Within the directory I then created a "test.txt" file and granted User access to just that file.

As User, I tried using FindFirstFile to test if the file exists. This failed with ERROR_ACCESS_DENIED. I then tried using GetFileAttributes which succeeded. As does opening the file with CreateFile,

Thus FindFirstFile can fail in situations where directly accessing the file would succeed.

[rgwood]

FYI, Nushell has run into this in the wild: nushell/nushell#4975

We'll work around it by falling back to FindFirstFileW if fs::metadata fails. Would be nice if fs::metadata could do something similar automatically, but I appreciate that there's a high bar for backwards compatibility and performance here.

[ChrisDenton]

To be clear, using FindFirstFile as a fallback (rather than replacement) answers my concerns and I think would be the best solution to this issue.