Add internal junction support to link APIs

[carlossanlop]

• Fixes Preview 7 inconsistent FileSystemInfo LinkTarget and ResolveLinkTarget results when using virtual drives #57582

In Windows, if a user creates a virtual drive using the subst command, they are actually creating a junction that points to a drive.

By adding junction support to our internal GetImmediateLinkTarget method, if the user creates symbolic links that point to files or directories inside that virtual drive, we can now ensure that the public API ResolveLinkTarget(returnFinalTarget: false) returns a FileSystemInfo in which the FullName points to a path that uses said virtual drive.

Note: When the user calls ResolveLinkTarget(returnFinalTarget: true), we use a different P/Invoke, which returns the absolute normalized path of the symlink's final target. This is expected behavior from Windows. So this code path remains untouched.

• Fixes Preview 7 Improper LinkTarget returned for a Directory Junction on Windows #57575

The root cause was the same as above: Both LinkTarget and ResolveLinkTarget(returnFinalTarget: false) call the internal method GetImmediateLinkTarget, which returns null if a reparse point that is not a symlink is encountered. And because ResolveLinkTarget(returnFinalTarget: true) calls a different P/Invoke, it is able to resolve any kind of reparse point, which explains the discrepancy. By adding support for mount points in GetImmediateLinkTarget, we can now return the target properly.

cc @fitdev

[ghost]

Tagging subscribers to this area: @dotnet/area-system-io
See info in area-owners.md if you want to be subscribed.

Issue Details

---

• Fixes Preview 7 inconsistent FileSystemInfo LinkTarget and ResolveLinkTarget results when using virtual drives #57582

In Windows, if a user creates a virtual drive using the subst command, they are actually creating a junction that points to a drive.

By adding junction support to our internal GetImmediateLinkTarget method, if the user creates symbolic links that point to files or directories inside that virtual drive, we can now ensure that the public API ResolveLinkTarget(returnFinalTarget: false) returns a FileSystemInfo in which the FullName points to a path that uses said virtual drive.

Note: When the user calls ResolveLinkTarget(returnFinalTarget: true), we use a different P/Invoke, which returns the absolute normalized path of the symlink's final target. This is expected behavior from Windows. So this code path remains untouched.

• Fixes Preview 7 Improper LinkTarget returned for a Directory Junction on Windows #57575 (Pending to fix)

We decided that when a FileSystemInfo wraps a junction, calling LinkTarget should return null for 6.0. This is so the user can determine that the file is not a symlink.

Author:	carlossanlop
Assignees:	carlossanlop, Jozkee
Labels:	area-System.IO
Milestone:	6.0.0

[carlossanlop]

I renamed this to match the names that were provided in the DUMMYUNIONNAME section of the official Windows docs.

[carlossanlop]

Also, the structs are not nested. This is how PowerShell had it already: https://github.com/PowerShell/PowerShell/blob/56d22bc3865fca445145fd685d6d6f6d63194701/src/System.Management.Automation/namespaces/FileSystemProvider.cs#L7957-L7985

[carlossanlop]

If instead of doing this, I splitted the virtual drive tests into multiple different test classes, then each test class would create its own virtual drive. We have so many symlink tests now, that we would end up depleting the available drive letters.

[carlossanlop]

• ReparseTag is not a flag, so the bitwise operation was unnecessary. Examples from PowerShell: 1, 2.
• The if logic is now reversed, so that we only return null if the link is not a symlink or a mount point.

[jozkee]

Source changes look mostly good to me; the tests are a bit hard to comprehend and I think we are missing some scenarios.

[carlossanlop]

Seems Windows Nano does not have subst available. The CI is failing in that leg.

I'll have to add the IsNotWindowsNanoServer condition attribute to the virtual drive tests.

[adamsitnik]

Overall LGTM, but please address the nits that I've found before we merge it. Thank you @carlossanlop !

[adamsitnik]

is there a way to ensure that it's called from Remote Executor?

[carlossanlop]

I'm not sure if we can add such a check here.

[danmoseley]

it would have to be something like this
Debug.Assert(Environment.StackTrace.Contains("RemoteExecutor"));
or it would be reasonable to add a static flag in RemoteExecutor so you could just test RemoteExecutor.IsInRemoteExecutor.

[adamsitnik]

I am totally OK with a test creating a temporary drive on my machine.

@ViktorHofer could it be an issue for the CI?

[ViktorHofer]

@dotnet/dnceng @MattGal is creating a temporary virtual drive as part of a test which runs on PRs and rolling builds something that we should avoid?

[MattGal]

I am totally OK with a test creating a temporary drive on my machine.

We should maybe make sure this is a general consensus with real users who might run your test on their personal machines; the odds of leaking are there.

As for whether it's something to avoid, that depends. TL;DR: Probably OK, but you might need to consider cleanup automation (there's not even a try around this code, if it blows up it just leaves things where it was...) I'll give a few responses since I don't know exactly where this will run.

• If running on an Azure Devops Hosted build machine - no issues, these are unique VMs per build
• If running on a helix-based buildpool.* machine: my (admittedly limited) understanding of subst is that the virtual drive mappings don't persist across reboots; we always reboot, no issues as long as this is accurate.
• If running on a VM-based Helix test machine: it's probably OK because you're the only user creating random drive mappings, but if there's a failure to cleanup and the same machine runs automation doing this repeatedly, as I don't think you can get more than 26 drive letters.
• If running on an on-premises, physical Helix test machine (every Windows ARM machine) : scariest possible choice; anything that's actually persistent might exist a long time, so if there's a letter-leak this is where I'd expect trouble.

All that said, I think if you had some sort of try {} finally {} here that cleans up the mapping when you're done, you're Ok.

[adamsitnik]

@MattGal thank you for a very detailed answer!

[carlossanlop]

I think if you had some sort of try {} finally {} here that cleans up the mapping when you're done, you're Ok.

I added a Dispose method to the test class, which clears the mapped drive. And by the way, the test class only creates one mapped drive and reuses it for all of its test methods. precisely because I was seeing the problem of using all the available drive letters. Now I only use one per test class.

I'm not sure what I would add to the finally part of the try catch I added to the Dispose method. Suggestions? Or is it good as it is?

[danmoseley]

BTW I have no preference, but note for other "somewhat impactful" tests we made them outerloop. That way we rarely impact our contributors, but still are properly covered. Example: the process.start tests that pop up the browser. It's an option but I have no opinion.

[jozkee]

Otherwise; LGTM. Only suggestions are for tests.

[jozkee]

If the subst call ever gets called twice, it would fail with the message Drive already SUBSTed.

Is there something that catches that error? my concern is that a race condition could occur and subst gets unintentionally executed twice and the CI would fail. Or maybe is not a big deal and we can wait for the CI to actually fail.

[jozkee]

Do you know why this is not supported in Nano?

[carlossanlop]

Windows Nano Server is a stripped down version of Windows. A bunch of components are removed to make it lightweight. The CI error says the subst executable was not found. It was supposed to be located in C:\Windows\System32, which is part of either PATH or some default env variable.

[jozkee]

if subst is not in the PATH for Windows Nano, can't we use the full path to the subst executable?

[jozkee]

Instead of this method you can add another parameter to the previous test, name it isIndirect or something alike.

[carlossanlop]

I attempted it but the test became way more complicated than it already is. I would rather keep them separate.

[adamsitnik]

the code above was not modified, but we should rather refactor it as well:

Depeneding on your personal preferences it could be:

public static bool CreateSymbolicLink(string linkPath, string targetPath, bool isDirectory)
{
    ProcessStartInfo startInfo = OperatingSystem.IsWindows()
        ? CreateStartInfo("cmd", "/c", "mklink", isDirectory ? "/D" : "", linkPath, targetPath)
        : CreateStartInfo("/bin/ln",  "-s", targetPath, linkPath);

    return RunProcess(startInfo);
}

Or

public static bool CreateSymbolicLink(string linkPath, string targetPath, bool isDirectory)
    => RunProcess(OperatingSystem.IsWindows()
        ? CreateStartInfo("cmd", "/c", "mklink", isDirectory ? "/D" : "", linkPath, targetPath)
        : CreateStartInfo("/bin/ln",  "-s", targetPath, linkPath));

[carlossanlop]

I also refactored it, initially. I used the code you shared the first time. It caused all the symlink enumeration tests to fail. It was taking me too long to figure out why so I decided to revert it and the tests passed again.
I can take a look again in the related PR for AppExecLinks, if you don't mind.

[carlossanlop]

/backport to release/6.0-rc1

[github-actions]

Started backporting to release/6.0-rc1: https://github.com/dotnet/runtime/actions/runs/1175308538