CreateProcess fails with ERROR_INVALID_PARAMETER when trying to run a file on mounted volume

[Karandra]

Environment

• Windows version: Windows 7 x64 SP1
• Processor architecture: x64
• Dokany version: 2.0
• Library type: Dokany

Check List

• I checked my issue doesn't exist yet
• My issue is valid with mirror default sample and not specific to my user-mode driver implementation
• I can always reproduce the issue with the provided description below.
• I have updated Dokany to the latest version and have reboot my computer after.
• I tested one of the last snapshot from appveyor CI

Description

Using mirror sample I mounted file system to M: drive using entire game folder as root. FS mounted well and I tried to run file enbhost.exe there from explorer. Program does not start with error "The parameter is incorrect.". Then I tried to run the file using CreateProcess function. It doesn't work either, CreateProcess failed and GetLastError returned ERROR_INVALID_PARAMETER (87).

Dokan 1 also has same issue, but only when I try to run programs in same process where Dokan is running, but I'm not sure if it's caused by Dokan or by my own FS implementation. For 2.0 I used mirror sample.

Why I can't run programs in mount point? Is it a Dokany bug or I doing something wrong? Here is the code I used, just in case.

STARTUPINFO tStartupInfo = {0};
tStartupInfo.cb = sizeof(tStartupInfo);

PROCESS_INFORMATION tProcessInfo = {0};
tProcessInfo.hProcess = INVALID_HANDLE_VALUE;
tProcessInfo.hThread = INVALID_HANDLE_VALUE;

WCHAR s[1024] = L"M:\\enbhost.exe";
bool bOK = CreateProcessW
(
	s,
	NULL,
	NULL,
	NULL,
	FALSE,
	0,
	NULL,
	NULL,
	&tStartupInfo,
	&tProcessInfo
);
fprintf(stdout, "%s: %u\r\n", bOK ? "true" : "false", GetLastError());

Logs

DebugView++.txt

[Liryna]

@KerberX Hi,

Can you try with the mount manager option and see if it improves the situation?

[Karandra]

@Liryna Hi, no, running as mirror.exe /o /d /r "D:\Game Files\The Elder Scrolls\Skyrim" /l M: doesn't change anything.

This can probably be useful, log from FileSpy. This time I tried to run SaveTool.exe in M:\Soft\ folder.
FileSpy.txt

[Karandra]

@Liryna Hi,
I found why I can't run programs. This has something to do with GetFileSecurity function, because returning STATUS_NOT_IMPLEMENTED in MirrorGetFileSecurity solves the problem and allowing me to run any app. I can't check it on Mirror sample (I can't compile it), but it works on my own implementation, which is almost one-to-one copy of mirror.

This isn't good solution, GetFileSecurity can be important to others, but in my case it isn't. I don't really need security information for my purposes, at least right now. Although I'd like to know why MirrorGetFileSecurity prevents process creation.

[Liryna]

@KerberX Hi,

Well found 👍Probably there is a case not handle in MirrorGetFileSecurity that return INVALID_PARAMETER.

In your logs, there is an error "GetSecurityInfo error: 5" ERROR_ACCESS_DENIED probably releated to it.

I could not get the hand on my env for now.
an issue on the version 2.x.x, I made a fix not so long time ago but still another issue pending #607 (during delete but should probably not be important for you)

[Liryna]

Hi @KerberX ,

Have you been able to get more information or a workaround about this issue ?

[Karandra]

Hi @Liryna ,
Except returning STATUS_NOT_IMPLEMENTED I can't do anything to make it work. Sorry, I have not worked on this since my last reply.

[Karandra]

Hi, @Liryna ,
I somehow managed to run a program from the same process though I don't really know how (I just did some refactoring). But it works only in test program I use to test VFS. It doesn't work in GUI app, I don't know why. I don't think I changed some crucial logic. Functions Get/SetFileSecurity are still set to return STATUS_NOT_IMPLEMENTED.

Also I found that file browse dialog opened in the same process also causes VFS to unmount itself, but only if the mounted folder has .exe files. If there are any other files (including .dll, they are fine), the VFS will work as expected.

Here's the code I use to test VFS: https://github.com/KerberX/KxVirtualFileSystem/blob/master/KxVirtualFileSystem/KxVirtualFileSystemMain.cpp#L57

When I open dialog and mounted folder has .exe files, app hungs for a few seconds. In Process Hacker I see that all threads are paused except one and it stuck doing something in NtMapViewOfSection. Then VFS will unmount itself (and call DOKAN_OPERATIONS::Unmounted), probably because of some timeout.

Threads list.
https://user-images.githubusercontent.com/32564546/50617110-7a6caa00-0efc-11e9-8bf8-1a5f96261c13.png
Only one active thread.
https://user-images.githubusercontent.com/32564546/50617109-7a6caa00-0efc-11e9-88a4-eb69d01742e0.png
Main thread (suspended currently)
https://user-images.githubusercontent.com/32564546/50617541-de906d80-0efe-11e9-86ea-50e798907e32.png

[Liryna]

Hi @KerberX ,

Do you think there is a possibility you can do the test on Dokan version 1 ? Since v2 release there is a fix that could be what you are facing.

[Karandra]

I'll try to test that on v1.

[Karandra]

@Liryna ,
I've tested on latest 1.x release (1.2.1.2000) on slightly modified mirror sample with same results. I pasted this part into mirror's main function and placed DokanMain call into another thread (using std::thread class).

I can run a program, but opening file browse dialog causes mirror to hung. Even worse, it can freeze entire system, v2 has some kind of timeout in such case, this version just freezes forever.

[Liryna]

This should be retried with the code at head or in the next release.

[Karandra]

Wonderful! I hope I could test this later.