The great unmount challenge

[lundman]

Currently, when we try to unmount a dataset (explicitly or as part of export) we need to ask all open files to flush and close.
However, we must be missing a step as it turns out to be a challenge.

In particular, the WPSettings.dat file, created very early on in the mounting process.

Because we never receive an IRP_MJ_CLOSE for the opened handle, it has section cache associated with it, in vnode_flushcache() it still has a DataSectionObject set. (not-NULL). We can not proceed without this released or we will BSOD in the future.

The winbtrfs project does call FSRTL_VOLUME_DISMOUNT which definitely sounds interesting, could this be the missing piece? Or do they only call it for the virtual disk volumes (and not mounted datasets)
(@maharmstone if you happen to see this, perhaps you can confirm?)

However, if I just call it, I get an error, as in:

		root_file = IoCreateStreamFileObject(NULL, zmo->deviceObject);
		ntstatus = FsRtlNotifyVolumeEvent(root_file, FSRTL_VOLUME_DISMOUNT);
		ObDereferenceObject(root_file);

Which happens also if I call FSRTL_VOLUME_MOUNT at mount time. Then by luck, I noticed that if I call (at least) these line
(which btrfs also calls)

	status = IoReportDetectedDevice(
		DriverObject,
		InterfaceTypeUndefined,
		0xFFFFFFFF, // 0
		0xFFFFFFFF, // 0
		NULL,
		NULL,
		FALSE,
		&pnpDeviceObject);
	if (IoAttachDeviceToDeviceStack(pnpDeviceObject, DeviceObject) != NULL) {

Suddenly, I can send FSRTL_VOLUME_MOUNT without error (and presumably FSRTL_VOLUME_DISMOUNT) but
frustratingly, now the dataset mount no longer works at all! Explorer will show it as an empty device and if you
click on it you get "invalid function" (and not from calling my dispatch).

Again, I suspect I am missing something. Sure it works without IoReportDetectedDevice / IoAttachDeviceToDeviceStack but I think that is a fluke, and I really should be calling them.

But by calling those functions, I also need to do something else to make it think it is mounted (and to call my dispatch). Then, perhaps DISMOUNT will work, and WPSettings.dat will be closed.

[maharmstone]

You should only be calling IoReportDetectedDevice once, for your bus device, not for every volume available (which from a cursory glance it looks like you're doing). Your bus device then reports its PDO children through IRP_MN_QUERY_DEVICE_RELATIONS, AddDevice gets called by pnpmgr to create device objects for each one, mountmgr assigns it a drive letter once it realizes it's a volume, and it gets mounted the first time someone accesses it.

Basically have a look at https://github.com/maharmstone/btrfs/blob/master/src/pnp.c. I suspect that once you've got PNP sorted out you'll be a lot closer to solving your issue.

[lundman]

Ah - thought I could avoid the pnp thing, but perhaps that's why I'm in trouble. Thanks for the reply, I'll give it a go.

[maharmstone]

No worries. If it helps, it's not all that useful to talk about unmounting on Windows - you can do it, but the kernel will mount it again in short order. If you want a device to go away for whatever reason, you need to be thinking rather about getting the underlying PDO to be removed.

[lundman]

OK so, I do set and handle PNP irps, it is just not in its own file - all handled in the big dispatcher. I had skipped the notification callbacks, assuming it just tells me about new things coming and going - which I don't believe comes into play with the unmounting issue. But I've thrown in that code now, just in case..

Using the word unmount is just used as a familiar terminology for me.. The "what ever is needed to ask windows to close everything so I can remove the device".

Going over btrfs and my code again, It looks like;

• I use IOCTL_MOUNTMGR_VOLUME_ARRIVAL_NOTIFICATION, but the documentation seems to indicate that I shouldn't need to, if my pnp is correct.
• There is no IOCTL_MOUNTMGR_VOLUME_REMOVAL_NOTIFICATION, so definitely need something else.
• I never forward irps. In some cases, btrfs calls IoCallDriver() to send the irp lower - I haven't.. figured it is just sending to myself - and hopefully missing this is not involved with unmounting.

about getting the underlying PDO to be removed.

I should focus on that. btrfs starts with FSRTL_VOLUME_DISMOUNT which is why I was trying to get that to work. But possibly it suggests my PDO and/or VPD is not correctly configured - I am unfamiliar with what they do, is there a util to test/display/verify/show-devices so I can confirm I have it correct? Or compared it against btrfs?

[imtiazdc]

@lundman Currently uninstall and reboot doesn't delete the driver from C:\Windows\system32\drivers. Also, it continues to show the controller in device manager. Are those issues related to this unmount issue? Should we file a different bug for it for tracking?

[lundman]

Ah let's see.. if you do not create a pool then unloading the module "should" work, and if it doesn't, we should fix it.
But if you create/import any pool, then you can't - due to the unmount issue.

[imtiazdc]

Clean install, uninstall, reboot on a physical machine shows OpenZFS controller in device manager and driver in C:\Windows\system32\drivers. No zpool created.

[lundman]

There was a fix from datacore, removing the symlink - I will double check it made it in..

[nils-trustnode]

@lundman Is there any progress after all these years? At the moment there is a reboot required because of the suspended state after exporting a pool. This is not suitable for daily use or server constellations.

[lundman]

Yes and no - lots of work has gone into unmount, but it is also not perfect. I've not had any suspended-state from unmounts in a while.