Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Switch to using lun to determine which persistent disk is attached? #135

Closed
cppforlife opened this issue Apr 28, 2016 · 9 comments
Closed

Comments

@cppforlife
Copy link
Contributor

I believe in the following scenario Agent cannot keep track of persistent disk attachment within the VM:

  • deploy VM with one persistent disk (eg /dev/sdc)
  • bump the size of the persistent disk
    • which causes second persistent disk to be attached (eg /dev/sdd)
    • first persistent disk is detached
  • at this time VM still has second persistent disk attached at /dev/sdd (nothing at /dev/sdc)
  • Azure migrates VM (may be reboot or start/stop will also cause this)
  • when VM boots up second persistent disk that was at /dev/sdd is not at /dev/sdc
  • agent will not start at this point since it cannot find persistent disk that it expects
[File System] 2016/04/27 22:07:19 DEBUG - Checking if file exists /dev/sdd
[main] 2016/04/27 ERROR - App setup Running bootstrap: Mounting persistent disk: Getting real device path: Resolving mapped device path: Timed out getting real device path for /dev/sdd

Is there a better way to determine which disk is which? OpenStack for example has a way to find it based on device ID. This article https://azure.microsoft.com/en-us/documentation/articles/virtual-machines-linux-classic-attach-disk/ seems to suggest lun might be the way to go. Do we know if lun would change after the reboot?

@sedouard
Copy link

sedouard commented Apr 28, 2016

Indeed, using LUNs are the correct way to map attached disks to drives. using lsscsi. This is a related issue Azure/azure-sdk-for-go#315. From everything I've seen, Azure always attaches to device 5. And that LUN is tracked by the API.

@AbelHu
Copy link
Contributor

AbelHu commented Apr 28, 2016

I am investigating it. From my test, the lun will never change after the reboot. I need to confirm it and update you later.

@sedouard
Copy link

Also @cppforlife if you don't have a disk at LUN 0 I believe the SCSCI driver can't detect disks at higher LUNS. So when you detach the first persisted disk, is it at LUN 0?

@cppforlife
Copy link
Contributor Author

@sedouard im not sure. @AbelHu hopefully can confirm otherwise.

@sedouard
Copy link

/cc @szarkos. He should know for sure if this is still the case.

Pretty sure if disk@ LUN 0 is gone disks at higher LUN's are not found. We hit this issue on another project.

@cppforlife
Copy link
Contributor Author

@sedouard what do you mean by "disks at higher LUN's are not found". do they not show up at all anywhere or do they just get renumbered?

@cppforlife
Copy link
Contributor Author

cc @AbelHu have you had a chance to observe this behaviour. im trying to determine if marking disks with uuids would even be helpful in the above situation described by @sedouard ?

@sedouard
Copy link

sedouard commented May 3, 2016

they won't show up at all on lsscsi.

@AbelHu
Copy link
Contributor

AbelHu commented May 4, 2016

@cppforlife if lun 0 is gone, disks at higher LUN still can be found. I do not observe this issue. I am investigating to use scsi as DevicePathResolutionType to use uuid as the device id.

AbelHu added a commit that referenced this issue May 18, 2016
Use LUN and host device id as the disk identifier. Close #135
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants