Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

qemu-kvm: could not open disk image ' ': Permission denied #996

Closed
caleyg opened this issue Oct 15, 2014 · 43 comments
Closed

qemu-kvm: could not open disk image ' ': Permission denied #996

caleyg opened this issue Oct 15, 2014 · 43 comments

Comments

@caleyg
Copy link

caleyg commented Oct 15, 2014

Following the KVM documentation I stumbled across this permissions error.

Running as root on Centos 6.5

[root@dev1 veewee]# bundle exec veewee kvm build 'bamboo-agent' --pool-name=default --network-name=default --force
Building Box bamboo-agent with Definition bamboo-agent:
- debug : false
- cwd : /root/veewee
- force : true
- nogui : false
- auto : false
- checksum : false
- postinstall_include : []
- postinstall_exclude : []
- skip_to_postinstall : false
- use_emulation : false
- network_name : default
- pool_name : default

The isofile CentOS-6.5-x86_64-netinstall.iso already exists.
/usr/local/rvm/gems/ruby-1.9.3-p547@veewee/gems/fog-1.24.0/lib/fog/libvirt/requests/compute/vm_action.rb:7:in `create': Call to virDomainCreateWithFlags failed: Unable to read from monitor: Connection reset by peer (Libvirt::Error)
internal error process exited while connecting to monitor: char device redirected to /dev/pts/19
qemu-kvm: -drive file=/root/veewee/iso/CentOS-6.5-x86_64-netinstall.iso,if=none,media=cdrom,id=drive-ide0-1-0,readonly=on,format=raw: could not open disk image /root/veewee/iso/CentOS-6.5-x86_64-netinstall.iso: Permission denied
        from /usr/local/rvm/gems/ruby-1.9.3-p547@veewee/gems/fog-1.24.0/lib/fog/libvirt/requests/compute/vm_action.rb:7:in `vm_action'
        from /usr/local/rvm/gems/ruby-1.9.3-p547@veewee/gems/fog-1.24.0/lib/fog/libvirt/models/compute/server.rb:69:in `start'
        from /root/veewee/lib/veewee/provider/kvm/box/up.rb:8:in `up'
        from /root/veewee/lib/veewee/provider/core/box/build.rb:50:in `kickstart'
        from /root/veewee/lib/veewee/provider/core/box/build.rb:111:in `build'
        from /root/veewee/lib/veewee/provider/kvm/box/build.rb:6:in `build'
        from /root/veewee/lib/veewee/command/kvm.rb:25:in `build'
        from /usr/local/rvm/gems/ruby-1.9.3-p547@veewee/gems/thor-0.19.1/lib/thor/command.rb:27:in `run'
        from /usr/local/rvm/gems/ruby-1.9.3-p547@veewee/gems/thor-0.19.1/lib/thor/invocation.rb:126:in `invoke_command'
        from /usr/local/rvm/gems/ruby-1.9.3-p547@veewee/gems/thor-0.19.1/lib/thor.rb:359:in `dispatch'
        from /usr/local/rvm/gems/ruby-1.9.3-p547@veewee/gems/thor-0.19.1/lib/thor/invocation.rb:115:in `invoke'
        from /usr/local/rvm/gems/ruby-1.9.3-p547@veewee/gems/thor-0.19.1/lib/thor.rb:235:in `block in subcommand'
        from /usr/local/rvm/gems/ruby-1.9.3-p547@veewee/gems/thor-0.19.1/lib/thor/command.rb:27:in `run'
        from /usr/local/rvm/gems/ruby-1.9.3-p547@veewee/gems/thor-0.19.1/lib/thor/invocation.rb:126:in `invoke_command'
        from /usr/local/rvm/gems/ruby-1.9.3-p547@veewee/gems/thor-0.19.1/lib/thor.rb:359:in `dispatch'
        from /usr/local/rvm/gems/ruby-1.9.3-p547@veewee/gems/thor-0.19.1/lib/thor/base.rb:440:in `start'
        from /root/veewee/bin/veewee:24:in `<top (required)>'
        from /usr/local/rvm/gems/ruby-1.9.3-p547@veewee/bin/veewee:23:in `load'
        from /usr/local/rvm/gems/ruby-1.9.3-p547@veewee/bin/veewee:23:in `<main>'
        from /usr/local/rvm/gems/ruby-1.9.3-p547@veewee/bin/ruby_executable_hooks:15:in `eval'
        from /usr/local/rvm/gems/ruby-1.9.3-p547@veewee/bin/ruby_executable_hooks:15:in `<main>'
@caleyg
Copy link
Author

caleyg commented Oct 15, 2014

SOLVED: found a way.
Changing /etc/libvirt/qemu.conf to make things work.
Uncomment user/group to work as root.
Then restart libvirtd

# The user for QEMU processes run by the system instance. It can be
# specified as a user name or as a user id. The qemu driver will try to
# parse this value first as a name and then, if the name doesn't exist,
# as a user id.
#
# Since a sequence of digits is a valid user name, a leading plus sign
# can be used to ensure that a user id will not be interpreted as a user
# name.
#
# Some examples of valid values are:
#
#       user = "qemu"   # A user named "qemu"
#       user = "+0"     # Super user (uid=0)
#       user = "100"    # A user named "100" or a user with uid=100
#
user = "root"

# The group for QEMU processes run by the system instance. It can be
# specified in a similar way to user.
group = "root"

# Whether libvirt should dynamically change file ownership
# to match the configured user/group above. Defaults to 1.
# Set to 0 to disable file ownership changes.
#dynamic_ownership = 1

[root@dev1 ~]# service libvirtd restart
Stopping libvirtd daemon:                                  [  OK  ]
Starting libvirtd daemon:                                  [  OK  ]

@Germano0
Copy link

Germano0 commented Feb 17, 2017

I don't think it is wise to make such edits letting qemu running as root. You should put the ISO file in a directory that is readable by qemu user, for example /tmp

@FossPrime
Copy link

FossPrime commented May 27, 2017

The qemu user needs search permissions all the way up the path, that means chmod a+x. For me the issue was I had ACL's set which blocked qemu... to remove them run "setfacl -b offending-dir/"

You can tell if that's your problem when unix permissions are correct but you can't ls every upstream dir with sudo su -s /bin/bash qemu and ls with an account that does have access yields a result with a permission line that contains a dot such as drwxrwxr-x. meaning extended ACL permissions.

@murlock
Copy link

murlock commented Mar 24, 2018

Encountered same issue, after digging ACL issue, I have to grant supplementary right on my HOME

murlock@asus:~/kubernet $ sudo getfacl -e /home/murlock
getfacl: Removing leading '/' from absolute path names
# file: home/murlock
# owner: murlock
# group: murlock
user::rwx
user:libvirt-qemu:--x		#effective:---
group::---			#effective:---
mask::---
other::---
murlock@asus:~/kubernet $ sudo setfacl -m u:libvirt-qemu:rx /home/murlock
murlock@asus:~/kubernet $ sudo getfacl -e /home/murlock
getfacl: Removing leading '/' from absolute path names
# file: home/murlock
# owner: murlock
# group: murlock
user::rwx
user:libvirt-qemu:r-x		#effective:r-x
group::---			#effective:---
mask::r-x
other::---

@davidbeesley
Copy link

Murlock's suggestions fixed the same issue for me as well. Only difference was the user was named "qemu" instead of "libvirt-qemo"

@rimsopheap
Copy link

@murlock
Pefect! That works like a magic.

@lacksfish
Copy link

lacksfish commented Jun 20, 2018

If you copy your ISO image to /var/lib/libvirt/images (and access it from there) all the problems will go away.

@nabeelpredmac
Copy link

@murlock
thanks ... Magic indeed ...

@mvitale1989
Copy link

mvitale1989 commented Oct 19, 2018

If none above worked for you (not even having qemu run as root:root; crazy uh?), here's another workaround that did the trick for me: instead of using the libvirt volume provider for your disks, in your machine XMLs, use plain files instead.

E.g., instead of this: <disk type='volume' device='disk'> ... </disk>
Use this: <disk type='file' device='disk'> .... </disk>

Edit: turns out it was AppArmour, denying accessing the volume through the libvirt pool interface as stated here #996 (comment)

@jvleminc
Copy link

@lacksfish's solution seems to be the easiest one. Worked for me.

@loxK
Copy link

loxK commented Feb 14, 2019

I have my images in a drive mounted in /media/user/Data sudo setfacl -m u:libvirt-qemu:rx /media/user fixed it.

@AcidGo
Copy link

AcidGo commented Apr 14, 2019

@murlock
NICE. Thanks.

@DMW007
Copy link

DMW007 commented Jun 1, 2019

Using Terraform with the libvirt Provider, I got the same error. But the problem was Apparmour: dmacvicar/terraform-provider-libvirt#97

For testing purpose, I simply set security_driver = "none" in /etc/libvirt/qemu.conf and restart the service service libvirtd restart.

@wwwlookformetop
Copy link

If you are using libvirt backup disks on openstack, you need to modify the permissions of users such as nova, glance, and cinder to the backup directory
Because openstack USES these users to create backup directories

@wwwlookformetop
Copy link

then,reboot

@dperygin
Copy link

Using Terraform with the libvirt Provider, I got the same error. But the problem was Apparmour: dmacvicar/terraform-provider-libvirt#97

For testing purpose, I simply set security_driver = "none" in /etc/libvirt/qemu.conf and restart the service service libvirtd restart.

This is what worked for me. Thanks.

@mrcloudhacker
Copy link

Simply add your username to libvirt group and then instead of root replace with your username and group in /etc/libvirt/qemu.conf file.

user = "howdy"
group = "rowdy"

No need to mess with acl or run it as root. After making necessary changes simply execute systemctl restart libvirtd and ta-da

@sevillaarvin
Copy link

I don't think it is wise to make such edits letting qemu running as root. You should put the ISO file in a directory that is readable by qemu user, for example /tmp

I followed this and did something like:

cd $(mktemp -d)
cp ~/my-installer.iso .
chmod 777 .
sudo virt-install ... --cdrom my-installer.iso

This is to bypass all the ACLs. Please tell me if there's something wrong with this.

@dzintars
Copy link

dzintars commented Nov 18, 2019

Tnx @codingenesis . Exactly what i needed! :)

sudo groupadd --system libvirt
sudo usermod -a -G libvirt $(whoami)
newgrp libvirt
id $(whoami)
sudo vim /etc/libvirt/libvirtd.conf
unix_sock_group = "libvirt"
unix_sock_rw_perms = "0770"
sudo vim /etc/libvirt/qemu.conf
user = "howdy"
group = "rowdy"
sudo systemctl restart libvirtd.service

@vap0rtranz
Copy link

After trying almost all of the above, a very simple "chmod all" fix appeared when someone over at StackExchange explain what kind of perms Qemu needs.

sudo chmod a+x [path_to_volume_pool]

Explanation given:

Qemu requires execution permission along the image file's path

@boostmachines
Copy link

After trying almost all of the above, a very simple "chmod all" fix appeared when someone over at StackExchange explain what kind of perms Qemu needs.

sudo chmod a+x [path_to_volume_pool]

Explanation given:

Qemu requires execution permission along the image file's path

Exactly. If admins are storing their ISOs in nontraditional directories, this should be implemented.

@peterwillcn
Copy link

Using Terraform with the libvirt Provider, I got the same error. But the problem was Apparmour: dmacvicar/terraform-provider-libvirt#97

For testing purpose, I simply set security_driver = "none" in /etc/libvirt/qemu.conf and restart the service service libvirtd restart.

Thx.

@AnonA2
Copy link

AnonA2 commented May 2, 2020

it just may be permissions + chmod+x and permissions of folder and home. It helped me to boot os x vm. Now trying to realise GPU passthrough...

williamcroberts pushed a commit to williamcroberts/selinux that referenced this issue May 14, 2020
ERROR    internal error: qemu unexpectedly closed the monitor: 2020-05-14T17:31:41.184108Z qemu-system-x86_64: -drive file=/home/travis/build/williamcroberts/selinux/Fedora-Cloud-Base-32-1.6.x86_64.raw,format=raw,if=none,id=drive-ide0-0-0: Could not open '/home/travis/build/williamcroberts/selinux/Fedora-Cloud-Base-32-1.6.x86_64.raw': Permission denied
Domain installation does not appear to have been successful.
If it was, you can restart your domain by running:
  virsh --connect qemu:///system start demo
otherwise, please restart your installation.

I found this:
  - jedi4ever/veewee#996

Maybe we don't need these sudo's

Signed-off-by: William Roberts <william.c.roberts@intel.com>
@ankitg12
Copy link

ankitg12 commented Feb 4, 2021

After trying almost all of the above, a very simple "chmod all" fix appeared when someone over at StackExchange explain what kind of perms Qemu needs.

sudo chmod a+x [path_to_volume_pool]

Explanation given:

Qemu requires execution permission along the image file's path

This was such an effing thing !!!!

@TeaCult
Copy link

TeaCult commented Jul 6, 2021

@vap0rtranz
In my case file was /media/username/4TBBackup/testMech.raw
setfacl or security_driver = "none" did not resolve it but

sudo chmod a+x /media
sudo chmod a+x /media/username
sudo chmod a+x /media/username/4TBBackup
sudo chmod a+x /media/username/4TBBackup/testMech.raw 

Indeed solved it
libvirtd needs execution permissions along the path

But ... Why the hell it needs it ?

@tiziano-pedrazzoli
Copy link

none of the solutions you proposed worked.
command:

virsh qemu-monitor-command --pretty ubuntu20.04 '
{"execute": "transaction",
   "arguments": {
     "actions": [
       {"type": "block-dirty-bitmap-add",
        "data": {"node": "libvirt-1-format", "name": "bitmap0"}},
       {"type": "drive-backup",
        "data": {"device": "libvirt-1-format", "target": "/home/user/vm/backup.qcow2",
                 "sync": "full", "format": "qcow2"}}
     ]
   }
} '

OUTPUT

{
   "id": "libvirt-14",
   "error": {
     "class": "GenericError",
     "desc": "Could not create file: No such file or directory"
   }
}

I modified /etc/libvirt/qemu.conf by uncommenting user / group.
i chmod on the folder putting the permissions to 777 and a + x.
i disabled apparmour in /etc/libvirt/qemu.conf with security_driver = "none".
i also tried sudo setfacl -m u: libvirt-qemu: rx / home / user / vm.
I don't know what to do anymore

@dperygin
Copy link

dperygin commented Jul 30, 2021 via email

@DomPro11
Copy link

@tiziano-pedrazzoli
I did everything as well, but also I decided to change the group and ownership of the file with
chgrp username /path/to/file/backup.qcow2
chown username /path/to/file/backup.qcow2
in addition to modifying /etc/libvirt/qemu.conf to use my username and group. After trying this last resort, it ended up working for me. I hope this is helpful to anyone that has at least gotten this far when nothing else worked.

@dperygin
Copy link

Sorry. I am no longer maintaining this. I used it a couple of years ago and don't need it anymore.

@juanbomfim22
Copy link

juanbomfim22 commented Nov 17, 2021

In the case I run the VM through the terminal (in my case from OSX-KVM). I do sudo ./Open-Boot.sh and as the docs says: run

sudo setfacl -R -m u:libvirt-qemu:rx /home/$USER
sudo setfacl -R -m u:libvirt-qemu:rx /home/$USER/OSX-KVM

@network-shark
Copy link

Keep it simple

sudo usermod -a -G disk $(whoami)

working on debian11

@DauntlessRony
Copy link

DauntlessRony commented Oct 26, 2022

Error starting domain: internal error: process exited while connecting to monitor: 2022-10-26T09:15:09.633143Z qemu-kvm: -blockdev {"driver":"file","filename":"/home/rony/data/soft/vm/kvm/win10.qcow2","node-name":"libvirt-2-storage","auto-read-only":true,"discard":"unmap"}: Could not open '/home/rony/data/soft/vm/kvm/win10.qcow2': Permission denied

1.

Comment out or assign root user in /etc/libvirt/qemu.conf file.

sudo vim /etc/libvirt/qemu.conf

###################################################################
#user = "root"
user = "root" #this is the user

# The group for QEMU processes run by the system instance. It can be
# specified in a similar way to user.
#group = "root"
group = "root" #this is the group
###################################################################

2.
Now, Restart the libvirtd service.

[user@dell ~]$ sudo systemctl restart libvirtd
[user@dell ~]$ sudo systemctl status libvirtd

@c2b2pss
Copy link

c2b2pss commented Apr 28, 2023

@murlock Thanks! Solution worked. Effective, simple.

@Toasterson
Copy link

Using Terraform with the libvirt Provider, I got the same error. But the problem was Apparmour: dmacvicar/terraform-provider-libvirt#97

For testing purpose, I simply set security_driver = "none" in /etc/libvirt/qemu.conf and restart the service service libvirtd restart.

If anybody is still looking to fix this and it's neither acl, perms or apparmor, Ubuntu 22.04 now has selinux. So disabling security_driver here is mandatory

@rdonnelly2
Copy link

#996 (comment)

mrcloudhacker had the best an easiest solution.

@ThaysonScript
Copy link

how do I resolve this?

virsh attach-disk debian125 /home/debian/kvm/disks_kvm/disk1.qcow2 --target vdb --persistent
error: Failed to attach disk
error: internal error: unable to execute QEMU command 'blockdev-add': Could not open '/home/debian/kvm/disks_kvm/disk1.qcow2': Permission denied

@Toasterson
Copy link

Hi, Check my comments above for the rough instructions. 3 things can go wrong, User and group filesystem permissions (libvirt and KVM runs as it's own user) Apparmor or SELunix permissions. Some ACL. Since you are placing the disk inside your home, it's probably security. Try placing it in the same directory as the first disk or /var/lib/libvirt/images alternatively. If that gives a permission error check you pool config in libvirt and place the disk in the correct location.

@ThaysonScript
Copy link

Hi, Check my comments above for the rough instructions. 3 things can go wrong, User and group filesystem permissions (libvirt and KVM runs as it's own user) Apparmor or SELunix permissions. Some ACL. Since you are placing the disk inside your home, it's probably security. Try placing it in the same directory as the first disk or /var/lib/libvirt/images alternatively. If that gives a permission error check you pool config in libvirt and place the disk in the correct location.

Thanks friend, I managed to solve it by changing the user and group to root in the /etc/libvirt/qemu.conf file and also changing the permissions of the folder where the disks are with chmod a+x

@ThaysonScript
Copy link

Hi, Check my comments above for the rough instructions. 3 things can go wrong, User and group filesystem permissions (libvirt and KVM runs as it's own user) Apparmor or SELunix permissions. Some ACL. Since you are placing the disk inside your home, it's probably security. Try placing it in the same directory as the first disk or /var/lib/libvirt/images alternatively. If that gives a permission error check you pool config in libvirt and place the disk in the correct location.

Thanks friend, I managed to solve it by changing the user and group to root in the /etc/libvirt/qemu.conf file and also changing the permissions of the folder where the disks are with chmod a+x

I also believe that I managed to solve why I changed the security_driver to none, at first it gave an error to manage the vm with libvirt, but I commented it again and finally everything worked fine

@ShayBox
Copy link

ShayBox commented Apr 10, 2024

I'm using Archlinux and it appears libvirt run as root which is why /var/lib/libvirt/images defaults to root:root 0644 but qemu runs as libvirt-qemu and requires o+x.

You can either sudo chmod o+x -R /var/lib/libvirt/images to allow other (non-root) users the additional execute permission, or you can sudo chown libvirt-qemu:root -R /var/lib/libvirt/images and narrow permissions to sudo chmod 0700 -R /var/lib/libvirt/images so only libvirt-qemu and root can access the files.

You could also just use ACL to allow libvirt-qemu access without changing user/group/permissions, but I prefer this.

@Doom4535
Copy link

@vap0rtranz In my case file was /media/username/4TBBackup/testMech.raw setfacl or security_driver = "none" did not resolve it but

sudo chmod a+x /media
sudo chmod a+x /media/username
sudo chmod a+x /media/username/4TBBackup
sudo chmod a+x /media/username/4TBBackup/testMech.raw 

Indeed solved it libvirtd needs execution permissions along the path

But ... Why the hell it needs it ?

This worked for me as well; but seriously, why is this needed?

@Toasterson
Copy link

A bit counterintuitively, but execute rights and not read rights are needed for directory traversal in POSIX based systems. It's been a thing so long it's almost impossible to change now.

@31702160136
Copy link

使用 Terraform 和 libvirt Provider 时,我遇到了同样的错误。但问题出在 Apparmour:dmacvicar/terraform-provider-libvirt#97

为了测试目的,我只需设置security_driver = "none"/etc/libvirt/qemu.conf重新启动服务service libvirtd restart

谢谢,这对我很有用

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests