Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

dnsmasq: libvirtd cannot activate virtual network: PATH environment variable not set #6121

Open
5 of 6 tasks
marek22k opened this issue Dec 10, 2023 · 11 comments
Open
5 of 6 tasks
Labels
bug Something isn't working networking Issues related to networking features (--net=, etc)

Comments

@marek22k
Copy link
Contributor

marek22k commented Dec 10, 2023

Description

I cannot activate the virtual network when firejail is activated.

Steps to reproduce the behavior

  1. Install Arch Linux
  2. Install qemu/kvm
  3. Run sudo virsh net-start default

Expected behavior

The network starts.

Actual behavior

The network does not start.

Behavior without a profile

Since dnsmasq is called by libvirt, it is difficult to do this manually. However, running it after firecfg --clean works.

Additional context

$ sudo virsh net-start default
error: Failed to start network default
error: internal error: Child process (VIR_BRIDGE_NAME=virbr0 /usr/local/bin/dnsmasq --conf-file=/var/lib/libvirt/dnsmasq/default.conf --leasefile-ro --dhcp-script=/usr/lib/libvirt/libvirt_leaseshelper) unexpected exit status 1: Error: PATH environment variable not set


$ sudo firecfg --clean
Removing all firejail symlinks:
   alacarte removed
   atril removed
   atril-previewer removed
   atril-thumbnailer removed
   calibre removed
   chromium removed
   claws-mail removed
   codium removed
   com.github.tchx84.Flatseal removed
   conplay removed
   cvlc removed
   dig removed
   display removed
   dnsmasq removed
   drill removed
   ebook-convert removed
   ebook-edit removed
   ebook-meta removed
   ebook-polish removed
   ebook-viewer removed
   enchant-2 removed
   enchant-lsmod-2 removed
   ffplay removed
   ffprobe removed
   filezilla removed
   ftp removed
   gajim removed
   gapplication removed
   geany removed
   gimp removed
   gimp-2.10 removed
   gpa removed
   hexchat removed
   host removed
   img2txt removed
   inkscape removed
   inkview removed
   keepassxc removed
   keepassxc-cli removed
   keepassxc-proxy removed
   libreoffice removed
   librewolf removed
   lobase removed
   localc removed
   lodraw removed
   loffice removed
   lofromtemplate removed
   loimpress removed
   lomath removed
   loweb removed
   lowriter removed
   man removed
   mate-color-select removed
   meld removed
   mousepad removed
   mpg123 removed
   mpg123-id3dump removed
   mpg123-strip removed
   mpv removed
   nslookup removed
   out123 removed
   parole removed
   patch removed
   pdftotext removed
   ping removed
   pluma removed
   qt-faststart removed
   qtox removed
   ristretto removed
   seahorse removed
   secret-tool removed
   smplayer removed
   soffice removed
   sqlitebrowser removed
   ssh removed
   strings removed
   telnet removed
   tshark removed
   unbound removed
   vlc removed
   vscodium removed
   wget removed
   whois removed
   wireshark removed
   xfburn removed
   xfce4-dict removed
   xfce4-notes removed
   xfce4-screenshooter removed
   yt-dlp removed
   zeal removed

$ sudo virsh net-start default
Network default started

Already reported several times, but apparently not yet resolved:

Workaround:

sudo sed -i 's/^dnsmasq/# dnsmasq/' /etc/firejail/firecfg.config

Environment

  • Linux distribution and version: Arch Linux x86_64 6.1.66-1-lts
  • Firejail version (firejail --version).
$firejail --version
firejail version 0.9.72

Compile time support:
	- always force nonewprivs support is disabled
	- AppArmor support is enabled
	- AppImage support is enabled
	- chroot support is enabled
	- D-BUS proxy support is enabled
	- file transfer support is enabled
	- firetunnel support is disabled
	- IDS support is disabled
	- networking support is enabled
	- output logging is enabled
	- overlayfs support is disabled
	- private-home support is enabled
	- private-cache and tmpfs as user enabled
	- SELinux support is disabled
	- user namespace support is enabled
	- X11 sandboxing support is enabled

Checklist

  • The issues is caused by firejail (i.e. running the program by path (e.g. /usr/bin/vlc) "fixes" it).
  • I can reproduce the issue without custom modifications (e.g. globals.local).
  • The program has a profile. (If not, request one in https://github.com/netblue30/firejail/issues/1139)
  • The profile (and redirect profile if exists) hasn't already been fixed upstream.
  • I have performed a short search for similar issues (to avoid opening a duplicate).
    • I'm aware of browser-allow-drm yes/browser-disable-u2f no in firejail.config to allow DRM/U2F in browsers.

Log

Output of LC_ALL=C firejail /path/to/program

$sudo LC_ALL=C firejail /usr/bin/virsh net-start default
Reading profile /etc/firejail/server.profile
Reading profile /etc/firejail/disable-common.inc
Reading profile /etc/firejail/disable-programs.inc
Reading profile /etc/firejail/disable-write-mnt.inc
Reading profile /etc/firejail/disable-xdg.inc

** Note: you can use --noprofile to disable server.profile **

Parent pid 9679, child pid 9680
The new log directory is /proc/9680/root/var/log
Warning: An abstract unix socket for session D-BUS might still be available. Use --net or remove unix from --protocol set.
Child process initialized in 25.35 ms
Error registering authentication agent: GDBus.Error:org.freedesktop.PolicyKit1.Error.Failed: Cannot determine user of subject (polkit-error-quark, 0)
error: Failed to start network default
error: internal error: Child process (VIR_BRIDGE_NAME=virbr0 /usr/local/bin/dnsmasq --conf-file=/var/lib/libvirt/dnsmasq/default.conf --leasefile-ro --dhcp-script=/usr/lib/libvirt/libvirt_leaseshelper) unexpected exit status 1: Error: PATH environment variable not set



Parent is shutting down, bye...

Output of LC_ALL=C firejail --debug /path/to/program

https://gist.github.com/marek22k/53c067d5a7e23121984dd8b6b74ebb5a

@glitsj16
Copy link
Collaborator

glitsj16 commented Dec 10, 2023

sudo sed -i 's/^dnsmasq/# dnsmasq/' /etc/firejail/firecfg.config

The upcoming Firejail release will have override support for firecfg.config:

.SH FILES
.PP
Configuration files are searched for and parsed in the following paths:
.PP
.RS
1. /etc/firejail/firecfg.d/*.conf (in alphabetical order)
.br
2. /etc/firejail/firecfg.config
.RE
.PP
The programs that are supported by default are listed in
/etc/firejail/firecfg.config.
It is recommended to leave it as is and put all customizations inside
/etc/firejail/firecfg.d/.
.PP
Profile files are also searched in the user configuration directory:
.PP
.RS
3. ~/.config/firejail/*.profile
.RE
.PP
For every \fBPROGRAM.profile\fR file found, firecfg attempts to create a
symlink for "PROGRAM", as if "PROGRAM" was listed in a configuration file.
.SH SYNTAX
Configuration file syntax:
.PP
A line that starts with \fB#\fR is considered a comment.
.br
A line that starts with \fB!PROGRAM\fR means to ignore "PROGRAM" when creating
symlinks.
.br
A line that starts with anything else is considered to be the name of an
executable and firecfg will attempt to create a symlink for it.
.PP
For example, to prevent firecfg from creating symlinks for "firefox" and
"patch" while attempting to create a symlink for "myprog", the following lines
could be added to /etc/firejail/firecfg.d/10-my.conf:
.PP
.RS
!firefox
.br
!patch
.br
.br
myprog

If you use firejail-git ftom the AUR you can have that functionality now. Dropping a file like the below will disable dnsmasq sandboxing persistently:

$ cat /etc/firejail/firecfg.d/10-disabled.conf
!dnsmasq

@ShellCode33
Copy link

@glitsj16 ideally I'd like to keep using the dnsmasq profile. While this can be a temporary workaround, it does not solve the underlying issue

@glitsj16
Copy link
Collaborator

@ShellCode33 Agreed, the underlying issue is still not very clear (to me).

I've zero experience with libvirt/dnsmasq. Going over the referenced issue threads, I did notice #5089 (comment) mentions caps.keep chown,dac_override,net_admin,net_bind_service,net_raw,setgid,setuid might be needed (besides whitelist /var/lib/libvirt/dnsmasq and whitelist /var/run). OP's response seems to suggest that fixes things, but the resulting commits ce6f792 and f3de2e3 don't touch caps.keep.

Have you tried using dnsmasq.profile with caps.keep chown,dac_override,net_admin,net_bind_service,net_raw,setgid,setuid yet?

@glitsj16
Copy link
Collaborator

Follow-up

I've installed libvirt/dnsmasq on my Arch Linux box to get a better understanding of this issue. With the below it works here, without the firecfg workaround:

$ cat ~/.config/firejail/dnsmasq.local
# Firejail profile for dnsmasq
# Persistent local customizations

allusers
caps.keep chown,net_admin,net_bind_service,net_raw,setgid,setuid
ignore caps.keep

Caveats:

@marek22k @ShellCode33 Can you try again with the above dnsmasq.local and report back here please? Hopefully we're closer to fixing this properly...

@ShellCode33
Copy link

ShellCode33 commented Dec 11, 2023

Thanks for taking the time to look at it @glitsj16 !

I still have the same PATH-related error:

$ sudo virsh net-start default
error: Failed to start network default
error: internal error: Child process (VIR_BRIDGE_NAME=virbr0 /usr/local/bin/dnsmasq --conf-file=/var/lib/libvirt/dnsmasq/default.conf --leasefile-ro --dhcp-script=/usr/lib/libvirt/libvirt_leaseshelper) unexpected exit status 1: Error: PATH environment variable not set

You said that for you virsh was just hanging, that sounds odd. Make sure the libvirtd service is running

Here's my /etc/firejail/dnsmasq.local :

noblacklist /run/libvirt
noblacklist /usr/lib/libvirt
noblacklist /usr/local/bin/dnsmasq
noblacklist /usr/bin/dnsmasq
noblacklist /usr/bin/libvirtd

whitelist /usr/lib/libvirt
whitelist /run/libvirt
whitelist /usr/local/bin/dnsmasq
whitelist /usr/bin/dnsmasq
whitelist /usr/bin/libvirtd

noblacklist /usr/lib
noblacklist /usr/bin
noblacklist /usr/local/bin/
noblacklist /run

allusers
caps.keep chown,net_admin,net_bind_service,net_raw,setgid,setuid
ignore caps.keep

(I'm deliberately trying to be very permissive to narrow it down after, but that still doesn't work)

@marek22k
Copy link
Contributor Author

$ sudo virsh net-start default
error: Failed to start network default
error: internal error: Child process (VIR_BRIDGE_NAME=virbr0 /usr/local/bin/dnsmasq --conf-file=/var/lib/libvirt/dnsmasq/default.conf --leasefile-ro --dhcp-script=/usr/lib/libvirt/libvirt_leaseshelper) unexpected exit status 1: Error: PATH environment variable not set


$ cat /etc/firejail/dnsmasq.local
allusers
caps.keep chown,net_admin,net_bind_service,net_raw,setgid,setuid
ignore caps.keep

@glitsj16
Copy link
Collaborator

You said that for you virsh was just hanging, that sounds odd. Make sure the libvirtd service is running

@ShellCode33
I did start libvirtd.service and virtlogd.service via systemd. Nothing special here. If you don't start those you'll indeed see errors:

$ sudo virsh net-start default
error: failed to connect to the hypervisor
error: Operation not supported: Cannot use direct socket mode if no URI is set

@ShellCode33 @marek22k
Did you re-enable the dnsmasq symlink in /usr/local/bin (via firecfg or manually)? To make absolutely sure I created a wrapper script:

$ cat /usr/local/bin/dnsmasq
#!/bin/sh
#
## wrapper for dnsmasq
#+ sandbox support via firejail

### vars
_app="dnsmasq"
_bin="/usr/bin/${_app}"

# sandboxing
_bin="firejail --name=${_app}-6121 --quiet ${_bin}"


### logic
${_bin} "$@"

exit 0

Mind the --name=${_app}-6121 part. It's another assisting param to double-check if sandboxing dnsmasq is or isn't working. After issueing the virsh command you can run:

$ firejail --list | grep dnsmasq
11943:root:dnsmasq:firejail --name=dnsmasq-6121 --quiet /usr/bin/dnsmasq --conf-file=/var/lib/libvirt/dnsmasq/default.conf --leasefile-ro --dhcp-script=/usr/lib/libvirt/libvirt_leaseshelper

You should see the same if you added name dnsmasq-6121 (or anything you choose really) to dnsmasq.local.

Did any of you both added his user to the libvirt group? Any polkit stuff we're missing eyes on in this context? Check these docs for details:
https://wiki.archlinux.org/title/Libvirt#Using_libvirt_group
https://wiki.archlinux.org/title/Libvirt#Using_polkit
https://wiki.archlinux.org/title/Polkit#Bypass_password_prompt
https://wiki.archlinux.org/title/Polkit#Globally
https://wiki.archlinux.org/title/Polkit#For_specific_actions

@ShellCode33
Copy link

ShellCode33 commented Dec 11, 2023

Did you re-enable the dnsmasq symlink in /usr/local/bin (via firecfg or manually)?

Yes it is currently enabled

Did any of you both added his user to the libvirt group?

Yes my user is part of this group, but I guess it doesn't matter considering we are running virsh using sudo, therefore polkit shouldn't be at play here


I tried to put your script in place of the /usr/local/bin/dnsmasq symlink, now virsh runs fine without error (at least it confirms this is not a PATH issue).

But now I have an apparmor denial 🥲

image

So I tried to disable this particular AppArmor profile, and now I observe the same behavior as you: virsh hangs.

While it's still hanging, I can see it's running within firejail:

 firejail --list | grep dnsmasq
63207:root:dnsmasq-6121:firejail --name=dnsmasq-6121 --quiet /usr/bin/dnsmasq --conf-file=/var/lib/libvirt/dnsmasq/default.conf --leasefile-ro --dhcp-script=/usr/lib/libvirt/libvirt_leaseshelper

EDIT: even if I CTRL+C virsh, dnsmasq is still running, but it's a bit annoying because all virsh commands hang, even sudo virsh net-list

EDIT2: my dnsmasq.locale is completely empty and it's still working. I'm starting to wonder if this is a bug in libvirt which does not resolve the symlink properly

EDIT3:

CTRL+C doesn't work, the libvirtd daemon will error after some time.

And when it's automatically restarted by systemd, the following errors/warning are emitted:

systemd logs
Dec 11 23:13:15 laptop libvirtd[66091]: End of file while reading data: Input/output error
Dec 11 23:15:45 laptop libvirtd[66091]: Make forcefull daemon shutdown
Dec 11 23:15:45 laptop systemd[1]: libvirtd.service: Main process exited, code=exited, status=1/FAILURE
░░ Subject: Unit process exited
░░ Defined-By: systemd
░░ Support: https://lists.freedesktop.org/mailman/listinfo/systemd-devel
░░
░░ An ExecStart= process belonging to unit libvirtd.service has exited.
░░
░░ The process' exit code is 'exited' and its exit status is 1.
Dec 11 23:15:45 laptop systemd[1]: libvirtd.service: Failed with result 'exit-code'.
░░ Subject: Unit failed
░░ Defined-By: systemd
░░ Support: https://lists.freedesktop.org/mailman/listinfo/systemd-devel
░░
░░ The unit libvirtd.service has entered the 'failed' state with result 'exit-code'.
Dec 11 23:15:45 laptop systemd[1]: libvirtd.service: Unit process 66475 (dnsmasq) remains running after unit stopped.
Dec 11 23:15:45 laptop systemd[1]: libvirtd.service: Unit process 66476 (firejail) remains running after unit stopped.
Dec 11 23:15:45 laptop systemd[1]: libvirtd.service: Unit process 66477 (firejail) remains running after unit stopped.
Dec 11 23:15:45 laptop systemd[1]: libvirtd.service: Unit process 66490 (dnsmasq) remains running after unit stopped.
Dec 11 23:15:45 laptop systemd[1]: libvirtd.service: Unit process 66491 (dnsmasq) remains running after unit stopped.
Dec 11 23:15:45 laptop systemd[1]: libvirtd.service: Consumed 1.201s CPU time, 12.4M memory peak, 0B memory swap peak.
░░ Subject: Resources consumed by unit runtime
░░ Defined-By: systemd
░░ Support: https://lists.freedesktop.org/mailman/listinfo/systemd-devel
░░
░░ The unit libvirtd.service completed and consumed the indicated resources.
Dec 11 23:15:45 laptop systemd[1]: libvirtd.service: Scheduled restart job, restart counter is at 1.
░░ Subject: Automatic restarting of a unit has been scheduled
░░ Defined-By: systemd
░░ Support: https://lists.freedesktop.org/mailman/listinfo/systemd-devel
░░
░░ Automatic restarting of the unit libvirtd.service has been scheduled, as the result for
░░ the configured Restart= setting for the unit.
Dec 11 23:15:45 laptop systemd[1]: libvirtd.service: Found left-over process 66475 (dnsmasq) in control group while starting unit. Ignoring.
Dec 11 23:15:45 laptop systemd[1]: libvirtd.service: This usually indicates unclean termination of a previous run, or service implementation deficiencies.
Dec 11 23:15:45 laptop systemd[1]: libvirtd.service: Found left-over process 66476 (firejail) in control group while starting unit. Ignoring.
Dec 11 23:15:45 laptop systemd[1]: libvirtd.service: This usually indicates unclean termination of a previous run, or service implementation deficiencies.
Dec 11 23:15:45 laptop systemd[1]: libvirtd.service: Found left-over process 66477 (firejail) in control group while starting unit. Ignoring.
Dec 11 23:15:45 laptop systemd[1]: libvirtd.service: This usually indicates unclean termination of a previous run, or service implementation deficiencies.
Dec 11 23:15:45 laptop systemd[1]: libvirtd.service: Found left-over process 66490 (dnsmasq) in control group while starting unit. Ignoring.
Dec 11 23:15:45 laptop systemd[1]: libvirtd.service: This usually indicates unclean termination of a previous run, or service implementation deficiencies.
Dec 11 23:15:45 laptop systemd[1]: libvirtd.service: Found left-over process 66491 (dnsmasq) in control group while starting unit. Ignoring.
Dec 11 23:15:45 laptop systemd[1]: libvirtd.service: This usually indicates unclean termination of a previous run, or service implementation deficiencies.
Dec 11 23:15:45 laptop systemd[1]: Starting libvirt legacy monolithic daemon...
░░ Subject: A start job for unit libvirtd.service has begun execution
░░ Defined-By: systemd
░░ Support: https://lists.freedesktop.org/mailman/listinfo/systemd-devel
░░
░░ A start job for unit libvirtd.service has begun execution.
░░
░░ The job identifier is 4850.
Dec 11 23:15:45 laptop systemd[1]: Started libvirt legacy monolithic daemon.
░░ Subject: A start job for unit libvirtd.service has finished successfully
░░ Defined-By: systemd
░░ Support: https://lists.freedesktop.org/mailman/listinfo/systemd-devel
░░
░░ A start job for unit libvirtd.service has finished successfully.
░░
░░ The job identifier is 4850.
Dec 11 23:17:45 laptop systemd[1]: libvirtd.service: Deactivated successfully.
░░ Subject: Unit succeeded
░░ Defined-By: systemd
░░ Support: https://lists.freedesktop.org/mailman/listinfo/systemd-devel
░░
░░ The unit libvirtd.service has successfully entered the 'dead' state.
Dec 11 23:17:45 laptop systemd[1]: libvirtd.service: Unit process 66475 (dnsmasq) remains running after unit stopped.
Dec 11 23:17:45 laptop systemd[1]: libvirtd.service: Unit process 66476 (firejail) remains running after unit stopped.
Dec 11 23:17:45 laptop systemd[1]: libvirtd.service: Unit process 66477 (firejail) remains running after unit stopped.
Dec 11 23:17:45 laptop systemd[1]: libvirtd.service: Unit process 66490 (dnsmasq) remains running after unit stopped.
Dec 11 23:17:45 laptop systemd[1]: libvirtd.service: Unit process 66491 (dnsmasq) remains running after unit stopped.

@kmk3 kmk3 added the networking Issues related to networking features (--net=, etc) label Sep 1, 2024
@kmk3 kmk3 changed the title Virtual network fails with firejail enabled for dnsmasq dnsmasq: cannot activate virtual network Sep 13, 2024
@kmk3 kmk3 changed the title dnsmasq: cannot activate virtual network dnsmasq: cannot activate virtual network: PATH environment variable not set Sep 13, 2024
@kmk3 kmk3 changed the title dnsmasq: cannot activate virtual network: PATH environment variable not set dnsmasq: libvirtd cannot activate virtual network: PATH environment variable not set Sep 13, 2024
@rieje
Copy link

rieje commented Oct 29, 2024

FWIW I'm having the same issue on Arch. I don't use AppArmor.

@Utini2000
Copy link

Same issue here on Arch. Disabled dnsmasq profile in firejail. Otherwise it won't function.

kmk3 added a commit to kmk3/firejail that referenced this issue Nov 7, 2024
There are multiple reports in netblue30#6121 that dnsmasq does not work when
called by libvirt:

    $ sudo virsh net-start default
    error: Failed to start network default
    error: internal error: Child process (VIR_BRIDGE_NAME=virbr0 /usr/local/bin/dnsmasq [...]) unexpected exit status 1: Error: PATH environment variable not set

Also, note that this is a server program, so it might be better to
disable it by default anyway.

Reported-by: @marek22k
kmk3 added a commit that referenced this issue Nov 8, 2024
There are multiple reports in #6121 that dnsmasq does not work when
called by libvirt:

    $ sudo virsh net-start default
    error: Failed to start network default
    error: internal error: Child process (VIR_BRIDGE_NAME=virbr0 /usr/local/bin/dnsmasq [...]) unexpected exit status 1: Error: PATH environment variable not set

Also, note that this is a server program, so it might be better to
disable it by default anyway.

Reported-by: @marek22k
@kmk3 kmk3 added the bug Something isn't working label Dec 29, 2024
@tl87

This comment has been minimized.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working networking Issues related to networking features (--net=, etc)
Projects
None yet
Development

No branches or pull requests

7 participants