mdns support for nss does not work when networkd is enabled

[poscat0x04]

Describe the bug
Remote .local domains cannot be resolved when networkd is enabled

I noticed that resolve comes before mdns_minimal in the generated nsswitch.conf:

passwd:    files systemd
group:     files systemd
shadow:    files

hosts:     files mymachines resolve [!UNAVAIL=return] mdns_minimal [NOTFOUND=return] dns mdns myhostname
networks:  files

ethers:    files
services:  files
protocols: files
rpc:       files

while archwiki recommended putting mdns_minimal before resolve: https://wiki.archlinux.org/index.php/Avahi#Hostname_resolution

not sure if this is the cause

To Reproduce
Steps to reproduce the behavior:

1. enable services.avahi.nssmdns

Expected behavior
A clear and concise description of what you expected to happen.

Screenshots
If applicable, add screenshots to help explain your problem.

Additional context
Add any other context about the problem here.

Notify maintainers

Metadata
Please run nix-shell -p nix-info --run "nix-info -m" and paste the result.

Maintainer information:

# a list of nixpkgs attributes affected by the problem
attribute:
# a list of nixos modules affected by the problem
module:

[poscat0x04]

possibly related modules:

• nsswitch
• resolved
• avahi-daemon

[doronbehar]

I noticed that resolve comes before mdns_minimal in the generated nsswitch.conf:

hosts:     files mymachines resolve [!UNAVAIL=return] mdns_minimal [NOTFOUND=return] dns mdns myhostname

While archwiki recommended putting mdns_minimal before resolve: wiki.archlinux.org/index.php/Avahi#Hostname_resolution . Not sure if this is the cause.

IIRC I experienced issues with .local name resolving a while ago with a certain printer on a certain network. I can't say I can reproduce the issue reliably though, unfortunately.

@flokli seems to have taken special care in fd21793 to put the mdns word after the dns word - opposite to what the Arch Wiki recommends:

nixpkgs/nixos/modules/services/networking/avahi-daemon.nix

Lines 241 to 244 in 3d36d46

	system.nssDatabases.hosts = optionals cfg.nssmdns (mkMerge [
	[ "mdns_minimal [NOTFOUND=return]" ]
	(mkOrder 1501 [ "mdns" ]) # 1501 to ensure it's after dns
	]);

I couldn't find any mention in upstream's docs to what the arch WiKi says.

[flokli]

@doronbehar I don't think that commit changed anything - it was merely a code move.

As for the general placement, nss-resolve(8) says:

  To activate the NSS module, add "resolve [!UNAVAIL=return]" to the line starting with "hosts:" in /etc/nsswitch.conf. Specifically, it is recommended to place "resolve" early in /etc/nsswitch.conf's "hosts:" line. It
  should be before the "files" entry, since systemd-resolved supports /etc/hosts internally, but with caching. To the contrary, it should be after "mymachines", to give hostnames given to local VMs and containers
  precedence over names received over DNS. Finally, we recommend placing "dns" somewhere after "resolve", to fall back to nss-dns if systemd-resolved.service is not available.

From my understanding, systemd-resolved should also support mDNS (also see https://wiki.archlinux.org/index.php/Systemd-resolved#mDNS), so having it early might make sense w.r.t caching.

I'm not sure if enabling both resolved and services.avahi.nssmdns is something that should work or not - I didn't really put much thought into it.

[lopsided98]

Yes, the problem is that mdns_minimal is after resolve. It used to work right until the config was split up into multiple modules in #87016, which messed up the order. mkOrder can be used to fix it.

[lopsided98]

See #99530 for a fix.

[nixos-discourse]

This issue has been mentioned on NixOS Discourse. There might be relevant details there:

https://discourse.nixos.org/t/how-to-make-systemd-resolved-and-mdns-work-together/10910/2

[wucke13]

This is still not working for me. I have the following relevant config:

services.avahi = {
  enable = true;
  nssmdns = true;
  publish = {
    enable = true;
    domain = true;
    addresses = true;
  };
};

networking = {
  useDHCP = false;
  useNetworkd = true;
  firewall.enable = lib.mkDefault false;
};

I can resolvectl query my local machine .local, but no other machine in the network:

resolvectl query othermachine.local
othermachine.local: resolve call failed: No appropriate name servers or networks for name found

It works however without .local. Why is that?

resolvectl query othermachine
othermachine: 10.0.0.13                                -- link: wlan0

-- Information acquired via protocol LLMNR/IPv4 in 115.9ms.
-- Data is authenticated: no

I'm on 21.05, so the fix mentioned above should be active?

[lopsided98]

I don't have a machine with the same config accessible right now, but I think this behavior might be expected. mDNS resolution with Avahi uses NSS, while resolvectl only goes through systemd-resolved, which doesn't do mDNS resolution with your config. It works without .local using LLMNR (basically Microsoft's version of mDNS), which is apparently enabled in your config (edit: it looks to be enabled by default). I think resolving the local machine works due to special handling with systemd-resolved.

Does ping othermachine.local work?

[flokli]

[…] I think this behavior might be expected. mDNS resolution with Avahi uses NSS, while resolvectl only goes through systemd-resolved, which doesn't do mDNS resolution with your config.

Yes, this is to be expected - avahi goes via nss, and networkd also provides a nss module.

ping othermachine[.local] will query nss modules configured in nsswitch.conf and should work.

[wucke13]

Does ping othermachine.local work?

Yes. Now I feel stupid. Thank you two for the fast input, I guess that settles it. Only one question left, is there an easy way to make it also work with resolvectl ?

[lopsided98]

@wucke13 You can use systemd-resolved's mDNS resolver instead of Avahi. Remove all the Avahi related config and add systemd.network.networks.<name>.networkConfig.MulticastDNS = true (or "resolve" if you want to only resolve and not advertise).

[Valodim]

For those still searching, this is what worked for me using NetworkManager:

# use resolved for hostname resolution
services.resolved.enable = true;

# enable mdns resolution for resolved on all connections
# see https://man.archlinux.org/man/NetworkManager.conf.5#CONNECTION_SECTION
networking.networkmanager.connectionConfig."connection.mdns" = 2;

# still required (at least mdns resolution didn't work for me without)
services.avahi.enable = true;
# nssmdns is NOT needed, since resolved will do the resolution, not the avahi nss module
# services.avahi.nssmdns = true;