nixos/nscd: start in early boot #154928

Mic92 · 2022-01-13T20:00:20Z

Services that have dynamic users require nscd to resolve users
via pam_systemd. Those services might not even create
their own dynamic users itself i.e. iptables.
To make sure nscd is always started when this is happening we move
nscd to sysinit.target and make sure that it is always started before
starting/reloading/restarting any other service.

Motivation for this change

This pr was original opened here: #106336

Things done

arcnmx · 2022-01-13T20:29:23Z

Thanks, I can confirm this fixes the issue with DynamicUser services on switch!

Before:

restarting the following units: ddclient.timer, home-manager-arc.service, home-manager-root.service, polkit.service
starting the following units: accounts-daemon.service, nscd.service
warning: the following units failed: ddclient.service
...
3jifr0sab601i8l2ka07c2c2cak5csg2-ddclient-prestart[430095]: install: invalid user ‘ddclient’

After:

starting nscd
restarting the following units: ddclient.timer, home-manager-arc.service, home-manager-root.service, polkit.service
starting the following units: accounts-daemon.service

dasJ · 2022-01-13T20:35:59Z

I highly recommend stopIfChanged = false. While the default of true is already crappy in most cases, it's even worst for nscd because stc can stop nscd, restart a service, and then only start nscd afterwards, causing the service that is restarted in between to be started without a working nscd. This way we don't need to modify stc

dasJ · 2022-01-13T20:37:46Z

Source: We have been doing so in our downstream repo for ~1 year and have had no issues since then

dasJ · 2022-01-13T20:38:44Z

nixos/modules/system/activation/switch-to-configuration.pl

@@ -474,6 +476,13 @@ sub filterUnits {
 print STDERR "setting up tmpfiles\n";
 system("@systemd@/bin/systemd-tmpfiles", "--create", "--remove", "--exclude-prefix=/dev") == 0 or $res = 3;

+# We need to start nscd before any other service, since they might need


Not a fan of yet another special case

I don't see a way around it without that does not involve getting rid of nscd.

dasJ · 2022-01-13T20:49:31Z

Also it may be helpful to just fix nscd.service instead:

$ systemctl show nscd | grep WantedBy
WantedBy=nss-user-lookup.target nss-lookup.target

$ systemctl show nss-lookup.target | grep By
$ systemctl show nss-user-lookup.target | grep By

Of course stc will not start nscd if nobody wants the service…

Mic92 · 2022-01-15T06:10:31Z

I highly recommend stopIfChanged = false. While the default of true is already crappy in most cases, it's even worst for nscd because stc can stop nscd, restart a service, and then only start nscd afterwards, causing the service that is restarted in between to be started without a working nscd. This way we don't need to modify stc

This also has the big downside that nss changes no longer are applied until reboot.

Mic92 · 2022-01-15T06:13:04Z

Also it may be helpful to just fix nscd.service instead:
$ systemctl show nscd | grep WantedBy
WantedBy=nss-user-lookup.target nss-lookup.target

$ systemctl show nss-lookup.target | grep By
$ systemctl show nss-user-lookup.target | grep By
Of course stc will not start nscd if nobody wants the service…

This is a foot gun at best. Non one expects a dependency on nscd when using DynamicUser.

Mic92 · 2022-01-15T06:20:18Z

If we don't fix this I would consider DynamicUser not as fit for purpose in NixOS.

revert if NixOS#154928 or a similar fix gets applied

arcnmx · 2022-01-15T14:47:17Z

This is a foot gun at best. Non one expects a dependency on nscd when using DynamicUser.

Would it be a footgun if the module (systemd unit generator) automatically did it for you? That wouldn't work for units coming from systemd.packages, but it could probably assert it at build time and request that it be added?

Mic92 · 2022-01-16T06:21:10Z

This is a foot gun at best. Non one expects a dependency on nscd when using DynamicUser.

Would it be a footgun if the module (systemd unit generator) automatically did it for you? That wouldn't work for units coming from systemd.packages, but it could probably assert it at build time and request that it be added?

Maybe but I feel this is more code/complexity in all of nixpkgs than just having them in the activation script. Also we can no longer use some of the upstream systemd units as is.

flokli · 2022-01-19T21:05:17Z

Can we drop this in favor of #155655?

With this in, requests before nscd is up should still work.

erikarvstedt · 2022-01-20T14:42:57Z

@flokli: It's common practice to use the stable NixOS release for the system modules and run services with binaries from unstable on top of that. When the glibc minor release differs between stable and unstable, #155655 won't work for these services.
So +1 for this PR, I don't see a simpler way to achieve this.

flokli · 2022-01-20T15:41:56Z

But I'd assume we can revert this after #155655 is merged, or in a followup PR, to get closer with upstream again?

arianvp · 2022-01-20T15:43:42Z

Given that #155655 still leaves nscd enabled by default this still seems a useful change no? Or would during early boot DynamicUser still work due to the fallback built in #155655 ?

erikarvstedt · 2022-01-20T15:47:03Z

For systems that have services using different glibc minor versions, #155655 is no replacement for this PR.
For all other systems, it is.
(#155655 only provides access to NSS modules for services using the system glibc.)
But we have to support the multi-glibc use case when nscd is enabled, so this PR is still needed, even with #155655.

Mic92 · 2022-01-27T04:14:23Z

rebased.

nrdxp

Everything appears in order. @Mic92 is it ready for merge? I can pull the trigger if so.

dasJ · 2022-01-27T22:08:36Z

Still not a fan at all. Adding yet another special case to stc that could also be implemented by native systemd functionality.
#154620 would immensely improve the current situation, combined with stopIfChanged there almost no issues.

A fix that would involve only systemd logic would be to add the After=s in the systemd builder when DynamicUser is set. This way there is no additional NixOS-specific tinkering in a language most people dislike.

But do what you think is best.

nrdxp · 2022-01-27T22:18:27Z

You suggestion sounds reasonable to me.I'll leave open for a bit to give time for consideration.

Services that have dynamic users require nscd to resolve users via pam_systemd. Those services might not even create their own dynamic users itself i.e. iptables. To make sure nscd is always started when this is happening we move nscd to sysinit.target and make sure that it is always started before starting/reloading/restarting any other service.

Mic92 · 2022-04-04T02:23:18Z

Rebased.

Mic92 · 2022-04-04T02:25:20Z

Also I will check if #154620 is enough.

symphorien · 2022-06-05T14:09:06Z

Still not a fan at all. Adding yet another special case to stc that could also be implemented by native systemd functionality. #154620 would immensely improve the current situation, combined with stopIfChanged there almost no issues.

A fix that would involve only systemd logic would be to add the After=s in the systemd builder when DynamicUser is set. This way there is no additional NixOS-specific tinkering in a language most people dislike.

But do what you think is best.

this is what I proposed in #105354, but it was rejected.

tilpner · 2022-06-05T14:13:54Z

Isn't this already fixed by #154320?

symphorien · 2022-06-05T14:23:41Z

I had the issue on 2022-05-12 on a system updated daily.

flokli · 2022-10-10T05:07:06Z

@dasJ @ajs124 is this still necessary, with #186785 merged?

dasJ · 2022-10-10T07:22:32Z

I highly doubt it

Mic92 requested a review from dasJ as a code owner January 13, 2022 20:00

github-actions bot added 6.topic: nixos 8.has: module (update) labels Jan 13, 2022

Mic92 mentioned this pull request Jan 13, 2022

nixos/nscd: start in early boot #106336

Closed

10 tasks

Mic92 requested a review from arianvp January 13, 2022 20:00

Mic92 mentioned this pull request Jan 13, 2022

nixos/ddclient: make nsupdate protocol usuable #154124

Merged

13 tasks

ofborg bot added 10.rebuild-darwin: 0 10.rebuild-linux: 1-10 labels Jan 13, 2022

dasJ reviewed Jan 13, 2022

View reviewed changes

Mic92 added a commit to Mic92/nixpkgs that referenced this pull request Jan 15, 2022

nixos/ddclient: don't chown secrets until dynamicuser issue is resolved

63971d1

revert if NixOS#154928 or a similar fix gets applied

Mic92 mentioned this pull request Jan 15, 2022

nixos/ddclient: don't chown secrets until dynamicuser issue is resolved #155075

Merged

13 tasks

flokli mentioned this pull request Jan 19, 2022

provide nss modules globally instead of hacking them with nscd #55276

Closed

This was referenced Jan 19, 2022

Provide NSS modules globally, make nscd unnecessary (v2) erikarvstedt/nixpkgs#1

Closed

Provide NSS modules globally, make nscd unnecessary (v2) #155655

Closed

arianvp approved these changes Jan 26, 2022

View reviewed changes

SuperSandro2000 added the 2.status: merge conflict label Jan 26, 2022

Mic92 force-pushed the nscd-fix branch from d153274 to 12310e1 Compare January 27, 2022 04:14

Mic92 removed the 2.status: merge conflict label Jan 27, 2022

nrdxp approved these changes Jan 27, 2022

View reviewed changes

tilpner mentioned this pull request Feb 3, 2022

Invidious: startup fails with local database due to auth failures #157361

Closed

Mic92 force-pushed the nscd-fix branch from 12310e1 to 6ea1fff Compare April 4, 2022 02:23

flokli mentioned this pull request Oct 10, 2022

use sd-notify to signal readyness twosigma/nsncd#35

Merged

dasJ closed this Oct 20, 2022

Mic92 deleted the nscd-fix branch November 20, 2022 19:15

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

nixos/nscd: start in early boot #154928

nixos/nscd: start in early boot #154928

Mic92 commented Jan 13, 2022 •

edited

Loading

arcnmx commented Jan 13, 2022

dasJ commented Jan 13, 2022 •

edited

Loading

dasJ commented Jan 13, 2022

dasJ Jan 13, 2022

Mic92 Jan 15, 2022

dasJ commented Jan 13, 2022

Mic92 commented Jan 15, 2022

Mic92 commented Jan 15, 2022 •

edited

Loading

Mic92 commented Jan 15, 2022

arcnmx commented Jan 15, 2022

Mic92 commented Jan 16, 2022 •

edited

Loading

flokli commented Jan 19, 2022

erikarvstedt commented Jan 20, 2022 •

edited

Loading

flokli commented Jan 20, 2022

arianvp commented Jan 20, 2022

erikarvstedt commented Jan 20, 2022 •

edited

Loading

Mic92 commented Jan 27, 2022

nrdxp left a comment

dasJ commented Jan 27, 2022

nrdxp commented Jan 27, 2022

Mic92 commented Apr 4, 2022

Mic92 commented Apr 4, 2022

symphorien commented Jun 5, 2022

tilpner commented Jun 5, 2022

symphorien commented Jun 5, 2022

flokli commented Oct 10, 2022

dasJ commented Oct 10, 2022

nixos/nscd: start in early boot #154928

nixos/nscd: start in early boot #154928

Conversation

Mic92 commented Jan 13, 2022 • edited Loading

Motivation for this change

Things done

arcnmx commented Jan 13, 2022

dasJ commented Jan 13, 2022 • edited Loading

dasJ commented Jan 13, 2022

dasJ Jan 13, 2022

Choose a reason for hiding this comment

Mic92 Jan 15, 2022

Choose a reason for hiding this comment

dasJ commented Jan 13, 2022

Mic92 commented Jan 15, 2022

Mic92 commented Jan 15, 2022 • edited Loading

Mic92 commented Jan 15, 2022

arcnmx commented Jan 15, 2022

Mic92 commented Jan 16, 2022 • edited Loading

flokli commented Jan 19, 2022

erikarvstedt commented Jan 20, 2022 • edited Loading

flokli commented Jan 20, 2022

arianvp commented Jan 20, 2022

erikarvstedt commented Jan 20, 2022 • edited Loading

Mic92 commented Jan 27, 2022

nrdxp left a comment

Choose a reason for hiding this comment

dasJ commented Jan 27, 2022

nrdxp commented Jan 27, 2022

Mic92 commented Apr 4, 2022

Mic92 commented Apr 4, 2022

symphorien commented Jun 5, 2022

tilpner commented Jun 5, 2022

symphorien commented Jun 5, 2022

flokli commented Oct 10, 2022

dasJ commented Oct 10, 2022

Mic92 commented Jan 13, 2022 •

edited

Loading

dasJ commented Jan 13, 2022 •

edited

Loading

Mic92 commented Jan 15, 2022 •

edited

Loading

Mic92 commented Jan 16, 2022 •

edited

Loading

erikarvstedt commented Jan 20, 2022 •

edited

Loading

erikarvstedt commented Jan 20, 2022 •

edited

Loading