podman PID != service MainPID happens when catatonit is not already running

[eriksjolund]

Just an observation:

I noticed that the podman PID is not equal to the service MainPID
when a systemd user service is started before any catatonit is running.

(This situation could for instance occur after a reboot when no other Podman command has been run)

I tried running this systemd service with Podman 4.5.0

[test@localhost ~]$ cat .config/systemd/user/test.service 
[Unit]
RequiresMountsFor=%t/containers

[Service]
Environment=PODMAN_SYSTEMD_UNIT=%n
KillMode=mixed
ExecStop=/usr/bin/podman rm -f -i --cidfile=%t/%N.cid
ExecStopPost=-/usr/bin/podman rm -f -i --cidfile=%t/%N.cid
Delegate=yes
Type=notify
NotifyAccess=main
Restart=no
SyslogIdentifier=%N
ExecStart=/usr/bin/podman --log-level=debug run --name=systemd-%N --cidfile=%t/%N.cid --replace --rm --cgroups=split --sdnotify=conmon -d docker.io/library/alpine sleep inf

The relevant difference to a normal quadlet service is:

NotifyAccess=main
Restart=no

I noticed that

systemctl --user start test.service

fails if catatonit is not already running.

journalctl showed

Got notification message from PID 1783, but reception only permitted for main PID 1777

and

Failed with result 'protocol'.

If catatonit is available beforehand, the podman PID is equal to the systemd service MAINPID and

systemctl --user start test.service

succeeds.

[test@localhost ~]$ pgrep -u test -f catatonit
[test@localhost ~]$ start=$(date +%s)
[test@localhost ~]$ systemctl --user start test.service
Job for test.service failed because the service did not take the steps required by its unit configuration.
See "systemctl --user status test.service" and "journalctl --user -xeu test.service" for details.
[test@localhost ~]$  journalctl -q --no-pager -xe -p warning --since=@$start
Jun 09 18:49:21 localhost.localdomain systemd-test[1801]: [conmon:d]: failed to write to /proc/self/oom_score_adj: Permission denied
Jun 09 18:49:21 localhost.localdomain systemd-test[1801]: 
Jun 09 18:49:21 localhost.localdomain systemd[834]: test.service: Got notification message from PID 1783, but reception only permitted for main PID 1777
Jun 09 18:49:22 localhost.localdomain systemd[834]: test.service: Failed with result 'protocol'.
░░ Subject: Unit failed
░░ Defined-By: systemd
░░ Support: https://lists.freedesktop.org/mailman/listinfo/systemd-devel
░░ 
░░ The unit UNIT has entered the 'failed' state with result 'protocol'.
Jun 09 18:49:22 localhost.localdomain systemd[834]: Failed to start test.service.
░░ Subject: A start job for unit UNIT has failed
░░ Defined-By: systemd
░░ Support: https://lists.freedesktop.org/mailman/listinfo/systemd-devel
░░ 
░░ A start job for unit UNIT has finished with a failure.
░░ 
░░ The job identifier is 56 and the job result is failed.
[test@localhost ~]$ systemctl --user reset-failed test.service
[test@localhost ~]$ pgrep -u test -f catatonit
1787
[test@localhost ~]$ systemctl --user start test.service
[test@localhost ~]$ systemctl --user is-active test.service
active
[test@localhost ~]$ 

[rhatdan]

Have you looked into using quadlet?

[eriksjolund]

Yes, quadlet is convenient. This service was manually edited from
a quadlet-generated service.