Invalid mode for zincati configuration on bootstrap

[vrutkovs]

Jun 13 04:34:17 localhost zincati[4583]: Error: Os { code: 13, kind: PermissionDenied, message: "Permission denied" }
Jun 13 04:34:17 localhost zincati[4583]: failed to open file '/etc/zincati/config.d/90-disable-feature.toml'
Jun 13 04:34:17 localhost zincati[4583]: failed to assemble configuration settings
Jun 13 04:34:17 localhost systemd[1]: zincati.service: Main process exited, code=exited, status=1/FAILURE
Jun 13 04:34:17 localhost systemd[1]: zincati.service: Failed with result 'exit-code'.

Zincati cannot read config file, probably due to an invalid file mode

[vrutkovs]

Fixed since https://origin-release.apps.ci.l2s4.p1.openshiftapps.com/releasestream/4.5.0-0.okd/release/4.5.0-0.okd-2020-07-01-233024

[LorbusChris]

What's weird in #392 (comment) is that the 90-disable-feature.toml file shouldn't even exist in 4.6 at all.
Anyway, I've opened openshift/okd-machine-os#24 to disable the service even before MCO runs.

[vrutkovs]

Here's a short description of the issue:

• on masters/workers Zincati is being stopped by a service dropin
• on bootstrap we add disable config in installer

Bootstrap configuration is apparently being created with invalid permissions. However this seems to do the trick - Zincati is stopped on bootstrap :) So its a minor nuisanse and it won't be resolved by MCO changes.

The cleanest approach here is to move installer file configuration to okd-machine-os and check file permissions.

[vrutkovs]

It actually got worse in latest 4.7/4.6:

[core@ip-10-0-15-78 ~]$ systemctl status zincati
● zincati.service - Zincati Update Agent
     Loaded: loaded (/usr/lib/systemd/system/zincati.service; enabled; vendor preset: enabled)
    Drop-In: /etc/systemd/system/zincati.service.d
             └─okd-machine-os-disabled.conf
     Active: active (running) since Sat 2020-12-05 09:55:38 UTC; 58s ago
       Docs: https://github.com/coreos/zincati
   Main PID: 742 (zincati)
      Tasks: 2 (limit: 9197)
     Memory: 15.4M
     CGroup: /system.slice/zincati.service
             └─742 /usr/libexec/zincati agent -v

Dec 05 09:55:38 localhost systemd[1]: Started Zincati Update Agent.
Dec 05 09:55:38 localhost zincati[742]: [INFO ] starting update agent (zincati 0.0.13)
Dec 05 09:55:44 ip-10-0-15-78 zincati[742]: [INFO ] Cincinnati service: https://updates.coreos.fedoraproject.org
Dec 05 09:55:44 ip-10-0-15-78 zincati[742]: [INFO ] agent running on node 'c97af433021d4245801732d157b0b0e0', in update group 'default'
Dec 05 09:55:44 ip-10-0-15-78 zincati[742]: [INFO ] initialization complete, auto-updates logic enabled
[core@ip-10-0-15-78 ~]$ cat /etc/systemd/system/zincati.service.d/okd-machine-os-disabled.conf 
[Unit]
ConditionPathExists=/enoent

[vrutkovs]

Fixed via installer dropin vrutkovs/installer@4a15883 :

[core@ip-10-0-0-171 ~]$ sudo systemctl status zincati
● zincati.service - Zincati Update Agent
     Loaded: loaded (/usr/lib/systemd/system/zincati.service; enabled; vendor preset: enabled)
    Drop-In: /etc/systemd/system/zincati.service.d
             └─okd-machine-os-disabled.conf
     Active: inactive (dead)
  Condition: start condition failed at Sat 2020-12-05 10:18:48 UTC; 1min 12s ago
       Docs: https://github.com/coreos/zincati

Dec 05 10:18:48 ip-10-0-0-171 systemd[1]: Condition check resulted in Zincati Update Agent being skipped.