SC4S Podman Rootless setup issue

[mathiznogoud]

I followed the documentation provided here: https://splunk.github.io/splunk-connect-for-syslog/main/gettingstarted/podman-systemd-general/

OS: RHEL 8.7
Podman version: 4.2.0

On the line: ExecStartPre=/usr/bin/bash -c "/usr/bin/systemctl set-environment SC4SHOST=$(hostname -s)" produce the following error

Apr 19 17:02:51 sc4s.local dbus-daemon[925]: [system] Rejected send message, 2 matched rules; type="method_call", sender=":1.335" (uid=1000 pid=24420 comm="/>
Apr 19 17:02:51 sc4s.local bash[24420]: Failed to set environment: Access denied
Apr 19 17:02:51 sc4s.local systemd[1]: sc4s.service: Control process exited, code=exited status=1
Apr 19 17:02:51 sc4s.local systemd[1]: sc4s.service: Failed with result 'exit-code'.

After that, I removed the line and need to change the env_file like below in order for the service to run since the config provided in the guide cause permission denied error

SC4S_DEST_SPLUNK_HEC_DEFAULT_URL=https://splunk.local:8088
SC4S_DEST_SPLUNK_HEC_DEFAULT_TOKEN=redacted
#Uncomment the following line if using untrusted SSL certificates
SC4S_DEST_SPLUNK_HEC_DEFAULT_TLS_VERIFY=no
SC4S_LISTEN_DEFAULT_TCP_PORT=8514
SC4S_LISTEN_DEFAULT_UDP_PORT=8514
SC4S_LISTEN_DEFAULT_RFC5426_PORT=8601
SC4S_LISTEN_DEFAULT_RFC6587_PORT=8601

However, the journalctl having these error line below

**Apr 19 17:20:13 sc4s.local podman[24908]: time="2023-04-19T17:20:13+07:00" level=error msg="unable to get systemd connection to add healthchecks: dial unix /tmp/podman-run-1000/systemd/private: connect: connection refused"
0" level=error msg="unable to get systemd connection to start healthchecks: dial unix /tmp/podman-run-1000/systemd/private: connect: connection refused"**
Apr 19 17:20:13 sc4s.local podman[24908]: SC4S_ENV_CHECK_HEC: Splunk HEC connection test successful to index=main for sourcetype=sc4s:fallback...
Apr 19 17:20:13 sc4s.local podman[24908]: SC4S_ENV_CHECK_HEC: Splunk HEC connection test successful to index=main for sourcetype=sc4s:events...
Apr 19 17:20:16 sc4s.local podman[24908]: syslog-ng checking config
Apr 19 17:20:16 sc4s.local podman[24908]: sc4s version=2.49.0
Apr 19 17:20:17 sc4s.local podman[24908]: starting goss

This could somehow be related to this topic containers/podman#12778

[rjha-splunk]

We need to spin the instance to check it @bparmar-splunk Can you please check this.

[mateuszpierzchala-splunk]

Hi @mathiznogoud , I have found a few issues with the instruction, I will prepare PR with updates. The main issue is that normal user can't run systemctl without sudo, all occurrences should be replaced with systemctl --user (both service file and starting service command). This fact also implicates that the service file should be placed under /home/sc4s/.config/systemd/user/sc4s.service instead of /lib/systemd/system/sc4s.service.