systemd-Journal by file (netdata#16038)

* query journal file by file: 17% faster

* maintain a registry of journal files in memory and support multiple journal directories; offer sources of journal directories

* fixes

* overloaded libc fstat64() call to speed up libsystemd

* do not just copy unset values, there is a flag that tracks them

* optimize facets_row_finished()

* use container name in ND_JOURNAL_PROCESS

* fix compatibility with versions of libsystemd without sd_journal_open_files_fd()

* added more statistics about the time spent per journal file

* optimize facets_row_finished()

* optimize facets_rows_begin()

* tuning

* progress reporting

* fix journal seek to precisely match log timestamps

* support remote sources and namespaces

* jf_is_mine() as function

* fixes

* fixes 2

* fixes 3

* added debug

* fixed log

* added source for fqs

* fix all source names

* fix jf_is_mine() to return a value

* add rows_useful to journal files

* sorted list of all sources

* make hostname visible by default

* rename sources

* increase number of columns

* updated apps_groups.conf

* support view only transformations

* add support for slicing

* add support for older versions of systemd

* cleanup

* added ordering of key values

* convert remote IPs to hostnames

* fix for hostname resolution

* standardize the source name length

* added versions

* added sources pills and info

* fix plural

* better formatting for durations

* support dynamic unset value

* fix sorting

* errno to still show numeric values

* maintain a used hashes registry

* fixed severity

* updated function help message with all current parameters accepted

* remove internal error

* always return null as empty values in data

* add default sd_journal_open flags

* validate anchor

* fix compiler warning

* calculate journal vs realtime delta per journal file

* up to 2 minutes journal vs realtime delta

* more detailed message

* do not log zero anchor

* fixed message

* fix seek to db

* request details and dump of all journal files in response

* sort files before processing them

* do not sort if fewer than 2 files

* documentation

* added documentation about performance

* added field transformations documentation and annotated _CAP_EFFECTIVE

* updated docs

* updated docs

* annotated SOURCE_REALTIME_TIMESTAMP

* updated docs

* workaround for old systems

* updated docs

* updated docs

* updated docs

* updated docs

* more fields to show by default

* filter data-only query by libsystemd on slice mode

* better tail

* restore operation of full queries

* updated docs

* updated docs

* added smart field _BOOT_ID to automatically extract the timestamp of the first message of this boot_id

* do not seek to anchor on full queries

* added tail and delta

* alphabetical sort on calculated columns

* simplify sorting of facet values

* fix sorting of transformed values

* simplify code

* numeric values for capabilities that do not exist in old systems

* do not log if directories do not exist or are not directories

collectors/apps.plugin/apps_groups.conf

@@ -83,14 +83,14 @@ xenstat.plugin: xenstat.plugin
 perf.plugin: perf.plugin
 charts.d.plugin: *charts.d.plugin*
 python.d.plugin: *python.d.plugin*
+systemd-journal.plugin:*systemd-journal.plugin*
 tc-qos-helper: *tc-qos-helper.sh*
 fping: fping
 ioping: ioping
 go.d.plugin: *go.d.plugin*
-slabinfo.plugin: slabinfo.plugin
+slabinfo.plugin: *slabinfo.plugin*
 ebpf.plugin: *ebpf.plugin*
 debugfs.plugin: *debugfs.plugin*
-systemd-journal.plugin: systemd-journal
 
 # agent-service-discovery
 agent_sd: agent_sd
@@ -137,7 +137,7 @@ modem: ModemManager
 netmanager: NetworkManager nm* systemd-networkd networkctl netplan connmand wicked* avahi-autoipd networkd-dispatcher
 firewall: firewalld ufw nft
 tor: tor
-bluetooth: bluetooth bluez bluedevil obexd
+bluetooth: bluetooth bluetoothd bluez bluedevil obexd 
 
 # -----------------------------------------------------------------------------
 # high availability and balancers
@@ -160,7 +160,7 @@ chat: irssi *vines* *prosody* murmurd
 # -----------------------------------------------------------------------------
 # monitoring
 
-logs: ulogd* syslog* rsyslog* logrotate systemd-journald rotatelogs sysklogd metalog
+logs: ulogd* syslog* rsyslog* logrotate *systemd-journal* rotatelogs sysklogd metalog
 nms: snmpd vnstatd smokeping zabbix* munin* mon openhpid tailon nrpe
 monit: monit
 splunk: splunkd
@@ -210,7 +210,7 @@ proxmox-ve: pve* spiceproxy
 # -----------------------------------------------------------------------------
 # containers & virtual machines
 
-containers: lxc* docker* balena*
+containers: lxc* docker* balena* containerd
 VMs: vbox* VBox* qemu* kvm*
 libvirt: virtlogd virtqemud virtstoraged virtnetworkd virtlockd virtinterfaced
 libvirt: virtnodedevd virtproxyd virtsecretd libvirtd
@@ -239,7 +239,7 @@ dhcp: *dhcp* dhclient
 # -----------------------------------------------------------------------------
 # name servers and clients
 
-dns: named unbound nsd pdns_server knotd gdnsd yadifad dnsmasq systemd-resolve* pihole* avahi-daemon avahi-dnsconfd
+dns: named unbound nsd pdns_server knotd gdnsd yadifad dnsmasq *systemd-resolve* pihole* avahi-daemon avahi-dnsconfd
 dnsdist: dnsdist
 
 # -----------------------------------------------------------------------------
@@ -272,7 +272,7 @@ backup: rsync lsyncd bacula* borg rclone
 # -----------------------------------------------------------------------------
 # cron
 
-cron: cron* atd anacron systemd-cron* incrond
+cron: cron* atd anacron *systemd-cron* incrond
 
 # -----------------------------------------------------------------------------
 # UPS
@@ -320,7 +320,7 @@ airflow: *airflow*
 # -----------------------------------------------------------------------------
 # GUI
 
-X: X Xorg xinit xdm Xwayland xsettingsd
+X: X Xorg xinit xdm Xwayland xsettingsd touchegg
 wayland: swaylock swayidle waypipe wayvnc
 kde: *kdeinit* kdm sddm plasmashell startplasma-* kwin* kwallet* krunner kactivitymanager*
 gnome: gnome-* gdm gconf* mutter
@@ -354,11 +354,11 @@ kswapd: kswapd
 zswap: zswap
 kcompactd: kcompactd
 
-system: systemd-* udisks* udevd* *udevd ipv6_addrconf dbus-* rtkit*
+system: systemd* udisks* udevd* *udevd ipv6_addrconf dbus-* rtkit*
 system: mdadm acpid uuidd upowerd elogind* eudev mdev lvmpolld dmeventd
 system: accounts-daemon rngd haveged rasdaemon irqbalance start-stop-daemon
 system: supervise-daemon openrc* init runit runsvdir runsv auditd lsmd
-system: abrt* nscd rtkit-daemon gpg-agent usbguard*
+system: abrt* nscd rtkit-daemon gpg-agent usbguard* boltd geoclue
 
 kernel: kworker kthreadd kauditd lockd khelper kdevtmpfs khungtaskd rpciod
 kernel: fsnotify_mark kthrotld deferwq scsi_* kdmflush oom_reaper kdevtempfs

collectors/plugins.d/pluginsd_parser.c

@@ -888,7 +888,7 @@ static void inflight_functions_delete_callback(const DICTIONARY_ITEM *item __may
     pf->result_cb(pf->result_body_wb, pf->code, pf->result_cb_data);
 
     string_freez(pf->function);
-    freez(pf->payload);
+    freez((void *)pf->payload);
 }
 
 void inflight_functions_init(PARSER *parser) {