documentation on accessing the json elements of the journald receiver logs in processors

[arnaudvre]

Component(s)

receiver/journald

What happened?

Cannot figure out how to reference the name value pairs from the journald log in attributeprocessor

Documentation bug / request for documentation.

Collector version

0.6.0

Environment information

Environment

OS: Ubuntu 22.04

OpenTelemetry Collector configuration

receivers:
  journald:

processors:
  resource:
    attributes:
      - action: insert
        key: loki.format
        value: json
      - action: insert
        key: environment
        value: dev
      - action: insert
        key: loki.attribute.labels
        value: hostname
  attributes:
    actions:
      - action: insert
        key: hostname
        from_attribute: body._HOSTNAME

exporters:
  loki:
    endpoint: "https://loki.net/loki/api/v1/push"
    sending_queue:
      num_consumers: 10
      queue_size: 50000
    headers:
      "X-Scope-OrgID": "xxx"
    tls:
      ca_file: "xxx"
      cert_file: "xxx"
      key_file: "xxx"
  file:
    path: /home/otelcol-contrib/temp.log
  debug:
    verbosity: detailed

service:
  pipelines:
    logs:
      receivers: [journald]
      processors: [resource,attributes]
      exporters: [debug]
  telemetry:
    logs:
      encoding: json

Log output

No response

Additional context

Example log json when written to file using the file exporter:

{
  "resourceLogs": [
    {
      "resource": {
        "attributes": [
          {
            "key": "loki.format",
            "value": {
              "stringValue": "json"
            }
          },
          {
            "key": "environment",
            "value": {
              "stringValue": "dev"
            }
          }
        ]
      },
      "scopeLogs": [
        {
          "scope": {},
          "logRecords": [
            {
              "timeUnixNano": "1721831731222344000",
              "observedTimeUnixNano": "1721831731422059659",
              "body": {
                "kvlistValue": {
                  "values": [
                    {
                      "key": "_SELINUX_CONTEXT",
                      "value": {
                        "stringValue": "unconfined\n"
                      }
                    },
                    {
                      "key": "_SYSTEMD_SLICE",
                      "value": {
                        "stringValue": "system.slice"
                      }
                    },
                    {
                      "key": "_HOSTNAME",
                      "value": {
                        "stringValue": "hostname-dev"
                      }
                    },
                    {
                      "key": "_SYSTEMD_CGROUP",
                      "value": {
                        "stringValue": "/system.slice/otelcol-contrib.service"
                      }
                    },
                    {
                      "key": "_CAP_EFFECTIVE",
                      "value": {
                        "stringValue": "0"
                      }
                    },
                    {
                      "key": "_PID",
                      "value": {
                        "stringValue": "2262330"
                      }
                    },
                    {
                      "key": "SYSLOG_FACILITY",
                      "value": {
                        "stringValue": "3"
                      }
                    },
                    {
                      "key": "_EXE",
                      "value": {
                        "stringValue": "/usr/bin/otelcol-contrib"
                      }
                    },
                    {
                      "key": "PRIORITY",
                      "value": {
                        "stringValue": "6"
                      }
                    },
                    {
                      "key": "SYSLOG_IDENTIFIER",
                      "value": {
                        "stringValue": "otelcol-contrib"
                      }
                    },
                    {
                      "key": "_TRANSPORT",
                      "value": {
                        "stringValue": "stdout"
                      }
                    },
                    {
                      "key": "_SYSTEMD_UNIT",
                      "value": {
                        "stringValue": "otelcol-contrib.service"
                      }
                    },
                    {
                      "key": "_UID",
                      "value": {
                        "stringValue": "997"
                      }
                    },
                    {
                      "key": "_SYSTEMD_INVOCATION_ID",
                      "value": {
                        "stringValue": "231cd2139b2f4664b755d43b4690aef0"
                      }
                    },
                    {
                      "key": "_GID",
                      "value": {
                        "stringValue": "997"
                      }
                    },
                    {
                      "key": "_STREAM_ID",
                      "value": {
                        "stringValue": "8a194d71567d4ae4b4752a9f132eb336"
                      }
                    },
                    {
                      "key": "_MACHINE_ID",
                      "value": {
                        "stringValue": "5fca68f59b9583a46efa7b9f6697ef74"
                      }
                    },
                    {
                      "key": "_BOOT_ID",
                      "value": {
                        "stringValue": "f0d1b58e375844c3961c527ffc365e6c"
                      }
                    },
                    {
                      "key": "_CMDLINE",
                      "value": {
                        "stringValue": "/usr/bin/otelcol-contrib --config=/etc/otelcol- contrib/config.yaml"
                      }
                    },
                    {
                      "key": "__MONOTONIC_TIMESTAMP",
                      "value": {
                        "stringValue": "598265666336"
                      }
                    },
                    {
                      "key": "_COMM",
                      "value": {
                        "stringValue": "otelcol-contrib"
                      }
                    },
                    {
                      "key": "MESSAGE",
                      "value": {
                        "stringValue": "2024-07-24T14:35:31.222Z\tinfo\tservice@v0.104.0/service.go:270\tShutdown complete."
                      }
                    },
                    {
                      "key": "__CURSOR",
                      "value": {
                        "stringValue": "s=92ed14a0f4d04f8f9b315bb73d09e814;i=20f19;b=f0d1b58e375844c3961c527ffc365e6c;m=8b4b699b20;t=61dff30b3d748;x=3a5de1616420ef73"
                      }
                    }
                  ]
                }
              },
              "traceId": "",
              "spanId": ""
            },
            {
              "timeUnixNano": "1721831731241174000",
              "observedTimeUnixNano": "1721831731422135940",
              "body": {
                "kvlistValue": {
                  "values": [
                    {
                      "key": "UNIT",
                      "value": {
                        "stringValue": "otelcol-contrib.service"
                      }
                    },
                    {
                      "key": "CODE_FUNC",
                      "value": {
                        "stringValue": "unit_log_success"
                      }
                    },
                    {
                      "key": "_MACHINE_ID",
                      "value": {
                        "stringValue": "5fca68f59b9583a46efa7b9f6697ef74"
                      }
                    },
                    {
                      "key": "_SELINUX_CONTEXT",
                      "value": {
                        "stringValue": "unconfined\n"
                      }
                    },
                    {
                      "key": "_SYSTEMD_SLICE",
                      "value": {
                        "stringValue": "-.slice"
                      }
                    },
                    {
                      "key": "_UID",
                      "value": {
                        "stringValue": "0"
                      }
                    },
                    {
                      "key": "TID",
                      "value": {
                        "stringValue": "1"
                      }
                    },
                    {
                      "key": "MESSAGE_ID",
                      "value": {
                        "stringValue": "7ad2d189f7e94e70a38c781354912448"
                      }
                    },
                    {
                      "key": "_TRANSPORT",
                      "value": {
                        "stringValue": "journal"
                      }
                    },
                    {
                      "key": "SYSLOG_FACILITY",
                      "value": {
                        "stringValue": "3"
                      }
                    },
                    {
                      "key": "_CAP_EFFECTIVE",
                      "value": {
                        "stringValue": "1ffffffffff"
                      }
                    },
                    {
                      "key": "_CMDLINE",
                      "value": {
                        "stringValue": "/lib/systemd/systemd autoinstall --system --deserialize 41"
                      }
                    },
                    {
                      "key": "SYSLOG_IDENTIFIER",
                      "value": {
                        "stringValue": "systemd"
                      }
                    },
                    {
                      "key": "__MONOTONIC_TIMESTAMP",
                      "value": {
                        "stringValue": "598265685166"
                      }
                    },
                    {
                      "key": "_BOOT_ID",
                      "value": {
                        "stringValue": "f0d1b58e375844c3961c527ffc365e6c"
                      }
                    },
                    {
                      "key": "_SOURCE_REALTIME_TIMESTAMP",
                      "value": {
                        "stringValue": "1721831731241074"
                      }
                    },
                    {
                      "key": "__CURSOR",
                      "value": {
                        "stringValue": "s=92ed14a0f4d04f8f9b315bb73d09e814;i=20f1a;b=f0d1b58e375844c3961c527ffc365e6c;m=8b4b69e4ae;t=61dff30b420d6;x=d6baded09f1b919b"
                      }
                    },
                    {
                      "key": "_EXE",
                      "value": {
                        "stringValue": "/usr/lib/systemd/systemd"
                      }
                    },
                    {
                      "key": "_PID",
                      "value": {
                        "stringValue": "1"
                      }
                    },
                    {
                      "key": "_SYSTEMD_CGROUP",
                      "value": {
                        "stringValue": "/init.scope"
                      }
                    },
                    {
                      "key": "PRIORITY",
                      "value": {
                        "stringValue": "6"
                      }
                    },
                    {
                      "key": "_SYSTEMD_UNIT",
                      "value": {
                        "stringValue": "init.scope"
                      }
                    },
                    {
                      "key": "_GID",
                      "value": {
                        "stringValue": "0"
                      }
                    },
                    {
                      "key": "INVOCATION_ID",
                      "value": {
                        "stringValue": "231cd2139b2f4664b755d43b4690aef0"
                      }
                    },
                    {
                      "key": "_HOSTNAME",
                      "value": {
                        "stringValue": "hostname-dev"
                          }
                    },
                    {
                      "key": "_COMM",
                      "value": {
                        "stringValue": "systemd"
                      }
                    },
                    {
                      "key": "CODE_LINE",
                      "value": {
                        "stringValue": "5553"
                      }
                    },
                    {
                      "key": "MESSAGE",
                      "value": {
                        "stringValue": "otelcol-contrib.service: Deactivated successfully."
                      }
                    },
                    {
                      "key": "CODE_FILE",
                      "value": {
                        "stringValue": "src/core/unit.c"
                      }
                    }
                  ]
                }
              }
              "traceId": "",
              "spanId": ""
            }
          ]
        }
      ]
    }
  ]
}

[github-actions]

Pinging code owners:

• receiver/journald: @sumo-drosiek @djaglowski

See Adding Labels via Comments if you do not have permissions to add labels yourself.

[sumo-drosiek]

Hi @arnaudvre. Is the collector version correct? 0.6.0 doesn't seem to be supported version

[arnaudvre]

hey @sumo-drosiek what version is supported? I'm happy to use whatever the supported version is but don't think that will change the issue.

[sumo-drosiek]

@arnaudvre I look at it more carefully and I think you should use transform processor if you want to use body attributes:

transform:
  error_mode: ignore
  log_statements:
    - context: log
      statements:
        - set(attributes["hostname"], body._HOSTNAME)

[arnaudvre]

Get this error:

otelcol-contrib[119767]: 2024/08/30 13:00:28 collector server run finished with error: invalid configuration: processors::transform: unable to parse OTTL statement "set(attributes[\"hostname\"], body._HOSTNAME)": statement has invalid syntax: 1:34: invalid input text "_HOSTNAME)"

tried escaping the underscore:

- set(attributes["hostname"], body.\_HOSTNAME)

also tried with a period just in case that worked although was sure it wouldn't:

- set(attributes["hostname"], body..HOSTNAME)

but the following does it!

- set(attributes["hostname"], body["_HOSTNAME"])

Thanks a lot.

@sumo-drosiek I'd like to abuse of your knowledge slightly if possible to ask if you know of how to collapse the encapsulating json element, in this case the hostname is coming in as attribute.hostname but I'd like hostname to be top level, not nested under the attribute element.

[sumo-drosiek]

@sumo-drosiek I'd like to abuse of your knowledge slightly if possible to ask if you know of how to collapse the encapsulating json element, in this case the hostname is coming in as attribute.hostname but I'd like hostname to be top level, not nested under the attribute element.

@arnaudvre Do you relate strictly to the file exporter? Logs are structured in OTLP, and AFAIK you cannot create your own property on the top-level. Please see the following documentation

[arnaudvre]

@sumo-drosiek we're currently using lokiexporter but will soon move to otlphttp

[macdewee]

@sumo-drosiek we're currently using lokiexporter but will soon move to otlphttp

I think it's not possible to add top level property in the lokiexporter neither

[arnaudvre]

nor otlphttp correct?

[macdewee]

correct śr., 4 wrz 2024, 16:48 użytkownik arnaudvre ***@***.***> napisał:

…

nor otlphttp correct? — Reply to this email directly, view it on GitHub <#34808 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ABVMK4MNB5I66GF3IQBZO53ZU4MVBAVCNFSM6AAAAABM5R4RG2VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDGMRZGI3TIMJYHA> . You are receiving this because you commented.Message ID: <open-telemetry/opentelemetry-collector-contrib/issues/34808/2329274188@ github.com>