[exporter/debug] change behavior of "normal" verbosity to be different from "basic"

[andrzej-stencel]

EDIT: Updated the issue to change the debug exporter instead of logging exporter.

Collector version: v0.88.0

Is your feature request related to a problem? Please describe.

Currently there are three verbosity levels available for the Debug exporter: basic, normal, detailed. The basic and normal level result in the same output, which only displays the number of records:

exporters:
  logging:
    verbosity: normal

2023-06-01T13:26:16.044+0200    info    MetricsExporter {"kind": "exporter", "data_type": "metrics", "name": "logging", "resource metrics": 1, "metrics": 1, "data points": 1}

On the other hand, the detailed verbosity displays all the details about telemetry, with multiple lines per span/data point/log record:

2023-06-01T13:30:22.669+0200    info    MetricsExporter {"kind": "exporter", "data_type": "metrics", "name": "logging", "resource metrics": 1, "metrics": 1, "data points": 1}
2023-06-01T13:30:22.669+0200    info    ResourceMetrics #0
Resource SchemaURL: 
ScopeMetrics #0
ScopeMetrics SchemaURL: 
InstrumentationScope otelcol/httpcheckreceiver 0.78.0
Metric #0
Descriptor:
     -> Name: httpcheck.duration
     -> Description: Measures the duration of the HTTP check.
     -> Unit: ms
     -> DataType: Gauge
NumberDataPoints #0
Data point attributes:
     -> http.url: Str(https://opentelemetry.io)
StartTimestamp: 2023-06-01 11:30:18.60847008 +0000 UTC
Timestamp: 2023-06-01 11:30:21.630471795 +0000 UTC
Value: 1038

Describe the solution you'd like

I propose to change the behavior for the normal verbosity to display telemetry details one line per span/data point/log record:

exporters:
  logging:
    verbosity: normal

2023-06-01T13:30:22.669+0200    info    MetricsExporter {"kind": "exporter", "data_type": "metrics", "name": "logging", "resource metrics": 1, "metrics": 1, "data points": 1}
2023-06-01T13:30:22.669+0200    info    MetricDataPoint httpcheck.duration{http.url=Str(https://opentelemetry.io)} 1038

The format is up for discussion, but I believe having one line per telemetry record might be a good "middle ground". The resource and scope details might or might not also be displayed, I'm open to suggestions.

Describe alternatives you've considered

I considered using the File exporter, which outputs JSON with full information on the telemetry to a file.

• It only outputs to a file. I don't think it's possible to make it output to the collector's console output?
• The output is too verbose to my taste, also it puts eeeverything in a single line. It would be much more readable to have one line per telemetry record (span/data point/log record).

Another alternative for metrics is to use Prometheus exporter, but it modifies data (replaces dots with underscores etc.).

[perk-sumo]

I also find logging exporter output a bit too verbose.

It would be awesome if I could provide a "template" for output - so that I can choose which metadata I want to see.
Eg. for logs sometimes I might be interested only in logs, sometimes in logs and some metadata and sometimes in metadata only.

Another name for this mode could be oneline (like in git log output).

[andrzej-stencel]

It would be awesome if I could provide a "template" for output

I agree this might be very useful. A bit more complex (need to define the "templating language", even if very basic), but very useful.

[herrbpl]

Also, what happened to log level ? Why cannot you filter out errors only?

[andrzej-stencel]

Also, what happened to log level ? Why cannot you filter out errors only?

@herrbpl, the logging exporter does not log errors. The exporter's name is in fact misleading, it should be named debug exporter (See #7769), as it is meant for sending telemetry to console for debugging purposes. All output from this exporter is considered "info" level. This is why the loglevel configuration property has been renamed to verbosity, as you do not filter on log levels, but rather you tell the exporter to be more or less verbose.

[andrzej-stencel]

Related issue:

• [exporter/debug] default verbosity should to be changed from normal to basic. #8844

I believe #8844 should be done first.

[atoulme]

I support this proposal.

[andrzej-stencel]

Proposing changes to change the formatting in normal verbosity:

• [exporter/debug] format log records as one-liners in normal verbosity #10225
• [exporter/debug] format metric data points as one-liners in normal verbosity #10462
• [exporter/debug] format spans as one-liners in normal verbosity #10280

[andrzej-stencel]

Given that all these PRs are merged, I'm closing this issue.

• [exporter/debug] format log records as one-liners in normal verbosity #10225
• [exporter/debug] format metric data points as one-liners in normal verbosity #10462
• [exporter/debug] format spans as one-liners in normal verbosity #10280