How to parse file exported from LoggingMetricsExporter?

[deshsidd]

I am using the following exporter to export my OpenTelemetry metrics : io.opentelemetry.exporter.logging.LoggingMetricExporter

Getting data in the following format in a log file:

metric: ImmutableMetricData{name=histo1, description=, unit=1, type=HISTOGRAM,
  data=ImmutableHistogramData{aggregationTemporality=DELTA,
    points=[ImmutableHistogramPointData{getStartEpochNanos=1650484261414000000, getEpochNanos=1650484271431000000}]
  }
}

I want to now parse this into JSON structure or any similar format and send this to a backend database for visualizing the metrics. Please help me understand if you provide a parser that I can use? Any suggestions on how I can achieve this?

[deshsidd]

cc @jkwatson @jack-berg Please help out on the above as its blocking our progress. Thanks

[jack-berg]

The LoggingMetricExporter (as well as other exporters in opentelemetry-exporter-logging) are meant for debugging, and do not have a structured format designed for parsing. For this type of thing, check out OtlpJsonLoggingMetricExporter (published in opentelemetry-exporter-logging-otlp).

I hope this relieves your blockage. 🙂

[deshsidd]

Thanks @jack-berg!

[deshsidd]

The above seems to work @jack-berg. Any suggestions on the best way to parse the OtlpJsonLoggingMetricExporter format? Do you provide a parser or something similar for the above? Is https://pypi.org/project/opentelemetry-proto/ the best way to parse this?

[deshsidd]

Trying to use the proto to parse this but looks like the file is now exported in the following format:

{
  "resource": {
    "attributes": [
      {
        "key": "service.name",
        "value": {
          "stringValue": "xxx"
        }
      }
    ]
  },
  "scopeMetrics": [
        {
          "name": "xxx.counter",
          "description": "Counter",
          "unit": "1",
          "sum": {
            "dataPoints": [
              {
                "startTimeUnixNano": "1723528410485877000",
                "timeUnixNano": "1723529250478593000",
                "asDouble": 20,
                "exemplars": [],
                "attributes": [
                  {
                    "key": "xxx",
                    "value": {
                      "intValue": "xxx"
                    }
                  }
                ]
              }
            ],
            "aggregationTemporality": 2,
            "isMonotonic": true
          }
        },
        {
          "name": "yyy.counter",
          "description": "Counter",
          "unit": "1",
          "sum": {
            "dataPoints": [
              {
                "startTimeUnixNano": "1723528410485877000",
                "timeUnixNano": "1723529250478593000",
                "asDouble": 12,
                "exemplars": [],
                "attributes": [
                  {
                    "key": "xxx",
                    "value": {
                      "stringValue": "xxx"
                    }
                  }
                ]
              }
            ],
            "aggregationTemporality": 2,
            "isMonotonic": true
          }
        }
      ]
    }
  ]
}

and the proto is expecting:

// MetricsData
// └─── ResourceMetrics
//   ├── Resource
//   ├── SchemaURL
//   └── ScopeMetrics
//      ├── Scope
//      ├── SchemaURL
//      └── Metric
//         ├── Name
//         ├── Description
//         ├── Unit
//         └── data
//            ├── Gauge
//            ├── Sum
//            ├── Histogram
//            ├── ExponentialHistogram
//            └── Summary

For some reason the expected json shown here has

{
  "resourceMetrics": [

Which is not present in my output.
@jack-berg Any suggestions on how to parse this using proto?

[deshsidd]

cc @jkwatson in case you are aware

[deshsidd]

Here is what I am trying :

parsed_data = metrics_pb2.ResourceMetrics().ParseFromString(data)

Error:

Error parsing message

[jack-berg]

Using dependencies:

• com.google.protobuf:protobuf-java-util:3.25.4
• io.opentelemetry.proto:opentelemetry-proto:1.3.2-alpha

ResourceMetrics.Builder builder = ResourceMetrics.newBuilder();
JsonFormat.parser().merge(message, builder);
System.out.println(builder.build());