Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add TraceFlags support to logs exporter #690

Merged
merged 1 commit into from
Jul 28, 2023
Merged

Add TraceFlags support to logs exporter #690

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
24 changes: 19 additions & 5 deletions exporter/collector/integrationtest/testdata/fixtures/logs/logs_span_trace_id.json
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -31,7 +31,7 @@
{
"key": "msg",
"value": {
"stringValue": "hello world 0"
"stringValue": "this log has IsSampled=0"
}
}
]
Expand All @@ -41,6 +41,7 @@
]
}
},
"flags": 0,
"attributes": [
{
"key": "file.name",
Expand Down Expand Up @@ -83,7 +84,7 @@
{
"key": "msg",
"value": {
"stringValue": "hello world 1"
"stringValue": "this log has IsSampled=1"
}
}
]
Expand All @@ -93,6 +94,7 @@
]
}
},
"flags": 1,
"attributes": [
{
"key": "file.name",
Expand Down Expand Up @@ -129,7 +131,7 @@
{
"key": "msg",
"value": {
"stringValue": "hello world 2"
"stringValue": "this log has neither flag nor gcp.trace_sampled"
}
},
{
Expand Down Expand Up @@ -187,7 +189,7 @@
{
"key": "msg",
"value": {
"stringValue": "hello world 3"
"stringValue": "this log has gcp.trace_sampled=false"
}
}
]
Expand All @@ -209,6 +211,12 @@
"value": {
"stringValue": "my-log-name-foo"
}
},
{
"key": "gcp.trace_sampled",
"value": {
"boolValue": false
}
}
],
"traceId": "ad57e7b40d6f172d04f8a0e1b80cafe4",
Expand Down Expand Up @@ -239,7 +247,7 @@
{
"key": "msg",
"value": {
"stringValue": "hello world 4"
"stringValue": "this log has gcp.trace_sampled=true"
}
}
]
Expand All @@ -261,6 +269,12 @@
"value": {
"stringValue": "my-log-name-foo"
}
},
{
"key": "gcp.trace_sampled",
"value": {
"boolValue": true
}
}
],
"traceId": "ad57e7b40d6f172d04f8a0e1b80cafe4",
Expand Down
20 changes: 11 additions & 9 deletions exporter/collector/integrationtest/testdata/fixtures/logs/logs_span_trace_id_expected.json
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -15,7 +15,7 @@
"jsonPayload": {
"body": {
"level": "info",
"msg": "hello world 0",
"msg": "this log has IsSampled=0",
"time": "2022-06-22T16:13:19Z"
}
},
Expand All @@ -39,7 +39,7 @@
"jsonPayload": {
"body": {
"level": "info",
"msg": "hello world 1",
"msg": "this log has IsSampled=1",
"time": "2022-06-22T16:13:19Z"
}
},
Expand All @@ -48,7 +48,8 @@
"file.name": "apache.log"
},
"trace": "projects/fakeprojectid/traces/ad57e7b40d6f172d04f8a0e1b80cafe4",
"spanId": "97c0f2f642dd1306"
"spanId": "97c0f2f642dd1306",
"traceSampled": true
},
{
"logName": "projects/fakeprojectid/logs/my-log-name-foo",
Expand All @@ -63,7 +64,7 @@
"jsonPayload": {
"body": {
"level": "info",
"msg": "hello world 2",
"msg": "this log has gcp.trace_sampled=false",
"time": "2022-06-22T16:13:19Z"
}
},
Expand All @@ -72,7 +73,7 @@
"file.name": "apache.log"
},
"trace": "projects/fakeprojectid/traces/ad57e7b40d6f172d04f8a0e1b80cafe4",
"spanId": "fded437fb39abed9"
"spanId": "6acb5055846322e7"
},
{
"logName": "projects/fakeprojectid/logs/my-log-name-foo",
Expand All @@ -87,7 +88,7 @@
"jsonPayload": {
"body": {
"level": "info",
"msg": "hello world 3",
"msg": "this log has gcp.trace_sampled=true",
"time": "2022-06-22T16:13:19Z"
}
},
Expand All @@ -96,7 +97,8 @@
"file.name": "apache.log"
},
"trace": "projects/fakeprojectid/traces/ad57e7b40d6f172d04f8a0e1b80cafe4",
"spanId": "6acb5055846322e7"
"spanId": "d9ae3a7b26ef239f",
"traceSampled": true
},
{
"logName": "projects/fakeprojectid/logs/my-log-name-foo",
Expand All @@ -111,7 +113,7 @@
"jsonPayload": {
"body": {
"level": "info",
"msg": "hello world 4",
"msg": "this log has neither flag nor gcp.trace_sampled",
"time": "2022-06-22T16:13:19Z"
}
},
Expand All @@ -120,7 +122,7 @@
"file.name": "apache.log"
},
"trace": "projects/fakeprojectid/traces/ad57e7b40d6f172d04f8a0e1b80cafe4",
"spanId": "d9ae3a7b26ef239f"
"spanId": "fded437fb39abed9"
}
],
"partialSuccess": true
Expand Down
6 changes: 3 additions & 3 deletions exporter/collector/logs.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -418,9 +418,9 @@ func (l logMapper) logToSplitEntries(
delete(attrsMap, SourceLocationAttributeKey)
}

// parse TraceSampled boolean from OTel attribute
if traceSampled, ok := attrsMap[TraceSampledAttributeKey]; ok {
entry.TraceSampled = traceSampled.Bool()
// parse TraceSampled boolean from OTel attribute or IsSampled OTLP flag
if traceSampled, ok := attrsMap[TraceSampledAttributeKey]; ok || logRecord.Flags().IsSampled() {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just curious, why did we have a "special" attribute in the first place?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In the spec we map all GCP LogEntry fields to a matching gcp.* attribute. LogEntry has a field traceSampled, so we get gcp.trace_sampled as the matching attribute.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should we update the spec to say that we should use the log record sampled flag instead of this attribute? We can keep it in the exporter for backwards-compatibility, but sampled isn't something I expect to get from attributes generally speaking.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yup good point, either way if we're adding support for log Flags we should update that spec doc to show it. I'll open a PR for that too

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

open-telemetry/opentelemetry-specification#3629

entry.TraceSampled = (traceSampled.Bool() || logRecord.Flags().IsSampled())
delete(attrsMap, TraceSampledAttributeKey)
}

Expand Down
18 changes: 18 additions & 0 deletions exporter/collector/logs_test.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -362,6 +362,24 @@ func TestLogMapping(t *testing.T) {
},
maxEntrySize: defaultMaxEntrySize,
},
{
name: "log with IsSampled",
mr: func() *monitoredrespb.MonitoredResource {
return nil
},
log: func() plog.LogRecord {
log := plog.NewLogRecord()
log.SetFlags(log.Flags().WithIsSampled(true))
return log
},
expectedEntries: []logging.Entry{
{
TraceSampled: true,
Timestamp: testObservedTime,
},
},
maxEntrySize: defaultMaxEntrySize,
},
{
name: "log with trace and span id",
mr: func() *monitoredrespb.MonitoredResource {
Expand Down