Elasticsearch exporter: Init JSON encoding support #3101
Conversation
|
@faec Can you help reviewing please? |
Codecov Report
@@ Coverage Diff @@
## main #3101 +/- ##
==========================================
- Coverage 91.91% 91.77% -0.14%
==========================================
Files 494 496 +2
Lines 23939 24103 +164
==========================================
+ Hits 22003 22120 +117
- Misses 1429 1472 +43
- Partials 507 511 +4
Flags with carried forward coverage won't be shown. Click here to find out more.
Continue to review full report at Codecov.
|
urso
force-pushed
the
esexporter-init-json-encoding
branch
from
8583a90 to
059f38d
Compare
|
@faec @blakerouse please review as codeowners. |
|
@urso please resolve the merge conflict. |
|
@faec @blakerouse please review. |
|
@urso please mark as resolved conversations that are resolved so easier to check the status. Please rebase |
urso
force-pushed
the
esexporter-init-json-encoding
branch
from
4bfa870 to
0462351
Compare
|
@bogdandrutu Done. |
urso
force-pushed
the
esexporter-init-json-encoding
branch
3 times, most recently
from
8d6f9be to
1a80aa9
Compare
|
This PR was marked stale due to lack of activity. It will be closed in 7 days. |
urso
force-pushed
the
esexporter-init-json-encoding
branch
2 times, most recently
from
e0d4b80 to
3ca7017
Compare
|
@bogdandrutu Any idea about the CI errors? Due to merge conflicts in |
|
Please rebase from main. It should fix the build. |
The change adds support for encoding OpenTelemetry log records to JSON. The encoder tries to remove duplicate entries in case the attribute map (which is an array if key value pairs) contains duplicates. Mixed style attributes, with key names having dots and other fields having attribute maps as value will be normalized, such that the JSON encoding will be either completely flat, or values are properly merged into a single JSON object (when dedot is enabled). The normalization helps with deduplication, and (not yet implemented) dedot support will allow us to present a well formated JSON event if Ingest Node is used (The dedotting in Elasticsearch does happen after Ingest Node). Next: - Dedotting support - Custom (configurable) field mapping - Testing testing
urso
force-pushed
the
esexporter-init-json-encoding
branch
from
3ca7017 to
a4a27ea
Compare
|
@tigrannajaryan @bogdandrutu Rebased yesterday. CI is green now. Thanks for fixing the CircleCI configs. |
|
If this fully resolves #1800 please close it. |
Description: The change adds support for encoding OpenTelemetry log records to JSON.
The encoder tries to remove duplicate entries in case the attribute map
(which is an array if key value pairs) contains duplicates.
Mixed style attributes, with key names having dots and other fields
having attribute maps as value will be normalized, such that the JSON
encoding will be either completely flat, or values are properly merged
into a single JSON object (when dedot is enabled). The normalization
helps with deduplication, and (not yet implemented) dedot support will
allow us to present a well formated JSON event if Ingest Node is used
(The dedotting in Elasticsearch does happen after Ingest Node).
Next:
publishLogsunit testingLink to tracking Issue: #1800
Testing:
The internal document type with field deduplication is fully tested (89%) via unit tests.
The change also hooks up
publishLogs, but this functionality is not covered by tests yet, as the PR has already grown quite a bit (I tested locally with a custom otel collector distribution). I would like to add additional tests in a separate PR, to keep focused on the JSON encoding only here.Documentation: None.