Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add EMF And Raw Log Support For Cloud Watch Logs #18730

Merged
merged 1 commit into from
Feb 24, 2023

Conversation

sethAmazon
Copy link
Contributor

@sethAmazon sethAmazon commented Feb 16, 2023

Description:
Add raw log and emf log to aws cloud watch logs package

Link to tracking Issue: #18758

Testing:
Unit tests
Created a docker image and tested the output

Documentation:
Added to the aws cloud watch logs readme for raw log feature.

@sethAmazon sethAmazon requested a review from a team February 16, 2023 21:40
@github-actions github-actions bot added exporter/awscloudwatchlogs awscloudwatchlogs exporter exporter/awsemf awsemf exporter internal/aws labels Feb 16, 2023
@runforesight
Copy link

runforesight bot commented Feb 16, 2023

Foresight Summary

    
Major Impacts

build-and-test duration(40 minutes 8 seconds) has decreased 28 minutes 23 seconds compared to main branch avg(1 hour 8 minutes 31 seconds).
View More Details

⭕  build-and-test-windows workflow has finished in 5 seconds (42 minutes 1 second less than main branch avg.) and finished at 23rd Feb, 2023.


Job Failed Steps Tests
windows-unittest-matrix -     🔗  N/A See Details
windows-unittest -     🔗  N/A See Details

✅  telemetrygen workflow has finished in 54 seconds (2 minutes 10 seconds less than main branch avg.) and finished at 23rd Feb, 2023.


Job Failed Steps Tests
build-dev -     🔗  N/A See Details
publish-latest -     🔗  N/A See Details
publish-stable -     🔗  N/A See Details

✅  check-links workflow has finished in 2 minutes and finished at 23rd Feb, 2023.


Job Failed Steps Tests
changed files -     🔗  N/A See Details
check-links -     🔗  N/A See Details

✅  changelog workflow has finished in 2 minutes 38 seconds and finished at 23rd Feb, 2023.


Job Failed Steps Tests
changelog -     🔗  N/A See Details

✅  build-and-test workflow has finished in 40 minutes 8 seconds (28 minutes 23 seconds less than main branch avg.) and finished at 23rd Feb, 2023.


Job Failed Steps Tests
unittest-matrix (1.20, internal) -     🔗  ✅ 561  ❌ 0  ⏭ 0    🔗 See Details
unittest-matrix (1.20, extension) -     🔗  ✅ 537  ❌ 0  ⏭ 0    🔗 See Details
correctness-traces -     🔗  ✅ 17  ❌ 0  ⏭ 0    🔗 See Details
unittest-matrix (1.19, internal) -     🔗  ✅ 561  ❌ 0  ⏭ 0    🔗 See Details
unittest-matrix (1.20, processor) -     🔗  ✅ 1511  ❌ 0  ⏭ 0    🔗 See Details
correctness-metrics -     🔗  ✅ 2  ❌ 0  ⏭ 0    🔗 See Details
unittest-matrix (1.19, extension) -     🔗  ✅ 537  ❌ 0  ⏭ 0    🔗 See Details
unittest-matrix (1.19, processor) -     🔗  ✅ 1511  ❌ 0  ⏭ 0    🔗 See Details
unittest-matrix (1.20, receiver-0) -     🔗  ✅ 2577  ❌ 0  ⏭ 0    🔗 See Details
unittest-matrix (1.19, receiver-0) -     🔗  ✅ 2577  ❌ 0  ⏭ 0    🔗 See Details
unittest-matrix (1.20, exporter) -     🔗  ✅ 2462  ❌ 0  ⏭ 0    🔗 See Details
unittest-matrix (1.19, exporter) -     🔗  ✅ 2462  ❌ 0  ⏭ 0    🔗 See Details
unittest-matrix (1.20, receiver-1) -     🔗  ✅ 1932  ❌ 0  ⏭ 0    🔗 See Details
unittest-matrix (1.19, other) -     🔗  ✅ 4772  ❌ 0  ⏭ 0    🔗 See Details
unittest-matrix (1.19, receiver-1) -     🔗  ✅ 1932  ❌ 0  ⏭ 0    🔗 See Details
unittest-matrix (1.20, other) -     🔗  ✅ 4772  ❌ 0  ⏭ 0    🔗 See Details
integration-tests -     🔗  ✅ 55  ❌ 0  ⏭ 0    🔗 See Details
setup-environment -     🔗  N/A See Details
check-collector-module-version -     🔗  N/A See Details
build-examples -     🔗  N/A See Details
lint-matrix (receiver-0) -     🔗  N/A See Details
lint-matrix (receiver-1) -     🔗  N/A See Details
lint-matrix (processor) -     🔗  N/A See Details
lint-matrix (exporter) -     🔗  N/A See Details
lint-matrix (extension) -     🔗  N/A See Details
lint-matrix (internal) -     🔗  N/A See Details
lint-matrix (other) -     🔗  N/A See Details
checks -     🔗  N/A See Details
check-codeowners -     🔗  N/A See Details
unittest (1.20) -     🔗  N/A See Details
unittest (1.19) -     🔗  N/A See Details
lint -     🔗  N/A See Details
cross-compile (darwin, amd64) -     🔗  N/A See Details
cross-compile (darwin, arm64) -     🔗  N/A See Details
cross-compile (linux, 386) -     🔗  N/A See Details
cross-compile (linux, amd64) -     🔗  N/A See Details
cross-compile (linux, arm) -     🔗  N/A See Details
cross-compile (linux, arm64) -     🔗  N/A See Details
cross-compile (linux, ppc64le) -     🔗  N/A See Details
cross-compile (windows, 386) -     🔗  N/A See Details
cross-compile (windows, amd64) -     🔗  N/A See Details
build-package (deb) -     🔗  N/A See Details
build-package (rpm) -     🔗  N/A See Details
windows-msi -     🔗  N/A See Details
publish-check -     🔗  N/A See Details
publish-stable -     🔗  N/A See Details
publish-dev -     🔗  N/A See Details

✅  prometheus-compliance-tests workflow has finished in 3 minutes 39 seconds (5 minutes 37 seconds less than main branch avg.) and finished at 23rd Feb, 2023.


Job Failed Steps Tests
prometheus-compliance-tests -     🔗  ✅ 21  ❌ 0  ⏭ 0    🔗 See Details

✅  load-tests workflow has finished in 15 minutes 30 seconds (1 minute 59 seconds less than main branch avg.) and finished at 23rd Feb, 2023.


Job Failed Steps Tests
loadtest (TestTraceAttributesProcessor) -     🔗  ✅ 3  ❌ 0  ⏭ 0    🔗 See Details
loadtest (TestIdleMode) -     🔗  ✅ 1  ❌ 0  ⏭ 0    🔗 See Details
loadtest (TestTraceNoBackend10kSPS|TestTrace1kSPSWithAttrs) -     🔗  ✅ 8  ❌ 0  ⏭ 0    🔗 See Details
loadtest (TestMetricResourceProcessor|TestTrace10kSPS) -     🔗  ✅ 12  ❌ 0  ⏭ 0    🔗 See Details
loadtest (TestMetric10kDPS|TestMetricsFromFile) -     🔗  ✅ 6  ❌ 0  ⏭ 0    🔗 See Details
loadtest (TestTraceBallast1kSPSWithAttrs|TestTraceBallast1kSPSAddAttrs) -     🔗  ✅ 10  ❌ 0  ⏭ 0    🔗 See Details
loadtest (TestBallastMemory|TestLog10kDPS) -     🔗  ✅ 18  ❌ 0  ⏭ 0    🔗 See Details
setup-environment -     🔗  N/A See Details

✅  e2e-tests workflow has finished in 13 minutes 1 second (2 minutes 52 seconds less than main branch avg.) and finished at 23rd Feb, 2023.


Job Failed Steps Tests
kubernetes-test (v1.26.0) -     🔗  N/A See Details
kubernetes-test (v1.25.3) -     🔗  N/A See Details
kubernetes-test (v1.24.7) -     🔗  N/A See Details
kubernetes-test (v1.23.13) -     🔗  N/A See Details

🔎 See details on Foresight

*You can configure Foresight comments in your organization settings page.

svcStructuredLog *cwlogs.Client
config component.Config
logger *zap.Logger
pusherMap map[cwlogs.PusherKey]cwlogs.Pusher
Copy link
Contributor Author

@sethAmazon sethAmazon Feb 16, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Added some clean up here. Map is unique by the log group / stream which we can turn into a key. Instead of having a map of map.

@sethAmazon
Copy link
Contributor Author

sethAmazon commented Feb 17, 2023

There are unrelated file changes due to make gotidy and make gofmt

@sethAmazon sethAmazon force-pushed the add-emf-log-support branch 3 times, most recently from 32cecc6 to 5d7b781 Compare February 17, 2023 15:16
@sethAmazon sethAmazon force-pushed the add-emf-log-support branch 2 times, most recently from 40576e8 to 119a5c1 Compare February 20, 2023 18:39
Copy link
Member

@mx-psi mx-psi left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Needs some formatting

exporter_test.go:23: File is not `gci`-ed with --skip-generated -s standard,default,prefix(github.com/open-telemetry/opentelemetry-collector-contrib) (gci)
	"github.com/aws/aws-sdk-go/service/cloudwatchlogs"

exporter_test.go:25: File is not `gci`-ed with --skip-generated -s standard,default,prefix(github.com/open-telemetry/opentelemetry-collector-contrib) (gci)
	"github.com/aws/aws-sdk-go/aws/awserr"

(you can run golangci-lint with the --fix option from the exporter folder to fix this automatically)

@sethAmazon
Copy link
Contributor Author

Thank you for how to fix the import lint issues. The fmt command does not work for this error. I resolved the issue and then pushed.

@mx-psi mx-psi merged commit 7206d6d into open-telemetry:main Feb 24, 2023
@sethAmazon sethAmazon deleted the add-emf-log-support branch February 24, 2023 14:19
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants