Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[chore] [exporter/splunkhec] Split the profiling data before processing #21909

Merged

Conversation

dmitryax
Copy link
Member

@dmitryax dmitryax commented May 14, 2023

To simplify the logic and make future improvements possible. Benchmarks shows no performance degradation for typical use cases (regular logs or profiling only) with small improvement on memory allocation. Benchmarks were adjusted to be applied on ConsumeLogs in both for before/after states. The only performance hit can be found for batches with both regular and profiling logs, but given that that use case is pretty rare, we can ignore it.

Before:

Benchmark_pushLogData_10_10_1024-10    	   12537	     95205 ns/op	   90389 B/op	    1302 allocs/op
Benchmark_pushLogData_10_10_8K
Benchmark_pushLogData_10_10_8K-10      	   12655	     95779 ns/op	   90434 B/op	    1302 allocs/op
Benchmark_pushLogData_10_10_2M
Benchmark_pushLogData_10_10_2M-10      	   12558	     95977 ns/op	   90488 B/op	    1302 allocs/op
Benchmark_pushLogData_10_200_2M
Benchmark_pushLogData_10_200_2M-10     	     566	   2069811 ns/op	 1829148 B/op	   26006 allocs/op
Benchmark_pushLogData_100_200_2M
Benchmark_pushLogData_100_200_2M-10    	      57	  18608189 ns/op	18349427 B/op	  259920 allocs/op
Benchmark_pushLogData_100_200_5M
Benchmark_pushLogData_100_200_5M-10    	      56	  18389893 ns/op	18463404 B/op	  259919 allocs/op

After:

Benchmark_pushLogData_10_10_1024
Benchmark_pushLogData_10_10_1024-10    	   12786	     93612 ns/op	   90023 B/op	    1299 allocs/op
Benchmark_pushLogData_10_10_8K
Benchmark_pushLogData_10_10_8K-10      	   12909	     93867 ns/op	   90060 B/op	    1299 allocs/op
Benchmark_pushLogData_10_10_2M
Benchmark_pushLogData_10_10_2M-10      	   12782	     94140 ns/op	   90085 B/op	    1299 allocs/op
Benchmark_pushLogData_10_200_2M
Benchmark_pushLogData_10_200_2M-10     	     577	   2069041 ns/op	 1824640 B/op	   26003 allocs/op
Benchmark_pushLogData_100_200_2M
Benchmark_pushLogData_100_200_2M-10    	      58	  18362833 ns/op	18347102 B/op	  259916 allocs/op
Benchmark_pushLogData_100_200_5M
Benchmark_pushLogData_100_200_5M-10    	      56	  18378540 ns/op	18687600 B/op	  259916 allocs/op

@dmitryax dmitryax requested a review from a team May 14, 2023 05:27
@dmitryax dmitryax requested a review from atoulme as a code owner May 14, 2023 05:27
To simplify the logic and make future improvements possible. Benchmarks shows no performance degradation for typical use cases (regular logs or profiling only) with small improvement on memory allocation. Benchmarks were adjusted to be applied on `ConsumeLogs` in both for before/after states. The only performance hit can be found for batches with both regular and profiling logs, but given that that use case is pretty rare, we can ignore it.
@dmitryax dmitryax force-pushed the splunkhec-remove-inline-splitting-logic branch from 6b282c5 to 555f82b Compare May 14, 2023 05:42
@dmitryax dmitryax merged commit d67cbca into open-telemetry:main May 14, 2023
@github-actions github-actions bot added this to the next release milestone May 14, 2023
@dmitryax dmitryax deleted the splunkhec-remove-inline-splitting-logic branch May 14, 2023 22:03
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants