Support protobuf extensions #11
Comments
|
Any update here? |
|
We've since landed extension support in Protozero, a way to use custom extension fields inside TrackEvents and support for bundling proto extension descriptors together with the trace so that Trace Processor and Perfetto UI can make sense of the custom types. So, in theory all the pieces are available, but perhaps the most important part is still work in progress: documentation. Stay tuned! (@betasheet FYI) |
|
Some documentation exists in https://cs.android.com/android/platform/superproject/+/master:external/perfetto/docs/design-docs/extensions.md, but probably needs updating :) @ddrone FYI |
primiano
pushed a commit
that referenced
this issue
Jan 12, 2023
TSAN reported the problem below on a CI run. There's a possible sequence of events that can lead to this race (I'm not sure it's the only one): * The perfetto SDK thread calls the OnStop callback. * The perfetto SDK thread is now inside WaitableTestEvent::Notify(). * The perfetto SDK thread sets notified_ to true and it unlocks the mutex, but does not call `cv_.notify_one()`. * The main thread executes `WaitableTestEvent::Wait()`, which at this point can return. * The main thread finishes executing the test and destroys the tracing sessions, which destroys the WaitableTestEvent and therefore `cv_`. * The perfetto SDK thread resumes execution and tries to execute `cv_.notify_one()`, but that races with `cv_` destructor. In order to prevent this race, we can just call `cv_.notify_one()` under the lock. It might be less efficient, but it's correct. This reverts part of https://r.android.com/1373304 https://ci.perfetto.dev/#!/logs/20221219192758--cls-2345694-4--linux-clang-x86_64-tsan ``` [00:07:40] [ RUN ] PerfettoApiTest/PerfettoApiTest.TrackEventObserver_TwoDataSources/System [00:07:40] [901.194] ing_service_impl.cc:980 Configured tracing session 1, #sources:1, duration:500 ms, #buffers:1, total buffer size:1024 KB, total sessions:1, uid:1337 session name: "" [00:07:41] [901.697] ng_service_impl.cc:1921 FlushAndDisableTracing(1) done, success=1 [00:07:41] [901.698] ing_service_impl.cc:980 Configured tracing session 2, #sources:1, duration:500 ms, #buffers:1, total buffer size:1024 KB, total sessions:2, uid:1337 session name: "" [00:07:41] [902.200] ng_service_impl.cc:1921 FlushAndDisableTracing(2) done, success=1 [00:07:42] ================== [00:07:42] WARNING: ThreadSanitizer: data race (pid=7841) [00:07:42] Write of size 8 at 0x7b2000019c40 by main thread: [00:07:42] #0 pthread_cond_destroy ??:? (perfetto_integrationtests+0x315f8d) (BuildId: 2d1df55c4bda2cdd) [00:07:42] #1 std::Cr::condition_variable::~condition_variable() ??:? (libc++.so+0xad729) (BuildId: bea413d45487d5c2) [00:07:42] #2 (anonymous namespace)::PerfettoApiTest::TearDown() api_integrationtest.cc:? (perfetto_integrationtests+0x398609) (BuildId: 2d1df55c4bda2cdd) [00:07:42] #3 testing::Test::Run() ??:? (perfetto_integrationtests+0x8de94f) (BuildId: 2d1df55c4bda2cdd) [00:07:42] #4 testing::TestInfo::Run() ??:? (perfetto_integrationtests+0x8e0a2c) (BuildId: 2d1df55c4bda2cdd) [00:07:42] #5 testing::TestSuite::Run() ??:? (perfetto_integrationtests+0x8e1b14) (BuildId: 2d1df55c4bda2cdd) [00:07:42] #6 testing::internal::UnitTestImpl::RunAllTests() ??:? (perfetto_integrationtests+0x8f9e3f) (BuildId: 2d1df55c4bda2cdd) [00:07:42] #7 testing::UnitTest::Run() ??:? (perfetto_integrationtests+0x8f8d36) (BuildId: 2d1df55c4bda2cdd) [00:07:42] #8 main api_integrationtest_main.cc:? (perfetto_integrationtests+0x4d7aa4) (BuildId: 2d1df55c4bda2cdd) [00:07:42] [00:07:42] Previous read of size 8 at 0x7b2000019c40 by thread T3: [00:07:42] #0 pthread_cond_signal ??:? (perfetto_integrationtests+0x315c28) (BuildId: 2d1df55c4bda2cdd) [00:07:42] #1 std::Cr::condition_variable::notify_one() ??:? (libc++.so+0xad479) (BuildId: bea413d45487d5c2) [00:07:42] #2 void std::Cr::__function::__policy_invoker<void ()>::__call_impl<std::Cr::__function::__default_alloc_func<(anonymous namespace)::PerfettoApiTest::NewTrace(perfetto::protos::gen::TraceConfig const&, perfetto::BackendType, int)::{lambda()#1}, void ()> >(std::Cr::__function::__policy_storage const*) api_integrationtest.cc:? (perfetto_integrationtests+0x39ba0a) (BuildId: 2d1df55c4bda2cdd) [00:07:42] #3 perfetto::base::UnixTaskRunner::RunImmediateAndDelayedTask() unix_task_runner.cc:? (perfetto_integrationtests+0x594cfd) (BuildId: 2d1df55c4bda2cdd) [00:07:42] #4 perfetto::base::UnixTaskRunner::Run() unix_task_runner.cc:? (perfetto_integrationtests+0x594133) (BuildId: 2d1df55c4bda2cdd) [00:07:42] #5 perfetto::base::ThreadTaskRunner::RunTaskThread(std::Cr::function<void (perfetto::base::UnixTaskRunner*)>) thread_task_runner.cc:? (perfetto_integrationtests+0x592e4f) (BuildId: 2d1df55c4bda2cdd) [00:07:42] #6 void* std::Cr::__thread_proxy[abi:v160000]<std::Cr::tuple<std::Cr::unique_ptr<std::Cr::__thread_struct, std::Cr::default_delete<std::Cr::__thread_struct> >, void (perfetto::base::ThreadTaskRunner::*)(std::Cr::function<void (perfetto::base::UnixTaskRunner*)>), perfetto::base::ThreadTaskRunner*, std::Cr::function<void (perfetto::base::UnixTaskRunner*)> > >(void*) thread_task_runner.cc:? (perfetto_integrationtests+0x59380b) (BuildId: 2d1df55c4bda2cdd) [00:07:42] [00:07:42] Location is heap block of size 120 at 0x7b2000019c00 allocated by main thread: [00:07:42] #0 operator new(unsigned long) ??:? (perfetto_integrationtests+0x3941a7) (BuildId: 2d1df55c4bda2cdd) [00:07:42] #1 (anonymous namespace)::PerfettoApiTest::NewTrace(perfetto::protos::gen::TraceConfig const&, int) api_integrationtest.cc:? (perfetto_integrationtests+0x39b864) (BuildId: 2d1df55c4bda2cdd) [00:07:42] #2 (anonymous namespace)::PerfettoApiTest::NewTraceWithCategories(std::Cr::vector<std::Cr::basic_string<char, std::Cr::char_traits<char>, std::Cr::allocator<char> >, std::Cr::allocator<std::Cr::basic_string<char, std::Cr::char_traits<char>, std::Cr::allocator<char> > > >, perfetto::protos::gen::TrackEventConfig, perfetto::protos::gen::TraceConfig) api_integrationtest.cc:? (perfetto_integrationtests+0x3a6385) (BuildId: 2d1df55c4bda2cdd) [00:07:42] #3 (anonymous namespace)::PerfettoApiTest_TrackEventObserver_TwoDataSources_Test::TestBody() api_integrationtest.cc:? (perfetto_integrationtests+0x4a61d2) (BuildId: 2d1df55c4bda2cdd) [00:07:42] #4 testing::Test::Run() ??:? (perfetto_integrationtests+0x8de8c3) (BuildId: 2d1df55c4bda2cdd) [00:07:42] #5 testing::TestInfo::Run() ??:? (perfetto_integrationtests+0x8e0a2c) (BuildId: 2d1df55c4bda2cdd) [00:07:42] #6 testing::TestSuite::Run() ??:? (perfetto_integrationtests+0x8e1b14) (BuildId: 2d1df55c4bda2cdd) [00:07:42] #7 testing::internal::UnitTestImpl::RunAllTests() ??:? (perfetto_integrationtests+0x8f9e3f) (BuildId: 2d1df55c4bda2cdd) [00:07:42] #8 testing::UnitTest::Run() ??:? (perfetto_integrationtests+0x8f8d36) (BuildId: 2d1df55c4bda2cdd) [00:07:42] #9 main api_integrationtest_main.cc:? (perfetto_integrationtests+0x4d7aa4) (BuildId: 2d1df55c4bda2cdd) [00:07:42] [00:07:42] Thread T3 'TracingMuxer' (tid=7845, running) created by main thread at: [00:07:42] #0 pthread_create ??:? (perfetto_integrationtests+0x3147ab) (BuildId: 2d1df55c4bda2cdd) [00:07:42] #1 perfetto::base::ThreadTaskRunner::ThreadTaskRunner(std::Cr::basic_string<char, std::Cr::char_traits<char>, std::Cr::allocator<char> > const&) thread_task_runner.cc:? (perfetto_integrationtests+0x592b80) (BuildId: 2d1df55c4bda2cdd) [00:07:42] #2 perfetto::(anonymous namespace)::PlatformPosix::CreateTaskRunner(perfetto::Platform::CreateTaskRunnerArgs const&) platform_posix.cc:? (perfetto_integrationtests+0x8c524a) (BuildId: 2d1df55c4bda2cdd) [00:07:42] #3 perfetto::internal::TracingMuxerImpl::TracingMuxerImpl(perfetto::TracingInitArgs const&) tracing_muxer_impl.cc:? (perfetto_integrationtests+0x5a731c) (BuildId: 2d1df55c4bda2cdd) [00:07:42] #4 perfetto::internal::TracingMuxerImpl::InitializeInstance(perfetto::TracingInitArgs const&) tracing_muxer_impl.cc:? (perfetto_integrationtests+0x5ad7fb) (BuildId: 2d1df55c4bda2cdd) [00:07:42] #5 perfetto::Tracing::InitializeInternal(perfetto::TracingInitArgs const&) tracing.cc:? (perfetto_integrationtests+0x5c0f0d) (BuildId: 2d1df55c4bda2cdd) [00:07:42] #6 (anonymous namespace)::ConcurrentSessionTest_ConcurrentBackends_Test::TestBody() api_integrationtest.cc:? (perfetto_integrationtests+0x4c3af4) (BuildId: 2d1df55c4bda2cdd) [00:07:42] #7 testing::Test::Run() ??:? (perfetto_integrationtests+0x8de8c3) (BuildId: 2d1df55c4bda2cdd) [00:07:42] #8 testing::TestInfo::Run() ??:? (perfetto_integrationtests+0x8e0a2c) (BuildId: 2d1df55c4bda2cdd) [00:07:42] #9 testing::TestSuite::Run() ??:? (perfetto_integrationtests+0x8e1b14) (BuildId: 2d1df55c4bda2cdd) [00:07:42] #10 testing::internal::UnitTestImpl::RunAllTests() ??:? (perfetto_integrationtests+0x8f9e3f) (BuildId: 2d1df55c4bda2cdd) [00:07:42] #11 testing::UnitTest::Run() ??:? (perfetto_integrationtests+0x8f8d36) (BuildId: 2d1df55c4bda2cdd) [00:07:42] #12 main api_integrationtest_main.cc:? (perfetto_integrationtests+0x4d7aa4) (BuildId: 2d1df55c4bda2cdd) [00:07:42] [00:07:42] SUMMARY: ThreadSanitizer: data race ??:? in __interceptor_pthread_cond_destroy [00:07:42] ================== ``` Change-Id: Ic822dba258b81a6c032633cf63763bd578739024
primiano
pushed a commit
that referenced
this issue
Sep 17, 2024
Some of our integration tests are randomly failing in CI with:
```
[00:16:35] [ RUN ] Run/HeapprofdEndToEnd.ConcurrentSession/StaticModeCustom
[00:16:35] [810.227] approfd_producer.cc:255 Disconnected from tracing service
[00:16:35] [810.428] approfd_producer.cc:255 Disconnected from tracing service
[00:16:35] [810.473] ng_service_impl.cc:1125 Configured tracing session 1, #sources:1, duration:5000 ms, #buffers:1, total buffer size:10240 KB, total sessions:1, uid:1337 session name: ""
[00:16:35] [810.829] approfd_producer.cc:245 Connected to the service, mode [child].
[00:16:35] [810.830] approfd_producer.cc:367 Setting up datasource: non-statsd initiator.
[00:16:36] [811.472] end_to_end_test.cc:1706 Starting concurrent.
[00:16:36] [811.473] paged_memory.cc:69 PERFETTO_CHECK(ptr && ptr != MAP_FAILED) (errno: 12, Cannot allocate memory)
[00:16:36]
[00:16:36] -----BEGIN PERFETTO PRE-CRASH LOG-----
[00:16:36] trace_uuid: ccd77e58-4684-4fcd-ce84-d54022f
[00:16:36] [810.079] ng_service_impl.cc:2102 FlushAndDisableTracing(2) done, success=1
[00:16:36] [810.139] _end_to_end_test.cc:726 Stats for 16809: unwinding_errors: 0?heap_samples: 198?map_reparses: 0?unwinding_time_us: [-inf, 1]: 0; [1, 2]: 0; [2, 4]: 0; [4, 8]: 0; [8, 16]: 98; [16, 32]: 96; [32, 64]: 3; [64, 128]: 0; [128, 256]: 0; [256, 512]: 0;
[00:16:36] [810.139] _end_to_end_test.cc:842 Shutting down profile target.
[00:16:36] [810.147] ng_service_impl.cc:2713 Tracing session 2 ended, total sessions:1
[00:16:36] [810.156] ng_service_impl.cc:2713 Tracing session 1 ended, total sessions:0
[00:16:36] [810.473] ng_service_impl.cc:1125 Configured tracing session 1, #sources:1, duration:5000 ms, #buffers:1, total buffer size:10240 KB, total sessions:1, uid:1337 session name: ""
[00:16:36] [811.472] end_to_end_test.cc:1706 Starting concurrent.
[00:16:36] [811.473] paged_memory.cc:69 PERFETTO_CHECK(ptr && ptr != MAP_FAILED) (errno: 12, Cannot allocate memory)
[00:16:36]
[00:16:36] -----END PERFETTO PRE-CRASH LOG-----
[00:16:36] [811.661] approfd_producer.cc:255 Disconnected from tracing service
[00:16:36] test/ci/linux_tests.sh: line 32: 16041 Illegal instruction (core dumped) ${OUT_PATH}/perfetto_integrationtests
```
It turns out this as little to do with heapprofd, it's just that mmap is
not able to allocate memory in trace_processor most of the times:
```
[00:02:56] AddressSanitizer: CHECK failed: sanitizer_common.cpp:61 "((0 && "unable to mmap")) != (0)" (0x0, 0x0) (tid=13072)
[00:02:56] #0 0x56e38cb2 in __asan::CheckUnwind() /b/s/w/ir/cache/builder/src/third_party/llvm/compiler-rt/lib/asan/asan_rtl.cpp:69:3
[00:02:56] #1 0x56e4d5bb in __sanitizer::CheckFailed(char const*, int, char const*, unsigned long long, unsigned long long) /b/s/w/ir/cache/builder/src/third_party/llvm/compiler-rt/lib/sanitizer_common/sanitizer_termination.cpp:86:5
[00:02:56] #2 0x56e3f580 in __sanitizer::ReportMmapFailureAndDie(unsigned int, char const*, char const*, int, bool) /b/s/w/ir/cache/builder/src/third_party/llvm/compiler-rt/lib/sanitizer_common/sanitizer_common.cpp:61:3
[00:02:56] #3 0x56e4640b in __sanitizer::MmapOrDieOnFatalError(unsigned int, char const*) /b/s/w/ir/cache/builder/src/third_party/llvm/compiler-rt/lib/sanitizer_common/sanitizer_posix.cpp:74:5
[00:02:56] #4 0x56da6fdd in __sanitizer::LargeMmapAllocator<__asan::AsanMapUnmapCallback, __sanitizer::LargeMmapAllocatorPtrArrayStatic, __sanitizer::LocalAddressSpaceView>::Allocate(__sanitizer::AllocatorStats*, unsigned int, unsigned int) /b/s/w/ir/cache/builder/src/third_party/llvm/compiler-rt/lib/asan/../sanitizer_common/sanitizer_allocator_secondary.h:98:9
[00:02:56] #5 0x56da6e19 in __sanitizer::CombinedAllocator<__sanitizer::SizeClassAllocator32<__asan::AP32<__sanitizer::LocalAddressSpaceView>>, __sanitizer::LargeMmapAllocatorPtrArrayStatic>::Allocate(__sanitizer::SizeClassAllocator32LocalCache<__sanitizer::SizeClassAllocator32<__asan::AP32<__sanitizer::LocalAddressSpaceView>>>*, unsigned int, unsigned int) /b/s/w/ir/cache/builder/src/third_party/llvm/compiler-rt/lib/asan/../sanitizer_common/sanitizer_allocator_combined.h:71:24
[00:02:56] #6 0x56da2b5b in __asan::Allocator::Allocate(unsigned int, unsigned int, __sanitizer::BufferedStackTrace*, __asan::AllocType, bool) /b/s/w/ir/cache/builder/src/third_party/llvm/compiler-rt/lib/asan/asan_allocator.cpp:582:29
[00:02:56] #7 0x56da3668 in __asan::asan_memalign(unsigned int, unsigned int, __sanitizer::BufferedStackTrace*, __asan::AllocType) /b/s/w/ir/cache/builder/src/third_party/llvm/compiler-rt/lib/asan/asan_allocator.cpp:1060:16
[00:02:56] #8 0x56e657a7 in operator new(unsigned int) /b/s/w/ir/cache/builder/src/third_party/llvm/compiler-rt/lib/asan/asan_new_delete.cpp:86:3
[00:02:56] #9 0xf7dc6a49 in void* std::__Cr::__libcpp_operator_new<unsigned int>(unsigned int) /ci/ramdisk/src/out/dist/../../buildtools/libcxx/include/new:270:10
[00:02:56] #10 0xf7dc6a49 in std::__Cr::__libcpp_allocate(unsigned int, unsigned int) /ci/ramdisk/src/out/dist/../../buildtools/libcxx/include/new:294:10
[00:02:56] #11 0xf7dc6a49 in std::__Cr::allocator<char>::allocate(unsigned int) /ci/ramdisk/src/out/dist/../../buildtools/libcxx/include/__memory/allocator.h:119:32
[00:02:56] #12 0xf7dc6a49 in std::__Cr::__allocation_result<std::__Cr::allocator_traits<std::__Cr::allocator<char>>::pointer> std::__Cr::__allocate_at_least<std::__Cr::allocator<char>>(std::__Cr::allocator<char>&, unsigned int) /ci/ramdisk/src/out/dist/../../buildtools/libcxx/include/__memory/allocate_at_least.h:41:19
[00:02:56] #13 0xf7dc6a49 in std::__Cr::basic_string<char, std::__Cr::char_traits<char>, std::__Cr::allocator<char>>::__grow_by(unsigned int, unsigned int, unsigned int, unsigned int, unsigned int, unsigned int) /ci/ramdisk/src/out/dist/../../buildtools/libcxx/include/string:2426:23
```
I've looked at the virtual address space and it's very fragmented, the
ASAN allocator seems to leave around a lot of 1MB sections.
I'm assuming that ASAN on 32-bit devices is harder to support because of
the limited amount of address space.
We don't really need 32-bit coverage for ASAN, we have 64-bit coverage
for ASAN (and 32-bit ASAN on android is becoming less relevant with
HWASAN).
Let's use this configuration to have some coverage for a 32-bit build.
Change-Id: I02ed354757874dd51b0c1c2b739d91e3c6584119
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
We should support protobuf extensions, so folks can define their own DataSource messages and TrackEvent arguments without having to check them in to Perfetto directly.
The text was updated successfully, but these errors were encountered: