Client Initialization may be blocked indefinitely #36
Labels
Comments
|
interesting, isn't there another thread calling I reckon the The other threads does not appear to be involved in this. |
|
Nope, that is the only thread. This was stuck in the build, i logged into to server and got stack trace for the process and put the stack traces of the threads. Shall research further. Something seems very innocent, who would think such a name resolution call would acquire a lock :) |
|
This is a duplicate of issue #18 |
ihsandemir
added a commit
to ihsandemir/hazelcast-cpp-client
that referenced
this issue
Apr 30, 2021
…be cancelled and underlying ringbuffer should be destroyed. Failing to do this, causes the runner continue working and it is possible that the lambda function in `MessageRunner::next` method access memory freed after shutdown which is the user executor pool resource and cause the `AddressSanitizer: heap-use-after-free` error with address sanitizer.
```
30/04/2021 13:38:16.296 INFO: [139693858485824] hz.client_1[dev] [4.1.0] (Fri Apr 30 05:10:59 2021 +0300:2b76000e5) LifecycleService::LifecycleEvent STARTING
30/04/2021 13:38:16.296 INFO: [139693858485824] hz.client_1[dev] [4.1.0] LifecycleService::LifecycleEvent STARTED
30/04/2021 13:38:16.307 INFO: [139693858485824] hz.client_1[dev] [4.1.0] Trying to connect to Address[127.0.0.1:5701]. Translated address: Address[127.0.0.1:5701].
Apr 30, 2021 1:38:16 PM com.hazelcast.client.impl.protocol.task.AuthenticationMessageTask
INFO: [localhost]:5701 [dev] [4.1.4-SNAPSHOT] Received auth from Connection[id=1, /127.0.0.1:5701->/127.0.0.1:52796, qualifier=null, endpoint=[127.0.0.1]:52796, alive=true, connectionType=CPP, planeIndex=-1], successfully authenticated, clientUuid: 123014ad-93cd-4fa4-a684-5ebb14e08b0f, client version: 4.1.0
30/04/2021 13:38:16.335 INFO: [139693858485824] hz.client_1[dev] [4.1.0] LifecycleService::LifecycleEvent CLIENT_CONNECTED
30/04/2021 13:38:16.336 INFO: [139693858485824] hz.client_1[dev] [4.1.0] Authenticated with server Address[localhost:5701]:393768ad-d085-42d7-80f3-19f4e73c62ec, server version: 4.1.4-SNAPSHOT, local address: Address[127.0.0.1:52796]. ClientConnection{alive=1, connectionId=1, remoteEndpoint=Address[localhost:5701], lastReadTime=2021-04-30 13:38:16.0-1, closedTime=never, connected server version=4.1.4-SNAPSHOT}
30/04/2021 13:38:16.340 INFO: [139693420803648] hz.client_1[dev] [4.1.0]
Members [1] {
Member[localhost]:5701 - 393768ad-d085-42d7-80f3-19f4e73c62ec
}
[ RUN ] ReliableTopicTest.testBasics
Apr 30, 2021 1:38:16 PM com.hazelcast.internal.partition.impl.PartitionStateManager
INFO: [localhost]:5701 [dev] [4.1.4-SNAPSHOT] Initializing cluster partition table arrangement...
[ OK ] ReliableTopicTest.testBasics (62 ms)
[ RUN ] ReliableTopicTest.testListenerSequence
[ OK ] ReliableTopicTest.testListenerSequence (9 ms)
[ RUN ] ReliableTopicTest.removeMessageListener_whenExisting
[ OK ] ReliableTopicTest.removeMessageListener_whenExisting (2008 ms)
[ RUN ] ReliableTopicTest.removeMessageListener_whenNonExisting
[ OK ] ReliableTopicTest.removeMessageListener_whenNonExisting (5 ms)
[ RUN ] ReliableTopicTest.publishMultiple
[ OK ] ReliableTopicTest.publishMultiple (13 ms)
[ RUN ] ReliableTopicTest.testConfig
30/04/2021 13:38:18.443 INFO: [139693858485824] hz.client_2[dev] [4.1.0] (Fri Apr 30 05:10:59 2021 +0300:2b76000e5) LifecycleService::LifecycleEvent Client (01a9971f-9da1-4182-9132-2cba5e70d255) is STARTING
30/04/2021 13:38:18.444 INFO: [139693858485824] hz.client_2[dev] [4.1.0] (Fri Apr 30 05:10:59 2021 +0300:2b76000e5) LifecycleService::LifecycleEvent STARTING
30/04/2021 13:38:18.444 INFO: [139693858485824] hz.client_2[dev] [4.1.0] LifecycleService::LifecycleEvent STARTED
30/04/2021 13:38:18.456 INFO: [139693858485824] hz.client_2[dev] [4.1.0] Trying to connect to Address[127.0.0.1:5701]. Translated address: Address[127.0.0.1:5701].
Apr 30, 2021 1:38:18 PM com.hazelcast.client.impl.protocol.task.AuthenticationMessageTask
INFO: [localhost]:5701 [dev] [4.1.4-SNAPSHOT] Received auth from Connection[id=2, /127.0.0.1:5701->/127.0.0.1:52798, qualifier=null, endpoint=[127.0.0.1]:52798, alive=true, connectionType=CPP, planeIndex=-1], successfully authenticated, clientUuid: 01a9971f-9da1-4182-9132-2cba5e70d255, client version: 4.1.0
30/04/2021 13:38:18.459 INFO: [139693858485824] hz.client_2[dev] [4.1.0] LifecycleService::LifecycleEvent CLIENT_CONNECTED
30/04/2021 13:38:18.459 INFO: [139693858485824] hz.client_2[dev] [4.1.0] Authenticated with server Address[localhost:5701]:393768ad-d085-42d7-80f3-19f4e73c62ec, server version: 4.1.4-SNAPSHOT, local address: Address[127.0.0.1:52798]. ClientConnection{alive=1, connectionId=1, remoteEndpoint=Address[localhost:5701], lastReadTime=2021-04-30 13:38:18.0-1, closedTime=never, connected server version=4.1.4-SNAPSHOT}
30/04/2021 13:38:18.461 INFO: [139692834874944] hz.client_2[dev] [4.1.0]
Members [1] {
Member[localhost]:5701 - 393768ad-d085-42d7-80f3-19f4e73c62ec
}
30/04/2021 13:38:18.473 INFO: [139693915061760] hz.client_2[dev] [4.1.0] LifecycleService::LifecycleEvent SHUTTING_DOWN
30/04/2021 13:38:18.473 INFO: [139693915061760] hz.client_2[dev] [4.1.0] Removed connection to endpoint: Address[localhost:5701], connection: ClientConnection{alive=0, connectionId=1, remoteEndpoint=Address[localhost:5701], lastReadTime=2021-04-30 13:38:18.0-1, closedTime=2021-04-30 13:38:18.000, connected server version=4.1.4-SNAPSHOT}
30/04/2021 13:38:18.473 INFO: [139693915061760] hz.client_2[dev] [4.1.0] LifecycleService::LifecycleEvent CLIENT_DISCONNECTED
Apr 30, 2021 1:38:18 PM com.hazelcast.internal.server.tcp.TcpServerConnection
INFO: [localhost]:5701 [dev] [4.1.4-SNAPSHOT] Connection[id=2, /127.0.0.1:5701->/127.0.0.1:52798, qualifier=null, endpoint=[127.0.0.1]:52798, alive=false, connectionType=CPP, planeIndex=-1] closed. Reason: Connection closed by the other side
30/04/2021 13:38:18.476 INFO: [139693915061760] hz.client_2[dev] [4.1.0] LifecycleService::LifecycleEvent SHUTDOWN
[ OK ] ReliableTopicTest.testConfig (40 ms)
[ RUN ] ReliableTopicTest.testMessageFieldSetCorrectly
=================================================================
==5793==ERROR: AddressSanitizer: heap-use-after-free on address 0x602000040c50 at pc 0x7f0d080b16fe bp 0x7f0cde11d250 sp 0x7f0cde11d248
READ of size 8 at 0x602000040c50 thread T141
[ OK ] ReliableTopicTest.testMessageFieldSetCorrectly (9 ms)
[ RUN ] ReliableTopicTest.testAlwaysStartAfterTail
#0 0x7f0d080b16fd in std::__uniq_ptr_impl<boost::asio::thread_pool, std::default_delete<boost::asio::thread_pool> >::_M_ptr() const /usr/include/c++/10/bits/unique_ptr.h:173
#1 0x7f0d080ad84b in std::unique_ptr<boost::asio::thread_pool, std::default_delete<boost::asio::thread_pool> >::get() const /usr/include/c++/10/bits/unique_ptr.h:422
#2 0x7f0d080a9f6d in std::unique_ptr<boost::asio::thread_pool, std::default_delete<boost::asio::thread_pool> >::operator*() const /usr/include/c++/10/bits/unique_ptr.h:408
hazelcast#3 0x7f0d080a3e35 in hazelcast::util::hz_thread_pool::submit(boost::detail::nullary_function<void ()>&&) /hazelcast-cpp-client/hazelcast/src/hazelcast/util/util.cpp:1094
hazelcast#4 0x114ec6d in boost::executors::executor_ref<hazelcast::util::hz_thread_pool>::submit(boost::detail::nullary_function<void ()>&&) /usr/local/include/boost/thread/executors/generic_executor_ref.hpp:73
hazelcast#5 0x1156578 in void boost::executors::executor::submit<boost::detail::run_it<boost::detail::continuation_shared_state<boost::future<hazelcast::client::rb::read_result_set>, void, hazelcast::client::reliable_topic::MessageRunner<hazelcast::client::topic::reliable_listener>::next()::{lambda(boost::future<hazelcast::client::rb::read_result_set>)#1}, boost::detail::shared_state<void> > > >(boost::detail::run_it<boost::detail::continuation_shared_state<boost::future<hazelcast::client::rb::read_result_set>, void, hazelcast::client::reliable_topic::MessageRunner<hazelcast::client::topic::reliable_listener>::next()::{lambda(boost::future<hazelcast::client::rb::read_result_set>)#1}, boost::detail::shared_state<void> > >&&) /usr/local/include/boost/thread/executors/executor.hpp:107
hazelcast#6 0x11502c5 in boost::detail::future_executor_continuation_shared_state<boost::future<hazelcast::client::rb::read_result_set>, void, hazelcast::client::reliable_topic::MessageRunner<hazelcast::client::topic::reliable_listener>::next()::{lambda(boost::future<hazelcast::client::rb::read_result_set>)#1}>::launch_continuation() /usr/local/include/boost/thread/future.hpp:4711
hazelcast#7 0xa0915e in boost::detail::shared_state<hazelcast::client::rb::read_result_set>::do_continuation(boost::unique_lock<boost::mutex>&) /usr/local/include/boost/thread/future.hpp:591
hazelcast#8 0x851af6 in boost::detail::shared_state_base::mark_finished_internal(boost::unique_lock<boost::mutex>&) /usr/local/include/boost/thread/future.hpp:345
hazelcast#9 0x85265b in boost::detail::shared_state_base::mark_exceptional_finish_internal(boost::exception_ptr const&, boost::unique_lock<boost::mutex>&) /usr/local/include/boost/thread/future.hpp:456
hazelcast#10 0x852744 in boost::detail::shared_state_base::mark_exceptional_finish() /usr/local/include/boost/thread/future.hpp:462
hazelcast#11 0xa2c839 in boost::detail::continuation_shared_state<boost::future<hazelcast::client::protocol::ClientMessage>, hazelcast::client::rb::read_result_set, hazelcast::client::ringbuffer::read_many(long, int, int)::{lambda(boost::future<hazelcast::client::protocol::ClientMessage>)#1}, boost::detail::shared_state<hazelcast::client::rb::read_result_set> >::call() (/hazelcast-cpp-client/build/hazelcast/test/src/client_test+0xa2c839)
hazelcast#12 0xa0acd1 in boost::detail::future_sync_continuation_shared_state<boost::future<hazelcast::client::protocol::ClientMessage>, hazelcast::client::rb::read_result_set, hazelcast::client::ringbuffer::read_many(long, int, int)::{lambda(boost::future<hazelcast::client::protocol::ClientMessage>)#1}>::launch_continuation() /usr/local/include/boost/thread/future.hpp:4638
hazelcast#13 0xa13720 in boost::detail::shared_state<hazelcast::client::protocol::ClientMessage>::do_continuation(boost::unique_lock<boost::mutex>&) (/hazelcast-cpp-client/build/hazelcast/test/src/client_test+0xa13720)
hazelcast#14 0x851af6 in boost::detail::shared_state_base::mark_finished_internal(boost::unique_lock<boost::mutex>&) /usr/local/include/boost/thread/future.hpp:345
hazelcast#15 0x85265b in boost::detail::shared_state_base::mark_exceptional_finish_internal(boost::exception_ptr const&, boost::unique_lock<boost::mutex>&) /usr/local/include/boost/thread/future.hpp:456
hazelcast#16 0x852744 in boost::detail::shared_state_base::mark_exceptional_finish() /usr/local/include/boost/thread/future.hpp:462
hazelcast#17 0x7f0d07e78134 in call /usr/local/include/boost/thread/future.hpp:4504
hazelcast#18 0x7f0d07e744a1 in run /usr/local/include/boost/thread/future.hpp:4536
hazelcast#19 0x7f0d07e81c7d in invoke<void (*)(boost::shared_ptr<boost::detail::shared_state_base>), boost::shared_ptr<boost::detail::future_async_continuation_shared_state<boost::future<hazelcast::client::protocol::ClientMessage>, hazelcast::client::protocol::ClientMessage, hazelcast::client::spi::impl::ClientInvocation::invoke()::<lambda(boost::future<hazelcast::client::protocol::ClientMessage>)> > > > /usr/local/include/boost/thread/detail/invoke.hpp:134
hazelcast#20 0x7f0d07e8100a in run2<1> /usr/local/include/boost/thread/detail/thread.hpp:79
hazelcast#21 0x7f0d07e80df3 in run /usr/local/include/boost/thread/detail/thread.hpp:85
hazelcast#22 0x7f0d06c007b6 in thread_proxy (/usr/local/lib/libboost_thread.so.1.72.0+0xe7b6)
hazelcast#23 0x7f0d06bcb3f8 in start_thread (/lib64/libpthread.so.0+0x93f8)
hazelcast#24 0x7f0d067afbb2 in __GI___clone (/lib64/libc.so.6+0x101bb2)
0x602000040c50 is located 0 bytes inside of 16-byte region [0x602000040c50,0x602000040c60)
freed by thread T0 here:
#0 0x7f0d086765b7 in operator delete(void*) (/lib64/libasan.so.6+0xad5b7)
#1 0x7f0d07b128a9 in std::default_delete<hazelcast::util::hz_thread_pool>::operator()(hazelcast::util::hz_thread_pool*) const /usr/include/c++/10/bits/unique_ptr.h:85
#2 0x7f0d07b0ce0c in std::unique_ptr<hazelcast::util::hz_thread_pool, std::default_delete<hazelcast::util::hz_thread_pool> >::~unique_ptr() /usr/include/c++/10/bits/unique_ptr.h:361
hazelcast#3 0x7f0d07b0526f in hazelcast::client::spi::impl::ClientExecutionServiceImpl::~ClientExecutionServiceImpl() (/hazelcast-cpp-client/build/libhazelcast-cpp-client.so.4.1.0+0xce826f)
hazelcast#4 0x7f0d07b052ab in void __gnu_cxx::new_allocator<hazelcast::client::spi::impl::ClientExecutionServiceImpl>::destroy<hazelcast::client::spi::impl::ClientExecutionServiceImpl>(hazelcast::client::spi::impl::ClientExecutionServiceImpl*) (/hazelcast-cpp-client/build/libhazelcast-cpp-client.so.4.1.0+0xce82ab)
hazelcast#5 0x7f0d07af8c7a in void std::allocator_traits<std::allocator<hazelcast::client::spi::impl::ClientExecutionServiceImpl> >::destroy<hazelcast::client::spi::impl::ClientExecutionServiceImpl>(std::allocator<hazelcast::client::spi::impl::ClientExecutionServiceImpl>&, hazelcast::client::spi::impl::ClientExecutionServiceImpl*) /usr/include/c++/10/bits/alloc_traits.h:531
hazelcast#6 0x7f0d07aebac0 in std::_Sp_counted_ptr_inplace<hazelcast::client::spi::impl::ClientExecutionServiceImpl, std::allocator<hazelcast::client::spi::impl::ClientExecutionServiceImpl>, (__gnu_cxx::_Lock_policy)2>::_M_dispose() (/hazelcast-cpp-client/build/libhazelcast-cpp-client.so.4.1.0+0xcceac0)
hazelcast#7 0x8aad08 in std::_Sp_counted_base<(__gnu_cxx::_Lock_policy)2>::_M_release() /usr/include/c++/10/bits/shared_ptr_base.h:158
hazelcast#8 0x872171 in std::__shared_count<(__gnu_cxx::_Lock_policy)2>::~__shared_count() /usr/include/c++/10/bits/shared_ptr_base.h:733
hazelcast#9 0xd002a3 in std::__shared_ptr<hazelcast::client::spi::impl::ClientExecutionServiceImpl, (__gnu_cxx::_Lock_policy)2>::~__shared_ptr() /usr/include/c++/10/bits/shared_ptr_base.h:1183
hazelcast#10 0xd002e5 in std::shared_ptr<hazelcast::client::spi::impl::ClientExecutionServiceImpl>::~shared_ptr() /usr/include/c++/10/bits/shared_ptr.h:121
hazelcast#11 0x7f0d07e2d152 in hazelcast::client::spi::impl::ClientInvocation::~ClientInvocation() /hazelcast-cpp-client/hazelcast/src/hazelcast/client/spi.cpp:1045
hazelcast#12 0x7f0d07e2d17d in hazelcast::client::spi::impl::ClientInvocation::~ClientInvocation() /hazelcast-cpp-client/hazelcast/src/hazelcast/client/spi.cpp:1045
hazelcast#13 0x7f0d07ef4487 in std::_Sp_counted_ptr<hazelcast::client::spi::impl::ClientInvocation*, (__gnu_cxx::_Lock_policy)2>::_M_dispose() /usr/include/c++/10/bits/shared_ptr_base.h:380
hazelcast#14 0x8aad08 in std::_Sp_counted_base<(__gnu_cxx::_Lock_policy)2>::_M_release() /usr/include/c++/10/bits/shared_ptr_base.h:158
hazelcast#15 0x872171 in std::__shared_count<(__gnu_cxx::_Lock_policy)2>::~__shared_count() /usr/include/c++/10/bits/shared_ptr_base.h:733
hazelcast#16 0x10102a7 in std::__shared_ptr<hazelcast::client::spi::impl::ClientInvocation, (__gnu_cxx::_Lock_policy)2>::~__shared_ptr() /usr/include/c++/10/bits/shared_ptr_base.h:1183
hazelcast#17 0x10102c3 in std::shared_ptr<hazelcast::client::spi::impl::ClientInvocation>::~shared_ptr() /usr/include/c++/10/bits/shared_ptr.h:121
hazelcast#18 0x7f0d07c37715 in std::pair<long const, std::shared_ptr<hazelcast::client::spi::impl::ClientInvocation> >::~pair() /usr/include/c++/10/bits/stl_pair.h:211
hazelcast#19 0x7f0d07c37735 in void __gnu_cxx::new_allocator<std::__detail::_Hash_node<std::pair<long const, std::shared_ptr<hazelcast::client::spi::impl::ClientInvocation> >, false> >::destroy<std::pair<long const, std::shared_ptr<hazelcast::client::spi::impl::ClientInvocation> > >(std::pair<long const, std::shared_ptr<hazelcast::client::spi::impl::ClientInvocation> >*) /usr/include/c++/10/ext/new_allocator.h:156
hazelcast#20 0x7f0d07c311ca in void std::allocator_traits<std::allocator<std::__detail::_Hash_node<std::pair<long const, std::shared_ptr<hazelcast::client::spi::impl::ClientInvocation> >, false> > >::destroy<std::pair<long const, std::shared_ptr<hazelcast::client::spi::impl::ClientInvocation> > >(std::allocator<std::__detail::_Hash_node<std::pair<long const, std::shared_ptr<hazelcast::client::spi::impl::ClientInvocation> >, false> >&, std::pair<long const, std::shared_ptr<hazelcast::client::spi::impl::ClientInvocation> >*) /usr/include/c++/10/bits/alloc_traits.h:531
hazelcast#21 0x7f0d07c295ea in std::__detail::_Hashtable_alloc<std::allocator<std::__detail::_Hash_node<std::pair<long const, std::shared_ptr<hazelcast::client::spi::impl::ClientInvocation> >, false> > >::_M_deallocate_node(std::__detail::_Hash_node<std::pair<long const, std::shared_ptr<hazelcast::client::spi::impl::ClientInvocation> >, false>*) /usr/include/c++/10/bits/hashtable_policy.h:2053
hazelcast#22 0x7f0d07c20d19 in std::__detail::_Hashtable_alloc<std::allocator<std::__detail::_Hash_node<std::pair<long const, std::shared_ptr<hazelcast::client::spi::impl::ClientInvocation> >, false> > >::_M_deallocate_nodes(std::__detail::_Hash_node<std::pair<long const, std::shared_ptr<hazelcast::client::spi::impl::ClientInvocation> >, false>*) (/hazelcast-cpp-client/build/libhazelcast-cpp-client.so.4.1.0+0xe03d19)
hazelcast#23 0x7f0d07c13fc1 in std::_Hashtable<long, std::pair<long const, std::shared_ptr<hazelcast::client::spi::impl::ClientInvocation> >, std::allocator<std::pair<long const, std::shared_ptr<hazelcast::client::spi::impl::ClientInvocation> > >, std::__detail::_Select1st, std::equal_to<long>, std::hash<long>, std::__detail::_Mod_range_hashing, std::__detail::_Default_ranged_hash, std::__detail::_Prime_rehash_policy, std::__detail::_Hashtable_traits<false, false, true> >::clear() /usr/include/c++/10/bits/hashtable.h:2030
hazelcast#24 0x7f0d07c06d57 in std::_Hashtable<long, std::pair<long const, std::shared_ptr<hazelcast::client::spi::impl::ClientInvocation> >, std::allocator<std::pair<long const, std::shared_ptr<hazelcast::client::spi::impl::ClientInvocation> > >, std::__detail::_Select1st, std::equal_to<long>, std::hash<long>, std::__detail::_Mod_range_hashing, std::__detail::_Default_ranged_hash, std::__detail::_Prime_rehash_policy, std::__detail::_Hashtable_traits<false, false, true> >::~_Hashtable() (/hazelcast-cpp-client/build/libhazelcast-cpp-client.so.4.1.0+0xde9d57)
hazelcast#25 0x7f0d07bfa41f in std::unordered_map<long, std::shared_ptr<hazelcast::client::spi::impl::ClientInvocation>, std::hash<long>, std::equal_to<long>, std::allocator<std::pair<long const, std::shared_ptr<hazelcast::client::spi::impl::ClientInvocation> > > >::~unordered_map() /usr/include/c++/10/bits/unordered_map.h:102
hazelcast#26 0x7f0d07bca7c2 in hazelcast::client::connection::Connection::~Connection() /hazelcast-cpp-client/hazelcast/src/hazelcast/client/network.cpp:822
hazelcast#27 0x7f0d07c476d1 in void __gnu_cxx::new_allocator<hazelcast::client::connection::Connection>::destroy<hazelcast::client::connection::Connection>(hazelcast::client::connection::Connection*) /usr/include/c++/10/ext/new_allocator.h:156
hazelcast#28 0x7f0d07c42e0e in void std::allocator_traits<std::allocator<hazelcast::client::connection::Connection> >::destroy<hazelcast::client::connection::Connection>(std::allocator<hazelcast::client::connection::Connection>&, hazelcast::client::connection::Connection*) /usr/include/c++/10/bits/alloc_traits.h:531
hazelcast#29 0x7f0d07c3e136 in std::_Sp_counted_ptr_inplace<hazelcast::client::connection::Connection, std::allocator<hazelcast::client::connection::Connection>, (__gnu_cxx::_Lock_policy)2>::_M_dispose() /usr/include/c++/10/bits/shared_ptr_base.h:560
previously allocated by thread T71 here:
#0 0x7f0d08675bb7 in operator new(unsigned long) (/lib64/libasan.so.6+0xacbb7)
#1 0x7f0d07e2c685 in hazelcast::client::spi::impl::ClientExecutionServiceImpl::start() /hazelcast-cpp-client/hazelcast/src/hazelcast/client/spi.cpp:1000
#2 0x7f0d07e1f5f8 in hazelcast::client::spi::lifecycle_service::start() /hazelcast-cpp-client/hazelcast/src/hazelcast/client/spi.cpp:292
hazelcast#3 0x7f0d07a3d876 in hazelcast::client::impl::hazelcast_client_instance_impl::start() /hazelcast-cpp-client/hazelcast/src/hazelcast/client/client_impl.cpp:198
hazelcast#4 0x7f0d07a3bc35 in hazelcast::client::hazelcast_client::hazelcast_client(hazelcast::client::client_config) /hazelcast-cpp-client/hazelcast/src/hazelcast/client/client_impl.cpp:77
hazelcast#5 0x7f0d07a455f9 in operator() /hazelcast-cpp-client/hazelcast/src/hazelcast/client/client_impl.cpp:840
hazelcast#6 0x7f0d07a51395 in invoke<hazelcast::new_client(hazelcast::client::client_config)::<lambda(hazelcast::client::client_config&&)>, hazelcast::client::client_config> /usr/local/include/boost/thread/detail/invoke.hpp:134
hazelcast#7 0x7f0d07a50704 in execute<1> /usr/local/include/boost/thread/detail/invoker.hpp:102
hazelcast#8 0x7f0d07a4f112 in operator() /usr/local/include/boost/thread/detail/invoker.hpp:95
hazelcast#9 0x7f0d07a4d590 in run /usr/local/include/boost/thread/future.hpp:982
hazelcast#10 0x7f0d07a57557 in invoke<void (*)(boost::shared_ptr<boost::detail::future_async_shared_state<hazelcast::client::hazelcast_client, boost::detail::invoker<hazelcast::new_client(hazelcast::client::client_config)::<lambda(hazelcast::client::client_config&&)>, hazelcast::client::client_config> > >, boost::detail::invoker<hazelcast::new_client(hazelcast::client::client_config)::<lambda(hazelcast::client::client_config&&)>, hazelcast::client::client_config>&&), boost::shared_ptr<boost::detail::future_async_shared_state<hazelcast::client::hazelcast_client, boost::detail::invoker<hazelcast::new_client(hazelcast::client::client_config)::<lambda(hazelcast::client::client_config&&)>, hazelcast::client::client_config> > >, boost::detail::invoker<hazelcast::new_client(hazelcast::client::client_config)::<lambda(hazelcast::client::client_config&&)>, hazelcast::client::client_config> > /usr/local/include/boost/thread/detail/invoke.hpp:134
hazelcast#11 0x7f0d07a5640a in run2<1, 2> /usr/local/include/boost/thread/detail/thread.hpp:79
hazelcast#12 0x7f0d07a54f2b in run /usr/local/include/boost/thread/detail/thread.hpp:85
hazelcast#13 0x7f0d06c007b6 in thread_proxy (/usr/local/lib/libboost_thread.so.1.72.0+0xe7b6)
Thread T141 created by T0 here:
#0 0x7f0d0861ef46 in __interceptor_pthread_create (/lib64/libasan.so.6+0x55f46)
#1 0x7f0d06bffb6d in boost::thread::start_thread_noexcept() (/usr/local/lib/libboost_thread.so.1.72.0+0xdb6d)
#2 0x7f0d07e7478a in thread<void (*)(boost::shared_ptr<boost::detail::shared_state_base>), boost::shared_ptr<boost::detail::future_async_continuation_shared_state<boost::future<hazelcast::client::protocol::ClientMessage>, hazelcast::client::protocol::ClientMessage, hazelcast::client::spi::impl::ClientInvocation::invoke()::<lambda(boost::future<hazelcast::client::protocol::ClientMessage>)> > > > /usr/local/include/boost/thread/detail/thread.hpp:379
hazelcast#3 0x7f0d07e6fbe7 in launch_continuation /usr/local/include/boost/thread/future.hpp:4619
hazelcast#4 0xa13720 in boost::detail::shared_state<hazelcast::client::protocol::ClientMessage>::do_continuation(boost::unique_lock<boost::mutex>&) (/hazelcast-cpp-client/build/hazelcast/test/src/client_test+0xa13720)
hazelcast#5 0x851af6 in boost::detail::shared_state_base::mark_finished_internal(boost::unique_lock<boost::mutex>&) /usr/local/include/boost/thread/future.hpp:345
hazelcast#6 0x85265b in boost::detail::shared_state_base::mark_exceptional_finish_internal(boost::exception_ptr const&, boost::unique_lock<boost::mutex>&) /usr/local/include/boost/thread/future.hpp:456
hazelcast#7 0x8bf1cd in boost::promise<hazelcast::client::protocol::ClientMessage>::~promise() (/hazelcast-cpp-client/build/hazelcast/test/src/client_test+0x8bf1cd)
hazelcast#8 0x7f0d07e2d0da in hazelcast::client::spi::impl::ClientInvocation::~ClientInvocation() /hazelcast-cpp-client/hazelcast/src/hazelcast/client/spi.cpp:1045
hazelcast#9 0x7f0d07e2d17d in hazelcast::client::spi::impl::ClientInvocation::~ClientInvocation() /hazelcast-cpp-client/hazelcast/src/hazelcast/client/spi.cpp:1045
hazelcast#10 0x7f0d07ef4487 in std::_Sp_counted_ptr<hazelcast::client::spi::impl::ClientInvocation*, (__gnu_cxx::_Lock_policy)2>::_M_dispose() /usr/include/c++/10/bits/shared_ptr_base.h:380
hazelcast#11 0x8aad08 in std::_Sp_counted_base<(__gnu_cxx::_Lock_policy)2>::_M_release() /usr/include/c++/10/bits/shared_ptr_base.h:158
hazelcast#12 0x872171 in std::__shared_count<(__gnu_cxx::_Lock_policy)2>::~__shared_count() /usr/include/c++/10/bits/shared_ptr_base.h:733
hazelcast#13 0x10102a7 in std::__shared_ptr<hazelcast::client::spi::impl::ClientInvocation, (__gnu_cxx::_Lock_policy)2>::~__shared_ptr() /usr/include/c++/10/bits/shared_ptr_base.h:1183
hazelcast#14 0x10102c3 in std::shared_ptr<hazelcast::client::spi::impl::ClientInvocation>::~shared_ptr() /usr/include/c++/10/bits/shared_ptr.h:121
hazelcast#15 0x7f0d07c37715 in std::pair<long const, std::shared_ptr<hazelcast::client::spi::impl::ClientInvocation> >::~pair() /usr/include/c++/10/bits/stl_pair.h:211
hazelcast#16 0x7f0d07c37735 in void __gnu_cxx::new_allocator<std::__detail::_Hash_node<std::pair<long const, std::shared_ptr<hazelcast::client::spi::impl::ClientInvocation> >, false> >::destroy<std::pair<long const, std::shared_ptr<hazelcast::client::spi::impl::ClientInvocation> > >(std::pair<long const, std::shared_ptr<hazelcast::client::spi::impl::ClientInvocation> >*) /usr/include/c++/10/ext/new_allocator.h:156
hazelcast#17 0x7f0d07c311ca in void std::allocator_traits<std::allocator<std::__detail::_Hash_node<std::pair<long const, std::shared_ptr<hazelcast::client::spi::impl::ClientInvocation> >, false> > >::destroy<std::pair<long const, std::shared_ptr<hazelcast::client::spi::impl::ClientInvocation> > >(std::allocator<std::__detail::_Hash_node<std::pair<long const, std::shared_ptr<hazelcast::client::spi::impl::ClientInvocation> >, false> >&, std::pair<long const, std::shared_ptr<hazelcast::client::spi::impl::ClientInvocation> >*) /usr/include/c++/10/bits/alloc_traits.h:531
hazelcast#18 0x7f0d07c295ea in std::__detail::_Hashtable_alloc<std::allocator<std::__detail::_Hash_node<std::pair<long const, std::shared_ptr<hazelcast::client::spi::impl::ClientInvocation> >, false> > >::_M_deallocate_node(std::__detail::_Hash_node<std::pair<long const, std::shared_ptr<hazelcast::client::spi::impl::ClientInvocation> >, false>*) /usr/include/c++/10/bits/hashtable_policy.h:2053
hazelcast#19 0x7f0d07c20d19 in std::__detail::_Hashtable_alloc<std::allocator<std::__detail::_Hash_node<std::pair<long const, std::shared_ptr<hazelcast::client::spi::impl::ClientInvocation> >, false> > >::_M_deallocate_nodes(std::__detail::_Hash_node<std::pair<long const, std::shared_ptr<hazelcast::client::spi::impl::ClientInvocation> >, false>*) (/hazelcast-cpp-client/build/libhazelcast-cpp-client.so.4.1.0+0xe03d19)
hazelcast#20 0x7f0d07c13fc1 in std::_Hashtable<long, std::pair<long const, std::shared_ptr<hazelcast::client::spi::impl::ClientInvocation> >, std::allocator<std::pair<long const, std::shared_ptr<hazelcast::client::spi::impl::ClientInvocation> > >, std::__detail::_Select1st, std::equal_to<long>, std::hash<long>, std::__detail::_Mod_range_hashing, std::__detail::_Default_ranged_hash, std::__detail::_Prime_rehash_policy, std::__detail::_Hashtable_traits<false, false, true> >::clear() /usr/include/c++/10/bits/hashtable.h:2030
hazelcast#21 0x7f0d07c06d57 in std::_Hashtable<long, std::pair<long const, std::shared_ptr<hazelcast::client::spi::impl::ClientInvocation> >, std::allocator<std::pair<long const, std::shared_ptr<hazelcast::client::spi::impl::ClientInvocation> > >, std::__detail::_Select1st, std::equal_to<long>, std::hash<long>, std::__detail::_Mod_range_hashing, std::__detail::_Default_ranged_hash, std::__detail::_Prime_rehash_policy, std::__detail::_Hashtable_traits<false, false, true> >::~_Hashtable() (/hazelcast-cpp-client/build/libhazelcast-cpp-client.so.4.1.0+0xde9d57)
hazelcast#22 0x7f0d07bfa41f in std::unordered_map<long, std::shared_ptr<hazelcast::client::spi::impl::ClientInvocation>, std::hash<long>, std::equal_to<long>, std::allocator<std::pair<long const, std::shared_ptr<hazelcast::client::spi::impl::ClientInvocation> > > >::~unordered_map() /usr/include/c++/10/bits/unordered_map.h:102
hazelcast#23 0x7f0d07bca7c2 in hazelcast::client::connection::Connection::~Connection() /hazelcast-cpp-client/hazelcast/src/hazelcast/client/network.cpp:822
hazelcast#24 0x7f0d07c476d1 in void __gnu_cxx::new_allocator<hazelcast::client::connection::Connection>::destroy<hazelcast::client::connection::Connection>(hazelcast::client::connection::Connection*) /usr/include/c++/10/ext/new_allocator.h:156
hazelcast#25 0x7f0d07c42e0e in void std::allocator_traits<std::allocator<hazelcast::client::connection::Connection> >::destroy<hazelcast::client::connection::Connection>(std::allocator<hazelcast::client::connection::Connection>&, hazelcast::client::connection::Connection*) /usr/include/c++/10/bits/alloc_traits.h:531
hazelcast#26 0x7f0d07c3e136 in std::_Sp_counted_ptr_inplace<hazelcast::client::connection::Connection, std::allocator<hazelcast::client::connection::Connection>, (__gnu_cxx::_Lock_policy)2>::_M_dispose() /usr/include/c++/10/bits/shared_ptr_base.h:560
hazelcast#27 0x8aad08 in std::_Sp_counted_base<(__gnu_cxx::_Lock_policy)2>::_M_release() /usr/include/c++/10/bits/shared_ptr_base.h:158
hazelcast#28 0x872171 in std::__shared_count<(__gnu_cxx::_Lock_policy)2>::~__shared_count() /usr/include/c++/10/bits/shared_ptr_base.h:733
hazelcast#29 0x8615a7 in std::__shared_ptr<hazelcast::client::connection::Connection, (__gnu_cxx::_Lock_policy)2>::~__shared_ptr() /usr/include/c++/10/bits/shared_ptr_base.h:1183
hazelcast#30 0x8615c3 in std::shared_ptr<hazelcast::client::connection::Connection>::~shared_ptr() /usr/include/c++/10/bits/shared_ptr.h:121
hazelcast#31 0x7f0d07c453e1 in hazelcast::client::internal::socket::BaseSocket<boost::asio::basic_stream_socket<boost::asio::ip::tcp, boost::asio::executor> >::do_read(std::shared_ptr<hazelcast::client::connection::Connection>)::{lambda(boost::system::error_code const&, unsigned long)#1}::~error_code() (/hazelcast-cpp-client/build/libhazelcast-cpp-client.so.4.1.0+0xe283e1)
hazelcast#32 0x7f0d07c45481 in boost::asio::detail::wrapped_handler<boost::asio::io_context::strand, hazelcast::client::internal::socket::BaseSocket<boost::asio::basic_stream_socket<boost::asio::ip::tcp, boost::asio::executor> >::do_read(std::shared_ptr<hazelcast::client::connection::Connection>)::{lambda(boost::system::error_code const&, unsigned long)#1}, boost::asio::detail::is_continuation_if_running>::~wrapped_handler() (/hazelcast-cpp-client/build/libhazelcast-cpp-client.so.4.1.0+0xe28481)
hazelcast#33 0x7f0d07c5c2c3 in boost::asio::detail::binder2<boost::asio::detail::wrapped_handler<boost::asio::io_context::strand, hazelcast::client::internal::socket::BaseSocket<boost::asio::basic_stream_socket<boost::asio::ip::tcp, boost::asio::executor> >::do_read(std::shared_ptr<hazelcast::client::connection::Connection>)::{lambda(boost::system::error_code const&, unsigned long)#1}, boost::asio::detail::is_continuation_if_running>, boost::system::error_code, unsigned long>::~binder2() (/hazelcast-cpp-client/build/libhazelcast-cpp-client.so.4.1.0+0xe3f2c3)
hazelcast#34 0x7f0d07c5c550 in boost::asio::detail::reactive_socket_recv_op<boost::asio::mutable_buffers_1, boost::asio::detail::wrapped_handler<boost::asio::io_context::strand, hazelcast::client::internal::socket::BaseSocket<boost::asio::basic_stream_socket<boost::asio::ip::tcp, boost::asio::executor> >::do_read(std::shared_ptr<hazelcast::client::connection::Connection>)::{lambda(boost::system::error_code const&, unsigned long)#1}, boost::asio::detail::is_continuation_if_running>, boost::asio::detail::io_object_executor<boost::asio::executor> >::do_complete(void*, boost::asio::detail::scheduler_operation*, boost::system::error_code const&, unsigned long) (/hazelcast-cpp-client/build/libhazelcast-cpp-client.so.4.1.0+0xe3f550)
hazelcast#35 0xcbc8f4 in boost::asio::detail::scheduler_operation::destroy() /usr/local/include/boost/asio/detail/scheduler_operation.hpp:45
hazelcast#36 0xcc285d in boost::asio::detail::scheduler::shutdown() /usr/local/include/boost/asio/detail/impl/scheduler.ipp:166
hazelcast#37 0x84719b in boost::asio::detail::service_registry::shutdown_services() /usr/local/include/boost/asio/detail/impl/service_registry.ipp:44
hazelcast#38 0x8473a7 in boost::asio::execution_context::shutdown() /usr/local/include/boost/asio/impl/execution_context.ipp:41
hazelcast#39 0x84730e in boost::asio::execution_context::~execution_context() /usr/local/include/boost/asio/impl/execution_context.ipp:34
hazelcast#40 0x7f0d07bf2451 in boost::asio::io_context::~io_context() /usr/local/include/boost/asio/impl/io_context.ipp:56
hazelcast#41 0x7f0d07c0ba73 in std::default_delete<boost::asio::io_context>::operator()(boost::asio::io_context*) const /usr/include/c++/10/bits/unique_ptr.h:85
hazelcast#42 0x7f0d07bfe9f8 in std::unique_ptr<boost::asio::io_context, std::default_delete<boost::asio::io_context> >::~unique_ptr() /usr/include/c++/10/bits/unique_ptr.h:361
hazelcast#43 0x7f0d07bc4a34 in hazelcast::client::connection::ClientConnectionManagerImpl::~ClientConnectionManagerImpl() /hazelcast-cpp-client/hazelcast/src/hazelcast/client/network.cpp:548
hazelcast#44 0x7f0d07b0539f in void __gnu_cxx::new_allocator<hazelcast::client::connection::ClientConnectionManagerImpl>::destroy<hazelcast::client::connection::ClientConnectionManagerImpl>(hazelcast::client::connection::ClientConnectionManagerImpl*) (/hazelcast-cpp-client/build/libhazelcast-cpp-client.so.4.1.0+0xce839f)
hazelcast#45 0x7f0d07af8cfa in void std::allocator_traits<std::allocator<hazelcast::client::connection::ClientConnectionManagerImpl> >::destroy<hazelcast::client::connection::ClientConnectionManagerImpl>(std::allocator<hazelcast::client::connection::ClientConnectionManagerImpl>&, hazelcast::client::connection::ClientConnectionManagerImpl*) /usr/include/c++/10/bits/alloc_traits.h:531
hazelcast#46 0x7f0d07aebf9c in std::_Sp_counted_ptr_inplace<hazelcast::client::connection::ClientConnectionManagerImpl, std::allocator<hazelcast::client::connection::ClientConnectionManagerImpl>, (__gnu_cxx::_Lock_policy)2>::_M_dispose() (/hazelcast-cpp-client/build/libhazelcast-cpp-client.so.4.1.0+0xccef9c)
hazelcast#47 0x8aad08 in std::_Sp_counted_base<(__gnu_cxx::_Lock_policy)2>::_M_release() /usr/include/c++/10/bits/shared_ptr_base.h:158
hazelcast#48 0x872171 in std::__shared_count<(__gnu_cxx::_Lock_policy)2>::~__shared_count() /usr/include/c++/10/bits/shared_ptr_base.h:733
hazelcast#49 0x7f0d07a7569d in std::__shared_ptr<hazelcast::client::connection::ClientConnectionManagerImpl, (__gnu_cxx::_Lock_policy)2>::~__shared_ptr() /usr/include/c++/10/bits/shared_ptr_base.h:1183
hazelcast#50 0x7f0d07a756b9 in std::shared_ptr<hazelcast::client::connection::ClientConnectionManagerImpl>::~shared_ptr() /usr/include/c++/10/bits/shared_ptr.h:121
hazelcast#51 0x7f0d07a3d6f7 in hazelcast::client::impl::hazelcast_client_instance_impl::~hazelcast_client_instance_impl() /hazelcast-cpp-client/hazelcast/src/hazelcast/client/client_impl.cpp:190
hazelcast#52 0x7f0d07aee1f8 in std::_Sp_counted_ptr<hazelcast::client::impl::hazelcast_client_instance_impl*, (__gnu_cxx::_Lock_policy)2>::_M_dispose() /usr/include/c++/10/bits/shared_ptr_base.h:380
hazelcast#53 0x8aad08 in std::_Sp_counted_base<(__gnu_cxx::_Lock_policy)2>::_M_release() /usr/include/c++/10/bits/shared_ptr_base.h:158
hazelcast#54 0x872171 in std::__shared_count<(__gnu_cxx::_Lock_policy)2>::~__shared_count() /usr/include/c++/10/bits/shared_ptr_base.h:733
hazelcast#55 0x7f0d07a75499 in std::__shared_ptr<hazelcast::client::impl::hazelcast_client_instance_impl, (__gnu_cxx::_Lock_policy)2>::~__shared_ptr() /usr/include/c++/10/bits/shared_ptr_base.h:1183
hazelcast#56 0x7f0d07a754b5 in std::shared_ptr<hazelcast::client::impl::hazelcast_client_instance_impl>::~shared_ptr() /usr/include/c++/10/bits/shared_ptr.h:121
hazelcast#57 0x7f0d07a3c196 in hazelcast::client::hazelcast_client::~hazelcast_client() /hazelcast-cpp-client/hazelcast/src/hazelcast/client/client_impl.cpp:120
hazelcast#58 0x10ab4b0 in hazelcast::client::test::ReliableTopicTest_testConfig_Test::TestBody() /hazelcast-cpp-client/hazelcast/test/src/HazelcastTests8.cpp:1072
hazelcast#59 0x7f0d06dad0c4 in void testing::internal::HandleSehExceptionsInMethodIfSupported<testing::Test, void>(testing::Test*, void (testing::Test::*)(), char const*) /hazelcast-cpp-client/build/hazelcast/test/googletest-src/googletest/src/gtest.cc:2433
hazelcast#60 0x7f0d06d9dfae in void testing::internal::HandleExceptionsInMethodIfSupported<testing::Test, void>(testing::Test*, void (testing::Test::*)(), char const*) /hazelcast-cpp-client/build/hazelcast/test/googletest-src/googletest/src/gtest.cc:2469
hazelcast#61 0x7f0d06d49a83 in testing::Test::Run() /hazelcast-cpp-client/build/hazelcast/test/googletest-src/googletest/src/gtest.cc:2508
hazelcast#62 0x7f0d06d4addc in testing::TestInfo::Run() /hazelcast-cpp-client/build/hazelcast/test/googletest-src/googletest/src/gtest.cc:2684
hazelcast#63 0x7f0d06d4ba71 in testing::TestSuite::Run() /hazelcast-cpp-client/build/hazelcast/test/googletest-src/googletest/src/gtest.cc:2816
hazelcast#64 0x7f0d06d67178 in testing::internal::UnitTestImpl::RunAllTests() /hazelcast-cpp-client/build/hazelcast/test/googletest-src/googletest/src/gtest.cc:5338
hazelcast#65 0x7f0d06db0075 in bool testing::internal::HandleSehExceptionsInMethodIfSupported<testing::internal::UnitTestImpl, bool>(testing::internal::UnitTestImpl*, bool (testing::internal::UnitTestImpl::*)(), char const*) /hazelcast-cpp-client/build/hazelcast/test/googletest-src/googletest/src/gtest.cc:2433
hazelcast#66 0x7f0d06da080d in bool testing::internal::HandleExceptionsInMethodIfSupported<testing::internal::UnitTestImpl, bool>(testing::internal::UnitTestImpl*, bool (testing::internal::UnitTestImpl::*)(), char const*) /hazelcast-cpp-client/build/hazelcast/test/googletest-src/googletest/src/gtest.cc:2469
hazelcast#67 0x7f0d06d63fdf in testing::UnitTest::Run() /hazelcast-cpp-client/build/hazelcast/test/googletest-src/googletest/src/gtest.cc:4925
hazelcast#68 0x12d0e2e in RUN_ALL_TESTS() /hazelcast-cpp-client/build/hazelcast/test/googletest-src/googletest/include/gtest/gtest.h:2473
hazelcast#69 0x12cf37c in main /hazelcast-cpp-client/hazelcast/test/src/main.cpp:71
hazelcast#70 0x7f0d066d5ba1 in __libc_start_main (/lib64/libc.so.6+0x27ba1)
Thread T71 created by T0 here:
#0 0x7f0d0861ef46 in __interceptor_pthread_create (/lib64/libasan.so.6+0x55f46)
#1 0x7f0d06bffb6d in boost::thread::start_thread_noexcept() (/usr/local/lib/libboost_thread.so.1.72.0+0xdb6d)
#2 0x7f0d07a4da66 in thread<void (*)(boost::shared_ptr<boost::detail::future_async_shared_state<hazelcast::client::hazelcast_client, boost::detail::invoker<hazelcast::new_client(hazelcast::client::client_config)::<lambda(hazelcast::client::client_config&&)>, hazelcast::client::client_config> > >, boost::detail::invoker<hazelcast::new_client(hazelcast::client::client_config)::<lambda(hazelcast::client::client_config&&)>, hazelcast::client::client_config>&&), boost::shared_ptr<boost::detail::future_async_shared_state<hazelcast::client::hazelcast_client, boost::detail::invoker<hazelcast::new_client(hazelcast::client::client_config)::<lambda(hazelcast::client::client_config&&)>, hazelcast::client::client_config> > >, boost::detail::invoker<hazelcast::new_client(hazelcast::client::client_config)::<lambda(hazelcast::client::client_config&&)>, hazelcast::client::client_config> > /usr/local/include/boost/thread/detail/thread.hpp:379
hazelcast#3 0x7f0d07a4bb13 in init /usr/local/include/boost/thread/future.hpp:974
hazelcast#4 0x7f0d07a49e9b in make_future_async_shared_state<hazelcast::client::hazelcast_client, boost::detail::invoker<hazelcast::new_client(hazelcast::client::client_config)::<lambda(hazelcast::client::client_config&&)>, hazelcast::client::client_config> > /usr/local/include/boost/thread/future.hpp:3853
hazelcast#5 0x7f0d07a47ed0 in async<hazelcast::new_client(hazelcast::client::client_config)::<lambda(hazelcast::client::client_config&&)>, hazelcast::client::client_config> /usr/local/include/boost/thread/future.hpp:3935
hazelcast#6 0x7f0d07a46894 in async<hazelcast::new_client(hazelcast::client::client_config)::<lambda(hazelcast::client::client_config&&)>, hazelcast::client::client_config> /usr/local/include/boost/thread/future.hpp:4294
hazelcast#7 0x7f0d07a45754 in hazelcast::new_client(hazelcast::client::client_config) /hazelcast-cpp-client/hazelcast/src/hazelcast/client/client_impl.cpp:843
hazelcast#8 0x10a9ce5 in hazelcast::client::test::ReliableTopicTest_testConfig_Test::TestBody() /hazelcast-cpp-client/hazelcast/test/src/HazelcastTests8.cpp:1049
hazelcast#9 0x7f0d06dad0c4 in void testing::internal::HandleSehExceptionsInMethodIfSupported<testing::Test, void>(testing::Test*, void (testing::Test::*)(), char const*) /hazelcast-cpp-client/build/hazelcast/test/googletest-src/googletest/src/gtest.cc:2433
hazelcast#10 0x7f0d06d9dfae in void testing::internal::HandleExceptionsInMethodIfSupported<testing::Test, void>(testing::Test*, void (testing::Test::*)(), char const*) /hazelcast-cpp-client/build/hazelcast/test/googletest-src/googletest/src/gtest.cc:2469
hazelcast#11 0x7f0d06d49a83 in testing::Test::Run() /hazelcast-cpp-client/build/hazelcast/test/googletest-src/googletest/src/gtest.cc:2508
hazelcast#12 0x7f0d06d4addc in testing::TestInfo::Run() /hazelcast-cpp-client/build/hazelcast/test/googletest-src/googletest/src/gtest.cc:2684
hazelcast#13 0x7f0d06d4ba71 in testing::TestSuite::Run() /hazelcast-cpp-client/build/hazelcast/test/googletest-src/googletest/src/gtest.cc:2816
hazelcast#14 0x7f0d06d67178 in testing::internal::UnitTestImpl::RunAllTests() /hazelcast-cpp-client/build/hazelcast/test/googletest-src/googletest/src/gtest.cc:5338
hazelcast#15 0x7f0d06db0075 in bool testing::internal::HandleSehExceptionsInMethodIfSupported<testing::internal::UnitTestImpl, bool>(testing::internal::UnitTestImpl*, bool (testing::internal::UnitTestImpl::*)(), char const*) /hazelcast-cpp-client/build/hazelcast/test/googletest-src/googletest/src/gtest.cc:2433
hazelcast#16 0x7f0d06da080d in bool testing::internal::HandleExceptionsInMethodIfSupported<testing::internal::UnitTestImpl, bool>(testing::internal::UnitTestImpl*, bool (testing::internal::UnitTestImpl::*)(), char const*) /hazelcast-cpp-client/build/hazelcast/test/googletest-src/googletest/src/gtest.cc:2469
hazelcast#17 0x7f0d06d63fdf in testing::UnitTest::Run() /hazelcast-cpp-client/build/hazelcast/test/googletest-src/googletest/src/gtest.cc:4925
hazelcast#18 0x12d0e2e in RUN_ALL_TESTS() /hazelcast-cpp-client/build/hazelcast/test/googletest-src/googletest/include/gtest/gtest.h:2473
hazelcast#19 0x12cf37c in main /hazelcast-cpp-client/hazelcast/test/src/main.cpp:71
hazelcast#20 0x7f0d066d5ba1 in __libc_start_main (/lib64/libc.so.6+0x27ba1)
SUMMARY: AddressSanitizer: heap-use-after-free /usr/include/c++/10/bits/unique_ptr.h:173 in std::__uniq_ptr_impl<boost::asio::thread_pool, std::default_delete<boost::asio::thread_pool> >::_M_ptr() const
Shadow bytes around the buggy address:
0x0c0480000130: fa fa fd fa fa fa fd fa fa fa fd fa fa fa fd fa
0x0c0480000140: fa fa fd fa fa fa fd fa fa fa fd fa fa fa fd fa
0x0c0480000150: fa fa fd fa fa fa fd fa fa fa fd fa fa fa fd fa
0x0c0480000160: fa fa fd fa fa fa fd fa fa fa fd fd fa fa fd fd
0x0c0480000170: fa fa fd fd fa fa fd fa fa fa fd fd fa fa fd fd
=>0x0c0480000180: fa fa fd fd fa fa fd fd fa fa[fd]fd fa fa fd fd
0x0c0480000190: fa fa fd fd fa fa fd fd fa fa fd fd fa fa fd fd
0x0c04800001a0: fa fa fd fd fa fa fd fd fa fa fd fd fa fa fd fd
0x0c04800001b0: fa fa fd fd fa fa fd fd fa fa fd fd fa fa fd fd
0x0c04800001c0: fa fa fd fd fa fa fd fd fa fa fd fd fa fa fd fd
0x0c04800001d0: fa fa fd fd fa fa fd fd fa fa fd fd fa fa fd fd
Shadow byte legend (one shadow byte represents 8 application bytes):
Addressable: 00
Partially addressable: 01 02 03 04 05 06 07
Heap left redzone: fa
Freed heap region: fd
Stack left redzone: f1
Stack mid redzone: f2
Stack right redzone: f3
Stack after return: f5
Stack use after scope: f8
Global redzone: f9
Global init order: f6
Poisoned by user: f7
Container overflow: fc
Array cookie: ac
Intra object redzone: bb
ASan internal: fe
Left alloca redzone: ca
Right alloca redzone: cb
Shadow gap: cc
==5793==ABORTING
```
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
The hazelcast client initialization seems to be blocked on the getnameinfo call. The log: https://hazelcast-l337.ci.cloudbees.com/view/Clients/job/cpp-play/84/console
When looking at the thread stack traces we observe the following:
(gdb) info threads
5 Thread 0xf0ea9b90 (LWP 4459) 0xffffe410 in __kernel_vsyscall ()
4 Thread 0xf7cb3b90 (LWP 4460) 0xffffe410 in __kernel_vsyscall ()
3 Thread 0xf36abb90 (LWP 4461) 0xffffe410 in __kernel_vsyscall ()
2 Thread 0xf54adb90 (LWP 4462) 0xffffe410 in __kernel_vsyscall ()
(gdb) bt
#0 0xffffe410 in __kernel_vsyscall ()
test issue #1 0x003f1776 in nanosleep () from /lib/libc.so.6
Added missing executeonX tests for Map. #2 0x0042c7dc in usleep () from /lib/libc.so.6
request and response classes and related hooks of old client protocol are deleted #3 0xf7ea037b in hazelcast::util::sleepmillis (milliseconds=1000) at /root/jenkins/workspace/cpp-play/hazelcast/src/hazelcast/util/Util.cpp:54
Decrement Pending packets for events also #4 0xf7ea0351 in hazelcast::util::sleep (seconds=1) at /root/jenkins/workspace/cpp-play/hazelcast/src/hazelcast/util/Util.cpp:47
Changes in Map getAll/putAll implementation #5 0xf7ea1a53 in hazelcast::util::CountDownLatch::await (this=0xffbd0234) at /root/jenkins/workspace/cpp-play/hazelcast/src/hazelcast/util/CountDownLatch.cpp:54
Reconnecting to instance on same address should preserve member list correctly #6 0xf7e05f89 in hazelcast::client::spi::ClusterService::start (this=0xffbd0228) at /root/jenkins/workspace/cpp-play/hazelcast/src/hazelcast/client/spi/ClusterService.cpp:52
Serialization resolution order #7 0xf7df94e5 in hazelcast::client::spi::LifecycleService::start (this=0xffbcfed8) at /root/jenkins/workspace/cpp-play/hazelcast/src/hazelcast/client/spi/LifecycleService.cpp:50
Add tests for serialization #8 0xf7e24e7a in hazelcast::client::HazelcastClient::HazelcastClient (this=0xffbcfe00, config=...) at /root/jenkins/workspace/cpp-play/hazelcast/src/hazelcast/client/HazelcastClient.cpp:49
Listener service should listen for events after clusterService has been started to prevent receiving duplicate events #9 0x0816fe52 in hazelcast::client::test::ClusterTest::testListenersWhenClusterDown (this=0xffbd1490) at /root/jenkins/workspace/cpp-play/hazelcast/test/cluster/ClusterTest.cpp:233
Fixing release scripts #10 0x08116092 in iTest::iTestFixturehazelcast::client::test::ClusterTest::executeTests (this=0xffbd1490) at /root/jenkins/workspace/cpp-play/hazelcast/include/iTest/iTestFixture.h:97
New client protocol implementation #11 0x08111cd6 in unitTests (address=0x818f1e7 "127.0.0.1") at /root/jenkins/workspace/cpp-play/hazelcast/test/main.cpp:65
Add All The Java Predicates To C++ #12 0x08112880 in main (argc=1, argv=0xffbd18d4) at /root/jenkins/workspace/cpp-play/hazelcast/test/main.cpp:99
(gdb) thread 2
[Switching to thread 2 (Thread 0xf54adb90 (LWP 4462))]#0 0xffffe410 in __kernel_vsyscall ()
(gdb) bt
#0 0xffffe410 in __kernel_vsyscall ()
#1 0x004c8839 in __lll_lock_wait () from /lib/libpthread.so.0
#2 0x004c3e9f in _L_lock_885 () from /lib/libpthread.so.0
#3 0x004c3d66 in pthread_mutex_lock () from /lib/libpthread.so.0
#4 0xf7fcb50c in _nss_files_gethostbyaddr_r () from /lib/libnss_files.so.2
#5 0x004499f7 in gethostbyaddr_r@@GLIBC_2.1.2 () from /lib/libc.so.6
#6 0x00451efd in getnameinfo () from /lib/libc.so.6
#7 0xf7e9dd0e in hazelcast::client::Socket::getAddress (this=0x95ef148) at /root/jenkins/workspace/cpp-play/hazelcast/src/hazelcast/client/Socket.cpp:170
#8 0xf7e2d550 in hazelcast::client::connection::Connection::connect (this=0x95ef100, timeoutInMillis=5000) at /root/jenkins/workspace/cpp-play/hazelcast/src/hazelcast/client/connection/Connection.cpp:69
#9 0xf7e358ab in hazelcast::client::connection::ConnectionManager::connectTo (this=0xffbcff98, address=..., ownerConnection=true) at /root/jenkins/workspace/cpp-play/hazelcast/src/hazelcast/client/connection/ConnectionManager.cpp:306
#10 0xf7e2f2c3 in hazelcast::client::connection::OwnerConnectionFuture::createNew (this=0xffbd01fc, address=...) at /root/jenkins/workspace/cpp-play/hazelcast/src/hazelcast/client/connection/OwnerConnectionFuture.cpp:42
#11 0xf7e33669 in hazelcast::client::connection::ConnectionManager::createOwnerConnection (this=0xffbcff98, address=...) at /root/jenkins/workspace/cpp-play/hazelcast/src/hazelcast/client/connection/ConnectionManager.cpp:98
#12 0xf7e06cc3 in hazelcast::client::spi::ClusterService::connectToOne (this=0xffbd0228) at /root/jenkins/workspace/cpp-play/hazelcast/src/hazelcast/client/spi/ClusterService.cpp:170
#13 0xf7e42730 in hazelcast::client::connection::ClusterListenerThread::run (this=0xffbd022c, currentThread=0xf6c010a0) at /root/jenkins/workspace/cpp-play/hazelcast/src/hazelcast/client/connection/ClusterListenerThread.cpp:56
#14 0xf7e4268d in hazelcast::client::connection::ClusterListenerThread::staticRun (args=...) at /root/jenkins/workspace/cpp-play/hazelcast/src/hazelcast/client/connection/ClusterListenerThread.cpp:44
#15 0xf7ea3823 in hazelcast::util::Thread::controlledThread (args=0xf6c010d8) at /root/jenkins/workspace/cpp-play/hazelcast/src/hazelcast/util/Thread.cpp:193
#16 0x004c1912 in start_thread () from /lib/libpthread.so.0
#17 0x0043347e in clone () from /lib/libc.so.6
(gdb) thread 3
[Switching to thread 3 (Thread 0xf36abb90 (LWP 4461))]#0 0xffffe410 in __kernel_vsyscall ()
(gdb) bt
#0 0xffffe410 in __kernel_vsyscall ()
#1 0x003f1776 in nanosleep () from /lib/libc.so.6
#2 0x0042c7dc in usleep () from /lib/libc.so.6
#3 0xf7ea037b in hazelcast::util::sleepmillis (milliseconds=10000) at /root/jenkins/workspace/cpp-play/hazelcast/src/hazelcast/util/Util.cpp:54
#4 0xf7ea0351 in hazelcast::util::sleep (seconds=10) at /root/jenkins/workspace/cpp-play/hazelcast/src/hazelcast/util/Util.cpp:47
#5 0xf7ea370f in hazelcast::util::Thread::interruptibleSleep (this=0xf6c0ab08, seconds=10) at /root/jenkins/workspace/cpp-play/hazelcast/src/hazelcast/util/Thread.cpp:167
#6 0xf7e3c33e in hazelcast::client::connection::HeartBeater::run (this=0xffbd01cc, currentThread=0xf6c0ab08) at /root/jenkins/workspace/cpp-play/hazelcast/src/hazelcast/client/connection/HeartBeater.cpp:81
#7 0xf7e3c021 in hazelcast::client::connection::HeartBeater::staticStart (args=...) at /root/jenkins/workspace/cpp-play/hazelcast/src/hazelcast/client/connection/HeartBeater.cpp:54
#8 0xf7ea3823 in hazelcast::util::Thread::controlledThread (args=0xf6c01050) at /root/jenkins/workspace/cpp-play/hazelcast/src/hazelcast/util/Thread.cpp:193
#9 0x004c1912 in start_thread () from /lib/libpthread.so.0
#10 0x0043347e in clone () from /lib/libc.so.6
(gdb) thread 4
[Switching to thread 4 (Thread 0xf7cb3b90 (LWP 4460))]#0 0xffffe410 in __kernel_vsyscall ()
(gdb) bt
#0 0xffffe410 in __kernel_vsyscall ()
#1 0x0042bf41 in select () from /lib/libc.so.6
#2 0xf7e3f4c0 in hazelcast::client::connection::OutSelector::listenInternal (this=0xffbd00b4) at /root/jenkins/workspace/cpp-play/hazelcast/src/hazelcast/client/connection/OutSelector.cpp:59
#3 0xf7e3cd48 in hazelcast::client::connection::IOSelector::listen (this=0xffbd00b4) at /root/jenkins/workspace/cpp-play/hazelcast/src/hazelcast/client/connection/IOSelector.cpp:68
#4 0xf7e3c7e7 in hazelcast::client::connection::IOSelector::staticListen (args=...) at /root/jenkins/workspace/cpp-play/hazelcast/src/hazelcast/client/connection/IOSelector.cpp:47
#5 0xf7ea3823 in hazelcast::util::Thread::controlledThread (args=0xf6c0aac8) at /root/jenkins/workspace/cpp-play/hazelcast/src/hazelcast/util/Thread.cpp:193
#6 0x004c1912 in start_thread () from /lib/libpthread.so.0
#7 0x0043347e in clone () from /lib/libc.so.6
(gdb) thread 5
[Switching to thread 5 (Thread 0xf0ea9b90 (LWP 4459))]#0 0xffffe410 in __kernel_vsyscall ()
(gdb) bt
#0 0xffffe410 in __kernel_vsyscall ()
#1 0x0042bf41 in select () from /lib/libc.so.6
#2 0xf7e3fc4f in hazelcast::client::connection::InSelector::listenInternal (this=0xffbd000c) at /root/jenkins/workspace/cpp-play/hazelcast/src/hazelcast/client/connection/InSelector.cpp:51
#3 0xf7e3cd48 in hazelcast::client::connection::IOSelector::listen (this=0xffbd000c) at /root/jenkins/workspace/cpp-play/hazelcast/src/hazelcast/client/connection/IOSelector.cpp:68
#4 0xf7e3c7e7 in hazelcast::client::connection::IOSelector::staticListen (args=...) at /root/jenkins/workspace/cpp-play/hazelcast/src/hazelcast/client/connection/IOSelector.cpp:47
#5 0xf7ea3823 in hazelcast::util::Thread::controlledThread (args=0xf6c0aa50) at /root/jenkins/workspace/cpp-play/hazelcast/src/hazelcast/util/Thread.cpp:193
#6 0x004c1912 in start_thread () from /lib/libpthread.so.0
#7 0x0043347e in clone () from /lib/libc.so.6
The text was updated successfully, but these errors were encountered: