Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Dangling pointer in continuation lambda #854

Closed
yemreinci opened this issue Apr 26, 2021 · 1 comment · Fixed by #858
Closed

Dangling pointer in continuation lambda #854

yemreinci opened this issue Apr 26, 2021 · 1 comment · Fixed by #858
Assignees
Labels
to-jira Use to create a placeholder Jira issue in Jira APIs Project Type: Defect
Milestone

Comments

@yemreinci
Copy link
Contributor

yemreinci commented Apr 26, 2021

All of 32 bit builds are failing with a segmentation fault. The problem started occurring after #850:

After reproducing and running with address sanitizer to spot the issue, I found that the thread executing the lambda function here tries to dereference an already-freed pointer to a cluster_view_listener object:

invocation->invoke_urgent().then([=] (boost::future<protocol::ClientMessage> f) {
if (f.has_value()) {
handler->on_listener_register();
return;
}
//completes with exception, listener needs to be reregistered
try_reregister_to_random_connection(connection);
});

The problem existed before #850, but seems like the change somehow made it more capable to cause errors.

The error can be reproduced on the i386/ubuntu docker image. I'm sharing a part of the ASAN's error report:

==26260==ERROR: AddressSanitizer: heap-use-after-free on address 0xf4240970 at pc 0xf724d744 bp 0xf07fed88 sp 0xf07fed78
WRITE of size 4 at 0xf4240970 thread T29
    #0 0xf724d743 in std::__atomic_base<hazelcast::client::connection::Connection*>::compare_exchange_strong(hazelcast::client::connection::Connection*&, hazelcast::client::connection::Connection*, std::memory_order, std::memory_order) /usr/include/c++/7/bits/atomic_base.h:752
    #1 0xf724d743 in std::atomic<hazelcast::client::connection::Connection*>::compare_exchange_strong(hazelcast::client::connection::Connection*&, hazelcast::client::connection::Connection*, std::memory_order) /usr/include/c++/7/atomic:511
    #2 0xf71e0833 in hazelcast::client::spi::impl::listener::cluster_view_listener::try_reregister_to_random_connection(std::shared_ptr<hazelcast::client::connection::Connection>) /hazelcast-cpp-client/hazelcast/src/hazelcast/client/spi.cpp:2203
    #3 0xf71dfaf8 in operator() /hazelcast-cpp-client/hazelcast/src/hazelcast/client/spi.cpp:2195
    #4 0xf7221df0 in call /usr/local/include/boost/thread/future.hpp:4563
    #5 0xf721b61b in run /usr/local/include/boost/thread/future.hpp:4596
    #6 0xf7230b0a in invoke<void (*)(boost::shared_ptr<boost::detail::shared_state_base>), boost::shared_ptr<boost::detail::future_async_continuation_shared_state<boost::future<hazelcast::client::protocol::ClientMessage>, void, hazelcast::client::spi::impl::listener::cluster_view_listener::try_register(std::shared_ptr<hazelcast::client::connection::Connection>)::<lambda(boost::future<hazelcast::client::protocol::ClientMessage>)> > > > /usr/local/include/boost/thread/detail/invoke.hpp:133
    #7 0xf722f321 in run2<1> /usr/local/include/boost/thread/detail/thread.hpp:79
    #8 0xf722efca in run /usr/local/include/boost/thread/detail/thread.hpp:85
    #9 0xf609108f in thread_proxy (/usr/local/lib/libboost_thread.so.1.75.0+0x708f)
    #10 0xf7a2d610  (/usr/lib/i386-linux-gnu/libasan.so.4+0x2f610)
    #11 0xf60663bc in start_thread (/lib/i386-linux-gnu/libpthread.so.0+0x63bc)
    #12 0xf5dd8c95 in clone (/lib/i386-linux-gnu/libc.so.6+0xf8c95)
@ihsandemir
Copy link
Collaborator

ihsandemir commented Apr 27, 2021

Solution suggestion:

                        auto this_view_listener = shared_from_this();
                        invocation->invoke_urgent().then(
                                [this_view_listener, handler, connection](boost::future<protocol::ClientMessage> f) {
                            if (f.has_value()) {
                                handler->on_listener_register();
                                return;
                            }
                            //completes with exception, listener needs to be reregistered
                            this_view_listener->try_reregister_to_random_connection(connection);
                        });

Can you try this solution @yemreinci and let me know if address sanitizer still reports a problem?

@ihsandemir ihsandemir added this to the 4.1 milestone Apr 27, 2021
ihsandemir added a commit to ihsandemir/hazelcast-cpp-client that referenced this issue Apr 28, 2021
…use-after-free` problem whn the view listener is destroyed.

fixes hazelcast#854
@degerhz degerhz added the to-jira Use to create a placeholder Jira issue in Jira APIs Project label Apr 28, 2021
@yemreinci yemreinci self-assigned this Apr 28, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
to-jira Use to create a placeholder Jira issue in Jira APIs Project Type: Defect
Projects
None yet
3 participants