Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[21232] Fix flaky latency tests on mac #5009

Merged
merged 3 commits into from
Jul 2, 2024
Merged

[21232] Fix flaky latency tests on mac #5009

merged 3 commits into from
Jul 2, 2024

Conversation

Mario-DL
Copy link
Member

@Mario-DL Mario-DL commented Jul 1, 2024

Description

Some of the Latency tests failed in mac from time to time, in particular, the intraprocess ones. Analysis revealed the following backtrace:

libfastdds.3.0.0.dylib!eprosima::fastdds::rtps::BaseReader::downcast(eprosima::fastdds::rtps::RTPSReader*) (/Users/mardom/dds_ws/src/fastdds/src/cpp/rtps/reader/BaseReader.cpp:236)
libfastdds.3.0.0.dylib!eprosima::fastdds::rtps::StatefulWriter::intraprocess_delivery(eprosima::fastdds::rtps::CacheChange_t*, eprosima::fastdds::rtps::ReaderProxy*) (/Users/mardom/dds_ws/src/fastdds/src/cpp/rtps/writer/StatefulWriter.cpp:483)
libfastdds.3.0.0.dylib!eprosima::fastdds::rtps::StatefulWriter::deliver_sample_to_intraprocesses(eprosima::fastdds::rtps::CacheChange_t*) (/Users/mardom/dds_ws/src/fastdds/src/cpp/rtps/writer/StatefulWriter.cpp:642)
libfastdds.3.0.0.dylib!eprosima::fastdds::rtps::StatefulWriter::deliver_sample_nts(eprosima::fastdds::rtps::CacheChange_t*, eprosima::fastdds::rtps::RTPSMessageGroup&, eprosima::fastdds::rtps::LocatorSelectorSender&, std::__1::chrono::time_point<std::__1::chrono::steady_clock, std::__1::chrono::duration<long long, std::__1::ratio<1l, 1000000000l> > > const&) (/Users/mardom/dds_ws/src/fastdds/src/cpp/rtps/writer/StatefulWriter.cpp:2194)
libfastdds.3.0.0.dylib!std::__1::enable_if<std::is_base_of<eprosima::fastdds::rtps::FlowControllerPureSyncPublishMode, eprosima::fastdds::rtps::FlowControllerSyncPublishMode>::value, bool>::type eprosima::fastdds::rtps::FlowControllerImpl<eprosima::fastdds::rtps::FlowControllerSyncPublishMode, eprosima::fastdds::rtps::FlowControllerFifoSchedule>::add_new_sample_impl<eprosima::fastdds::rtps::FlowControllerSyncPublishMode>(eprosima::fastdds::rtps::RTPSWriter*, eprosima::fastdds::rtps::CacheChange_t*, std::__1::chrono::time_point<std::__1::chrono::steady_clock, std::__1::chrono::duration<long long, std::__1::ratio<1l, 1000000000l> > > const&) (/Users/mardom/dds_ws/src/fastdds/src/cpp/rtps/flowcontrol/FlowControllerImpl.hpp:1191)
libfastdds.3.0.0.dylib!eprosima::fastdds::rtps::FlowControllerImpl<eprosima::fastdds::rtps::FlowControllerSyncPublishMode, eprosima::fastdds::rtps::FlowControllerFifoSchedule>::add_new_sample(eprosima::fastdds::rtps::RTPSWriter*, eprosima::fastdds::rtps::CacheChange_t*, std::__1::chrono::time_point<std::__1::chrono::steady_clock, std::__1::chrono::duration<long long, std::__1::ratio<1l, 1000000000l> > > const&) (/Users/mardom/dds_ws/src/fastdds/src/cpp/rtps/flowcontrol/FlowControllerImpl.hpp:1012)
libfastdds.3.0.0.dylib!eprosima::fastdds::rtps::StatefulWriter::unsent_change_added_to_history(eprosima::fastdds::rtps::CacheChange_t*, std::__1::chrono::time_point<std::__1::chrono::steady_clock, std::__1::chrono::duration<long long, std::__1::ratio<1l, 1000000000l> > > const&) (/Users/mardom/dds_ws/src/fastdds/src/cpp/rtps/writer/StatefulWriter.cpp:455)
libfastdds.3.0.0.dylib!eprosima::fastdds::rtps::WriterHistory::notify_writer(eprosima::fastdds::rtps::CacheChange_t*, std::__1::chrono::time_point<std::__1::chrono::steady_clock, std::__1::chrono::duration<long long, std::__1::ratio<1l, 1000000000l> > > const&) (/Users/mardom/dds_ws/src/fastdds/src/cpp/rtps/history/WriterHistory.cpp:124)
libfastdds.3.0.0.dylib!eprosima::fastdds::rtps::WriterHistory::add_change_(eprosima::fastdds::rtps::CacheChange_t*, eprosima::fastdds::rtps::WriteParams&, std::__1::chrono::time_point<std::__1::chrono::steady_clock, std::__1::chrono::duration<long long, std::__1::ratio<1l, 1000000000l> > >) (/Users/mardom/dds_ws/src/fastdds/src/cpp/rtps/history/WriterHistory.cpp:145)
libfastdds.3.0.0.dylib!eprosima::fastdds::rtps::WriterHistory::add_change(eprosima::fastdds::rtps::CacheChange_t*) (/Users/mardom/dds_ws/src/fastdds/src/cpp/rtps/history/WriterHistory.cpp:52)
libfastdds.3.0.0.dylib!eprosima::fastdds::rtps::EDPSimple::removeLocalReader(eprosima::fastdds::rtps::RTPSReader*) (/Users/mardom/dds_ws/src/fastdds/src/cpp/rtps/builtin/discovery/endpoint/EDPSimple.cpp:738)
libfastdds.3.0.0.dylib!eprosima::fastdds::rtps::BuiltinProtocols::removeLocalReader(eprosima::fastdds::rtps::RTPSReader*) (/Users/mardom/dds_ws/src/fastdds/src/cpp/rtps/builtin/BuiltinProtocols.cpp:304)
libfastdds.3.0.0.dylib!eprosima::fastdds::rtps::RTPSParticipantImpl::deleteUserEndpoint(eprosima::fastdds::rtps::GUID_t const&) (/Users/mardom/dds_ws/src/fastdds/src/cpp/rtps/participant/RTPSParticipantImpl.cpp:2140)
libfastdds.3.0.0.dylib!eprosima::fastdds::rtps::RTPSDomainImpl::removeRTPSReader(eprosima::fastdds::rtps::RTPSReader*) (/Users/mardom/dds_ws/src/fastdds/src/cpp/rtps/RTPSDomain.cpp:532)
libfastdds.3.0.0.dylib!eprosima::fastdds::rtps::RTPSDomain::removeRTPSReader(eprosima::fastdds::rtps::RTPSReader*) (/Users/mardom/dds_ws/src/fastdds/src/cpp/rtps/RTPSDomain.cpp:516)
libfastdds.3.0.0.dylib!eprosima::fastdds::dds::DataReaderImpl::stop() (/Users/mardom/dds_ws/src/fastdds/src/cpp/fastdds/subscriber/DataReaderImpl.cpp:335)
libfastdds.3.0.0.dylib!eprosima::fastdds::dds::DataReaderImpl::~DataReaderImpl() (/Users/mardom/dds_ws/src/fastdds/src/cpp/fastdds/subscriber/DataReaderImpl.cpp:349)
libfastdds.3.0.0.dylib!eprosima::fastdds::statistics::dds::DataReaderImpl::~DataReaderImpl() (/Users/mardom/dds_ws/src/fastdds/src/cpp/statistics/fastdds/subscriber/DataReaderImpl.hpp:43)
libfastdds.3.0.0.dylib!eprosima::fastdds::statistics::dds::DataReaderImpl::~DataReaderImpl() (/Users/mardom/dds_ws/src/fastdds/src/cpp/statistics/fastdds/subscriber/DataReaderImpl.hpp:43)
libfastdds.3.0.0.dylib!eprosima::fastdds::statistics::dds::DataReaderImpl::~DataReaderImpl() (/Users/mardom/dds_ws/src/fastdds/src/cpp/statistics/fastdds/subscriber/DataReaderImpl.hpp:43)
libfastdds.3.0.0.dylib!eprosima::fastdds::dds::SubscriberImpl::delete_datareader(eprosima::fastdds::dds::DataReader const*) (/Users/mardom/dds_ws/src/fastdds/src/cpp/fastdds/subscriber/SubscriberImpl.cpp:298)
libfastdds.3.0.0.dylib!eprosima::fastdds::dds::Subscriber::delete_datareader(eprosima::fastdds::dds::DataReader const*) (/Users/mardom/dds_ws/src/fastdds/src/cpp/fastdds/subscriber/Subscriber.cpp:129)
LatencyTest!LatencyTestPublisher::~LatencyTestPublisher() (/Users/mardom/dds_ws/src/fastdds/test/performance/latency/LatencyTestPublisher.cpp:74)
LatencyTest!LatencyTestPublisher::~LatencyTestPublisher() (/Users/mardom/dds_ws/src/fastdds/test/performance/latency/LatencyTestPublisher.cpp:59)

In intraprocess there is a race in destruction since the LatencySubsriber deletes its participant and then, when the LatencyPublisher deletes their entites it tries to access the EDP builtins on the LatencySubscriber which are deleted.

This PR lets the user endpoints be destroyed while guaranteeing that both are alive before deleting the participants.

In addition, proposes a refactor for using a BaseReader instead of a RTPSReader for the local_reader_ member of the readerproxy to save up a dynamic_cast when downcasting.

Contributor Checklist

  • Commit messages follow the project guidelines.
  • The code follows the style guidelines of this project.
  • NO Tests that thoroughly check the new feature have been added/Regression tests checking the bug and its fix have been added; the added tests pass locally
  • Any new/modified methods have been properly documented using Doxygen.
  • NO Any new configuration API has an equivalent XML API (with the corresponding XSD extension)
  • ❌ Changes are backport compatible: they do NOT break ABI nor change library core behavior.
  • ❌ Changes are API compatible.
  • NO New feature has been added to the versions.md file (if applicable).
  • NO New feature has been documented/Current behavior is correctly described in the documentation.
  • NO Applicable backports have been included in the description.

Reviewer Checklist

  • The PR has a milestone assigned.
  • The title and description correctly express the PR's purpose.
  • Check contributor checklist is correct.
  • If this is a critical bug fix, backports to the critical-only supported branches have been requested.
  • Check CI results: changes do not issue any warning.
  • Check CI results: failing tests are unrelated with the changes.

Signed-off-by: Mario Dominguez <mardom@Marios-iMac.local>
Signed-off-by: Mario Domínguez López <mariodominguez@eprosima.com>
@Mario-DL Mario-DL added this to the v3.0.0 milestone Jul 1, 2024
@Mario-DL Mario-DL requested a review from elianalf July 1, 2024 08:21
@github-actions github-actions bot added the ci-pending PR which CI is running label Jul 1, 2024
@elianalf elianalf added skip-ci Automatically pass CI and removed ci-pending PR which CI is running labels Jul 2, 2024
Signed-off-by: Mario Dominguez <mariodominguez@eprosima.com>
@elianalf
Copy link
Contributor

elianalf commented Jul 2, 2024

LGTM: CI passed here ce9fc78 and the failed tests are unrelated.

@elianalf elianalf added ready-to-merge Ready to be merged. CI and changes have been reviewed and approved. and removed skip-ci Automatically pass CI labels Jul 2, 2024
@EduPonz EduPonz merged commit 9961e61 into master Jul 2, 2024
3 checks passed
@EduPonz EduPonz deleted the fix/21232 branch July 2, 2024 12:18
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
ready-to-merge Ready to be merged. CI and changes have been reviewed and approved.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants