Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support actions / Add communication benchmarks directory #137

Open
wants to merge 9 commits into
base: mauro/iron-events-executor
Choose a base branch
from

Conversation

mauropasse
Copy link
Collaborator

@mauropasse mauropasse commented Nov 14, 2024

Commits:
d50eebd Support Actions and add comms_benchmark dir

  • comms_benchmark directory contains scripts to run benchmarks plus topologies:
    • Benchmarks for Pub/Sub, Cli/Srv, Actions
      • Single-process / Multi-process
      • Uses 10B/100KB/1MB/4MB
    • For example, irobot_benchmark/comms_benchmark/scripts/run_all_benchmarks.sh is the script to run all tests.
    • irobot_benchmark/comms_benchmark/scripts/parse-results contains scripts to parse the results.
  • This commit also adds support for actions, allowing to create them from the usual json topology files.

e67baa4 Improve error handling in data parsing and file operations
2fdd9bc warning if negative latency for multi-host benchmarks
5d4e998 Show latency even if msg arrived late
0f646cf Add option to create results directory
9def392 Remove irobot_events_executor dependency
24cf6b2 Remove option and always name threads

@mauropasse mauropasse requested a review from alsora as a code owner November 14, 2024 16:10
@mauropasse mauropasse changed the title Mauro/comms benchmark iron Support actions / Add communication benchmarks directory Nov 14, 2024
topology_basename.substr(0, topology_basename.length() - 5) + "_log";
std::string result_dir_name;

if (result_dir != "") {
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
if (result_dir != "") {
if (!result_dir.empty()) {

@@ -219,6 +252,7 @@ class PerformanceNodeBase
// Client blocking call does not work with timers
// Use a lock variable to avoid calling when you are already waiting
std::atomic<bool> m_client_lock {false};
std::atomic<bool> m_action_client_lock {false};
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

if i recall correctly, this assumes that you are only doing 1 request from each node.
i think that each client should have their own "lock", rather than 1 per node

// A service-name indexed map to store the client pointers with their
// trackers.
std::map<std::string, ClientsTuple> m_clients;
std::map<std::string, ActionClientsTuple> m_action_clients;
std::map<std::string, ActionServersTuple> m_action_servers;
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

minor: these could probably be unordered maps

auto client = std::static_pointer_cast<rclcpp_action::Client<Action>>(std::get<0>(client_tuple));

// Wait for action server to become available
if (!client->wait_for_action_server(std::chrono::seconds(1))) {
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this will block the whole node (e.g. maybe there are pubs-subs also that should be running).
I wonder if we should wait 0 here and just skip the request if not discovered.
then the test validation should check how many goals were sent.

maybe the same comment applies also to services


size_t non_sep_pos = split_right.find_first_not_of(sep);
if (non_sep_pos == std::string::npos) {
std::cerr << "Non-separator character not found after separator in line: " << line << std::endl;
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

what can cause this?
should we assert?

} else if (action_client_json.find("period_ms") != action_client_json.end()) {
period_ms = action_client_json["period_ms"];
} else {
std::cout << "Error! Action Clients must set period_ms or freq_hz in json file" << std::endl;
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

should we assert here?

@@ -39,7 +39,7 @@
{"topic_name":"parana", "msg_type":"stamped3_float32"}
],
"publishers": [
{"topic_name": "salween", "msg_type": "stamped12_float32", "period_ms": 100, "msg_pass_by":"loaned_msg"}
{"topic_name": "salween", "msg_type": "stamped12_float32", "period_ms": 100, "msg_pass_by":"shared_ptr"}
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why this change?

@mauropasse mauropasse force-pushed the mauro/comms-benchmark-iron branch 2 times, most recently from 1fb00f4 to d9fdf5c Compare December 11, 2024 15:34
@mauropasse mauropasse force-pushed the mauro/comms-benchmark-iron branch from d9fdf5c to 5a939f0 Compare December 13, 2024 18:55
@mauropasse mauropasse force-pushed the mauro/comms-benchmark-iron branch from 5a939f0 to 08f967b Compare December 16, 2024 16:25
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants