Making 2+ processes run in non-blocking mode

[coder137]

Code

  reproc::options options;
  options.nonblocking = true;

  const char *sargv[] = { "ping", "localhost", "-n", "6", NULL };
  std::tie(status, ec) = reproc::run(sargv, options);
  std::cout << "status1: " << status << std::endl;
  if (ec) {
    std::cerr << ec.message() << std::endl;
  }

  const char *margv[] = { "cmake", "--help", NULL };
  std::tie(status, ec) = reproc::run(margv, options);
  std::cout << "status2: " << status << std::endl;
  if (ec) {
    std::cerr << ec.message() << std::endl;
  }

Unfortunately, the second reproc::run waits for the ping command to complete.

Question

1. Is there a way to run multiple processes simultaneously?
2. Is there a way to put a hard cap on the number of processes that can run simultaneously?

[DaanDeMeyer]

The reproc_poll (or reproc::poll in C++) should allow to implement this. If you check the C examples, there's a detailed example on how to use reproc_poll to run multiple processes at once. Although it might be nice to write an extra function on top of it to run multiple processes concurrently and wait for them all to complete. We don't currently have that though. Note that the reproc++ API for poll is currently a bit bad until we can update reproc++ to minimum C++17 so we can use std::optional.

[coder137]

Thank you, This worked for me!

On a different note:

I am currently checking to see the benefits of using the regular system C API over reproc since I just need to invoke an external program and get a return code.
According to you would reproc be more performant in this scenario?

Please let me know if I should start a new discussion for this.
Thanks

[DaanDeMeyer]

You're not likely to notice a difference in performance. But a process launched by system cannot be interrupted, stopped, or killed if it hangs. You also can't separate stdout and stderr and you lose out on the myriad of redirection options that reproc provides.

reproc also does stuff like closing file descriptors when starting a new process and resetting signal handlers on POSIX that prevent child processes from interfering with the parent process, but those things only become a big problem when you have long running child processes. Finally, system relies on the shell which can be a bit of a security issue sometimes.

If all those things aren't that important to you and you're just running a short-lived program that's wellbehaved and you just need a return code, system might fit your needs and you won't need reproc.

[coder137]

Security and performance are the main reasons for me to consider using reproc over system. I can see that reproc does not allow spaced input to process.start which is a good first step for sanitizing my inputs.

With respect to performance, however, I have started a new discussion #57 where I have run some basic benchmarks.