hello world example returns empty packets on packet.get() after converting to wasm

[tso996]

Hello,
I'm trying to make the mediapipe hello world cpp example run in the browser by converting the cpp file to web assembly. I have converted it to webassembly and loaded the wasm file into the browser and received debug messages via cout strings as output in the chrome browser console. I followed this repo to achieve this. However, the wasm hangs at poller.next(&packet), as is evident from observing the debug flags I had put in the cpp code from the console. I did some digging in the internet and found out that it is hanging because poller.next() waits until the next packet is available. The same code is working properly when run from the terminal as a cpp file but upon converting to wasm something is not working. It would be great if someone could provide more information on why the code behaves as such upon being converted to wasm.

build file for cpp helloworld

cc_binary(
    name = "hello_world",
    srcs = ["hello_world.cc"],
    visibility = ["//visibility:public"],
    deps = [
        "//mediapipe/calculators/core:pass_through_calculator",
        "//mediapipe/framework:calculator_graph",
        "//mediapipe/framework/port:logging",
        "//mediapipe/framework/port:parse_text_proto",
        "//mediapipe/framework/port:status",
    ],
)

cpp helloworld code that runs properly

#include "mediapipe/framework/calculator_graph.h"
#include "mediapipe/framework/port/logging.h"
#include "mediapipe/framework/port/parse_text_proto.h"
#include "mediapipe/framework/port/status.h"

//namespace mediapipe {

int PrintHelloWorld() {
  // Configures a simple graph, which concatenates 2 PassThroughCalculators.
  mediapipe::CalculatorGraphConfig config =
      mediapipe::ParseTextProtoOrDie<mediapipe::CalculatorGraphConfig>(R"pb(
        input_stream: "in"
        output_stream: "out"
        node {
          calculator: "PassThroughCalculator"
          input_stream: "in"
          output_stream: "out1"
        }
        node {
          calculator: "PassThroughCalculator"
          input_stream: "out1"
          output_stream: "out"
        }
      )pb");

  mediapipe::CalculatorGraph graph;
  graph.Initialize(config);
  auto status_or_poller = graph.AddOutputStreamPoller("out");
  mediapipe::OutputStreamPoller poller = std::move(status_or_poller.value());
  graph.StartRun({});
  // Give 10 input packets that contains the same std::string "Hello World!".
  for (int i = 0; i < 10; ++i) {
    graph.AddPacketToInputStream(
        "in", mediapipe::MakePacket<std::string>("Hello World!").At(mediapipe::Timestamp(i)));
  }
  // Close the input stream "in".
  graph.CloseInputStream("in");
  mediapipe::Packet packet;
  // Get the output packets std::string.
  while (poller.Next(&packet)) {
    std::cout << poller.QueueSize()<< std::endl;
    std::cout << packet.Get<std::string>()<<std::endl;
  }
  // return graph.WaitUntilDone();
}
//}  // namespace mediapipe

int main(int argc, char** argv) {
  google::InitGoogleLogging(argv[0]);
  PrintHelloWorld();
  //std::cout << "hello world!!!!!!!!!" << std::endl;
  return 0;
}

output as expected in the terminal

build file for the wasm helloworld:

cc_binary(
    name = "hello-world-web",
    srcs = ["helloWorldWasm.cc"],
    deps = [
        "//mediapipe/calculators/core:pass_through_calculator",
        "//mediapipe/framework:calculator_graph",
        "//mediapipe/framework/port:parse_text_proto",
        "//mediapipe/framework/port:status",
        "//third_party:glog",
        "//mediapipe/framework/port:logging",
    ],
    linkopts = [
        "-s USE_PTHREADS=0",
        "-s ALLOW_MEMORY_GROWTH=1",
        "-s ASSERTIONS=1",
        "-s USE_WEBGL2=1",
        "-s ERROR_ON_UNDEFINED_SYMBOLS=0", 
        "--bind",
    ]
)


wasm_cc_binary(
    name = "hello-world-wasm",
    cc_target = ":hello-world-web",
)

cpp code used to compile the wasm file:

#include "mediapipe/framework/calculator_graph.h"
#include "mediapipe/framework/port/logging.h"
#include "mediapipe/framework/port/parse_text_proto.h"
#include "mediapipe/framework/port/status.h"

#include <emscripten/bind.h>
//#include <emscripten.h>


    int PrintHelloWorld() {
                std::cout << "flag 1"<< std::endl;
                // Configures a simple graph, which concatenates 2 PassThroughCalculators.
                mediapipe::CalculatorGraphConfig config =
        mediapipe::ParseTextProtoOrDie<mediapipe::CalculatorGraphConfig>(R"pb(
            input_stream: "in"
            output_stream: "out"
            node {
            calculator: "PassThroughCalculator"
            input_stream: "in"
            output_stream: "out1"
            }
            node {
            calculator: "PassThroughCalculator"
            input_stream: "out1"
            output_stream: "out"
            }
        )pb");

            mediapipe::CalculatorGraph graph;
            graph.Initialize(config);
            auto status_or_poller = graph.AddOutputStreamPoller("out");
            mediapipe::OutputStreamPoller poller = std::move(status_or_poller.value());
            graph.StartRun({});
            std::cout << "flag 2"<< std::endl;
            // Give 10 input packets that contains the same std::string "Hello World!".
            for (int i = 0; i < 10; ++i) {
                graph.AddPacketToInputStream(
                    "in", mediapipe::MakePacket<std::string>("Hello World!").At(mediapipe::Timestamp(i)));
            }
            // Close the input stream "in".
            graph.CloseInputStream("in");
            std::cout << "flag 3"<< std::endl;
            mediapipe::Packet packet;
            // Get the output packets std::string.
            while (poller.Next(&packet)) {
                std::cout << "flag 4"<< std::endl;
                std::cout << poller.QueueSize()<< std::endl;
                std::cout << packet.Get<std::string>()<<std::endl;
            }
            // return graph.WaitUntilDone();
            std::cout << "flag 5"<< std::endl;
            return 1;
    }
        int main(int argc, char** argv) {
            google::InitGoogleLogging(argv[0]);
            std::cout << "hello world wasm file loaded." << std::endl;
            PrintHelloWorld();
            return 0;
          }


     EMSCRIPTEN_BINDINGS(hello_world_module) {
             emscripten::function("PrintHelloWorld", &PrintHelloWorld);
          
     }

the output from chrome console:

The chrome tab hangs and will need multiple clicks to close upon loading the wasm file. I guess it is not throwing any errors since packet.next() function is behaving as it is intended to behave, however, I would like to understand why the packets aren't in the queue like it is in the cpp example when run in the terminal.
The machine I used to compile the code to wasm is mac m1 running MacOS Montenery.
Any help is much appreciated.

[sureshdagooglecom]

Hi @tso996 ,
Did you get a chance to check on this closed issue #2444

[tso996]

@sureshdagooglecom Hi, I've read that and it wasn't helpful. I think this issue is related to js and not ios as you have tagged. Anyway, I did manage to solve it with guidance from @tyrmullen. I will outline them below.

• replace OutputStreamPoller with ObserveOutputStream
• use WaitUntilIdle instead of graph.WaitUntilDone()
  The issue was because the cpp example uses multithreading. So any function like poller.next() that waits on other threads won't work on webassembly.
  WaitUntilIdle() forces all the commands to execute inline (if I understand correctly).
  For anyone that is stuck here, check out the comments here.

[google-ml-butler]

Are you satisfied with the resolution of your issue?
Yes
No