isFrameNew() erratic behavior with multiple cameras

[adielfernandez]

Using 3 Orbbec Astras (not Pro or S) for only their depth stream, i.e. initColorStream() never invoked.

All three cameras update and display their raw depth stream just fine, but the isFrameNew() method behaves strangely. I'm doing some really basic frame counting immediately after calling update on all three instances:

    cout << "Current Frame: " << ofGetFrameNum() << endl;
    if(astra1.isFrameNew()){
        cout << "Cam 1 Frame Number: " << cam1FrameNum << endl;
        cam1FrameNum++;
    }
    if(astra2.isFrameNew()){
        cout << "Cam 2 Frame Number: " << cam2FrameNum << endl;
        cam2FrameNum++;
    }
    if(astra3.isFrameNew()){
        cout << "Cam 3 Frame Number: " << cam3FrameNum << endl;
        cam3FrameNum++;
    }

"isFrameNew()" fires somewhat regularly for astra1, but I'm not convinced it's every frame. It's true about 10% as often for astra2 and never fires for astra3. What's weird is that the video feed appears smooth and clear, it's just the boolean that's not behaving. But I need the boolean to do some threading stuff.

[mattfelsen]

Hm, that's strange. I assume this is in your ofApp::update()? Can you try printing the frame number both there and inside ofxOrbbecAstra::on_frame_ready()? I wonder if there's something up with the order in which things get called which causes the bIsFrameNew to get set back to false before you get a chance to query it. Also do the cameras always print in the same order or does it seem random? (I'm guessing this may be tough to tell since you'd need multiple cameras to have new frames in the same app update loop)

[adielfernandez]

Sure, here's the print out. https://gist.github.com/adielfernandez/acb2ea819cd4a57b399115100fbc8bad

I let it run for a few seconds. You can search for "cam 1", "cam 2", "cam 3" to see when the isFrameNew() is true.

Also weird is that the on_frame_ready method runs kind of erratically in that print out, but I've seen it run more smoothly other times, i.e. 3 prints from on_frame_ready for every print from ofApp::update().

[mattfelsen]

Hm, that is pretty weird. Looking at frames 0 or 1, it looks like on_frame_ready() gets called twice, so you'd expect to see two cams update.

Here's an idea...I took a look at the MultiSensorViewer example, and there are a few similar things going on: each cam gets its own StreamSet, StreamReader, and listener class, just as would happen with the way the addon is set up now. However, astra_temp_update() only gets called once per update loop whereas each ofxOrbbecAstra instance calls it so it will get called 3 times per update loop. Maybe that's the source of some problems? You could try commenting that out in ofxOrbbecAstra::update() and instead calling it from your ofApp::update() to see if that fixes something.

Also what could be helpful is adding a string name property to each cam and setting it to "Cam 1" etc. so that you can tell which cam's on_frame_ready() is getting called

[adielfernandez]

Sorry for the delay, got pulled away.

I couldn't figure out how to call astra_temp_update() from ofApp::update() so I just used the device names to ensure that only sensor0 was actually invoking it, i.e. once per update:
https://github.com/adielfernandez/ofxOrbbecAstra/blob/master/src/ofxOrbbecAstra.cpp#L130

Didn't quite fix the issue though. In fact, it might have made it worse. Now, isFrameNew() is never invoked for the sensors 1 and 2. Feel free to dig around my fork of the repo and the multi cam example to see how I'm doing it if it helps:
https://github.com/adielfernandez/ofxOrbbecAstra/tree/master/multicam_example

Here's the new printout with better labeling for clarity since we're printing from a load of different places: https://gist.github.com/adielfernandez/908c2830599fe39da4195b05fdb8ec2c

As a side note, the main reason I needed these was to make sure my multi-threaded version wasn't flooding the thread with duplicate frames BETWEEN the new incoming frames. If sent too often it caused a build up and lag because of the queueing. I got around it by only sending the current cam pixels to the thread every 34ms (~30fps). Ideally isFrameNew() would be more precise, but this works for now :P

[mattfelsen]

Hrm, what happens when you try to call astra_temp_update()? ofApp.h includes ofxOrbbecAstra.h which includes <astra/astra.h>, so that function should be visible. Xcode seems fine when I do it. Weird, but your workaround seems fine.

Two thoughts here. One is you could change the implementation of isFrameNew(). Something like this, and also remove bNewFrame = false in ofxOrbbecAstra::update().

bool wasNew = bNewFrame;
bNewFrame = false;
return wasNew;

This would ensure that a frame is reported as new...I think that the flag is getting toggled back to false before you get a chance to check it. Something like that..

On a separate note regarding the queueing...you mean that you're sending data with ofThreadChannel which uses a queue so you end up with frames backing up, yea? I just ran into this myself. Instead of using while (channel.receive(pixels)){} in your threadedFunction() you can use a while loop with tryReceive() to pop all the data out of the queue and just end up with the most recent one. I did the same recently in another project and it worked well. Here's an example:
https://github.com/openframeworks/openFrameworks/blob/604902ad33fc4486eb539e8bad20d9fe55f1641d/examples/threads/threadChannelExample/src/ImgAnalysisThread.cpp#L38-L51