Develop proper facial expressions handler – Stint 2

[Nicogene]

Task description

This is the follow-up of #50.

We ought to address these points:

• Refactor the module in order to use thrift
• Investigate and overcome the waitkey issue
• investigate and overcome the fullscreen issue
• When the module starts a default emotion or a black image has to be shown
• Manage the images using the context/ResourceFinder machinery
• Add ergoCubEmotions to CI
• Add it to the root CMakeLists.txt with an option (e.g. COMPILE_ergocubEmotions)

Definition of done

All those points are addressed by a PR and it has been merged

cc @pattacini

[lrapetti]

Hi @Nicogene @pattacini @martinaxgloria, is there any plan to load also video instead of images?

cc @Gianmarcogatti93

[martinaxgloria]

Yes @lrapetti, I already tried to show simple mp4 videos and it's possible to do 👍

[martinaxgloria]

Regarding the waitKey() issue, today I found a way to trigger the function and get the reply before the loop starts by using the function pollKey() in the HighGUI module.

The video below shows how it works now. In this way, it's possible to switch from one image to another without remaining blocked. Of course it could be improved but this was the biggest limit of the module so far.

module.mp4

[pattacini]

Nice progress!

[pattacini]

Regarding this:

Manage the images using the context/ResourceFinder machinery

I propose organizing the conf files as follows:

ergoCubEmotions/
├── config.ini
└── expressions
    ├── images
    │   ├── exp_img_0.png
    │   └── exp_img_1.png
    │   └── ...
    └── videos
        ├── exp_mov_0.mp4
        └── exp_mov_1.mp4
        └── ...

The content of the main file config.ini containing the configuration options may look like this:

[general]
num_expressions 10

[expression_0]
name happy
type image
file images/exp_img_0.png

[expression_1]
name sad
type image
file images/exp_img_1.png

...

[expression_9]
name cunny
type video
file videos/exp_mov_0.mp4

The module may then manage a std::map to handle the association between the tag of type string (e.g., happy, sad...) and the corresponding information for which we could create an ad-hoc structure to contain the type, the file, and the object¹ to display (image or video).

Footnotes

1. By exploiting the object-oriented paradigm of C++, we can design a class to wrap images and videos and expose a simple API for displaying purposes. ↩

[martinaxgloria]

Hi @pattacini, thank you for the advice. Do you suggest handling this task before thrifting the module or it's the same?

[pattacini]

Hi @pattacini, thank you for the advice. Do you suggest handling this task before thrifting the module or it's the same?

They're two independent sub-tasks. You could complete thrifting the module first and then address its configuration. While doing the latter, you may also need to adjust thrift methods. This is perfectly fine.

[martinaxgloria]

Clear, thank you!

[martinaxgloria]

Hi, I added the thrift to the module in this fork but now I've some problems with the rpc protocol I haven't before.

In particular, after launching the module and the yarp rpc /port , when the service of the idl.thrift is called, it takes some time to execute (it should be called two or three times since the module seems to recall it and respond to the request as accomplished). Below there is the video that shows better what happens (even if it's not shown in the capture, the module executes the command the third time I tried to execute it)

module.2.mp4

cc @pattacini @Nicogene

[martinaxgloria]

After chatting with @pattacini about the problems reported above, we came up to these conclusions:

• regarding the waitKey(), it's enough to use it only the first time the window used to show images is open and not after each imshow() function as I did before;
• in order to not remain blocked in the loop until a key is pressed, it's mandatory to pass to the function waitKey a delay in ms: for example, waitKey(1000) means that the function waits for a key event for 1000 ms and then if no key event is registered, it goes ahead;
• we tried different delays but, for example, with 25 ms (a small value) no event is registered and no window is open.

After these attempts, we decided to maintain a delay of 1000 ms for now.

[pattacini]

The use of a magic number (i.e., 1000) is an antipattern. At any rate, we will go on this way for the moment.

[martinaxgloria]

Today I implemented the context/ResourceFinder mechanism to manage the images and the updated module is here. If you're ok, I could open a PR to see the current state of the implementation.

cc @Nicogene @pattacini

[pattacini]

Fine with me!

[pattacini]

PR available here:

• ergoCubEmotions module - stint 2 #87