Use the GPU

[qbonnard]

The image processing part of Chilitags could be done on a GPU if available. This would speed up detection a lot.

[ayberkozgur]

Séverin and I discussed this and did some profiling. It turns out that the most time consuming single algorithm is Canny. As a first step, we will try to implement the Canny algorithm of FindQuads in a shader and see if that gets us somewhere.

[qbonnard]

Have you had a look at previous work ? https://duckduckgo.com/?q=canny+gpu

[severin-lemaignan]

yes ;-)

[qbonnard]

Any news on the GPU front ?

[ayberkozgur]

I spent a few days on it last week, and managed to get a working implementation of the Canny filter on the CPU. I tested it with Chilitags and it works as well as the OpenCV's implementation. After I got the first part of the filter running in shaders, I realized that getting the data back from the framebuffer into the CPU is extremely slow. There are faster alternatives to this, like pixelbuffers, but they are limited to GLES 3.0, which is not yet supported by the majority of the devices out there.

So it seems that our effort of running just Canny on the GPU is futile at this point. Either we port the whole algorithm to shaders and hope that the data we get in the end (tag transforms) is small enough to be transferred to the CPU fast, or we accept the CPU speed processing and tweak things with the performance parameters.

[qbonnard]

An intermediate could be to "only" port FindQuads to shaders. It outputs only (x,y) coordinates for each corner of each quadrilateral, which shouldn't be too much data to copy. I guess the biggest missing thing to port in this case would be the algorithm to extract and simplify contours...

[ayberkozgur]

You're right, that might be an option. The reason we tried to port Canny first is because we profiled the whole thing and Canny took more than 50% of the time.

[qbonnard]

Ah yes, that's definitely the best place to start !
(This close & comment button is really annoying ;) )

[ayberkozgur]

Exactly, I keep hitting it with my finger accidentally on the phone :)

[severin-lemaignan]

OpenGL transformFeedback is probably the way to go.