ROIPooling layer in fast and faster R-CNN

[chuzui]

Now Fast R-CNN and Faster R-CNN are start-of-the-art image detection methods. The most important component of these method is a ROI pooling layer and the authors implemented it in caffe.

I find it may be difficult to implement the ROI pooling layer using the ops in theano. Is there anyone has any idea? Or we can only implement it with C extension?

[benanne]

It looks like this is just max-pooling with a pool size dependent on the input, so that the output always has the same size (e.g. 7x7)? That should be fairly simple to implement in pure Theano. And since it's unlikely to be a very time-consuming part of a network, making a faster C implementation probably isn't worth it.

[chuzui]

It looks like this is just max-pooling with a pool size dependent on the input, so that the output always has the same size (e.g. 7x7)? That should be fairly simple to implement in pure Theano. And since it's unlikely to be a very time-consuming part of a network, making a faster C implementation probably isn't worth it.

But the input of ROIPooling layer in a batch is several object proposal sub-windows of the same image. They have different sizes and all max-pooling to the same size (e.g. 7x7) with different pooling sizes. So i think it's not very easy to implement.

[benanne]

Right, in that case it's going to be tough to avoid scan, or something like that. A custom CUDA kernel might even be worth considering (it's fairly easy to wrap them in Theano using PyCUDA).

[kshmelkov]

Can't it be emulated via TransformerLayer? IIRC, it should do the trick if you transform bbox coordinates into affine transform parameters.

[f0k]

The TransformerLayer will do bilinear interpolation, though, not max-pooling. You could use it to extract regions scaled to a fixed target size, but not to implement the ROI pooling discussed here.
Looking at the implementation, it seems the most efficient solution will indeed be wrapping it into a custom kernel.
Implementing it in pure Theano will probably be slow. You'd need to theano.scan() over the region proposals, extract the corresponding subtensor, subdivide that again (to get, e.g., 7x7 subregions) and take the maximum of each.

[kshmelkov]

Fair point. Somehow I missed that it is called pooling for reason. Anyway I am messing around faster rcnn and I almost finished implementation of ROI 'pooling' via TransformerLayer. I hope it doesn't make a significant difference.

[faizankshaikh]

Has anything been done for this issue?

[f0k]

Has anything been done for this issue?

No, but the deepdetect issue linking to ours has a Theano implementation posted: https://github.com/ddtm/theano-roi-pooling
This could be integrated into Theano and wrapped as a Lasagne layer, or integrated into Lasagne, or just be used as a basis for a Lasagne Recipe.

[faizankshaikh]

That seems reasonable. Thanks!

[f0k]

Feel free to submit a PR to Lasagne/Recipes when you got it working, or send a PR to Theano for that Op and ping us back!

[Sentient07]

Hi, I have made a draft here, Theano/Theano#5189 . Could you please have a look and let me know if the Op is implemented correctly?

Ramana

[Sentient07]

Hi, If anyone is interested, the code for fast-RCNN, with installation instructions are here, Lasagne/Recipes#35 (comment)