Augmentation by Perspective Transform

[soupault]

Hi!

I think the case of augmentation by perspective transform is very useful for real-life applications. Would you be interested in such function?

See http://www.euclideanspace.com/maths/discrete/groups/categorise/finite/cube/rotFace.png . Let the plane with a green 1 be an image. I'd suggest to parametrize the augmentation by angles around z and y, i.e. img_aug = aug_by_perspective(img, angle_vert, angle_horiz).

Implementing this should be relatively simple using OpenCV.

[aleju]

If there's an OpenCV/skimage function that works with z- and y-angles (or something close to that) it shouldn't be too hard to implement and seems kinda useful.
I likely won't have the time to implement it in the next weeks, but I can add it to the list.

[soupault]

@aleju ok, good! I'll try to find some time to implement this. Will keep you posted.

[soupault]

For the info: implementation of the proposed method in Augmentor - https://github.com/mdbloice/Augmentor/blob/master/Augmentor/Operations.py#L230

[soupault]

@aleju Hi! I've just noticed that you added an implementation in ecbcfea. Thanks a lot!
Should we close this issue? Do you find the API mentioned here to be a better option than the current one?

[aleju]

Yeah, there's now a perspective transform in the library, which is based on a four-point transformation. Currently, the four source points are randomly located and the four target points are just the corners of the image plane. Maybe in the future I change that so that the target points are also randomly sampled, which then might randomly sometimes perform the same augmentation as the perspective transform with vertical and horizontal angle.

For some applications vertical+horizontal angle might still be more useful, so at some point it will probably be added too.

[ergysr]

Is it possible to drop the restriction "PerspectiveTransform is currently limited to images with 4 or less channels."? It seems cv2.warpPerspective can support multiple channels.

[aleju]

Last time I tried images with more than 4 channels, OpenCV generated an error that seemed like the input was expected to a max of 4 channels.
Though should be possible to convert this to a loop that augments one (or four) channel(s) at a time.

[ergysr]

A more straightforward option is to remove the assertion on L992 from the code. Would it be possible to include this change in a future release?

[aleju]

I just pushed a change that allows to use any number of channels in PerspectiveTransform.

Just deleting the assert was not enough, because then cv2.warpPerspective() crashes for inputs with >4 channels.

[ergysr]

Perfect, thank you @aleju!

[JaviFuentes94]

Have you guys considered adding a fit_output parameter to the perspective transformation? In a lot of cases it is not feasible to have half an object in the image, so you need the perspective transformation to return the whole image padded.

[ZhengRui]

@JaviFuentes94 I have same feature request today and made a pull request, you may try this and see if it fits you. #452