Transformations in GPU

[edgarriba]

Is there any plan to support image transformations for GPU?
Doing big transformations e.g resizing (224x224) <-> (64x64) with PIL seems a bit slow.

[soumith]

@edgarriba we are thinking about it. It seems like a good idea, and NVIDIA has the relevant GPU kernels already implemented.

[edgarriba]

nice! that good could be a very appreciate feature 😉

[edgarriba]

also OpenCV seems a bit faster with this, however I think that such GPU routines are not available in Python

[Maratyszcza]

@edgarriba You may try pytorch/accimage with PR #15 to leverage Intel IPP for preprocessing. Keep in mind that this package is beta (like in beta than nothing) and barely tested.

[edgarriba]

@Maratyszcza thx to point me to this, looks pretty good. I'll try to run and give feedback

[ghost]

I'd be happy to give this a go, if there's still interest?

[alykhantejani]

I think there might still be interest in having this cc @soumith who might have a stronger opinion here

[radenmuaz]

so any plans for GPU data-augmentations or do we still stick to cpu PIL?

[edgarriba]

@radenmuaz in kornia.org we recently introduced an API for that. Not only supports GPU but also differentiablility. Please check the kornia.augmentation module.

[qhaas]

Those of us on non-x86-64 systems would benefit from GPU accelerated transformations given the lack of optimizations for our architecture at times. For example, we are seeing significant performance disparities between x86-64 and power9 systems that use the same GPU and profiling points the finger at the Power9 CPU taking longer to crunch torchvision/transforms and PIL.

[fmassa]

@qhaas #2278 is a first step towards making the transforms run seamlessly on the CPU / GPU via torch tensors. For now it will be on a per-image basis so GPUs will probably be slow, but in the future with NestedTensor support we might be able to make it efficient for batches of different image sizes

[kaoutar55]

Could someone clarify if this piece of code is doing the transformation in CPU or GPU. It is not clear to me if the current torchvision library is supporting GPU transformations or all is done on CPUs at this point.
train_dataset =
datasets.ImageFolder(args.train_dir,
transform=transforms.Compose([
transforms.RandomResizedCrop(224),
transforms.RandomHorizontalFlip(),
transforms.ToTensor(),
transforms.Normalize(mean=[0.485, 0.456, 0.406],
std=[0.229, 0.224, 0.225])
]))

[frgfm]

@kaoutar55 In your example, the transformations will be performed on CPU

The reason behind that is the following:

• when transformations are passed to a VisionDataset, the transforms will be applied to each sample in each worker run by the dataloader.
• here ImageFolder is inherited from VisionDataset and what I mentioned above is applicable

Since v0.8.0, you can use transforms on GPU (cf. example in the release note).

[fmassa]

@kaoutar55 the comments from @frgfm are spot on.

I'm closing this as we have added GPU support for the transforms with the 0.8.0 release of torchvision https://github.com/pytorch/vision/releases/tag/v0.8.0

[wetliu]

Thank you so much for your new GPU support. @fmassa

May I ask why there is some slight differences results between the nn.Sequencial version and the transforms.Compose version even just in the resize case? Which one should be correct?

import numpy as np
import torch
from torchvision import datasets, transforms
from PIL import Image
from torch.utils.data import Dataset, DataLoader
import random
torch.manual_seed(222)

class FunnyDataset(Dataset):

    def __init__(self, transform1=None, transform2=None):
        self.transform1 = transform1
        self.transform2 = transform2
        self.data = np.random.randint(256, size=(5, 84, 84, 3), dtype=np.uint8)
    
    def __len__(self):
        return 5

    def __getitem__(self, idx):
        x = self.data[idx]

        x = Image.fromarray(x)
        x1 = self.transform1(x)
        
        x2 = self.transform2(x)

        return x1, x2


transform1 = transforms.Compose([
            transforms.Resize((32,32)),
            transforms.ToTensor(),
            ])

transform2 = transforms.Compose([transforms.ToTensor()])

transforms = torch.nn.Sequential(
            transforms.Resize((32,32)),
            )

funny_dataset = FunnyDataset(transform1, transform2)

dl = DataLoader(funny_dataset, 5, shuffle=False, num_workers=1, pin_memory=False)
for data in dl:
    x1, x2 = data
    x2 = transforms(x2)
    print(x1.min(), x1.max(), x2.min(), x2.max())
    print(((x1-x2)**2).sum())

output:

tensor(0.1961) tensor(0.7765) tensor(0.0179) tensor(0.9847)
tensor(365.3533)

[frgfm]

Hi @wetliu 👋

The difference comes from the different behaviours between PIL Image interpolation (in your x1) and PyTorch tensor interpolation (in your x2). This matter is quite different from the question of the device where the operation is performed.

I would suggest checking #2950 👍

Hope this helps!

[binn77]

How to switch processing to GPU

resize = transforms.Compose([ transforms.Resize((128,128)), transforms.ToTensor()])

[frgfm]

How to switch processing to GPU

resize = transforms.Compose([ transforms.Resize((128,128)), transforms.ToTensor()])

Hello @binn77 👋

Transforms, even in their "Module/Compose" form, have no learnable parameters (at least in the current API, to the best of my knowledge). In PyTorch, an operation depending on the location of input tensors will:

• throw an error if tensors are on different devices
• go along on CPU if both of them are on the CPU
• go along on GPU if both of them are on the same one

So you need to move the input tensor to your GPU (and your model if you're using one afterwards). You have two options:

• if your input image is a Pillow one, you can only move it after it was turned into a tensor

from PIL import Image
from torchvision.transforms import Compose, Resize, ToTensor

with Image.open("path/to/img.jpg", mode='r') as f:
    img = f

transfo = Compose([ Resize((128,128)), ToTensor()]) 
input_tensor = transfo(img)
input_tensor = input_tensor.cuda()

Hope this helps!

[binn77]

How to switch processing to GPU
resize = transforms.Compose([ transforms.Resize((128,128)), transforms.ToTensor()])

Hello @binn77 👋

Transforms, even in their "Module/Compose" form, have no learnable parameters (at least in the current API, to the best of my knowledge). In PyTorch, an operation depending on the location of input tensors will:

• throw an error if tensors are on different devices
• go along on CPU if both of them are on the CPU
• go along on GPU if both of them are on the same one

So you need to move the input tensor to your GPU (and your model if you're using one afterwards). You have two options:

• if your input image is a Pillow one, you can only move it after it was turned into a tensor

from PIL import Image
from torchvision.transforms import Compose, Resize, ToTensor

with Image.open("path/to/img.jpg", mode='r') as f:
    img = f

transfo = Compose([ Resize((128,128)), ToTensor()]) 
input_tensor = resize(img)
input_tensor = input_tensor.cuda()

Hope this helps!

[frgfm]

@binn77 my bad I fixed the snippet, it's input_tensor = transfo(img)