Pre-chipped datasets: How to convert geotiff chips from 32 to 8 bits per band

[bitsandbricks]

Hi there!

I have a pre-chipped dataset (RGB images and their masks) to import via SemanticSegmentationImageDataset()

The problem is that these chips are 32 bit multiband geotiff files, so they can't be read -a maximum of 8 bits per channel is currently supported.

Do you have any recipes to batch convert these chips to 8 bits per band, so they can be used with Rastervision ?

Thanks!

[AdeelH]

a maximum of 8 bits per channel is currently supported.

Hmm, I'm not sure if this is the case. What error are you running into?

[bitsandbricks]

Hi @AdeelH ! Thanks for looking into this.

So, I setup my pre-chipped dataset like this:

from rastervision.pytorch_learner import SemanticSegmentationImageDataset

ds = SemanticSegmentationImageDataset(
    img_dir='/home/havb/Downloads/CHIPS/images',
    label_dir='/home/havb/Downloads/CHIPS/labels',
    transform=A.Resize(chip_size, chip_size),
)

No problem there. Then when I try to read any of the images...

x, y = ds[0]
x.shape, y.shape

I get this kind of error:

UnidentifiedImageError: cannot identify image file '/home/havb/Downloads/CHIPS/images/2016-NOV-MOSAIC-SENTINEL_RGB_236_108.tif'

That error seems to come from this operation:

File ~/micromamba/envs/rastervision/lib/python3.9/site-packages/rastervision/pytorch_learner/dataset/utils/utils.py:41, in load_image(path)
     39     img = np.load(path)
     40 else:
---> 41     img = np.array(Image.open(path))
     43 if img.ndim == 2:
     44     # (h, w) --> (h, w, 1)
     45     img = img[..., np.newaxis]

According to this, that error is to be expected because the Python Imaging Library is only able to open 8 bit per channel in multi-channel images. Mine are 32 bit per channel so I need to convert them, and making sure that the output is compatible with the RV pipeline.

I've seen -and tried- a few recipes online, but my results so far don't look like usable RGB files.

So, I wonder if you have solved something like this before :)

[AdeelH]

Oh, I was not aware of that limitation of PIL. I suppose we should use rasterio if the file is a TIFF.

You could try modifying the source to do that as a workaround if that's convenient for you. (A PR would be welcome as well!)

Another option would be to convert them to numpy arrays. You could write a simple script that reads in each image and saves it in the numpy file format.

[bitsandbricks]

Got it! I'll see what I can do.