Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add iNaturalist dataset #3292

Closed
vfdev-5 opened this issue Jan 25, 2021 · 9 comments · Fixed by #4123
Closed

Add iNaturalist dataset #3292

vfdev-5 opened this issue Jan 25, 2021 · 9 comments · Fixed by #4123

Comments

@vfdev-5
Copy link
Collaborator

vfdev-5 commented Jan 25, 2021

🚀 Feature

A feature request to discuss about about adding iNaturalist dataset wrapper of ImageFolder (maybe, even without downloading tools).

Motivation

We can see in the research code custom dataset implementation which can be also in torchvision. For example, here:

https://github.com/facebookresearch/deit/blob/ab5715372db8c6cad5740714b2216d55aeae052e/datasets.py#L13

Being also aware of our not-yet-defined procedure on which datasets to add, feel free to close the FR if we can not move forward.

cc @pmeier

@pmeier
Copy link
Collaborator

pmeier commented Jan 26, 2021

It seems to be cited 159 times. Given that this is not a general purpose classification dataset, I think it is reasonable popular. @vfdev-5 do you have insights if this is still used in contemporary publications or is even picking up in popularity?

@vfdev-5
Copy link
Collaborator Author

vfdev-5 commented Jan 26, 2021

According to scholar, there are already 9 papers citing the dataset in 2021. Code cited above is for Deit (12/2020) paper.
Looks like a popular dataset. Let's see other opinions here before going further.

@pmeier
Copy link
Collaborator

pmeier commented Jan 28, 2021

Just to be clear: I'm in favor of adding this.

@arch-user-france1

This comment was marked as resolved.

@pmeier
Copy link
Collaborator

pmeier commented Feb 28, 2023

@arch-user-france1 There is no mistake in the documentation since it doesn't describe the module, but rather the class:

class INaturalist(VisionDataset):

Seeing the AttributeError's above, I guess something is wrong with your environment. Please open a new issue following the steps in the form so we can have a look.

@arch-user-france1

This comment was marked as outdated.

@arch-user-france1
Copy link

arch-user-france1 commented Feb 28, 2023

I think there is a memory leak in the dataset or DataLoader - is it supposed to use more than 30GB of RAM? It does not free any memory, so a dataset of a this size would be unusable. Nobody has 200GB RAM...
I have not spotted any issue in my code, and I have seen a stackoverflow question describing the same problem.

@pmeier
Copy link
Collaborator

pmeier commented Feb 28, 2023

@arch-user-france1 This has nothing to do with this issue. Please open a new one following the steps in the form.

@nhewadehigah
Copy link

Hi Do anyone know how to access the supercategory label in 2021 version. In tensorflow it has supercategory label but not seen in pytorch.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging a pull request may close this issue.

5 participants