-
Notifications
You must be signed in to change notification settings - Fork 641
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Update readme to detail the lerobot dataset format #275
Conversation
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks so much @thomwolf . I have comments but my disposition is approved / non-blocking.
Here are the important details and internal structure organization of a typical `LeRobotDataset` instantiated with `dataset = LeRobotDataset("lerobot/aloha_static_coffee")`. The exact features will change from dataset to dataset but not the main aspects: | ||
|
||
``` | ||
dataset attributes: |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I wish there was a way of flagging this snippet so that there's a CI mechanism to draw our attention to it in case we change anything in the datasets module ... or something, anything to make sure it doesn't go stale. I wonder if someone at HF has solved this problem...
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Yes indeed, we have that in transformers
for docstring examples. It's quite a lot of work to setup and maintain so I would advise to wait that the library has really built a large community of users before diving in this.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Very nice!
One change for me, feel free to take the nits or not.
Thanks @alexander-soare and @aliberts Added all the suggestions. Feel free to merge when you feel like. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thank you @thomwolf! 🙏
All good to me, I'll let @alexander-soare merge if it's good for him too
What this does
This Pull Request detail how the LeRobot Dataset format is structured and created.
It also adds a mention of the often overlooked
DATA_DIR
environment variable.It's extracted from #246
How it was tested
It's only a readme update thus no tests.
How to checkout & try? (for the reviewer)
Read the changes.
This change is