Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Create license-compliant version of the Pile: subsets #301

Open
Tracked by #65
albertvillanova opened this issue Dec 3, 2021 · 0 comments · Fixed by huggingface/datasets#3378
Open
Tracked by #65

Create license-compliant version of the Pile: subsets #301

albertvillanova opened this issue Dec 3, 2021 · 0 comments · Fixed by huggingface/datasets#3378
Assignees
@albertvillanova
Labels
data catalog Gathering data from data sources

Comments

@albertvillanova
Copy link
Member

Subsets of The Pile:

  • pubmed
  • ubuntu_irc
  • europarl
  • hacker_news
  • nih_exporter
@albertvillanova albertvillanova added the data catalog Gathering data from data sources label Dec 3, 2021
@albertvillanova albertvillanova self-assigned this Dec 3, 2021
This was referenced Dec 3, 2021
Open
Merged
Repository owner moved this from Todo to Done in Data Tooling: Collecting Primary Catalogue Data Sources Dec 9, 2021
@albertvillanova albertvillanova added the language modeling script Need Language Modeling loading script label Jan 20, 2022
@albertvillanova albertvillanova removed the language modeling script Need Language Modeling loading script label Jan 27, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@albertvillanova albertvillanova

Labels
data catalog Gathering data from data sources
Projects
Data Tooling: Collecting Primary Catalogue Data Sources
Status: Todo
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

Add The Pile subsets huggingface/datasets
1 participant
@albertvillanova