Getting started datasets

[bartjkdp]

The tutorial dataset is rather small and the Wikipedia dataset is rather large. It would be nice to provide some datasets that are somewhere in between to help people getting started.

We could also provide a nice interface to download these datasets, similar to scikit-learn that aids at downloading these datasets. This will simplify the tutorials.

[robotcator]

Hello, I also think it is nice to provide a nice API to download the corpus. So anyone works on this?
As I often use the text8.zip dataset in my experiment, I write an script to download from the Internet. (https://gist.github.com/robotcator/1fb0cdc1437515f5662d33368554f4c8)

[macks22]

Some good candidate datasets are:

• 20 Newsgroups
• NSF Research Award Abstracts 1990-2003
• Reuters-21578 Text Categorization Collection
• Reuters Corpus Volume I (RCV1) v2
• WebKB -- original here
• USPTO Patent Grant Full Text subsets

[macks22]

Some nice references for how to implement:

• scikit 20 newsgroup fetch function
• streaming file download in Python
• nltk downloader code

[menshikh-iv]

Resolved in #1705.

[piskvorky]

@menshikh-iv which of the datasets from @macks22 are there? #1705 is rather terse on information.

We especially want all the practical, domain-specific corpora: USPTO patents, the Quora QA duplicates dataset, the PubMed corpus…

Having a clean, uniform interface for downloading and opening these corpora from Python is already super useful.