Skip to content

AndyTheFactory/romanian-nlp-datasets

Folders and files

NameName
Last commit message
Last commit date

Latest commit

ย 

History

45 Commits
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 

Repository files navigation

Awesome

A list of Romanian NLP Datasets

A curated list of open source and open access Romanian Language NLP Datasets. For the moment we don't add parallel copora to the list.

For additions or any other changes please submit a pull request.

Table of contents

Unlabeled text Corpora

The FuLG dataset is a comprehensive Romanian language corpus comprising
150 billion tokens, carefully extracted from Common Crawl. 

arXiv

Part of a large multilanguage corpus originated from Common Crawl.
It's a raw, unannotated corpus. It has roughly 50 GB of Romanian text
in 4.5 million documnets. For details check its homepage 
and the paper

arXiv Homepage

 Similar to Oscar, part of a multilanguage corpus also based on Common Crawl
 from 2018. Romanian text is 16GB large

arXiv Homepage

  Romanian language wikipedia dump. 
  A collection of varoius unannotated corpora collected around 2018-2019.
  Includes books, scraped newspapers and juridical documents  
  A collection of written and spoken text from various
  sources: Articles, Fairy tales, Fiction, History, Theatre, News
 Romanian national legilation from  1881 to 2021. The corpus
 includes mainly: governmental decisions, ministerial orders,
 decisions, decrees and laws.
 Automatically annotated for Named Entities

ACL Homepage

Mega-COV is a billion-scale dataset from Twitter for studying COVID-19. It is available in over 100+ languages, Romanian being one of them. Tweets need to be rehydrated

arXiv Medium

A corpus of Romanian tweets related to COVID and vaccination against COVID, created and collected between January 2021 and February 2022. It contains 19319 tweets.

Minutes of the Sittings of the Chamber of Deputies of Romania (2016-2018)
Unannotated corpus
contains 500k+ instances of speech from the parliament podium from
1996 to 2018. Sentence splitting and deduplication onm sentence level
have been applied as processing steps
Unannotated corpus
Romanian presidential discouses (1990-2020) split in 4 files
one for each president. Unannotated corpus
Monolingual Romanian corpus, including content from public websites related to culture

Monolingual (ron) corpus, containing 38063991 tokens and 854096 lexical types in the law domain.

Monolingual Romanian corpus, containing 360833 sentences (9064764 words) in the public administration domain.

The New Civil Procedure Code in Romanian (monolingual) comprising 297888 words.

The Romanian updated criminal code: text with law content.

news articles dataset from romanian newssites title, summary and article

multi-language corpus from online available news sources. It contains also 43mil words in Romanian language from Twitter, Blogs and Newspapers

Homepage

The Romanian novel collection for ELTeC, the European Literary Text Collection Sources: Biblioteca Metropolitana din Bucuresti, Biblioteca Universitara "Mihai Eminescu" din Iasi, Biblioteca Judeteana din Botosani, personal micro-collections uploaded on Zenodo under the following labels: "Hajduks Library"; "RomanianNovel Library"; "CityMysteries Library"; "BibliotecaDHL_Iasi"

Public dataset of 1447 manually annotated Romanian business-oriented emails. The corpus is annotated with 5 token-related labels, as well as 5 sequence-related classes

MDPI

The corpus consists of texts written by Romanian authors between 19th century and present, representing stories, short-stories, fairy tales and sketches. The current version contains 19 authors, 1263 full texts and 12516 paragraphs of around 200 words each, preserving paragraphs integrity.

A dataset containing 400 Romanian texts written by 10 authors The dataset contains stories, short stories, fairy tales, novels, articles, and sketches written by Ion Creangฤƒ, Barbu ลžtefฤƒnescu Delavrancea, Mihai Eminescu, Nicolae Filimon, Emil Gรขrleanu, Petre Ispirescu, Mihai Oltean, Emilia Plugaru, Liviu Rebreanu, Ioan Slavici.

MDPI

891 Cooking Recipes in Romanian Language

Semantic Textual Similarity / Paraphrasing

Semantic Textual Similarity dataset for the Romanian language RO-STS contains 8,628 sentence pairs with their similarity scores

NeurIPS

A paraphprase corpus created from 10 different Romanian language Bible versions. The final dataset contains 904,815 similar records and 218,977 non matching records, totaling 1,123,927

Around ~100k examples of paraphrases. No clear explanation on how the dataset was built

A multi-language paraphrase corpus for 73 languages extracted from the Tatoeba database. It has ~ 2000 romanian phrases totaling 941 paraphrase groups.

ACL Homepage

Natural Language Inference

The repository seems to be just an attempt at starting to build the dataset

Summarization

Around ~72k Full texts and their summary. Source seems to be news websites. No description or explanation available

Dialect and regional speech identification

varied compilation of speech samples from five distinct regions of Romania, covering both urban and rural environments. Around 2800 records labeled with age, gender and type of dialect

arXiv

MOROCO: The Moldavian and Romanian Dialectal Corpus The MOROCO data set contains Moldavian and Romanian samples of text collected from the news domain. The samples belong to one of the following six topics: culture, finance, politics, science, sports, tech totaling over 32.000 labeled records

arXiv

Named Entity Recognition (NER)

Autorship Attribution

Sentiment Analysis

Dependency Parsing

Diacritics Restoration / Grammar Correction

Fake News / Clickbait / Satirical News

Offensive Language

manually annotated 4,052 comments on a Romanian local news website into one of the following classes: non-offensive, targeted insults, racist, homophobic, and sexist.

arXiv

4455 organic generated comments from Facebook live broadcasts annotated not binary offensive language detection tasks and for fine-grained offensive language detection

IEEE

4800 Romanian comments annotated with offensive text spans Offensive span detection

MDPI

3860 labeled hate speech records

Dataset consists of 5000 tweets, from which 924 were labeled as offensive (18.48 %) and 4076 tweets as non-offensive.

ACL

The corpus contains 39 245 tweets, annotated by multiple annotators, following the sexist label set of a recent study.

ACL

Questions and Answers

This dataset is just the translation of the gsm8k dataset. GSM8K (Grade School Math 8K) is a dataset of 8.5K high quality linguistically diverse grade school math word problems. There is no information on the quality of the translation

RoCode, a competitive programming dataset, consisting of 2,642 problems written in Romanian, 11k solutions in C, C++ and Python and comprehensive testing suites for each problem. The purpose of RoCode is to provide a benchmark for evaluating the code intelligence of language models trained on Romanian / multilingual text as well as a fine-tuning set for pretrained Romanian models.

arXiv

Spelling, Dictionaries and Gramatical Errors

Synthetic dataset with ~1.9M records. Altered and correct statement as columns

Romanian Archaisms Regionalisms Lexicon containing ~ 1940 Word definitions

Romanian Rules for Dialects - 1940 regionalisms, meanings and the region of provenience