Unsupervised Word Segmentation for Neural Machine Translation and Text Generation
-
Updated
Aug 7, 2024 - Python
#
subword-units
Here are 13 public repositories matching this topic...
Morfessor is a tool for unsupervised and semi-supervised morphological segmentation
-
Updated
Oct 6, 2020 - Python
Properly handle position-dependent phones in a subword lexicon FST
-
Updated
Oct 26, 2020 - Python
Morfessor FlatCat
-
Updated
Aug 20, 2019 - Python
Morfessor EM+Prune
python natural-language-processing segmentation neural-machine-translation word-segmentation subword-units subword-segmentation
-
Updated
Jul 22, 2020 - Python
Finalfusion embeddings in Python
-
Updated
Nov 30, 2021 - Python
A python package to build a corpus vocabulary using the byte pair methodology and also a tokenizer to tokenize input texts based on the built vocab.
nlp natural-language-processing tokenizer vocabulary nlp-library vocabulary-builder natural-language-understanding subword-units bpe bytepairencoding subwordtokenization subwordtokens
-
Updated
May 21, 2020 - Python
Cognate-aware morphological segmentation
-
Updated
Aug 20, 2019 - Python
A tool for generating sub-word (phone or grapheme) level embeddings from an HTK-style MLF ASR corpus
-
Updated
Oct 11, 2019 - Python
Morfessor EM+Prune
-
Updated
Apr 20, 2020 - Python
Morfessor demonstration
-
Updated
Mar 13, 2015 - Python
Subword Neural Machine Translation
-
Updated
Feb 6, 2019 - Python
🕵️ Language Model based on RNN for generating Sherlock Holmes stories.
natural-language-processing recurrent-neural-networks language-model sherlock-holmes textgeneration subword-units
-
Updated
Jun 21, 2021 - Python
Improve this page
Add a description, image, and links to the subword-units topic page so that developers can more easily learn about it.
Add this topic to your repo
To associate your repository with the subword-units topic, visit your repo's landing page and select "manage topics."