Skip to content

Latest commit

 

History

History

get_panlex_data

Folders and files

NameName
Last commit message
Last commit date

parent directory

..
README.md
README.md
 
 
get_panlex_embs.py
get_panlex_embs.py
 
 

README.md

Get PanLex-BLI Data

  • Download bilingual dictionaries from the PanLex-BLI repo: 
    cd /media/data/
git clone https://github.com/cambridgeltl/panlex-bli.git
cd panlex-bli/lexicons/
unzip all-l1-l2.zip
  • Install fastText: 
    git clone https://github.com/facebookresearch/fastText.git
cd fastText
pip install .
  • Get monolingual embeddings: 
    python get_panlex_embs.py

Attention

In PanLex-BLI, source->target and target->source translation tasks have different training and test dictionaries. So, it is needed to train two BLI models respectively for source->target and target->source translations.