Skip to content

hhzrd/chinese_newword_discovery

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

3 Commits
 
 
 
 

Repository files navigation

CHINESE NEW WORD DISCOVERY (Open-source-version)

Based on Branch Entropy and Mutual Information

This method uses Mutual Information(MI) to find possible words.

Left and Right Branch Entropy to check word boundaries.

This is a open-source-version, you can also improve the speed of calculation by optimizing the data structure.

Python version and something may you need to install:

python 3.6

tqdm

numpy

About

No description, website, or topics provided.

Resources

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published

Languages