-
-
Notifications
You must be signed in to change notification settings - Fork 4.4k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
oov word prob is zero #536
Comments
Can you do ls `python -c "import spacy; print(spacy.get_data_path())"` And tell me what you see? |
ls Traceback (most recent call last): |
Ah. ls `python -c "import spacy; print(spacy.util.get_data_path())"` |
Here it is:
|
Also gives the following in en-1.1.0/vocab directory:
|
Sorry I was in a hurry and didn't read your issue properly. This is obviously a bug — sec. |
Published v1.0.3 on PyPi. Should be fixed. Thanks again! |
Fantastic! Thanks Matthew. |
This thread has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs. |
Hi,
I just installed 1.0.1 on MacOSx. I find that the model is assigning zero probability to oov words:
import spacy
nlp=spacy.load('en')
x=nlp(u'this is an oovword')
[(t, t.is_oov, t.prob) for t in x]
[(this, False, -5.36181640625), (is, False, -4.457748889923096), (an, False, -6.014852046966553), (oovword, True, 0.0)]
More context: I had the same experience as the issue referenced here #535, and so I did these sequence of steps:
uninstall 0.100.0, install 1.0.1, download data, uninstall 1.0.1, install 1.0.1, download data
The text was updated successfully, but these errors were encountered: