Spacy Scikit-learn Multithreaded Training Prediction Issue

[azzuwan]

I'm using the multithreaded-training branch and when I run:
python -m rasa_nlu.train -c config.json -t 3
I get this:
/usr/local/lib/python2.7/dist-packages/sklearn/metrics/classification.py:1113: UndefinedMetricWarning: F-score is ill-defined and being set to 0.0 in labels with no predicted samples. 'precision', 'predicted', average, warn_for)

[amn41]

TL;DR this isn't an error but I would recommend you add more training data.

The model still trains and runs correctly I presume? The warning you see is actually related to how the training data is used & can also occur with 1 thread.

Let's say you have 2 intents. Part of the f-score is the number of examples correctly predicted to be each intent. If you have very few training examples for, say, intent_1, then when your data is split into training and test sets it can happen that your test set doesn't have any examples of intent_1. That means the f-score isn't defined & scikit-learn has to fall back on a default.

[azzuwan]

That explains it thoroughly. Thank you Alan.

[vduckute1812]

Thank you very much amn41 👍

[ugarciac]

hello I have the same problem and I have not been able to solve it
I have
training 1277 objects
test 15000 objects
If I use accuracy or f1_micro, it works for me, how can it be solved, the same thing happens to me if I use a perceptron, HELP!