Dialogflow import bug "No training examples found for the dialogflow file data\intents\<every intent>.json!"

[A-A-R0N]

Rasa version: Rasa 1.10.3

Rasa SDK version (if used & relevant):

Rasa X version (if used & relevant):

Python version: python 3.6.8

Operating system (windows, osx, ...): windows 10

Issue:

For every imported intent, a warning message is given saying it cannot find the example data even though the files are there. I have tried this will several exported chatbots including very simple sample ones.

c:\users\aaron\appdata\local\programs\python\python36\lib\site-packages\rasa\utils\common.py:363: UserWarning: No training examples found for dialogflow file data\intents\every_intent.json!

Cause:
This issue is caused when the language variable set to None in the /nlu/training_data/formats/dialogflow.py file. When generating the intent example filename in the _read_example_js function, it generates it as "intent_name_usersays_None.json" instead of "intent_name_usersays_en.json". I've already addressed the issue in my local code with a simple check against the language variable that corrects the value if it is None. I'm looking for the best place to update the code and will submit a pull request. I see calling request actually set a default value, but somewhere down the line a None value is passed for language.

Error (including full traceback):

c:\users\aaron\appdata\local\programs\python\python36\lib\site-packages\rasa\utils\common.py:363: UserWarning: No training examples found for dialogflow file data\intents\intent_name.json!
  More info at https://rasa.com/docs/rasa/migrate-from/google-dialogflow-to-rasa/

Command or request that led to error:

rasa train nlu

Content of configuration file (config.yml) (if relevant):
It doesn't seem to be relevant, it is the default file.

# Configuration for Rasa NLU.
# https://rasa.com/docs/rasa/nlu/components/
language: en
pipeline:
  - name: WhitespaceTokenizer
  - name: RegexFeaturizer
  - name: LexicalSyntacticFeaturizer
  - name: CountVectorsFeaturizer
  - name: CountVectorsFeaturizer
    analyzer: "char_wb"
    min_ngram: 1
    max_ngram: 4
  - name: DIETClassifier
    epochs: 100
  - name: EntitySynonymMapper
  - name: ResponseSelector
    epochs: 100

# Configuration for Rasa Core.
# https://rasa.com/docs/rasa/core/policies/
policies:
  - name: MemoizationPolicy
  - name: TEDPolicy
    max_history: 5
    epochs: 100
  - name: MappingPolicy

Content of domain file (domain.yml) (if relevant):
This is the default file.

intents:
  - greet
  - goodbye
  - affirm
  - deny
  - mood_great
  - mood_unhappy
  - bot_challenge

responses:
  utter_greet:
  - text: "Hey! How are you?"

  utter_cheer_up:
  - text: "Here is something to cheer you up:"
    image: "https://i.imgur.com/nGF1K8f.jpg"

  utter_did_that_help:
  - text: "Did that help you?"

  utter_happy:
  - text: "Great, carry on!"

  utter_goodbye:
  - text: "Bye"

  utter_iamabot:
  - text: "I am a bot, powered by Rasa."

session_config:
  session_expiration_time: 60
  carry_over_slots_to_new_session: true

[sara-tagger]

Thanks for the issue, @alwx will get back to you about it soon!

You may find help in the docs and the forum, too 🤗

[indam23]

@A-A-R0N When would language not be set in agent.json? I'm looking at the example data in data/examples/dialogflow since I don't have an actual dialogflow bot. Is this not getting exported from dialogflow?

[A-A-R0N]

@melindaloubser1 The language is set in the agent.json file for every chatbot I have tried importing. It is just somehow being lost in the processing. I've attached an example chatbot using one one of the dialogflow samples. You can see that the agent.json file does populate the language, but if you go through the steps to import the chatbot, it will produce the errors I'm talking about because at some point the the language value gets overwritten.

The steps I followed are here:
https://rasa.com/docs/rasa/migrate-from/google-dialogflow-to-rasa/

Coffee-Shop.zip

[A-A-R0N]

A bit more debugging info on this.....

I'm not exactly sure why, but the get_nlu_data function is called twice. Since default values for python functions are set when they are defined, they will only be used the first time they are called. You can see below the full stack for the first call where the default value is being used...

But later in the process it is called again with no language value passed. Since default arguments are mutable, this time the default is not used. Without a check within the function itself, a subsequent call like this will result a None value for language being passed through the series of functions and eventually result in the inability to find example training data.

I hope that helps clarify the issue.

[indam23]

I see what you mean; it's called again during the train function and then the language is not set. Thanks for the clarification!

[indam23]

It looks like the issue is on this line:

rasa/rasa/nlu/train.py

Line 82 in b932c9f

training_data = await data.get_nlu_data(nlu_config.data)

The expected argument is language, but nlu_config.data will be None when it gets there in the importer case. So language is actually explicitly getting set to None. I'm not sure why nlu_config.data is passed at all; I tried it with just

        training_data = await data.get_nlu_data()

and that works fine. @wochinge do you know the reason for passing nlu_config.data as language here?

[wochinge]

@melindaloubser1 should be nlu_config.language 👍🏼 Recently discovered that as well and fixed it (but that branch is not yet merged back to master)

[indam23]

Ok cool! @A-A-R0N if you wanted to change your PR to make that fix, it could be merged into 1.10.x?

[A-A-R0N]

Thanks @melindaloubser1! I've updated the PR as requested.