Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Chinese Training using List Trainer #215

Closed
nicarq opened this issue Jul 21, 2016 · 7 comments
Closed

Chinese Training using List Trainer #215

nicarq opened this issue Jul 21, 2016 · 7 comments

Comments

@nicarq
Copy link

nicarq commented Jul 21, 2016

Hi. I'm having problems trying to use the ChatterBot to talk to me in Chinese.

In the terminal I get:

/usr/local/lib/python2.7/site-packages/chatterbot/conversation/statement.py:46: UnicodeWarning: Unicode equal comparison failed to convert both arguments to Unicode - interpreting them as being unequal if response.text == self.in_response_to[index].text: Type something to begin...

for this first case (talk to english and respond in chinese works ok):

Hello
你好。 你好吗?

but it doesn't work for talking in chinese to the bot.

我很好你呢
我很好你呢

I also get this error:

/usr/local/lib/python2.7/site-packages/chatterbot/adapters/logic/closest_match.py:35: UnicodeWarning: Unicode equal comparison failed to convert both arguments to Unicode - interpreting them as being unequal if input_statement.text in text_of_all_statements:

The code of what I'm using:

# -*- coding: utf-8 -*-
from chatterbot import ChatBot
from chatterbot.training.trainers import ListTrainer

chatterbot = ChatBot("Terminal",
    storage_adapter="chatterbot.adapters.storage.MongoDatabaseAdapter",
    logic_adapters=[
        "chatterbot.adapters.logic.ClosestMatchAdapter"
    ],
    input_adapter="chatterbot.adapters.input.TerminalAdapter",
    output_adapter="chatterbot.adapters.output.TerminalAdapter",
    database="chatterbot-database"
)

chatterbot.set_trainer(ListTrainer)

chatterbot.train([
    "你好。 你好吗?",
    "Hello",
])

chatterbot.train([
    "我很好太感谢",
    "我很好你呢",
])

print("Type something to begin...")

while True:
    try:
        bot_input = chatterbot.get_response(None)

    # Press ctrl-c or ctrl-d on the keyboard to exit
    except (KeyboardInterrupt, EOFError, SystemExit):
        break

Thanks!

@mbarisa
Copy link

mbarisa commented Jul 22, 2016

Hi, I made a pull request you might want to check out, I think that is same issue I encountered with Croatian language, so I just added utf-8 support for corpus. I tried ListTrainer with your dataset, and it seems to work fine.

You want to check this pull request

@ickma
Copy link

ickma commented Jul 28, 2016

I have the same problem with @Nykho ,and I have merged the commit from @mbarisa said,but it still didn't work for me !

@gunthercox
Copy link
Owner

Hi @Nykho, I've been able to confirm this issue on my end. I will attempt to issue a patch as soon as I can.

@gunthercox
Copy link
Owner

gunthercox commented Jul 28, 2016

Just a quick note for anyone else running into this issue. This issue is fixed in Python 3. Although this project aims to fully support multiple versions of Python, if you are using Python 2.7 and you can upgrade that may be the best course of action.

@nicarq
Copy link
Author

nicarq commented Aug 13, 2016

Thanks @gunthercox!

@manhcuogntin4
Copy link

Hi, I met this issue for unicode in python 3.5 in the training phase :
File "/usr/lib/python3.5/json/decoder.py", line 357, in raw_decode
raise JSONDecodeError("Expecting value", s, err.value) from None
json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0)
Hope for help !

@lock
Copy link

lock bot commented Mar 10, 2019

This thread has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs.

@lock lock bot locked and limited conversation to collaborators Mar 10, 2019
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

No branches or pull requests

5 participants