Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

trained model takes longer to get a response #1056

Closed
Karamya opened this issue Nov 6, 2017 · 4 comments
Closed

trained model takes longer to get a response #1056

Karamya opened this issue Nov 6, 2017 · 4 comments

Comments

@Karamya
Copy link

Karamya commented Nov 6, 2017

I trained my corpus with storage_adapter= SQLStorageAdapter and database= database.sqlite3. The file size of the sqlite3 database is about 45 MB, which I got after about one week of training using a corpus size of ~78 MB. I am trying to get a response from the trained model and it is almost one and half hours and still I didn't get any response. Here is the code for training my data based on a corpus in the folder /data

from chatterbot.trainers import ChatterBotCorpusTrainer, Trainer
from chatterbot import ChatBot

bot = ChatBot(
    "Remedy",
    storage_adapter='chatterbot.storage.SQLStorageAdapter',
    #input_adapter = 'chatterbot.input.TerminalAdapter',
    #output_adapter = 'chatterbot.output.TerminalAdapter',
    logic_adapters=[
        {
            "import_path": "chatterbot.logic.BestMatch",
            "statement_comparison_function": "chatterbot.comparisons.levenshtein_distance",
            "response_selection_method": "chatterbot.response_selection.get_first_response"
        },
{
            'import_path': 'chatterbot.logic.LowConfidenceAdapter',
            'threshold': 0.65,
            'default_response': 'I am sorry, but I do not understand.'
        }

    ],
    database = './database.sqlite3'
)

bot.set_trainer(ChatterBotCorpusTrainer)
bot.train('chatterbot.corpus.german', './data/') 

After training my corpus, I used the following code to get a response,

from chatterbot import ChatBot
from chatterbot.trainers import ChatterBotCorpusTrainer, Trainer
bot = ChatBot("Remedy",
    storage_adapter='chatterbot.storage.SQLStorageAdapter',
    #input_adapter = 'chatterbot.input.TerminalAdapter',
    #output_adapter = 'chatterbot.output.TerminalAdapter',
    logic_adapters=[
        {
            "import_path": "chatterbot.logic.BestMatch",
            "statement_comparison_function": "chatterbot.comparisons.levenshtein_distance",
            "response_selection_method": "chatterbot.response_selection.get_first_response"
        },
{
            'import_path': 'chatterbot.logic.LowConfidenceAdapter',
            'threshold': 0.65,
            'default_response': 'I am sorry, but I do not understand.'
        }

    ],
    database = './database.sqlite3', read_only = True)
print(bot.get_response('Ich arbeite an einem Projekt'))

Can someone figure out the problem, why it takes really long to get a response (already waited close to 2 hours).

@vkosuri
Copy link
Collaborator

vkosuri commented Nov 7, 2017

Known issue is going address by this feature #925

@ghost
Copy link

ghost commented Nov 21, 2017

Does changing the database away from sqlite3 help the efficiency here? I trained the bot on 10,000 movie scripts and it's been hours since a response with a simple "hello"

@gunthercox
Copy link
Owner

@kyle-sorensen The database type won't effect the efficiency on any noticeable level for large datasets.

@lock
Copy link

lock bot commented Mar 10, 2019

This thread has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs.

@lock lock bot locked and limited conversation to collaborators Mar 10, 2019
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants