Keras/TensorFlow implementation

[monokal]

IMO, ChatterBot would really benefit from a rewrite to integrate Keras/TensorFlow as the Machine/Deep Learning backend. It's not a difficult implementation now that Keras has been re-written by Google, complete with high-level Python APIs.

It provides powerful, well-maintained Machine Learning models/algorithms which have numerous NLP/chatbot examples on the web, will handle training storage, provides a web-UI to dig in to the bot's Neural Network, training progress, etc), and far more.

I'd be very happy to contribute should the proposal be accepted as I was thinking of migrating my project away from ChatterBot to Keras anyways, but I'd much rather make a good thing great here.

Some useful links:

• https://blog.keras.io/keras-as-a-simplified-interface-to-tensorflow-tutorial.html
• https://www.tensorflow.org/
• https://www.youtube.com/watch?v=SJDEOWLHYVo
• https://www.youtube.com/watch?v=5DknTFbcGVM
• https://www.youtube.com/playlist?list=PLQVvvaa0QuDeETZEOy4VdocT7TOjfSA8a

[peterel]

@monokal 2 questions for you:

1. Do you know what Chatterbot is actually using in the backend as Machine/Deep Learning?
2. Im not so familiar with TF or Keras. Do you have any good tutorial for creating a chatbot that you have tried out and liked?

Peter

[monokal]

1. It uses a bunch of simple logic adapter functions as opposed to a framework like TensorFlow, which can be seen in the repo here: https://github.com/gunthercox/ChatterBot/tree/master/chatterbot/logic

2. There's some examples on the first page of Google and YouTube. Just search "tensorflow chatbot".

[peterel]

1. Ok.
2. Yes, there are indeed a lot of examples. My question is, have you tried any of them and found them "good"?

[vkosuri]

It's all about you how you are going to apply your application and usecase.

Tensorflow is all about google products for example Gmail replay suggestions, YouTube recommended videos, Google Photos i.e it is good for Classification of problems and predictions.

Keras is built on top of Tensorflow, Theano deep learning regression models API into more useful manner. The (Tensorflow, Theano, Keras) main motive is classification your data and make prediction for you.

What Does Chatterbot is:

Chatterbot is a bot, It embedded knowledge to answer your questions using Natural Language Processing Library, for that you have to train with some predefined data with question and answer, later it starts learning from responses and question update your database to get more accurate predictions.

If you are looking an chat application using NLP(Natural Language Processing), i believe chatterbot is good for you.

What is Machine Learning

A computer program is said to learn from experience 'E' with respect to some class of tasks 'T"and performance measure 'P , if its performance at tasks in 'T' as measured by 'P', improves with experience 'E' that is called Machine Learning

[peterel]

@vkosuri Thanks! Very useful reading!

[bilalbayasut]

I'm agree with @vkosuri , but @monokal idea is cool.

IMHO we need to see the whole picture first, how to utilize deep learning models in chatbot implementation, and me my self to be honest still got confused.

for instance, if you mean chatbot by just throwing away conversation logs to the ML model and chat with it , then what is the goal ?

you have to consider this case : let say user send multiple messages within short time, then how to receive and recognize those utterances , how to track the state of the conversation to the goal state, etc.

if you know how to solve that, it will be cool to use deep learning I think.

me myself, I've used this framework chatterbot, and modified it, embedded it with Machine Learning model to classify intent, however I'm still figuring out how to implement the deep learning libraries so that the bot can be used in real life scenario :)

[vikramsoni2]

@monokal Have you had any success with writing Keras logic adapter, are you using the same storage to get the data/train model/save the trained model/weights ?

[johndpope]

chiming in here - I plugged in chatterbot to duel against @oswaldoludwig keras conversational agent
Adversarial-Learning-for-Generative-Conversational-Agents

Architecture is using using GAN. My implementation is a hack and results at the moment - round 1 - kind of suck and things get stuck in a loop. Was thinking of retraining the model against chatterbot.
oswaldoludwig/Adversarial-Learning-for-Generative-Conversational-Agents#4

here's code (apparently not the best gan discriminator example - https://github.com/oswaldoludwig/Seq2seq-Chatbot-for-Keras/blob/master/conversation_discriminator.py)
https://gist.github.com/johndpope/b0d9a025c6e54dc1e07ab6100c34a24a

Microsoft released this paper a year back on hybrid code networks
HCNs considerably reduce the amount of training data required, while retaining the key benefit of inferring a latent representation of dialog state.
https://arxiv.org/abs/1702.03274
this keeps a track / gist of conversation and pursue outcome within defined action templates

    Action Templates
    1. 'any preference on a type of cuisine',
    2. 'api_call <party_size> <rest_type>',
    3. 'great let me do the reservation',
    4. 'hello what can i help you with today',
    5. 'here it is <info_address>',
    6. 'here it is <info_phone>',
    7. 'how many people would be in your party',
    8. "i'm on it",
    9. 'is there anything i can help you with',
    10. 'ok let me look into some options for you',
    11. 'sure is there anything else to update',
    12. 'sure let me find an other option for you',
    13. 'what do you think of this option: ',
    14. 'where should it be',
    15. 'which price range are looking for',
    16. "you're welcome",

https://github.com/johndpope/hcn

this supposedly outperformed commercial applications 12 mths ago.
Thinking out loud - I guess if you could social crowd curate these action templates / have a standard in format / corresponding training data - then this would move things a step forward.

[vkosuri]

Awesome i loved it