Deep Learning and NLP A-Z: How to create a Chatbot – Additional Resources
Published by SuperDataScience Team
Live Training With Hadelin
Discover the 5 steps to Unlock your Career!
Greetings
Welcome to the data repository for the Deep Learning and NLP: How to build a ChatBot course by Hadelin de Ponteves and Kirill Eremenko. The supplementary materials are below. Enjoy!
Welcome to the Course!
Section 1. Welcome to the course!
- Deep Learning and NLP A-Z (Folder Structure. Updated 20180201)
- Google Colab Cloud Training File for GPU
Deep NLP Intuition
Lecture 13 – Attention Mechanisms (Part 2)
Additional reading:
- Minh-Thang, Luong, 2015, Effective Approached to Attention-based Neural Machine Translation
Building a Chatbot with Deep NLP
Part 1: Data Pre-Processing
Part 2: Building the SEQ2SEQ Model
Lecture 35 – Step 18
The TensorFlow placeholder function:
Lecture 36 – Step 19
The most important tools used:
- https://www.tensorflow.org/api_docs/python/tf/fill
- https://www.tensorflow.org/api_docs/python/tf/strided_slice
- https://www.tensorflow.org/api_docs/python/tf/concat
Lecture 37 – Step 20
The most important tools used:
- https://www.tensorflow.org/api_docs/python/tf/contrib/rnn/BasicLSTMCell
- https://www.tensorflow.org/api_docs/python/tf/contrib/rnn/DropoutWrapper
- https://www.tensorflow.org/api_docs/python/tf/contrib/rnn/MultiRNNCell
- https://www.tensorflow.org/api_docs/python/tf/nn/bidirectional_dynamic_rnn
Lecture 38 – Step 21
The most important tools used:
- https://www.tensorflow.org/programmers_guide/embedding
- https://www.tensorflow.org/api_docs/python/tf/variable_scope
- https://www.tensorflow.org/versions/r1.0/api_docs/python/tf/contrib/seq2seq/prepare_attention
- https://www.tensorflow.org/versions/r1.0/api_docs/python/tf/contrib/seq2seq/attention_decoder_fn_train
- https://www.tensorflow.org/versions/r1.0/api_docs/python/tf/contrib/seq2seq/dynamic_rnn_decoder
- https://www.tensorflow.org/api_docs/python/tf/nn/dropout
Lecture 39 – Step 22
The most important tools used:
- https://www.tensorflow.org/programmers_guide/embedding
- http://web.stanford.edu/class/cs20si/lectures/notes_04.pdf
- https://www.tensorflow.org/api_docs/python/tf/variable_scope
- https://www.tensorflow.org/versions/r1.0/api_docs/python/tf/contrib/seq2seq/prepare_attention
- https://www.tensorflow.org/versions/r1.0/api_docs/python/tf/contrib/seq2seq/attention_decoder_fn_inference
- https://www.tensorflow.org/versions/r1.0/api_docs/python/tf/contrib/seq2seq/dynamic_rnn_decoder
- https://www.tensorflow.org/api_docs/python/tf/nn/dropout
Lecture 40 – Step 23
The most important tools used:
- https://www.tensorflow.org/api_docs/python/tf/variable_scope
- https://www.tensorflow.org/api_docs/python/tf/contrib/rnn/BasicLSTMCell
- https://www.tensorflow.org/api_docs/python/tf/contrib/rnn/DropoutWrapper
- https://www.tensorflow.org/api_docs/python/tf/contrib/rnn/MultiRNNCell
- https://www.tensorflow.org/api_docs/python/tf/truncated_normal_initializer
- https://www.tensorflow.org/api_docs/python/tf/zeros_initializer
- https://www.tensorflow.org/api_docs/python/tf/contrib/layers/fully_connected
Lecture 41 – Step 24
The most important tools used:
- https://www.tensorflow.org/api_docs/python/tf/contrib/layers/embed_sequence
- https://www.tensorflow.org/api_docs/python/tf/Variable
- https://www.tensorflow.org/api_docs/python/tf/random_uniform
- https://www.tensorflow.org/api_docs/python/tf/nn/embedding_lookup
Part 3: Training the SEQ2SEQ Model
Lecture 44 – Step 25
Geoffrey Hinton’s Paper:
Lecture 45 – Step 26
The most important tools used:
- https://www.tensorflow.org/api_docs/python/tf/reset_default_graph
- https://www.tensorflow.org/api_docs/python/tf/InteractiveSession
Lecture 47 – Step 28
The TensorFlow placeholder_with_default function:
Lecture 48 – Step 29
The TensorFlow shape function:
Lecture 49 – Step 30
The TensorFlow reverse function:
- https://www.tensorflow.org/api_docs/python/tf/reverse
- https://docs.scipy.org/doc/numpy-1.13.0/reference/generated/numpy.reshape.html
Lecture 50 – Step 31
The most important tools used:
- https://www.tensorflow.org/api_docs/python/tf/name_scope
- https://www.tensorflow.org/api_docs/python/tf/contrib/seq2seq/sequence_loss
- https://www.tensorflow.org/api_docs/python/tf/ones
- https://www.tensorflow.org/api_docs/python/tf/train/AdamOptimizer
- https://www.tensorflow.org/versions/r0.12/api_docs/python/train/gradient_clipping
- https://www.tensorflow.org/api_docs/python/tf/clip_by_value
Lecture 52 – Step 33
Difference between return and yield:
Lecture 54 – Step 35
The most important tools used:
Part 4: Testing the SEQ2SEQ Model
Lecture 58 – Step 37
The most important tools used:
- https://www.tensorflow.org/api_docs/python/tf/InteractiveSession
- https://www.tensorflow.org/api_docs/python/tf/global_variables_initializer
- https://www.tensorflow.org/api_docs/python/tf/train/Saver
Other ChatBot Implementations
Lecture 63
Intuition and Code resources for the Best ChatBot:
- http://suriyadeepan.github.io/2016-06-28-easy-seq2seq/
- http://suriyadeepan.github.io/2016-12-31-practical-seq2seq/
- https://github.com/suriyadeepan/practical_seq2seq
Lecture 64
Find below a great ChatBot implementation in TensorFlow 1.4.(This ChatBot was built by Luka Anicin, who I am very happy to rank among the students of this course.)
Thanks very much Luka for sharing:
https://github.com/lucko515/chatbot-startkit
Downloadable Materials:
Lecture 65
For PyTorch lovers, please find below a ChatBot Implementation in PyTorch.To run this make sure to have PyTorch installed in anaconda (either the anaconda main environment or any virtual environment you are using). (This ChatBot was built by Alexis Jacq, a PhD student in Robotics and also a good friend of mine.)
Thanks Alexis for sharing:
https://github.com/alexis-jacq/Pytorch-ChatBot
Downloadable Materials: