Artificial Intelligence Masterclass – Additional Resources

Greetings

Welcome to the data repository for the Artificial Intelligence Masterclass course by Kirill Eremenko and Hadelin de Ponteves. The datasets and other supplementary materials are below. Enjoy!

Section 1. Introduction

The Course Structure

The Full World Model

Additional Reading:

• The Full World Model Article
• The Best Resource on Evolution Strategies

Section 2. Artificial Neural Networks (ANN)

Additional Reading:

• Yann LeCun et al., 1998, Efficient BackProp
• By Xavier Glorot et al., 2011, Deep sparse rectifier neural networks
• CrossValidated, 2015, A list of cost functions used in neural networks, alongside applications
• Andrew Trask, 2015, A Neural Network in 13 lines of Python (Part 2 – Gradient Descent)
• Michael Nielsen, 2015, Neural Networks and Deep Learning

Section 3. Convolutional Neural Networks (CNN)

Additional Reading:

• Yann LeCun et al., 1998, Gradient-Based Learning Applied to Document Recognition
• Jianxin Wu, 2017, Introduction to Convolutional Neural Networks
• C.-C. Jay Kuo, 2016, Understanding Convolutional Neural Networks with A Mathematical Model
• Kaiming He et al., 2015, Delving Deep into Rectifiers: Surpassing Human-Level Performance on ImageNet Classification
• Dominik Scherer et al., 2010, Evaluation of Pooling Operations in Convolutional Architectures for Object Recognition
• Adit Deshpande, 2016, The 9 Deep Learning Papers You Need To Know About (Understanding CNNs Part 3)
• Rob DiPietro, 2016, A Friendly Introduction to Cross-Entropy Loss
• Peter Roelants, 2016, How to implement a neural network Intermezzo 2

Section 4. AutoEncoder (AE)

Additional Reading:

• Malte Skarupke, 2016, Neural Networks Are Impressively Good At Compression
• Francois Chollet, 2016, Building Autoencoders in Keras
• Chris McCormick, 2014, Deep Learning Tutorial – Sparse Autoencoder
• Eric Wilkinson, 2014, Deep Learning: Sparse Autoencoders
• Alireza Makhzani, 2014, k-Sparse Autoencoders
• Pascal Vincent, 2008, Extracting and Composing Robust Features with Denoising Autoencoders
• Salah Rifai, 2011, Contractive Auto-Encoders: Explicit Invariance During Feature Extraction
• Pascal Vincent, 2010, Stacked Denoising Autoencoders: Learning Useful Representations in a Deep Network with a Local Denoising Criterion
• Geoffrey Hinton, 2006, Reducing the Dimensionality of Data with Neural Networks

Section 5. Variational AutoEncoder (VAE)

Additional Reading:

• Irhum Shafkat, 2018, Intuitively Understanding Variational Autoencoders
• Diederik P. Kingma and Max Welling, 2014, Auto-Encoding Variational Bayes
• Francois Chollet, 2016, Building Autoencoders in Keras
• Chris McCormick, 2014, Deep Learning Tutorial – Sparse Autoencoder
• Eric Wilkinson, 2014, Deep Learning: Sparse Autoencoders
• Alireza Makhzani et al., 2014, k-Sparse Autoencoders
• Pascal Vincent et al., 2008, Extracting and Composing Robust Features with Denoising Autoencoders
• Salah Rifai et al., 2011, Contractive Auto-Encoders: Explicit Invariance During Feature Extraction
• Pascal Vincent et al., 2010, Stacked Denoising Autoencoders: Learning Useful Representations in a Deep Network with a Local Denoising Criterion
• Geoffrey Hinton et al., 2006, Reducing the Dimensionality of Data with Neural Networks

Section 6. Implementing the CNN-VAE

TBA

Section 7. Recurrent Neural Networks (RNN)

Additional Reading:

• Oscar Sharp & Benjamin, 2016, Sunspring
• Sepp (Josef) Hochreiter, 1991, Untersuchungen zu dynamischen neuronalen Netzen
• Yoshua Bengio, 1994, Learning Long-Term Dependencies with Gradient Descent is Difficult
• Razvan Pascanu, 2013, On the difficulty of training recurrent neural networks
• Sepp Hochreiter & Jurgen Schmidhuber, 1997, Long Short-Term Memory
• Christopher Olah, 2015, Understanding LSTM Networks
• Shi Yan, 2016, Understanding LSTM and its diagrams
• Andrej Karpathy, 2015, The Unreasonable Effectiveness of Recurrent Neural Networks
• Andrej Karpathy, 2015, Visualizing and Understanding Recurrent Networks
• Klaus Greff, 2015, LSTM: A Search Space Odyssey
• Xavier Glorot, 2011, Deep sparse rectifier neural networks

Section 7. Mixture Density Network (MDN)

TBA

Section 8. Implementing the MDN-RNN

TBA

Section 9. Reinforcement Learning

Additional Reading:

• Arthur Juliani, 2016, Simple¬†Reinforcement Learning with Tensorflow (10 Parts)
• Richard Sutton et al., 1998, Reinforcement Learning I: Introduction
• Richard Bellman, 1954, The Theory of Dynamic Programming
• D. J. White, 1993, A Survey of Applications of Markov Decision Processes
• Martijn van Otterlo, 2009, Markov Decision Processes: Concepts and Algorithms
• Richard Sutton, 1988, Learning to Predict by the Methods of Temporal Differences

Section 10. Deep Neuro Evolution (GA, ES)

TBA