Suggested Reading:

- N-gram Language Models and Perplexity
- The Unreasonable Effectiveness of Recurrent Neural Networks
- Recurrent Neural Networks Tutorial
- Sequence Modeling: Recurrent and Recursive Neural Nets

Assignment:

- Complete assignment 2.
- Implement an RNN in this tutorial
- Implement the code behind this tutorial to gain an intuition of back propagation through time
- Jupyter notebook on vanishing gradient as discussed in lecture. Discuss: why is the sum of magnitude of gradients from the second layer larger? How do the values of gradients compare when RELU and sigmoid is used?
- Share your favourite article on RNN