Wiki: Lecture 8: Recurrent Neural Networks and Language Models


#1

Lecture video

Lecture Notes

Suggested Reading:

Assignment:

  • Complete assignment 2.
  • Implement an RNN in this tutorial
  • Implement the code behind this tutorial to gain an intuition of back propagation through time
  • Jupyter notebook on vanishing gradient as discussed in lecture. Discuss: why is the sum of magnitude of gradients from the second layer larger? How do the values of gradients compare when RELU and sigmoid is used?
  • Share your favourite article on RNN