Wiki: Improving NN: Assignment 3


Stochastic vs Batch gradient descent

Follow this tutorial on implementing stochastic gradient descent. Try extending the model by using batch or minibatch gradient descent. How do your results differ?

Implementing Minibatch Gradient Descent for Neural Networks

A module to implement the stochastic gradient descent learning algorithm for a feedforward neural network. github and tutorial

Reply to this thread:

  • How does stochastic compare to minibatch gradient descent? Does one provide better convergence over the other?