Welcome to Part 2 of the Deep Learning Illustrated series. In it Previous article (definitely read that first!), we cover how a neural network works and how trained The neural network makes predictions.
In this article we will delve into the training process and explore how a neural network learns.
If you haven't read my previous articles, I recommend that you start with my series of articles covering machine learning basicsspecifically that of gradient descent because you will find that much of the material covered there is relevant here.
Machine Learning Starter Pack
Let's say we want to create a neural network that predicts daily revenue from ice cream sales using temperature and day of the week characteristics.
This is the training (synthetic) data set we are using:
To build a neural network, as we learned in the previous article, we must first decide on its architecture. This includes determining the number of hidden layers, the number of neurons in each layer, and the activation function of each neuron.
Let's say we decide our architecture is: 1 hidden layer with 2 neurons and 1 output neuron, all using the rectifier activation function.
Segue terminology: In the previous article, we learned about using subscripts to differentiate between different weights. We stick to the same convention here and, in addition, we will use superscripts to…
