Neural networks
Every model in this tour is built from the same simple parts. Watch a small network pass a signal forward, see the functions that shape it, and watch it learn from its mistakes.
A neural network is a stack of simple steps. Each step multiplies numbers by weights, adds them up and passes the result on. Training is how a network finds good weights. This module shows all three parts on a network small enough to see whole.
The forward pass¶
Set the two inputs, then run the forward pass and watch the signal move from left to right. Green lines are positive weights and orange lines are negative ones. A thicker line has more influence.
Each hidden neuron adds up what reaches it, adds a bias, and turns any negative total into zero. The output neuron squashes its total into a number between 0 and 1. Change one input and run it again: every connection plays a part in the answer.
Activation functions¶
The step that turns a neuron's total into its output is called the activation function. Switch between ReLU, sigmoid and tanh, and move z to see what each one does to a total.
Watch the orange line. It is the slope where you are, and it is what training multiplies by when it works out how to change a weight. Push sigmoid out towards either edge and the slope flattens to almost zero. A flat slope means almost no learning, a problem called the vanishing gradient.
Learning from mistakes¶
Training means showing the network an example, measuring how wrong its answer is, and nudging every weight to make it a little less wrong. The nudges are worked out backwards, from the output towards the inputs, which is why this is called backpropagation.
Train one step and follow the four stages: the forward pass, the loss, the error flowing backwards, and the weight update. Then fast-forward and watch the loss fall as the network gets closer to its target.
What to take away¶
A model is a very large stack of these parts. It learns by being wrong, measuring how wrong, and adjusting. Nothing in that process checks whether an answer is true. It only makes the model better at matching the examples it was trained on.
Where next
How AI works: Module 2: Language models
A shared question channel is on the way. When it opens, each answer will be written once and shared with everyone taking the course.