How AI Learns: Forward Pass and Loss Explained with 2 + 1
Our inputs are 2 and 1, and the correct answer is 3. But the network initially predicts 0.7. Why? A neural network doesn’t automatically know the rules of addition. Its prediction depends on its current weights and biases. The forward pass A forward pass means sending inputs through the network to produce a prediction. Each neuron multiplies its inputs by weights, adds a bias, and applies an…
In a neural network, the starting point is to feed in inputs of 2 and 1, with the anticipated result being 3. However, initially, the network's prediction comes out to be 0.7. This discrepancy arises due to the network's initial weights and biases. The process begins with a forward pass. During this phase, information passes through the network, with each neuron performing a calculation that involves multiplying inputs by weights, adding a bias, and then applying an activation function.
For instance, a hidden neuron might compute: (2 x 0.5) + (1 x -1) + 0.5 = 0.5. With ReLU as the activation function, negative results turn into zero, while positive values remain unchanged. In this scenario, two hidden neurons each yield 0.5. The final output neuron then amalgamates these results: Prediction = (0.5 x 0.8) + (0.5 x 0.4) + 0.1 = 0.7.
This forward pass produces a prediction, but crucially, it doesn't alter the network's weights. However, with the prediction in hand, we can move to the next step: calculating the loss. The loss is a numerical measure indicating how far off our prediction is from the correct answer. In this case, the difference between the prediction (0.7) and the correct answer (3) is -2.3.
By squaring this error, we arrive at a loss value of 5.29. A lower loss figure signifies that the prediction is nearer to the correct answer. Understanding loss alone is insufficient to teach the network. That's where backpropagation comes into play. It determines the gradients, which are essentially the direction and magnitude of change needed in each weight and bias to reduce the loss.
An optimizer then utilizes these gradients to adjust the weights and biases. This cycle of making a prediction, computing the loss, calculating gradients, and updating the weights is repeated over numerous examples. This iterative process is the backbone of training a neural network. The goal is to refine the network's weights and biases so that when presented with new inputs it hasn't seen before, it can make accurate predictions.
In essence, the forward pass tells the network what it predicts, the loss calculation tells us how far off that prediction is, and the training phase indicates how we should adjust the network's parameters.
Written by urgent.news from Dev.to's reporting — not their text. Machine-written — may contain errors; check the original before relying on it.
