How does a network know it's wrong?
Loss functions turn every mistake into one number. Here's how squared and absolute loss score the same misses very differently. A neural network can't feel embarrassed when it's wrong. It needs a way to measure its mistakes, so it knows which direction to improve. That measurement is the loss function. It turns all of a network's errors into a single number, and training is the process of pushing…
A network learns by striving to minimize a single number, which is the result of a loss function. This number is the sum of all the network's errors compared to the correct answers. Squared loss and absolute loss are two common types of loss functions. Squared loss squares each error before adding them up, which makes it highly sensitive to large errors.
Absolute loss, on the other hand, simply adds up the size of each error without squaring it. This makes absolute loss less sensitive to outliers, as a single large error will not have as significant an impact on the total loss. During training, the network adjusts its weights, taking small steps to reduce the loss number. The type of loss function chosen influences what the network cares about most - whether it's big mistakes or all errors equally.
Written by urgent.news from Dev.to's reporting — not their text. Machine-written — may contain errors; check the original before relying on it.