Chapter 02 · Not written yet
Measuring the error
Every mistake, boiled down to one number.
An error is a gap, and a gap can be drawn as a square. Add the squares and you have the single number that the whole of training exists to push down.
- error
- squares
- loss
How this chapter will go
Read a cost, say why the errors are squared, and know that training is the act of pushing that one number down.
The image it is built onTasting the dish. One mouthful, one verdict, and the verdict is a number.
One example, one gap
The machine answers, the truth is known, and the gap between them is drawn as a vertical segment. That segment is the error on that example, and nothing else.
Why adding gaps fails
Add the gaps and a guess that is too high cancels one that is too low, so a bad model can score zero. Drag a point across the line and watch the total lie to you.
Turn each gap into a square
Give every segment a square of its own. A gap twice as large gets a square four times as big, so one bad answer costs more than two mediocre ones.
Drag the line to slide it, or either handle to tilt it. Arrow keys nudge; hold shift for more.
- Cost (total area)
- 17.81
- Slope
- -0.10
- Intercept
- 4.00
- Best possible
- 2.03
The cost is the total shaded area. Moving the line grows some squares and shrinks others, and the eye reads the trade directly. One number for the whole set
Add the areas and divide by how many there are. The result no longer depends on the number of examples, so two models can be compared on it.
What the number will not tell you
It says how much error there is, never where it is. Two very different models can share a cost, which is why the picture stays on screen next to it.
Written with symbols, using a hat on the letter to mean what the machine answered, and no hat for the truth:
Show it in symbols
cost = average of (ŷ − y)²
Nothing here is new. It is the picture above, written in a shorter way.
The misreading to head off
Squaring is not a mathematical trick to remove the minus sign. It is a decision about what counts as bad, and it makes one large mistake worse than several small ones.
There is no date, no list to join and nothing to subscribe to. The chapter simply appears here once the drawing works.