How models learn

An interactive explainer

How models learn

Machine learning tends to be explained twice over: once in metaphors that leave nothing behind, once in equations that assume you already agree. This is the third way.

  1. A cook who has lost the recipe still gets there: taste, adjust, taste again. Learning is that loop, written down. Opens chapter 1, Learning without a rule.

  3. Before anything can be fixed, one number has to say how wrong the answer was. Opens chapter 2, Measuring the error.

  5. A slope is an exchange rate. Turn the knob a little, and see how much error you get back. Opens chapter 3, The slope.

  7. Step against the slope. Too short and you never arrive, too long and you fly over the valley. Opens chapter 4, Gradient descent.

  9. One slope per knob, stacked in a column. That column is the whole of the gradient. Opens chapter 5, The gradient.

  11. Nobody reads all the data before each step. A handful of examples already knows roughly where down is. Opens chapter 6, Fast, or straight.

  13. A network is a chain of small machines. Each one receives, transforms, and passes it on. Opens chapter 7, A machine in layers.

  15. Along a train of gears the ratios multiply. Sensitivities travel the same way. Opens chapter 8, Gears in a chain.

  17. Thirteen thousand knobs, and one pass backwards tells you what each of them owes the error. Opens chapter 9, Backpropagation.

  19. Predict, measure, hand out the blame, adjust. Then again, a few thousand times. Opens chapter 10, The loop.

  21. A model that answers perfectly on what it has already seen may have understood nothing at all. Opens chapter 11, Learning by heart.

  23. None of it runs on words. Meaning has to become coordinates first. Opens chapter 12, Embeddings.

  25. Reading everything at once is not the same as knowing where to look. Opens chapter 13, Attention.

  27. Something has to read all of it, squeeze it down, and write something back. Opens chapter 14, Encoder and decoder.