Chapter 01
What a model is
Nobody writes the recipe.
Some things cannot be written down as a rule, and recognising handwriting is one of them. So instead of the rule you build a machine full of adjustable numbers, and a way of checking how well it is doing.
- examples
- parameters
- scoring
The image it is built onA cook who has lost the recipe. Taste, adjust, taste again. Nothing else is going on.
Some things cannot be written down as a rule. This chapter is about what you do instead.
A rule nobody can write
You recognise a handwritten seven instantly, and you cannot say how. Try to write the rule and every version of it has an exception, and the exceptions have exceptions of their own.
A machine reads that same handwriting and gets it right. So the rule is not the only way in, and the interesting question is what the machine has instead.
So build something that can be checked
Rather than the rule, build a machine with numbers you can change, and a way of scoring what comes out of it. Those numbers are the parameters. They are the only thing training ever touches, and the rule is what they settle into.
Nothing about this is particular to handwriting. Parameters and a score work the same way on digits, on letters, on words, on images and on sound. What changes from one to the next is only how the thing is turned into numbers first, which is a chapter of its own.
Setting them by hand
With two parameters you can find good values yourself, by feel, in about a minute. Nothing tells you which way to turn; you try, you look, you try again.
Turn the two dials until your line sits on the dashed one. Then ask for twelve and try the same thing again.
- the shape wanted
- what your parameters make
- parameters
- 2
- how close
- 0%
- best so far
- 0%
- adjustments
- 0
Turn the two dials until the solid line sits on the dashed one. There is no rule to follow; go by feel.
Where the hand stops
A real model has thousands of parameters, and nobody sets those by feel. Everything that follows is one method for setting them without looking.
A model is a machine full of adjustable numbers, plus a way of scoring what it produces. Training is the search for values of those numbers that score well.
Before anything can be adjusted, the score has to exist. That is the next chapter.
The misreading to head off
A model does not store the examples it was shown. When training ends the examples are thrown away, and only the parameters are kept.