How models learn
← The series

Chapter 03 · Not written yet

The slope

Which way is down.

Nudge one knob, watch how much the error moves, and divide. Shrink the nudge until the answer settles: that number is the slope, and its sign is a decision.

  • nudge
  • derivative
  • tangent

How this chapter will go

Measure a slope at a point, read its sign, and decide which way to turn a knob from it alone.

The image it is built onFog on a hillside. You cannot see the bottom, you can feel the ground tilt under your feet, and that is enough to take one step.

  1. One knob, one curve

    Fix everything but a single knob and plot the cost against it. The whole chapter happens on this one curve.

  2. A nudge, and an exchange rate

    Move the knob by 0.1 and the cost falls from 4.00 to 3.60. So 0.1 of knob bought 0.40 of cost, which is a rate of 4. That rate is the thing this chapter is about.

  3. The rate depends on the nudge

    Use a nudge of 2 instead and the rate comes out different, because the curve bends between the two points. The problem appears before the fix does.

  4. Shrink until it settles

    Try a nudge of 1, then 0.1, then 0.01. The rate moves less and less, until it stops moving in the digits you are watching. That settled number is the slope at that point, and it is what you get by zooming in until the bend disappears.

    The plot takes focus. Left and right arrow keys move the measurement point along the curve, Home and End take it to either end. The sliders and the readout below carry the same values.

    costwhΔ costsecanttangent
    w
    1.800
    h
    2.00000
    cost(w)
    1.868608
    cost(w + h)
    1.306688
    (Δ cost) / h
    -0.280960
    Drag the nudge towards zero. The line stops swinging and lies along the curve, and the rate stops changing.

  5. The sign decides, the size warns

    Negative slope, go right. Positive slope, go left. A large slope means a small turn changes a lot; a small one means you are on a plateau and this will take a while.

    it falls, so go rightit rises, so go left
    The same curve twice. The tangent and the arrow beneath it share a colour, so the decision is read rather than computed.

That settled rate has a name: the derivative. It arrives last, for something you have been handling all chapter. Written out, with h standing for the nudge you have been shrinking:

Show it in symbols

rate = ( cost(w + h) − cost(w) ) ÷ h

Once h is small enough for the answer to settle, that answer gets written dL/dw. Read it as one thing — how much the cost moves when the knob moves — and not as one number divided by another.

The misreading to head off

The value of the cost and the slope of the cost are different things. A cost can be enormous and perfectly flat, which is exactly what being stuck feels like.

There is no date, no list to join and nothing to subscribe to. The chapter simply appears here once the drawing works.