Interactive: Bias-Variance Tradeoff — the fundamental ML curve
DRAG the model complexity slider. SEE bias decrease, variance increase, and total error form a U-shape. The sweet spot is the minimum.
If it doesn't appear in 2 seconds, press Ctrl+Shift+R to hard refresh.
THE FUNDAMENTAL TRADEOFF OF ML: Every model has TWO sources of error: BIAS: the model is too SIMPLE to capture the pattern. Linear model on curved data = high bias. Symptom: train error HIGH, test error HIGH. VARIANCE: the model is too COMPLEX, sensitive to noise. Degree-15 polynomial on 20 points = high variance. Symptom: train error LOW, test error HIGH (the gap). TOTAL ERROR = bias² + variance + irreducible noise. Forms a U-curve. The minimum = best model. LEFT of min: underfitting (increase complexity). RIGHT of min: overfitting (decrease complexity or regularize). This curve is why we can't just use the most complex model — variance kills you. And we can't use the simplest — bias kills you. The art of ML is finding the sweet spot.
-
Set complexity=1 — extreme underfitting→ bias is high, variance is low, total error is high -
Set complexity=5 — the sweet spot→ bias + variance are balanced, total error is minimum -
Set complexity=20 — extreme overfitting→ bias is low but variance is huge, total error is high