sequence of step sizes $αk(a)\alpha_k(a) αk(a) will allow an iterative estimate to converge to the true value. The intuition is exactly what the text says:
- ∑k=1∞αk(a)=∞\sum_{k=1}^{\infty} \alpha_k(a) = \infty ∑k=1∞αk(a)=∞: Step sizes must be large enough in total to overcome any initial bias or bad starting estimates
- ∑k=1∞αk2(a)<∞\sum_{k=1}^{\infty} \alpha_k^2(a) < \infty ∑k=1∞αk2(a)<∞: Step sizes must shrink fast enough that the noise eventually dies out and the estimates stabilize