Gradient bandits

$$\mathrm{Pr}(A_t=a)=\frac{e^{H_t(a)}}{{\sum }_{b=1}^n{e}^{H_t(a)}}={\pi }_t(a)$$

Learning these probability distributions is done with algorithm based on Stochastic Gradient Descent.

$$\begin{array}{rlrl}{H}_{t+1}(A_t)& =H_t(A_t)+\alpha (R_t-{\overline{R}}_t)(1-{\pi }_t(A_t))& & \text{and}\\ H_{t+1}(a)& =H_t(a)-\alpha (R_t-{\overline{R}}_t)({\pi }_t(a))& & \forall a\ne A_t\end{array}$$