Central limit theorem

SAN Statistics

Central Limit Theorem (CLT)

Theorem: Let $X_1,X_2,X_3,\dots$ be a sequence of independent and identically distributed (i.i.d.) random variables with mean $\mu =E[X_i]$ and finite variance ${\sigma }^2=\text{Var}(X_i)>0$.

Let $S_n=X_1+X_2+\cdots +X_n$ denote the sum. Then:

$\frac{S_n-n\mu }{\sigma \sqrt{n}}=\frac{S_n-E[S_n]}{\sqrt{\text{Var}(S_n)}}\stackrel{d}{\to }N(0,1)$

where $\stackrel{d}{\to }$ denotes convergence in distribution.

Equivalently for the sample mean ${\bar{X}}_n=\frac{S_n}{n}$:

$\frac{{\bar{X}}_n-\mu }{\sigma /\sqrt{n}}\stackrel{d}{\to }N(0,1)$

or: ${\bar{X}}_n\sim \text{approximately }N\left(\mu ,\frac{{\sigma }^2}{n}\right)\text{ for large }n$

Key insight: Standardize by dividing by $\sigma \sqrt{n}$ (or $\sigma /\sqrt{n}$ for the mean), not just $\sigma$.

---

Applications

1. Approximating distributions

• Even if $X_i$ are not normally distributed, their sum/mean is approximately normal for large $n$
• Rule of thumb: $n\ge 30$ often sufficient

2. Confidence intervals

• Construction: ${\bar{X}}_n\pm z_{\alpha /2}\cdot \frac{\sigma }{\sqrt{n}}$

3. Hypothesis testing

• Z-tests for means with large samples

4. Quality control

• Process monitoring (control charts)

5. Finance

• Portfolio returns modeling (sum of many small effects)

6. Monte Carlo simulation

• Error estimation

Why it matters: CLT explains why the normal distribution appears so frequently in practice—any phenomenon that results from the sum of many independent factors will be approximately normally distributed.