# Central Limit Theorem

## Theorem

Let $X_1,X_2,\dots$ be a sequence of independent identically distributed random variables with

Mean $E\left[X_i\right] = \mu \in (-\infty,\infty)$
Variance $V\left(X_i\right) = \sigma^2>0$

Let $\displaystyle S_n = \sum_{i=1}^{n}X_i$

Then

$\displaystyle \frac{S_n - n\mu}{\sqrt{n\sigma^2}} \xrightarrow{D} N\left(0,1\right)$ as $n\to \infty$

that is, converges in distribution to a standard normal

## Proof

Let $\displaystyle Y_i = \frac{X_i - \mu}{\sigma}$

We have that $E\left[Y_i\right] = 0$ and $E[Y_i^2] = 1$

Then the characteristic function can be written

$\displaystyle \phi_{Y_i} = 1 - \frac{t^2}{2} +o\left(t^2\right)$ by Taylor's Theorem

Now let

 $$\displaystyle$$ $$\displaystyle$$ $$\displaystyle$$ $$\displaystyle U_n$$ $$=$$ $$\displaystyle$$ $$\displaystyle \frac {S_n - n \mu} {\sqrt {n\sigma^2} }$$ $$\displaystyle$$ $$\displaystyle$$ $$\displaystyle$$ $$\displaystyle$$ $$\displaystyle$$ $$\displaystyle$$ $$=$$ $$\displaystyle$$ $$\displaystyle \sum_{i=1}^{n} \frac {X_i - \mu} {\sqrt {n\sigma^2} }$$ $$\displaystyle$$ $$\displaystyle$$ $$\displaystyle$$ $$\displaystyle$$ $$\displaystyle$$ $$\displaystyle$$ $$=$$ $$\displaystyle$$ $$\displaystyle \frac {1} {\sqrt{n} } \sum_{i=1}^{n} \left( \frac {X_i - \mu} {\sigma }\right)$$ $$\displaystyle$$ $$\displaystyle$$ $$\displaystyle$$ $$\displaystyle$$ $$\displaystyle$$ $$\displaystyle$$ $$=$$ $$\displaystyle$$ $$\displaystyle \frac {1} {\sqrt{n} } \sum_{i=1}^{n} Y_i$$ $$\displaystyle$$ $$\displaystyle$$

Then its characteristic function is given by

 $$\displaystyle$$ $$\displaystyle$$ $$\displaystyle$$ $$\displaystyle \phi_{U_n}\left({t}\right)$$ $$=$$ $$\displaystyle$$ $$\displaystyle E\left[ e^{itU_n} \right]$$ $$\displaystyle$$ $$\displaystyle$$ $$\displaystyle$$ $$\displaystyle$$ $$\displaystyle$$ $$\displaystyle$$ $$=$$ $$\displaystyle$$ $$\displaystyle E\left[ exp\left(\frac{it}{\sqrt{n} } \sum_{n}^{i=1}Y_i\right) \right]$$ $$\displaystyle$$ $$\displaystyle$$ $$\displaystyle$$ $$\displaystyle$$ $$\displaystyle$$ $$\displaystyle$$ $$=$$ $$\displaystyle$$ $$\displaystyle \prod_{n}^{i=1} E\left[ exp\left(\frac{it }{\sqrt{n} }Y_i \right) \right]$$ $$\displaystyle$$ $$\displaystyle$$ Since $Y_i$ are independent identically distributed $$\displaystyle$$ $$\displaystyle$$ $$\displaystyle$$ $$\displaystyle$$ $$=$$ $$\displaystyle$$ $$\displaystyle \prod_{n}^{i=1} \phi_{Y_i}\left(\frac{t}{\sqrt{n} }\right)$$ $$\displaystyle$$ $$\displaystyle$$ $$\displaystyle$$ $$\displaystyle$$ $$\displaystyle$$ $$\displaystyle$$ $$=$$ $$\displaystyle$$ $$\displaystyle \left(\phi_{Y_i}\left(\frac{t}{\sqrt{n} } \right)\right)^n$$ $$\displaystyle$$ $$\displaystyle$$ Since $Y_i$ are independent identically distributed $$\displaystyle$$ $$\displaystyle$$ $$\displaystyle$$ $$\displaystyle$$ $$=$$ $$\displaystyle$$ $$\displaystyle \left[1-\frac{t^2}{2n} +o\left( {t^2} \right) \right]^n$$ $$\displaystyle$$ $$\displaystyle$$

Recall that the characteristic equation of a standard normal is given by $\displaystyle e^{-\frac{1}{2}t^2}$.

Indeed the characteristic equations of the series converges to the standard normal characteristic equation.

$\displaystyle \left[1-\frac{t^2}{2n} +o\left( {t^2} \right) \right]^n \to e^{-\frac{1}{2}t^2}$ as $n\to \infty$

Then Lévy’s continuity theorem applies, giving, in particular, that the convergence in distribution of the $U_n$ to some random variable with standard normal distribution is equivalent to continuity of the limiting characteristic equation at $t=0$. But, $\displaystyle e^{-\frac{1}{2}t^2}$ is clearly continuous at $0$, so we have that $\displaystyle \frac{S_n - n\mu}{\sqrt{n\sigma^2}}$ converges in distribution to a standard normal random variable.

$\blacksquare$

## Sources

G. Grimmett & D. Stirzaker : Probability and Random Processes (2001)