Central Limit Theorem

Theorem

Let $X_1,X_2,\dots$ be a sequence of independent identically distributed random variables with

Mean $E\left[X_i\right] = \mu \in (-\infty,\infty) $

Variance $V\left(X_i\right) = \sigma^2>0$

Let $\displaystyle S_n = \sum_{i=1}^{n}X_i$

Then

$\displaystyle \frac{S_n - n\mu}{\sqrt{n\sigma^2}} \xrightarrow{D} N\left(0,1\right)$ as $n\to \infty$

that is, converges in distribution to a standard normal

Proof

Let $\displaystyle Y_i = \frac{X_i - \mu}{\sigma}$

We have that $E\left[Y_i\right] = 0$ and $E[Y_i^2] = 1$

Then the characteristic function can be written

$\displaystyle \phi_{Y_i} = 1 - \frac{t^2}{2} +o\left(t^2\right)$ by Taylor's Theorem

Now let

	\(\displaystyle \)	\(\displaystyle \)	\(\displaystyle \)	\(\displaystyle U_n\)	\(=\)	\(\displaystyle \)	\(\displaystyle \frac {S_n - n \mu} {\sqrt {n\sigma^2} }\)	\(\displaystyle \)	\(\displaystyle \)
	\(\displaystyle \)	\(\displaystyle \)	\(\displaystyle \)	\(\displaystyle \)	\(=\)	\(\displaystyle \)	\(\displaystyle \sum_{i=1}^{n} \frac {X_i - \mu} {\sqrt {n\sigma^2} }\)	\(\displaystyle \)	\(\displaystyle \)
	\(\displaystyle \)	\(\displaystyle \)	\(\displaystyle \)	\(\displaystyle \)	\(=\)	\(\displaystyle \)	\(\displaystyle \frac {1} {\sqrt{n} } \sum_{i=1}^{n} \left( \frac {X_i - \mu} {\sigma }\right)\)	\(\displaystyle \)	\(\displaystyle \)
	\(\displaystyle \)	\(\displaystyle \)	\(\displaystyle \)	\(\displaystyle \)	\(=\)	\(\displaystyle \)	\(\displaystyle \frac {1} {\sqrt{n} } \sum_{i=1}^{n} Y_i\)	\(\displaystyle \)	\(\displaystyle \)

Then its characteristic function is given by

	\(\displaystyle \)	\(\displaystyle \)	\(\displaystyle \)	\(\displaystyle \phi_{U_n}\left({t}\right)\)	\(=\)	\(\displaystyle \)	\(\displaystyle E\left[ e^{itU_n} \right]\)	\(\displaystyle \)	\(\displaystyle \)
	\(\displaystyle \)	\(\displaystyle \)	\(\displaystyle \)	\(\displaystyle \)	\(=\)	\(\displaystyle \)	\(\displaystyle E\left[ exp\left(\frac{it}{\sqrt{n} } \sum_{n}^{i=1}Y_i\right) \right]\)	\(\displaystyle \)	\(\displaystyle \)
	\(\displaystyle \)	\(\displaystyle \)	\(\displaystyle \)	\(\displaystyle \)	\(=\)	\(\displaystyle \)	\(\displaystyle \prod_{n}^{i=1} E\left[ exp\left(\frac{it }{\sqrt{n} }Y_i \right) \right]\)	\(\displaystyle \)	\(\displaystyle \)	Since $Y_i$ are independent identically distributed
	\(\displaystyle \)	\(\displaystyle \)	\(\displaystyle \)	\(\displaystyle \)	\(=\)	\(\displaystyle \)	\(\displaystyle \prod_{n}^{i=1} \phi_{Y_i}\left(\frac{t}{\sqrt{n} }\right)\)	\(\displaystyle \)	\(\displaystyle \)
	\(\displaystyle \)	\(\displaystyle \)	\(\displaystyle \)	\(\displaystyle \)	\(=\)	\(\displaystyle \)	\(\displaystyle \left(\phi_{Y_i}\left(\frac{t}{\sqrt{n} } \right)\right)^n\)	\(\displaystyle \)	\(\displaystyle \)	Since $Y_i$ are independent identically distributed
	\(\displaystyle \)	\(\displaystyle \)	\(\displaystyle \)	\(\displaystyle \)	\(=\)	\(\displaystyle \)	\(\displaystyle \left[1-\frac{t^2}{2n} +o\left( {t^2} \right) \right]^n\)	\(\displaystyle \)	\(\displaystyle \)

Recall that the characteristic equation of a standard normal is given by $\displaystyle e^{-\frac{1}{2}t^2} $.

Indeed the characteristic equations of the series converges to the standard normal characteristic equation.

$\displaystyle \left[1-\frac{t^2}{2n} +o\left( {t^2} \right) \right]^n \to e^{-\frac{1}{2}t^2} $ as $n\to \infty$

Then Lévy’s continuity theorem applies, giving, in particular, that the convergence in distribution of the $U_n$ to some random variable with standard normal distribution is equivalent to continuity of the limiting characteristic equation at $t=0$. But, $\displaystyle e^{-\frac{1}{2}t^2} $ is clearly continuous at $0$, so we have that $\displaystyle \frac{S_n - n\mu}{\sqrt{n\sigma^2}} $ converges in distribution to a standard normal random variable.

$\blacksquare$

Sources

G. Grimmett & D. Stirzaker : Probability and Random Processes (2001)