Central Limit Theorem

From ProofWiki
Jump to: navigation, search

Theorem

Let $X_1,X_2,\dots$ be a sequence of independent identically distributed random variables with

Mean $E\left[X_i\right] = \mu \in (-\infty,\infty) $
Variance $V\left(X_i\right) = \sigma^2>0$

Let $\displaystyle S_n = \sum_{i=1}^{n}X_i$

Then

$\displaystyle \frac{S_n - n\mu}{\sqrt{n\sigma^2}} \xrightarrow{D} N\left(0,1\right)$ as $n\to \infty$

that is, converges in distribution to a standard normal

Proof

Let $\displaystyle Y_i = \frac{X_i - \mu}{\sigma}$

We have that $E\left[Y_i\right] = 0$ and $E[Y_i^2] = 1$

Then the characteristic function can be written

$\displaystyle \phi_{Y_i} = 1 - \frac{t^2}{2} +o\left(t^2\right)$ by Taylor's Theorem

Now let

\(\displaystyle \) \(\displaystyle \) \(\displaystyle \) \(\displaystyle U_n\) \(=\) \(\displaystyle \) \(\displaystyle \frac {S_n - n \mu} {\sqrt {n\sigma^2} }\) \(\displaystyle \) \(\displaystyle \)                    
\(\displaystyle \) \(\displaystyle \) \(\displaystyle \) \(\displaystyle \) \(=\) \(\displaystyle \) \(\displaystyle \sum_{i=1}^{n} \frac {X_i - \mu} {\sqrt {n\sigma^2} }\) \(\displaystyle \) \(\displaystyle \)                    
\(\displaystyle \) \(\displaystyle \) \(\displaystyle \) \(\displaystyle \) \(=\) \(\displaystyle \) \(\displaystyle \frac {1} {\sqrt{n} } \sum_{i=1}^{n} \left( \frac {X_i - \mu} {\sigma }\right)\) \(\displaystyle \) \(\displaystyle \)                    
\(\displaystyle \) \(\displaystyle \) \(\displaystyle \) \(\displaystyle \) \(=\) \(\displaystyle \) \(\displaystyle \frac {1} {\sqrt{n} } \sum_{i=1}^{n} Y_i\) \(\displaystyle \) \(\displaystyle \)                    

Then its characteristic function is given by

\(\displaystyle \) \(\displaystyle \) \(\displaystyle \) \(\displaystyle \phi_{U_n}\left({t}\right)\) \(=\) \(\displaystyle \) \(\displaystyle E\left[ e^{itU_n} \right]\) \(\displaystyle \) \(\displaystyle \)                    
\(\displaystyle \) \(\displaystyle \) \(\displaystyle \) \(\displaystyle \) \(=\) \(\displaystyle \) \(\displaystyle E\left[ exp\left(\frac{it}{\sqrt{n} } \sum_{n}^{i=1}Y_i\right) \right]\) \(\displaystyle \) \(\displaystyle \)                    
\(\displaystyle \) \(\displaystyle \) \(\displaystyle \) \(\displaystyle \) \(=\) \(\displaystyle \) \(\displaystyle \prod_{n}^{i=1} E\left[ exp\left(\frac{it }{\sqrt{n} }Y_i \right) \right]\) \(\displaystyle \) \(\displaystyle \)          Since $Y_i$ are independent identically distributed          
\(\displaystyle \) \(\displaystyle \) \(\displaystyle \) \(\displaystyle \) \(=\) \(\displaystyle \) \(\displaystyle \prod_{n}^{i=1} \phi_{Y_i}\left(\frac{t}{\sqrt{n} }\right)\) \(\displaystyle \) \(\displaystyle \)                    
\(\displaystyle \) \(\displaystyle \) \(\displaystyle \) \(\displaystyle \) \(=\) \(\displaystyle \) \(\displaystyle \left(\phi_{Y_i}\left(\frac{t}{\sqrt{n} } \right)\right)^n\) \(\displaystyle \) \(\displaystyle \)          Since $Y_i$ are independent identically distributed          
\(\displaystyle \) \(\displaystyle \) \(\displaystyle \) \(\displaystyle \) \(=\) \(\displaystyle \) \(\displaystyle \left[1-\frac{t^2}{2n} +o\left( {t^2} \right) \right]^n\) \(\displaystyle \) \(\displaystyle \)                    

Recall that the characteristic equation of a standard normal is given by $\displaystyle e^{-\frac{1}{2}t^2} $.

Indeed the characteristic equations of the series converges to the standard normal characteristic equation.

$\displaystyle \left[1-\frac{t^2}{2n} +o\left( {t^2} \right) \right]^n \to e^{-\frac{1}{2}t^2} $ as $n\to \infty$

Then Lévy’s continuity theorem applies, giving, in particular, that the convergence in distribution of the $U_n$ to some random variable with standard normal distribution is equivalent to continuity of the limiting characteristic equation at $t=0$. But, $\displaystyle e^{-\frac{1}{2}t^2} $ is clearly continuous at $0$, so we have that $\displaystyle \frac{S_n - n\mu}{\sqrt{n\sigma^2}} $ converges in distribution to a standard normal random variable.

$\blacksquare$

Sources