Deviation For Central Limit Theorem

Background Knowledge

Characteristic Function

For any random variable \(X\), its characteristic function \(\phi_X(t)\) is

\[\phi_X(t) = E[e^{itX}].\]

If the moment-generating function of \(X\), \(M_X(t)\), exists, then

\[\phi_X(t) = M_{iX}(t) = M_X (it).\]

Different from moment-generating function, characteristic function always exists.

A good property of Characteristic Function is that the map between characteristic function and random distribution is one-to-one. (We are going to use)

Derivation

Let \(f(x)\) to be the pdf of random variable \(X\), then its characteristic function is

\[\phi(t) = \int_{-\infty}^{+\infty} e^{itx} f(x) dx.\]

• Lemma 1: \(Y = X_1 + X_2\), and \(X_1\) is independent from \(X_2\), then

\[\begin{split}\begin{align} & \phi_Y(t) = \phi_{X_1+X_2}(t) \\ &= \int \int e^{it(x_1 + x_2)} f(x_1) g(x_2) d x_1 d x_2 \\ &= \int e^{it x_1)} f(x_1) d x_1 \int e^{it x_2)} g(x_2) d x_2\\ &= \phi_{X_1}(t) \phi_{X_2}(t) \end{align}\end{split}\]

• Lemma 2: \(Y = aX + b\), then

\[\begin{split}\begin{align} & \phi_Y(t) = \phi_{aX + b}(t) \\ &= \int e^{it(ax + b)} f(x) d x \\ &= e^{itb} \int e^{i(at)x} f(x) dx \\ &= e^{itb} \phi_{X}(at) \end{align}\end{split}\]

• Lemma 3: Let \(X \sim N(0, 1)\), then its characteristic function is

\[\begin{split}\begin{align} & \phi(t) = \int e^{itx} \frac{1}{\sqrt{2 \pi}} e^{- \frac{x^2}{2}} dx \\ &= \int \frac{1}{\sqrt{2 \pi}} e^{- \frac{(x - it)^2}{2}} e^{-\frac{t^2}{2}} d(x-it)\\ &= e^{-\frac{t^2}{2}} \int \frac{1}{\sqrt{2 \pi}} e^{- \frac{(x - it)^2}{2}} d(x-it)\\ &= e^{-\frac{t^2}{2}} \end{align}\end{split}\]

Proof

Suppose \(X_1, X_2, \cdots, X_n\) are i.i.d, and \(E[X] = \mu\), \(Var[X] = \sigma^2\). Let \(S = \sum_{i=1}^n X_i\), \(\bar{X} = \frac{S}{n}\), \(Y = \sqrt{n} \frac{\bar{X} - \mu}{\sigma}\).

From Lemma 1, we have

\[\phi_S(t) = [\phi_X(t)]^n.\]

From Lemma 2, we have

\[\phi_{\bar{X}}(t) = \phi_S(t/n) = [\phi_X(t/n)]^n,\]

and

\[\begin{split}\begin{align} & \phi_{Y}(t) = e^{i(-\frac{\sqrt{n}}{\sigma}\mu)t} \phi_{\bar{X}}(\frac{\sqrt{n}}{\sigma}t)\\ &= e^{i(-\frac{\sqrt{n}}{\sigma}\mu)t} [ \phi_{X}(\frac{1}{\sigma\sqrt{n}}t) ]^n . \end{align}\end{split}\]

Take naturn logarithm to both side of \(\phi_{Y}(t)\), then

\[\begin{split}\begin{align} & \ln \phi_{Y}(t) = \ln \big\{ e^{i(-\frac{\sqrt{n}}{\sigma}\mu)t} [ \phi_{X}(\frac{1}{\sigma\sqrt{n}}t) ]^n \big\}\\ &= - i\frac{\sqrt{n}}{\sigma}\mu t + n \ln \big[ \phi_{X}(\frac{1}{\sigma\sqrt{n}}t) \big] \\ &= n \big\{ -i \frac{1}{\sigma\sqrt{n}} \mu t + \ln \big[ \phi_{X}(\frac{1}{\sigma\sqrt{n}}t) \big] \big\} \end{align}\end{split}\]

Let \(p = \frac{t}{\sigma \sqrt{n}}\). When \(n \rightarrow \infty\), \(p \rightarrow 0\). Thus,

\[\begin{split}\begin{align} & \lim_{n \rightarrow \infty} \ln \phi_Y(t) = \lim_{n \rightarrow \infty} n \big\{ -i \frac{1}{\sigma\sqrt{n}} \mu t + \ln \big| \phi_{X}(\frac{1}{\sigma\sqrt{n}}t) \big| \big\}\\ &= \frac{t^2}{\sigma^2} \lim_{p \rightarrow 0} \frac{-i\mu p + \ln|\phi_X(p)|}{p^2}. \end{align}\end{split}\]

Since \(\phi_X(0)= 1\), \(\phi'_X(0) = \int i x f(x) = i\mu\), and \(\phi''_X(0) = \int - x^2 f(x) = -\mu^2 - \sigma^2\). The limit is \(\frac{0}{0}\). Apply L’Hospital once. We then have

\[\begin{align} \frac{t^2}{\sigma^2} \lim_{p \rightarrow 0} \frac{-i\mu p + \ln|\phi_X(p)|}{p^2} &= \frac{t^2}{\sigma^2} \lim_{p \rightarrow 0} \frac{-i\mu + \frac{1}{\phi_X(p) \phi'_X(p)}}{2p} \end{align}\]

\(\frac{0}{0}\) again. Apply L’Hospital once more, and

\[\begin{split}\begin{align} &= \frac{t^2}{\sigma^2} \lim_{p \rightarrow 0} \frac{\phi''_X(p) \phi_X(p) - \phi'_X(p) \phi'_X(p)}{2|\phi_X(p)|^2} \\ &= \frac{t^2}{\sigma^2} \frac{\phi''_X(0) \phi_X(0) - \phi'_X(0) \phi'_X(0)}{2|\phi_X(0)|^2}\\ &= \frac{t^2}{\sigma^2} \frac{(-\mu^2 - \sigma^2) \cdot 1 - i\mu \cdot i\mu}{2 \cdot 1} \\ &= - \frac{t^2}{2}. \end{align}\end{split}\]

Remember we took logarithm? \(\lim_{n \rightarrow \infty} \phi_Y(t) = e^{-t^2/2}\). Apply the one-to-one mapping between characteristic function and random distribution, \(Y \sim N(0, 1)\).