Variance

The variance of a Random Variable is

$$\mathrm{Var}(X)=\mathbb{E}[(X-\mathbb{E}[X]{)}^2]=\mathbb{E}[X^2]-(\mathbb{E}[X]{)}^2.$$

Properties

• If $X_1$ and $X_2$ are independent, then $\mathrm{Var}(X_1+X_2)=\mathrm{Var}(X_1)+\mathrm{Var}(X_2)$.
  • Generally, this is not true, unlike Expectation
• For any linear transformation on random vector $X$, $\mathrm{Var}(AX+b)=A\mathrm{Var}(X)A^T$
  • special case: $\mathrm{Var}(aX+b)=a^2\mathrm{Var}(X)$

Sample Variance

For a finite sample of size $n$, the sample variance is

$$s^2=\frac{{\sum }_{i=1}^n(x_i-\overline{x}{)}^2}{n-1},$$

where $\overline{x}=\frac{{\sum }_{i=1}^n{x}_i}{n}$. The $n-1$ in the denominator makes it a unbiased estimator of the real variance of the population. This is because

$$\begin{array}{rl}\mathbb{E}[s^2]& =\frac{1}{n-1}\sum _{i=1}^n\mathbb{E}[x_i^2-2x_i\overline{x}+{\overline{x}}^2]\\ & =\frac{n}{n-1}\left(\left(1-\frac{2}{n}+\frac{n}{n^2}\right)\mathbb{E}[x_i^2]+\left(\frac{n^2-n}{n^2}-\frac{2(n-1)}{n}\right)\mathbb{E}[x_1{x}_2]\right)\\ & =\frac{n}{n-1}\frac{n-1}{n}(\mathbb{E}[x_1^2]-\mathbb{E}[x_1{]}^2)\\ & =\mathrm{Var}(x_1)\end{array}$$