Law of Total Variance

For Random Variables $X$, $Y$ on the same probability space, if $Y$ has finite variance, then

$$\mathrm{Var}(X)=\mathbb{E}[\mathrm{Var}(X|Y)]+\mathrm{Var}(\mathbb{E}[X|Y]).$$

Proof

$$\begin{array}{rl}\mathrm{Var}(X)=& \mathbb{E}{X}^2-(\mathbb{E}X{)}^2\\ =& \mathbb{E}\left[\mathbb{E}[X^2\mid Y]\right]-{\left(\mathbb{E}[\mathbb{E}[X\mid Y]]\right)}^2\\ =& \mathbb{E}\left[\mathbb{E}[X^2\mid Y]-{\left(\mathbb{E}[X\mid Y]\right)}^2\right]+\mathbb{E}\left[{\left(\mathbb{E}[X\mid Y]\right)}^2\right]-{\left(\mathbb{E}[\mathbb{E}[X\mid Y]]\right)}^2\\ =& \mathbb{E}\left[\mathrm{Var}(X\mid Y)\right]+\mathrm{Var}\left(\mathbb{E}[X\mid Y]\right).\end{array}$$

Interpretation

Typically, we want to find a causal random variable $Y$ that explains (the variance of) $X$.

• For example, let $X$ be the weight of a dog, and $Y$ be the dog’s breed.

Intuitively, fixing a breed $Y=y$, the variance of $X\mid Y=y$ should be small; this contributes to the “unexplained” part of the variance of $X$: $\mathbb{E}[\mathrm{Var}(X\mid Y)]$. And the dog’s weight should be close to the average weight of the breed $\mathbb{E}[X\mid Y]$, whose variance is the “explained” part of the variance of $X$: $\mathrm{Var}(\mathbb{E}[X\mid Y])$.