Independence

Independent Events

The following definitions of independence are equivalent for two events $A,B$:

• $P(A\cap B)=P(A)P(B)$.
• If $P(B)>0$, $P(A|B)=P(A)$ using Conditional Probability.

For a family (potentially infinite) of events $\{A_i{\}}_{i\in \mathcal{I}}$, they are independent if for any finite subset $\{A_{i_1},\dots ,A_{i_n}\}$, the events are independent.

Independent Sigma Fields

Two Sigma Fields ${\mathcal{F}}_1,{\mathcal{F}}_2$ on the same probability space are independent if any two events $A\in {\mathcal{F}}_1,B\in {\mathcal{F}}_2$ are independent.

For a family of Sigma Fields $\{{\mathcal{F}}_i{\}}_{i\in \mathcal{I}}$, they are independent if we pick one arbitrary event $A_i$ from each ${\mathcal{F}}_i$, the events $\{A_i{\}}_{i\in \mathcal{I}}$ are independent.

Thm

If ${\mathcal{F}}_i=\sigma ({\mathcal{G}}_i)$ and ${\mathcal{G}}_i$ is closed under intersection, then ${\mathcal{F}}_i$ are independent if and only if ${\mathcal{G}}_i$ are independent, i.e., for arbitrary events $A_i\in {\mathcal{G}}_i$, the events $\{A_i\}$ are independent.

Independent Random Variables

• Intuitively, two random variables are independent if any partial information on the realized value of one random variable does not change the distribution of the other.

For two Random Variables $X,Y$, the following statements are equivalent

• $X,Y$ are independent
• $P(X\in A,Y\in B)=P(X\in A)\cdot P(Y\in B)$ for any borel sets $A,B$
  • That is, events $\{X\in A\}$ and $\{Y\in B\}$ are Independent Events
• $P(X\in A|Y\in B)=P(X\in A)$ for any possible event $A,B$
  • Equivalently, $f_{X\mid Y}(a\mid b)=f_X(a)$ if they are jointly continuous¹
  • $p_{X\mid Y}(a\mid b)=p_X(a)$ if they are jointly discrete
• $F(a,b)=F_X(a)\cdot F_Y(b)$
  • This is a special case of the second statement
• $f(a,b)=f_X(a)\cdot f_Y(b)$ if they are jointly continuous¹
• $p(a,b)=p_X(a)\cdot p_Y(b)$ if they are jointly discrete

For a collection of r.v.s $\{X_i{\}}_{i\in \mathcal{I}}$, where $\mathcal{I}$ is a potentially infinite index set, they are independent if any finite subset of them are independent.

Independence Preserved by Functions

Let $X$, $Y$ be independent r.v.s. For any functions $f,g$, $f(X)$ and $g(Y)$ are also independent.

Joint Probability Space

It is intuitive to discuss the independence of Random Variables on different Probability Spaces. On the other hand, it is more convenient to discuss the Joint Distribution of $X$ and $Y$ if they have the same probability space; and then we can formally discuss their conditional probability, independence, etc. Here we introduce a canonical way to construct independent random variables on the same probability space with $X\in ({\Omega }_1,{\mathcal{F}}_1,P_1)$ and $Y\in ({\Omega }_2,{\mathcal{F}}_2,P_2)$. We first construct the probability space:

$$\{\begin{array}{l}\Omega ={\Omega }_1\times {\Omega }_2,\\ \mathcal{F}=\sigma ({\mathcal{F}}_1\times {\mathcal{F}}_2),\\ P=P_1\otimes P_2:A_1\times A_2\mapsto P(A_1\times A_2)=P(A_1)P(A_2).\end{array}$$

Then we define the new random variables $\overline{X}({\omega }_1,{\omega }_2)=X({\omega }_1)$ and $\overline{Y}({\omega }_1,{\omega }_2)=Y({\omega }_2)$ on $\Omega$. One can easily verify that $\overline{X}$ and $\overline{Y}$ are $(\mathcal{F},\mathcal{B})$-measurable, and $F_{\overline{X}}=F_X$, $F_{\overline{Y}}=F_Y$. Further, $\overline{X}\perp \overline{Y}$.

Rmk

• $\mathcal{F}=\sigma \left(\{A_1\times {\Omega }_2:A_1\in {\mathcal{F}}_1\}\cup \{{\Omega }_1\times A_2:A_2\in {\mathcal{F}}_2\}\right)$
• $P$ satisfying the condition is unique

Footnotes

1. $f(a,b)=\frac{{\partial }^2}{\partial a\partial b}F(a,b)$. ↩ ↩²