Abstract Integration

On a Measure space $(\Omega ,\mathcal{F},\mu )$, we want to define the integration of a function $f:\Omega \to \mathbb{R}$ denoted as

$${\int }_{\Omega }f\mathrm{d}\mu .$$

• Throughout this note, we use convention $0\cdot \infty =0$.

When the integral is well-defined, we have the following properties:

General Measure	Probability Measure
${\int }_{\Omega }{1}_B\mathrm{d}\mu =\mu (b)$	$\mathbb{E}[1_B]=P(B)$
$f\ge 0⟹\int g\mathrm{d}\mu \ge 0$	$X\ge 0⟹\mathbb{E}[X]\ge 0$
$f=0\text{ a.e. }⟹\int f\mathrm{d}\mu =0$	$X=0\text{ a.s.}⟹\mathbb{E}[X]=0$
$f\le g\text{ a.e. }⟹\int f\mathrm{d}\mu \le \int g\mathrm{d}\mu$	$X\le Y\text{ a.s. }⟹\mathbb{E}[X]\le \mathbb{E}[Y]$
$f=g\text{ a.e. }⟹\int f\mathrm{d}\mu =\int g\mathrm{d}\mu$	$X=Y\text{ a.s. }⟹\mathbb{E}[X]=\mathbb{E}[Y]$
$f\ge 0\wedge \int f\mathrm{d}\mu =0⟹f=0\text{ a.e.}$	$X\ge 0\wedge \mathbb{E}[X]=0⟹X=0\text{ a.s.}$
$\int (f+g)\mathrm{d}\mu =\int f\mathrm{d}\mu +\int g\mathrm{d}\mu$	$\mathbb{E}[X+Y]=\mathbb{E}[X]+\mathbb{E}[Y]$
$\int (af)\mathrm{d}\mu =a\int f\mathrm{d}\mu$	$\mathbb{E}[aX]=a\mathbb{E}[X]$
$0\le f_n\uparrow f\text{ a.e. }⟹\int f_n\mathrm{d}\mu \uparrow \int f\mathrm{d}\mu$	$0\le X_n\uparrow X\text{ a.s. }⟹\mathbb{E}[X_n]\uparrow \mathbb{E}[X]$
$f\ge 0⟹\nu (B):={\int }_{B}f\mathrm{d}\mu$ is a measure	$f\ge 0\wedge \int f\mathrm{d}P=1⟹\nu (B):={\int }_{B}f\mathrm{d}P$ is a probability measure

To define such an integral, we follow a standard program: “simple → nonnegative → general” approach.

Integral of Simple Functions

$g:\Omega \to \mathbb{R}$ is called simple if it is measurable and has a finite range. The canonical representation of a simple function is

$$g(\omega )=\sum _{i=1}^n{a}_i{1}_{A_i}(\omega ),$$

where $a_i\in \mathbb{R}$ are distinct and $A_i=\left\{\omega :g(\omega )=a_i\right\}\in \mathcal{F}$ form a disjoint partition of $\Omega$. A simple function have have multiple representations, but the canonical representation is unique.

For a simple function $g$, we define its integral as

$$\int g\mathrm{d}\mu =\sum _{i=1}^n{a}_i\mu (A_i).$$

We can verify that:

• This definition is well-defined, i.e., it does not depend on the representation of $g$.
• The sum of two simple functions is simple.
• All properties in ^tab-prop.

Integral of Nonnegative Functions

For a nonnegative (extended-valued) measurable function $g$, denote $S(g)$ the set of all nonnegative simple functions $q$ such that $q\le g$, and define

$$\int g\mathrm{d}\mu =\underset{q\in S(g)}{\sup }\int q\mathrm{d}\mu .$$

We can verify that this definition satisfies all properties in ^tab-prop; we will provide some proofs below.

Zero Variation

We prove that $g\ge 0\wedge \int g\mathrm{d}\mu =0$ implies $g=0$ a.e. We will show contradiction if $g\ne 0$ on a set of positive measure. Let $B:=\left\{\omega :g(\omega )>0\right\}$. Then, $\mu (B)>0$. Let $B_n:=\left\{\omega :g(\omega )>1/n\right\}$. Then $B_n\uparrow B$. Thus $\mu (B_n)\uparrow \mu (B)$, which indicates that $\mu (B_{n_0})>0$ for some $n_0$. We have

$$\int g\mathrm{d}\mu \ge \int 1_{B_{n_0}}g\mathrm{d}\mu \ge \frac{1}{n_0}\mu (B_{n_0})>0.$$

Monotone Convergence Theorem

We first prove the case where $g_n\uparrow q$ and $q$ is a simple function ($g_n$ are not necessarily simple). Suppose $q$ has the canonical representation $q={\sum }_{i=1}^m{a}_i{1}_{A_i}$. If $\int q\mathrm{d}\mu =\infty$, then there exists $i$ such that $a_i>0$ and $\mu (A_i)=\infty$. Let $B_n=\left\{\omega \in A_i:g_n(\omega )>a_i/2\right\}$. Since $q_n\uparrow q$, $B_n\uparrow A_i$. Then we have

$$\underset{n\to \infty }{\lim }\int g_n\mathrm{d}\mu \ge \underset{n\to \infty }{\lim }\int 1_{B_n}{g}_n\mathrm{d}\mu \ge \underset{n\to \infty }{\lim }\frac{a_i}{2}\mu (B_n)=\frac{a_i}{2}\mu (A_i)=\infty .$$

If $\int q\mathrm{d}\mu <\infty$, then $\mu (A_i)<\infty$ for all $i$ such that $a_i>0$. Let $A={\bigsqcup }_{i=1}^m{A}_i$. By the countable additivity of $\mu$, we have $\mu (A)<\infty$. Fixing a $R>0$, let $B_n=\left\{\omega \in A:g_n(\omega )\ge q(\omega )-1/R\right\}$. Then $B_n\uparrow A$. We have

$$\int g_n\mathrm{d}\mu +\int 1_{B_n}/R\mathrm{d}\mu \ge \int 1_{B_n}q\mathrm{d}\mu =\int q\mathrm{d}\mu -{\int }_{A\setminus B_n}q\mathrm{d}\mu \ge \int q\mathrm{d}\mu -\underset{i}{\max }{a}_i\mu (A\setminus B_n).$$

Since $B_n\uparrow A$, we have

$$\underset{n\to \infty }{\lim }\int g_n\mathrm{d}\mu +\mu (A)/R\ge \int q\mathrm{d}\mu -\underset{i}{\max }{a}_i\mu (A\setminus A)=\int q\mathrm{d}\mu .$$

Letting $R\to \infty$ gives ${\lim }_{n\to \infty }\int g_n\mathrm{d}\mu \ge \int q\mathrm{d}\mu$. On the other hand, we have

$$\int g_n\mathrm{d}\mu \le \int q\mathrm{d}\mu ⟹\underset{n\to \infty }{\lim }\int g_n\mathrm{d}\mu \le \int q\mathrm{d}\mu .$$

We now consider the more general case where $g_n\uparrow g$. For any simple function $q\in S(g)$, we have

$$0\le \min \left\{g_n,q\right\}\uparrow \min \left\{g,q\right\}=q.$$

From the previous case, we have

$$\underset{n\to \infty }{\lim }\int g_n\mathrm{d}\mu \ge \underset{n\to \infty }{\lim }\int \min \left\{g_n,q\right\}\mathrm{d}\mu =\int q\mathrm{d}\mu .$$

Taking supremum over $q\in S(g)$ gives

$$\underset{n\to \infty }{\lim }\int g_n\mathrm{d}\mu \ge \underset{q\in S(g)}{\sup }\int q\mathrm{d}\mu =\int g\mathrm{d}\mu .$$

On the other hand, we have $g_n\le g⟹\int g_n\mathrm{d}\mu \le \int g\mathrm{d}\mu$.

Now suppose $g_n\uparrow g$ a.e. Then there exists $g_n^{\prime }$ and $g^{\prime }$ such that $g_n=g_n^{\prime }$ a.e., $g=g^{\prime }$ a.e., and $g_n^{\prime }\uparrow g^{\prime }$. By the previous case, we have

$$\underset{n\to \infty }{\lim }\int g_n\mathrm{d}\mu =\underset{n\to \infty }{\lim }\int g_n^{\prime }\mathrm{d}\mu =\int g^{\prime }\mathrm{d}\mu =\int g\mathrm{d}\mu .$$

Constructing Approximation

The proof of MCT does not provide a way to construct the approximating sequence. An explicit construction is as follows:

$$g_n(\omega )=\{\begin{array}{llcc}n,& \text{if }g(\omega )\ge n,& & \\ k/2^n,& \text{if }k/2^n\le g(\omega )<(k+1)/2^n,& & \text{ for }k=0,1,\dots ,n{2}^n-1.\end{array}$$

In words, the function $g_n$ is a quantized version of $g$. For every $\omega$, the value of $g(\omega )$ is first capped at $n$, and then rounded down to the nearest multiple of $2^{-n}$. One can verify that $g_n$ is simple and $g_n\uparrow g$.

• A byproduct of this construction is that we show that a nonnegative function is measurable if and only if it is the monotonic and pointwise limit of simple functions. This conclusion generalizes to general functions.

Integral of General Functions

Consider now a general measurable function $f:\Omega \to \bar{\mathbb{R}}$. Let $f^{+}=\max \left\{f,0\right\}$ and $f^{-}=\max \left\{-f,0\right\}$. Then $f=f^{+}-f^{-}$ and $f^{+},f^{-}$ are nonnegative. We define

$$\int f\mathrm{d}\mu =\int f^{+}\mathrm{d}\mu -\int f^{-}\mathrm{d}\mu .$$

This definition is well-defined as long as one of the integrals on the RHS is finite. We can verify that this definition satisfies all properties in ^tab-prop.

Consistency with Expectation

With the help of abstract integration, we can define the expectation, the integral of probability measures, for general random variables. However, functions like Random Variables and Densities carry the operations between different measure spaces. It is important to verify the consistency of abstract integration among different measure spaces.

---

Consider a Probability Space $(\Omega ,\mathcal{F},\mathbb{P})$ and a Random Variable $X:\Omega \to \mathbb{R}$. This random variable induces another probability space $(\mathbb{R},\mathcal{B},{\mathbb{P}}_X)$. Further, let $g:\mathbb{R}\to \mathbb{R}$ be a measurable function, $Y=g(X)$ be the new random variable, whose induced probability space is $(\mathbb{R},\mathcal{B},{\mathbb{P}}_Y)$. We want to show that

$$\mathbb{E}[Y]=\int Y\mathrm{d}\mathbb{P}=\int g\mathrm{d}{\mathbb{P}}_X=\int y\mathrm{d}{\mathbb{P}}_Y.$$

A special case of this consistency is proved in Proof of Function Transformation. To prove the general results, we adopt the stnadard program, which starts with simple functions. Suppose $g$ is a simple function whose range is $\{y_i\}$. We have

$$\int Y\mathrm{d}\mathbb{P}=\sum _i{y}_i\mathbb{P}(\{\omega :g\circ X(\omega )=y_i\})=\sum _i{y}_i\mathbb{P}(\{\omega :Y(\omega )=y_i\})=\sum _i{y}_i{\mathbb{P}}_Y(y_i)=\int y\mathrm{d}{\mathbb{P}}_Y.$$

Similarly

$$\int g\mathrm{d}{\mathbb{P}}_X=\sum _i{y}_i{\mathbb{P}}_X(\{x:g(x)=y_i\})=\sum _i{y}_i{\mathbb{P}}_X(g^{-1}(y_i))=\sum _i{y}_i\mathbb{P}(\{\omega :X(\omega )\in g^{-1}(y_i)\})=\sum _i{y}_i\mathbb{P}(\{\omega :g(X(\omega ))=y_i\}),$$

which establishes $\int Y\mathrm{d}\mathbb{P}=\int g\mathrm{d}{\mathbb{P}}_X$.

The general case follows by approximating $g$ by simple functions.

---

We now consider a continuous r.v. $X$ with Density $f$. We want to show that

$$\mathbb{E}[g(X)]=\int g\mathrm{d}{\mathbb{P}}_X=\int (gf)\mathrm{d}\lambda ,$$

where $\lambda$ is the Lebesgue Measure on $\mathbb{R}$.

Again, we start with a simple function $g={\sum }_i{y}_i{1}_{A_i}$. We have

$$\int g\mathrm{d}{\mathbb{P}}_X=\sum _i{y}_i{\mathbb{P}}_X(A_i)=\sum _i{y}_i{\int }_{A_i}f\mathrm{d}\lambda =\int \sum _i{y}_i{1}_{A_i}f\mathrm{d}\lambda =\int gf\mathrm{d}\lambda .$$

The general case follows by approximating $g$ by simple functions.

Dominated Convergence Theorem

$X_n\to X$ a.e. Suppose $|X_n|\le Y$ a.e. for some integrable $Y$. Then ${\lim }_{n\to \infty }\mathbb{E}[X_n]=\mathbb{E}[X]$. Or $X_n\stackrel{L_1}{\to }X$.

Proof

WLOG, we assume $X_n\to X$ and $|X_n|\le Y$ everywhere. By Fatou’s Lemma, we have

$$\mathbb{E}[X]=\mathbb{E}[\underset{n}{\mathrm{liminf}}{X}_n]\le \underset{n}{\mathrm{liminf}}\mathbb{E}[X_n]\le \underset{n}{\mathrm{limsup}}\mathbb{E}[X_n]\le \mathbb{E}[\underset{n}{\mathrm{limsup}}{X}_n]=\mathbb{E}[X].$$

Therefore, ${\lim }_{n\to \infty }\mathbb{E}[X_n]=\mathbb{E}[X]$.