Stochastic Asymptotic Notation

For Random Variable sequences $R_n$ and $X_n$, we denote

$$\{\begin{array}{ll}{X}_n=o_P(R_n)& \text{if }{X}_n=Y_n{R}_n\text{ where }{Y}_n=o_P(0),\\ X_n=O_P(R_n)& \text{if }{X}_n=Y_n{R}_n\text{ where }{Y}_n=O_P(1),\end{array}$$

where $o_P(1)$ represents a sequence that weakly converges to 0, and $O_P(1)$ represents a uniformly tight sequence.

Arithmetic Properties

• $o_P(1)+o_P(1)=o_P(1)$
• $o_P(1)+O_P(1)=O_P(1)$
• $o_P(1)O_P(1)=o_P(1)$
• $(1+o_P(1){)}^{-1}=O_P(1)$
• $o_P(R_n)=R_n{o}_P(1)$
• $O_P(R_n)=R_n{O}_P(1)$
• $o_P(O_P(1))=o_P(1)$
• $R(h)=o(\Vert h{\Vert }^p),h\to 0⟹R(X_n)=o_P(\Vert X_n{\Vert }^p),X_n\stackrel{P}{\to }0$, for any $p$
• $R(h)=O(\Vert h{\Vert }^p),h\to 0⟹R(X_n)=O_P(\Vert X_n{\Vert }^p),X_n\stackrel{P}{\to }0$, for any $p$

Proof

We only prove the last two properties. Define $r(h)=R(h)/\Vert h{\Vert }^p$ for $h\ne 0$ and $r(0)=0$. For the first property, by the condition, $r$ is continuous at $0$. By Continuous Mapping Theorem, $r(X_n)\stackrel{P}{\to }r(0)=0$. Then,

$$R(X_n)=r(X_n)\Vert X_n{\Vert }^p=o_P(\Vert X_n{\Vert }^p).$$

For the second property, we have $r(h)=O(1)$ for $h$ near 0. There exists $M,\delta >0$ such that $|g(h)|\le M$ for $\Vert h\Vert \le \delta$. Thus,

$$P(|g(X_n)|>M)\le P(\Vert X_n\Vert >\delta )\to 0,$$

which implies the uniform tightness of $\{g(X_n)\}$, and thus $R(X_n)=O_P(\Vert X_n{\Vert }^p)$.