Bayes Optimal Test

Recall that Hypothesis Testing is a decision-making problem, where we make a binary decision $A(X)\in \{0,1\}$. Therefore, given a loss function $L(a,\theta )$, we can define the Risk of a test, and thus derive the (Bayes) optimal test. For a binary HT, the loss function always takes the form

$$L(a,\theta )=\{\begin{array}{ll}0,& \text{if }(a=0\wedge \theta \in {\Theta }_0)\vee (a=1\wedge \theta \in {\Theta }_1)\\ c_{\mathrm{FP}},& \text{if }a=1\wedge \theta \in {\Theta }_0\\ c_{\mathrm{FN}},& \text{if }a=0\wedge \theta \in {\Theta }_1\end{array}$$

From now on we consider a simple test. Similarly, a Bayes prior always takes the form

$$Q({\theta }_0)={\pi }_0\in (0,1),Q({\theta }_1)={\pi }_1=1-{\pi }_0.$$

Then, the Bayes Risk is

$$R_B(A,Q)={\mathbb{E}}_{\theta \sim Q}{\mathbb{E}}_{X\sim P_{\theta }}[L(A(X),\theta )]={\pi }_0{P}_{{\theta }_0}(A(X)=1)\cdot c_{\mathrm{FP}}+{\pi }_1{P}_{{\theta }_1}(A(X)=0)\cdot c_{\mathrm{FN}}.$$

When ${\pi }_0={\pi }_1=1/2$ and $c_{\mathrm{FP}}=c_{\mathrm{FN}}=1$, we call it the unbiased risk.

Optimal Test

Suppose $P_{{\theta }_i}$ has density $p_i$. Then, the Bayes-optimal test is given by

$$A^{\ast }(x)=1\{p_1(x)/p_0(x)>({\pi }_0{c}_{\mathrm{FP}})/({\pi }_1{c}_{\mathrm{FN}})\}=1\{p_1{\pi }_1{c}_{\mathrm{FN}}>f_0{\pi }_0{c}_{\mathrm{FP}}\}.$$

Proof

We can simply calculate the posterior:

$$p({\theta }_0\mid x)=\frac{{\pi }_0{p}_0(x)}{{\pi }_0{p}_0(x)+{\pi }_1{p}_1(x)},p({\theta }_1\mid x)=\frac{{\pi }_1{p}_1(x)}{{\pi }_0{p}_0(x)+{\pi }_1{p}_1(x)}.$$

Then, the Bayes-optimal test is given by

$$\begin{array}{rl}{A}^{\ast }(x)=& \underset{a}{\mathrm{argmin}}{\mathbb{E}}_{\theta }[L(a,\theta )\mid x]\\ =& \{\begin{array}{ll}1,& \text{if }p({\theta }_1\mid x)c_{\mathrm{FN}}>p({\theta }_0\mid x)c_{\mathrm{FP}}\\ 0,& \text{otherwise}.\end{array}\\ =& \{\begin{array}{ll}1,& \text{if }{p}_1(x)/p_0(x)>({\pi }_0{c}_{\mathrm{FP}})/({\pi }_1{c}_{\mathrm{FN}})\\ 0,& \text{otherwise}.\end{array}\end{array}$$