Admissibility

A statistical procedure $A$ is admissible if it is not dominated by any other procedure $A^{\prime }$. Formally, given a Risk function $R$, we say $A$ is inadmissible if there exists another procedure $A^{\prime }$ such that $R(A^{\prime },\theta )\le R(A,\theta )$ for all $\theta \in \Theta$ and there exists ${\theta }_0$ such that $R(A^{\prime },{\theta }_0)<R(A,{\theta }_0)$. We say $A$ is admissible if it is not inadmissible.

We have the following useful results regarding admissibility.

Unique Minimax Estimator is Admissible

Cor

A unique minimax estimator is admissible.

Pf

If the minimax estimator $A$ is inadmissible, then there exists $A^{\prime }$ such that $R(A^{\prime },\theta )\le R(A,\theta )$ for all $\theta$. Then,

$$R_M(A^{\prime })=\underset{\theta }{\sup }R(A^{\prime },\theta )\le \underset{\theta }{\sup }R(A,\theta )=R_M(A)=\underset{A}{\inf }{R}_M(A),$$

indicating that $A^{\prime }$ is also minimax optimal, contradicting the uniqueness of $A$.

Unique Bayes Estimator is Admissible

Thm

A unique Bayes estimator $A^{\ast }$ w.r.t prior $Q$ is admissible.

Pf

Suppose $A^{\ast }$ is inadmissible. Then there exists $A^{\prime }$ such that $R(A^{\prime },\theta )\le R(A^{\ast },\theta )$ for all $\theta$. Then,

$$R_B(A^{\prime },Q)={\int }_{Q}R(A^{\prime },\theta )\mathrm{d}\theta \le {\int }_{Q}R(A^{\ast },\theta )\mathrm{d}\theta =R_B(A^{\ast },Q).$$

By the Bayes optimality, $R_B(A^{\prime },Q)\ge R_B(A^{\ast },Q)$. Thus, $R_B(A^{\prime },Q)=R_B(A^{\ast },Q)$, which implies $A^{\ast }$ is not unique, making a contradiction.

Cor

If the unique Bayes estimator is also minimax with equal Bayes risk and minimax risk, then $A^{\ast }$ is the unique minimax estimator.

Pf

Suppose $A^{\prime }$ is another minimax estimator. By definitions of the minimax optimality and Bayes optimality, we have

$$\{\begin{array}{l}{R}_M(A^{\ast })\ge R_M(A^{\prime })\ge R_B(A^{\prime },Q)\\ \\ R_B(A^{\prime },Q)\ge R_B(A^{\ast },Q).\end{array}$$

Then, since $R_M(A^{\ast })=R_B(A^{\ast },Q)$, we have $R_B(A^{\prime },Q)=R_B(A^{\ast },Q)$, contradicting the uniqueness of the Bayes estimator.

Bayes Estimator with Strictly Positive Prior is Admissible

Thm

Suppose the risk $R$ is continuous on $\Theta$ and the prior has full support on $\Theta$. Then if the Bayes risk is finite, the Bayes estimator is admissible.

Pf

If a Bayes estimator $A^{\ast }$ is inadmissible, then there exists another procedure $A^{\prime }$ such that $R(A^{\prime },\theta )\le R(A^{\ast },\theta )$ for all $\theta \in \Theta$ and there exists ${\theta }_0$ such that $R(A^{\prime },{\theta }_0)<R(A^{\ast },{\theta }_0)$. Then,

$$\begin{array}{rl}{R}_B(A^{\prime },Q)-R_B(A^{\ast },Q)=& {\int }_{\Theta }R(A^{\prime },\theta )-R(A^{\ast },\theta )\mathrm{d}Q(\theta )\\ \le & {\int }_{B({\theta }_0;ϵ)}R(A^{\prime },\theta )-R(A^{\ast },\theta )\mathrm{d}Q(\theta )\\ \le & -c{\int }_{B({\theta }_0;ϵ)}\mathrm{d}Q(\theta )\\ <& 0,\end{array}$$

where $B({\theta }_0;ϵ)$ is a small neighborhood of ${\theta }_0$ such that $R(A^{\prime },\theta )-R(A^{\ast },\theta )<-c$ on $B({\theta }_0;ϵ)$; Such $ϵ$ and $c>0$ exist due to the continuity of $R$; and the last inequality is due to the strictly positivity of $Q$.

Sample mean

By Anderson’s Lemma, we know that the sample mean $\overline{X}$ is a Bayes Optimal Estimator w.r.t a Bowl-Shaped Loss and a Gaussian prior $Q=\mathcal{N}({\mu }_0,{\tau }^2)$ with $\tau \to \infty$. Since this prior is strictly positivity, the sample mean is admissible.