Likelihood Hypothesis Test

Rejection Region

We can also construct a rejection region using the Likelihood ratio:

$$\Lambda (\mathbf{x})=\frac{{\sup }_{\theta \in {\Theta }_0}L(\theta \mid \mathbf{x})}{{\sup }_{\theta \in \Theta }L(\theta \mid \mathbf{x})}=\frac{L({\hat{\theta }}_0)}{L({\hat{\theta }}_{\mathrm{MLE}})},$$

where ${\Theta }_0$ is the null hypothesis parameter space. To highlight the role of the alternative, we can also constrain the supremum to the alternative parameter space ${\Theta }_1$ in the denominator. By definition, the maximizers are called constrained MLEs.

Then, the rejection region is given by

$$\mathrm{RR}=\{\mathbf{x}\mid \Lambda (\mathbf{x})\le k\},$$

where $k$ is chosen such that the test has a significance level $\alpha$.

This method is called the likelihood ratio test (LRT).

We can see that LRT is closely related to Wald Test with MLE: Wald statistic measures the closeness of the MLE to the null value (x-axis), while LRT measures the closeness of their likelihoods (y-axis). Under certain regularity conditions, the two measures are equivalent.

Asymptotic LRT

Under For Maximum Likelihood Estimation, as $n\to \infty$, we have

$$-2\log \Lambda (X)=:G^2\stackrel{d}{\to }{𝜒}_k^2,$$

where the degrees of freedom $k=\dim (\Theta )–\dim ({\Theta }_0)$. This is the Wilks' theorem.

For a simple null hypothesis, $\dim ({\Theta }_0)=0$, and the result can be derived using the fact about the unconstrained MLE:

$$\begin{array}{rl}-2\log \Lambda (X)=& 2\left(\ell (\hat{\theta })-\ell ({\theta }_0)\right)\\ \approx & 2\cdot \left(-\frac{1}{2}{\left(\hat{\theta }-{\theta }_0\right)}^T{\ddot{\ell }}_{{\theta }_0}\left(\hat{\theta }-{\theta }_0\right)\right)\\ =& {\left(\hat{\theta }-\theta \right)}^{T}I({\theta }_0)\left(\hat{\theta }-{\theta }_0\right)\\ \to & {\chi }_k^2,\end{array}$$

where $\ell$ is the log-likelihood, the approximation follows the Proof Sketch of the asymptotic normality of M-Estimators.

• 💡 LRT is useful to find a convenient test statistic. For instance, we can transform the inequality $\Lambda \le$ into an inequality in $-2\log \Lambda$, which is a convenient test statistic because of the theorem of asymptotic LRT.