Hypothesis Testing: MP, UMP and Likelihood Ratio Tests
CSIR NET statistics: errors, size and power, Neyman–Pearson lemma, randomised MP tests, MLR and Karlin–Rubin UMP tests, likelihood ratio tests, Wilks.
- 19 min
On this page · 13 sections
Prerequisites
- Point Estimation and Confidence Intervals: likelihoods, MLEs, sufficient statistics and exponential families.
- Sampling Distributions, Standard Errors and Order Statistics: the \( t \), \( \chi^2 \) and \( F \) laws that give critical values.
- Standard Discrete and Continuous Distributions: tail probabilities of the Poisson, binomial and exponential.
The Hook: A Fixed Budget for False Alarms
The Framework
The parameter space \( \Theta \) is split into \( \Theta_0 \) (null hypothesis \( H_0: \theta \in \Theta_0 \)) and \( \Theta_1 \) (alternative \( H_1: \theta \in \Theta_1 \)). A hypothesis is simple if it specifies the distribution completely (\( \theta = \theta_0 \)) and composite otherwise.
A test is a function \( \phi(x) \in [0, 1] \): the probability of rejecting \( H_0 \) when the data are \( x \). A non-randomised test takes only the values 0 and 1; \( \{x : \phi(x) = 1\} \) is the critical region.
| \( H_0 \) true | \( H_0 \) false | |
|---|---|---|
| reject \( H_0 \) | Type I error (probability \( \alpha \)) | correct (power \( 1 - \beta \)) |
| do not reject | correct | Type II error (probability \( \beta \)) |
- Power function: \( \beta_\phi(\theta) = E_\theta\,\phi(X) = P_\theta(\text{reject } H_0) \).
- Size: \( \sup_{\theta \in \Theta_0}\beta_\phi(\theta) \). A test is of level \( \alpha \) if its size is \( \le \alpha \).
- Unbiased test: \( \beta_\phi(\theta) \ge \alpha \) for every \( \theta \in \Theta_1 \) (you are never less likely to reject when \( H_0 \) is false than when it is true).
- p-value: the smallest level at which the observed data would lead to rejection; for a test rejecting large values of \( T \), \( p = \sup_{\theta \in \Theta_0}P_\theta(T \ge t_{\text{obs}}) \). Reject at level \( \alpha \) iff \( p \le \alpha \).
The Neyman–Pearson Lemma
For testing a simple \( H_0: \theta = \theta_0 \) against a simple \( H_1: \theta = \theta_1 \) with densities \( f_0, f_1 \), the test
$$\phi(x) = \begin{cases} 1 & f_1(x) > k f_0(x) \\ \gamma & f_1(x) = k f_0(x) \\ 0 & f_1(x) < k f_0(x) \end{cases}$$with \( k \ge 0 \) and \( \gamma \in [0, 1] \) chosen so that \( E_{\theta_0}\phi(X) = \alpha \), is most powerful (MP) among all tests of level \( \alpha \). Such a test exists for every \( \alpha \in (0, 1) \), and any MP test of size \( \alpha \) has this form (except on a set of probability zero under both hypotheses).
Two consequences:
- The MP test depends on the data only through the likelihood ratio \( \frac{f_1(x)}{f_0(x)} \), so only through a sufficient statistic.
- An MP test is unbiased: its power is at least \( \alpha \) (compare with the trivial test \( \phi \equiv \alpha \)), and strictly greater unless \( f_0 = f_1 \).
The method in three steps
- Write the ratio \( \frac{L(\theta_1)}{L(\theta_0)} \) and simplify.
- Show it is a monotone function of a statistic \( T \); then “ratio \( > k \)” is “\( T > c \)” (or \( T < c \)).
- Choose \( c \) (and \( \gamma \) if \( T \) is discrete) from the size condition under \( H_0 \).
Randomisation for discrete data
With a discrete statistic, “reject when \( T > c \)” can only reach certain sizes. To get exact size \( \alpha \), reject outright above \( c \) and with probability \( \gamma \) at \( T = c \):
$$P_{\theta_0}(T > c) + \gamma\,P_{\theta_0}(T = c) = \alpha.$$Uniformly Most Powerful Tests
A level-\( \alpha \) test is UMP for \( H_0 \) vs a composite \( H_1 \) if it is MP against every \( \theta_1 \in \Theta_1 \) simultaneously. The key is that the MP test from the lemma often does not depend on which \( \theta_1 \) you picked.
A one-parameter exponential family \( f(x; \theta) = h(x)\exp\left(\eta(\theta)T(x) - A(\theta)\right) \) has MLR in \( T \) when \( \eta \) is increasing (in \( -T \) when \( \eta \) is decreasing). \( U(0, \theta) \) has MLR in \( X_{(n)} \). The Cauchy location family does not have MLR.
| Model | \( H_1 \) | UMP test rejects when |
|---|---|---|
| \( N(\theta, \sigma^2) \), \( \sigma \) known | \( \theta > \theta_0 \) | \( \bar{X} > \theta_0 + z_\alpha\sigma/\sqrt{n} \) |
| \( N(\mu, \sigma^2) \), \( \mu \) known | \( \sigma^2 > \sigma_0^2 \) | \( \sum (X_i - \mu)^2 > \sigma_0^2\chi^2_{n,\alpha} \) |
| Bernoulli\( (p) \) | \( p > p_0 \) | \( \sum X_i \) large (randomised) |
| Poisson\( (\lambda) \) | \( \lambda > \lambda_0 \) | \( \sum X_i \) large (randomised) |
| Exponential, mean \( \theta \) | \( \theta > \theta_0 \) | \( \sum X_i > \frac{\theta_0}{2}\chi^2_{2n,\alpha} \) |
| \( U(0, \theta) \) | \( \theta > \theta_0 \) | \( X_{(n)} > \theta_0(1 - \alpha)^{1/n} \) |
When no UMP test exists
For \( N(\theta, 1) \) and \( H_0: \theta = \theta_0 \) vs \( H_1: \theta \ne \theta_0 \): the MP test against \( \theta_1 > \theta_0 \) rejects for large \( \bar{X} \), the MP test against \( \theta_1 < \theta_0 \) rejects for small \( \bar{X} \). By the uniqueness part of the lemma, no single test is MP against both, so no UMP test exists. The usual two-sided test \( \lvert \bar{X} - \theta_0 \rvert > z_{\alpha/2}/\sqrt{n} \) is UMP among unbiased tests (UMPU), a weaker optimality.
Likelihood Ratio Tests
When hypotheses are composite and no UMP test exists, use the generalised likelihood ratio
$$\Lambda(x) = \frac{\sup_{\theta \in \Theta_0}L(\theta; x)}{\sup_{\theta \in \Theta}L(\theta; x)} \in [0, 1],$$and reject \( H_0 \) when \( \Lambda \le c \). The numerator uses the MLE under \( H_0 \), the denominator the unrestricted MLE.
Under regularity conditions, if \( H_0 \) is true and \( \Theta_0 \) is a smooth subset of the open set \( \Theta \subseteq \mathbb{R}^k \) of dimension \( k - r \), then
$$-2\log\Lambda \xrightarrow{d} \chi^2_r, \qquad r = \dim\Theta - \dim\Theta_0 = \text{number of restrictions imposed by } H_0.$$| Problem (normal populations) | LRT reduces to |
|---|---|
| \( \mu = \mu_0 \), \( \sigma \) known | \( \lvert z \rvert \) large; \( -2\log\Lambda = n(\bar{x} - \mu_0)^2/\sigma^2 \sim \chi^2_1 \) exactly |
| \( \mu = \mu_0 \), \( \sigma \) unknown | \( \lvert t \rvert \) large, \( t_{n-1} \) |
| \( \sigma^2 = \sigma_0^2 \), \( \mu \) unknown | \( \frac{(n-1)S^2}{\sigma_0^2} \) too small or too large, \( \chi^2_{n-1} \) |
| \( \mu_1 = \mu_2 \), common unknown \( \sigma \) | pooled two-sample \( t \), \( t_{n_1+n_2-2} \) |
| \( \sigma_1^2 = \sigma_2^2 \) | \( F = S_1^2/S_2^2 \) too small or too large, \( F_{n_1-1,n_2-1} \) |
| \( k \) means equal, common \( \sigma \) | one-way ANOVA \( F_{k-1,\,N-k} \) |
The LRT is not guaranteed to be optimal, but it is always available, it is invariant under reparametrisation, and it reproduces the classical tests. The Wald test (based on \( \hat{\theta} \)) and the score test (based on the slope of \( \ell \) at \( \theta_0 \)) have the same \( \chi^2_r \) limit. Pearson’s chi-square statistic on the next page is asymptotically equivalent to \( -2\log\Lambda \) for multinomial data.
graph TD
A["Testing problem"] --> B{"Both hypotheses simple?"}
B -->|yes| C["Neyman-Pearson: reject when f1/f0 large"]
B -->|no| D{"One-sided, family with MLR?"}
D -->|yes| E["Karlin-Rubin UMP: reject when T large (or small)"]
D -->|no| F{"Two-sided?"}
F -->|"normal-type family"| G["No UMP; UMPU two-sided test"]
F -->|"general"| H["Likelihood ratio test; -2 log Lambda ~ chi-square(r)"]Memory Tricks
- “Ratio big, reject.” Every MP test rejects when \( f_1/f_0 \) is large; the rest is algebra to find which statistic that means.
- “One side, one winner.” One-sided alternatives in MLR families have UMP tests; two-sided ones usually do not (uniform is the exception).
- “Count the restrictions.” Wilks’ degrees of freedom = free parameters under \( H_1 \cup H_0 \) minus free parameters under \( H_0 \).
- “Discrete needs a coin.” Exact size with a Poisson or binomial statistic needs randomisation at the boundary value.
- “Size from H0, power from H1.” Compute the critical value under \( \theta_0 \); compute power by plugging the alternative into the same region.
Don’t Confuse
| This | Is not this | The difference |
|---|---|---|
| size \( \alpha \) | p-value | \( \alpha \) is fixed beforehand; the p-value is computed from data |
| level \( \alpha \) (size \( \le \alpha \)) | size exactly \( \alpha \) | a conservative test has size below its level |
| MP (simple vs simple) | UMP (composite \( H_1 \)) | UMP is MP against every alternative at once |
| UMP | UMPU | UMPU is best only among unbiased tests |
| Type II error \( \beta \) | \( 1 - \alpha \) | \( \beta \) is computed under \( H_1 \); it is not determined by \( \alpha \) alone |
| \( -2\log\Lambda \sim \chi^2_r \) (asymptotic) | exact \( \chi^2_r \) | exact only in special cases (normal mean, known \( \sigma \)) |
| “fail to reject \( H_0 \)” | “\( H_0 \) is true” | non-rejection is lack of evidence, not proof |
Common Mistakes
Practice
All questions below are original practice questions written for this page. They are not from any CSIR NET paper. In Level 3, as in Part C, one or more options may be correct.
Level 1: Foundation
- Define the Type I error of a test.
- What does the Neyman–Pearson lemma say the most powerful test depends on?
- In Wilks’ theorem, how many degrees of freedom does \( -2\log\Lambda \) have if \( H_0 \) fixes one of three free parameters?
Answers
- Rejecting \( H_0 \) when it is true.
- The likelihood ratio \( f_1(x)/f_0(x) \): reject when it exceeds a constant.
- One.
Level 2: Part B Style
Practice Q1. \( X_1, \dots, X_{25} \) are i.i.d. \( N(\theta, 1) \). The most powerful test of size 0.05 for \( H_0: \theta = 0 \) vs \( H_1: \theta = 1 \) rejects \( H_0 \) when \( \bar{X} > c \), where \( c \) is about (use \( z_{0.05} = 1.645 \)):
(a) 0.066 (b) 0.329 (c) 0.392 (d) 1.645
Answer: (b)
\( c = 0 + 1.645 \times \frac{1}{\sqrt{25}} = 0.329 \). Option (c) uses 1.96 (a two-sided value); (d) forgets the \( \sqrt{n} \).
Practice Q2. A coin is tossed 10 times to test \( H_0: p = \frac{1}{2} \) against \( H_1: p > \frac{1}{2} \), rejecting when the number of heads is at least 9. The size of the test is:
(a) \( \frac{1}{1024} \) (b) \( \frac{10}{1024} \) (c) \( \frac{11}{1024} \) (d) \( \frac{56}{1024} \)
Answer: (c)
\( P(X \ge 9) = \frac{\binom{10}{9} + \binom{10}{10}}{2^{10}} = \frac{11}{1024} \approx 0.0107 \). Option (d) is \( P(X \ge 8) \).
Practice Q3. One observation \( X \) from \( U(0, \theta) \). To test \( H_0: \theta = 1 \) vs \( H_1: \theta = 2 \), the test rejects when \( X > 0.9 \). Its size and power are:
(a) 0.1 and 0.45 (b) 0.1 and 0.55 (c) 0.9 and 0.55 (d) 0.05 and 0.55
Answer: (b)
Size \( P_1(X > 0.9) = 0.1 \). Power \( P_2(X > 0.9) = \frac{1.1}{2} = 0.55 \). Option (a) gives the Type II error.
Practice Q4. \( X_1, \dots, X_n \) are i.i.d. \( N(\mu, \sigma^2) \). For \( H_0: \mu = 0, \sigma^2 = 1 \) against an unrestricted alternative, Wilks’ theorem gives the asymptotic null distribution of \( -2\log\Lambda \) as:
(a) \( \chi^2_1 \) (b) \( \chi^2_2 \) (c) \( \chi^2_{n-1} \) (d) \( t_{n-1} \)
Answer: (b)
\( \dim\Theta = 2 \), \( \dim\Theta_0 = 0 \): two restrictions.
Level 3: Part C Style
Practice Q5. \( X_1, \dots, X_n \) are i.i.d. \( N(\theta, 1) \). Which statements are true?
(a) A UMP level-\( \alpha \) test exists for \( H_0: \theta \le 0 \) vs \( H_1: \theta > 0 \). (b) A UMP level-\( \alpha \) test exists for \( H_0: \theta = 0 \) vs \( H_1: \theta \ne 0 \). (c) The MP size-\( \alpha \) test of \( \theta = 0 \) vs \( \theta = 1 \) is also MP for \( \theta = 0 \) vs \( \theta = 2 \). (d) For fixed \( \theta > 0 \) and \( \alpha \), the power of the UMP test in (a) increases with \( n \).
Answer: (a), (c) and (d)
- (a) True. MLR in \( \bar{X} \); Karlin–Rubin.
- (b) False. The MP tests against \( \theta > 0 \) and \( \theta < 0 \) reject in opposite tails.
- (c) True. Both reject when \( \bar{X} > z_\alpha/\sqrt{n} \); the region does not depend on \( \theta_1 > 0 \).
- (d) True. Power \( = 1 - \Phi(z_\alpha - \sqrt{n}\,\theta) \), increasing in \( n \).
Practice Q6. One observation \( X \) has density \( \theta e^{-\theta x} \), \( x > 0 \). Test \( H_0: \theta = 2 \) vs \( H_1: \theta = 1 \) at size 0.05. Which statements are true?
(a) The MP test rejects when \( X > \frac{\log 20}{2} \approx 1.498 \). (b) Its power is \( \sqrt{0.05} \approx 0.224 \). (c) Its Type II error probability is 0.95. (d) The same test is UMP for \( H_0: \theta = 2 \) vs \( H_1: \theta < 2 \).
Answer: (a), (b) and (d)
- (a) True. \( \frac{f_1}{f_0} = \frac{e^{-x}}{2e^{-2x}} = \frac{e^x}{2} \), increasing in \( x \). Size: \( e^{-2c} = 0.05 \Rightarrow c = \frac{\log 20}{2} \).
- (b) True. \( P_1(X > c) = e^{-c} = (e^{-2c})^{1/2} = \sqrt{0.05} \).
- (c) False. \( \beta = 1 - 0.224 \approx 0.776 \).
- (d) True. For any \( \theta_1 < 2 \), \( \frac{f_{\theta_1}}{f_2} \propto e^{(2 - \theta_1)x} \) is increasing in \( x \), so the same region is MP against each.
Practice Q7. One observation \( X \sim \) Poisson\( (\lambda) \). For \( H_0: \lambda = 1 \) vs \( H_1: \lambda = 2 \) at size 0.05, which statements are true? (Use \( P_1(X \ge 3) \approx 0.0803 \), \( P_1(X \ge 4) \approx 0.0190 \), \( P_1(X = 3) \approx 0.0613 \), \( P_2(X \ge 4) \approx 0.1429 \), \( P_2(X = 3) \approx 0.1804 \).)
(a) The non-randomised test “reject iff \( X \ge 3 \)” has size larger than 0.05. (b) The MP size-0.05 test rejects when \( X \ge 4 \) and, when \( X = 3 \), rejects with probability about 0.506. (c) The power of the MP test is about 0.23. (d) The MP test rejects for small values of \( X \).
Answer: (a), (b) and (c)
- (a) True. \( 0.0803 > 0.05 \).
- (b) True. \( \gamma = \frac{0.05 - 0.0190}{0.0613} \approx 0.506 \).
- (c) True. \( 0.1429 + 0.506 \times 0.1804 \approx 0.234 \).
- (d) False. The ratio \( e^{-1}2^x \) increases with \( x \).
Practice Q8. \( X_1, \dots, X_n \) (\( n \ge 2 \)) are i.i.d. \( N(\mu, \sigma^2) \), both unknown. Which statements are true?
(a) The LRT of \( H_0: \mu = \mu_0 \) rejects for large \( \lvert t \rvert \), \( t = \sqrt{n}(\bar{X} - \mu_0)/S \). (b) For that LRT, \( -2\log\Lambda = n\log\left(1 + \frac{t^2}{n - 1}\right) \). (c) Under \( H_0 \), \( -2\log\Lambda \) is asymptotically \( \chi^2_1 \). (d) The LRT of \( H_0: \sigma^2 = \sigma_0^2 \) is based on \( \frac{(n-1)S^2}{\sigma_0^2} \), which is \( \chi^2_n \) under \( H_0 \).
Answer: (a), (b) and (c)
- (a) True and (b) True. See the worked example: \( \Lambda = (1 + t^2/(n-1))^{-n/2} \).
- (c) True. One restriction.
- (d) False. The statistic is right, but its null law is \( \chi^2_{n-1} \) (the mean is estimated).
Quick Revision Table
| Idea | Fact |
|---|---|
| Size | \( \sup_{\Theta_0}P_\theta(\text{reject}) \) |
| Power | \( P_\theta(\text{reject}) \), \( \theta \in \Theta_1 \); \( \beta = 1 - \) power |
| NP lemma | reject when \( f_1/f_0 > k \); randomise on equality |
| MP test | unbiased; depends on a sufficient statistic |
| MLR | ratio monotone in \( T \); exponential families with monotone \( \eta \) |
| Karlin–Rubin | one-sided UMP: reject for large \( T \) |
| Two-sided normal | no UMP; UMPU exists |
| \( U(0, \theta) \) two-sided | UMP exists: \( X_{(n)} > \theta_0 \) or \( X_{(n)} \le \theta_0\alpha^{1/n} \) |
| LRT | \( \Lambda = \sup_{\Theta_0}L/\sup_\Theta L \); reject small |
| Wilks | \( -2\log\Lambda \to \chi^2_r \), \( r = \) number of restrictions |
| Normal LRTs | \( t \), \( \chi^2_{n-1} \), pooled \( t \), \( F \), ANOVA |
Key Takeaways
What’s Next
- Chi-Square Goodness of Fit and Large-Sample Tests: the \( \chi^2 \) limit at work on counts, and \( z \)-tests for means and proportions.
- Point Estimation and Confidence Intervals: inverting these tests gives confidence intervals.
- Experimental Design, Sampling and Data: the Part A view of hypotheses, controls and error.
- Back to the chapter overview.