Sampling Distributions, Standard Errors and Order Statistics
CSIR NET statistics: chi-square, t and F laws, normal sampling theory, standard errors, CLT and delta method, sample quantiles and order statistics.
- 19 min
On this page · 14 sections
Prerequisites
- Standard Discrete and Continuous Distributions: gamma, beta, normal and exponential laws, and their MGFs.
- Descriptive Statistics and Exploratory Data Analysis: sample mean, sample variance, quantiles and the median.
- Quadratic Forms: \( \sum (X_i - \bar{X})^2 \) is a quadratic form \( X^TAX \) with \( A = I - \frac{1}{n}J \), an orthogonal projection of rank \( n - 1 \). That rank is where the “\( n - 1 \) degrees of freedom” come from.
The Hook: Why \( n - 1 \)?
Statistics and Standard Errors
A statistic is a function of the sample that does not involve unknown parameters. Its probability distribution over repeated samples is its sampling distribution, and the standard deviation of that distribution is its standard error (SE). When the SE depends on unknown parameters, we plug in estimates and call the result the estimated standard error.
| Statistic | Standard error | Notes |
|---|---|---|
| Sample mean \( \bar{X} \) | \( \frac{\sigma}{\sqrt{n}} \) | exact, any population with finite variance |
| \( \bar{X} \), sampling without replacement from \( N \) | \( \frac{\sigma}{\sqrt{n}}\sqrt{\frac{N - n}{N - 1}} \) | finite population correction |
| Sample proportion \( \hat{p} \) | \( \sqrt{\frac{pq}{n}} \) | exact |
| \( \bar{X}_1 - \bar{X}_2 \), independent samples | \( \sqrt{\frac{\sigma_1^2}{n_1} + \frac{\sigma_2^2}{n_2}} \) | exact |
| \( \hat{p}_1 - \hat{p}_2 \) | \( \sqrt{\frac{p_1q_1}{n_1} + \frac{p_2q_2}{n_2}} \) | exact |
| Sample variance \( S^2 \) (normal population) | \( \sigma^2\sqrt{\frac{2}{n - 1}} \) | exact; \( \approx \sigma^2\sqrt{2/n} \) |
| Sample s.d. \( S \) (normal, large \( n \)) | \( \approx \frac{\sigma}{\sqrt{2n}} \) | delta method |
| Sample median (normal, large \( n \)) | \( \approx \sigma\sqrt{\frac{\pi}{2n}} \approx \frac{1.2533\sigma}{\sqrt{n}} \) | median is less precise than the mean |
| Correlation \( r \) (bivariate normal, large \( n \)) | \( \approx \frac{1 - \rho^2}{\sqrt{n}} \) | use Fisher’s \( z \) for tests |
The SE of the mean falls like \( \frac{1}{\sqrt{n}} \): to halve it you need four times the sample.
The Three Sampling Distributions
Chi-square
If \( Z_1, \dots, Z_n \) are i.i.d. \( N(0, 1) \), then \( \chi^2 = Z_1^2 + \dots + Z_n^2 \) has the \( \chi^2_n \) distribution, which is Gamma\( \left(\frac{n}{2}, \frac{1}{2}\right) \):
$$f(x) = \frac{1}{2^{n/2}\Gamma(n/2)}x^{n/2 - 1}e^{-x/2}, \quad x > 0.$$Mean \( n \), variance \( 2n \), MGF \( (1 - 2t)^{-n/2} \) for \( t < \frac{1}{2} \), mode \( n - 2 \) for \( n \ge 2 \).
- Additive: independent \( \chi^2_m + \chi^2_n = \chi^2_{m+n} \).
- \( \chi^2_2 \) is exponential with mean 2.
- For large \( n \), \( \frac{\chi^2_n - n}{\sqrt{2n}} \to N(0, 1) \) (CLT); Fisher’s approximation treats \( \sqrt{2\chi^2_n} \) as roughly \( N(\sqrt{2n - 1}, 1) \).
Student’s \( t \)
- Symmetric about 0; heavier tails than \( N(0, 1) \).
- Mean 0 for \( n > 1 \); variance \( \frac{n}{n - 2} \) for \( n > 2 \). Moments of order \( k \) exist only for \( k < n \).
- \( t_1 \) is the standard Cauchy distribution.
- \( t_n \to N(0, 1) \) in distribution as \( n \to \infty \).
Snedecor’s \( F \)
- Mean \( \frac{n}{n - 2} \) for \( n > 2 \): it depends on the denominator degrees of freedom only.
- \( \frac{1}{F} \sim F_{n,m} \). Hence the lower percentage point: \( F_{m,n;\,1-\alpha} = \frac{1}{F_{n,m;\,\alpha}} \).
- \( T \sim t_n \Rightarrow T^2 \sim F_{1,n} \).
- If \( F \sim F_{m,n} \), then \( \frac{(m/n)F}{1 + (m/n)F} \sim \) Beta\( \left(\frac{m}{2}, \frac{n}{2}\right) \).
graph LR
Z["N(0,1)"] -->|"sum of n squares"| C["Chi-square(n)"]
Z -->|"Z / sqrt(V/n)"| T["t(n)"]
C -->|"(U/m) / (V/n)"| F["F(m, n)"]
T -->|"square"| F1["F(1, n)"]
T -->|"n = 1"| Ca["Cauchy"]
T -->|"n to infinity"| ZSampling From a Normal Population
Let \( X_1, \dots, X_n \) be i.i.d. \( N(\mu, \sigma^2) \), \( \bar{X} = \frac{1}{n}\sum X_i \), \( S^2 = \frac{1}{n - 1}\sum (X_i - \bar{X})^2 \). Then
- \( \bar{X} \sim N\left(\mu, \frac{\sigma^2}{n}\right) \);
- \( \frac{(n - 1)S^2}{\sigma^2} \sim \chi^2_{n-1} \);
- \( \bar{X} \) and \( S^2 \) are independent;
- \( \frac{\sqrt{n}(\bar{X} - \mu)}{S} \sim t_{n-1} \).
Proof sketch. Let \( P \) be an orthogonal matrix whose first row is \( \frac{1}{\sqrt{n}}(1, \dots, 1) \), and \( Y = PX \). Then \( Y_1 = \sqrt{n}\bar{X} \), the \( Y_i \) are independent normals with variance \( \sigma^2 \) (orthogonal maps preserve i.i.d. normality), \( Y_2, \dots, Y_n \) have mean 0, and \( \sum_{i \ge 2} Y_i^2 = \sum X_i^2 - n\bar{X}^2 = (n - 1)S^2 \). Statement 4 follows from 1–3 and the definition of \( t \).
Consequences:
- \( E(S^2) = \sigma^2 \), \( \operatorname{Var}(S^2) = \frac{2\sigma^4}{n - 1} \).
- \( \sum (X_i - \mu)^2/\sigma^2 \sim \chi^2_n \) (known mean: no degree of freedom lost).
- Independence of \( \bar{X} \) and \( S^2 \) characterises the normal distribution: for any other non-degenerate population they are dependent.
- \( \bar{X} \) is independent of each deviation \( X_i - \bar{X} \) (jointly normal, covariance \( \frac{\sigma^2}{n} - \frac{\sigma^2}{n} = 0 \)).
Two samples. Independent samples of sizes \( n_1, n_2 \) from \( N(\mu_1, \sigma_1^2) \) and \( N(\mu_2, \sigma_2^2) \):
$$\frac{S_1^2/\sigma_1^2}{S_2^2/\sigma_2^2} \sim F_{n_1 - 1,\, n_2 - 1}.$$If \( \sigma_1 = \sigma_2 = \sigma \), with the pooled variance \( S_p^2 = \frac{(n_1 - 1)S_1^2 + (n_2 - 1)S_2^2}{n_1 + n_2 - 2} \):
$$\frac{(\bar{X}_1 - \bar{X}_2) - (\mu_1 - \mu_2)}{S_p\sqrt{\frac{1}{n_1} + \frac{1}{n_2}}} \sim t_{n_1 + n_2 - 2}.$$Asymptotics
Central limit theorem and Slutsky
CLT (i.i.d., finite variance). \( \frac{\sqrt{n}(\bar{X} - \mu)}{\sigma} \xrightarrow{d} N(0, 1) \).
Slutsky’s theorem. If \( U_n \xrightarrow{d} U \) and \( V_n \xrightarrow{p} c \) (a constant), then \( U_n + V_n \xrightarrow{d} U + c \), \( U_nV_n \xrightarrow{d} cU \), and \( U_n/V_n \xrightarrow{d} U/c \) for \( c \ne 0 \). Since \( S \xrightarrow{p} \sigma \), the studentised mean \( \frac{\sqrt{n}(\bar{X} - \mu)}{S} \xrightarrow{d} N(0, 1) \) for any population with finite variance. That is the justification for large-sample \( z \)-tests on non-normal data.
The delta method
If \( \sqrt{n}(T_n - \theta) \xrightarrow{d} N(0, \tau^2) \) and \( g \) is differentiable at \( \theta \) with \( g'(\theta) \ne 0 \), then
$$\sqrt{n}\left(g(T_n) - g(\theta)\right) \xrightarrow{d} N\left(0, \left[g'(\theta)\right]^2\tau^2\right).$$The approximate SE of \( g(T_n) \) is \( \lvert g'(\theta) \rvert \times \) SE\( (T_n) \).
Variance-stabilising transformations. Choose \( g \) so that \( g'(\theta)^2\tau^2(\theta) \) is constant:
| Setting | Transform | Asymptotic variance of \( \sqrt{n}\,(g(T_n) - g(\theta)) \) |
|---|---|---|
| Poisson\( (\lambda) \), \( T_n = \bar{X} \) | \( \sqrt{\bar{X}} \) | \( \frac{1}{4} \) |
| Binomial proportion \( \hat{p} \) | \( \sin^{-1}\sqrt{\hat{p}} \) | \( \frac{1}{4} \) |
| Exponential mean, \( T_n = \bar{X} \) (variance \( \theta^2 \)) | \( \log \bar{X} \) | 1 |
| Correlation \( r \) (bivariate normal) | Fisher’s \( z = \tanh^{-1} r = \frac{1}{2}\log\frac{1 + r}{1 - r} \) | 1; in practice \( \operatorname{Var}(z) \approx \frac{1}{n - 3} \) |
Sample variance, any population with finite fourth moment: \( \sqrt{n}(S^2 - \sigma^2) \xrightarrow{d} N(0, \mu_4 - \sigma^4) \). For a normal population \( \mu_4 = 3\sigma^4 \), giving variance \( 2\sigma^4 \), which matches the exact \( \frac{2\sigma^4}{n - 1} \) for large \( n \).
Sample quantiles
Let \( F \) have density \( f \), continuous and positive at the \( p \)-th quantile \( \xi_p \). The sample \( p \)-th quantile \( \hat{\xi}_p \) satisfies
$$\sqrt{n}\left(\hat{\xi}_p - \xi_p\right) \xrightarrow{d} N\left(0, \frac{p(1 - p)}{f(\xi_p)^2}\right).$$For the median (\( p = \frac{1}{2} \)): asymptotic variance \( \frac{1}{4f(m)^2} \).
For \( N(\mu, \sigma^2) \), \( f(\mu) = \frac{1}{\sigma\sqrt{2\pi}} \), so \( \frac{1}{4f(\mu)^2} = \frac{\pi\sigma^2}{2} \): the median’s variance is about \( \frac{\pi}{2} \approx 1.57 \) times the mean’s. For the Laplace (double exponential) distribution the comparison reverses: the median beats the mean. For the Cauchy, the mean is useless and the median is \( \sqrt{n} \)-consistent with variance \( \frac{\pi^2}{4n} \).
Order Statistics
Let \( X_1, \dots, X_n \) be i.i.d. with continuous CDF \( F \) and density \( f \). Sorting gives the order statistics \( X_{(1)} < X_{(2)} < \dots < X_{(n)} \) (ties have probability 0).
| Quantity | Formula |
|---|---|
| CDF of the maximum | \( F_{(n)}(x) = F(x)^n \) |
| CDF of the minimum | \( F_{(1)}(x) = 1 - (1 - F(x))^n \) |
| Density of \( X_{(r)} \) | \( \frac{n!}{(r - 1)!\,(n - r)!}F(x)^{r-1}\left(1 - F(x)\right)^{n-r}f(x) \) |
| CDF of \( X_{(r)} \) | \( \sum_{k=r}^n \binom{n}{k}F(x)^k(1 - F(x))^{n-k} \) |
| Joint density of \( X_{(r)}, X_{(s)} \), \( r < s \), at \( x < y \) | \( \frac{n!}{(r-1)!(s-r-1)!(n-s)!}F(x)^{r-1}[F(y) - F(x)]^{s-r-1}[1 - F(y)]^{n-s}f(x)f(y) \) |
| Joint density of min and max | \( n(n - 1)[F(y) - F(x)]^{n-2}f(x)f(y) \), \( x < y \) |
| Joint density of all \( n \) | \( n!\,f(x_1)\cdots f(x_n) \), \( x_1 < \dots < x_n \) |
Reading the density of \( X_{(r)} \). Of the \( n \) observations, \( r - 1 \) fall below \( x \), one falls at \( x \), and \( n - r \) fall above. The multinomial coefficient counts the ways to choose which. With this picture you never need to memorise the formula.
The uniform case
For \( U(0, 1) \): \( X_{(r)} \sim \) Beta\( (r, n - r + 1) \), so
$$E\,X_{(r)} = \frac{r}{n + 1}, \qquad \operatorname{Var}X_{(r)} = \frac{r(n - r + 1)}{(n + 1)^2(n + 2)}.$$The \( n \) points cut \( (0, 1) \) into \( n + 1 \) gaps of equal expected length \( \frac{1}{n + 1} \). The range \( R = X_{(n)} - X_{(1)} \sim \) Beta\( (n - 1, 2) \), with \( E\,R = \frac{n - 1}{n + 1} \). For \( U(0, \theta) \), multiply by \( \theta \): \( E\,X_{(n)} = \frac{n\theta}{n + 1} \).
By the probability integral transform, \( F(X_{(r)}) \) is the \( r \)-th uniform order statistic for any continuous \( F \). This is why order-statistic results are distribution-free.
The exponential case
For i.i.d. Exponential\( (\lambda) \):
- \( X_{(1)} \sim \) Exponential\( (n\lambda) \).
- Rényi’s representation: the normalised spacings \( (n - i + 1)\left(X_{(i)} - X_{(i-1)}\right) \), \( i = 1, \dots, n \) (with \( X_{(0)} = 0 \)), are i.i.d. Exponential\( (\lambda) \). After the \( i - 1 \)-th failure, \( n - i + 1 \) items remain, and by memorylessness the wait to the next failure is the minimum of \( n - i + 1 \) fresh exponentials.
- Hence \( E\,X_{(r)} = \frac{1}{\lambda}\sum_{i=1}^r \frac{1}{n - i + 1} \), and \( E\,X_{(n)} = \frac{1}{\lambda}\left(1 + \frac{1}{2} + \dots + \frac{1}{n}\right) \).
- \( X_{(1)} \) is independent of \( X_{(n)} - X_{(1)} \), but \( X_{(1)} \) and \( X_{(n)} \) themselves are dependent (\( X_{(n)} \ge X_{(1)} \)).
A distribution-free interval for the median
For a continuous population with median \( m \), each observation is below \( m \) with probability \( \frac{1}{2} \), so
$$P\left(X_{(r)} < m < X_{(s)}\right) = \sum_{k=r}^{s-1}\binom{n}{k}\left(\frac{1}{2}\right)^n.$$In particular \( P(X_{(1)} < m < X_{(n)}) = 1 - 2\left(\frac{1}{2}\right)^n = 1 - 2^{1-n} \). The same count underlies the sign test on the nonparametric page.
Memory Tricks
- “Mean n, variance 2n.” For \( \chi^2_n \). The \( F \) mean \( \frac{n}{n-2} \) uses the bottom degrees of freedom; the \( t \) variance \( \frac{n}{n-2} \) uses the only one.
- “Lose one for each mean estimated.” \( \sum (X_i - \mu)^2 \) has \( n \) degrees of freedom; \( \sum (X_i - \bar{X})^2 \) has \( n - 1 \); a pooled two-sample variance has \( n_1 + n_2 - 2 \).
- “r over n plus one.” The expected \( r \)-th uniform order statistic. Then rescale.
- “Below, at, above.” \( r - 1 \) below, 1 at, \( n - r \) above: the density of \( X_{(r)} \) writes itself.
- Median SE = 1.2533 times mean SE for a normal sample (\( \sqrt{\pi/2} \)).
Don’t Confuse
| This | Is not this | The difference |
|---|---|---|
| \( \sum (X_i - \bar{X})^2/\sigma^2 \sim \chi^2_{n-1} \) | \( \sum (X_i - \mu)^2/\sigma^2 \sim \chi^2_n \) | estimating \( \mu \) costs one degree of freedom |
| \( \frac{\sqrt{n}(\bar{X} - \mu)}{\sigma} \sim N(0, 1) \) | \( \frac{\sqrt{n}(\bar{X} - \mu)}{S} \sim t_{n-1} \) | known vs estimated \( \sigma \) |
| standard deviation of the data | standard error of a statistic | SE \( = \sigma/\sqrt{n} \) for the mean |
| \( F_{m,n;\,1-\alpha} \) | \( F_{n,m;\,\alpha} \) | they are reciprocals: \( F_{m,n;\,1-\alpha} = 1/F_{n,m;\,\alpha} \) |
| \( X_{(1)} \sim \) Exp\( (n\lambda) \) | \( X_{(n)} \) exponential | the maximum is not exponential |
| \( \bar{X} \perp S^2 \) for normal samples | \( \bar{X} \perp S^2 \) always | independence characterises the normal |
Common Mistakes
Practice
All questions below are original practice questions written for this page. They are not from any CSIR NET paper. In Level 3, as in Part C, one or more options may be correct.
Level 1: Foundation
- What are the mean and variance of \( \chi^2_{10} \)?
- \( X_1, \dots, X_n \) are i.i.d. \( U(0, 1) \). What is \( E\,X_{(1)} \)?
- If \( T \sim t_n \), what is the distribution of \( T^2 \)?
Answers
- Mean 10, variance 20.
- \( \frac{1}{n + 1} \).
- \( F_{1,n} \).
Level 2: Part B Style
Practice Q1. \( X_1, \dots, X_{10} \) are i.i.d. \( N(\mu, \sigma^2) \) with sample variance \( S^2 \) (divisor 9). The distribution of \( \frac{9S^2}{\sigma^2} \) is:
(a) \( \chi^2_{10} \) (b) \( \chi^2_9 \) (c) \( t_9 \) (d) \( F_{9,10} \)
Answer: (b)
\( \frac{(n - 1)S^2}{\sigma^2} \sim \chi^2_{n-1} \) with \( n = 10 \).
Practice Q2. \( X_1, \dots, X_5 \) are i.i.d. \( U(0, 1) \). Then \( E\,X_{(2)} \) equals:
(a) \( \frac{2}{5} \) (b) \( \frac{1}{3} \) (c) \( \frac{1}{4} \) (d) \( \frac{1}{2} \)
Answer: (b)
\( X_{(2)} \sim \) Beta\( (2, 4) \), mean \( \frac{2}{6} = \frac{1}{3} \). Option (a) is the tempting \( \frac{r}{n} \).
Practice Q3. For a random sample of size 11 from \( N(\mu, 4) \), \( \operatorname{Var}(S^2) \) (divisor 10) is:
(a) 0.8 (b) 1.6 (c) 3.2 (d) 2.9
Answer: (c)
\( \operatorname{Var}(S^2) = \frac{2\sigma^4}{n - 1} = \frac{2 \times 16}{10} = 3.2 \). Option (d) uses \( n = 11 \) in the denominator.
Practice Q4. \( X_1, \dots, X_n \) are i.i.d. Poisson\( (\lambda) \). The asymptotic variance of \( \sqrt{n}\left(\sqrt{\bar{X}} - \sqrt{\lambda}\right) \) is:
(a) \( \lambda \) (b) \( \frac{1}{2} \) (c) \( \frac{1}{4} \) (d) \( \frac{1}{4\lambda} \)
Answer: (c)
Delta method with \( g(x) = \sqrt{x} \), \( g'(\lambda) = \frac{1}{2\sqrt{\lambda}} \), \( \tau^2 = \lambda \): \( \frac{1}{4\lambda} \cdot \lambda = \frac{1}{4} \). The square root stabilises the Poisson variance.
Level 3: Part C Style
Practice Q5. \( X_1, \dots, X_n \) (\( n \ge 2 \)) are i.i.d. \( N(\mu, \sigma^2) \). Which statements are true?
(a) \( \bar{X} \) and \( S^2 \) are independent. (b) \( \bar{X} \) and \( X_1 - \bar{X} \) are independent. (c) \( \sum_{i=1}^n (X_i - \mu)^2/\sigma^2 \sim \chi^2_{n-1} \). (d) \( \frac{\sqrt{n}(\bar{X} - \mu)}{S} \sim t_{n-1} \).
Answer: (a), (b) and (d)
- (a) True. The normal sampling theorem.
- (b) True. \( (\bar{X}, X_1 - \bar{X}) \) is jointly normal and \( \operatorname{Cov}(\bar{X}, X_1 - \bar{X}) = \frac{\sigma^2}{n} - \frac{\sigma^2}{n} = 0 \).
- (c) False. With the true mean \( \mu \), the sum of \( n \) independent squared standard normals is \( \chi^2_n \).
- (d) True. \( N(0, 1) \) over \( \sqrt{\chi^2_{n-1}/(n-1)} \), independent.
Practice Q6. \( X_1, \dots, X_n \) (\( n \ge 2 \)) are i.i.d. exponential with rate 1. Which statements are true?
(a) \( X_{(1)} \) is exponential with rate \( n \). (b) \( E\,X_{(n)} = \sum_{k=1}^n \frac{1}{k} \). (c) \( X_{(2)} - X_{(1)} \) is exponential with rate \( n - 1 \). (d) \( X_{(1)} \) and \( X_{(n)} \) are independent.
Answer: (a), (b) and (c)
- (a) True. \( P(X_{(1)} > x) = e^{-nx} \).
- (b) True. Sum of the expected spacings \( \frac{1}{n} + \frac{1}{n-1} + \dots + 1 \).
- (c) True. Rényi: \( (n - 1)(X_{(2)} - X_{(1)}) \sim \) Exp(1).
- (d) False. \( X_{(n)} \ge X_{(1)} \); for example \( P(X_{(1)} > 1, X_{(n)} < 1) = 0 \ne P(X_{(1)} > 1)P(X_{(n)} < 1) \). It is \( X_{(n)} - X_{(1)} \) that is independent of \( X_{(1)} \).
Practice Q7. A random sample of 100 observations from a normal population gives a sample standard deviation of 15. Which statements are true (large-sample approximations where needed)?
(a) The estimated standard error of the sample mean is 1.5. (b) The approximate standard error of the sample median is about 1.88. (c) The approximate standard error of the sample standard deviation is about 1.06. (d) A sample of 200 would halve the standard error of the mean.
Answer: (a), (b) and (c)
- (a) True. \( \frac{15}{\sqrt{100}} = 1.5 \).
- (b) True. \( 1.2533 \times 1.5 \approx 1.88 \).
- (c) True. \( \frac{15}{\sqrt{200}} \approx 1.06 \).
- (d) False. Halving the SE needs \( n = 400 \).
Practice Q8. \( X_1, \dots, X_4 \) are i.i.d. \( U(0, 1) \) with median \( \frac{1}{2} \). Which statements are true?
(a) \( P\left(X_{(1)} < \frac{1}{2} < X_{(4)}\right) = \frac{7}{8} \). (b) \( E\left(X_{(4)} - X_{(1)}\right) = \frac{3}{5} \). (c) \( X_{(2)} \sim \) Beta\( (2, 3) \). (d) \( E\,X_{(4)} = \frac{3}{4} \).
Answer: (a), (b) and (c)
- (a) True. \( 1 - 2^{1-4} = 1 - \frac{1}{8} \).
- (b) True. Range \( \sim \) Beta\( (3, 2) \), mean \( \frac{n-1}{n+1} = \frac{3}{5} \).
- (c) True. Beta\( (r, n - r + 1) = \) Beta\( (2, 3) \).
- (d) False. \( E\,X_{(4)} = \frac{4}{5} \).
Quick Revision Table
| Item | Fact |
|---|---|
| \( \chi^2_n \) | Gamma\( (\frac{n}{2}, \frac{1}{2}) \); mean \( n \), variance \( 2n \); additive |
| \( t_n \) | \( Z/\sqrt{V/n} \); variance \( \frac{n}{n-2} \); \( t_1 = \) Cauchy; \( t_n^2 = F_{1,n} \) |
| \( F_{m,n} \) | \( \frac{U/m}{V/n} \); mean \( \frac{n}{n-2} \); \( 1/F \sim F_{n,m} \) |
| Normal sample | \( \bar{X} \sim N(\mu, \sigma^2/n) \), \( (n-1)S^2/\sigma^2 \sim \chi^2_{n-1} \), independent |
| \( \operatorname{Var}(S^2) \) (normal) | \( \frac{2\sigma^4}{n-1} \) |
| SE of mean / proportion | \( \sigma/\sqrt{n} \), \( \sqrt{pq/n} \) |
| SE of median (normal) | \( \approx 1.2533\,\sigma/\sqrt{n} \) |
| Delta method | variance \( \times\, g'(\theta)^2 \) |
| Sample quantile | \( \frac{p(1-p)}{n f(\xi_p)^2} \) |
| \( X_{(r)} \) density | \( \frac{n!}{(r-1)!(n-r)!}F^{r-1}(1-F)^{n-r}f \) |
| Uniform \( X_{(r)} \) | Beta\( (r, n-r+1) \), mean \( \frac{r}{n+1} \) |
| Exponential spacings | \( (n-i+1)(X_{(i)} - X_{(i-1)}) \) i.i.d. Exp\( (\lambda) \) |
Key Takeaways
What’s Next
- Point Estimation and Confidence Intervals: these sampling distributions become pivots for interval estimates.
- Hypothesis Testing: MP, UMP and Likelihood Ratio Tests: the \( t \), \( \chi^2 \) and \( F \) tests as likelihood ratio tests.
- Inner Product Spaces and Orthonormal Bases: the orthogonal rotation behind the normal sampling theorem.
- Back to the chapter overview.