Skip to content

Simple Random, Stratified and Systematic Sampling

CSIR NET sampling theory: SRSWR and SRSWOR, finite population correction, stratified allocation (proportional, Neyman, cost-optimum) and systematic sampling.

  • 21 min read
  • Updated
On this page · 13 sections
  1. Prerequisites
  2. The Hook: One Population, Three Spreads
  3. Notation and the Finite Population Set-Up
  4. Simple Random Sampling
  5. Stratified Random Sampling
  6. Systematic Sampling
  7. Memory Tricks
  8. Don’t Confuse
  9. Common Mistakes
  10. Practice
  11. Quick Revision Table
  12. Key Takeaways
  13. What’s Next

Prerequisites

The Hook: One Population, Three Spreads

Notation and the Finite Population Set-Up

A finite population has \( N \) identifiable units with values \( Y_1, \dots, Y_N \) of the study variable. The values are fixed numbers; all randomness comes from the sampling design.

SymbolMeaning
\( Y = \sum Y_i \), \( \bar Y = \frac{Y}{N} \)population total and mean
\( \sigma^2 = \frac{1}{N}\sum (Y_i - \bar Y)^2 \)population variance with divisor \( N \)
\( S^2 = \frac{1}{N - 1}\sum (Y_i - \bar Y)^2 \)population mean square, divisor \( N - 1 \)
\( f = \frac{n}{N} \)sampling fraction
\( 1 - f = \frac{N - n}{N} \)finite population correction (fpc)
\( \bar y, \ s^2 = \frac{1}{n - 1}\sum (y_i - \bar y)^2 \)sample mean and sample mean square

The two population variances are related by \( \sigma^2 = \frac{N - 1}{N}S^2 \). Most sampling formulas look cleanest in \( S^2 \), which is why Cochran-style texts use it; check which one a question uses before you compute.

Simple Random Sampling

Without replacement (SRSWOR)

Every subset of size \( n \) has the same probability \( \frac{1}{\binom{N}{n}} \) of being the sample. Equivalently, draw units one by one, each time with equal probability from those not yet drawn.

QuantitySRSWOR value
Number of possible samples\( \binom{N}{n} \)
\( P(\text{unit } i \text{ in sample}) = \pi_i \)\( \frac{n}{N} \)
\( P(\text{units } i, j \text{ both in sample}) = \pi_{ij} \)\( \frac{n(n - 1)}{N(N - 1)} \)
\( E(\bar y) \)\( \bar Y \) (unbiased)
\( V(\bar y) \)\( \frac{N - n}{Nn}S^2 = (1 - f)\frac{S^2}{n} \)
\( E(s^2) \)\( S^2 \), so \( v(\bar y) = (1 - f)\frac{s^2}{n} \) is unbiased
Total \( \hat Y = N\bar y \)\( V(\hat Y) = N^2(1 - f)\frac{S^2}{n} \)

Derivation in one line. Write \( \bar y = \frac{1}{n}\sum_{i=1}^N a_i Y_i \) with \( a_i \) the indicator that unit \( i \) is drawn. Then \( V(a_i) = \frac{n}{N}(1 - \frac{n}{N}) \), \( \operatorname{Cov}(a_i, a_j) = \pi_{ij} - \pi_i\pi_j = -\frac{n(N - n)}{N^2(N - 1)} \), and expanding the variance of the sum and using \( \sum_{i \ne j}(Y_i - \bar Y)(Y_j - \bar Y) = -\sum (Y_i - \bar Y)^2 \) gives \( (1 - f)\frac{S^2}{n} \).

Proportions. If \( Y_i \in \{0, 1\} \) with population proportion \( P \) (and \( Q = 1 - P \)), then \( S^2 = \frac{N}{N - 1}PQ \), so

$$V(p) = \frac{N - n}{N - 1}\cdot\frac{PQ}{n}, \qquad v(p) = \frac{N - n}{N}\cdot\frac{pq}{n - 1} \ \text{(unbiased)}.$$

With replacement (SRSWR)

Each of \( n \) draws picks any unit with probability \( \frac{1}{N} \), independently. There are \( N^n \) equally likely ordered samples.

QuantitySRSWR value
\( V(\bar y) \)\( \frac{\sigma^2}{n} = \frac{N - 1}{N}\cdot\frac{S^2}{n} \)
\( E(s^2) \)\( \sigma^2 \)
\( P(\text{unit } i \text{ appears at least once}) \)\( 1 - \left(1 - \frac{1}{N}\right)^n \)
Expected number of distinct units \( E(\nu) \)\( N\left[1 - \left(1 - \frac{1}{N}\right)^n\right] \)

Comparison. For \( n > 1 \),

$$\frac{V_{\text{WR}}(\bar y)}{V_{\text{WOR}}(\bar y)} = \frac{N - 1}{N - n} > 1,$$

so SRSWOR is always more efficient; the gain is negligible when \( f \) is small, which is why the fpc is usually dropped for \( f < 0.05 \).

Worked example: enumerate everything

Population \( \{2, 4, 6, 8, 10\} \), \( N = 5 \), \( n = 2 \). Here \( \bar Y = 6 \), \( \sum(Y_i - \bar Y)^2 = 40 \), so \( S^2 = 10 \), \( \sigma^2 = 8 \).

  • SRSWOR: 10 samples with means \( 3, 4, 5, 6, 5, 6, 7, 7, 8, 9 \). Their average is 6 and their variance (divisor 10) is 3, matching \( \frac{5 - 2}{5 \cdot 2} \cdot 10 = 3 \).
  • SRSWR: 25 ordered samples; the variance of \( \bar y \) is \( \frac{\sigma^2}{n} = 4 \).
  • Ratio \( \frac{4}{3} = \frac{N - 1}{N - n} \). Check.

Choosing the sample size

To estimate \( \bar Y \) with margin \( d \) at confidence coefficient corresponding to the normal point \( z \), ignore the fpc first:

$$n_0 = \frac{z^2S^2}{d^2}, \qquad n = \frac{n_0}{1 + \frac{n_0}{N}}.$$

For a proportion, \( n_0 = \frac{z^2PQ}{d^2} \). Some books write \( 1 + \frac{n_0 - 1}{N} \) in the denominator; the two versions agree to the nearest integer except in tiny populations. Always round up.

Stratified Random Sampling

Divide the population into \( L \) non-overlapping strata of sizes \( N_1, \dots, N_L \) (\( \sum N_h = N \)), and draw an independent SRSWOR of size \( n_h \) from stratum \( h \). Write \( W_h = \frac{N_h}{N} \), \( f_h = \frac{n_h}{N_h} \), and \( \bar Y_h, S_h^2 \) for the stratum mean and mean square.

$$\bar y_{st} = \sum_{h=1}^L W_h\bar y_h, \qquad V(\bar y_{st}) = \sum_{h=1}^L W_h^2(1 - f_h)\frac{S_h^2}{n_h}.$$

\( \bar y_{st} \) is unbiased, and \( v(\bar y_{st}) = \sum W_h^2(1 - f_h)\frac{s_h^2}{n_h} \) is unbiased for its variance (needs \( n_h \ge 2 \) in every stratum). Because the strata are sampled independently, only within-stratum variation enters the variance. Stratification pays when strata are internally homogeneous and differ from one another.

The three allocations

graph TD
    A[Total sample size n fixed] --> B{What do you know?}
    B -->|Only stratum sizes N_h| C["Proportional: n_h = n N_h / N"]
    B -->|Sizes and stratum SDs S_h| D["Neyman: n_h = n N_h S_h / sum N_h S_h"]
    B -->|Sizes, SDs and unit costs c_h| E["Cost-optimum: n_h proportional to N_h S_h / sqrt c_h"]
Allocation\( n_h \)Variance of \( \bar y_{st} \)
Proportional\( n W_h \)\( V_{prop} = \frac{1 - f}{n}\sum W_hS_h^2 \)
Neyman (optimum for fixed \( n \))\( n\frac{N_hS_h}{\sum N_hS_h} \)\( V_{opt} = \frac{(\sum W_hS_h)^2}{n} - \frac{\sum W_hS_h^2}{N} \)
Optimum for cost \( C = c_0 + \sum c_hn_h \)\( \propto \frac{N_hS_h}{\sqrt{c_h}} \)minimises \( V \) for fixed cost (or cost for fixed \( V \))

Where Neyman comes from. Minimise \( \sum \frac{W_h^2S_h^2}{n_h} \) subject to \( \sum n_h = n \). By Cauchy-Schwarz, \( \left(\sum \frac{W_h^2S_h^2}{n_h}\right)\left(\sum n_h\right) \ge \left(\sum W_hS_h\right)^2 \), with equality iff \( \frac{W_hS_h}{n_h} \) is the same for all \( h \), i.e. \( n_h \propto W_hS_h \propto N_hS_h \). The same argument with \( \sum c_hn_h \) fixed gives \( n_h \propto \frac{N_hS_h}{\sqrt{c_h}} \). The rule in words: sample more where the stratum is larger, more variable, and cheaper.

For a fixed budget \( C - c_0 \) on sampling, the total size is

$$n = (C - c_0)\frac{\sum N_hS_h/\sqrt{c_h}}{\sum N_hS_h\sqrt{c_h}}.$$

Comparing the designs

Ignoring terms of order \( \frac{1}{N_h} \) (large strata),

$$V_{ran} - V_{prop} \approx \frac{1 - f}{n}\sum W_h(\bar Y_h - \bar Y)^2 \ \ge 0, \qquad V_{prop} - V_{opt} = \frac{1}{n}\sum W_h(S_h - \bar S)^2 \ \ge 0,$$

where \( \bar S = \sum W_hS_h \). So \( V_{opt} \le V_{prop} \le V_{ran} \) (the second inequality in the large-stratum approximation). Read the two gaps carefully:

  • Proportional allocation gains over SRS through differences between stratum means.
  • Neyman gains over proportional through differences between stratum standard deviations. If all \( S_h \) are equal, Neyman is proportional.

Worked example

Three strata, \( N = 1000 \), \( n = 90 \):

Stratum\( N_h \)\( W_h \)\( S_h \)\( \bar Y_h \)\( N_hS_h \)Proportional \( n_h \)Neyman \( n_h \)
12000.2155030001830
23000.3104030002730
35000.563030004530
  • \( \sum W_hS_h^2 = 0.2(225) + 0.3(100) + 0.5(36) = 93 \), so \( V_{prop} = \frac{0.91}{90} \times 93 = 0.9403 \).
  • \( \bar S = \sum W_hS_h = 3 + 3 + 3 = 9 \), so \( V_{opt} = \frac{81}{90} - \frac{93}{1000} = 0.900 - 0.093 = 0.807 \).
  • Check: \( \frac{1}{90}\sum W_h(S_h - 9)^2 = \frac{1}{90}[0.2(36) + 0.3(1) + 0.5(9)] = \frac{12}{90} = 0.1333 = 0.9403 - 0.807 \).
  • With the stratum means given, \( \bar Y = 37 \) and the population mean square is \( S^2 \approx 153.79 \), so SRSWOR of 90 has \( V_{ran} = \frac{0.91}{90} \times 153.79 \approx 1.555 \). Stratification with proportional allocation cuts the variance by about 40%, mostly because the stratum means differ.

Costs. If the costs per unit are \( c_h = 1, 4, 9 \) and the budget for sampling is 540, then \( n_h \propto \frac{3000}{1}, \frac{3000}{2}, \frac{3000}{3} \), i.e. in ratio \( 6 : 3 : 2 \); \( n = 540 \times \frac{5500}{18000} = 165 \), giving \( n_h = 90, 45, 30 \) at cost \( 90 + 180 + 270 = 540 \).

Systematic Sampling

Linear systematic sampling

Let \( N = nk \). Choose a random start \( i \) from \( 1, \dots, k \) and take units \( i, i + k, i + 2k, \dots, i + (n - 1)k \). There are only \( k \) possible samples, each with probability \( \frac{1}{k} \). Each unit has inclusion probability \( \frac{1}{k} = \frac{n}{N} \), so \( \bar y_{sy} \) is unbiased for \( \bar Y \), but two units in different systematic samples are never drawn together (\( \pi_{ij} = 0 \)).

Arrange the population as a \( k \times n \) array, row \( i \) being the \( i \)-th systematic sample. Then

$$V(\bar y_{sy}) = \frac{1}{k}\sum_{i=1}^k (\bar y_{i\cdot} - \bar Y)^2 = \frac{N - 1}{N}S^2 - \frac{k(n - 1)}{N}S^2_{wsy},$$

where \( S^2_{wsy} = \frac{1}{k(n - 1)}\sum_i\sum_j (y_{ij} - \bar y_{i\cdot})^2 \) is the mean square within systematic samples. Equivalently, in terms of the intraclass correlation \( \rho_w \) between units of the same systematic sample,

$$V(\bar y_{sy}) = \frac{N - 1}{N}\cdot\frac{S^2}{n}\left[1 + (n - 1)\rho_w\right].$$

Systematic vs SRS

Subtracting \( V_{ran} = \frac{N - n}{Nn}S^2 \) gives

$$V(\bar y_{sy}) - V_{ran} = \frac{n - 1}{n}\left(S^2 - S^2_{wsy}\right).$$

So systematic sampling beats SRSWOR iff \( S^2_{wsy} > S^2 \): each systematic sample must be more heterogeneous than the population as a whole. That is the same principle as stratification, turned around: strata should be homogeneous; systematic samples should be heterogeneous.

Three kinds of population

PopulationComparisonWhy
Random order\( V_{sy} \approx V_{ran} \) on averagethe systematic sample behaves like a random one
Linear trend \( Y_i = a + bi \)\( V_{st} < V_{sy} < V_{ran} \)each sample spans the trend; one-per-stratum spans it best
Periodic, period \( k \) (or a divisor of \( k \))\( V_{sy} \) as large as possibleevery unit of a sample has the same phase, so \( \rho_w = 1 \)

For the linear trend \( Y_i = i \), \( N = nk \), exact values are

$$V_{ran} = \frac{(k - 1)(nk + 1)}{12}, \qquad V_{sy} = \frac{k^2 - 1}{12}, \qquad V_{st} = \frac{k^2 - 1}{12n},$$

which with \( n = k = 10 \) are the hook’s \( 75.75 \), \( 8.25 \) and \( 0.825 \).

Memory Tricks

Don’t Confuse

ThisvsThat
\( S^2 \) (divisor \( N - 1 \)): SRSWOR uses it, \( E(s^2) = S^2 \)\( \sigma^2 \) (divisor \( N \)): SRSWR uses it, \( E(s^2) = \sigma^2 \)
Proportional allocation: \( n_h \propto N_h \), self-weighting (\( \bar y_{st} = \bar y \))Neyman allocation: \( n_h \propto N_hS_h \), not self-weighting in general
Gain of proportional over SRS: between-stratum meansGain of Neyman over proportional: between-stratum SDs
Stratified: want homogeneous strataSystematic: want heterogeneous samples (\( S^2_{wsy} > S^2 \))
Inclusion probability \( \frac{n}{N} \) in SRSWOR (for each unit)probability of a particular sample \( \frac{1}{\binom{N}{n}} \)
Linear systematic (\( N = nk \), start in \( 1..k \))circular systematic (start in \( 1..N \), wrap round)

Common Mistakes

Practice

All questions below are original practice questions written for this page. They are not from any CSIR NET paper. In Level 3, as in Part C, one or more options may be correct.

Level 1: Foundation

  1. How many SRSWOR samples of size 4 can be drawn from \( N = 12 \) units, and what is the inclusion probability of a given unit?
  2. In SRSWR of size 3 from \( N = 10 \) units, find the expected number of distinct units in the sample.
  3. Write the variance of \( \bar y_{st} \) under proportional allocation.
Answers
  1. \( \binom{12}{4} = 495 \) samples; \( \pi_i = \frac{4}{12} = \frac{1}{3} \).
  2. \( E(\nu) = 10\left[1 - (0.9)^3\right] = 10(0.271) = 2.71 \).
  3. \( V_{prop} = \frac{1 - f}{n}\sum_h W_hS_h^2 \), with \( f = \frac{n}{N} \) and \( W_h = \frac{N_h}{N} \).

Level 2: Part B Style

Practice Q1. From the population \( \{2, 4, 6, 8, 10\} \) a sample of size 2 is drawn by SRSWOR. The variance of the sample mean is

(a) 2 (b) 3 (c) 4 (d) 5

Answer: (b)

\( \bar Y = 6 \), \( \sum(Y_i - 6)^2 = 16 + 4 + 0 + 4 + 16 = 40 \), so \( S^2 = \frac{40}{4} = 10 \). \( V(\bar y) = \frac{N - n}{Nn}S^2 = \frac{3}{10} \times 10 = 3 \). Option (c), 4, is the SRSWR value \( \frac{\sigma^2}{n} = \frac{8}{2} \); option (d) is \( \frac{S^2}{n} \) without the fpc.

Practice Q2. A population of 1000 units is split into three strata with \( N_h = 200, 300, 500 \) and \( S_h = 15, 10, 6 \). A sample of 90 is allocated by Neyman allocation. The number drawn from the first stratum is

(a) 18 (b) 27 (c) 30 (d) 45

Answer: (c)

\( N_hS_h = 3000, 3000, 3000 \), so each stratum gets \( \frac{90}{3} = 30 \). Option (a), 18, is the proportional allocation \( 90 \times 0.2 \); (d), 45, is the proportional share of stratum 3.

Practice Q3. A survey must estimate a proportion believed to be near 0.4 within \( \pm 0.05 \), using \( z = 2 \), in a population of \( N = 2000 \). Using \( n = \frac{n_0}{1 + n_0/N} \), the required SRSWOR sample size is

(a) 384 (b) 323 (c) 322 (d) 300

Answer: (b)

\( n_0 = \frac{z^2PQ}{d^2} = \frac{4 \times 0.24}{0.0025} = 384 \). Then \( n = \frac{384}{1 + 0.192} = \frac{384}{1.192} \approx 322.15 \), rounded up to 323. (The variant \( 1 + \frac{n_0 - 1}{N} \) gives 322.28, also 323.) Option (a) ignores the fpc; (c) rounds down.

Practice Q4. The population consists of \( Y_i = i \), \( i = 1, \dots, 100 \). A linear systematic sample of size 10 is drawn. The variance of the sample mean is

(a) 75.75 (b) 9.09 (c) 8.25 (d) 0.825

Answer: (c)

\( k = 10 \). The 10 possible samples have means \( i + 45 \), \( i = 1, \dots, 10 \): a shift of the discrete uniform on \( \{1, \dots, 10\} \), whose variance is \( \frac{k^2 - 1}{12} = \frac{99}{12} = 8.25 \). Option (a) is the SRSWOR variance \( \frac{(k - 1)(nk + 1)}{12} \); (d) is the one-unit-per-stratum value.

Level 3: Part C Style

Practice Q5. For the three-stratum population of Q2, take also the stratum means \( \bar Y_h = 50, 40, 30 \) and \( n = 90 \). Which statements are true?

(a) Under proportional allocation, \( V(\bar y_{st}) \approx 0.940 \). (b) Under Neyman allocation, \( V(\bar y_{st}) = 0.807 \). (c) \( V_{prop} - V_{opt} = \sum_h W_h(S_h - \bar S)^2 \) where \( \bar S = \sum W_hS_h \). (d) An SRSWOR of 90 units from the whole population has a variance of the sample mean exceeding 1.5.

Answer: (a), (b) and (d)
  • (a) True. \( \sum W_hS_h^2 = 45 + 30 + 18 = 93 \); \( V_{prop} = \frac{1 - 0.09}{90} \times 93 = 0.9403 \).
  • (b) True. \( \bar S = 9 \); \( V_{opt} = \frac{9^2}{90} - \frac{93}{1000} = 0.9 - 0.093 = 0.807 \).
  • (c) False. The identity has a factor \( \frac{1}{n} \): \( V_{prop} - V_{opt} = \frac{1}{n}\sum W_h(S_h - \bar S)^2 = \frac{12}{90} = 0.1333 \), not 12.
  • (d) True. \( \bar Y = 0.2(50) + 0.3(40) + 0.5(30) = 37 \). The total sum of squares is \( \sum (N_h - 1)S_h^2 + \sum N_h(\bar Y_h - \bar Y)^2 = (199 \cdot 225 + 299 \cdot 100 + 499 \cdot 36) + (200 \cdot 169 + 300 \cdot 9 + 500 \cdot 49) = 92639 + 61000 = 153639 \), so \( S^2 = \frac{153639}{999} \approx 153.79 \) and \( V_{ran} = \frac{0.91}{90} \times 153.79 \approx 1.555 \).

Practice Q6. A linear systematic sample of size \( n \) is drawn from a population of size \( N = nk \). Which statements are true?

(a) Every unit has inclusion probability \( \frac{1}{k} \), so the sample mean is unbiased for \( \bar Y \). (b) An unbiased estimator of \( V(\bar y_{sy}) \) can be computed from the single sample drawn. (c) If the mean square within systematic samples exceeds the population mean square \( S^2 \), the systematic sample mean has smaller variance than the SRSWOR mean. (d) If the population is periodic with period exactly \( k \), systematic sampling is at least as precise as stratified sampling with one unit per stratum of \( k \) consecutive units.

Answer: (a) and (c)
  • (a) True. Unit \( i + jk \) is drawn iff the start is \( i \), probability \( \frac{1}{k} = \frac{n}{N} \).
  • (b) False. Units in different systematic samples have joint inclusion probability 0, so no unbiased variance estimator exists from one sample.
  • (c) True. \( V_{sy} - V_{ran} = \frac{n - 1}{n}(S^2 - S^2_{wsy}) < 0 \).
  • (d) False. With period \( k \), every unit of a systematic sample has the same value, so \( \rho_w = 1 \) and \( V_{sy} = \frac{N - 1}{N}S^2 \), the largest possible. Stratified sampling with one unit from each run of \( k \) is unaffected by the periodicity in this way.

Practice Q7. From the population \( \{2, 4, 6, 8, 10\} \), samples of size 2 are drawn. Which statements are true?

(a) \( \frac{V_{\text{SRSWR}}(\bar y)}{V_{\text{SRSWOR}}(\bar y)} = \frac{4}{3} \). (b) Under SRSWOR, \( s^2 \) is unbiased for \( \sigma^2 = 8 \). (c) Under SRSWR, the probability that the unit with value 2 appears at least once is \( \frac{9}{25} \). (d) Under SRSWOR, the probability that the units with values 2 and 4 are both drawn is \( \frac{1}{10} \).

Answer: (a), (c) and (d)
  • (a) True. \( 4 / 3 = \frac{N - 1}{N - n} = \frac{4}{3} \).
  • (b) False. Under SRSWOR, \( E(s^2) = S^2 = 10 \). (Under SRSWR, \( E(s^2) = \sigma^2 = 8 \).)
  • (c) True. \( 1 - \left(\frac{4}{5}\right)^2 = \frac{9}{25} \).
  • (d) True. \( \pi_{ij} = \frac{n(n - 1)}{N(N - 1)} = \frac{2}{20} = \frac{1}{10} \); equivalently one of the \( \binom{5}{2} = 10 \) equally likely samples.

Quick Revision Table

IdeaFact
SRSWOR\( V(\bar y) = (1 - f)\frac{S^2}{n} \); \( E(s^2) = S^2 \); \( \pi_i = \frac{n}{N} \), \( \pi_{ij} = \frac{n(n-1)}{N(N-1)} \)
SRSWR\( V(\bar y) = \frac{\sigma^2}{n} \); \( E(s^2) = \sigma^2 \); \( E(\nu) = N[1 - (1 - \frac{1}{N})^n] \)
WR vs WORratio of variances \( \frac{N - 1}{N - n} \)
Proportion\( V(p) = \frac{N - n}{N - 1}\frac{PQ}{n} \)
Sample size\( n_0 = \frac{z^2S^2}{d^2} \), \( n = \frac{n_0}{1 + n_0/N} \), round up
Stratified mean\( \sum W_h\bar y_h \), \( V = \sum W_h^2(1 - f_h)\frac{S_h^2}{n_h} \)
Allocationsproportional \( \propto N_h \); Neyman \( \propto N_hS_h \); cost \( \propto \frac{N_hS_h}{\sqrt{c_h}} \)
Ordering\( V_{opt} \le V_{prop} \le V_{ran} \) (large strata)
Systematic\( V = \frac{N-1}{N}\frac{S^2}{n}[1 + (n-1)\rho_w] \); beats SRS iff \( S^2_{wsy} > S^2 \)
Linear trend\( V_{ran} : V_{sy} : V_{st} = \frac{(k-1)(nk+1)}{12} : \frac{k^2-1}{12} : \frac{k^2-1}{12n} \)

Key Takeaways

What’s Next