Skip to content

Probability Distributions and the Central Limit Theorem

CSIR NET Physical Sciences: random variables, binomial, Poisson and normal distributions, random walks, counting errors and the central limit theorem.

  • 16 min read
  • Updated
On this page · 15 sections
  1. Prerequisites
  2. The Hook: Why a Geiger Counter Never Ticks Evenly
  3. Random Variables, Mean and Variance
  4. The Binomial Distribution
  5. The Poisson Distribution
  6. The Normal (Gaussian) Distribution
  7. The Central Limit Theorem
  8. Propagation of Errors
  9. Memory Tricks
  10. Don’t Confuse
  11. Common Mistakes
  12. Practice
  13. Quick Revision Table
  14. Key Takeaways
  15. What’s Next

Prerequisites

The Hook: Why a Geiger Counter Never Ticks Evenly

Random Variables, Mean and Variance

A random variable \( X \) takes values \( x_i \) with probabilities \( p_i \) (discrete) or has a probability density \( f(x) \) with \( P(a < X < b) = \int_a^bf\,dx \) (continuous). In both cases the probabilities sum or integrate to 1.

QuantityDiscreteContinuous
Mean \( \mu = \langle X\rangle \)\( \sum x_ip_i \)\( \int xf(x)\,dx \)
Variance \( \sigma^2 \)\( \langle X^2\rangle - \langle X\rangle^2 \)same
Standard deviation\( \sigma = \sqrt{\text{variance}} \)same

Rules that do most of the work.

  • \( \langle aX + b\rangle = a\langle X\rangle + b \) and \( \operatorname{Var}(aX + b) = a^2\operatorname{Var}X \).
  • Means always add: \( \langle X + Y\rangle = \langle X\rangle + \langle Y\rangle \).
  • For independent \( X \), \( Y \): variances add, \( \operatorname{Var}(X + Y) = \operatorname{Var}X + \operatorname{Var}Y \), and \( \operatorname{Var}(X - Y) \) is the same sum, not the difference.
  • The characteristic function \( \phi_X(k) = \langle e^{ikX}\rangle \) is the Fourier transform of the density; for independent variables \( \phi_{X+Y} = \phi_X\phi_Y \). The density of a sum is the convolution of the densities.

Two continuous distributions you need in passing.

  • Uniform on \( (a, b) \): mean \( \frac{a + b}{2} \), variance \( \frac{(b - a)^2}{12} \).
  • Exponential with rate \( \lambda \), \( f(t) = \lambda e^{-\lambda t} \) for \( t \ge 0 \): mean \( 1/\lambda \), variance \( 1/\lambda^2 \). It describes the waiting time between events of a Poisson process (the time to the next decay), and it is memoryless: \( P(T > s + t \mid T > s) = P(T > t) \). A nucleus that has not decayed is as good as new.

The Binomial Distribution

\( n \) independent trials, each a success with probability \( p \) (failure \( q = 1 - p \)). The number of successes \( k \) has

$$\boxed{P(k) = \binom{n}{k}p^kq^{n-k}, \qquad \mu = np, \qquad \sigma^2 = npq}$$

The most probable value is near \( np \) (precisely \( \lfloor(n + 1)p\rfloor \), with a tie when \( (n + 1)p \) is an integer). The distribution is symmetric only for \( p = \tfrac{1}{2} \).

Examples. Four fair coin tosses: \( P(\text{2 heads}) = 6/16 = 0.375 \). Ten tosses: \( P(5) = 252/1024 \approx 0.246 \), so even the most likely outcome happens less than a quarter of the time.

The one-dimensional random walk

A walker takes \( N \) steps of length \( l \), each to the right with probability \( p \) and to the left with \( q \). If \( n_R \) steps go right, the displacement is \( x = (2n_R - N)l \). Since \( n_R \) is binomial,

$$\langle x\rangle = N(p - q)l, \qquad \sigma_x^2 = 4Npq\,l^2.$$

For the unbiased walk (\( p = q = \tfrac{1}{2} \)): \( \langle x\rangle = 0 \) and

$$\boxed{x_{\text{rms}} = l\sqrt N}$$

Diffusion is this result in disguise: with \( N \) proportional to time, the mean-square displacement grows linearly in time, \( \langle x^2\rangle = 2Dt \) in one dimension.

The Poisson Distribution

Let \( n \to \infty \) and \( p \to 0 \) with \( \lambda = np \) fixed. The binomial distribution becomes

$$\boxed{P(k) = \frac{\lambda^ke^{-\lambda}}{k!}, \qquad \mu = \sigma^2 = \lambda}$$

Properties.

  • Mean equals variance. A count \( N \) therefore has standard deviation \( \sqrt N \) and fractional error \( 1/\sqrt N \): 100 counts give 10%, 10 000 counts give 1%.
  • \( P(k + 1)/P(k) = \lambda/(k + 1) \): the probabilities rise while \( k + 1 < \lambda \) and fall afterwards, so the mode is \( \lfloor\lambda\rfloor \) (with a tie at \( \lambda - 1 \) and \( \lambda \) for integer \( \lambda \)).
  • The sum of independent Poisson variables is Poisson with the summed mean.
  • \( P(0) = e^{-\lambda} \), so \( P(\text{at least one}) = 1 - e^{-\lambda} \).
  • For large \( \lambda \) the Poisson distribution approaches a normal distribution with \( \mu = \sigma^2 = \lambda \).

The Normal (Gaussian) Distribution

$$\boxed{f(x) = \frac{1}{\sigma\sqrt{2\pi}}\exp\left[-\frac{(x - \mu)^2}{2\sigma^2}\right]}$$

Standardise with \( z = (x - \mu)/\sigma \).

RangeProbability
\( \lvert z\rvert < 1 \)0.683
\( \lvert z\rvert < 2 \)0.954
\( \lvert z\rvert < 3 \)0.997
\( z > 1.645 \)0.050
\( z > 1.96 \)0.025
\( z > 2 \)0.023

Facts to have ready.

  • Full width at half maximum: \( \text{FWHM} = 2\sqrt{2\ln2}\,\sigma \approx 2.355\sigma \).
  • Moments about the mean: \( \langle(x - \mu)^2\rangle = \sigma^2 \), \( \langle(x - \mu)^4\rangle = 3\sigma^4 \); all odd central moments vanish.
  • A sum of independent normal variables is normal, with means and variances added.
  • Its characteristic function is again a Gaussian, \( e^{i\mu k - \sigma^2k^2/2} \), the probabilistic face of “a Gaussian is its own Fourier transform”.

Normal approximation to the binomial (de Moivre–Laplace): for large \( n \) with \( np \) and \( nq \) both large, the binomial is close to a normal with \( \mu = np \), \( \sigma^2 = npq \). Because the binomial is discrete, use a continuity correction: \( P(X \ge 60) \) becomes \( P(X > 59.5) \) in the normal approximation.

The Central Limit Theorem

Let \( X_1, \ldots, X_N \) be independent and identically distributed with finite mean \( \mu \) and finite variance \( \sigma^2 \). Then as \( N \to \infty \) the standardised sum

$$\boxed{Z_N = \frac{\sum_{i=1}^{N}X_i - N\mu}{\sigma\sqrt N}}$$

approaches the standard normal distribution, whatever the distribution of the individual \( X_i \). Equivalently, the sample mean \( \bar X \) is approximately normal with mean \( \mu \) and standard deviation \( \sigma/\sqrt N \) (the standard error).

Why it works. The characteristic function of \( Z_N \) is \( [\phi(k/\sigma\sqrt N)]^N \) (after centring). Expanding \( \phi(k) = 1 - \sigma^2k^2/2 + \cdots \) and taking the limit gives \( e^{-k^2/2} \), the transform of a standard Gaussian. The finite variance is exactly what makes the \( k^2 \) term exist.

Consequences.

  • Averaging beats noise as \( 1/\sqrt N \). To reduce the standard error tenfold you need a hundred times as many measurements.
  • Gaussian errors are the norm because a measurement error is usually the sum of many small independent effects.
  • A classic demonstration: the sum of 12 independent uniform \( (0, 1) \) variables has mean 6 and variance \( 12\times\frac{1}{12} = 1 \), so (sum \( - 6 \)) is already a good approximation to a standard normal variable.

When it fails. If the variance is infinite, the theorem does not apply. The Lorentzian (Cauchy) distribution \( f(x) = \frac{1}{\pi}\frac{\gamma}{x^2 + \gamma^2} \), the line shape of a decaying state, has no finite variance (not even a finite mean in the usual sense). The average of \( N \) Cauchy variables has the same Cauchy distribution as one of them: averaging does not help at all. Its characteristic function \( e^{-\gamma\lvert k\rvert} \) has no \( k^2 \) term, which is where the derivation above breaks.

graph LR
    A[Binomial n, p] -->|n large, p small, np = lambda fixed| B[Poisson lambda]
    A -->|n large, np and nq large| C[Normal mean np, variance npq]
    B -->|lambda large| C
    D[Any distribution with finite variance] -->|sum of many independent copies| C

Propagation of Errors

For \( f(x_1, \ldots, x_n) \) with independent errors \( \sigma_i \) in each \( x_i \), to first order,

$$\boxed{\sigma_f^2 = \sum_i\left(\frac{\partial f}{\partial x_i}\right)^2\sigma_i^2}$$
  • Sum or difference: absolute errors add in quadrature.
  • Product or quotient: relative errors add in quadrature.
  • Power \( f = x^n \): \( \sigma_f/f = \lvert n\rvert\sigma_x/x \).

Memory Tricks

Don’t Confuse

ThisvsThat
Standard deviation of one measurement \( \sigma \)Standard error of the mean \( \sigma/\sqrt N \)
\( \operatorname{Var}(X - Y) = \operatorname{Var}X + \operatorname{Var}Y \) (independent)\( \langle X - Y\rangle = \langle X\rangle - \langle Y\rangle \)
Poisson: \( \sigma^2 = \mu \)Binomial: \( \sigma^2 = \mu q < \mu \)
Random walk: \( x_{\text{rms}} \propto \sqrt N \)Ballistic motion: distance \( \propto N \)
CLT: needs finite varianceCauchy/Lorentzian: averages do not converge
Exponential waiting time: memorylessNormal or uniform lifetimes: not memoryless

Common Mistakes

Practice

All questions below are original practice questions written for this page. They are not from any CSIR NET paper.

Level 1: Foundation

  1. Mean and variance of a binomial distribution with \( n = 10 \), \( p = 0.5 \)?
  2. For a Poisson distribution with \( \lambda = 2 \), what is \( P(0) \)?
  3. A Geiger counter registers 10 000 counts. What is the fractional statistical error?
Answers
  1. Mean 5, variance 2.5. \( np = 5 \), \( npq = 2.5 \).
  2. \( e^{-2} \approx 0.135 \).
  3. 1%. \( \sqrt{10000}/10000 = 0.01 \).

Level 2: Part B Style

Practice Q1. A counter records on average 4 counts per second from a steady source. The probability of exactly 2 counts in a given one-second interval is closest to:

(a) 0.073 (b) 0.147 (c) 0.195 (d) 0.25

Answer: (b) 0.147

\( P(2) = \frac{4^2e^{-4}}{2!} = 8e^{-4} \approx 8\times0.0183 = 0.147 \). Option (c) is \( P(3) = P(4) \approx 0.195 \), the mode; (a) is \( P(1) \).

Practice Q2. A particle performs an unbiased one-dimensional random walk of 100 steps, each of length 1 µm. Its root-mean-square displacement from the start is:

(a) 1 µm (b) 10 µm (c) 50 µm (d) 100 µm

Answer: (b) 10 µm

\( x_{\text{rms}} = l\sqrt N = 1\times\sqrt{100} = 10 \) µm. Option (d) is the total path length; (c) confuses the rms displacement with \( N/2 \) steps.

Practice Q3. Twelve independent random numbers are drawn uniformly from \( (0, 1) \) and added. The mean and standard deviation of the sum are:

(a) 6 and 1 (b) 6 and \( \sqrt{12} \) (c) 6 and 12 (d) 0.5 and 1

Answer: (a) 6 and 1

Each term has mean \( \tfrac{1}{2} \) and variance \( \tfrac{1}{12} \). Means add to 6; variances add to 1, so \( \sigma = 1 \). By the central limit theorem the sum is already close to a normal distribution.

Practice Q4. For a Gaussian line shape of standard deviation \( \sigma \), the full width at half maximum is approximately:

(a) \( \sigma \) (b) \( 1.177\sigma \) (c) \( 2\sigma \) (d) \( 2.355\sigma \)

Answer: (d) 2.355σ

Half maximum where \( e^{-x^2/2\sigma^2} = \tfrac{1}{2} \), i.e. \( x = \sigma\sqrt{2\ln2} \approx 1.177\sigma \). The full width is twice that. Option (b) is the half width.

Level 3: Part C Style

Practice Q5. A fair coin is tossed 100 times; \( X \) is the number of heads. Which of the following are correct?

  1. \( \langle X\rangle = 50 \) and \( \sigma_X = 5 \).
  2. With the continuity correction, the normal approximation gives \( P(X \ge 60) \approx 0.029 \).
  3. \( P(40 \le X \le 60) \) is approximately 0.96.
  4. The Poisson distribution with \( \lambda = 50 \) is a good approximation to the distribution of \( X \).

(a) 1 and 2 only (b) 1, 2 and 3 only (c) 1 and 4 only (d) all four

Answer: (b) 1, 2 and 3 only

Statement 1: \( np = 50 \), \( \sqrt{npq} = \sqrt{25} = 5 \). True.

Statement 2: \( z = (59.5 - 50)/5 = 1.9 \), upper tail \( \approx 0.0287 \). The exact binomial value is 0.0284. True.

Statement 3: with correction, \( P(39.5 < X < 60.5) = P(\lvert z\rvert < 2.1) \approx 0.964 \); the exact value is 0.965. True.

Statement 4: Poisson with \( \lambda = 50 \) has variance 50, twice the true variance 25. \( p = \tfrac{1}{2} \) is not small. False.

Practice Q6. In a counting experiment, a detector records 900 counts in 100 s with a source in place and 400 counts in 100 s with the source removed. Assume Poisson statistics. Which of the following are correct?

  1. The net count rate due to the source is \( 5.0 \) s\( ^{-1} \).
  2. The standard deviation of the net rate is about \( 0.36 \) s\( ^{-1} \).
  3. The standard deviation of the net rate is about \( 0.22 \) s\( ^{-1} \).
  4. Counting both runs for 400 s instead of 100 s would halve the uncertainty in the net rate.

(a) 1 and 2 only (b) 1, 2 and 4 only (c) 1, 3 and 4 only (d) 1 and 3 only

Answer: (b) 1, 2 and 4 only

Statement 1: \( 9.0 - 4.0 = 5.0 \) s\( ^{-1} \). True.

Statements 2 and 3: the rate errors are \( 30/100 = 0.30 \) and \( 20/100 = 0.20 \) s\( ^{-1} \); add in quadrature: \( \sqrt{0.13} \approx 0.36 \). Statement 2 true; statement 3 (\( \sqrt{0.09 - 0.04} \approx 0.22 \)) subtracts variances. False.

Statement 4: counts grow fourfold, so each count’s relative error falls by \( \sqrt4 = 2 \), and the rate errors become 0.15 and 0.10 s\( ^{-1} \), giving 0.18 s\( ^{-1} \). True.

Practice Q7. Which of the following statements are correct?

  1. The waiting time between successive decays from a long-lived source is exponentially distributed and memoryless.
  2. The mean of \( N \) independent measurements, each with standard deviation 0.5, has standard error 0.05 when \( N = 100 \).
  3. The average of \( N \) independent Lorentzian (Cauchy) distributed variables approaches a Gaussian as \( N \to \infty \).
  4. If \( X \) and \( Y \) are independent Poisson variables with means 3 and 5, \( X + Y \) is Poisson with mean 8.

(a) 1 and 2 only (b) 1, 2 and 4 only (c) 2, 3 and 4 only (d) all four

Answer: (b) 1, 2 and 4 only

Statement 1: decays form a Poisson process; inter-event times are exponential and memoryless. True.

Statement 2: \( 0.5/\sqrt{100} = 0.05 \). True.

Statement 3: the Cauchy distribution has no finite variance, so the central limit theorem does not apply; the average has the same Cauchy distribution as a single variable. False.

Statement 4: sums of independent Poisson variables are Poisson with added means. True.

Quick Revision Table

Distribution\( P \) or \( f \)MeanVariance
Binomial\( \binom{n}{k}p^kq^{n-k} \)\( np \)\( npq \)
Poisson\( \lambda^ke^{-\lambda}/k! \)\( \lambda \)\( \lambda \)
Normal\( \frac{1}{\sigma\sqrt{2\pi}}e^{-(x-\mu)^2/2\sigma^2} \)\( \mu \)\( \sigma^2 \)
Uniform \( (a, b) \)\( 1/(b - a) \)\( (a + b)/2 \)\( (b - a)^2/12 \)
Exponential\( \lambda e^{-\lambda t} \)\( 1/\lambda \)\( 1/\lambda^2 \)
Random walk\( N \) steps of \( l \)0 (unbiased)\( Nl^2 \)
Sample mean (CLT)approx. normal\( \mu \)\( \sigma^2/N \)

Key Takeaways

What’s Next

Advertisement
Advertisement