Skip to content

Sample Space, Independence and Bayes' Theorem

CSIR NET probability: sample spaces, sigma-fields, Kolmogorov axioms, inclusion-exclusion, conditional probability, independence and Bayes' theorem.

  • 3 min read
  • Updated
On this page · 12 sections
  1. Prerequisites
  2. The Hook: The Test That Is 95% Accurate
  3. Sample Space, Events and the Axioms
  4. Conditional Probability
  5. Independence
  6. Memory Tricks
  7. Don’t Confuse
  8. Common Mistakes
  9. Practice
  10. Quick Revision Table
  11. Key Takeaways
  12. What’s Next

Prerequisites

The Hook: The Test That Is 95% Accurate

Sample Space, Events and the Axioms

A random experiment has a set \( \Omega \) of possible outcomes, the sample space. Events are subsets of \( \Omega \), but when \( \Omega \) is uncountable not every subset can be assigned a probability consistently, so we fix a collection \( \mathcal{F} \) of events.

\( \sigma \)-field. \( \mathcal{F} \subseteq \mathcal{P}(\Omega) \) is a \( \sigma \)-field if (i) \( \Omega \in \mathcal{F} \), (ii) \( A \in \mathcal{F} \Rightarrow A^c \in \mathcal{F} \), (iii) \( A_1, A_2, \dots \in \mathcal{F} \Rightarrow \bigcup_n A_n \in \mathcal{F} \). It is then closed under countable intersections, differences and limsup/liminf of sequences. The Borel \( \sigma \)-field on \( \mathbb{R} \) is the smallest one containing all intervals.

Kolmogorov’s axioms. A probability \( P : \mathcal{F} \to [0, 1] \) satisfies \( P(\Omega) = 1 \) and countable additivity: for pairwise disjoint \( A_1, A_2, \dots \), \( P\left(\bigcup A_n\right) = \sum P(A_n) \). The triple \( (\Omega, \mathcal{F}, P) \) is a probability space.

Consequences of the axioms

RuleStatement
Complement\( P(A^c) = 1 - P(A) \); \( P(\emptyset) = 0 \)
Monotonicity\( A \subseteq B \Rightarrow P(A) \le P(B) \), and \( P(B \setminus A) = P(B) - P(A) \)
Addition\( P(A \cup B) = P(A) + P(B) - P(A \cap B) \)
Boole (union bound)\( P\left(\bigcup A_n\right) \le \sum P(A_n) \)
Bonferroni\( P\left(\bigcap_{i=1}^{n} A_i\right) \ge \sum P(A_i) - (n - 1) \)
Continuity\( A_n \uparrow A \Rightarrow P(A_n) \uparrow P(A) \); \( A_n \downarrow A \Rightarrow P(A_n) \downarrow P(A) \)

Inclusion–exclusion.

$$P\left(\bigcup_{i=1}^{n} A_i\right) = \sum_i P(A_i) - \sum_{i Truncating after an odd number of sums gives an upper bound, after an even number a lower bound.

The probability that exactly \( m \) of the \( n \) events occur is \( \sum_{k=m}^{n} (-1)^{k-m}\binom{k}{m} S_k \), where \( S_k \) is the \( k \)-th inclusion–exclusion sum.

Three ways to assign probabilities

ModelRuleTypical question
Classical\( P(A) = \frac{\lvert A \rvert}{\lvert \Omega \rvert} \), equally likely outcomesdice, cards, committees, arrangements
Geometric\( P(A) = \frac{\text{measure}(A)}{\text{measure}(\Omega)} \)random points on a segment, square, disc
Axiomatic / givenany \( P \) obeying the axiomsdiscrete models, data-based models

Three classical results worth carrying:

  • Birthday problem. With 365 equally likely birthdays, the probability that \( n \) people all differ is \( \prod_{k=0}^{n-1}\left(1 - \frac{k}{365}\right) \); it drops below \( \frac{1}{2} \) at \( n = 23 \).
  • Broken stick. Break a unit stick at two uniformly chosen points. The three pieces form a triangle (each piece \( < \frac{1}{2} \)) with probability \( \frac{1}{4} \).
  • Buffon’s needle. A needle of length \( l \) dropped on parallel lines at spacing \( d \ge l \) crosses a line with probability \( \frac{2l}{\pi d} \).

Conditional Probability

For \( P(B) > 0 \),

$$\boxed{P(A \mid B) = \frac{P(A \cap B)}{P(B)}}$$

For fixed \( B \), \( A \mapsto P(A \mid B) \) is itself a probability on \( \mathcal{F} \), so every rule above holds conditionally: \( P(A^c \mid B) = 1 - P(A \mid B) \), and so on. Conditioning does not work in the other slot: \( P(A \mid B^c) \ne 1 - P(A \mid B) \) in general.

Multiplication rule. \( P(A_1 A_2 \cdots A_n) = P(A_1)\,P(A_2 \mid A_1)\,P(A_3 \mid A_1A_2)\cdots P(A_n \mid A_1 \cdots A_{n-1}) \).

Law of total probability. If \( B_1, \dots, B_k \) partition \( \Omega \) with \( P(B_i) > 0 \),

$$P(A) = \sum_{i=1}^{k} P(A \mid B_i)\,P(B_i).$$

Bayes’ theorem.

$$\boxed{P(B_j \mid A) = \frac{P(A \mid B_j)\,P(B_j)}{\sum_i P(A \mid B_i)\,P(B_i)}}$$

Read it as posterior \( \propto \) likelihood \( \times \) prior. In odds form for two hypotheses:

$$\frac{P(B_1 \mid A)}{P(B_2 \mid A)} = \frac{P(A \mid B_1)}{P(A \mid B_2)} \cdot \frac{P(B_1)}{P(B_2)}.$$
graph LR
    S[Start] -->|P of B1| B1[Cause B1]
    S -->|P of B2| B2[Cause B2]
    S -->|P of B3| B3[Cause B3]
    B1 -->|P of A given B1| A1[Effect A]
    B2 -->|P of A given B2| A2[Effect A]
    B3 -->|P of A given B3| A3[Effect A]

Total probability adds the products along every path that ends in \( A \). Bayes takes one such path product and divides by that total.

Pólya’s urn: dependence without asymmetry

An urn holds \( r \) red and \( b \) black balls. Draw one, return it together with \( c \) more of the same colour, repeat. Then:

  • \( P(\text{red on draw } n) = \frac{r}{r + b} \) for every \( n \) (the draws are exchangeable);
  • the draws are not independent: \( P(R_1 R_2) = \frac{r}{r+b}\cdot\frac{r+c}{r+b+c} \ne \left(\frac{r}{r+b}\right)^2 \) when \( c > 0 \).

Exchangeability lets you compute \( P(R_1 \mid R_2) = P(R_2 \mid R_1) \).

Independence

Two events. \( A \) and \( B \) are independent if \( P(A \cap B) = P(A)\,P(B) \). If \( P(B) > 0 \) this says \( P(A \mid B) = P(A) \).

If \( A, B \) are independent, so are \( A, B^c \); \( A^c, B \); and \( A^c, B^c \). An event of probability 0 or 1 is independent of every event, including itself; conversely \( A \) is independent of itself iff \( P(A) \in \{0, 1\} \).

Many events. \( A_1, \dots, A_n \) are mutually independent if for every subcollection \( \{i_1 < \dots < i_k\} \), \( k \ge 2 \),

$$P(A_{i_1} \cdots A_{i_k}) = P(A_{i_1}) \cdots P(A_{i_k}).$$

That is \( 2^n - n - 1 \) equations. Pairwise independence asks only for the \( \binom{n}{2} \) equations with \( k = 2 \).

Independence versus disjointness. If \( A \cap B = \emptyset \) and both have positive probability, then \( P(AB) = 0 \ne P(A)P(B) \): disjoint events with positive probability are always dependent. Disjointness is a property of sets; independence is a property of \( P \).

Independent trials. Repeating an experiment independently gives product probabilities; this is the setting of the binomial and geometric distributions and of the laws of large numbers in Inequalities, Convergence and Limit Theorems.

Conditional independence (\( P(AB \mid C) = P(A \mid C)P(B \mid C) \)) neither implies nor follows from independence. In the Pólya urn, the colours are conditionally independent given the limiting proportion but not independent; in the two-coin example above, \( A \) and \( B \) are independent but not conditionally independent given \( C \).

Memory Tricks

Don’t Confuse

ThisvsThat
\( P(A \mid B) \)\( P(B \mid A) \) (Bayes converts one to the other)
mutually exclusive: \( P(AB) = 0 \)independent: \( P(AB) = P(A)P(B) \)
pairwise independentmutually independent
\( P(A^c \mid B) = 1 - P(A \mid B) \) (true)\( P(A \mid B^c) = 1 - P(A \mid B) \) (false)
exchangeable draws (Pólya)independent draws
finitely additivecountably additive (the axiom)

Common Mistakes

Practice

All questions below are original practice questions written for this page. They are not from any CSIR NET paper. In Level 3, as in Part C of Mathematical Sciences, one or more options may be correct.

Level 1: Foundation

  1. \( A \) and \( B \) are independent with \( P(A) = 0.5 \), \( P(B) = 0.4 \). Find \( P(A \cup B) \).
  2. Can two events with positive probabilities be both mutually exclusive and independent?
  3. How many product conditions define mutual independence of 5 events?
Answers
  1. \( 0.5 + 0.4 - 0.2 = 0.7 \).
  2. No. Disjointness gives \( P(AB) = 0 \), independence gives \( P(AB) = P(A)P(B) > 0 \).
  3. \( 2^5 - 5 - 1 = 26 \).

Level 2: Part B Style

Practice Q1. A condition has prevalence 1%. A test has sensitivity 95% and false-positive rate 5%. Given a positive test, the probability of the condition is closest to:

(a) 0.95 (b) 0.50 (c) 0.16 (d) 0.05

Answer: (c)

\( \frac{0.0095}{0.0095 + 0.0495} = \frac{0.0095}{0.059} \approx 0.161 \). (a) confuses \( P(+ \mid D) \) with \( P(D \mid +) \).

Practice Q2. Box I has 3 red and 2 white balls; box II has 1 red and 4 white. A box is chosen at random and one ball is drawn; it is red. The probability that it came from box I is:

(a) \( \frac{3}{5} \) (b) \( \frac{3}{4} \) (c) \( \frac{2}{5} \) (d) \( \frac{1}{2} \)

Answer: (b)

\( P(\text{red}) = \frac{1}{2}\cdot\frac{3}{5} + \frac{1}{2}\cdot\frac{1}{5} = \frac{2}{5} \). Then \( P(\text{I} \mid \text{red}) = \frac{3/10}{2/5} = \frac{3}{4} \). (c) is \( P(\text{red}) \) itself.

Practice Q3. A fair die is rolled three times. The probability that all three faces are different is:

(a) \( \frac{1}{2} \) (b) \( \frac{5}{6} \) (c) \( \frac{5}{9} \) (d) \( \frac{4}{9} \)

Answer: (c)

\( \frac{6 \cdot 5 \cdot 4}{6^3} = \frac{120}{216} = \frac{5}{9} \).

Practice Q4. \( X \) and \( Y \) are independent and uniform on \( [0, 1] \). Then \( P(\lvert X - Y \rvert < \frac{1}{3}) \) equals:

(a) \( \frac{1}{3} \) (b) \( \frac{5}{9} \) (c) \( \frac{2}{3} \) (d) \( \frac{4}{9} \)

Answer: (b)

\( 1 - \left(\frac{2}{3}\right)^2 = \frac{5}{9} \): the complement is two right triangles with legs \( \frac{2}{3} \). (d) is the complement.

Level 3: Part C Style

Practice Q5. Let \( \Omega = \{1, 2, 3, 4\} \) with \( P(\{i\}) = \frac{1}{4} \), and \( A = \{1, 2\} \), \( B = \{1, 3\} \), \( C = \{1, 4\} \). Which statements are true?

(a) \( A \) and \( B \) are independent. (b) \( A \), \( B \), \( C \) are mutually independent. (c) \( A \) and \( B^c \) are independent. (d) \( A \) and \( B \cap C \) are independent.

Answer: (a) and (c)
  • (a) True. \( P(AB) = P(\{1\}) = \frac{1}{4} = \frac{1}{2}\cdot\frac{1}{2} \).
  • (b) False. \( P(ABC) = \frac{1}{4} \ne \frac{1}{8} \).
  • (c) True. Complements of independent events are independent: \( P(AB^c) = P(\{2\}) = \frac{1}{4} = \frac{1}{2}\cdot\frac{1}{2} \).
  • (d) False. \( B \cap C = \{1\} \subset A \), so \( P(A \cap \{1\}) = \frac{1}{4} \ne \frac{1}{2}\cdot\frac{1}{4} \). This is exactly the failure of mutual independence.

Practice Q6. An urn contains 2 red and 3 black balls. A ball is drawn, then returned together with one more ball of the same colour, and a second ball is drawn. Let \( R_i \) = red on draw \( i \). Which statements are true?

(a) \( P(R_2) = \frac{2}{5} \). (b) \( P(R_1 \mid R_2) = \frac{1}{2} \). (c) \( R_1 \) and \( R_2 \) are independent. (d) \( P(R_1 R_2) = \frac{1}{5} \).

Answer: (a), (b) and (d)
  • (d) True. \( P(R_1R_2) = \frac{2}{5}\cdot\frac{3}{6} = \frac{1}{5} \).
  • (a) True. \( P(R_2) = \frac{2}{5}\cdot\frac{3}{6} + \frac{3}{5}\cdot\frac{2}{6} = \frac{6 + 6}{30} = \frac{2}{5} \), the same as for the first draw.
  • (b) True. \( P(R_1 \mid R_2) = \frac{1/5}{2/5} = \frac{1}{2} \).
  • (c) False. \( \frac{1}{5} \ne \left(\frac{2}{5}\right)^2 = \frac{4}{25} \).

Practice Q7. Four letters are placed at random, one each, into four addressed envelopes. Which statements are true?

(a) The probability that no letter is in its correct envelope is \( \frac{3}{8} \). (b) The probability that exactly three letters are correct is 0. (c) The probability that exactly one letter is correct is \( \frac{1}{3} \). (d) The probability that at least one letter is correct exceeds \( \frac{2}{3} \).

Answer: (a), (b) and (c)
  • (a) True. There are 9 derangements of 4 objects: \( \frac{9}{24} = \frac{3}{8} \).
  • (b) True. Three correct forces the fourth to be correct.
  • (c) True. Choose the correct letter (4 ways) and derange the other three (2 ways): \( \frac{8}{24} = \frac{1}{3} \).
  • (d) False. \( 1 - \frac{3}{8} = \frac{5}{8} = 0.625 < \frac{2}{3} \).

Check: none 9, exactly one 8, exactly two \( \binom{4}{2} \cdot 1 = 6 \), exactly four 1; total 24.

Quick Revision Table

IdeaFact
Probability space\( (\Omega, \mathcal{F}, P) \); \( \sigma \)-field closed under complements and countable unions
Axioms\( P(\Omega) = 1 \), countable additivity
Inclusion–exclusionalternating sums \( S_1 - S_2 + S_3 - \dots \); truncations are bounds
Matching problem\( P(\text{no match}) = \sum_{k=0}^{n} \frac{(-1)^k}{k!} \to \frac{1}{e} \)
Conditional\( P(A \mid B) = P(AB)/P(B) \), a probability in \( A \)
Total probability\( P(A) = \sum P(A \mid B_i)P(B_i) \)
Bayes\( P(B_j \mid A) \propto P(A \mid B_j)P(B_j) \)
Independence\( P(AB) = P(A)P(B) \); passes to complements
Mutual, \( n \) events\( 2^n - n - 1 \) conditions; pairwise is weaker
Disjoint + positive probabilitydependent
Geometric\( P(\lvert X - Y \rvert < a) = 1 - (1 - a)^2 \); broken stick \( \frac{1}{4} \); Buffon \( \frac{2l}{\pi d} \)

Key Takeaways

What’s Next