Conditional probability

When new information arrives, our estimate of probability changes. Conditional probability is the formal tool for updating: $P(A \mid B)$ is the probability that event $A$ occurs given that event $B$ has occurred.

Definition

For events $A$ and $B$ in a sample space, with $P(B) > 0$: $P(A \mid B) = \frac{P(A \cap B)}{P(B)}.$

The condition "$B$ has occurred" effectively shrinks the sample space to $B$. Within this reduced space, the proportion of outcomes that are also in $A$ is $P(A \cap B)/P(B)$.

If $P(B) = 0$, the conditional $P(A \mid B)$ is undefined (or arbitrary).

Key properties

For fixed $B$ with $P(B) > 0$, the function $A \mapsto P(A \mid B)$ is a probability , it satisfies all axioms of probability:

• $0 \le P(A \mid B) \le 1$.
• $P(B \mid B) = 1$.
• $P(\emptyset \mid B) = 0$.
• $P(A_1 \cup A_2 \mid B) = P(A_1 \mid B) + P(A_2 \mid B)$ if $A_1, A_2$ disjoint.
• $P(A^c \mid B) = 1 - P(A \mid B)$.

But note: $P(A \mid B^c)$ is not generally $1 - P(A \mid B)$. The complement of $B$ is a different conditioning event.

Multiplication theorem

Rearranging the definition: $P(A \cap B) = P(A \mid B)\,P(B) = P(B \mid A)\,P(A).$

This is the multiplication rule: the joint probability is the product of the conditional and the marginal. It is the key to computing probabilities by sequential reasoning.

For three events: $P(A \cap B \cap C) = P(A) \cdot P(B \mid A) \cdot P(C \mid A \cap B).$

This generalises: peel off one event at a time, conditioning on what's accumulated.

Tree diagrams

When probabilities depend on sequential outcomes, tree diagrams help. At each node, branches represent possible next outcomes; the probability of each branch is conditional on the path so far. The probability of any path is the product of conditional probabilities along it.

Worked examples

Example 1. A fair die is rolled. Let $A$ = the result is odd, $B$ = the result is at least $4$. Find $P(A \mid B)$.

$B = \{4, 5, 6\}$, $|B| = 3$. $A \cap B = \{5\}$, $|A \cap B| = 1$. $P(A \mid B) = 1/3$.

Example 2. A card is drawn from a standard deck. Find $P(\text{spade} \mid \text{black})$.

$P(\text{black}) = 26/52 = 1/2$. $P(\text{spade} \cap \text{black}) = P(\text{spade}) = 13/52 = 1/4$. So $P(\text{spade} \mid \text{black}) = (1/4)/(1/2) = 1/2$.

Example 3. A bag has 4 red and 6 blue balls. Two balls are drawn without replacement. Find the probability that both are red.

$P(\text{first red}) = 4/10 = 2/5$. After removing one red: $P(\text{second red} \mid \text{first red}) = 3/9 = 1/3$. Joint: $(2/5)(1/3) = 2/15$.

Example 4. Three cards are drawn from a deck (without replacement). Find $P$(all kings).

$P(\text{1st K}) = 4/52$. $P(\text{2nd K} \mid \text{1st K}) = 3/51$. $P(\text{3rd K} \mid \text{1st 2 K}) = 2/50$. Product: $\dfrac{4 \cdot 3 \cdot 2}{52 \cdot 51 \cdot 50} = \dfrac{24}{132600} = \dfrac{1}{5525}$.

Example 5. Two fair coins are flipped. Given that at least one shows heads, find the probability that both show heads.

$P(\text{both H}) = 1/4$. $P(\text{at least one H}) = 3/4$. $P(\text{both H} \mid \text{at least one H}) = (1/4)/(3/4) = 1/3$.

Example 6. A box contains 3 defective and 7 good bulbs. Two bulbs are drawn at random one after the other (without replacement). Find the probability that both are good.

$P(\text{1st good}) = 7/10$. $P(\text{2nd good} \mid \text{1st good}) = 6/9 = 2/3$. Joint: $7/10 \cdot 2/3 = 14/30 = 7/15$.

Try it yourself

1. Roll two dice. Given the sum is $7$, find $P(\text{both odd})$.
2. A card is drawn. Find $P(\text{queen} \mid \text{face card})$.
3. Toss three coins. Given at least one tails, find $P(\text{exactly one tails})$.
4. From a deck, two cards are drawn without replacement. Find $P(\text{both aces})$.
5. A family has two children. Given at least one is a girl, find $P(\text{both girls})$. (Assume equally likely outcomes.)
6. Roll a die twice. Find $P(\text{2nd is 6} \mid \text{1st is even})$.
7. From 5 boys and 4 girls, two are chosen. Find $P(\text{2nd is girl} \mid \text{1st is boy})$.
8. A jar has 5 white and 4 red balls. Three balls are drawn. Find $P(\text{all red})$.
9. Given $P(A) = 0.6, P(B) = 0.4, P(A \cap B) = 0.2$, find $P(A \mid B)$ and $P(B \mid A)$.
10. If $P(A \mid B) = 0.5, P(B) = 0.4$, find $P(A \cap B)$.
11. Two events: $P(A) = 0.3, P(B) = 0.5, P(A \cup B) = 0.6$. Find $P(B \mid A)$.
12. Bag of 6 W, 4 B balls; draw without replacement. Find $P(\text{1st W, 2nd B})$.
13. Two cards from a deck: $P(\text{both spades})$.
14. Two coins: one is biased with $P(H) = 0.7$, other fair. Choose one at random and flip. Given heads, find $P(\text{biased})$.

Pitfalls and tricks

• Conditional probability requires $P(B) > 0$.
• The reduced sample space view is often easier: when computing $P(A \mid B)$, just count outcomes in $B$ and ask what fraction are also in $A$.
• Tree diagrams are great for two- or three-step problems.
• Multiplication rule gives the joint as a product of conditional and marginal. Memorise both directions: $P(A \cap B) = P(A|B)P(B) = P(B|A)P(A)$.
• Conditional probability is not "given that $B$ might occur" , it's "given that $B$ has occurred." Always update accordingly.