Coins!

Coins are our first model of a statistical machine — a simple device that randomly produces one of a finite number of results with some probability. For our purposes, a coin is any object or mechanism that, when flipped, produces one of two outcomes: heads ($H$) or tails ($T$). The Romans called this practice navia aut caput (“ship or head”), after the galley and portrait that appeared on either side of their currency — one of the earliest ways humans took chance into their own hands.

The Basics

A coin is characterized by a single parameter $p \in (0, 1)$:

$$\mathbb{P}(H) = p, \qquad \mathbb{P}(T) = 1 - p.$$

A fair coin has $p = \tfrac{1}{2}$, meaning heads and tails appear with equal likelihood. We assume the coin must land on one side or the other.

Counting probability. When all outcomes are equally likely, probability is:

$$\mathbb{P}(\text{event}) = \frac{\text{number of desired outcomes}}{\text{number of total outcomes}}$$

⚠️ Common mistake. The formula above only applies when every outcome is equally likely. A classic confusion: “there are two outcomes — winning or losing the lottery — so the probability is $\tfrac{1}{2}$.” The error lies in treating distinct outcomes as equally likely ones. If 10,000 lottery tickets exist and any could win with the same probability, the correct denominator is 10,000, not 2.

Question 1. You flip a fair coin 10 times and get tails every time. What is the probability the next flip is heads?

Answer

$\tfrac{1}{2}$. Each flip is independent — past results have no influence on future flips.

Question 2. What is the probability of flipping $HHH$ with a fair coin?

Answer

We can list all 8 outcomes for 3 flips:

Flip 1	Flip 2	Flip 3	$HHH$?
H	H	H	✓
H	H	T
H	T	H
H	T	T
T	H	H
T	H	T
T	T	H
T	T	T

One desired outcome out of eight total:

$\mathbb{P}(HHH) = \frac{1}{8}$

Question 3. What is the probability of flipping exactly one head in three flips?

Answer

Three sequences have exactly one head — $HTT$, $THT$, $TTH$ — out of eight total:

Flip 1	Flip 2	Flip 3	Exactly one $H$?
H	H	H
H	H	T
H	T	H
H	T	T	✓
T	H	H
T	H	T	✓
T	T	H	✓
T	T	T

$\mathbb{P}(\text{exactly one } H) = \frac{3}{8}$

Question 4. What is the probability of flipping exactly 5 heads in 10 flips of a fair coin?

Answer

Ten flips produce too many outcomes to list — there are $2^{10} = 1024$ of them. Let’s split the problem.

Number of desired outcomes

We need to choose which 5 of the 10 positions are heads. We have 10 choices for the first head, 9 for the second, and so on, giving

$10 \cdot 9 \cdot 8 \cdot 7 \cdot 6 = \frac{10!}{5!}$

ordered placements. But this overcounts: labeling the five heads $H_1, H_2, H_3, H_4, H_5$ treats them as distinguishable when they are not.

Each of the $5! = 120$ orderings of those labels maps to the same physical sequence of heads and tails — so we divide by $5!$:

$\frac{10!}{5! \cdot 5!} = \binom{10}{5} = 252$

This is the choose function, also written ${}_{10}C_5$ and read “10 choose 5.”

Number of total outcomes

Each flip has 2 outcomes, so 10 flips give $2^{10} = 1024$ equally likely sequences.

Combining both:

$\mathbb{P}(\text{5 heads in 10 flips}) = \frac{\binom{10}{5}}{2^{10}} = \frac{252}{1024} = \frac{63}{256} \approx 0.246$

The choose function is just the entrance to combinatorics — a rich field dedicated to the art of counting. We will use it repeatedly going forward.

Question 5. What is the probability of flipping at least one head in 10 flips?

Answer

Summing over all non-zero head counts is tedious. Instead, use the complement: only one sequence has no heads at all ($TTTTTTTTTT$), so

$\mathbb{P}(\text{at least one } H) = 1 - \mathbb{P}(\text{no heads}) = 1 - \left(\frac{1}{2}\right)^{10} = 1 - \frac{1}{1024} = \frac{1023}{1024} \approx 0.999.$

Question 6. A weighted coin has $p = \tfrac{3}{4}$. What is the probability of flipping $HHH$?

Answer

Counting probability does not apply here because $HHH$ and $TTT$ are not equally likely. We need a different strategy.

Notice that each flip is independent: the outcome of one does not affect any other. This means we can multiply:

$\mathbb{P}(HHH) = \mathbb{P}(H) \cdot \mathbb{P}(H) \cdot \mathbb{P}(H) = \left(\frac{3}{4}\right)^3 = \frac{27}{64}$

Implementing a Coin

import random

class Coin:
    """A coin that lands heads with probability p."""

    def __init__(self, p=0.5):
        self.p = p

    def flip(self):
        return 'H' if random.random() < self.p else 'T'

Set Notation

Now that we can compute basic probabilities, the next step is combining them. Set theory gives us a precise language.

Define the sample space $\Omega$ as the set of all possible outcomes. An event is any subset $A \subseteq \Omega$.

For three coin flips: $\Omega = \{HHH, HHT, HTH, HTT, THH, THT, TTH, TTT\}$. The event “exactly one head” is $\{HTT, THT, TTH\} \subset \Omega$.

Given an event $A$, we write its probability as $\mathbb{P}(A)$. Set operations produce new events:

Operation	Notation	Probabilistic meaning
Complement	$\mathbb{P}(A^c)$	Probability that $A$ does not occur
Intersection	$\mathbb{P}(A \cap B)$	Probability that both $A$ and $B$ occur
Union	$\mathbb{P}(A \cup B)$	Probability that $A$ or $B$ (or both) occur

Two facts follow immediately:

1. Independence. Events $A$ and $B$ are independent if and only if $\mathbb{P}(A \cap B) = \mathbb{P}(A) \cdot \mathbb{P}(B)$.
2. Inclusion-Exclusion (PIE). $\mathbb{P}(A \cup B) = \mathbb{P}(A) + \mathbb{P}(B) - \mathbb{P}(A \cap B).$

Question 6 (revisited with set notation). A weighted coin has $p = \tfrac{3}{4}$. What is $\mathbb{P}(HHH)$?

Answer

Let $A$, $B$, $C$ be the events “first flip is $H$”, “second flip is $H$”, and “third flip is $H$.” Since flips are independent, $\mathbb{P}(A \cap B) = \mathbb{P}(A) \cdot \mathbb{P}(B)$, and the same holds for any pair. Applying this twice:

$$\begin{align*} \mathbb{P}(A \cap B \cap C) &= \mathbb{P}(A \cap B) \cdot \mathbb{P}(C) \\ &= \mathbb{P}(A) \cdot \mathbb{P}(B) \cdot \mathbb{P}(C) \\ &= \frac{3}{4} \cdot \frac{3}{4} \cdot \frac{3}{4} = \frac{27}{64}. \end{align*}$$

Question 7. Given a coin with $\mathbb{P}(H) = p$, derive the probability of getting exactly $h$ heads in $n$ flips.

Answer

From Question 4, there are $\binom{n}{h}$ sequences of $n$ flips containing exactly $h$ heads. By independence, any specific sequence of $h$ heads and $n-h$ tails has probability $p^h \cdot (1-p)^{n-h}$. Since there are $\binom{n}{h}$ such sequences:

$\boxed{\mathbb{P}(\text{exactly } h \text{ heads in } n \text{ flips}) = \binom{n}{h} p^h (1-p)^{n-h}}$

Question 8. For $p \in (0, 1)$, prove:

$\sum_{h=0}^{n} \binom{n}{h} p^h (1-p)^{n-h} = 1$

Answer

Apply the Binomial Theorem: $(x + y)^n = \sum_{h=0}^{n} \binom{n}{h} x^h y^{n-h}$.

Set $x = p$ and $y = 1-p$:

$\sum_{h=0}^{n} \binom{n}{h} p^h (1-p)^{n-h} = \bigl(p + (1-p)\bigr)^n = 1^n = 1. \qquad \square$

Simulation

The formula from Question 7 makes a testable prediction: if we flip a weighted coin $n$ times and repeat the experiment many times, the proportion of trials with each head count $h$ should approach $\binom{n}{h} p^h (1-p)^{n-h}$ as the number of trials grows. Let’s verify this.

import math

NUM_TRIALS  = 10_000
NUM_FLIPS   = 8
COIN_WEIGHT = 3/4
GOAL_HEADS  = 5

coin = Coin(p=COIN_WEIGHT)

probability_of_goal = (
    math.comb(NUM_FLIPS, GOAL_HEADS)
    * COIN_WEIGHT ** GOAL_HEADS
    * (1 - COIN_WEIGHT) ** (NUM_FLIPS - GOAL_HEADS)
)

num_heads = []
for _ in range(NUM_TRIALS):
    flips = [coin.flip() for _ in range(NUM_FLIPS)]
    num_heads.append(flips.count('H'))

After 10,000 trials the simulated distribution closely matches the theoretical binomial — for $n = 8$ and $p = \tfrac{3}{4}$, the most likely outcome is 6 heads, with $P(\text{5 heads}) \approx 0.208$ as predicted.

The controls below mirror the four parameters in the code. Press Run Simulation to generate your own trials, then try changing the goal, adjusting the coin weight, or cranking up the trial count to see how quickly the empirical distribution converges to theory.

Bayes’ Rule

The final concept we introduce here is conditional probability — the probability of an event given information about another event.

Given events $A$ and $B$ with $\mathbb{P}(B) > 0$, the conditional probability of $A$ given $B$ is:

$\mathbb{P}(A \mid B) = \frac{\mathbb{P}(A \cap B)}{\mathbb{P}(B)}$

From a set-theory perspective, this restricts the sample space from all of $\Omega$ down to $B$: we are asking “among the outcomes in $B$, what fraction also belong to $A$?” We will see a geometric picture of this shortly.

Question 9. Events $A$ and $B$ are independent, with $\mathbb{P}(A) = \tfrac{1}{2}$ and $\mathbb{P}(B) = \tfrac{1}{3}$. What are $\mathbb{P}(A \mid B)$ and $\mathbb{P}(B \mid A)$?

$\mathbb{P}(A \mid B)$

By independence, $\mathbb{P}(A \cap B) = \mathbb{P}(A) \cdot \mathbb{P}(B)$, so:

$\mathbb{P}(A \mid B) = \frac{\mathbb{P}(A \cap B)}{\mathbb{P}(B)} = \frac{\mathbb{P}(A)\,\mathbb{P}(B)}{\mathbb{P}(B)} = \mathbb{P}(A) = \frac{1}{2}$

When $A$ and $B$ are independent, conditioning on $B$ does not affect the probability of $A$. The converse is also true: if conditioning leaves the probability unchanged, the events are independent.

$\mathbb{P}(B \mid A)$

By the same reasoning:

$\mathbb{P}(B \mid A) = \frac{\mathbb{P}(B \cap A)}{\mathbb{P}(A)} = \frac{\mathbb{P}(B)\,\mathbb{P}(A)}{\mathbb{P}(A)} = \mathbb{P}(B) = \frac{1}{3}$

Note: $\mathbb{P}(A \cap B) = \mathbb{P}(B \cap A)$ (intersection is commutative), but conditioning is not — $\mathbb{P}(A \mid B) \neq \mathbb{P}(B \mid A)$ in general.

Question 10. You flip a fair coin 3 times. Given that the first flip is heads, what is the probability that all three flips are heads?

Answer

Left to the reader. Hint: let $A = \{HHH\}$ and $B =$ “first flip is $H$.” How many outcomes are in $B$? Does $A \subseteq B$?

Question 11. You have two coins: a fair coin ($p = \tfrac{1}{2}$) and a weighted coin ($p = \tfrac{2}{3}$). You pick one at random and flip it, and it comes up heads. What is the probability that the next flip is also heads?

Answer

Left to the reader. Hint: use the first flip to update the probability you are holding the weighted coin (Bayes’ theorem), then compute the expected probability of heads on the second flip as a weighted average over both coins.

Geometric Interpretations

Visualize Bayes’ theorem as regions of a sample space. The full grid represents $\Omega$. Paint event $A$ (blue) and event $B$ (orange) and watch the stats panel compute $\mathbb{P}(A)$, $\mathbb{P}(B)$, and $\mathbb{P}(A \mid B) = \mathbb{P}(A \cap B) / \mathbb{P}(B)$ in real time.

Penney’s Game

Coming soon.

Player A selects a sequence of heads and tails (length 3 or longer) and shows it to Player B. Player B then selects a different sequence of the same length. A fair coin is tossed until one player’s sequence appears as a consecutive run. The player whose sequence appears first wins.

Despite appearances, this game is not fair — for any sequence Player A picks, Player B can always choose a sequence that is strictly more likely to appear first. We will analyze specific matchups and derive a general strategy.

Additional Problems

Coming soon.