Bayes' Theorem

Theorem

Let $\Pr$ be a probability measure on a probability space $\left({\Omega, \Sigma, \Pr}\right)$.

Let $\Pr \left({A | B}\right)$ denote the conditional probability of $A$ given $B$.

Let $\Pr \left({A}\right) > 0$ and $\Pr \left({B}\right) > 0$.

Then:

$\displaystyle \Pr \left({B | A}\right) = \frac {\Pr \left({A | B}\right) \Pr \left({B}\right)} {\Pr \left({A}\right)}$

Generalized Versions

There are other more or less complicated ways of saying very much the same thing, all of which can be derived from the basic version with the help of other fairly elementary results.

For example:

Let $\left\{{B_1, B_2, \ldots}\right\}$ be a partition of the event space $\Sigma$.

Then, for any $B_i$ in the partition:

$\displaystyle \Pr \left({B_i | A}\right) = \frac {\Pr \left({A | B_i}\right) \Pr \left({B_i}\right)} {\Pr \left({A}\right)} = \frac {\Pr \left({A | B_i}\right) \Pr \left({B_i}\right)} {\sum_j \Pr \left({A | B_j}\right) \Pr \left({B_j}\right)}$

where here $\sum_j$ denotes the sum over $j$.

Proof

From the definition of conditional probabilities, we have:

• $\displaystyle \Pr \left({A | B}\right) = \frac{\Pr \left({A \cap B}\right)} {\Pr \left({B}\right)}$

• $\displaystyle \Pr \left({B | A}\right) = \frac{\Pr \left({A \cap B}\right)} {\Pr \left({A}\right)}$

After some algebra:

$\displaystyle \Pr \left({A | B}\right) \Pr \left({B}\right) = \Pr \left({A \cap B}\right) = \Pr \left({B | A}\right) \Pr \left({A}\right)$

Dividing both sides by $\Pr \left({A}\right)$ (we are told that it is non-zero), the result follows:

$\displaystyle \Pr \left({B | A}\right) = \frac {\Pr \left({A | B}\right) \Pr \left({B}\right)} {\Pr \left({A}\right)}$

$\blacksquare$

Proof of Generalized Version

Follows directly from the Total Probability Theorem:

$\displaystyle \Pr \left({A}\right) = \sum_i \Pr \left({A | B_i}\right) \Pr \left({B_i}\right)$

$\blacksquare$

Note

This result is also known as Bayes' Formula.

The formula:

$\displaystyle \Pr \left({A | B}\right) \Pr \left({B}\right) = \Pr \left({A \cap B}\right) = \Pr \left({B | A}\right) \Pr \left({A}\right)$

is sometimes called the product rule for probabilities.

Source of Name

This entry was named for Thomas Bayes.

Sources

• Geoffrey Grimmett and Dominic Welsh: Probability: An Introduction (1986): $\S 1.6$: Exercise $18$