What if there are more than two mutually exclusive hypotheses?

Expand the denominator using the Law of Total Probability over all $n$ mutually exclusive and exhaustive hypotheses: $P(E) = \sum_{i=1}^{n} P(E|H_i)P(H_i)$. Then, $P(H_k|E) = \frac{P(E|H_k)P(H_k)}{\sum P(E|H_i)P(H_i)}$.

What if the prior probability $P(H)$ is unknown?

Bayes' theorem cannot be computed without a prior. In some fields, an uninformative prior is assumed, but this requires explicit mathematical justification and alters the nature of the result.

Can Bayes' theorem be applied sequentially for multiple pieces of evidence?

Yes. Assuming the pieces of evidence are conditionally independent given the hypothesis, the posterior probability from the first observation becomes the new prior probability for calculating the impact of the second observation.

How to use Bayes' theorem

Bayes' theorem calculates the probability of an event based on prior knowledge of conditions that might be related to the event. It applies when you know the probability of the evidence given a hypothesis, the prior probability of the hypothesis, and the overall probability of the evidence.

The setup

Identify the hypothesis $H$ and the evidence $E$ . State the known probabilities: the prior $P(H)$ , the likelihood $P(E|H)$ , and the false positive rate $P(E|H^c)$ . If the total probability of the evidence $P(E)$ is not given directly, calculate it using the Law of Total Probability: $P(E) = P(E|H)P(H) + P(E|H^c)P(H^c)$ .

The steps

Define events $H$ (hypothesis) and $E$ (evidence). 2. Extract $P(H)$ , $P(E|H)$ , and $P(E|H^c)$ from the problem statement. Calculate $P(H^c) = 1 - P(H)$ . 3. Compute the denominator $P(E)$ using the Law of Total Probability. 4. Apply Bayes' formula: $P(H|E) = \frac{P(E|H)P(H)}{P(E)}$ .

Checking the result

Ensure $0 \le P(H|E) \le 1$ . Verify that if the evidence is more likely under the hypothesis than in general ( $P(E|H) > P(E)$ ), then the posterior probability $P(H|E)$ must be greater than the prior probability $P(H)$ .

Common errors

Confusing the likelihood $P(E|H)$ with the posterior $P(H|E)$ . Failing to use the Law of Total Probability for the denominator when the total probability of the evidence is not explicitly provided.

Worked example

A disease affects 1% of a population. A test for the disease is 99% accurate (both true positive and true negative rates are 0.99). If a person tests positive, what is the probability they actually have the disease?

Let $D$ be having the disease and $T$ be testing positive. We know: $P(D) = 0.01$ , $P(T|D) = 0.99$ , and $P(T^c|D^c) = 0.99$ . Therefore, $P(T|D^c) = 1 - 0.99 = 0.01$ and $P(D^c) = 0.99$ . We need $P(D|T)$ . Step 1: Calculate the denominator $P(T) = P(T|D)P(D) + P(T|D^c)P(D^c)$ . Substituting the values gives $P(T) = (0.99)(0.01) + (0.01)(0.99) = 0.0099 + 0.0099 = 0.0198$ . Step 2: Apply Bayes' theorem: $P(D|T) = \frac{P(T|D)P(D)}{P(T)} = \frac{0.0099}{0.0198} = 0.5$ . The probability is 50%.

FAQ

Run your own problem

Answer form

References: OpenStax Introductory Statistics, Chapter 3 · Introduction to Probability by Bertsekas and Tsitsiklis, Chapter 1