Skip to content

Bayes Theorem

·
The same joint event can be factored as P(B)P(A∣B)\mathbb{P}(B)\mathbb{P}(A\mid B) or as P(A)P(B∣A)\mathbb{P}(A)\mathbb{P}(B\mid A).
Bayes' theorem is a rule for reversing conditional probabilities. It tells us how to compute the probability of a hypothesis after observing evidence.
Let AA be a hypothesis and let BB be an observed event. The conditional probability of AA given BB is
P(A∣B)=P(A∩B)P(B),P(B)>0.\begin{equation*}\mathbb{P}(A\mid B)=\frac{\mathbb{P}(A\cap B)}{\mathbb{P}(B)}, \qquad \mathbb{P}(B)>0.\end{equation*}
The terms in the formula have standard names:
P(A∣B)⏟posterior=P(B∣A)⏟likelihoodP(A)⏟priorP(B)⏟evidence.\begin{equation*}\underbrace{\mathbb{P}(A\mid B)}_{\text{posterior}} =\frac{ \underbrace{\mathbb{P}(B\mid A)}_{\text{likelihood}} \underbrace{\mathbb{P}(A)}_{\text{prior}}}{\underbrace{\mathbb{P}(B)}_{\text{evidence}}}.\end{equation*}
The prior is the probability of AA before seeing BB.
BAYES' THEOREM
Let AA and BB be events with P(A)>0\mathbb{P}(A)>0 and P(B)>0\mathbb{P}(B)>0. Then
P(A∣B)=P(B∣A)P(A)P(B).\begin{equation*}\mathbb{P}(A\mid B) = \frac{\mathbb{P}(B\mid A)\mathbb{P}(A)}{\mathbb{P}(B)}.\end{equation*}
The proof is only the definition of conditional probability used twice. Since
P(A∣B)=P(A∩B)P(B)andP(B∣A)=P(A∩B)P(A),\begin{equation*}\mathbb{P}(A\mid B)=\frac{\mathbb{P}(A\cap B)}{\mathbb{P}(B)}\quad \text{and} \quad\mathbb{P}(B\mid A)=\frac{\mathbb{P}(A\cap B)}{\mathbb{P}(A)},\end{equation*}
we get
P(A∩B)=P(B∣A)P(A).\begin{equation*}\mathbb{P}(A\cap B)=\mathbb{P}(B\mid A)\mathbb{P}(A).\end{equation*}
Substitution into the first identity gives
P(A∣B)=P(B∣A)P(A)P(B).□\begin{equation*}\mathbb{P}(A\mid B)=\frac{\mathbb{P}(B\mid A)\mathbb{P}(A)}{\mathbb{P}(B)}.\qquad \Box\end{equation*}

References

  • [Durrett2019]Durrett, Rick. Probability: Theory and Examples. Fifth edition. Cambridge University Press, 2019. PDF.