The paradox of induction. Consider a statement whose truth is unknown. If we see many examples that are compatible with it, we are tempted to view the statement as more probable. Such reasoning is often referred to as inductive inference (in a philosophical, rather than mathematical sense). Consider now the statement that “all cows are white.” An equivalent statement is that “everything that is not white is not a cow.” We then observe several black crows. Our observations are clearly compatible with the statement. but do they make the hypothesis “all cows are white” more likely?
To analyze such a situation, we consider a probabilistic model. Let us assume that there are two possible states of the world, which we model as complementary events:
$A:$ all cows are white,
$A^{c}: 50 \%$ of all cows are white.
Let $p$ be the prior probability $\mathbf{P}(A)$ that all cows are white. We make an observation of a cow or a crow, with probability $q$ and $1-q$, respectively, independent of whether event $A$ occurs or not. Assume that $0<p<1,0<q<1$, and that all crows are black.
(a) Given the event $B=\{$ a black crow was observed $\}$, what is $\mathbf{P}(A \mid B) ?$
(b) Given the event $C=\{$ a white cow was observed $\}$, what is $\mathbf{P}(A \mid C) ?$
(a) We use the formula
$$
\mathbf{P}(A \mid B)=\frac{\mathbf{P}(A \cap B)}{\mathbf{P}(B)}=\frac{\mathbf{P}(A) \mathbf{P}(B \mid A)}{\mathbf{P}(B)}
$$
Since all crows are black, we have $\mathbf{P}(B)=1-q .$ Furthermore, $\mathbf{P}(A)=p$. Finally, $\mathbf{P}(B \mid A)=1-q=\mathbf{P}(B)$, since the probability of observing a (black) crow is not affected by the truth of our hypothesis. We conclude that $\mathbf{P}(A \mid B)=\mathbf{P}(A)=p .$ Thus, the new evidence, while compatible with the hypothesis “all cows are white,” does not change our beliefs about its truth.
(b) Once more,
$$
\mathbf{P}(A \mid C)=\frac{\mathbf{P}(A \cap C)}{\mathbf{P}(C)}=\frac{\mathbf{P}(A) \mathbf{P}(C \mid A)}{\mathbf{P}(C)}
$$
Given the event $A$, a cow is observed with probability $q$, and it must be white. Thus, $\mathbf{P}(C \mid A)=q .$ Given the event $A^{c}$, a cow is observed with probability $q$, and it is white with probability $1 / 2 .$ Thus, $\mathbf{P}\left(C \mid A^{c}\right)=q / 2$. Using the total probability theorem,
$$
\mathbf{P}(C)=\mathbf{P}(A) \mathbf{P}(C \mid A)+\mathbf{P}\left(A^{c}\right) \mathbf{P}\left(C \mid A^{c}\right)=p q+(1-p) \frac{q}{2}
$$
Hence,
$$
\mathbf{P}(A \mid C)=\frac{p q}{p q+(1-p) \frac{q}{2}}=\frac{2 p}{1+p}>p
$$
Thus, the observation of a white cow makes the hypothesis “all cows are white” more likely to be true.