## Sunday, September 04, 2005

### Raven Paradox Essay

Suppose we are wondering whether the hypothesis (S1) All ravens are black is true. Intuitively, it seems that observing black ravens ought to boost the credence we give to S1. For similar reasons, observing a non-black non-raven presumably provides confirming evidence for the statement (S2): Whatever is not black is not a raven. But note that S1 and S2 are logically equivalent. Each is true if and only if there are no non-black ravens.[1] So whatever confirms the one statement must also confirm the other. In particular, a red herring is a non-black non-raven, and thus confirms S2, and thus confirms S1. But it seems incredible that observing a red herring could provide any evidence whatsoever for the claim that all ravens are black. I will argue that this conclusion is no ‘paradox’. On the contrary, it is demonstrably true. Building on the insights this yields into the nature of confirmation, I will further argue that black ravens are not always confirming evidence that all ravens are black. Common sense is wrong on both counts.

There are two general approaches that one might take in seeking evidence related to a universal statement like S1. One can try to find “confirming instances” of the hypothesis, i.e. objects that it is true of. Alternatively, one can seek disconfirming instances that would falsify the hypothesis.[2] Discussions of the raven paradox have traditionally focused on the first approach. I will argue that this is a mistake, and that there is no such thing as direct confirmation of a universal statement. Instead, the way to confirm S1 is, roughly, to try as hard as one can to falsify it. If it survives all such attempts, then this is evidence that there is no falsity to be had, i.e. that the hypothesis is true.

On this approach, the way to confirm that all ravens are black is to look for a raven that isn’t. More precisely, we employ a method that might reasonably be expected to turn up a non-black (henceforth, ‘coloured’) raven, were such to exist. When it fails to do so, this provides some evidence that there are no such coloured ravens to be found.[3] The strength of this evidence will depend upon how likely the method would be to expose coloured ravens were there any. Clearly, if the method is guaranteed to expose any coloured ravens that exist, then, when it fails to do so, this would conclusively prove that none do. By contrast, if the method has only a very faint chance of picking up a coloured raven, then its failure to do so provides correspondingly faint evidence against them.

The perfect test of S1 would be to exhaustively check every object in existence to see whether any are coloured ravens. If we found one, S1 would be shown to be false, and otherwise we would have established its truth. But we can also do imperfect tests, which merely sample the population of worldly objects, rather than doing an exhaustive census. So let’s consider the method of randomly sampling objects from the world. Every sampled object that is not a coloured raven would count as (weak) evidence that there are no such creatures. For if there were coloured ravens then we should expect them to turn up occasionally in random samples. The fact that they never do counts against them. Put another way, an object of any given type is more likely to turn up in a random sample if there are comparatively few objects of other types in the sample population. In particular, red herrings are slightly more like to be observed if there are no coloured ravens in the population than if there are some, all else being equal. This explains why it is that red herrings can be confirming evidence for S1.

The intuitive oddness of this conclusion can be partly dissolved when we recall that the strength of confirmation depends upon the proportion of coloured ravens we should expect to find in the sample population, were any to exist. If our sample population comprises all worldly objects, then this proportion is vanishingly small. We can improve it by shrinking the size of the sample population without excluding any possible coloured ravens from consideration. One way to do this would be to sample from the population of coloured objects. But of course there are still an impractically large number of those. Better yet, we could sample from the population of ravens. This would provide much stronger results. Hence our intuition that the way to confirm S1 is to observe black ravens, not red herrings. For all practical purposes, our intuition is quite right. But the difference is only one of degree, not kind. To illustrate: imagine a world bursting full of ravens, but with extremely few coloured (non-black) objects.[4] In such a world, the best way to confirm that all ravens are black would not be to look first for ravens and then check their colour. It would be much more efficient to instead search through the coloured objects for one that is a raven.

Despite the above arguments, one might remain suspicious of the ‘paradoxical conclusion’. Intuitively, it seems that when faced with a bucket of herrings, going through it and identifying each red herring would do nothing at all to confirm that all ravens are black. In fact, I will argue that this intuition is correct. Although it would confirm S1 to observe of coloured things that they are non-ravens, this is different from observing of non-ravens that they are coloured.

The distinction can be clarified in terms of the ‘sample population’. If we sample objects from the population of all coloured objects, then any coloured ravens that exist will be included in this population, and so stand a chance of being sampled. However, this is not true of the population of non-ravens, nor, for that matter, the population of black things. Even if coloured ravens existed, they would stand no chance of being sampled from such a restricted population. So our failure to find them in such a sample provides no evidence whatsoever about their existential status. Thus, whether any given object counts as confirming a universal statement like S1 will depend upon the circumstances of the observation. This shows that the commonly assumed principle of Direct Confirmation is false:

(DC) Whenever an object has two attributes C1, C2, it constitutes confirming evidence for the hypothesis that every object which has the attribute C1 also has the attribute C2.[5]

To illustrate: Suppose I am wondering whether the hypothesis (B) Nothing exists beyond the boundaries of my backyard is true. DC implies that I could obtain evidence for this hypothesis without having to look beyond my backyard at all. Exploring my lawn, I notice plenty of objects that both (1) exist, and (2) are in my backyard; hence, by DC, each such object serves as evidence for B, the claim that all existing objects are in my backyard. But this is clearly absurd. Exploring the confines of my backyard tells me nothing about the existential status of the world beyond it. For the same reason, exploring a bucket of herrings cannot tell us whether coloured ravens exist elsewhere.

This shows that DC is a mistaken account of confirmation. Hypotheses are not directly confirmed by their instances. Instead, they are confirmed indirectly, when a potentially falsifying test returns a negative result. The reasoning behind indirect confirmation was explained earlier in this essay. Let us call a population ‘CR-open’ if it would contain coloured ravens were any to exist, and ‘CR-closed’ if it would not. So, for example, the populations of ravens, of coloured objects, and of everything that exists, are all CR-open, whereas populations of black objects, or of herrings, are CR-closed. The upshot of our earlier discussion is that evidence concerning the existence of coloured ravens can be obtained by randomly sampling CR-open populations, but not CR-closed ones. Thus finding a red herring in the population of coloured objects is confirming evidence for S1, whereas finding a black raven by exploring the population of black objects is not. Sometimes red herrings really do confirm S1, and sometimes black ravens really don’t. Insofar as our intuitions rebel against this ‘paradoxical’ result, they are mistaken to do so, as my account of the nature of confirmation makes clear.

Hempel defends DC against the above objections by arguing that CR-closed populations are corrupted by background knowledge: we “know anyhow” that they contain no coloured ravens, so further testing is simply unnecessary.[6] The suggestion here is that the bucket of herrings does provide evidence for S1, but it does this as soon as you find out that it is a bucket of herrings. You learn nothing new from searching through it. But this is insufficient to explain our intuitions here. DC implies that the number of herrings in the bucket makes a difference. Each one provides an additional evidential boost to the hypothesis S1. But this is not plausible. Whether the bucket contains two herrings or two thousand can make no difference to the likelihood of a coloured raven existing elsewhere. This is something that my theory of indirect confirmation can clearly explain. DC appears quite unmotivated in comparison. Indeed, it doesn’t really explain why confirmation ever occurs at all.

One possible foundation for direct confirmation comes from what I will call ‘the argument from exclusion’.[7] We begin, a priori, with various rival hypotheses about the existential status of coloured ravens, and assign each some non-zero probability. We are primarily interested in the claim (S3) There are no coloured ravens; but one rival to this might[8] be the hypothesis (EC) Everything is a coloured raven. When we observe a bucket of red herrings, in any circumstances whatsoever, this allows us to rule out EC, thus boosting the a posteriori probability of all its rivals, including S3.

However, this is only part of the story. When we observe the bucket of red herrings, that allows us to rule out any hypothesis about the world which is inconsistent with the existence of such a bucket. Some of these, like EC, may be rivals to S3. But others – such as (ED) Everything is a dog – will be consistent with S3, and so our credence in the latter will be harmed by their exclusion. We have no reason to think that the sum effect of these exclusions will be favourable towards S3. So the argument from exclusion fails.

To recap: I have argued that the standard account of direct confirmation is mistaken. This helps to explain some of our intuitive resistance to the ‘paradoxical conclusion’. We are right to think that there is no evidence to be found from observing objects that are already known not to be ravens. It is only through sampling objects that might be coloured ravens that the possibility of falsification arises, and this possibility is a prerequisite for indirect confirmation. The raven paradox can survive this modification, however, since we can still establish the surprisingly conclusion that a red herring can, in the right circumstances, constitute confirming evidence that all ravens are black.

There is another source of intuitive resistance worth exploring. The red herring confirms that all ravens are black because it is evidence that there are no coloured ravens. But we could just as well say that the herring is evidence that there are no non-green ravens, and hence that all ravens are green. However, it seems decidedly odd to say that one piece of evidence can confirm both that all ravens are black and that all ravens are green. The problem is that the sampled red herring, on its own, serves as evidence against the existence of all other object types. In particular, it is evidence that there are no ravens at all. It is only by factoring in our background knowledge that there are some black ravens that we get to rule out the more general hypothesis that there are no ravens at all, and hence that all zero of them are green.

How we interpret this result will depend upon whether we take universal statements to have existential import – that is, whether “all ravens are black” entails that “some ravens exist”. Modern logicians deny this, holding that empty universal statements are vacuously true. That is, “there are no ravens” entails “all ravens are X” for any X whatsoever. Since the red herring confirms (in isolation) that there are no ravens at all, so it confirms the logical entailment that “all ravens are X” for every possible X. It is important to note that on this view, “all ravens are black” and “all ravens are green” are not inconsistent. Both will be true if there are no ravens at all. So this is no problem for the herring’s confirmation of S1.

Let us now consider how affirming existential import would affect this. It would imply that S1 and S2 are no longer logically equivalent,[9] so the standard raven argument would fail. After all, observing the herring does nothing at all to confirm the existence of a black raven. Quite the opposite, in fact: in isolation from our background knowledge, it provides evidence that there are no ravens at all, and hence no black ones. So, if S1 requires that some black ravens exist, then observing a red herring does not in isolation confirm S1. What it does support – even in isolation – is the claim (S3) there are no coloured ravens. When we combine this with our background knowledge, we obtain support for S1. But again, the core of the raven paradox survives these minor alterations. It is intuitively surprising that a red herring could confirm S3, or that in conjunction with our background knowledge of black ravens it could confirm S1. But we might consider these results to be less surprising than the original claim that a red herring alone is evidence that all ravens are black. This reaction would show that we were (mis)understanding S1 as having existential import. Our intuitions are quite right to reject the view that red herrings can confirm the existence of black ravens. But the raven argument, properly understood, makes no such claim.

We are now in a position to agree with Hempel that the appearance of ‘paradox’ in the raven argument is misleading. I have shown how a red herring can, in appropriate circumstances, constitute probabilistic evidence that all ravens are black. Further, three potential causes of intuitive resistance to this conclusion can be safely dispelled. Firstly, differences in population size mean that much stronger confirmation is to be had by finding a raven to be black than by finding a coloured object to be other than a raven. The degree of confirmation from the latter is so weak as to be unworthy of notice for all practical purposes. Second, there is no evidence to be found by exploring a population already known to not contain coloured ravens. This intuitively pleasing result has some surprising implications, however, including (i) that traditional accounts of ‘direct confirmation’ are mistaken; and (ii) that black ravens are not always evidence that all ravens are black. Finally, our intuitions might be misled by interpreting the universal claim to have existential import. We are right to think that the herring cannot serve as evidence for the existence of black ravens; all it suggests is that there are no non-black ravens.

Despite this agreement, my disagreements with Hempel may be more significant. I have disputed the traditional principle of Direct Confirmation, and proposed an alternative theory of indirect confirmation, which clearly explains how the ‘paradoxical conclusion’ arises and why it is true. It also deepens the ‘paradox’ by implying that in some circumstances, black ravens do not confirm that all ravens are black. The advocated theory of indirect confirmation does more than just solve Hempel’s raven paradox, it also expands and illuminates it.

Bibliography

Fitelson, B. & Hawthorne, J. (forthcoming) ‘How Bayesian Confirmation Theory Handles the Paradox of the Ravens’ http://fitelson.org/ravens.pdf

Hempel, C. (1945) ‘Studies in the Logic of Confirmation I’ Mind, 54 (213): 1-26.

Hempel, C. (1946) ‘A Note on the Paradoxes of Confirmation’ Mind, 55 (217): 79-82.

Musgrave, A. (2004) ‘How Popper [Might Have] Solved the Problem of Induction’ Philosophy, 79: 19-31.

Whiteley, C. (1945) ‘Hempel’s Paradoxes of Confirmation’ Mind, 54 (214): 156-158.

[1] I will discuss the implications of denying this equivalence later in the essay.

[2] Cf. Musgrave, p.24, though I depart from the Popperians in holding that resistance to falsification is itself evidence for the truth of the hypothesis in question, at least in the circumstances I describe below.

[3] Whiteley, p.157, hints at this line of argument, though he fails to follow through on it. In particular, he makes the mistake of thinking that if an event is “not improbable even if S1 is false” then it “provides no evidence” for S1. But it is relative, not absolute, probabilities that matter here. If an event is more probable given S1’s truth than its falsity (even if this latter probability is itself high in absolute terms) then the event boosts the a posteriori probability of S1.

[4] I owe this thought experiment to Doug Campbell.

[5] Hempel (1946), p.79. There he calls it the “R1” requirement, and elsewhere in the literature it is referred to as “Nicod’s Condition”.

[6] Hempel (1945), pp.19-20. Though Fitelson & Hawthorne, p.7, argue that this contradicts Hempel’s own theory of confirmation.

[7] Thanks to Doug Campbell for suggesting this argument to me.

[8] To ensure that the two hypotheses really are mutually inconsistent, we must assume that the universe is not empty, i.e. that at least one object exists.

[9] In particular, S1 would entail the existence of a raven, whereas S2 would instead entail the existence of a non-black object. So, in a possible world containing nothing but red herrings, S2 would be true but S1 false.

1. It's only baffling because there is such a large number of "all things". If you divide the weight contributed by observing a red herring by the number of things in the set of which it is a member, you'll find the level of contribution is almost infinitesimal (and possibly actually infinitesimal!)

What you have is an observation which matches your theory. It's baffling because it looks like the test for "is not a raven" is a trifling matter. But consider if it were difficult to distinguish herrings from ravens, and there were a lot of controversy.

In that situation, what you have intuitively is really good confirmation. It's only because your example doesn't give any credence to whether the raven-or-herring part of the test is difficult. You've got this assumption that the breaking up of the world into ravens and non-ravens is trivial.

Cheers,
-MP

2. I keep looking at the bottom of your essays for somthing like "MelbournePhilosopher (2005)" !

3. Very interesting. It seems your account has two parts, each designed to handle a distinct version of the paradox.

First version of the paradox: From reasonable assumptions, it follows that observing of a colored object that it is not a raven confirms "All ravens are black".

Part One of your account (here I think your account is just the mere content cutting account): Yes, but this only seems counterinuitive because it only confirms "All ravens are black" to a very small (perhaps vanishingly small) degree. Why? Because by observing of a colored object that it is non-raven merely raises the probability of "All ravens are black" from the probability of "Colored object one is non-raven and colored object two is non-raven and colored object three is non-raven... and colored object n is non-raven" to the probability of "colored object two is non-raven and colored object three is non-raven and colored object n is non raven". Since n is so very large, this increase is very small.

Second version of the paradox: From reasonable assumptions, it follows that observing of a non-raven that it is colored confirms "All ravens are black".

Part two of your account: While observing of a colored object that it is non-raven confirms "All ravens are black" (a very little bit) observing of a non-raven that it is colored does not. Why? Roughly, because this observation had no "potential" to falsify "All ravens are black".

I really like how you've drawn out these two aspects (versions) of the paradox and I really like your solutions to both of them.

But what if we observe of some object that it is colored and non-raven—that is, what if take a sample from the total population and find that it is both colored and non-raven? Of course, given the right background beliefs (most importantly: the belief that the population of ravens is relatively small compared to the population at large), “part one” of your account will be able to hand this case. But suppose our background beliefs were different. In particular, suppose we believe that there are exactly 100 objects in the universe and that exactly 50 of them are ravens. Now suppose scientist A takes a random sample from the population at a large and discovers that they are all black ravens and scientist B takes a random sample from the population at large and discovers that they are all colored non-ravens. According to part one of your account (part two is irrelevant here), A and B have confirmed “All ravens are black” equally. But, intuitively, that doesn’t seem right. While in the case of B’s evidence, it seems that we ought to raise our probability of “All ravens are black” in accordance with mere content cutting, but in the case of A’s evidence, it seems that we ought to raise it more than that. This is precisely the sort of case that Lange’s account is supposed to handle (although I’m not sure if he ultimately gets it).

4. Thanks for the kind remarks!

'observing of a colored object that it is non-raven merely raises the probability of "All ravens are black" from the probability of "Colored object one is non-raven and colored object two is non-raven and colored object three is non-raven... and colored object n is non-raven" to the probability of "colored object two is non-raven and colored object three is non-raven and colored object n is non raven".'

I'm not so sure about that. Random sampling seems to involve more than just content cutting. Content-cutting sounds a lot like what I called "the argument from exclusion", which I argue doesn't guarantee any (even tiny) confirmation at all. If God gives me a specially selected bucket of red herrings, that doesn't tell me anything at all about the rest of the world. But if I randomly sample the universe and come up with a bucket of red herrings, then that seems to tell me something about the entire population. It makes me think that there probably aren't heaps of coloured ravens around, or else they should have turned up in my sample. Isn't that saying something more than mere content cutting?

The case in your final paragraph is tricky! Could I say that scientist A has effectively received information about a sample of ravens, in addition to the primary sample of everything? Suppose we sample 10 objects from the universe, 5 of which are black ravens. The latter part of this seems in some sense equivalent to a sample of 5 objects from the raven population, all of which are black. (Granted, non-ravens could have turned up in their place. But that doesn't seem especially important here. The crucial point is that all ravens had an equal chance of being sampled.) If this sort of interpretation is legitimate (a big 'if'!), then my account is still able to explain the disparity you point to.

5. But wouldn't that be contradictory to your sampling account--i.e. the idea that it matters which population we are REALLY sampling from?

The key point is this: The sense in which this is a sample from the RAVEN population is the same sense in which it is also a sample from the BLACK OBJECT population. But if we interpret the sample THAT way, then, according to your account, it's not going to tell you anything (beyond mere content cutting) about "All ravens are black" because examining black objects doesn't even have the potential to falsify "All ravens are black".

By the way, I'm really glad you came across my post and pointed me towards yours. You've really helped me clarify some of my thinking on this stuff. Thanks!

Also, can a simple bayesian framework explain why observing OF a raven that it is black confirms "All ravens are black" while observing OF a black obect that it is a raven does not? Let me see if I can work it out.

I think these are reasonable priors:

pr(observing OF a raven that is black) = .1

pr(observing OF a black object that it is a raven) = .000000001

pr(all ravens are black) = .5^x = a

So, if we conditionalize using bayes' theorem, then the posterior probability of "All ravens are black" after we observe OF a raven that it is black will be 1a/.1 = 10a and the posterior probability of "All ravens are black" after observing OF a black object that it is raven will be .00000001a/.000000001 = a.

The reason we have 1 in the numerator in the former case is of course because pr(observing of a raven that it is black given all ravens are black) = 1. And the reason we have .00000001 in the numerator in the latter case is because pr(observing of a black object that it is a raven given all ravens are black) = pr(observing of a black object that it is a raven) = .0000001. In other words (and this is the key idea), the truth of "All ravens are black" doesn't make it anymore likely that we observe OF a black object that it is raven than if it were false (well, maybe it makes it a LITTLE more likely but that won't affect the argument).

Is this right? Well, of course I don't think it is exactly right, because I think we raise the probability by more than mere content cutting when we observe of a raven that it is black, but I think we can ignore this for the sake of argument (we could have consider the case of, e.g., observing of a coin flip that it is followed by a landing heads vs observing of a landing heads that it is preceeded by a coin flip).

6. I don't know about the Bayesian equations; I find it easier to understand this stuff on an intuitive level. But if the numbers are supportive, I'm happy to hear it :)

"But wouldn't that be contradictory to your sampling account--i.e. the idea that it matters which population we are REALLY sampling from?"

But I think we really are sampling from the raven population here, albeit indirectly, at least in the sense that matters. After all, every raven has an equal chance of being selected. The fact that all five we came across happened to be black statistically indicates that there are a lot more black than coloured ravens around.

"The sense in which this is a sample from the RAVEN population is the same sense in which it is also a sample from the BLACK OBJECT population. But if we interpret the sample THAT way, then, according to your account, it's not going to tell you anything (beyond mere content cutting) about "All ravens are black" because examining black objects doesn't even have the potential to falsify "All ravens are black"."

Indeed, I want to say that this interpretation provides no evidence at all (not even "content cutting", which in this case at least presumably rests upon the fallacious argument from exclusion). But that's no problem. We should simply treat the various interpretations as distinct 'sub-samples', the aggregate of which amounts to the full evidence produced by the super-sample. Some evidence from RAVEN sample + no evidence from BLACK sample = some evidence in total.

If that sounds too much like cheating, do let me know :)

7. I see what you mean about every raven having an equal chance of being selected and that

"We should simply treat the various interpretations as distinct 'sub-samples', the aggregate of which amounts to the full evidence produced by the super-sample."

Very interesting suggestion. I'm going have to think more about that.

Visitors: check my comments policy first.
Non-Blogger users: If the comment form isn't working for you, email me your comment and I can post it on your behalf. (If your comment is too long, first try breaking it into two parts.)