Saturday, November 08, 2008

Perceptual and Rational Bias

Suppose I learn that most (but not all) people have a condition which leads them to misperceive purple line segments as only being about half as long as they really are. Further suppose I have no special evidence regarding whether I myself am one of the people with this odd condition. Now I see a purple line, which seems to me about an inch long. How long should I believe it is? (Here I use 'belief' to simply indicate the bulk of one's credence. It is what one would choose if forced to bet one way or the other.) Two inches, presumably.

Next, suppose I learn that most academics in my vicinity have been exposed to a drug that leads them to make inverted evaluations without realizing it: they're inclined to give A grades to C-meriting papers, and vice versa. As before, I have no special grounds for expecting myself to be free of this condition. Now I read a paper by Joe Shmoe that seems hopelessly incoherent and deserving of a C. What grade should I give it? An A?

Erica Roedder proposes an analogy roughly along these lines, but I think the two cases are importantly different. In the first case -- what I'll call 'perceptual bias' -- the only reason for believing that the line is an inch long is the phenomenology of it seeming so. You only have access to the phenomenal 'output', and not to any of the raw data or 'inputs' that form the bases of this judgment. So once you learn that the phenomenology is actually evidence of something else -- namely, the line being twice that length -- that settles the matter. There is no further evidence in play.

However, in case of 'rational bias' -- cases like the second where we make bad inferences or otherwise misinterpret some independently available evidence -- we have more to go on. In addition to the output of our initial judgment, there is also the raw data of the paper itself. So presumably the epistemic fact of what we should believe depends on all this evidence, and not only the higher-order evidence provided by our initial judgment. My initial judgment of badness provides some evidence (due to the known likelihood of bias) that the paper is actually good. But that doesn't settle the matter, for surely the details of the paper itself are also relevant to what grade I ought to give it!

Let's consider the two possibilities in turn. Either I'm biased and the paper is actually good, or I'm unbiased and the paper is truly bad. If the paper is actually good, then both the first-order evidence provided by the paper and the higher-order evidence provided by my initial judgment agree: I should conclude that the paper is good. Simple.

Now for the more interesting possibility: suppose the paper truly is bad. It's still the case that my initial judgment of badness provides some higher-order evidence that the paper is actually good. But there is also the first-order evidence of the incoherent paper itself, which pretty strongly establishes its low quality. I have access not only to my initial judgment, but also to the paper itself and the fact that it reads as follows: (*insert copy of incoherent paper here*). It seems entirely possible that this latter evidence could outweigh the former, in which case I really should conclude that the paper is bad, and give it a C.

In summary: we should correct for the likelihood of perceptual bias, even if it turns out we weren't biased. This is because we have no other evidence to go on in such a case. But cases of rational bias are different, and the appropriate response is contingent on further details. If we are biased, then of course we should correct for it -- that's uncontroversial. But the actually non-biased person arguably should not recalibrate their judgments, or at least not to the same degree. This is not just because they happen to be non-biased (they obviously have no access to this fact), but because of other facts they do have access to, namely the first order evidence.

1. I'm not really clear on how it makes more sense to make a distinction between the seeming and the evidence for the paper than for the line. I can tell a more detailed story about how the evidence works for the paper than for the line, because the workings of perception are almost entirely opaque to me, while my reasoning about paper quality includes some elements I seem to be consciously aware of. But from what the social psychologists tell us, it looks like probably most of my judgment in the latter case is pretty opaque too; indeed, the bits of it that seem transparent I may very well be wrong about (we're pretty unreliable at determining why we come to the beliefs we do). Perhaps even slightly lesser opacity makes a difference, but I don't see nearly as sharp a line as you seem to suggest here.

2. Aaron -- can you clarify how issues of psychological 'transparency', etc., are relevant to my post? I didn't take myself to be making any significant psychological claims. Regardless of what causal processes led to my initial judgments, it remains true that in the paper case (only) I have access to some additional evidence, namely the paper itself.

(Perhaps the confusion arose because of my sloppy reference to "the raw data or 'inputs' that form the bases of this judgment". This might be interpreted to mean that it's psychological opacity per se that is the problem. But that is not what I meant. What matters is simply that we have access to some further evidence beyond the fact of our judgment itself; not whether this is secured through introspection on the judgment or simply by perceiving other relevant evidence, e.g. with our eyes.)

3. Hi Richard. You say there is extra evidence in the paper case, namely the paper. This is extra, presumably, because it is non-phenomenal. (I assume you could have made the same argument with any old phenomenal state akin to seeing purple and any old non-phenomenal evidence akin to the paper.) However you have no epistemic access to the paper except by cognising it somehow or other. And contrastingly there is something about the world in addition to the phenomenon as of purple, namely the brain states in play and the causal interaction with surface reflectances. Do you think this pair of considerations dulls the distinction between the two cases?

4. Hi Barry, I don't think so. We can interpret the extra evidence phenomenally, as concerning what the paper looks like (squiggles of text in such-and-such a particular arrangement). What's important is that this is a different fact (even if also phenomenal) from the one we are forming a judgment about, namely how good the paper is.

In the purely perceptual case, we have no other experiences that speak to the length of the line. It is a direct judgment. The paper case is indirect or inferential; we have extra evidence to consider besides our opinion on the immediate question. That's the essential difference.

5. The issue I see is that, while we have this extra evidence in the paper case, the evidence is meaningless to us without some sort of inferential procedure -- we can say "we have this set of squiggly lines," but we can't say whether that is a good thing or a bad thing. Since there's no way to make any use of the evidence itself, for all practical purposes the possibly-biased person is in teh same boat with regard to the line and the paper.

The distinction is plausible insofar as the separation between evidence and conclusion in the paper case means that you can try out a *different* inferential procedure and compare the two. So, for example, your holistic impression of the paper says it's a C. You can then go through and scrutinize each component of what makes a good paper (proper use of sources, adequate length, etc.) and evaluate it individually then take the average -- and if that procedure comes out with "C" as well, you know you're not biased, whereas if it comes out with "A" you know one of them is biased (and can decide that the latter procedure is more trustworthy and award the A).

(And perhaps the same thing could be done with the line -- you could double-check your direct perceptions by using a ruler, or photoshopping the line a different color to see how long it looks, etc.)

6. How does this drug work, exactly? Does it just affect first impressions, or does it affect all-things-considered judgments as well? If it just affects first impressions, then it looks like you should give the paper an A, or maybe some mixture of A and C- determined by how likely you think it is that you've taken the drug. (Setting aside any ethical questions about grading papers at all in this situation.) If the drug affects all-things-considered judgments, then I don't see how you can be assured of doing the right thing. The best thing you can do is probably to be agnostic, and to search for an antidote to the drug. (Setting aside the inevitable questions about how you're going to get any useful information off PubMed, now that you've been exposed to the drug.)

7. Wizard - it just affects the 'seemings' (intuitive judgments). Someone who knew they had taken the drug could reliably grade papers by giving them the opposite of what they seem to deserve.

But can you justify treating the paper case exactly like the ordinary perceptual case? You seem to be suggesting that the only relevant evidence is one's intuitive judgments about the paper, as in the perceptual case. But the point of my post is that there's a very important disanalogy here. What do you make of my claim: "My initial judgment of badness provides some evidence (due to the known likelihood of bias) that the paper is actually good. But that doesn't settle the matter, for surely the details of the paper itself are also relevant to what grade I ought to give it!"

Do you really think the known facts about what the paper says are strictly irrelevant to what grade one should give it?

8. I don't think that adjusting the grade to correct for suspected drug-induced bias counts as treating the known facts about what the paper says as "strictly irrelevant to what grade one should give it". What you want is to secure the right kind of reliable counterfactual dependence between paper content and grades. I think that's the sense in which you want paper content to be relevant.

9. Wizard - the question is, what descriptive facts determine the epistemic fact of what grade I ought to give a paper? If this is determined simply by -
(i) My initial judgment/inclination, and (ii) An adjustment for possible bias
- then obviously this is logically independent of "the known facts about what the paper says". So if we think these first-order facts are relevant, we must include them among the metaphysical base determinants of epistemic facts, i.e.:

(i) My initial judgment/inclination,
(ii) An adjustment for possible bias, and
(iii) What the paper actually says.

But now, depending on the weighting, it's entirely possible that determinant (iii) will outweigh the other two, such that I shouldn't, in fact, give the paper whatever grade is adjusted from my initial judgment. I should give the paper (something closer to) what it actually deserves.

I think this is clearer in my new bootstrapping example:

"Suppose I receive an incoherent C- paper, and irrationally misjudge it to deserve an A. If the author happens to be a black student and I have evidence that instructors are typically biased against black students, does that automatically mean that all things considered I should boost the grade and give it an A+? Surely not. My initial judgment may be some evidence that the paper warrants an A+. But it's not the only relevant fact. In particular, it can't make up for the fact that the details of the paper itself constitute overwhelming evidence that the paper should be given a C-. The presence of higher-order evidence doesn't excuse me from concluding what the first-order evidence demands."

Now, if you think I really ought, in such a case, to award the adjusted grade of A+, then it does indeed follow that you are treating the known facts about what the paper says as strictly irrelevant to what grade one should give it. Taking those facts into account mandates a far lower grade, in the imagined scenario.

Visitors: check my comments policy first.
Non-Blogger users: If the comment form isn't working for you, email me your comment and I can post it on your behalf. (If your comment is too long, first try breaking it into two parts.)

Note: only a member of this blog may post a comment.