I had the pleasure yesterday of meeting Carl Shulman and Eliezer Yudkowsky (of Overcoming Bias fame) while they were in town, and hashing out some of our philosophical disagreements. It was interesting, because they're both very smart, and Eliezer's starkly materialist/reductionist ideology was shared by my past self. So I'm not entirely unsympathetic. But it was also frustrating in some respects, since he seemed to assume that any disagreement was simply due to a failure to appreciate his basic arguments, rather than a considered judgment that they aren't wholly compelling. So let me discuss a couple of issues in more detail, and attempt to lay out some of the reasons why I've shifted away from his blanket reductionism over the years.
(A) Fundamental Normativity. Eliezer holds that normative terms (e.g. 'should') are reducible to a particular framework of assessment -- roughly, the ultimate norms endorsed by the speaker. He calls this 'objective subjectivism', and it bears some similarity to the 'Objective Moral Relativism' I endorsed back in 2005.
I now find this unsatisfying, for several reasons. (1) The most obvious is that there's nothing really normative here, in the sense of an ideal that potentially outstrips any purely descriptive facts (incl. my current preferences and accepted norms). Though Eliezer wouldn't like to admit it, this is less a reduction than an elimination. Anti-realist maneuvers can save many of the appearances of normative practice, but its deepest aspirations are ultimately rejected. (2) His view implies that many normative disagreements are simply terminological; different people mean different things by the term 'ought', so they're simply talking past each other. This is a popular stance to take, especially among non-philosophers, but it is terribly superficial. See my 'Is Normativity Just Semantics?' for more detail. (3) We can go beyond the impoverished instrumental conception of rationality on which this view depends. Ultimate ends may themselves be assessed as more or less irrational. (I first realized this here.)
(B) Fundamental Mentality. My post on 'Dualist Explanations' outlines the case for property dualism, and defuses typical worries of the scientifically minded. Now, Eliezer seems to think that the causal inefficacy of non-physical phenomenal properties ("irreducible consciousness") is a knock-down argument against them. I once agreed, but again, have since changed my mind. My post, 'Why do you think you're conscious?' addresses this challenge in some detail.
There are some bullets to bite either way. I admit it's a bit odd to think that the words I type are not causally related to the facts I purport to describe. (That's an extreme way of putting it; do follow my above link to put this in perspective.) But, upon reflection, I find this commitment less absurd than denying the manifest reality of first-personal conscious experience (as reductive materialists like Dennett and Eliezer do), or engaging in the metaphysical contortions that non-reductive materialists must (see my 'dualist explanations' post).
(C) Epistemology. Eliezer writes:
When "pure thought" tells you that 1 + 1 = 2, "independently of any experience or observation", you are, in effect, observing your own brain as evidence.
I responded:
It's just fundamentally mistaken to conflate reasoning with "observing your own brain as evidence". For one thing, no amount of mere observation will suffice to bring us to a conclusion, as Lewis Carroll's tortoise taught us. Further, it mistakes content and vehicle. When I judge that p, and subsequently infer q, the basis for my inference is simply p - the proposition itself - and not the psychological fact that I judge that p. I could infer some things from the latter fact too, of course, but that's a very different matter.
In discussion, Eliezer emphasized the demands of (what I call) 'meta-coherence' between our first-order and higher-order beliefs. If you reason from p to q, but further believe that your reasoning in this instance was faulty or unreliable, then this should undermine your belief in q. I agree that reasoning presupposes that one's thought processes are reliable, and a subjectively convincing line of thought may be undermined by showing that the thinker was rationally incapacitated at the time (due to a deceptive drug, say). But presuppositions are not premises. So it simply doesn't follow that the following are equally good arguments:
(1) P, therefore Q
(2) If I were to think about it, I would conclude that Q. Therefore Q.
(Related issues are raised in my post on 'Meta-Evidence'. See also my argument for the inescapability of a priori justification.)
Concluding Remarks. Oops, this is too long already -- I've shifted my concluding thoughts to a new post.
