[This is something of a manifesto for my current research project...]
Critics of consequentialism often object to how a consequentialist agent would (allegedly) think. They claim that the consequentialist agent is, in some sense, a bad character. Defenders of consequentialism typically dismiss such objections by citing the distinction between 'criteria of rightness' and 'decision procedures'. (Utility provides the criterion that determines the moral status of an act; it's a further question whether agents ought to attempt to calculate utilities themselves.) This is not entirely satisfactory. There remains a real objection here that needs to be addressed, not just dismissed. As I will explain, consequentialists still need to say something about what a 'rational' or fitting moral (consequentialist) agent would look like -- and when they do, this leaves room for others to object that the agent thus pictured is not in any sense morally 'rational' or non-instrumentally ideal.
To begin, we must distinguish two very different kinds of normative evaluation: the 'fortunate', and the 'fitting'. On the one hand, we can ask whether an agent's psychology is recommended by the normative theory as something to aim at -- roughly, whether it is desirable, or ought to be pursued, or such like. This is to ask whether it is a good or fortunate psychology to have. On the other hand, we can ask whether the agent's psychology embodies or "fits with" the normative theory -- roughly, whether the agent is responsive to the reasons posited by the theory: whether he desires what the theory says is desirable, etc. This is to ask whether the agent is, in a sense, rational or (as I will say) fit.
This distinction is illustrated in cases of 'rational irrationality', where the best disposition to have is one that embodies irrationality. Parfit's threat ignorer (for example) is irrational, but his psychology is rationally recommended or fortunate, since by being intrinsically defective in this way the agent is more likely to attain rational goods (he will no longer be vulnerable to threats or blackmail).
Similarly, a hedonist might say that only desires for pleasure are fitting, but it's fortunate -- better achieves the goal of pleasure -- to have other desires besides. Hedonists will think there's some sense in which an agent with other desires is rationally defective: they're desiring things which don't really warrant desire, after all. But, they'll say, it's fortunate to be defective in this way.
Finally, in case of ethics, we should likewise distinguish 'morally fortunate' from 'morally fitting' character. The fortunate character is that which serves to promote the good. The fitting character is that which embodies an orientation towards the good. This is the sense in which someone might have "good intentions", even if the intention has bad consequences, and so is unfortunate. Talk of "virtuous" character also plausibly concerns the 'fitting' mode of evaluation.
To aid your intuitive grip of the distinction, we can identify two families of evaluative terms. In the first family, we find terms like 'desirable', 'fortunate', 'good [on net]', and their opposites. These mark a kind of evaluation that is at least partly instrumental. In the second family, we find terms like 'rational', competent, virtuous/vicious, fitting/perverse, well-meaning, 'well-functioning'/'defective', and perhaps 'intrinsically good'. I should emphasize that, while these terms mark a kind of intrinsic evaluation (whether an agent is rational, virtuous, etc., does not depend on the outside world), one needn't think that there is any value to being in this fitting state. That's a substantive axiological question.
We're now in position to distinguish two anti-consequentialist objections. One claims that the fitting consequentialist psychology is unfortunate or 'self-defeating'. This is a very poor objection, as I explain in my old post, 'What's wrong with self-effacing moral theories?'
But when deontologists complain about the bad character of a committed consequentialist agent, there is something else that they might mean. They might mean that the fitting consequentialist psychology is (contrary to the consequentialist's claims) not actually morally fitting. For example, they argue that the "ideally rational/virtuous" consequentialist agent is incapable of friendship or commitment to projects -- but, they add, this seems like an intrinsic defect: surely genuine virtue and rationality are not incompatible with these important goods. So, they conclude, the consequentialist's conception of rationality (virtue, fittingness) must be in error.
This objection is the real challenge. Consequentialists have typically neglected it, because they have focused exclusively on evaluations of fortunateness. They haven't appreciated that their theory also commits them to a conception of the morally fitting agent. To take up this challenge, we must either (i) bite the bullet and insist that what the deontologist identifies as moral 'defects' are not really so, or else (ii) argue that, properly understood, the fitting consequentialist agent would not in fact possess the identified defect. (See, for example, my response to Stocker: 'Satisficing and Salience'.)
Of course, the first step towards a solution is recognizing that you have a problem.
[One might understand the developers of 'indirect' or 'sophisticated' consequentialism as working in this vein. But they have not always been clear about whether their theory commends the 'indirect' decision procedure as fitting or merely fortunate. Hence my previous post exploring the relation between sophisticated consequentialism and 'rational irrationality'.]