We cannot justifiably have ill will towards [...] wrong-doers, wishing things to go badly for them. Nor can we justifiably cease to have good will towards them, by ceasing to wish things to go well for them. We could at most be justified in ceasing to like these people, and trying, in morally acceptable ways, to have nothing to do with them.
This strikes me as a pretty unappealing view (especially if we extend it to the positive reactive attitudes: is gratitude likewise never warranted?). Parfit defends it by assuming that desert-involving moral responsibility requires the impossible, namely ultimate sourcehood: "To be responsible for our acts in some way that could make us deserve to suffer, we must be responsible for being in the relevant ways how we are."
I'm not sure why we should accept this theoretical assumption. I certainly don't find it anywhere near as plausible as the concrete intuitions about cosmic (in)justice: e.g. that all else equal, it's preferable that the vicious become miserable rather than happy. Granted, the notion of 'desert' is a bit slippery. But we needn't appeal to it here. We can instead appeal directly to our axiological intuitions concerning the combination of virtue/vice and welfare harms/benefits. It's just a bad thing when good things happen to bad people, just as it's bad when bad things happen to good people. Virtue-welfare mismatches are amongst the intrinsic bads. How one got to be vicious or virtuous has nothing to do with it.
Parfit doesn't really address this view. All he does is mention the weakest reason for holding it, namely religion:
Some people would reject [the ultimate sourcehood requirement]. There are people who believe that, though our wrong acts are merely events in time, and are causally inevitable, we could deserve to be sent by God to suffer in Hell. On such views, to deserve to suffer, we don't have to have any kind of contra-causal freedom, or to be in any way responsible for our own character, or for being how we are.
Of those who make such claims, some admit that they cannot understand how such claims could be true. God's justice, these people claim, is incomprehensible. [... But we] have no reason to expect such moral truths to be incomprehensible.
I do not find it "incomprehensible" to think that virtue-welfare mismatches might be intrinsically bad. I probably couldn't give any argument for this axiological claim, but then nor can you say much to argue for the claim that pain is intrinsically bad. It's just bedrock. But that is not the same as being incomprehensible.
Peter Singer suggests the stronger objection from evolutionary debunking: we can understand how selection pressures would give rise to norms of punishment in social animals like ourselves, so why think these norms have any independent truth to them? To this I'm inclined to respond, 'Why not?' Such genetic challenges do nothing to show that the belief in question is false; they merely prompt us to examine them more closely. If, on closer examination, we found that our concrete intuition conflicted with more plausible general principles (as in the case of various historical biases, etc.) then we'd have good grounds for revising our beliefs. But I find, on the contrary, that reflection on the ultimate sourcehood requirement (and the impossibility of anything ever satisfying it) instead undermines this conflicting theoretical belief, whereas the supporting theoretical principle -- that virtue/welfare mismatches are a bad thing -- seems as inherently plausible as ever. But maybe that's just me?