It's a familiar enough idea. A young child, pained by another's insults, might be comforted by his mother with the suggestion, "You wouldn't want to be friends with such a nasty person anyway." Similarly, a person whose immense beauty was "all on the inside" (so to speak) might be comforted in face of romantic rejection with the friendly suggestion that "Such a shallow person doesn't deserve you anyway." In both cases, the sheer fact that someone would devalue you in such a way is presented as a reason for not taking their evaluations seriously. It's a comforting thought, to be sure. But is it a reasonable one?
We surely cannot be granted universal immunity here. Another's low opinion of us can sometimes be justified, after all. If we want an accurate self-conception, we must be willing to consider others' criticisms seriously, painful though this may be. On the other hand, this may only apply on a fairly specific level. A person may be sub-par in various particular respects. But I have trouble imagining a person being thoroughly worthless in themselves (or "in general", where this is not the same thing as averaging their particular qualities; rather it is the idea of 'bare personhood', or what lies beneath the particulars. Cf. Velleman on Love). It's all too easy for us to not truly see each other, to not appreciate all that is there. Whenever I've caught glimpses of that "all" in another, I can't help but feel a kind of awed respect for it, i.e. for them. So if someone were to devalue you generally, I'm inclined to think this must be an error on their part. They simply aren't seeing the light that would dazzle them if they could.
Still, such glimses are so rare that it would be unreasonable to blame someone for failing to see this light. Iris Murdoch once wrote that "love is the extremely difficult realisation that something other than oneself is real". There's something to that - it's very hard to fully comprehend the reality and value of others. Especially if we don't much try. Such blindness might be blameworthy in some specific circumstances, say if one failed to love a family member. ("You've known each other your whole lives! How could you still not see them?") But in most cases, that's simply the way of things. Every person deserves to be loved. But not every person can love them.
So rejection needn't entail wholesale despair. To be unloved - though undeniably painful - is not to be unlovable, for no person is ever the latter (in either its modal or normative sense). But we may accept this without thereby blaming the rejector. Their oversight is understandable. At most, the rejected might reasonably feel disappointed that the other wasn't more willing to "look" for them. But of course disappointment is not condemnation.
The above arguments are very general in scope, however. Perhaps we need instead to address the more specific circumstances described at the start. So let us move beyond consideration of self-love (the recognition - and hence valuing - of one's own personhood) to that of self-esteem (the valuing of one's own particular qualities). The person with high self-esteem takes themselves to be skilled or talented, or otherwise worthy of note, in various particular respects. These are taken to not be outweighed by particular vices, such as being a dirty, smelly, asshole. As the qualities in question are readily recognizable, one would expect others to hold you in similarly high esteem. They should thus want to be closer to you, want to get to know you better, and so forth. We might say that to have high self-esteem is simply to take oneself to warrant this kind of response in others. But then, insofar as the rejector fails to respond in the appropriate way to your obvious qualities, they might be held to demonstrate poor taste. ("Their loss," you might say.)
I've yet to bring into consideration the distinctive feature of romantic relationships, i.e. sexual attraction. It might well be that the other recognizes and is appropriately (albeit platonically) drawn to your obvious qualities. But they just think you're batshit ugly (nothing personal, I'm sure).
Is it "shallow" to avoid intimacy with someone solely on the basis of their physical features? I'm not sure what to make of this. On the one hand, I'm inclined to go pop-sentimentalist and say, "You can't help who you're attracted to. It's just the way things are." But on the other hand, I'm not convinced that's true. If one's spouse has a disfiguring accident, one wouldn't cease to love them, even romantically. I've read a bit from conservative Christians - like Macht - who have a rather "unromantic" view of romantic love as a choice rather than an uncontrollable feeling, and a lot of what they say here (as opposed to every other topic under the sun) sort of makes sense.
A key question here is whether romantic rejection entails a kind of devaluation. The pop-sentimentalist view provides a convenient way to deny this. You can say, "Oh, no, I think you're wonderful in every way, I just don't feel that way about you." - and that lack of feeling is taken to be a brute fact about the speaker, and not any kind of rational response to, or evaluation of, the rejected. But is this really plausible? I don't think our emotions are so disconnected from our reasoned appreciation of value. Indeed, the claim in quotes sounds deeply incoherent to my ear. Surely if you considered someone wonderful in every way, then you would feel "that way" about them? So the fact that you don't entails a kind of negative evaluation of them. For whatever reason - and there must be some reason, even if it's not transparent - you think they're not good enough for you.
That's why it's so painful to be rejected. There's no reason to feel bad about someone lacking some brute inexplicable feeling. But if they're judging you to be "sub-par", then that's clearly something different.
What if they were to make the more qualified claim: "I think you're wonderful in every non-physical way, and would love to be friends. I just think you're batshit ugly, and so don't feel 'that way' about you."
The discussion above suggests that it might still have been possible for you to have a great relationship. But that alone doesn't suffice to establish that the rejector is "shallow" for not wanting to try it. That would be to disallow the consideration of aesthetic values, and it isn't clear that there's any reasonable basis for such disqualification. Sure, aesthetic values shouldn't trump all; it would be shallow to consider them and nothing else. But it is surely reasonable to consider them alongside everything else ('depth' doesn't require blindness), which opens the possibility of their outweighing other values, depending on what precisely one is looking for in a romantic partner.
This complicates the rejection-as-criticism account, because aesthetic criticism seems less objective than other sorts. So perhaps we can cede some ground to the subjectivity of pop-sentimentalism after all. One could be judged to be batshit ugly, without this judgment necessarily reflecting poorly on either you or the evaluator (depending on who is "right"). Reasonable people may have differing tastes. Though it's always a pity when someone you like doesn't have a taste for you, I suppose.
Anyway, I'm all reflected out. I'll leave it to any commentators to figure out the appropriate response to rejection in light of the above. As far as unreflective intuition goes, "forget about it and move on" sounds pretty sensible to my ear. And they do say that our gut reactions are surprisingly reliable. Perhaps not quite so fun, though ;-).