Because utilitarianism denies the (moral) separateness of persons, it doesn't actually give equal concern to all persons. Rather, it gives equal concerns to all interests, and it thereby ignores the way in which various interests (and the thwarting or satisfaction of such interests) are integrated into individual human lives.
It only makes sense to be concerned with interests, thus detached from their integral role in individual, separate lives, if we assume there is some aggregate super-life to which they ultimately belong. Otherwise the utilitarian has mistaken the importance of people's interests within their lives for the importance of people's interests tout court.
This may not be fair, and I'll be interested to see your reply.
I'm not sure that it's even possible to dis-integrate a welfare 'interest' from the life to which it belongs. (And if it's not possible, then utilitarians can't be doing it!) Rather, I think all the relevant facts about the role of the interest, as integrated in the life, are already built into the weight of the interest itself.
For example, consider my interest in dinner, in contrast to a starving man's interest in dinner. His interest in dinner will be much the greater, presumably because of the way it relates to the rest of his life. (Whether these interests are fulfilled or not will have a much more significant impact on his life than mine.) Utilitarianism takes all this into account, of course. If some interest plays an important role in one's life, then that makes it an important interest, and utilitarianism will recognize it as such. So I'm not sure how to understand the objection that's being suggested here.