Fashionable or fancy appearance has no intrinsic value. The only value is to make yourself look comparatively better than others. If everyone dressed up, no-one would be any better off than if all had stuck to casual attire. In fact, everyone would be worse off, because less comfortable, not to mention the wasted time and effort. It's a mere rat race, so those who put effort into their appearance impose externalities on those who don't (by making them look worse). That's obviously bad, and there is no general benefit to justify imposing this costly transfer of status. So, we may conclude, it is immoral to 'dress up', follow fashion, put much effort into your appearance, etc.
N.B. This complaint does not extend to basic hygiene, since that is a non-comparative value: a world full of smelly people really is worse, in a way that a world of unkempt people is not.
Possible objections: I see two ways one might rebut the claim of 'no general benefit' here:
1. Insist that mere appearance is a non-comparative value after all. (Apparently cosmetic surgery gives lasting satisfaction, unlike most luxury purchases which people soon adjust to. This at least suggests it isn't comparison to one's recently past self that one values here. But it may still be the comparison to other people.) I remain skeptical.
2. Appeal to status pluralism. If some people care more about appearances than others, then maybe those who care can obtain great subjective benefits while the rest of us don't much care about the imposed "cost" of looking worse in comparison. I'm sympathetic to this line of thought -- the only flaw is that it ignores the run-on consequences: other people think appearances matter even if we don't, and so may treat us worse, and we certainly care about that.
Absent any more convincing objections, we seem led to the conclusion that caring for appearances is indeed a mere 'rat race', or Prisoner's Dilemma, such that deliberators in the Original Position (behind the veil of ignorance) would make a collective agreement not to start down that track. Is there anything to stop me drawing the convenient conclusion that dressing up is not just tiresome, but unjust?