I should begin by conceding that the universe doesn't care what we do or how we fare. Value doesn't come pre-built into the fabric of the cosmos. If something matters, it's because it matters to us. And, indeed, things do matter to us. We have desires, and their fulfillment is of value to us.
So I think the existence of agent-relative non-moral value is particularly difficult to deny. Surely everyone would agree that some things can be better or worse for me. (I argue for the more specific claim that what is good for me is the fulfillment of my desires, but the specifics are not important here.) But once we grant the existence of non-moral value, we can easily construct an agent-objective aggregate, which could plausibly be equated with moral value.
There's a particularly nice argument to this effect in Stephen Darwall's Philosophical Ethics (p.125), inspired by J.S. Mill:
1) Morality, by its very nature, is concerned with what is good from the perspective of the moral community.
2) What is good from the perspective of the moral community is the greatest amount of what is good to the individuals comprising it.
3) What is good to any individual is that person's pleasure or happiness.
The details of (2) and (3) could be revised without damaging the overall force of this argument for naturalism. For instance, one might replace the maximization principle of (2) with a more Rawlsian 'maximin' principle, or something along those lines. The result would no longer be utilitarianism, but it would be no less naturalistic. Similarly, you can plug your favourite account of wellbeing (or non-moral value) into premise (3). The overall structure is flexible enough to handle it. Really all we need is the eminently plausible premise (1), in conjunction with any naturalistically specifiable account of "what is good from the perspective of the moral community" (which shouldn't be too difficult to provide).
When ethical naturalism is this easy, why would you ever resort to nihilism?