What things are like non-naturally is not relevant to our normative judgments in the way we would expect them to be if such judgments were beliefs about those sorts of properties. Non-natural properties would belong to a menagerie of curiosities if we could map and catalog them, but our deepest normative convictions do not hang on how they are arranged.
Or, as Frank Jackson put it (From Metaphysics to Ethics, p.127):
[I]t is hard to see how the further properties [posited by non-naturalists] could be of any ethical significance. Are we supposed to take seriously someone who says, 'I see that this action will kill many and save no-one, but that is not enough to justify my not doing it; what really matters is that the action has an extra property that only ethical terms are suited to pick out'? In short, the extra properties would ethical 'idlers'.
I've previously suggested a couple of moves that I think non-naturalists should make to clarify their view in response to these kinds of concerns.
Firstly (in response to Jackson's formulation in particular), we should note that general considerations of moral fetishism already indicate that we should be motivated by the right-making features of acts, rather than by considerations of rightness itself. Only the latter is posited to be non-natural, so even non-naturalists will agree that it's natural features that should (suffice to) motivate us. Roughly speaking, the role of non-natural properties is not to be the reasons that justify or properly motivate our actions (or that are directly practically important), but rather to give content to the attribution of importance to these natural properties rather than others. Their role is, in this way, more theoretical than practical.
Second, the "menagerie of curiosities" objection seems to depend on the idea that so-called "non-natural" properties have some descriptive (i.e. not explicitly normative) essence by which we could track them and intelligibly raise questions about the normative significance of the items to which they attach. Bedke invites us to "imagine that God hands you magic spectacles that reveal all non-natural properties and how they are distributed." The thought is that the observed distribution wouldn't tempt us to revise our moral beliefs, even if it differed from our antecedent beliefs about the distribution of moral properties. (Suppose that pleasure and pain were revealed to have all the same non-natural properties, or none at all.)
This thought experiment makes sense if non-natural properties are imagined to be a kind of ghostly ectoplasm. We can then ask: Why think that this ghostly ectoplasm tracks anything of normative significance? But non-naturalists will agree that any such independently identifiable properties can't be inherently normative: Our agreement on this point is precisely why we're not metaethical naturalists! (Recall that Moore's so-called "naturalistic fallacy" applies equally to super-naturalistic reductions. The point is that normative properties are primitive, irreducible, or purely normative, such that they cannot be adequately captured or characterized via any non-normative guise at all.) But if the non-naturalist's posited properties have no non-normative guises at all, then the only way to perceive them is to perceive them as normative: e.g., to put on magic spectacles that reveal to you the distribution of goodness as such. And could you see the distribution of goodness, and see it as such, without seeing this as relevant to your beliefs about what things are good? That seems hard to make sense of.
Of course, we can imagine an oracle giving us generic info about non-natural properties, e.g. that pleasure and pain have no such properties. This should not cause us to revise our normative beliefs that pleasure is good and pain is bad. But that's just because we should be more confident of these normative claims than we are of our metaethical views. That is, we would properly take the oracle's pronouncement as evidence that non-naturalism is false -- in just the same way that oracular pronouncements that there are no non-physical properties should lead mind-body dualists to revise their theoretical beliefs rather than giving up the datum that we are in fact conscious.
So, that's my defense of non-naturalism (against "ethical idler" / "menagerie of curiosity" objections) in a nutshell. For more detail, see my work in progress, 'A Non-Natural Reason by Any Other Name...' (pdf).