I'm suspicious of the notion of metaphysical possibility, as I don't see how we could have any grasp at all on what is or isn't possible in the broadest sense. When I discussed the matter with Dave Chalmers, I was surprised to find that he was broadly sympathetic to these concerns. He suggested that ideal conceivability is all that we have to go on, and that if other philosophers have some other conception of metaphysical possibility in mind (i.e. one which does not take ideal conceivability as its standard), then he doesn't know what they're talking about either.
Now, I would want to simply avoid all talk of "metaphysical possibility", and just discuss "ideal conceivability" or "logical possibility" instead. This is because, as previously explained, I'm not convinced that there's any sense to be given to the notion of "what really could have been". Or even if there is, we surely aren't in a position to be making any claims about it. Unless one thinks that the human imagination has spooky magical powers, we don't seem to have any more reason to take ideal conceivability as the standard of "what really could have been" than, say, nomological possibility, or anything else for that matter.
But I do think that 'ideal conceivability' is philosophically useful and important, in relation to our explanatory practices. If something is conceptually necessary, then it stands in no further need of explanation. Given that grass is green, I do not need to further explain why grass is coloured, for it is inconceivable that the latter fact could fail to obtain given the former. But conceptual contingencies do demand further explanation. Take zombies, for instance. We can conceive of physical duplicates of ourselves that lack phenomenal conscious experience. So the physical facts are insufficient to explain our consciousness. (This is an epistemic, rather than metaphysical, conclusion.)
This conclusion does not worry me. In fact, even the conclusion that zombies are metaphysically possible wouldn't greatly worry me. That might merely lead to some sort of property dualism, which doesn't seem so bad. What I really want to avoid is substance dualism. My intuitions commit me very strongly to the view that it's nomologically impossible for a physical duplicate of myself to lack consciousness. Given the natural laws as they actually are, there can be no zombies. There is no further 'soul', 'spirit', or special substance that needs to be "injected" into a brain and body before it becomes a conscious person. I find such views ludicrous, and want to avoid them like the plague. But if the view is simply that our natural laws are somewhat special, in giving rise to consciousness when different natural laws would fail to do so (given physically identical duplicates), then that seems a far less extravagant claim, and doesn't concern me nearly so much. Whether that means I'm not so committed to physicalism as I'd previously thought, I'm not quite sure. (I guess it depends what one means by "physicalism".)