[T]he likelihood that life will arise in this universe does not change even if there are millions of non-life-supporting universes "out there" (assuming that universes are closed systems that don't interfere with each other). If this reasoning is correct then, from our perspective, it is no more rational to believe that there are infinitely many universes than to believe there is just one--it's merely an aesthetic preference.
This strikes me as mistaken. It's true that the existence of other universes doesn't affect the probability of this universe - "u42" let's say - being capable of supporting life. But that isn't what needs explaining. We merely need to show how there could be some universe or other that supports life. The locative fact that it's ours in particular can be got for free, by appeal to the Anthropic Principle.
Compare: "Of all the planets in the universe, how is it that we ended up on one of the few capable of supporting life? Isn't this monumentally unlikely?" This sounds like a silly question. It's not as if we might instead have been asking the question from the blistered surface of Mercury. Living - and hence being somewhere capable of supporting life - is a precondition for even asking the question. Finding ourselves alive on a lifeless planet is not a possibility that ever needed to be ruled out.
A more troubling question would be: "How is it that there are any life-supporting planets at all?" For clearly the total absence of life is a coherent alternative, so we need some explanation of why that didn't come to be. (Otherwise we must appeal to brute chance or coincidence, but that isn't much of an explanation!) And here the appeal to multiple universes might help. If there are zillions of universes, the chance that life exists somewhere or other suddenly looks a lot more likely.
Note that once it has been explained how life could plausibly exist somewhere, it's no great mystery how come it exists in our universe in particular. Maybe it's unlikely that u42 would contain life, but we don't really care about u42. For explanatory purposes, we care about "the universe we're in", de dicto not de re. And the probability that whatever universe we're in contains life is pretty well certain.
Again, the Anthropic Principle by itself cannot solve the whole problem. It cannot address the question of why we (or life-supporting universes) exist at all. For that we need the multiverse hypothesis. The Anthropic Principle can only help with the locative question: presupposing that there are life-supporting locations, why do we find ourselves in one of them rather than somewhere else? Combine the two and we get a relatively satisfying explanation, I guess. (More so than any alternative I've yet come across, anyway.)
So, contra Dom, I think we could rationally believe the multiverse hypothesis, as an inference to the best explanation. We simply need to be clear on what it serves to explain. It does not explain why u42 supports life. That's not something I feel any need to explain. Rather, what the multiverse hypothesis initially explains is how any actual universe could support life. Conjoined with the anthropic principle, it can then explain why we find ourselves in a life-supporting universe.
(See also: Why does the universe exist?)