Surely this post's title is a false dilemma. Nevertheless, it seems to be a popular defense of religious belief, to say that "atheism is a faith position too", e.g. because we need to assume that our senses are a reliable guide to reality [HT: OB], or because objective morality is no less "mysterious" than God, etc. This strikes me as not so much an argument as a negotiation: "Your beliefs are unjustified too, so I won't say anything if you don't!" Let's all just lower our epistemic standards and be one big irrational family.
Or let's not.
Not all axioms are created equal. Some assumptions are more reasonable than others. Given our commitment to making sense of the world as best we can, we are rationally obliged to believe in the preconditions of our success, i.e. that our basic methods of inquiry (science and reason) are on the right track. This is an entirely reasonable assumption to make, and in no way does it legitimize making further - arbitrary - assumptions in addition.
This becomes even clearer if we reject the foundationalist model of justification in favour of coherentism. One's maximally coherent belief-set would contain the claim that one's senses are generally reliable. It would not (atheists argue) contain the claims made by pop theism. They just don't fit together so well with everything else we (take ourselves to) know about the world.
If it's really the case that we have no good reason to believe something, then we shouldn't believe it. So it would be irresponsible to accept the theist's cease-fire; their reasoned criticisms should instead be welcomed! But, I would argue, we actually do have good reasons to believe in those other things (e.g. morality, the external world) in the alleged analogy. On the other hand, we don't have such good reasons to believe in pop theism. So, I think we should simply reject the analogies.
Granted, even atheists must make some assumptions. But, again, some assumptions are more reasonable than others. It's not always obvious what we should (most reasonably) believe. It's not always easy to avoid falling off into either extreme of complacent skepticism or complacent relativism. But of course this challenge calls for more, rather than less, critical epistemic discernment. Don't throw up your hands -- think!