Jason Brennan's 'Scepticism about Philosophy' raises a very interesting challenge, arguing that "[w]idespread disagreement shows that pursuing philosophy is not a reliable method of discovering true answers to philosophical questions."
I want to explore three ways of responding to the challenge (I should flag that these extend beyond the scope of Jason's paper, and so should be taken as "semi-related thoughts" rather than "objections"). We may question (1) whether there is any better alternative to doing philosophy as best we can -- however unreliable that may be; (2) the extent to which "discovering true answers" is an important goal of philosophy; and (3) whether "philosophy" in general is the relevant reference class for assessing reliability, or whether we should instead be assessing a range of more fine-grained methods.
(1) The Alternative: Jason imagines an agnostic who is equally interested in achieving true beliefs and avoiding false ones. He concludes that the agnostic has no epistemic reason to engage in philosophy, the alternative being to retain their default agnosticism.
How should we understand this agnosticism? It might involve (i) suspending all judgment, or else (ii) positively assigning balanced credences across competing philosophical views. My old post, Why Suspend Judgment?, argues that there's something odd, even unattractive, about the former attitude (especially, I think, when applied so broadly!). So let's consider the second case. We have an agent with fairly balanced priors across competing philosophical theories. Does she have epistemic reason to engage in philosophical reflection?
I think that she plausibly does. By ironing out any internal inconsistencies, and coming to a more informed understanding of the issues, she can only end up in an epistemic position that is an improvement by her own lights. And even taking an "outside view", it seems hard to deny that philosophical reflection could be expected to do at least slightly more epistemic good than harm. (Though people might disagree about some of the specifics, philosophy has surely solved at least some problems. So long as philosophical reflection isn't likely to positively lead you astray on other issues, the cases where it is uncontroversially helpful suggest that there is some epistemic benefit here.)
So, while granting the point that philosophical reflection is extremely fallible, I'm inclined to think that it is the best we can do, epistemically speaking, and better than nothing. So, all else equal, we should always consider it epistemically worthwhile to engage in more philosophical reflection rather than less. We have no reason to think it'll lead us to a worse balance of truth-over-falsehoods, and some reason to expect it to lead us to a (slightly) better such balance.
(2) How much does (non-conditional) truth matter? Jason assumes that "discovering true answers" is the primary epistemic goal of philosophy, though of course it may have various non-epistemic virtues -- it's a lot of fun, etc. And in this respect, I think we need to at least partly agree with Jason: "discovering true answers" to its central questions is certainly one important goal of philosophy, and one that -- despite my above comments -- it does a somewhat disappointing job at. (It may be our best available method whilst still being not as good as we might have hoped!)
But even so far as epistemic value is concerned, I've previously argued that understanding is more important than mere truth. And aiding our understanding is I think something that philosophy is very good at. We come to better understand the inter-relations between various views (one might rephrase this in terms of "discovering the truth" of a whole raft of conditionals). And, while it's unlikely that all our philosophical views are correct, at least some of them probably are, and in those cases philosophical reflection will likely have given us a better understanding of why they are correct.
(Perhaps by valuing this understanding more than I disvalue the "false understanding" that accompanies our false philosophical beliefs, I am simply contradicting the stipulated values of Jason's imagined agnostic. But I think these are the right epistemic values to have! In part, I guess, because I think even our false philosophical beliefs involve some genuine understanding of the issues involved -- e.g. certain merits of the view, or what would justify believing it if it weren't for whatever defeaters we've mistakenly neglected. In other words, again, we might reasonably claim that philosophy reliably leads to knowledge of many important conditionals.)
(3) Finer-grained Methods. Philosophy as a whole is extremely diverse, and displays a high degree of dissensus. But various philosophical sub-groups enjoy greater consensus, which opens the possibility that some of them are highly reliable. Of course, we lack any neutral means for identifying which philosophical sub-group is really on the right track, so this doesn't help Jason's agnostic outsider decide which philosophical tradition to join. But it might be epistemically significant all the same. As I argue in Knowing What Matters, there may simply be an objective epistemic fact about which philosophical "starting points" are more rational than others, such that agents who use this fine-grained method of philosophical reflection starting from the right place are genuinely reliable and eligible to have their beliefs constitute knowledge -- despite the fact that there's no "neutral" way to distinguish them from unreliable philosophers. On this view, the correct fine-grained non-neutral method is constitutive of ideal rationality, and hence is, in an important sense, the method that an agent rationally ought to be using.
Of course, since we lack any neutral method for identifying which philosophical school exemplifies the correct fine-grained method, in practice the best we can do is to implement the (neutrally described) method of philosophical reflection starting from our own priors -- whatever they may be. Some will be lucky enough to have this coincide with what is in fact (close enough to) the objectively rational method. Others will not be so lucky. But it's our best and only chance to believe as we ought, so I figure we should take it.