Alan Templeton, whose text Population Genetics and Microevolutionary Theory is right below Hartl & Clark in my book, recently published a strongly worded paper, Coherent and incoherent inference in phylogeography and human evolution. The possibility of statistical errors in published work is not shocking, I have heard that when statisticians are asked to sort through papers in medical genetics journals there are elementary errors in ~3/4 of those which have made it beyond peer review. That being said Templeton seems to be making a stronger case than simple refutation of basic errors, in particular he is suggesting that the “ABC” method which lay at the heart of the paper I reviewed last week is incoherent at the root. Here’s Templeton’s abstract:

A hypothesis is nested within a more general hypothesis when it is a special case of the more general hypothesis. Composite hypotheses consist of more than one component, and in many cases different composite hypotheses can share some but not all of these components and hence are overlapping. In statistics, coherent measures of fit of nested and overlapping composite hypotheses are technically those measures that are consistent with the constraints of formal logic. For example, the probability of the nested special case must be less than or equal to the probability of the general model within which the special case is nested. Any statistic that assigns greater probability to the special case is said to be incoherent. An example of incoherence is shown in human evolution, for which the approximate Bayesian computation (ABC) method assigned a probability to a model of human evolution that was a thousand-fold larger than a more general model within which the first model was fully nested. Possible causes of this incoherence are identified, and corrections and restrictions are suggested to make ABC and similar methods coherent. Another coalescent-based method, nested clade phylogeographic analysis, is coherent and also allows the testing of individual components of composite hypotheses, another attribute lacking in ABC and other coalescent-simulation approaches. Incoherence is a highly undesirable property because it means that the inference is mathematically incorrect and formally illogical, and the published incoherent inferences on human evolution that favor the out-of-Africa replacement hypothesis have no statistical or logical validity.

The method which Templeton favors is naturally one which he has pushed in the past. In any case, I don’t know the statistical details well enough to comment with much knowledge, but I see that a statistician has responded to Templeton already, so I would recommend checking that out. I immediately went looking for responses because the paper uses really strong and dismissive language, and I am somewhat wary of that sort of thing when attempting to tear down the fundamentals of a whole field of research (I want to emphasize that overall I enjoy Templeton’s work, but the paper reminded me a bit too much of Jerry Fodor). His citation of Popper in particular seems an appeal to authority that aims to convince the non-statisticians in the audience, and I don’t see the point of that besides rhetorical utility. I do tend to accept somewhat Templeton’s critique of models which assume very little gene flow between hominin populations before the Out-of-Africa migration, though from what I can tell it does seem that Africa has had relatively little back-migration south of the Sahara over the past 50,000 years, so perhaps this is an older dynamic as well. I am cautiously optimistic that DNA extraction from fossils themselves may put to bed some of these arguments over the dance of parameters, though naturally interpretation is always an issue outside of pure mathematics.

For what it’s worth, here’s the model which Templeton’s method favors:

The thin lines represent continuous gene flow between populations, and the thick lines extremely strong demographic & genetic pulses which overwhelm the genetic structure status quo periodically. I have implied something similar as operative on the smaller scale of H. sapiens sapiens.

Citation: Coherent and incoherent inference in phylogeography and human evolution, PNAS 2010 107 (14) 6376-6381; doi:10.1073/pnas.0910647107