Interactions of neanderthals and modern humans: what can be inferred from mitochondrial DNA?

Alternatives to the Wright-Fisher model: the robustness of mitochondrial Eve dating

Methods of calculating the distributions of the time to coalescence depend on the underlying model of population demography. In particular, the models assuming deterministic evolution of population size may not be applicable to populations evolving stochastically. Therefore the study of coalescence models involving stochastic demography is important for applications. One interesting approach which includes stochasticity is the O'Connell limit theory of genealogy in branching processes. Our paper explores how many generations are needed for the limiting distributions of O'Connell to become adequate approximations of exact distributions. We perform extensive simulations of slightly supercritical branching processes and compare the results to the O'Connell limits. Coalescent computations under the Wright-Fisher model are compared with limiting O'Connell results and with full genealogy-based predictions. These results are used to estimate the age of the so-called mitochondrial Eve, i.e., the root of the mitochondrial polymorphisms of the modern humans based on the DNA from humans and Neanderthal fossils.