BGPT: Paper Review: Rooting a phylogenetic tree with nonreversible substitution models

Explore by Goal

Quick Explanation Copied

Key result

Allowing an explicit nonstationary initial base-composition distribution (NONSTA; π not tied to equilibrium) substantially improves likelihood discrimination among candidate rooted trees—especially compared with the stationary nonreversible model (STA)—and can be used when no outgroup is available.

Evidence comes from multiple-gene mitochondrial + nuclear datasets where NONSTA more often assigns the known correct root branch and yields much larger log-likelihood differences than STA/REV.

Long Explanation

Paper Review (visual + critical): Rooting a phylogenetic tree with nonreversible substitution models

Bing Yap & Terry Speed, BMC Evolutionary Biology (published 2005) — evaluates rooting without outgroups by comparing NONSTA vs STA vs REV substitution processes.

1) Visualize the reported root-discrimination gains

The paper uses three verification groups where the correct root branch is assumed known: (1) human–chimp–gorilla (13 genes), (2) adds orangutan (13 genes), (3) human–mouse–chicken–frog (15 genes), plus a 9-primate mtDNA analysis on an assumed unrooted topology.

2) Visualize log-likelihood separation (NONSTA vs STA vs REV)

The paper argues that NONSTA often produces a noticeably higher log-likelihood for the correct rooted tree, while differences between STA and REV are often small.

3) Visualize “stage-gating” intuition: stationarity constraint can erase rooting signal

Conceptually, the stationary model ties the root base distribution π to the equilibrium distribution π_Q of the rate matrix Q; the nonstationary model allows π ≠ π_Q. The paper claims this decoupling yields a better explanation of data and greater root discrimination, likely because sequence composition evolves through time.

4) Critical evaluation (skeptical, evidence-based)

4.1 Strengths

Direct likelihood-based rooting without outgroups: the paper’s approach compares rooted candidates compatible with the same unrooted topology by ML fit and likelihood ranking, explicitly targeting rooting signal rather than only tree inference.
Nonstationarity is the only substantive model change relative to stationarity: by allowing π to be free (π ≠ π_Q), NONSTA increases model flexibility in a targeted way (base-composition dynamics).
Multiple datasets + gene-level replication: the verification studies analyze many mitochondrial protein-coding genes plus nuclear genes (albumin, c-myc) across primate groupings where the correct root branch is assumed known by the paper.

4.2 Limitations / blind spots

Model misspecification risk (equidistant leaves constraint): the paper explicitly notes NONSTA imposes a constraint: leaves at the same distance from the root must share the same base composition. This is “patently unrealistic” and may explain failures on some genes.
Root correctness is assumed “known” in verification groups: the paper treats certain root placements as correct for evaluation, but the reader should note this is not an independent experimental ground truth within the paper; it is a validation-by-consensus design.
Site-variation modeling is partially structured: the paper uses site classes primarily based on codon position (and an extra class for RNA-coding bases in the primate mtDNA dataset) and notes other sources (e.g., structure-driven rate variation) are not modeled; it mentions gamma distribution or HMM approaches as possible remedies.
Computational/optimization pitfalls may exist: the authors state they implemented simplex/gradient-based procedures and used multiple initial estimates to avoid local maxima, but the exact robustness across broader tree sizes is not established in the provided text.

4.3 Where the conclusion would be most vulnerable (what could disprove it)

If the same-root-discrimination advantage disappears under richer nonstationary/nonhomogeneous models that relax the “equidistant leaves” base-composition constraint, then NONSTA’s gains may partly reflect compensatory flexibility rather than uniquely informative directionality.
If likelihood-ratio testing assumptions (including distributional approximations) fail systematically, then “statistical significance” claims about NONSTA vs STA/REV would require caution; the paper itself notes chi-square distribution may be inappropriate but seems satisfactory in practice.

5) Practical takeaways (within the paper’s scope)

When outgroups are unavailable, NONSTA can be a straightforward likelihood-based rooting method over an unrooted topology by ranking candidate rooted trees.
Expect failure modes when lineage-specific base-composition evolution violates the equidistant leaves constraint; the paper suggests NONSTA could still be used to diagnose deviations and motivate richer lineage-specific models.

Author reviews

Click for deeper critique focused on the authors themselves (full-name author pages):

Feedback:

Updated: March 21, 2026