BGPT: Paper Review: Beyond Explained Variance: A Cautionary Tale of PCA

Explore by Goal

Quick Answer Copied

PCA’s “explained variance” can still yield a misleading 2D story for nonlinear data.

This paper re-analyses a fossil-teeth dataset and argues that the PCA 2D scatterplot’s apparent clustering does not correspond to true structure: t-SNE (with cosine distance) + persistent homology indicate a loop (ring-like) organization and an intrinsic dimensionality near 1, supported qualitatively by a circle-based generative model that predicts an arcsine-like distribution of cosine distances.

Long Answer

Paper Review: Beyond Explained Variance: A Cautionary Tale of PCA

Author: Gionni Marchetti.

Paper DOI: 10.48550/arXiv.2605.13520

Core question: Can a 2D PCA embedding be reliable for nonlinear-manifold structure even when PC1+PC2 explain most variance?

Visual 1 — PCA explained variance vs PCA-based rank heuristics

Data used by the paper: standardized fossil teeth matrix X* (ns=88 samples, n=9 variables).

The paper reports PC1=74.9% and PC2=18.8% (PC1+PC2=93.7%) for its PCA run, while prior work using R reports PC1=78.8% and PC2=16.7% (PC1+PC2=95.5%).

Interpretation claim: “high explained variance ≠ structure-preserving 2D embedding.” The paper uses this as its central caution.

Visual 2 — Topological evidence: bottleneck distances for H0/H1/H2

The paper compares persistent homology diagrams computed with Euclidean vs cosine distance; bottleneck distances are reported separately for H0, H1, and H2.

The paper’s numerical summary supports this qualitative claim: H2 (voids) is highly consistent across metrics (tiny bottleneck distance), while H0 and especially H1 show greater metric dependence.

Visual 3 — Intrinsic dimensionality (ID): what the paper infers and what PCA heuristics imply

The paper states that PCA-based heuristics yield an upper bound m*≈2 for a nonlinear manifold, and then interprets the true intrinsic dimensionality as m*=1.

Critical note: the manuscript’s logic is internally consistent (PCA heuristics → upper bound, then nonlinear-manifold bias → lower “true” m*), but this depends on the correctness and applicability of the PCA overestimation principle to this dataset and measurement setup. The paper does not (in the provided text) show a ground-truth manifold for direct falsification; it relies on concordance among t-SNE + PH + a circle-based distance-distribution fit.

Visual 4 — Model selection of cosine-distance distribution: Beta mixture vs Gaussian mixture (AIC/BIC)

The paper fits cosine-distance histograms using a two-component Beta mixture model (BMM) and a two-component Gaussian mixture model (GMM), reporting AIC/BIC values.

Preferential fit supports the paper’s qualitative narrative (arcsine-like distance distribution compatible with a circular/1D-directional structure), but mixture fit ≠ geometric ground truth. Many distributions can produce bimodality; AIC/BIC only compare among the specific candidate parametric families chosen in the fitting procedure.

What the paper gets right (strengths)

Methodological triangulation. Instead of trusting PCA’s 2D scatterplot, the paper uses multiple independent checks on structure: t-SNE embeddings (with careful metric choice), persistent homology (loop evidence in H1), and a directional-statistics probabilistic-geometric model predicting an arcsine-like cosine-distance distribution.
Explicit metric dependence. The paper emphasizes that switching from Euclidean to cosine distance materially changes t-SNE and persistent homology behavior, and then quantifies that sensitivity via bottleneck distances per homology class.
Quantitative summaries accompany qualitative claims. Examples: reported PCA variance percentages; bottleneck distances; and AIC/BIC comparisons for mixture model fits.

Skeptical critique (limitations & potential blind spots)

1) Metric choice can “manufacture structure.” The paper’s central positive claim (“ring exists”) depends on cosine distance being the right similarity notion. While that’s a legitimate scientific hypothesis, it is also a vulnerability: if cosine distance is tuned (explicitly or implicitly) to produce loops, then the loop may reflect the metric’s geometry rather than intrinsic data structure.

2) Small sample size (ns=88) + low-dimensional measurements (n=9) limit identifiability. The paper uses nine linear measurements per tooth across 88 specimens and then interprets low-dimensional topology/ID. With such a compact feature set, multiple mechanistic explanations can collapse into similar distance structures (e.g., measurement proxy geometry, scaling/standardization artifacts, or projection effects).

3) t-SNE is inherently sensitive to hyperparameters and initialization. The paper uses PCA initialization and a perplexity τp=29 and claims ring-like structures emerge for τp≥12. That supports robustness somewhat, but t-SNE does not provide a theorem guaranteeing that embeddings preserve topology in the same way across settings; it is a heuristic embedding optimized for neighbor probabilities.

4) Generative model is idealized; agreement is qualitative. The unit-circle model assumes points on an exact circle with uniform angle sampling, but the paper acknowledges empirical points are not exactly on a circle and are not uniformly sampled. Thus, the observed arcsine-like distance distribution provides qualitative consistency, not a decisive proof.

Where this could be falsified (in principle): the paper’s claim would be undermined if (i) cosine-distance PH and distance-distribution fits fail to reproduce the loop-like evidence on independently re-measured or re-parameterized versions of similar nonlinear datasets, or (ii) alternative metric choices that are justified independently (not tuned) yield similarly strong loop evidence for a different “metric-driven” explanation.

Overall scientific takeaway

The paper’s main lesson is not that PCA is “wrong,” but that explained variance is not a guarantee of a structure-faithful 2D map when the data may lie on a nonlinear manifold, and that distance/metric selection can dominate what structure is detected by downstream nonlinear methods and topology summaries.

Confidence: High that the paper’s internal results are coherent and numerically reported (PCA variance, PH bottleneck distances, mixture-model IC). Moderate that the loop should be interpreted as intrinsic rather than metric/feature-proxy induced, because the evidence is conditional on the cosine-distance choice and on an idealized generative story.

Bespoke BGPT follow-ups (optional)

Author reviews

Feedback:

Updated: July 01, 2026