Review papers by their claims

• Methods: 14 SVG/SVP approaches spanning graph-based, kernel/Gaussian-process-based, and hybrid designs, including SPARK-X and Moran’s I. 14-method benchmark scope
• Data/simulation: builds simulation datasets using scDesign3 with realistic spatial variability derived from real spatial transcriptomics references, and then blends spatial vs non-spatial mean components via a parameter α controlling spatial signal. scDesign3-based simulation strategy and α mixing scDesign3 simulation realism motivation
• Metrics: ranking accuracy (Kendall correlation), classification (auPRC), p-value calibration (KS distance under null by permuting spots), computational scalability (memory/runtime vs spot count), and downstream effect (spatial domain clustering using ARI for RNA-seq; CHAOS for ATAC-seq). Multi-metric evaluation design

The paper reports that most methods are poorly calibrated under null conditions (spots permuted to remove spatial variability), producing p-values that are either too conservative or too liberal, while SPARK and SPARK-X are among the best-calibrated in their evaluation. p-value miscalibration findings

Skeptical implication: If p-values are miscalibrated, then “significant SVG sets” chosen by thresholds like p<0.05 may yield biased false-positive/false-negative rates, which can contaminate downstream spatial domain clustering and biological downstream interpretation. The paper’s own recommendation is to rely on ranking thresholds (e.g., top-K genes) rather than p-value significance. Recommendation: ranking thresholds vs p-values

• The study explicitly addresses a known benchmarking weakness: simulated SVG/non-SVG labels can be inflated if simulations only use simplistic spatial pattern families. The authors’ scDesign3-driven strategy aims to incorporate more biologically plausible expression structure and variable spatial signal strength via α mixing. Motivation for more realistic simulation scDesign3 realism basis

• What we know: the paper uses scDesign3 simulations to define ground truth spatial variability, since gold-standard SVG ground truth in real tissues is not generally available. Why simulations are used for ground truth
• Skeptical risk: if scDesign3’s assumed generative mechanisms (e.g., statistical forms for marginals/joint distributions and spatial covariance structure) do not match the true biological data-generating process for certain tissues/platforms, the benchmarking could favor methods whose assumptions align more closely with that simulator. (This is not an accusation—just the inherent identifiability issue of simulation-only ground truth.) Simulation-based ground truth limitation

• The paper shows that high ranking accuracy does not imply correct uncertainty quantification (p-values). In other words, two methods can rank genes well yet still be unreliable for “statistical significance” unless calibration holds. Ranking vs calibration dissociation
• Practical consequence: users should treat “SVG p-values” as method-dependent and validate calibration with permuted-null checks when possible—especially before using p-value thresholds for downstream biology. Method-dependent p-value risk

• The paper attempts applying SVG methods to spatial ATAC-seq cell-by-peak matrices to detect SVPs, but many methods either fail computationally or underperform, with only some showing better clustering locality via CHAOS. SVG→SVP transfer difficulties
• Interpretation caution: because SVP “ground truth” is absent, the paper uses a proxy (spatial cluster continuity/locality via CHAOS). That proxy can help, but it is still not direct SVP correctness; a method could yield coherent clusters for reasons not strictly tied to biologically true SVPs. CHAOS proxy limitation for SVP

• Novelty: The paper’s main contribution is the breadth and structure of the benchmark (14 methods, multi-metric comparison, and a realistic simulation strategy grounded in scDesign3). Benchmark scope and realism claim
• Impact: The “SPARK-X top overall” conclusion is actionable for ranking-based workflows; the “p-value calibration mostly fails” finding is actionable for inferential/statistical workflows; and the “ATAC transfer fails” finding helps prioritize method development for SVPs. Reported practical conclusions

SpatialDE is a classic Gaussian-process-based spatial variability approach that underlies much subsequent work. SpatialDE modeling foundation

The benchmarking paper’s calibration warning is especially relevant here: even when GP-based models can detect spatial variability, correct p-value calibration can still fail depending on test construction and data preprocessing—so ranking-only conclusions should not be over-extended to inferential claims. Calibration results vs spatial model families