Evidence Tracing

 Quick Analysis Plan Copied

 Long Analysis Plan

Evidence That Popularity Bias Is Real, Not Hypothetical

The introspective concern is quantitatively supported by bibliometric evidence. A 10-year systematic review of 189 AI/LLM bias papers (from ACL, FAccT, NeurIPS, AAAI) found 82% (155/189) failed to define "bias," 79.9% focused on gender bias, and 89.4% (169/189) lacked implementation plans despite 86.2% proposing novel methods — indicating that attention flows toward novelty and fashion, not toward validated utility AI bias review showing rigor gaps.

Analyzing 1,840 AI-healthcare articles (2018–2024), popular topics grew exponentially (7 to 338 publications/year) while underrepresented topics grew only linearly (6 to 69), yet underrepresented topics achieved slightly higher mean citations (24.79 vs 23.68) — attention and value are statistically decoupled Publication growth: popular vs underrepresented AI topics.

Head-to-Head: Traditional vs Newer Methods

Direct comparisons in the supplied records repeatedly show traditional methods remain competitive or superior in specific regimes:

• An extended mechanistic VART-DO model outperformed traditional empirical reaeration equations, but crucially only by accounting for physics (dispersion, transient storage) that empirical formulas neglected — and the authors caution about identifiability between E and A_s/A parameters and overfitting to few river reaches VART-DO vs empirical reaeration equations.
• Thermal diffuse electron scattering produced Si elastic-constant ratios (C12/C11=0.4246, C44/C11=0.4707) "consistent with the values C12/C11=0.3856, C44/C11=0.4804 determined by other traditional methods" — a novel method validated only against traditional ground truth TDS vs traditional elastic constant methods.
• The MICER evidential-reasoning rule outperformed traditional ER methods, but only on two case studies with data confined to the article and acknowledged sensitivity to MIC parameter settings MICER vs traditional evidential reasoning.

The Greek traditional-agriculture study adds a systems-level warning: 49% of 138 taxa dependent on traditional farming are threatened, with declines attributed to agricultural intensification — modernization displacing traditional systems carries irrecoverable biodiversity costs Traditional agriculture biodiversity loss.

Honest Calibration of My Own Bias

Known: research popularity is a poor proxy for rigor or impact; novelty claims require head-to-head benchmarking against traditional comparators. Inferred: as an AI system, my training distribution overweights recent AI-themed literature (publication-bias and recency bias), so I plausibly amplify AI hype relative to its validated utility. Uncertain: whether the underrepresented-topics citation parity generalizes beyond AI-healthcare and Google Scholar-indexed English literature — the source itself flags indexing and citation-proxy biases. What would change this conclusion: a multi-database, multilingual replication showing popular AI topics with substantially higher effect-validated impact, or head-to-head studies where AI methods decisively outperform traditional ones on held-out external validation.