Review Papers Claim by Claim

Interpretation, strengths, and core limitations (visual → concise)

• What the data show: Median COVID preprint-distributed papers had higher median citation ratios than directly submitted non‑COVID papers in 2020 (citation impact 1.52 vs 1.42), and the preprint advantage was most pronounced in 2020 when COVID research was nascent and attention concentrated Tsunoda et al. (2024) — medians & journal list
• Methodological strengths: uses server APIs (bioRxiv/medRxiv and PubMed E‑utilities) and transparent indicators (paper ratio, citation ratio, citation impact) allowing per‑journal comparisons over time Tsunoda et al. (2024) — methods
• Primary limitations / blindspots (critical):
  1. Confounding by attention and media: COVID preprints in early 2020 received broad media and policy attention — citations may reflect social/urgent interest rather than scientific quality Fleerackers et al. (2022) — media and preprints
  2. Selection bias: authors limited analysis to 110 journals with complete data; journals excluded could systematically differ (specialty, language, indexing), biasing observed effects Tsunoda et al. (2024) — selection note
  3. Metric validity: PubMed 'Cited by' counts are convenient but differ from Clarivate/Scopus/Dimensions; citation practices vary across fields and time (field size, review articles), so CR/PR may partially reflect field composition rather than pure preprint effect Sadegh & colleagues (2023) — metric caution
  4. Omitted covariates: no regression adjusting for journal impact factor, article type (methods, reviews), author prominence, or open‑access status; Table 2 lists high‑CI journals but inference on causation requires multivariable control Tsunoda et al. (2024b) — related larger analysis
• Reproducibility note: authors provide method descriptions (APIs, selection window), but raw journal‑level data and code were not attached; reproducibility would be strengthened by sharing the extraction scripts and per‑journal csvs (PubMed API calls are feasible) Tsunoda et al. (2024) — data availability

Concrete suggestions to strengthen the work

1. Share extraction scripts + per‑journal CSVs (PubMed ID lists, preprint linkages) to enable full reproduction.
2. Run multivariable models (e.g., negative binomial or zero‑inflated regressions) predicting citations controlling for journal IF, article type (original vs review), time since publication, OA status, author country, and media/Altmetric activity.
3. Compare across citation sources (PubMed cited‑by vs Scopus/Dimensions/Web of Science) to confirm robustness across databases.
4. Analyze time‑to‑citation curves (survival / cumulative citation trajectories) to disentangle early attention vs long‑term sustained impact.

How to falsify the study's main claim

Demonstrate—using the same journals and time window but (a) controlling for journal impact factors, article types, and time since publication; and (b) using an alternative citation database (Scopus or WoS)—that preprint distribution has no independent positive association with later citation counts (i.e., adjusted incidence rate ratio ≤1). This is the stated falsification pathway provided by the authors (compare preprint vs direct submission controlling for confounders) Tsunoda et al. (2024) — falsification remark

Confidence and final judgment

The observed descriptive patterns (higher median CR and CI for COVID preprint‑distributed papers, especially in 2020) are credible and align with parallel studies showing preprints increase visibility and early citations during the pandemic; however, causal claims that preprint dissemination per se produced higher scholarly impact remain tentative without multivariable adjustment and external database replication Tsunoda et al. (2024b) — corroborating dataset