Why BGPT?
logo

The Evidence Layer for Scientific AI

Ask questions across claims linked to experiments, exact results, limitations, and sources.Know what falsifiable science actually supports before you trust the answer.

Press Enter ↡ to search evidence


     Quick Explanation



    Concise verdict

    Large, carefully controlled observational study showing women are ~28% less likely than men to self-promote scholarly papers on Twitter (2013–2018), robust across fields and status strata, and with slightly smaller visibility returns when women do self-promote; methods, code, and data are publicly available but the study remains observational and platform-specific so causal claims require further testing. Key resources: open code and dataset for replication.

    Primary source: Peng et al. Nature Communications 2025 paper.





     Long Explanation



    Detailed paper review and critique

    1) What the paper does, in plain terms

    Peng and colleagues assemble a multidisciplinary dataset linking Altmetric mentions to Microsoft Academic Graph metadata to analyze 2,834,829 research papers published 2013–2018 and 3,503,674 unique authors (11,396,752 paper-author pairs) to answer whether authors self-promote on Twitter differently by gender and whether returns to self-promotion differ by gender. They infer gender from first names, match authors to Twitter accounts using a validated heuristic, and model (a) the probability that a given author tweeted their paper (mixed-effects logistic regressions with rich controls), and (b) the paper-level tweet mentions with and without self-promotion (negative binomial regression with interaction terms) to estimate returns. Their reproducibility resources include code on GitHub and datasets on Figshare.

    Primary claims are summarized succinctly in the paper and reproduced below with direct citations:

    • Women are ~28% less likely than men to self-promote their papers on Twitter after controls
    • The gap exists across all four broad research areas and increases with status/performance (higher impact journals, more productive authors, top affiliation rank)
    • Self-promotion raises paper tweet mentions for both genders, but the boost is slightly smaller when the promoter is a woman

    2) Strengths

    • Scale and scope: massive multi-million record dataset linking 23M tweets to 2.8M papers and 3.5M authors, enabling precise estimates and narrow confidence intervals
    • Careful statistical controls and multiple robustness checks (paper random effects, field controls, propensity-score matching, exclusion of East Asian names, gender-neutral names, solo-author subset, Twitter-active subset) that consistently reproduce the gap
    • Transparency and reproducibility resources: code repository and Figshare dataset are publicly available for replication and further tests

    3) Limitations, potential biases, and alternative explanations

    The authors acknowledge several limitations; below I expand and critique them with attention to what would be required to change the interpretation.

    1. Observational design and causal ambiguity. The study is observational: associations cannot alone prove why women self-promote less or why returns differ. For example, unmeasured confounders such as language style, disciplinary norms not captured by MAG fields, offline networks (who retweets/promotes), or temporal follower growth patterns could explain part of the measured gender differences. The authors attempt to control many covariates but residual confounding remains plausible
    2. Gender inference from names. Gender is inferred from first names with validation (F1 β‰ˆ 0.9), but name-based inference can misclassify nonbinary individuals and struggle with cross-cultural names and pseudonyms; misclassification could bias effect sizes if errors are differential by gender or region. The authors test excluding East Asian names and treating gender-neutral names separately, but name-inference remains a blind spot for full confidence
    3. Matching authors to Twitter accounts is imperfect. The heuristic matching achieves out-of-sample F1 > 0.95 per the paper, but if women disproportionately use pseudonyms, private accounts, or non-name usernames, the method could undercount women self-promotion and therefore bias toward observing a larger gap; the authors note this possibility
    4. Platform specificity and temporal scope. The study focuses on Twitter/X 2013–2018. Social-media ecosystems change rapidly (platform demographics, norms, moderation, and harassment levels), so generalization to other platforms (LinkedIn, Facebook, ResearchGate, Mastodon, TikTok) or later years (post-2018, post-X changes) is uncertain. The authors make this limitation explicit
    5. Measurement timing for follower counts and audience. Follower counts are taken at data collection time, not at publication date, so models that include follower counts as a covariate risk post-treatment bias: followers may be influenced by prior self-promotion and are not strictly exogenous. The authors therefore exclude many Twitter usage variables from main specifications to avoid endogeneity, which is appropriate but leaves audience dynamics less controlled
    6. Return metric is aggregated tweets rather than engagement by source or sentiment. The return is measured as change in total paper tweet mentions, which conflates direct engagement with self-promotional tweets and other amplification (media coverage, influencer reposts). While the authors also analyze mentions by scientists vs non-scientists and early self-promotion, a finer-grained causal test (e.g., randomized controlled trial of identical promotional messages assigned by gendered sender profiles) would be needed to isolate audience bias vs content/style differences vs network effects

    4) Key technical checks and reproducibility

    • Code is on GitHub with Jupyter notebooks, processing scripts, and plotting notebooks allowing replication and extension
    • Primary dataset for promotion is hosted on Figshare (3.71 GB) to enable re-analysis; Altmetric and MAG data sources are documented and available under their respective access programs

    5) Interpretation and policy-relevant implications

    Peng et al. argue that differential self-promotion practices and smaller visibility returns for women can contribute to measurable gender gaps in online visibility and downstream career markers (citations, invitations, awards). They emphasize institutional responses that "fix institutions not fix women" and suggest targeted interventions (sponsorship, awareness training, alternative evaluation metrics) and further causal research.

    My assessment: the evidence supports the existence of a measurable gendered association in self-promotion and in visibility returns on Twitter during 2013–2018, and the robustness checks increase confidence. However, policy prescriptions should be conservative about causal pathways: implementing interventions that shift evaluation away from raw online visibility is sensible as a precaution because visibility metrics can be gamed or biased; but direct interventions to change individuals behaviors should be tested (A/B trials or randomized interventions) before widescale adoption because unintended costs (e.g., increasing harassment risk or additional labor burdens disproportionately placed on women) are plausible.

    6) Concrete ways to strengthen or falsify the conclusions

    1. Randomized field experiment: assign matched researcher accounts (or controlled bot accounts) with identical promotion text but differing signaled gender to measure causal audience response differences; this would directly separate audience bias from content and network structure effects.
    2. Time-resolved follower/audience measurements: re-run analyses with follower counts measured at publication time (or use archived snapshots) to reduce post-treatment bias and better control for audience size when estimating returns.
    3. Cross-platform replication: apply the same (or similar) pipeline to LinkedIn, ResearchGate, Facebook, Mastodon, or discipline-specific platforms to test generality beyond Twitter/X and different moderation/harassment regimes.
    4. Survey-linked validation: combine observational data with surveys of authors about their promotion intent, perceived harassment risk, and use of pseudonymous accounts to evaluate measurement errors in matching and gender inference and to capture psychological drivers.

    7) Practical notes for readers wishing to reuse the data/code

    • Access promotion dataset on Figshare (Promotion) and review GitHub notebooks; if using Altmetric or MAG data, request appropriate access keys and cite providers per their terms
    • Be cautious about reusing follower counts measured post hoc; prefer archived snapshots or exclude potentially endogenous follower covariates or treat them as mediators rather than controls.

    8) Final critical synthesis

    Peng et al. present a rigorous large-scale observational study documenting a robust gender gap in scholarly self-promotion on Twitter between 2013 and 2018 and a modestly smaller visibility return when women self-promote. The strengths include scale, careful modeling, and public code/data. The main caveats are observational design, measurement limitations for gender and account matching, single-platform focus, and endogeneity concerns for some covariates (follower counts). The paper substantially advances measurement of online scholarly self-promotion and provides a reproducible foundation for causal follow-ups that I recommend. Overall confidence in the core descriptive result (a gender gap in self-promotion on Twitter in 2013–2018) is high; confidence that observed associations fully explain downstream career inequalities is moderate and requires experimental or longitudinal causal tests.

    Useful links and reproduction resources

    Replication starter: To reproduce Fig.2A predictions, run notebook 1_1_reg_data_alt_papers_basic_control.ipynb from the GitHub repo; the authors provide notebooks mapping to main analyses. See the GitHub repo for notebook names and ordering.

    If you want I can (a) produce re-created plots from the paper using the Figshare sample metadata, (b) draft a protocol for an experiment that causally tests audience bias, or (c) generate code to re-run regressions with alternative gender-inference rules; choose below.









    Feedback:    

    Updated: September 20, 2025

     BGPT Paper Review



    Study Novelty

    90%

    Large-scale empirical documentation of an online self-promotion gender gap with cross-field coverage and robust sensitivity analyses is novel in scale and practical implications for altmetrics and visibility studies.



    Scientific Quality

    80%

    High-quality data engineering and appropriate statistical models plus replication resources raise confidence; primary weaknesses are observational design, name-based gender inference, and platform-limited scope which the authors acknowledge.



    Study Generality

    80%

    Findings apply across four broad research areas and to high-status authors; however, generality is limited to Twitter and the 2013–2018 window and may not hold on other platforms or later periods.



    Study Usefulness

    90%

    Useful for scientometrics, research-evaluation policy, and institutions considering altmetric use; provides reproducible resources enabling follow-up and intervention design.



    Study Reproducibility

    90%

    High reproducibility: authors publish code and a Figshare dataset and document data sources (Altmetric, MAG); some access requires API keys but notebooks and processing scripts are provided to replicate analyses.



    Explanatory Depth

    70%

    The paper thoroughly documents patterns and associations and offers plausible mechanisms (harassment risk, language style, network effects) but cannot disentangle causal pathways; deeper mechanistic social-psychological tests are needed.


    🎁 Authors: Collect 500 Free Science Tokens (β‰ˆ $50.0 USD)

    Claim My Author Tokens

    Use for 125 days of free BGPT access (4 tokens = 1 day) or trade/sell (β‰ˆ $50.0 USD)

     Top Data Sources ExportMCP



     Analysis Wizard



    Not applicable to this social-science paper; no bioinformatics code provided.



     Hypothesis Graveyard



    Hypothesis that the gap is entirely due to Twitter adoption (women simply not on Twitter) is falsified because the gap persists among Twitter-active authors.


    Hypothesis that name-inference errors fully explain the gap is unlikely given validations (F1~0.9) and sensitivity tests excluding East Asian and gender-neutral names.

     Science Movie



    Make a narrated HD Science movie for this answer ($64 per minute)




     Discussion


    Stay current without chasing every paper.

    Know what changed, what holds up, and what remains uncertain. Every Friday. No ads.


    My BGPT