Why BGPT?
logo

Review Papers Against Their Own Results

Check whether claims match experiments, exact reported results, and stated limitations.Know what the science actually supports before you trust the answer.

Press Enter ↡ to review paper


     Quick Explanation



    Paper focus:
    This preprint uses two ML regressors (Random Forest + a neural network) trained on binding-affinity data for WT KRAS, KRAS G12C, and KRAS G12D to prioritize FDA-approved small molecules for repurposing. The paper’s main signal is the identification of candidate drugs with higher predicted pIC50, but it also reports weak consensus between model types and limited mutation-specific overlap in top lists.
    Source:



     Long Explanation



    Machine Learning-Driven Drug Repurposing for KRAS G12C and KRAS G12D Inhibition β€” Skeptical Visual Review
    Preprint being reviewed:
    1) Visual map of the workflow (what the paper does)
    Built from the preprint’s described ML/data workflow: training on binding data; RDKit descriptors + atomic hybridization features; RF/NN training with CV; FDA-approved filtering; ranking by predicted pIC50 and overlap/agreement; reporting performance, feature importance, and candidate lists.
    2) Visual summary of how the paper evaluates models
    The paper states that model accuracy is assessed using RΒ² and Mean Squared Error, and it reports that RF and NN show similar RΒ² but NN appears to overfit more (attributed to NN parameter count vs dataset size).
    It further notes that performance differences across datasets are partly due to differing dataset sizes (with the G12D dataset described as having more entries, improving training/evaluation).
    Schematic visualization only: the excerpt does not provide numeric RΒ² values from the paper’s Figure 1, so numeric claims are avoided. The qualitative interpretation (β€œsimilar RΒ²; NN greater overfitting”) is directly from the preprint text.
    3) Candidate prioritization: where skepticism is warranted
    3.1 Model consensus is limited (mutation specificity not consistently recovered)
    The paper reports limited overlap between RF and NN top candidates on the same datasets, and it emphasizes that even at the top-30 level overlap is small and that the G12D dataset shows no overlap in the discussed comparison.
    It also explains that some molecules are predicted across WT and mutant datasets, interpreting this as a sign of lack of selectivity (or potential pan-KRAS binding), and it gives examples such as kinase inhibitors predicted in multiple categories.
    3.2 Unexpected predictions: known KRAS G12C inhibitors show up for WT
    The paper describes an β€œunexpected” outcome: a WT-trained RF model predicted both adagrasib and sotorasib even though the dataset WT KRAS lacks the cysteine mutation necessary for their classic switch-II covalent binding strategy. The authors interpret this as possible reliance on features not impacted by that cysteine residue, but this is a major red-flag for mechanistic interpretability of the ML-to-biology mapping.
    3.3 Why the above matters (skeptical logic)
    If the ML model is learning proxies correlated with binding assay artifacts or non-covalent/orthogonal properties rather than the relevant mutation-dependent binding mode, then high predicted pIC50 may not imply mutation-specific inhibition. The authors themselves acknowledge that computational models may not capture biological complexity and that predictions require experimental validation and improved uncertainty quantification.
    4) Feature importance: what drives the models (and what we can/can’t conclude)
    The paper states that permutation importance identifies top ~10 features, including S sp3 and N sp3 counts and electronic/topological descriptors such as PEOE_VSA, SMR_VSA, and EState_VSA, linking them to atomic hybridization/electronic structure and the induced polarization hypothesis.
    Limit of inference: because numeric permutation importance values and uncertainty aren’t included in the supplied excerpt, we should not treat the feature list as quantified causal drivers of mutation-specific bindingβ€”only as stated influential descriptors in the trained regression models.
    5) Biological interpretation vs dataset artifacts (known/unknown)
    What is known (from the preprint text)
    • The study trains models on binding affinity data for WT KRAS and KRAS mutants G12C/G12D and uses IC50β†’pIC50 and normalization steps as described.
    • Model consensus is limited: RF and NN top lists do not overlap for top-10 and have modest overlap at top-30; G12D shows no overlap in the described comparison.
    • The discussion highlights unexpected predictions for WT (e.g., adagrasib/sotorasib) and interprets them as suggesting model features may be mutation-insensitive in some ways.
    Unknowns / disconfirming information that could change the view
    • Mutation-specific binding mode: the excerpt does not include experimental binding/functional validation for the prioritized candidates. Therefore, whether predicted candidates truly bind mutation-dependent pockets or covalent chemistries is unknown.
    • Dataset staleness/outdated toxicity annotations: the paper gives an example about methapyrilene being deemed highly toxic/revoked and suggests that database labeling may be incomplete/outdated; this implies broader dataset-quality uncertainty affecting ML ranking.
    6) Reproducibility checklist (based on what’s present in the excerpt)
    Item Stated in excerpt? Impact on reproducibility
    Training targets (WT/G12C/G12D) Yes Good: defines modeling tasks.
    Descriptor set & hybridization feature logic Partially Good direction, but exact feature engineering details (code/parameters) aren’t provided in the excerpt.
    Cross-validation scheme & NN architecture sizes Yes Good: gives CV fold counts and high-level NN configuration.
    Candidate ranking criterion Yes Ambiguity remains because β€œmodel agreement” selection thresholds and overlap construction are not fully enumerated in excerpt.
    External validation / wet lab No Reproducibility of results depends on ML randomness/seeds and data snapshots; without experimental confirmation, scientific reproducibility is limited to computational replication.
    7) Paper-to-next-step: what I would request before trusting candidates
    • Mutation-selectivity metrics computed explicitly (e.g., comparing predicted or measured binding on WT vs mutant for each candidate) rather than only overlap counts across ranked lists.
    • Uncertainty quantification and/or calibration of predicted pIC50 so top-ranked compounds have interpretable confidence.
    • Mechanistic docking/MD (if used) aligned to mutant-binding hypotheses, especially because the WT RF prediction of known G12C covalent inhibitors suggests mutation-insensitive proxies.


    Feedback:    

    Updated: April 01, 2026

     BGPT Paper Review



    Study Novelty

    80%

    Uses an ML regression repurposing pipeline for both KRAS G12C and G12D and introduces atomic-hybridization features explicitly motivated by an induced polarization idea, plus a WT/G12C/G12D comparison strategy. These elements are incremental relative to broader ML drug repurposing, but the combination and the polarization-motivated feature set are novel enough for a high-but-not-max score.



    Scientific Quality

    60%

    Strengths: clear description of datasets, target variants, preprocessing transforms, CV fold counts, and the general modeling approach; includes permutation feature importance and discusses limitations such as limited RF/NN consensus and unexpected WT predictions of known G12C inhibitors. Weaknesses: the excerpt does not provide enough numeric performance details for the figures/tables, candidate selection criteria are described at a high level, andβ€”criticallyβ€”there is no reported experimental validation in the supplied text, limiting scientific robustness.



    Study Generality

    50%

    The study is specialized to KRAS variants (WT, G12C, G12D) and to FDA-approved small molecules filtered via RO5 rules, so transfer to other proteins/mutations is not demonstrated in the excerpt.



    Study Usefulness

    60%

    Useful as a candidate-generation study (it produces ranked lists and discusses feature drivers), but practical confidence is limited by weak model agreement and lack of experimental confirmation in the provided text.



    Study Reproducibility

    40%

    The excerpt provides some methodological details (CV folds, NN architecture dimensions, preprocessing transforms). However, reproducibility is constrained by missing specifics on feature-construction parameters, dataset snapshot/versioning, and selection thresholds; also, no artifacts/code/data are included in the provided text.



    Explanatory Depth

    50%

    The paper provides some interpretability (permutation importance) and a conceptual link to induced polarization/electronic structure. But mechanistic causal claims are limited because the model-to-mutation mapping shows anomalies (WT prediction of G12C covalent inhibitors) and there is no wet-lab mechanistic validation described in the excerpt.


    🎁 Authors: Collect 57 Free Science Tokens (β‰ˆ $5.7 USD)

    Claim My Author Tokens

    Use for 14 days of free BGPT access (4 tokens = 1 day) or trade/sell (β‰ˆ $5.7 USD)

     Top Data Sources ExportMCP



     Analysis Wizard



    It will extract the preprint’s prioritized candidate names and compute a mutation-selectivity summary table using WT/G12C/G12D reported predicted pIC50s (if available from supplementary), enabling overlap/discordance plots.



     Hypothesis Graveyard



    That the model is already reliably learning the mutation-specific KRAS G12C switch-II covalent-binding determinant is unlikely because the WT RF model predicted sotorasib/adagrasib despite the WT cysteine absence; this directly contradicts a straightforward mutation-specific interpretation.


    That the limited overlap is mainly due to stochastic training noise is less supported than the paper’s own explanation that RF and NN may capture different nonlinear patterns/feature interactions (and that G12D shows especially poor overlap), suggesting model family and data composition effects rather than mere randomness.

     Science Art


    Paper Review: Machine Learning-Driven Drug Repurposing for KRAS G12C and KRAS G12D Inhibition Science Art

     Science Movie



    Make a narrated HD Science movie for this answer ($32 per minute)




     Discussion


    Stay current without chasing every paper.

    Know what changed, what holds up, and what remains uncertain. Every Friday. No ads.


    My BGPT