Review papers with raw data transparency

Brief critical summary

The preprint presents a pragmatic pipeline that extracts residue embeddings with ESM2 and projects them with UMAP to produce low dimensional variant spaces used for binary classification of missense variants in seven amyloidosis proteins; reported mean ROC AUC improves modestly over selected baselines and the method provides intuitive visualization of VUS in intercluster regions Main

Detailed review and critique

What the paper did

• Collected ClinVar missense variants for seven human amyloidosis related proteins and mapped to canonical UniProt sequences as described by the authors ClinVar
• Computed residue level embeddings with pretrained ESM2 T36 extracting 1280 dimension vectors for the mutated residue without fine tuning then applied UMAP (min_dist 0.0 metric euclidean variable n_neighbors) to reduce to 2D and used distance from wild type to threshold pathogenicity; trained classifiers (LR SVM RBF RF) with 5 fold stratified CV and compared ROC AUC to AlphaMissense and VESM++ Method

Major positive points

• The idea of combining PLM embeddings with nonlinear manifold learning to produce interpretable low dimensional variant spaces is conceptually sound and aligns with prior evidence that PLM embeddings capture structure and function signals ESM2
• The pipeline is computationally straightforward reproducible in concept using public ESM2 and umap implementations and standard scikit learn classifiers which makes it practical for exploratory variant interpretation in a clinical research setting Tools

Primary concerns and limitations

1. Label quality and training labels The authors treat likely benign and likely pathogenic labels as benign and pathogenic respectively which can introduce label noise and circularity because ClinVar labels vary in evidence strength and submitter concordance; downstream performance and claimed improvements can be driven by label artifacts rather than true functional separation. The paper acknowledges reliance on ClinVar but does not quantify label concordance or perform sensitivity analyses excluding low confidence submissions Label
2. Small focused protein set and generalizability The study is limited to seven amyloidosis related proteins which are clinically important but atypical in mutation patterns; the method performance may not generalize to other gene families or proteins with different evolutionary constraints. No external protein set or cross protein validation is reported to support generalization claims Protein
3. Method of classification and thresholds Classifying variants by distance from the wild type in UMAP space is an intuitive heuristic but raises concerns: UMAP is nonlinear and stochastic its distances are not isometric to original embedding space and vary with initialization and parameters (n_neighbors min_dist) making a single distance threshold brittle across proteins and datasets. Authors report empirically chosen UMAP params but do not explore robustness to different random seeds UMAP parameters or calibration of thresholds using independent sets UMAP
4. Comparative baselines and fairness AlphaMissense and VESM++ are reasonable comparators but the paper lacks clarity about whether baselines used the same variant filtering and training/test splits and whether those methods had coverage across all proteins; Table 2 mentions completeness differences but does not fully reconcile results across missing proteins which complicates claims that ESM2+UMAP is superior Baselines
5. Lack of experimental validation No wet lab functional assay or orthogonal clinical data is used to validate the predicted reclassifications or VUS placements; therefore it is unknown whether the clusters correspond to meaningful functional changes in protein stability activity or interaction patterns. The paper correctly frames this as exploratory but clinical claims should be restrained until validated No

Quantitative results summary

These numbers indicate modest improvement in aggregate but heterogenous per protein performance where some proteins (eg P02766) show large gains while others are near parity or worse. This heterogeneity argues for protein specific evaluation and for reporting confidence intervals and statistical tests on paired comparisons which are not fully shown in the preprint Performance

Reproducibility assessment

• Strengths: uses publicly available models ESM2 and UMAP and standard classifiers which are accessible ESM2
• Weaknesses: the preprint does not release code preprocessing scripts exact random seeds UMAP initialization parameters beyond the few reported nor the precise thresholds used for classification making strict reproduction and sensitivity checks difficult; authors should release code data splits and seed values to reach high reproducibility

How to improve the study

1. Report and filter ClinVar labels by review status and perform sensitivity analyses excluding likely labels or low evidence submissions to quantify label noise impact.
2. Release code data splits and random seeds and provide UMAP seeds and parameter sweeps to show robustness of distance thresholding.
3. Compare fairly to baselines by harmonizing variant sets coverage and by including additional established predictors (REVEL CADD PolyPhen SIFT) and report paired statistical tests (eg DeLong test for ROC comparison) with confidence intervals.
4. Use independent external datasets and ideally functional assay results (multiplexed assays of variant effect MAVEs when available) to validate predicted separations of benign and pathogenic clusters.
5. Consider projecting embeddings into higher dimensional UMAP spaces (eg 10D) before classification or using PCA or supervised manifold methods and show that the choice of UMAP 2D is not artificially driving results.

Final balanced take

The paper proposes a practical and interpretable pipeline combining ESM2 embeddings with UMAP projections that can help visualize and prioritize missense variants; the idea is well motivated and technically feasible given public tools however current evidence in the manuscript is exploratory limited by label noise small protein scope lack of robustness analyses and absence of orthogonal functional validation; with code release robustness checks and experimental validation this approach could become a useful component in variant interpretation toolkits rather than a standalone clinical predictor Authors