Evidence for paper review

Overview and Context

This paper introduces a novel Positive-Unlabeled (PU) learning framework, termed BIN-PU, aimed at predicting Compound-Protein Interactions (CPIs) for bacterial systems where curated negative samples are unavailable. The study addresses the significant challenge in bacterial CPI prediction arising from limited negative datasets, leveraging only positive samples to generate pseudo labels (both positive and negative) in order to train deep learning models Study Motivation and Dataset.

Methodological Innovations

• PU Learning Framework: BIN-PU diversifies the training set by generating unlabeled pairs from known positives and then categorizes them using multiple bins and averaged prediction scores. This binning strategy enables the method to effectively assign pseudo labels and overcome the lack of explicit negatives.
• Weighted Positive Loss Function: By introducing a loss function that emphasizes errors on truly positive samples, the framework ensures better calibration of prediction probabilities. The combined Binary Cross Entropy and weighted positive loss term help reduce false negatives, an inherent risk in PU learning approaches Loss Function Details.
• Validation Strategy: The method is validated numerically using F1-scores derived from cross-validation, comparisons with existing approaches such as PUCPI, and further validated using independent datasets from curated human CYP data and additional bacterial literature. The extensive experimental design uses both computational metrics (e.g., Spy Capture Rate) and wet-lab validations (biological assays such as HPLC and biophysical docking studies) for comprehensive verification Validation Strategy Overview.

Results and Impact

The BIN-PU framework significantly outperforms the benchmark PU method (PUCPI) across several deep learning backbone models, such as TransformerCPI, CPIprediction, and SSNet. Notably, improvements in F1-scores (with gains approximately in the range of 7%-9%) and high Spy Capture Rates (up to 93% for TransformerCPI) highlight the method's superior discriminative ability. The experimental validations further strengthen the claims by demonstrating that the predicted enzyme-substrate interactions are consistent with biological assays Performance Metrics and Experimental Validation.

Strengths and Limitations

• Strengths: The approach is innovative in its sole reliance on positive data, thereby bypassing the conventional requirement for curated negatives. Methodologically, the integration of multiple CPI models with a novel loss function, as well as the use of robust statistical validations and real wet-lab experiments, enhances the confidence in the predictive performance.
• Limitations: The paper acknowledges limitations concerning the availability of labeled data for different reaction types (inhibitors vs activators), and the potential biases introduced by relying on curated training data that may not capture the full spectrum of bacterial diversity. Additionally, there could be an over-reliance on predictions derived solely from positive data, which might not generalize across all bacterial systems Paper Limitations Discussion.

Conclusion and Future Directions

This paper makes a significant contribution by introducing an effective framework for predicting bacterial CPIs using only positive samples. Its multi-faceted validation approach and innovative loss function offer strong potential for applications in drug discovery, biocatalysis, and industrial biotechnology. Future work may include the integration of additional biochemical data, extension of the framework to other protein families, and exploration of more complex interaction networks to further generalize the findings.

Data and code availability on GitHub further enhance reproducibility, making this a valuable resource for the scientific community Data Availability Statement.