Check Claims Before You Publish

Executive summary

Liam Brierley has produced a body of computationally rigorous work at the intersection of viral ecology, epidemiology, and machine learning. His publications and preprints demonstrate (1) reproducible code and data sharing, (2) careful handling of sampling and phylogenetic confounders, and (3) use of modern ML ensembles and transformer models for host and zoonotic inference. Areas needing strengthening include: clearer ground truth definition for negative labels in zoonosis prediction, prospective experimental validation of in silico predictions, and more explicit treatment of sampling bias and uncertainty quantification in some studies. The detailed evidence below cites his major relevant works and reproducible resources.

Primary evidence and assessment

1. Zoonotic prediction with transparent code and data

   The 2025 preprint An AI for an AI (DOI 10.1101/2025.09.16.676011) describes a segment-aware ensemble trained on curated influenza A genomes (618 zoonotic and 18,913 avian genomes, 120 subtypes) using clustering (Linclust/MMseqs2 at 70% id/coverage) to reduce phylogenetic overfitting, explicit feature sets spanning nucleotide kmers, codon/dinucleotide biases, amino acid composition and protein physicochemical descriptors, and a stacking meta-learner evaluated on held‑out subtypes achieving AUROC ~0.947 and F1 ~0.896 on subtype holdouts; code and full feature/model archives are publicly deposited on GitHub and Zenodo. The preprint also acknowledges limitations around imperfect ground-truth negatives and lack of wet-lab validation, which the authors document and make available for reanalysis AI for an AI influenza preprint

2. Transformer based host inference and community datasets

   HAVEN (DOI 10.1101/2025.06.09.658367) is a separate, independent effort by a multi‑group team including Brierley, presenting a pretrained viral protein transformer (1.2M viral proteins from UniRef90) fine‑tuned for virus host inference with prototype few‑shot modules, showing strong generalization to rare hosts and unseen viruses; all code and datasets (Zenodo) are available, enabling reproducibility and independent benchmarking HAVEN preprint and code

3. Contributions to preprint science and meta research

   Brierley is first or coauthor on influential analyses of preprints during COVID19, notably Tracking changes between preprint posting and journal publication during a pandemic (PLOS Biology 2022, DOI 10.1371/journal.pbio.3001285). That work used automated and manual annotations to quantify textual and figure changes between preprints and published versions, finding only modest changes in conclusions for most papers (7.2% non-COVID abstracts change; 17.2% COVID), and hence supporting preprints as largely robust signals during rapid response science; data and code were shared Preprint vs published PLOS Biology

4. Foundational viral ecology and epidemiology work

   Earlier collaborative work with M Woolhouse and colleagues produced high‑utility datasets and comparative analyses (e.g., Epidemiological characteristics of human-infective RNA viruses, Scientific Data 2018, DOI 10.1038/sdata.2018.17; and Assessing the Epidemic Potential of RNA and DNA Viruses, EID 2016, DOI 10.3201/eid2212.160123). These outputs are widely used in comparative risk analyses and demonstrate rigor in systematic literature curation and trait annotation Human infective RNA viruses datasetEID epidemic potential analysis

Strengths observed

• Reproducibility culture: multiple projects provide code repositories and Zenodo archives enabling reanalysis and reuse AI for an AI code and data
• Methodological diversity: uses classical statistical ecology, clustering de-biasing, modern ML ensembles, and transformer pretrained language models across projects (HAVEN) HAVEN approach
• Attention to sampling and phylogenetic confounding: explicit clustering and holdout by subtype in influenza project is good practice to test generalisation beyond phylogeny Clustering to mitigate phylogenetic bias

Limitations and blindspots

• Ground truth definition for negative labels in zoonosis/host prediction is inherently imperfect. The influenza work explicitly acknowledges that avian sequences labeled non-zoonotic may simply be unsampled zoonoses or lack epidemiological context; this weakens causal interpretation of model decisions and operational deployment without prospective validation Ground truth negative limitation
• No published wet-lab validation yet for sequence-derived zoonotic predictions. Authors call for experimental follow-up which is necessary before public‑health operationalization Need for experimental validation
• Potential funding conflict to disclose: AI for an AI lists CSL Seqirus UK Ltd funding which is declared; industry funding demands careful transparency though it does not invalidate methods if data and code are open for community scrutiny Industry funding disclosure
• Uncertainty quantification and calibration beyond AUROC/F1: reporting AUROC and F1 is useful but operational risk systems need calibrated probabilities, confidence intervals, and prospective false positive/negative impact analysis which are only partially addressed.

Reproducibility and data access

Code and data availability are explicitly provided for multiple projects: the influenza project GitHub repository GitHub ai_for_ai and Zenodo model archive Zenodo 17068424 support full reanalysis; HAVEN provides datasets via Zenodo and GitHub (see citations). These public resources considerably strengthen the credibility and scientific utility of the work Zenodo model archive

Overall judgement

Brierley demonstrates high competence in data curation, reproducible computational pipelines, and appropriate use of model validation schemes to address phylogenetic confounding. His work sits at the rigorous end of computational viral ecology and machine learning applied to zoonotic risk. The principal scientific gap is prospective biological validation and refined uncertainty quantification for operational use.

Quick practical recommendations for next steps

1. Design and run prospective prospective blind validation using newly collected avian influenza sequences and follow-up epidemiological surveillance to test model predictions.
2. Collaborate with wet‑lab virology groups to test a small panel of high predicted‑risk sequences for mammalian cell entry, replication, or receptor binding to ground computational signals in mechanism.
3. Publish calibration analyses and decision thresholds with cost‑sensitive evaluation (false positive cost vs false negative cost) for practical surveillance deployment.

Useful links and machine readable citations

• AI for an AI influenza preprint
• HAVEN preprint and code
• Preprint vs published PLOS Biology
• Human infective RNA viruses dataset
• EID epidemic potential analysis