Review Papers Claim by Claim

1. Build groups of arrays by repeat, genus, and Cas type; remove groups lacking spacer overlap, producing a dataset of 518 groups and 5934 arrays. dataset size (518 groups, 5934 arrays)
2. For each group, run SpacerPlacer ancestral reconstruction twice: once on input spacer order, once on reversed spacer order (as integer labels), generating forward vs reverse likelihoods. forward vs reverse reconstructions
3. Compute likelihood ratio Lratio = log Lforward − log Lreverse and compare to an empirically chosen confidence threshold; output forward, reverse, or not determined. Lratio + confidence threshold rule

Reported result: CRISPR-evOr confidently predicts orientation for 28.3% of arrays in the considered CRISPRCasdb subset. 28.3% confident arrays (abstract)

Skeptical note: because the method is explicitly designed to output “not determined” when likelihoods are close (Lratio near 0), the confidence metric is necessarily information-limited by group diversity and evolutionary signal. why

The paper compares CRISPR-evOr to PAM-orientation (Vink et al.) and repeat/leader-based tools (CRISPRDirection and CRISPRstrand, as used in CRISPRCasFinder/CRISPRidentify). comparison targets (PAM, CRISPRDirection, CRISPRstrand)

Important context: CRISPRDirection uses a mixture of repeat and flanking features to predict transcribed strand; CRISPRstrand uses consensus repeat mutation structure. CRISPRDirection feature-based strand prediction CRISPRstrand repeat-orientation approach

The provided paper text explicitly states that for Cas type II-C, PAM-orientation and CRISPR-evOr disagree in 21.3% of cases when both methods are confident. II-C disagreement 21.3%

Counterpoint: the disagreement is measured in a conditional set (“confident cases”, within specific subtypes with enough data) so it does not directly translate to an unconditional error rate across all arrays. conditional evaluation framing

Benchmarking logic used by the paper

• They simulate datasets under two tree regimes: coalescent trees and “more realistic” core-genome-derived trees from CRISPRCasdb. simulation datasets A/B
• Using the original simulation tree with a decision threshold c=0, they report 97.8% correct predictions and describe the remaining errors as mostly wrong few-Lratio or Lratio=0 cases. 97.8% correct (simulations, original trees)
• When trees are inferred from the spacer arrays (harder regime), correctness drops to 95.5%, with 1.5% Lratio=0 and some fraction of wrong-orientation decisions tied to low-resolution/incorrect trees. 95.5% correct (simulations, inferred trees)

Skeptical interpretation: the reported simulation performance is extremely high, but simulation realism depends on how well the evolutionary block deletion model (and tree estimation assumptions) match real CRISPR dynamics. The paper itself notes complexity beyond the simple model (e.g., ectopic insertions) and that real data can be more complex than their model assumptions. model misspecification caveat

• CRISPR spacer acquisition is described as polarized (new spacers typically inserted at the leader end) and the paper treats this as the key universal property enabling orientation inference. polarized acquisition basis spacer acquisition mechanism review direction inference background
• The paper’s mechanism is intentionally sequence-marker independent: it avoids relying on leader/repeat/PAM calls for orientation decisions, instead comparing evolutionary likelihoods. sequence-marker independence claim

Strengths (evidence-linked)

• Orthogonality: unlike purely sequence-feature methods (leader/repeat/PAM), CRISPR-evOr uses evolutionary likelihood consistency under reversal, which can resolve conflicts where marker-based tools fail or disagree. orthogonal evolutionary advantage claim
• Explicit 'not determined': the model rejects low-information cases rather than forcing a guess, which is scientifically safer for downstream inference. conservative decision rule

Weak points / blind spots (what could break the logic)

• Tree estimation dependence: accuracy drops when trees are inferred from spacer arrays rather than provided from core genes, implying sensitivity to phylogenetic reconstruction quality. tree quality sensitivity
• Model misspecification: SpacerPlacer uses a block deletion model and a simplified insertion/deletion process; real CRISPR evolution may include ectopic insertions and other complex events, potentially biasing likelihood comparisons. evolutionary mechanism complexity caveat
• Database sampling bias: confidence may be limited by availability of closely related, diverse groups; the paper suggests 0-likelihood cases may arise from under-diversity and sampling biases. sampling bias explanation