BGPT: Paper Review: High false sign rates in transcriptome-wide association studies

Explore by Goal

Quick Explanation Copied

Core finding: TWAS direction-of-effect is frequently wrong—reported FSR = 23% for plasma proteins and FSR = 33% for complex traits—only partially improved by colocalization filtering, with large further gains when restricting to trait-relevant tissues plus colocalization.

Long Explanation

Paper review (skeptical, evidence-based): High false sign rates in transcriptome-wide association studies

Paper DOI: https://doi.org/10.64898/2025.12.19.695550 • Date: Dec 20, 2025

Note: the paper states the 23% unfiltered plasma-protein FSR and explicit downstream values 11% and 5% after the stated restrictions; it does not provide an explicit single “coloc-filtered baseline” number in the supplied text excerpt.

Mechanistic hypothesis graph (failure mode the paper emphasizes)

What the paper does well (strengths)

Direct evaluation target: instead of only false discovery rate (FDR) for “causal genes,” the study quantifies false sign rate (FSR) for TWAS direction-of-effect using concrete expectations (expression→plasma protein; LoF/duplication→direction) in the same underlying cohort framework.
Two ground-truth polarizers: plasma proteins provide a mechanistic “expression→protein abundance” expectation, while complex traits use LoF burden direction-of-effect as the sign label.
Implementation transparency: TWAS performed with FUSION, tissue handling via a hierarchical majority vote across tissues/categories, and colocalization via coloc and coloc-SuSiE, with fine-mapping via SuSiE.

Skeptical critique & potential failure points

“Expectation” may not equal mechanistic truth: The plasma-protein ground truth assumes a generally positive relationship between genetically driven gene expression and protein abundance, then polarizes using LoF vs duplication. The paper reports strong empirical agreement for significant burden hits (98.6% and 100%), but that does not guarantee correctness for borderline/signless loci or for all genes/tissues.
Direction assignment via majority vote can be fragile: TWAS direction-of-effect is collapsed using hierarchical majority voting across tissue categories. This can systematically down-weight minority-but-causal tissue effects and can amplify discordance when multiple plausible tissue directions exist.
Colocalization filtering trades off sign accuracy vs recall: The paper explicitly shows that tighter PPH4 thresholds lower FSR while substantially reducing recall, and that fine-mapping (coloc-SuSiE) can sometimes worsen FSR at matched thresholds.
Single-causal assumptions vs multiple causal signals: coloc (approx Bayes factor) assumes one causal variant per locus, whereas SuSiE-based approaches accommodate multiple signals. Misfit here can bias which signal(s) are considered “shared,” affecting inferred direction.
Tissue relevance remains an unsolved problem for complex traits: The biggest FSR improvement is achieved for plasma proteins by restricting direction assignment to trait-relevant tissues. For complex traits, the paper notes that identifying the relevant tissue is unclear, which limits applying the same strategy.
LD-reference sensitivity: TWAS and colocalization often rely on LD from an external reference panel. The authors discuss LD mismatches and model uncertainty in imputed expression as possible technical contributors to sign discordance.

Evaluation logic (FSR as a diagnostic)

Step 1: TWAS predicts genetically regulated expression and tests association with the phenotype.

Step 2: Define an expected sign using: (i) LoF/duplication→protein abundance direction for plasma proteins; (ii) LoF burden sign for complex traits.

Step 3: Compute FSR = fraction where TWAS direction disagrees with expectation; then evaluate interventions (PPH4 filtering; trait-relevant tissue restriction) and observe FSR vs recall tradeoffs.

Values are taken from the paper excerpt: 85.3% before filtering and 43.3% after filtering at PPH4 ≥ 0.5 among proteins with at least one colocalizing TWAS association.

Actionable takeaways for TWAS users (interpreting direction-of-effect)

Treat TWAS sign as high-variance unless you explicitly quantify directional accuracy with an FSR-style benchmark; the paper demonstrates large discordance even when FDR-like expectations could look acceptable.
If using direction-of-effect downstream (e.g., mechanism inference), prefer strategies that incorporate trait-relevant tissues and filtering; for plasma proteins the paper reports a strong reduction to 11% and 5% under additional restrictions.
Expect a recall penalty when filtering with stringent PPH4 thresholds; the paper explicitly shows improved FSR accompanied by substantial recall loss.

Author reviews (navigate to BGPT)

Feedback:

Updated: April 02, 2026