BGPT: Paper Review: Effect Size Estimation in Linear Mixed Models

Explore by Goal

Quick Explanation Copied

Core contribution

Defines an operational way to compute Cohen’s f² for linear mixed models (LMMs) using the estimated fixed-effects covariance (e.g., via vcov() in R/lme4), and shows how f² becomes model-, estimation-, and variance-component–dependent in practice.

Long Explanation

Paper Review (Science-Grade, Skeptical, Visual): “Effect Size Estimation in Linear Mixed Models”

Published: 2025-08-11. DOI: 10.1007/s40300-025-00295-w.

Visual map of what the paper does (logic-first)

The paper’s strategy is: start from the partitioned regression view of Cohen’s f² (based on an F statistic), then replace the unknown model covariance in the LMM setting by the estimated fixed-effects covariance returned by an LMM fit (e.g., using R/lme4).

1) The paper’s core definitions (with where uncertainty enters)

LMM setup (known-structure derivation first)

The paper uses the standard LMM representation y = Xβ + Zu + e with u and e centered and covariance matrices proportional to unknown parameters, yielding Cov(y)=σ²V with V = ZDZᵀ + T.

Operational f² depends on estimation of Cov(β)

If V were known, the fixed-effects covariance would be Cov(β)=σ²B and f² can be defined in a way that removes the F distribution’s scaling. But in practice V is not known; the paper therefore defines an operational f² by plugging in the estimated fixed-effects covariance (from an LMM fit), i.e., it computes f² from restriction matrix R1, the vector R1β, and the estimated Cov(β), including the necessary ν compensation because the estimated Cov(β) itself uses σ².

Key skepticism checkpoint: because Cov(β) depends on how variance components are estimated (and potentially on the assumed random-effects structure), the resulting f² is not a model-free invariant; it is “effect size for that particular fitted LMM.” The paper explicitly states this dependence.

Supporting background the paper relies on

Cohen’s effect size f² is introduced as a regression effect-size measure connected to an F test for linear hypotheses under multivariate normality.
The lme4 modeling machinery that returns fixed-effect variance-covariance is a standard route for operationalizing the paper’s formula in R.
For context on “R² for fixed effects” in LMMs, the paper’s discussion cross-references the fixed-effects R² literature (though its main computational path is via Cov(β)).

2) Results shown in the paper (only what we can support from the provided text)

Synthetic example (n=1000) — what the paper reports

No random effects case yields f² for the binary variable X1 conditional on X2: f² = 0.0017767.
With random effects Z, the paper defines k=σ²_u/σ² and reports a representative operational f² at the estimated k: f² = 0.0946626 when k is estimated as 0.4586.

Figure-like plot: f² vs k (endpoints only, since the text provides only these numeric anchors)

Interpretation constraint: the paper includes a full curve in its Fig. 4, but the provided full text only contains these numeric anchors; therefore the plot shows only the two supported points.

Figure-like plot: age effect f² across random-effects structures (reported values only)

The paper reports that the p-value for Age is 0.0286 in the first random-intercept model and that its computed f² is 0.0286; when adding random slopes for Stimulus and/or Participant, p-values increase (less evidence) and computed f² decreases (0.0163, then 0.0121).

3) Strengths (what’s scientifically valuable)

Operational clarity: the paper gives a direct recipe for computing an f²-like effect size in LMMs by leveraging variance-covariance output already produced by standard LMM fitting workflows.
Explicit dependence on model specification: instead of pretending effect sizes are invariant, the paper emphasizes that the computed f² depends on the LMM estimation procedure (variance component estimates, and therefore Cov(β)).
Connectivity to established computation: it positions its approach relative to existing mixed-model effect-size computation guidance (including R²-based approaches and Cohen’s f² pragmatics in mixed models).
Reproducibility intent: the paper indicates an associated GitHub repository with code and data for the LMM Cohen’s f² example.

4) Limitations & skeptical red flags (where results may mislead)

(A) Effect size is conditional on the fitted LMM

Because f² is computed from estimated Cov(β), it inherits sensitivity to: (i) chosen random-effects structure; (ii) variance component estimation; (iii) the restriction matrix for the parameter subset being targeted. The paper demonstrates sensitivity by comparing f² across random-effects inclusion in both its synthetic and rating examples.

(B) Operational formula uses estimated Cov(β); estimation error propagates

Even if the formula is algebraically coherent given Cov(β), the sampling variability and possible biases in variance component estimation can shift f². The paper explicitly notes that computed f² depends on the estimation procedures used for Cov(β), but the provided examples do not quantify uncertainty in f² itself (e.g., confidence intervals via resampling/Delta-method).

The paper mainly illustrates computation using an artificial dataset and a publicly available dataset (as stated) rather than a comprehensive empirical evaluation across many real applications and model-check regimes. This limits confidence that f² behaves desirably under common LMM perils in applied biology (e.g., mis-specified random effects, small numbers of groups, convergence issues). The broader LMM literature emphasizes that mixed-model inference can be sensitive to hierarchical specification and diagnostic checks.

(D) Assumptions remain: multivariate normality and LMM covariance structure

The LMM derivation relies on the LMM’s Gaussian/variance structure assumptions as used for the F-statistic distributional arguments, even though the operational computation works off Cov(β) estimates. If those assumptions fail in practice, f²—like the corresponding F tests—can reflect misspecification. (The paper itself does not claim robustness to violations; it frames the need for model-dependent operationalization.)

5) What you should do with this method (practical checklist)

State the target: clearly define which fixed-effect parameters are in the restriction (R1β). The paper’s f² is local to that hypothesis subset.
Report the LMM specification: because f² depends on random-effects structure and variance components, you need to report the model formula and how variance components were estimated.
Check LMM diagnostics: follow mixed-model diagnostic principles to reduce risk of misleading inference.
Quantify uncertainty (not emphasized here): consider adding uncertainty intervals for f² derived from Cov(β) estimation uncertainty (e.g., resampling or analytic approximation). The paper discusses dependence but does not center on f² confidence intervals.

6) Compact “scorecard” tables (from reported paper contents only)

Scenario	Model targeting	Reported f²	What changed
Synthetic (fixed only)	X1 given X2	0.0017767	No random-effects variance
Synthetic (with random effects)	X1 given X2 and Z	0.0946626	Estimated k=0.4586
Rating dataset	Age with (1\|Participant)	0.0286	Baseline random-intercept
Rating dataset	Age with (Age\|Stimulus)	0.0163	Adds random slope for Stimulus
Rating dataset	Age with (1\|Participant)+(Age\|Stimulus)	0.0121	Richer random-effects structure

All numeric values above are taken directly from the provided paper text.

Final critique: what’s most likely to matter in real applied work

This paper gives a mathematically grounded and computationally convenient way to produce an effect-size metric analogous to Cohen’s f² in LMMs, by converting the abstract effect-size definition into an operational estimator that uses Cov(β). That convenience is the strength—but the paper’s own results show the metric can change drastically when random-effects structure and variance component estimates change. Therefore, the correct scientific use is to treat f² as conditional on the specific LMM you fit, and to report enough modeling detail that readers can interpret what the f² is “local” to.

Author Reviews (jump to bespoke BGPT pages)

Feedback:

Updated: April 12, 2026