Review papers with raw data transparency

Immediate strengths (data-driven)

• Large sample for scale validation (N=660) with split-sample EFA (n=330) and CFA (n=330) — best practice for cross-validation Validation procedure summary
• High internal consistency across dimensions (α >= 0.898; PCK α=0.990) which supports scale reliability for this sample (but see caveats below).
• CFA indices (CFI=0.911; RMSEA=0.062; CMIN/DF=2.26) fall within commonly accepted thresholds, supporting model adequacy for these data.

Key concerns, limitations, and potential biases (critical)

1. Borderline sampling adequacy (KMO = 0.544). KMO values <0.6 are generally described as 'mediocre'—this weakens confidence that the correlation matrix is sufficiently factorable and raises risk of unstable factor recovery across samples KMO & sampling adequacy
2. Fit indices mixed — GFI is low. GFI=0.806 (authors call acceptable for complex models) but many psychometricians prefer >0.90; relying on PCFI/CFI to rescue model fit risks overstating confirmatory success. The model is acceptable but not excellent.
3. No cross-group measurement invariance reported. India is linguistically and educationally heterogeneous; without invariance across languages/regions/experience levels we cannot assume the factor structure or item functioning is equivalent across subgroups (limits comparisons and longitudinal tracking).
4. Self‑report + online convenience sampling risk. Data collected via Google Forms in English; inclusion required English literacy and full-time faculty status — excludes adjuncts, non‑English speakers and likely biases toward digitally engaged teachers. Social desirability / response-set effects may inflate reported competence.
5. High Cronbach's α (e.g., PCK α=0.990) can indicate item redundancy. Very high α may mean overlapping items and limited content breadth; authors should report inter-item correlations and item-total statistics to evaluate redundancy and scale efficiency.
6. No external validational criterion. Authors did not report concurrent/criterion validity (e.g., correlations with observed classroom AI use, student outcomes, or AI-usage logs), so predictive validity remains unknown.
7. Data availability and code not provided. Reproducibility is limited when raw responses, codebooks, or syntax are unavailable; authors did not share a data repository link in the paper (per article metadata).

Recreated/reinterpreted quantitative snapshots (from reported values)

(A) Variance explained by each extracted factor — authors reported cumulative 79.06% with first three factors explaining ~51%.

Interpretation & practical implications

What the scale is ready for:

• Population-level surveys of AI-related teacher knowledge in Indian higher education (descriptive profiling, needs assessment).
• Baseline and post‑professional‑development measurement if used within similar populations/language (but see invariance caveat).

What it does not yet support:

• High‑stakes comparisons across states, languages, or K‑12 vs higher‑education without invariance testing and language adaptation.
• Claims about classroom impact of AI competence (no criterion/predictive validation provided).

Concrete suggestions for follow-up studies / improvements

1. Conduct measurement invariance tests across language groups, academic rank, subject (Arts/Science/Commerce) and region — report configural, metric, scalar invariance and partial invariance where needed.
2. Publish item-level descriptive statistics, item–total correlations, Cronbach alpha‑if‑item‑deleted, and confirm no problematic high inter-item correlations (>0.80), to check redundancy.
3. Collect concurrent validity data: observed teacher behaviour (classroom recordings or LMS logs), student learning outcomes, or independent assessor ratings to establish criterion validity.
4. Preregister confirmatory analyses on a fresh, stratified sample (and share data/syntax) to enhance reproducibility and reduce researcher degrees of freedom.
5. Translate and culturally adapt the instrument into major Indian languages with forward-back translation, cognitive interviews, and local piloting—then re-run invariance testing.

Minimal set of falsifying results (what would disprove this validation)

• Failure to replicate the seven-factor structure (e.g., CFA CFI < 0.90, RMSEA > 0.08) in independent Indian samples.
• Dimension alphas dropping below conventional thresholds (α < 0.70) or evidence of severe cross-loadings in EFA/CFA.
• Measurement non-invariance across language/region making the scale non-comparable across Indian subgroups.