Analyze What Papers Actually Reported

Plan summary — what I'll deliver

I will produce a reproducible analysis pipeline that compares PATCH-AP to traditional MEG inverse methods (MNE/dSPM/sLORETA/cMEM/beamformers) across shared metrics: localization error (Dmin / Euclidean error), spatial dispersion, Earth Mover's Distance (EMD), and robustness to noise/scarce labels — using both the provided published numeric benchmarks and new reproducible analyses on your real/simulated MEG data. Key references: PATCH-AP (2025), PINN MEG inversion (2026), cluster permutation statistics (2007), method-comparisons (2010, 2020).

Core evidence anchors: PATCH-AP preprint • PINN inversion benchmark

Visual-first plan: Comparative MEG localization analysis (PATCH-AP vs traditional methods)

This plot uses the PINN vs MNE mean Euclidean localization errors reported in the PINN paper (PINN mean error 0.45 cm; optimized MNE mean error 0.84 cm; N=1000 synthetic samples) — see citation below for full context and caveats about synthetic geometry vs real MEG data.

Primary methodological sources and what they contribute to the plan:

• PATCH-AP (2025) — proposes PATCH-AP tailored to extended/patch-like sources; authors report improved localization and alignment to fusiform face area in a face dataset, and use Earth Mover's Distance (EMD) to quantify spatial match. This is the method we are comparing against traditional methods and new ML/PINN approaches. PATCH-AP preprint
• PINN + FEM inversion (2026) — demonstrates physics-informed neural networks reduce localization error vs MNE in ellipsoidal synthetic head models and maintain accuracy in scarce-label regimes; provides a reproducible codebase. Use its reported numeric benchmarks as a reproducibility anchor for ML-based inversion approaches. PINN inversion benchmark
• Nonparametric cluster tests (Maris & Oostenveld 2007) — establishes cluster-based permutation statistics for M/EEG, required for rigorous inference across space/time/frequency when comparing methods on empirical data (controls false alarm rate). We'll use this for statistical comparisons across methods on subject-level and group-level maps. Cluster-based permutation testing
• Distributed methods comparison (2010, 2020) — MNE/dSPM/sLORETA/cMEM comparisons and resolution metrics (PSF/CTF/Dmin/SD) that we will replicate to compute spatial-resolution metrics and to interpret trade-offs (localization bias vs spatial extent). MNE / dSPM / sLORETA resolution study Clinical comparison of distributed methods

Stepwise analysis plan (reproducible pipeline)

1. Inputs required: (A) your raw MEG sensor-level epochs (or provide the SPM face dataset referenced by PATCH-AP), (B) subject MRIs or standardized template, (C) forward model (BEM/FEM) or sensor calibration files, (D) ground truth where available (intracranial, fMRI ROIs, simulated source locations). Cite sources when used.
2. Preprocessing: Maxwell / signal-space separation (if Elekta), band-pass per task, artifact rejection (ICA for ECG/EOG), baseline correction; downsample to analysis rate. Use procedures consistent with Cichy et al. 2017 for MVPA-like comparisons when reusing face data. MEG preprocessing/decoding pipeline
3. Forward modeling: build both BEM and (if available) FEM forward models; for physics-informed comparisons use high-fidelity FEM (FEniCS) as in the PINN paper; for standard pipelines use 1-layer or 3-layer BEM (OpenMEEG/MNE-BEM). Document conductivity choices and mesh resolution. PINN FEM forward modeling
4. Implement algorithms (containerized):
   • Traditional distributed: MNE (with depth weighting), dSPM, sLORETA, beamformer (LCMV), cMEM (if available), and averaging approaches (Ave) per Human Brain Mapping (2020) benchmarks. Clinical distributed methods benchmarks
   • PATCH-AP: obtain author code (request if not public) or reimplement per methods section; ensure EMD metric reproduction using Python POT package (authors used POT). PATCH-AP methods note
   • PINN baseline (optional comparative ML): use the published PINN implementation as an additional modern baseline (reproducible code available). Compare PINN, PATCH-AP, and MNE families on identical forward model and data. PINN repo and hyperparams
5. Evaluation metrics (compute all):
   • Localization error (Euclidean distance) when ground-truth dipoles/patch centers exist (report mean, median, 90th percentile, and full error CDF).
   • Dmin (minimum distance to ground-truth source surface) and PSF/CTF metrics per Hauk et al./W.2002/2010 frameworks to evaluate spatial spread. MEG sensitivity mapping
   • Earth Mover's Distance (EMD) between estimated maps and ground truth for extended/patch sources (this is the metric used in PATCH-AP paper).
   • Spatial dispersion, map-size, contrast, and off-focus activity (as in the 2020 epilepsy study). Spatial metrics in clinical comparison
   • Robustness tests: additive sensor noise, head-movement perturbations (simulate marker-coil error sources per triangular coil paper) and scarce-label regimes (for ML methods), quantify error increase. Marker coil position uncertainty
6. Statistics: use cluster-based permutation tests (Maris & Oostenveld 2007) to compare source maps across conditions/methods (control family-wise error). For per-subject numeric summaries (Dmin, EMD) use paired/permutation tests (Wilcoxon or permutation paired t) and report effect sizes and 95% CIs obtained by bootstrap. Cluster permutation statistics
7. Visualization:
   • Report per-method mean/median errors (bar + CDF), spatial maps (surface projections) with consistent colormap and thresholding rules (use algorithm-agnostic percentile thresholds), and EMD heatmaps for patches.
   • Include interactive Plotly figures (error CDFs, bar charts, voxel-wise difference maps) and tabular results (DataTables) for reproducibility.
8. Reproducibility and code: deliver container (Docker) with notebooks, dependencies pinned (MNE-Python, numpy, scipy, POT, PyTorch, FEniCS if PINN used), and a CWL/Snakemake pipeline to run end-to-end. Publish results and minimal data snapshots to a stable repo (GitHub + Zenodo) and include instructions to request/ingest original SPM face data used by PATCH-AP. Cite original dataset source. SPM Face Dataset reference

Planned figures (all interactive Plotly)

1. Bar chart + error bars: mean/median localization error per method (MNE/dSPM/sLORETA/cMEM/PATCH-AP/PINN) — where PATCH-AP numbers come from their reported metrics; if none available, show 'no numeric reported' label and present EMD instead.
2. Error CDF overlay (log-linear x-axis) for each method to show tails and robustness to outliers (important: PINN claims to reduce long-tail high-error solutions).
3. Surface maps: Method-specific source maps (average over epochs) and pairwise difference maps (PATCH-AP minus MNE) with cluster-based significance overlays.
4. EMD heatmap across simulated patch locations showing where methods fail (spatial map of EMD per cortical patch center).

Caveats, blind spots, and critical tests

• PATCH-AP reported superiority on simulations and a face dataset but did not (in the available extract) publish full numeric error distributions or open code — we must request code/data or reimplement exactly to avoid implementation-bias; results may depend on source model assumptions (coherence, orientation, patch extent). PATCH-AP limitations
• PINN benchmarks are synthetic (ellipsoidal head) — improved numbers there need real-MEG validation; forward-model mismatch and MEG noise structure (environmental + correlated sensor noise) can reduce ML advantages. PINN validation limits
• Traditional methods trade localization bias and spread (PSF/CTF); averaging methods (Ave) may reduce Dmin but mask method-specific biases — interpret map averaging carefully. Ave vs cMEM tradeoffs
• Statistical inference must use nonparametric cluster tests (Maris & Oostenveld 2007) to control for multiple comparisons across cortical vertices and time/frequency windows. Cluster permutation testing best practice

Deliverables

• Executable pipeline (Docker + Snakemake) to run: preprocessing -> forward model -> inversion (all methods) -> metrics -> stats -> interactive Plotly report.
• Interactive Plotly figures: bar charts, CDFs, cortical maps with cluster overlays, EMD maps.
• Reproducible notebooks and static manuscript-ready figures and a plain-English technical report with a critical discussion of limitations, model assumptions, and what would falsify each claim (e.g., if PATCH-AP fails to outperform alternatives on independent datasets or under noise/marker perturbations it would falsify robustness claims).

Essential citations (anchor evidence used above)

• PATCH-AP preprint
• PINN inversion study
• Cluster-permutation testing
• MNE/dSPM/sLORETA resolution analysis
• Distributed methods clinical comparison
• MEG sensitivity and depth effects