The Evidence Layer for Scientific AI

 Quick Analysis Plan Copied

 Long Analysis Plan

1. Reproduction pipeline (highest priority)

Clone the authors' repository (data-driven-elm-design.git) once released; stack versions match the paper: TabPFN v2 (v8.0.3), scikit-learn v1.3, Python 3.10, NumPy 1.24, pandas 2.0 Software stack for TabPFN hydrogel models. Build the training table as 45 rows (15 conditions × n=3 hydrogels) with columns [PAS_n, IPTG_µM] → [G′, thickness, curli]; 12-row independent test set (n=4, 8, 15 at 15 µM; n=5 at 10 µM) and 9-row permeability test set.

Verification targets from Table 2: TabPFN R²=0.851, RMSE=1001 Pa, RPIQ=4.59; GPR (Matérn 3/2) R²=0.813; linear regression on log₁₀G′ R²=0.454 Independent test set model comparison. Also confirm the coverage claim: all independent G′ measurements fell within TabPFN's 90% predictive interval 90% predictive interval coverage.

2. Cross-validation stress tests

• Re-run LOCO (leave-one-condition-out) and LOIO (leave-one-IPTG-out) exactly as reported; expect permeability LOCO R² to be negative while independent-test R²=0.690 — reproduce this inconsistency and quantify how much of it stems from only 15 training points Permeability model LOCO inconsistency.
• Reproduce the two-way ANOVA: PAS length explains 81.8% of G′ variance (P < 0.0001); G′ spans 1227±138 Pa (PAS20, 5 µM) to 8615±1196 Pa (PAS1, 25 µM) G′ range and ANOVA variance decomposition.

3. Inverse-design module

Implement the grid-based 2D sweep (PAS n × IPTG) over the four TabPFN property models, normalize properties, and rank candidates by Manhattan (ℓ₁) distance. Validation check: multi-property targeting (G′ + thickness + curli) should recover PAS8/15 µM with distance ≈1 vs 14 for single-property G′ targeting Multi-property inverse design recovery.

4. Extensions beyond the paper (BGPT inference, clearly flagged)

• Active learning loop: use TabPFN uncertainty to select the next PAS/IPTG experiment — a natural fit since the design space is only 2D and uncertainty was realistic on the test set.
• UQ stress test: verify the 90% interval coverage on additional unseen conditions; the paper's test set (12 points) is too small for precise R²/RPIQ confidence — treat published values as point estimates, not population parameters.
• Thinness caveat: thickness R²=−1.505 despite RMSE=21 µm (~5–8% of the 267 µm mean) — replicate with bootstrapped CIs on R² before using thickness in any design loop Thickness model negative R² caveat.

5. Known blind spots

One chassis (E. coli PHL628 ΔCsgA), one matrix protein (CsgA-PASn), two design parameters, one permeability probe (40 kDa FITC-dextran); extrapolation beyond the PAS/IPTG space is untested, and no mechanistic experiments validated why PAS length dominates G′ variance. Generality to other ELM platforms is asserted, not demonstrated.