Test Hypotheses Against Reported Results

 BGPT Odds of True

The hypothesis predicts genre/register, not AI exposure, drives detector false positives. Direct support: a pre-LLM constitution flagged 98.5% AI (n=1 document); corroborated by L2 false-positive bias across 30 detectors and near-chance style-based human discrimination. Not yet tested via systematic multi-genre, multi-detector benchmarks, so confidence is moderate.

 Hypothesis Novelty

The general idea that detectors rely on stylistic/register features is widely discussed in 2023–2026 literature, but the specific prediction that pre-LLM constitutional and legal boilerplate would be flagged is a sharpened, testable framing less commonly formalized.

 Quick Analysis Plan Copied

 Long Analysis Plan

Genre, Not Exposure, Drives False Positives

The hypothesis makes a strong, falsifiable prediction: detectors flag register (formulaic, low-perplexity institutional prose) rather than actual AI authorship. The strongest direct support is a single-document diagnostic test: ZeroGPT scored the 1987 Constitution of the Philippines—drafted and ratified decades before LLMs existed—as 98.5% AI-generated. This is a near-decisive single case because ground truth (human authorship) is certain, isolating genre/register as the only plausible driver ZeroGPT constitutional false positive.

Two converging lines corroborate the mechanism. First, detector error varies systematically by textual population: across 30 detectors, English L2 student essays showed consistently higher false-positive rates than L1 essays—another register-adjacent signal independent of any AI exposure L2 false-positive bias across 30 detectors. Second, in a blinded medical-school grading study, expert human assessors identified AI vs. student sources at only 53% (36/68)—near chance—with perceived "writing style" the most cited (unreliable) rationale, and one assessor using AI detection tools classified zero responses correctly. Human judgment, like detector judgment, appears keyed to surface style rather than provenance Blinded expert source-identification study.

Note: the 98.5% value is a single-document anecdote from a narrative review (n=1 text, one detector); the 53% figure comes from a small exploratory study (10 assessors, 68 responses, one institution). Neither represents a systematic genre-vs-exposure benchmark.

What is missing: no supplied evidence systematically benchmarks constitutions, legal templates, and formulaic academic prose across multiple detectors. The Weber-Wulff paraphrase-evasion findings and near-chance human accuracy (40–55%) further suggest detectors rely on fragile stylometric cues that legal boilerplate happens to share with LLM output Paraphrase evasion and near-chance human accuracy. Falsification would require a multi-detector, multi-genre corpus (pre-LLM vs post-LLM institutional texts) in which false positives track authorship rather than register.

Test Hypothesis: Genre vs. Exposure Benchmark Meta-analysis of Detector False Positives