The Evidence Layer for Scientific AI

 Quick Explanation Copied

 Long Explanation

What “influence hidden states” means

A neural network transforms an input through hidden states, for example h_l+1 = f(W_lh_l + b_l). An adversarial perturbation changes the input or an intermediate representation so that these hidden activations cross a decision boundary. The final error is therefore often mediated by altered internal features, not merely by visible input noise.

Evidence

In a simulated EEG–AI system, attacks were applied at three levels: raw EEG, feature space, and Transformer attention layers. Raw-signal FGSM reduced accuracy from 85.2–90.1% to 62.7–68.4%, while user-intent misalignment increased from 4.2–5.3% to 31.8–39.1%. An attention-level attack reduced Transformer accuracy to 72.5% and produced 29.7% misalignment, directly supporting the possibility of manipulating internal attention representations; however, the study was simulation-based and did not measure every hidden state or demonstrate real-hardware transfer EEG attacks at input, feature, and attention levels.

A separate intrusion-detection study used JSMA to modify a limited number of input features. When a model trained on normal data was tested on adversarial inputs, class accuracies fell to 32.11–71.41% and F1 scores to 1.68–34.01%, indicating that small, strategically selected changes can substantially redirect learned internal representations. Yet the single-dataset design, fixed perturbation budget, and lack of cross-dataset validation limit generalization JSMA-induced degradation in intrusion detection.

Important qualification

The supplied evidence supports the chain adversarial perturbation → altered internal representation or attention → changed output, but it does not prove that hidden states are always the unique cause. A stronger test would compare clean and attacked activation vectors layer by layer, intervene on the most changed units, and test whether restoring those states restores the prediction. The evidence is therefore moderate for hidden-state involvement and limited for neuron-level causality or real-world generalization.