Brain-to-Image Generation: Reconstructing Visual Stimuli from EEG using Generative Adversarial Networks

大脑到图像生成：利用生成对抗网络从脑电图（EEG）中重建视觉刺激

Abstract: Reconstructing visual stimuli from electroencephalography (EEG) is difficult because scalp measurements have high temporal but limited spatial resolution, and paired EEG-image datasets remain small relative to modern generative-model training corpora.

摘要： 从脑电图（EEG）中重建视觉刺激非常困难，因为头皮测量具有高时间分辨率但空间分辨率有限，且配对的 EEG-图像数据集相对于现代生成模型的训练语料库而言仍然较小。

We present a reproducible single-subject baseline on THINGS-EEG2 that first tests the more defensible question of whether EEG can retrieve the viewed stimulus in a visual embedding space. A compact temporal-spatial convolutional encoder maps repetition-averaged EEG (63 by 250) to provided 512-dimensional ViT-B/32 image features.

我们在 THINGS-EEG2 数据集上提出了一个可复现的单受试者基准，首先测试了一个更具合理性的问题：EEG 是否能在视觉嵌入空间中检索到所观察到的刺激。一个紧凑的时空卷积编码器将重复平均后的 EEG 数据（63 x 250）映射到提供的 512 维 ViT-B/32 图像特征中。

Model selection uses a concept-disjoint validation split, and final evaluation uses the official 200-image, 200-concept test gallery. Across three training seeds, the model obtains 12.83 +/- 0.58%, 39.17 +/- 1.76%, and 58.00 +/- 1.73% image recall at 1, 5, and 10 (mean +/- sample standard deviation), compared with analytical chance levels of 0.5%, 2.5%, and 5.0%.

模型选择采用了概念不相交的验证集划分，最终评估使用了官方的 200 张图像、200 个概念的测试库。在三个训练种子下，该模型在 Top-1、Top-5 和 Top-10 的图像召回率分别为 12.83 +/- 0.58%、39.17 +/- 1.76% 和 58.00 +/- 1.73%（平均值 +/- 样本标准差），而分析得出的随机概率水平分别为 0.5%、2.5% 和 5.0%。

A session-balanced ablation shows that averaging more test repetitions generally improves ranking. Applying the Subject 01 model to the other nine subjects without adaptation causes a sharp performance drop, exposing subject specificity.

一项会话平衡消融研究表明，平均更多的测试重复次数通常会提高排名效果。将受试者 01 的模型直接应用于其他九名受试者而不进行适配，会导致性能急剧下降，这揭示了受试者之间的特异性。

We further report exploratory stress tests of direct conditional generators trained without external visual weights: single-subject and ten-subject variants produce noise-dominated outputs, with early validation improvements reversing after one to four epochs.

我们进一步报告了在没有外部视觉权重的情况下训练的直接条件生成器的探索性压力测试：单受试者和十受试者变体产生的输出以噪声为主，且在训练一到四个 epoch 后，早期的验证改进会出现逆转。

Finally, we distinguish direct reconstruction from semantic rendering with a pretrained diffusion prior. The results support above-chance coarse semantic decoding under a closed-set, repetition-averaged protocol, but do not support faithful recovery of stimulus pixels.

最后，我们将直接重建与使用预训练扩散先验的语义渲染进行了区分。结果支持在闭集、重复平均协议下进行高于随机水平的粗略语义解码，但不支持对刺激像素的忠实恢复。