Sampling Reveals Style: Unsupervised, Training-Free Discovery of Prompt-Conditional Stylistic Axes in LLM Activations

采样揭示风格：在大语言模型激活中无监督、免训练地发现提示词条件下的风格轴

Abstract: Large language models (LLMs) encode rich stylistic structure in their hidden activations, but discovering which stylistic dimensions are salient for a given prompt typically requires supervised contrastive data. 摘要： 大语言模型（LLMs）在其隐藏层激活中编码了丰富的风格结构，但要发现哪些风格维度对于给定的提示词（prompt）是显著的，通常需要有监督的对比数据。

We present a training-free, prompt-conditional alternative: we repeatedly sample completions of a single prompt at elevated temperature, apply Principal Component Analysis (PCA) to the pooled hidden activations, and label the resulting axes automatically from the pole generations. 我们提出了一种免训练、基于提示词条件的替代方案：我们在较高的温度（temperature）下对单个提示词进行重复采样补全，对汇总后的隐藏层激活应用主成分分析（PCA），并根据两极生成的文本自动标记所得出的轴。

We validate the discovered axes against 245 human-elicited stylistic annotations in a two-phase study. On our strongest model (Qwen-3.5-4B-Instruct), the top two axes match spontaneously requested human dimensions with 72.8% precision and 43.6% macro-recall, and 75.6% of validity ratings judge the axes’ polar generations accurate to their labels, with 90.9% adjacent inter-annotator agreement. 我们在两阶段研究中，针对 245 条人工诱导的风格标注对所发现的轴进行了验证。在我们最强的模型（Qwen-3.5-4B-Instruct）上，前两个轴与人类自发要求的维度匹配，精确率（precision）达到 72.8%，宏观召回率（macro-recall）达到 43.6%；75.6% 的有效性评分认为这些轴的两极生成结果与其标签准确对应，且相邻标注者间的一致性达到 90.9%。

Discoverability is strongly model-dependent: both Qwen models and Llama-3.2-3B expose human-salient axes, while DeepSeek-7B-Chat drops to 35.3% precision, its leading components dominated by structural rather than stylistic variance. 这种可发现性在很大程度上取决于模型：Qwen 系列模型和 Llama-3.2-3B 都能揭示出人类显著的风格轴，而 DeepSeek-7B-Chat 的精确率则下降至 35.3%，其主要成分更多地受结构性而非风格性方差的影响。

Simple PCA over a model’s own decoding variance is thus an effective, low-cost probe of stylistic structure in LLM representations, one that also exposes sharp cross-model differences in how that structure is organized. 因此，对模型自身的解码方差进行简单的 PCA 分析，是一种有效且低成本的探测 LLM 表示中风格结构的方法，同时也揭示了不同模型在组织这些结构方式上的显著差异。