Physiognomy Revisited: How Vision-Language Models Encode Sensitive Attributes Through Image Transformation

Computer Science > Computer Vision and Pattern Recognition arXiv:2610.35910 (cs) [Submitted on 6 Oct 2026] Title: Physiognomy Revisited: How Vision-Language Models Encode Sensitive Attributes Through Image Transformation Authors: Soumyajit Datta, Arka Dutta, Ashiqur R. KhudaBukhsh.

计算机科学 > 计算机视觉与模式识别 arXiv:2610.35910 (cs) [提交于 2026 年 10 月 6 日] 标题：重访相面术：视觉语言模型如何通过图像转换编码敏感属性 作者：Soumyajit Datta, Arka Dutta, Ashiqur R. KhudaBukhsh。

Abstract: Emergent generative AI technologies, while remarkably beneficial across several tasks and domains, can suffer from unforeseen vulnerabilities and difficult-to-anticipate failure modes. Such failure modes can disparately affect vulnerable communities.

摘要：新兴的生成式人工智能技术虽然在多个任务和领域中具有显著的益处，但也可能遭受不可预见的漏洞和难以预料的故障模式的影响。此类故障模式可能会对弱势群体产生差异化的影响。

This paper investigates two well-known vision language models (VLMs) and inspects how these models encode protected identities while generating images. We show that when VLMs are requested to transform human face images to reflect race identities (e.g., Hispanic, Black, White, or Asian) and sexual orientation (e.g., gay or straight), VLMs often comply with such requests.

本文调查了两种知名的视觉语言模型（VLM），并考察了这些模型在生成图像时如何编码受保护的身份信息。我们展示了当要求 VLM 转换人脸图像以反映种族身份（例如：西班牙裔、黑人、白人或亚裔）和性取向（例如：同性恋或异性恋）时，VLM 往往会顺从这些请求。

We demonstrate that compliance (and refusal) is non-deterministic, group-sensitive, and generated images are statistically distinguishable, often accentuating deep-seated stereotypes. We further show that representation biases introduced during image generation can re-emerge in textual reasoning and characterization tasks.

我们证明了这种顺从（以及拒绝）是非确定性的、对群体敏感的，且生成的图像在统计学上是可区分的，往往加剧了根深蒂固的刻板印象。我们进一步表明，在图像生成过程中引入的表征偏差可能会在文本推理和定性任务中再次出现。

Finally, we show that this worrisome finding is not limited to racial identity or sexual orientation, image generation can perpetuate bias around criminality, raising concerning possibilities of AI-assisted profiling.

最后，我们表明这一令人担忧的发现并不局限于种族身份或性取向，图像生成还可能延续关于犯罪行为的偏见，从而引发了人工智能辅助侧写（AI-assisted profiling）的令人担忧的可能性。