Why Do VLMs Fail at Remote Physiological Measurement? Rethinking with Physiological Grounding and Numerical Alignment

Computer Science > Computer Vision and Pattern Recognition arXiv:2610.19052 (cs) [Submitted on 13 Oct 2026] Title: Why Do VLMs Fail at Remote Physiological Measurement? Rethinking with Physiological Grounding and Numerical Alignment.

计算机科学 > 计算机视觉与模式识别 arXiv:2610.19052 (cs) [提交于 2026 年 10 月 13 日] 标题：为什么视觉语言模型（VLM）在远程生理测量中表现不佳？基于生理基础与数值对齐的重新思考。

Abstract: Video-based remote physiological measurement (RPM) has broad applications, while Vision-Language Models (VLMs) provide a promising opportunity to enable end-to-end physiological monitoring for future human-aware intelligent systems. However, existing RPM methods rely on task-specific models and cannot be seamlessly integrated with the multimodal understanding and interaction capabilities of VLMs.

摘要：基于视频的远程生理测量（RPM）具有广泛的应用前景，而视觉语言模型（VLM）为未来人机感知智能系统实现端到端生理监测提供了契机。然而，现有的 RPM 方法依赖于特定任务模型，无法与 VLM 的多模态理解和交互能力无缝集成。

We revisit RPM with VLMs and provide empirical evidence for two major bottlenecks: generic visual semantic representations fail to preserve fine-grained physiological cues; while discrete token prediction in autoregressive generation is inherently misaligned with continuous numerical measurement.

我们重新审视了 VLM 在 RPM 中的应用，并为两个主要瓶颈提供了实证证据：通用的视觉语义表示无法保留细粒度的生理线索；而自回归生成中的离散标记预测与连续数值测量本质上是不匹配的。

To address these challenges, we propose VITAL (Visual-physiological Token Alignment for Language Models), a visual-physiological token alignment framework for RPM and preserving general vision tasks. VITAL retains the original visual pathway for semantic understanding and introduces an independent physiological pathway to jointly model semantic and physiological information.

为了解决这些挑战，我们提出了 VITAL（用于语言模型的视觉-生理标记对齐），这是一个用于 RPM 并保留通用视觉任务的视觉-生理标记对齐框架。VITAL 保留了用于语义理解的原始视觉路径，并引入了一个独立的生理路径来联合建模语义和生理信息。

Moreover, we propose a structured numerical alignment approach to improve the reliability of generative heart rate estimation by enhancing numerical relationship modeling. Extensive experiments show that VITAL improves cross-domain estimation performance on multiple rPPG datasets while preserving the general visual understanding capability of VLMs, demonstrating its effectiveness as a unified visual-physiological model.

此外，我们提出了一种结构化数值对齐方法，通过增强数值关系建模来提高生成式心率估计的可靠性。大量实验表明，VITAL 在多个 rPPG 数据集上提升了跨域估计性能，同时保留了 VLM 的通用视觉理解能力，证明了其作为统一视觉-生理模型的有效性。