Benchmarking Confidential GPU Inference on NVIDIA H100 under Intel TDX
Benchmarking Confidential GPU Inference on NVIDIA H100 under Intel TDX
基于 Intel TDX 的 NVIDIA H100 机密 GPU 推理基准测试
Abstract: Confidential computing is becoming a practical deployment requirement for AI inference workloads that process sensitive inputs or protect proprietary model assets. However, the performance cost of enabling confidential execution for GPU-accelerated large language model serving remains workload dependent and operationally important.
摘要： 对于处理敏感输入或保护专有模型资产的 AI 推理工作负载而言，机密计算正逐渐成为一种实际的部署需求。然而，为 GPU 加速的大语言模型服务启用机密执行所带来的性能成本，仍取决于具体的工作负载，且在运营层面至关重要。
This paper presents a benchmark study comparing standard non-confidential execution with confidential computing mode on a single NVIDIA H100 80GB GPU hosted in an Intel TDX confidential instance. The evaluation uses two representative language models, Mistral-7B v0.1 and Qwen3-30B-A3B, and measures time to first token, end-to-end request latency, per-request token generation throughput, global token throughput, and closed-loop request throughput under increasing concurrency.
本文提出了一项基准测试研究，对比了在 Intel TDX 机密实例中托管的单块 NVIDIA H100 80GB GPU 上，标准非机密执行模式与机密计算模式的性能差异。评估使用了两种具有代表性的语言模型：Mistral-7B v0.1 和 Qwen3-30B-A3B，并测量了首字延迟（TTFT）、端到端请求延迟、单请求令牌生成吞吐量、全局令牌吞吐量以及在不断增加的并发量下的闭环请求吞吐量。
In fixed request-rate experiments, confidential mode increases average TTFT by 21.8% for Mistral-7B and 27.8% for Qwen3-30B-A3B, while global token throughput drops by 17.7% and 21.1%, respectively. In closed-loop concurrency experiments, throughput gaps remain in the 11.5-20.2% range, but the larger model reaches its saturation knee earlier under confidential mode.
在固定请求速率的实验中，机密模式使 Mistral-7B 的平均首字延迟（TTFT）增加了 21.8%，Qwen3-30B-A3B 增加了 27.8%，而全局令牌吞吐量分别下降了 17.7% 和 21.1%。在闭环并发实验中，吞吐量差距保持在 11.5% 到 20.2% 之间，但较大的模型在机密模式下会更早达到饱和拐点。
The results suggest that confidential GPU inference can retain usable throughput under load, but capacity planning must account for both the steady throughput penalty and the earlier saturation behavior observed for larger models.
研究结果表明，机密 GPU 推理在负载下仍能保持可用的吞吐量，但容量规划必须同时考虑到稳定的吞吐量损耗以及大型模型所表现出的更早饱和行为。