Speaking the Navigator's Language: Trajectory-Grounded Instruction Translation for Frozen Aerial VLN Agents

Computer Science > Artificial Intelligence arXiv:2610.10635 (cs) [Submitted on 7 Oct 2026] Title: Speaking the Navigator’s Language: Trajectory-Grounded Instruction Translation for Frozen Aerial VLN Agents Authors: Xi Chen, Zhe Liu, Xiaogang Xu, Jiafei Xu, Chunyi Zhou, Yuan Su, Rui Zeng, Tianyu Du, Kelu Yao, Chao Li, Shouling Ji.

计算机科学 > 人工智能 arXiv:2610.10635 (cs) [提交于 2026 年 10 月 7 日] 标题：说导航员的语言：面向冻结空中视觉语言导航（VLN）智能体的轨迹基础指令翻译。作者：Xi Chen, Zhe Liu, Xiaogang Xu, Jiafei Xu, Chunyi Zhou, Yuan Su, Rui Zeng, Tianyu Du, Kelu Yao, Chao Li, Shouling Ji。

Abstract: Aerial vision-and-language navigation (VLN) agents are typically trained on detail-rich, trajectory-aligned commands, whereas users issue short, intent-driven instructions; on a frozen OpenFly navigator, this \emph{instruction gap} drops success rate (SR) from $31.03%$ to $11.33%$.

摘要：空中视觉语言导航（VLN）智能体通常在细节丰富、与轨迹对齐的指令上进行训练，而用户发布的是简短的、以意图为导向的指令；在冻结的 OpenFly 导航器上，这种“指令差距”导致成功率（SR）从 $31.03%$ 下降到 $11.33%$。

To scale translator training, we prompt a language model with human-written style examples to convert original commands into paired, intent-centered Weak commands, which yield $15.27%$ SR. We introduce the \textbf{Trajectory-Grounded Instruction Translator (TGIT)}, a front-end that keeps the navigator frozen and translates Weak inputs into agent-executable commands by learning from its trajectory outcomes.

为了扩展翻译器训练，我们利用人工编写的风格示例提示语言模型，将原始指令转换为成对的、以意图为中心的弱指令，这产生了 $15.27%$ 的成功率。我们引入了“轨迹基础指令翻译器（TGIT）”，这是一个前端组件，它在保持导航器冻结的同时，通过学习轨迹结果将弱输入翻译为智能体可执行的指令。

The resulting Weak-trained translator raises Weak-input SR to $37.93%$ and transfers zero-shot to real human instructions ($11.33%{\rightarrow}32.51%$); it also improves held-out OpenFly ($4.95%{\rightarrow}20.79%$) and yields recovery on CityNav and AirVLN.

由此产生的弱训练翻译器将弱输入的成功率提高到了 $37.93%$，并实现了向真实人类指令的零样本迁移（从 $11.33%$ 提升至 $32.51%$）；它还改进了留出的 OpenFly 数据集表现（从 $4.95%$ 提升至 $20.79%$），并在 CityNav 和 AirVLN 上实现了性能恢复。