When Does Language Matter? Multilingual Instructions Reveal Step-wise Language Sensitivity in Vision-Language-Action Models
Authors: Xuan Dong, Zhe Han, Tianhao Niu, Qingfu Zhu, Wanxiang Che
The authors conduct the first systematic multilingual evaluation of VLA models, finding that non-English instructions lead to a significant drop in success rates. The study reveals that language influence is highly uneven across task execution steps and proposes a reasoning-time intervention method based on step-wise language sensitivity to align representations.
Paper Summary
Problem
While VLA models perform strongly in language-conditioned robotic manipulation, their robustness to language variants such as multilingual instructions is poorly understood, and existing unified alignment methods have limited effectiveness.
Method Overview
- Backbone: Vision-Language-Action (VLA) 模型
- Key module: 步骤级语言敏感性评分模块、推理时选择性干预机制
- Loss: 策略损失(Policy Loss),干预阶段采用表示对齐策略
Takeaways
Personal understanding — corrections welcome
Experiments indicate that the language robustness of VLA models is essentially a step-wise control problem, where language bias concentrates on specific critical steps rather than being uniformly distributed.
Limitations & Questions
The evaluation is primarily based on the LIBERO simulation benchmark, requiring verification of generalization in real robotic scenarios; the intervention method relies on offline English data to construct sensitivity standards.
Reading Notes
Personal reading notes. Original paper: arXiv:2606.11906v1.
Paper Reading Note: When Does Language Matter? Multilingual Instructions Reveal Step-wise Language Sensitivity in Vision-Language-Action Models
Paper ID: 2606.11906v1
Title: When Does Language Matter? Multilingual Instructions Reveal Step-wise Language Sensitivity in Vision-Language-Action Models
Authors: Xuan Dong, Zhe Han, Tianhao Niu, Qingfu Zhu, Wanxiang Che
Affiliation: Harbin Institute of Technology
Date: June 10, 2026 (arXiv Preprint)
This note was included in the arXiv daily digest 20260616. More roundups at blog archive .