Vision-Language-Action (VLA) models are increasingly used for end-to-end driving due to their world knowledge and reasoning ability. Most prior work, however, inserts textual chains-of-thought (CoT) as intermediate steps tailored to the current scene. Such symbolic compressions can blur spatio-temporal relations and discard fine visual cues, creating a cross-modal gap between perception and planning. We propose FSDrive, a visual spatio-temporal CoT framework that enables VLAs to think in images. The model first acts as a world model to generate a unified future frame that overlays coarse but physically-plausible priors-future lane dividers and 3D boxes-on the predicted future image. This unified frame serves as the visual CoT, capturing both spatial structure and temporal evolution. The same VLA then functions as an inverse-dynamics model, planning trajectories from current observations and the visual CoT. To equip VLAs with image generation while preserving understanding, we introduce a unified pre-training paradigm that expands the vocabulary to include visual tokens and jointly optimizes VQA (for semantics) and future-frame prediction (for dynamics). A progressive easy-to-hard scheme first predicts lane/box priors to enforce physical constraints, then completes full future frames for fine details. On nuScenes and NAVSIM, FSDrive improves trajectory accuracy and reduces collisions under both ST-P3 and UniAD metrics, and attains competitive FID for future-frame generation despite using lightweight autoregression. It also advances scene understanding on DriveLM. Together, these results indicate that visual CoT narrows the cross-modal gap and yields safer, more anticipatory planning. Code is available at https://github.com/MIV-XJTU/FSDrive.
翻译:视觉-语言-动作(VLA)模型因其世界知识与推理能力,正日益广泛地应用于端到端驾驶任务。然而,现有研究大多通过插入针对当前场景的文本思维链作为中间步骤。此类符号化压缩可能模糊时空关联并丢失细微视觉线索,导致感知与规划之间存在跨模态鸿沟。本文提出FSDrive,一种视觉时空思维链框架,使VLA模型能够通过图像进行推理。该模型首先作为世界模型,生成融合粗略但物理合理的先验信息(未来车道分隔线与三维边界框)的统一未来帧,叠加于预测的未来图像之上。此统一帧作为视觉思维链,同时捕捉空间结构与时序演化。随后,同一VLA模型作为逆动力学模型,基于当前观测与视觉思维链规划轨迹。为赋予VLA图像生成能力并保持其理解性能,我们提出统一预训练范式:扩展词汇表以纳入视觉标记,并联合优化视觉问答(用于语义理解)与未来帧预测(用于动态建模)。采用渐进式由易到难策略,先预测车道/边界框先验以强化物理约束,再补全完整未来帧以细化细节。在nuScenes与NAVSIM数据集上,FSDrive在ST-P3和UniAD指标下均提升了轨迹精度并降低碰撞率,且通过轻量级自回归方法实现了具有竞争力的未来帧生成FID分数。在DriveLM场景理解任务中也取得显著进展。综合结果表明,视觉思维链能有效弥合跨模态鸿沟,实现更安全、更具前瞻性的规划。代码已开源:https://github.com/MIV-XJTU/FSDrive。