World-to-Wrist Task-Conditioned Future Wrist Modeling for Fine-Grained Robot Manipulation

Yuhao Pan^1,* Haosong Peng^1,* Zhengshen Zhang² Zhengyang Yan¹

Yalun Dai³ Fushuo Huo⁷ Chujie Wang⁴ Tianyu Qi⁵

Xiucheng Wang⁶ Nan Cheng⁶ Wenchao Xu^1,†

¹The Hong Kong University of Science and Technology · ²National University of Singapore

³Nanyang Technological University · ⁴Wuhan University · ⁵Sun Yat-sen University

⁶Xidian University · ⁷Southeast University

^*Equal contribution. · ^†Corresponding author.

PPaper CCode MW²-VLA Model CW²-CoT DDataset

W²-VLA turns global task understanding into action-proximal foresight by predicting future wrist latents from a compact, task-conditioned interface and wrist history.

98.5%LIBERO average

60.71%RoboTwin Easy

18.21%RoboTwin Hard

80+ Hzreal-time generation

Abstract

Vision-language-action (VLA) models often treat main-view and wrist-view observations as parallel visual inputs, overlooking their distinct roles in robot manipulation. Fine-grained manipulation, however, benefits from anticipating how wrist-local interactions may evolve under the global task context. To address this limitation, we present World-to-Wrist VLA (W²-VLA), a VLA model for fine-grained robot manipulation with task-conditioned future wrist modeling. Given current multi-view observations and an instruction, W²-VLA contextualizes a set of latent modeling tokens as a compact interface between the VLM and the wrist predictor. Conditioned on this interface and wrist history, the wrist predictor forecasts future wrist latents, which are converted into future-aware context for action prediction. In addition, we propose W²-CoT, a synthesis pipeline that produces structured annotations for manipulation progress, physical transition cues, and wrist-local evidence. These structured annotations provide auxiliary supervision to help shape the task-conditioned latent interface. Experiments on LIBERO, RoboTwin 2.0, and real-world manipulation tasks demonstrate improved fine-grained and contact-sensitive manipulation across single-arm and bimanual settings, while maintaining real-time action-generation above 80 Hz.

Directional modeling

Global world context is explicitly routed toward wrist-local dynamics.

Latent foresight

Future wrist representations capture contact, alignment, and release.

Fast inference

A fixed-length latent interface avoids explicit CoT generation at deployment.

Method

A two-branch policy joined by one task-conditioned interface.

World

①

Task-conditioned interface

Current multi-view observations and language contextualize 16 latent modeling tokens inside Qwen-VL.

Wrist

②

Future wrist predictor

Frozen V-JEPA wrist-history features are queried by task-conditioned states to forecast local future latents.

Action

③

Future-aware control

A lightweight adapter fuses the predicted wrist context with VLM states for flow-matching action generation.

Benchmark results

Strong across single-arm and bimanual manipulation.

Real-world evaluation

Robust progress, even when the world changes.

Condition

Metric

Paper visuals

From task-level plans to wrist-local evidence.

Real-world rollouts for table cleaning, occluded placement, and bimanual plug insertion — Real-world rollouts span long-horizon cleaning, obstacle-aware placement, and contact-rich bimanual insertion.

Attention visualizations over main and wrist views during simulation and real-world manipulation — Latent modeling token attention tracks task-relevant evidence across main and wrist views. CoT text is decoded for visualization only.

Real-world videos

Three tasks across four visual conditions.

Citation

BibTeX

@article{w2vla_placeholder,
  title   = {World-to-Wrist: Task-Conditioned Future Wrist Modeling for Fine-Grained Robot Manipulation},
  author  = {Author list to be updated},
  journal = {Venue to be updated},
  year    = {2027}
}