OSVI-WM

OSVI-WM Contributions

Architecture: An efficient end-to-end imitation learning architecture trained solely on in-domain data, without requiring large-scale pretraining.
World Model: A novel world-model-guided trajectory generation module tailored for OSVI on unseen tasks.
Re-Planning: Robustness enhancement at test time by using a waypoint controller with re-planning.
Experiments: Extensive experiments in both simulated and real-world settings, demonstrating that OSVI-WM outperforms existing methods on unseen tasks.

Video

Results

Table 2: Real-World experiments: Success rates and execution breakdowns (in %) are reported. T-OSVI* denotes T-OSVI [10] aided with end-effector depth sensing for improved grasping.

Experimental result clips for all the benchmarks are available in the project video.

Citation

@article{goswami2025osvi,
  title={Osvi-wm: One-shot visual imitation for unseen tasks using world-model-guided trajectory generation},
  author={Goswami, Raktim Gautam and Krishnamurthy, Prashanth and LeCun, Yann and Khorrami, Farshad},
  journal={arXiv preprint arXiv:2505.20425},
  year={2025}
}