Lune

NeurIPS2022Top-tier venue

Motion Transformer with Global Intention Localization and Local Movement Refinement

Shaoshuai Shi, Li Jiang, Dengxin Dai, Bernt Schiele

Abstract

Predicting multimodal future behavior of traffic participants is essential for robotic vehicles to make safe decisions. Existing works explore to directly predict future trajectories based on latent features or utilize dense goal candidates to identify agent's destinations, where the former strategy converges slowly since all motion modes are derived from the same feature while the latter strategy has efficiency issue since its performance highly relies on the density of goal candidates. In this paper, we propose the Motion TRansformer (MTR) framework that models motion prediction as the joint optimization of global intention localization and local movement refinement. Instead of using goal candidates, MTR incorporates spatial intention priors by adopting a small set of learnable motion query pairs. Each motion query pair takes charge of trajectory prediction and refinement for a specific motion mode, which stabilizes the training process and facilitates better multimodal predictions. Experiments show that MTR achieves state-of-the-art performance on both the marginal and joint motion prediction challenges, ranking 1 st on the leaderboards of Waymo Open Motion Dataset. The source code is available at https://github.com/sshaoshuai/MTR .

Ask Lune

Know this paper in minutes, not an afternoon.

Lune can read every page of it and the top-tier papers that cite it, and gives you its claims, its evidence and its limits, each one cited to the paper it comes from. No skimming, no AI slop.

74 top-tier papers have cited it since. Know what they found before your reviewers do.

Questions to start from

Lune can draw on

the full paper · the top-tier papers that cite it · related work

Get the answer

Free to start. No credit card required.

Lineage

  • DAB-DETR: Dynamic Anchor Boxes are Better Queries for DETR cites Conditional DETR for Fast Training Convergence
  • DAB-DETR: Dynamic Anchor Boxes are Better Queries for DETR cites Deformable DETR: Deformable Transformers for End-to-End Object Detection
  • BEiT: BERT Pre-Training of Image Transformers cites An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale
  • UniTR: A Unified and Efficient Multi-Modal Transformer for Bird's-Eye-View Representation cites An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale
  • Conditional DETR for Fast Training Convergence cites Deformable DETR: Deformable Transformers for End-to-End Object Detection
  • Forecast-MAE: Self-supervised Pre-training for Motion Forecasting with Masked Autoencoders cites An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale
Builds onCited byLeft to right by year; the more cited, the higher.

Cited by top-tier papers74

  1. GameFormer: Game-theoretic Modeling and Learning of Transformer-based Interactive Prediction and Planning for Autonomous DrivingZhiyu Huang, Haochen Liu, Chen LvICCV 2023 · 209 citations
  2. MotionLM: Multi-Agent Motion Forecasting as Language ModelingAri Seff, Brian Cera, Dian Chen, Mason Ng et al.ICCV 2023 · 186 citations
  3. Forecast-MAE: Self-supervised Pre-training for Motion Forecasting with Masked AutoencodersJie Cheng, Xiaodong Mei, Ming LiuICCV 2023 · 123 citations
  4. LMDrive: Closed-Loop End-to-End Driving with Large Language ModelsHao Shao, Yuxuan Hu, Letian Wang, Guanglu Song et al.CVPR 2024 · 114 citations
  5. UniTR: A Unified and Efficient Multi-Modal Transformer for Bird's-Eye-View RepresentationHaiyang Wang, Hao Tang, Shaoshuai Shi, Aoxue Li et al.ICCV 2023 · 106 citations

Builds on24

  1. An Image is Worth 16x16 Words: Transformers for Image Recognition at ScaleAlexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn et al.ICLR 2021 · 21,477 citations
  2. Deformable DETR: Deformable Transformers for End-to-End Object DetectionXizhou Zhu, Weijie Su, Lewei Lu, Bin Li et al.ICLR 2021 · 7,353 citations
  3. BEiT: BERT Pre-Training of Image TransformersHangbo Bao, Li Dong, Songhao Piao, Furu WeiICLR 2022 · 3,632 citations
  4. DAB-DETR: Dynamic Anchor Boxes are Better Queries for DETRShilong Liu, Feng Li, Hao Zhang, Xiao Yang et al.ICLR 2022 · 1,218 citations
  5. Conditional DETR for Fast Training ConvergenceDepu Meng, Xiaokang Chen, Zejia Fan, Gang Zeng et al.ICCV 2021 · 974 citations

Related papers

  1. Multimodal Motion Prediction With Stacked TransformersYicheng Liu, Jinghuai Zhang, Liangji Fang, Qinhong Jiang et al.CVPR 2021
  2. M2I: From Factored Marginal Trajectory Prediction to Interactive PredictionQiao Sun, Xin Huang, Junru Gu, Brian C. Williams et al.CVPR 2022 · 104 citations
  3. Multi-Person 3D Motion Prediction with Multi-Range TransformersJiashun Wang, Huazhe Xu, Medhini Narasimhan, Xiaolong WangNeurIPS 2021 · 102 citations
  4. UniMotion: A Unified Motion Framework for Simulation, Prediction and PlanningNan Song, Junzhe Jiang, Jingyu Li, Xiatian Zhu et al.NeurIPS 2025 · 2 citations
  5. TPNet: Trajectory Proposal Network for Motion PredictionLiangji Fang, Qinhong Jiang, Jianping Shi, Bolei ZhouCVPR 2020
Dusk over the sea between two cliffs drawn in fine vertical lines