Lune

NeurIPS2020Top-tier venue

PRANK: motion Prediction based on RANKing

Yuriy Biktairov, Maxim Stebelev, Irina Rudenko, Oleh Shliazhko, Boris Yangel

Abstract

Predicting the motion of agents such as pedestrians or human-driven vehicles is one of the most critical problems in the autonomous driving domain. The overall safety of driving and the comfort of a passenger directly depend on its successful solution. The motion prediction problem also remains one of the most challenging problems in autonomous driving engineering, mainly due to high variance of the possible agent's future behavior given a situation. The two phenomena responsible for the said variance are the multimodality caused by the uncertainty of the agent's intent (e.g., turn right or move forward) and uncertainty in the realization of a given intent (e.g., which lane to turn into). To be useful within a real-time autonomous driving pipeline, a motion prediction system must provide efficient ways to describe and quantify this uncertainty, such as computing posterior modes and their probabilities or estimating density at the point corresponding to a given trajectory. It also should not put substantial density on physically impossible trajectories, as they can confuse the system processing the predictions. In this paper, we introduce the PRANK method, which satisfies these requirements. PRANK takes rasterized bird-eye images of agent's surroundings as an input and extracts features of the scene with a convolutional neural network. It then produces the conditional distribution of agent's trajectories plausible in the given scene. The key contribution of PRANK is a way to represent that distribution using nearest-neighbor methods in latent trajectory space, which allows for efficient inference in real time. We evaluate PRANK on the in-house and Argoverse datasets, where it shows competitive results.

Ask Lune

Know this paper in minutes, not an afternoon.

Lune can read every page of it and the top-tier papers that cite it, and gives you its claims, its evidence and its limits, each one cited to the paper it comes from. No skimming, no AI slop.

6 top-tier papers have cited it since. Know what they found before your reviewers do.

Questions to start from

Lune can draw on

the full paper · the top-tier papers that cite it · related work

Get the answer

Free to start. No credit card required.

Lineage

  • Vehicle trajectory prediction works, but not everywhere cites VectorNet: Encoding HD Maps and Agent Dynamics From Vectorized Representation
  • VectorNet: Encoding HD Maps and Agent Dynamics From Vectorized Representation cites PRECOG: PREdiction Conditioned on Goals in Visual Multi-Agent Settings
  • CoverNet: Multimodal Behavior Prediction Using Trajectory Sets cites PRECOG: PREdiction Conditioned on Goals in Visual Multi-Agent Settings
  • Motion Transformer with Global Intention Localization and Local Movement Refinement cites VectorNet: Encoding HD Maps and Agent Dynamics From Vectorized Representation
  • Motion Transformer with Global Intention Localization and Local Movement Refinement cites PRECOG: PREdiction Conditioned on Goals in Visual Multi-Agent Settings
  • Motion Transformer with Global Intention Localization and Local Movement Refinement cites CoverNet: Multimodal Behavior Prediction Using Trajectory Sets
  • Latent Variable Sequential Set Transformers for Joint Multi-Agent Motion Prediction cites VectorNet: Encoding HD Maps and Agent Dynamics From Vectorized Representation
  • Latent Variable Sequential Set Transformers for Joint Multi-Agent Motion Prediction cites CoverNet: Multimodal Behavior Prediction Using Trajectory Sets
  • Reasoning Multi-Agent Behavioral Topology for Interactive Autonomous Driving cites VectorNet: Encoding HD Maps and Agent Dynamics From Vectorized Representation
  • Reasoning Multi-Agent Behavioral Topology for Interactive Autonomous Driving cites Motion Transformer with Global Intention Localization and Local Movement Refinement
  • ADAPT: Efficient Multi-Agent Trajectory Prediction with Adaptation cites VectorNet: Encoding HD Maps and Agent Dynamics From Vectorized Representation
  • ADAPT: Efficient Multi-Agent Trajectory Prediction with Adaptation cites PRECOG: PREdiction Conditioned on Goals in Visual Multi-Agent Settings
  • ADAPT: Efficient Multi-Agent Trajectory Prediction with Adaptation cites Latent Variable Sequential Set Transformers for Joint Multi-Agent Motion Prediction
Builds onCited byLeft to right by year; the more cited, the higher.

Cited by top-tier papers6

  1. Motion Transformer with Global Intention Localization and Local Movement RefinementShaoshuai Shi, Li Jiang, Dengxin Dai, Bernt SchieleNeurIPS 2022 · 515 citations
  2. Latent Variable Sequential Set Transformers for Joint Multi-Agent Motion PredictionRoger Girgis, Florian Golemo, Felipe Codevilla, Martin Weiss et al.ICLR 2022 · 200 citations
  3. ADAPT: Efficient Multi-Agent Trajectory Prediction with AdaptationGörkay Aydemir, Adil Kaan Akan, Fatma GüneyICCV 2023 · 85 citations
  4. Vehicle trajectory prediction works, but not everywhereMohammadhossein Bahari, Saeed Saadatnejad, Ahmad Rahimi, Mohammad Shahverdikondori et al.CVPR 2022 · 65 citations
  5. Reasoning Multi-Agent Behavioral Topology for Interactive Autonomous DrivingHaochen Liu, Li Chen, Yu Qiao, Chen Lv et al.NeurIPS 2024 · 52 citations

Builds on3

  1. PRECOG: PREdiction Conditioned on Goals in Visual Multi-Agent SettingsNicholas Rhinehart, Rowan McAllister, Kris Kitani, Sergey LevineICCV 2019 · 407 citations
  2. VectorNet: Encoding HD Maps and Agent Dynamics From Vectorized RepresentationJiyang Gao, Chen Sun, Hang Zhao, Yi Shen et al.CVPR 2020
  3. CoverNet: Multimodal Behavior Prediction Using Trajectory SetsTung Phan-Minh, Elena Corina Grigore, Freddy A. Boulton, Oscar Beijbom et al.CVPR 2020

Related papers

  1. ProphNet: Efficient Agent-Centric Motion Forecasting with Anchor-Informed ProposalsXishun Wang, Tong Su, Fang Da, Xiaodong YangCVPR 2023
  2. Gaussian-Mixture Latent Flow for Stochastic 3D Human Motion PredictionYue Ma, Frederick W. B. Li, Xiaohui LiangCVPR 2026
  3. FJMP: Factorized Joint Multi-Agent Motion Prediction over Learned Directed Acyclic Interaction GraphsLuke Rowe, Martin Ethier, Eli-Henry Dykhne, Krzysztof CzarneckiCVPR 2023
  4. Multimodal Motion Prediction With Stacked TransformersYicheng Liu, Jinghuai Zhang, Liangji Fang, Qinhong Jiang et al.CVPR 2021
  5. Real-Time Motion Prediction via Heterogeneous Polyline Transformer with Relative Pose EncodingZhejun Zhang, Alexander Liniger, Christos Sakaridis, Fisher Yu et al.NeurIPS 2023 · 79 citations
Dusk over the sea between two cliffs drawn in fine vertical lines