Safe Local Motion Planning With Self-Supervised Freespace Forecasting

Figure 1 : What are good 3D representations that support planning in dynamic environments? We visualize a typical urban motion planning scenario from a bird's-eye view, where an autonomous vehicle (AV) awaits an unprotected left turn. We highlight a candidate plan with a blue arrow, whose endpoint represents where the AV will be in 1s. An object-centric representation (left), as adopted by standard perception stacks, focuses on object properties (their shape, orientation, position, etc.) both at the current time step and the future. Alternatively, a freespace-centric representation directly captures the freespace of the surrounding scene and can be readily obtained by raycasting measurements from a depth (e.g., LiDAR) sensor. Forecasting a future version (in 1s) of either representation could help the AV identify a potential collision associated with the candidate plan, however at wildly different annotation costs. Forecasting future object trajectories requires a massive amount of object and track labels to train perceptual modules. Instead, we explore future freespace, whose forecasting can be naturally self-supervised by simply letting time move forward and raycasting future sensor measurements. We propose approaches to planning with forecasted freespace and learning to plan with future freespace.