Think Twice before Driving: Towards Scalable Decoders for End-to-End Autonomous Driving

Throttle 0.05 Steer 0.05 × 𝑵𝑵 Adjust Driving Plan Figure 1. We propose ThinkTwice, a scalable decoder paradigm that generates the future trajectory and action of the ego vehiclefor endto-end autonomous driving. Conditioned on the coarse action/trajectory, we propose the Look Module to retrieve information from critical regions and the Prediction Module to anticipate the outcome of the ego behavior. Taking features from the two modules as input, the coarse prediction is refined by predicting its offset from ground-truth. We could stack multiple such layers and scale up the capacity of the decoder with dense supervision and spatial-temporal prior.