An Efficient Accelerator for Point-based and Voxel-based Point Cloud Neural Networks

The 3D point cloud neural networks, including point-based and voxel-based networks, play an essential role in various 3D applications. Many previous works have proposed dedicated accelerators to speed up 3D point cloud neural network processing. Yet, two major challenges still exist: (1) Inefficient memory access due to large off-chip data access volume. The point-based method visits massive redundant points, while the voxel-based method fails to reuse on-chip voxel data, leading to up to 983× data access compared with original input data. (2) Poor scalability due to low computing unit utilization. The computing unit is under-utilized when scaled with a larger computing array size, as low as 16.37% when scaling the current accelerator’s computing capability to general-purpose processors (e.g., GPUs).To solve the above challenges, we propose MARS, a memory access reduced and scalable accelerator for both point-based and voxel-based 3D point cloud neural networks. To reduce the memory access, MARS filters out unnecessary off-chip point data access by 6.52× in volume for point-based networks and increases on-chip data reuse to reduce off-chip data access by 26.31× for voxel-based networks. To improve scalability, MARS also features an elastic computing array architecture that can be dynamically configured at runtime to fit different tasks, providing 7.09× higher computing unit utilization. Extensive experiments show that MARS achieves 1.76× over speedup and 3.97× PointAcc for point-based and end-to-end voxel-based point cloud neural networks, respectively.