Pose-Transformed Equivariant Network for 3D Point Trajectory Prediction

Predicting 3D point trajectory is a fundamental learning task which commonly should be equivariant under Eu-clidean transformation, e.g., SE(3). The existing equivari-ant models are commonly based on the group equivariant convolution, equivariant message passing, vector neuron, frame averaging, etc. In this paper, we propose a novel pose-transformed equivariant network, in which the points are firstly uniquely normalized and then transformed by the learned pose transformations, upon which the points after motion are predicted and aggregated. Under each trans-formed pose, we design the point position predictor consisting of multiple Pose- Transformed Points Prediction blocks, in which the global and local motions are estimated and aggregated. This framework can be proven to be equiv-ariant to SE(3) transformation over 3D points. We eval-uate the pose-transformed equivariant network on exten-sive datasets including human motion capture, molecular dynamics modeling and dynamics simulation. Extensive experimental comparisons demonstrated our SOTA performance compared with the existing equivariant networks for 3D point trajectory prediction.