Proper Velocity Neural Networks

Hyperbolic Neural Networks (HNNs) have shown remarkable success in representing hierarchical and tree-like structures, yet most existing work relies on the Poincaré ball and hyperboloid models. While these models admit closed-form Riemannian operators, their constrained nature potentially leads to numerical instabilities, especially near model boundaries. In this work, we explore the Proper Velocity (PV) space, an unconstrained representation of hyperbolic space rooted in Einstein's special relativity, as a stable alternative. We first establish the complete Riemannian toolkit of the PV space. Building on this foundation, we introduce Proper Velocity Neural Networks (PVNNs) with core layers including Multinomial Logistic Regression (MLR), Fully Connected (FC), convolutional, activation, and batch normalization layers. Extensive experiments across four tasks, namely numerical stability, image classification, graph node classification, and genomic sequence learning, demonstrate the stability and effectiveness of PVNNs. The code is available at https://github.com/NickyoyoSu/PVNN . * Equal contribution. Published as a conference paper at ICLR 2026 and batch normalization layers. Based on these layers, one can construct different network architectures. We validate the framework through four sets of experiments, including numerical stability, computer vision, graph learning, and genomic sequence learning, demonstrating both the stability of PV embeddings and effectiveness of PVNNs. To our knowledge, the PV model has remained largely unexplored in machine learning, and our work provides the first systematic study of its use for representation learning. In summary, our contributions are threefold: 1. We establish the complete Riemannian geometric toolkit of the PV manifold, deriving closedform operators that enable its use as a new alternative to classical hyperbolic models. 2. We develop fundamental building blocks in PV space, including MLR, FC, convolutional, activation, and batch normalization layers. 3. We validate the stability and effectiveness of PVNNs through experiments on four tasks: numerical stability, image classification, graph node classification, and genomic sequence learning. RELATED WORK Hyperbolic representation. Hyperbolic embeddings have been widely explored for hierarchical and non-Euclidean structures in networks, trees, and text (