ShapeEmbed: a self-supervised learning framework for 2D contour quantification

The shape of objects is an important source of visual information in a wide range of applications. One of the core challenges of shape quantification is to ensure that the extracted measurements remain invariant to transformations that preserve an object's intrinsic geometry, such as changing its size, orientation, and position in the image. In this work, we introduce ShapeEmbed, a self-supervised representation learning framework designed to encode the contour of objects in 2D images, represented as a Euclidean distance matrix, into a shape descriptor that is invariant to translation, scaling, rotation, reflection, and point indexing. Our approach overcomes the limitations of traditional shape descriptors while improving upon existing state-of-the-art autoencoder-based approaches. We demonstrate that the descriptors learned by our framework outperform their competitors in shape classification tasks on natural and biological images. We envision our approach to be of particular relevance to biological imaging applications.

1. We introduce, to the best of our knowledge, the first self-supervised representation learning model that learns shape descriptors from distance matrices. The descriptors are, by design, invariant to scaling, translation, rotation, reflection, and re-indexing. 2. We are, to the best of our knowledge, the first to propose a solution to achieve indexation invariance in a VAE architecture for shape description based on a padding operation in the encoder, operating jointly with a new loss function. 3. We show that our method outperforms the representation learning state-of-the-art and classical baselines on downstream shape classification tasks.