Lune

CVPR2023Top-tier venue

Heat Diffusion Based Multi-Scale and Geometric Structure-Aware Transformer for Mesh Segmentation

Chi-Chong Wong

Abstract

Triangle mesh segmentation is an important task in 3D shape analysis, especially in applications such as digital humans and AR/VR. Transformer model is inherently permutation-invariant to input, which makes it a suitable candidate model for 3D mesh processing. However, two main challenges involved in adapting Transformer from natural languages to 3D mesh are yet to be solved, such as i) extracting the multi-scale information of mesh data in an adaptive manner; ii) capturing geometric structures of mesh data as the discriminative characteristics of the shape. Current point based Transformer models fail to tackle such challenges and thus provide inferior performance for discretized surface segmentation. In this work, heat diffusion based method is exploited to tackle these problems. A novel Transformer model called MeshFormer is proposed, which i) integrates Heat Diffusion method into Multi-head Self-Attention operation (HDMSA) to adaptively capture the features from local neighborhood to global contexts; ii) applies a novel Heat Kernel Signature based Structure Encoding (HKSSE) to embed the intrinsic geometric structures of mesh instances into Transformer for structure-aware processing. Extensive experiments on triangle mesh segmentation validate the effectiveness of the proposed Mesh-Former model and show significant improvements over current state-of-the-art methods.

Ask Lune

Know this paper in minutes, not an afternoon.

Lune can read every page of it and the top-tier papers that cite it, and gives you its claims, its evidence and its limits, each one cited to the paper it comes from. No skimming, no AI slop.

Questions to start from

Lune can draw on

the full paper · related work

Get the answer

Free to start. No credit card required.

Lineage

  • Swin Transformer: Hierarchical Vision Transformer using Shifted Windows cites Pyramid Vision Transformer: A Versatile Backbone for Dense Prediction without Convolutions
  • Point Transformer V2: Grouped Vector Attention and Partition-based Pooling cites Swin Transformer: Hierarchical Vision Transformer using Shifted Windows
  • Point Transformer V2: Grouped Vector Attention and Partition-based Pooling cites Pyramid Vision Transformer: A Versatile Backbone for Dense Prediction without Convolutions
Builds onCited byLeft to right by year; the more cited, the higher.

Builds on12

  1. Language Models are Few-Shot LearnersTom B. Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah et al.NeurIPS 2020 · 64,255 citations
  2. Swin Transformer: Hierarchical Vision Transformer using Shifted WindowsZe Liu, Yutong Lin, Yue Cao, Han Hu et al.ICCV 2021 · 31,683 citations
  3. Pyramid Vision Transformer: A Versatile Backbone for Dense Prediction without ConvolutionsWenhai Wang, Enze Xie, Xiang Li, Deng-Ping Fan et al.ICCV 2021 · 4,909 citations
  4. Point Transformer V2: Grouped Vector Attention and Partition-based PoolingXiaoyang Wu, Yixing Lao, Li Jiang, Xihui Liu et al.NeurIPS 2022 · 924 citations
  5. Rethinking Graph Transformers with Spectral AttentionDevin Kreuzer, Dominique Beaini, William L. Hamilton, Vincent Létourneau et al.NeurIPS 2021 · 854 citations

Related papers

  1. SchellingFormer: Laplacian Matrix-guided Geometric Transformer for Robust Schelling Point DetectionYihao Chen, Haobo Jiang, Liang Yu, Jianmin ZhengAAAI 2026
  2. Mesh GraphormerKevin Lin, Lijuan Wang, Zicheng LiuICCV 2021 · 399 citations
  3. Deformable Mesh Transformer for 3D Human Mesh RecoveryYusuke YoshiyasuCVPR 2023
  4. MeshFlow: Mesh Generation with Equivariant Flow MatchingQi Sun, Kiyohiro Nakayama, Jing Nathan Yan, Qixing Huang et al.SIGGRAPH 2026
  5. FeatER: An Efficient Network for Human Reconstruction via Feature Map-Based TransformERCe Zheng, Matías Mendieta, Taojiannan Yang, Guo-Jun Qi et al.CVPR 2023
Dusk over the sea between two cliffs drawn in fine vertical lines