Two-dimensional quantization for geometry-aware audio coding
Tal Shuster, Eliya Nachmani
摘要
Recent neural audio codecs have achieved impressive reconstruction quality, typically relying on quantization methods such as Residual Vector Quantization (RVQ), Vector Quantization (VQ) and Finite Scalar Quantization (FSQ). However, these quantization techniques limit the geometric structure of the latent space, make it harder to capture correlations between features leading to inefficiency in representation learning, codebook utilization and token rate. In this paper we introduce Two-Dimensional Quantization (Q2D2), a quantization scheme in which feature pairs are projected onto structured 2D grids, such as hexagonal, rhombic, or rectangular tiling and quantized to the nearest grid values, yielding an implicit codebook defined by the product of grid levels, with codebook sizes comparable to conventional methods. Despite its simple geometric formulation, Q2D2 improves audio compression efficiency, with low token rates and high codebook utilization while maintaining state of the art reconstruction quality. Specifically, Q2D2 achieves competitive to superior performance in various objective and subjective reconstruction metrics, across extensive experiments in speech, audio and music domains compared to state of the art models. Comprehensive ablation studies further confirm the effectiveness of our design choices.
问问这篇 Paper
智能体会读完全文。
Lune 把这篇 Paper 索引到了每一个公式,引用它的顶会 Paper 也一样。你提问,回答直接引用原文。
它引用的顶会 Paper17
- Language Models are Few-Shot LearnersTom B. Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah 等NeurIPS 2020 · 被引用 64,255 次
- data2vec: A General Framework for Self-supervised Learning in Speech, Vision and LanguageAlexei Baevski, Wei-Ning Hsu, Qiantong Xu, Arun Babu 等ICML 2022 · 被引用 1,123 次
- High-Fidelity Audio Compression with Improved RVQGANRithesh Kumar, Prem Seetharaman, Alejandro Luebs, Ishaan Kumar 等NeurIPS 2023 · 被引用 910 次
- Finite Scalar Quantization: VQ-VAE Made SimpleFabian Mentzer, David Minnen, Eirikur Agustsson, Michael TschannenICLR 2024 · 被引用 442 次
- NaturalSpeech 3: Zero-Shot Speech Synthesis with Factorized Codec and Diffusion ModelsZeqian Ju, Yuancheng Wang, Kai Shen, Xu Tan 等ICML 2024 · 被引用 341 次
相关 Paper
- Self-Guidance: Enhancing Neural Codecs via Decoder Manifold AlignmentXiang Li, Yixuan Zhou, Xie, Zhiyong Wu 等ICML 2026
- IVQ: Structured and Lightweight Vector Quantization via Binary Hierarchical Composition Inspired by Heda Zuo, Junxian Wu, Fengjie Lu, Pei Chen 等ICML 2026
- ESC: Efficient Speech Coding with Cross-Scale Residual Vector Quantized TransformersYuzhe Gu, Enmao DiaoEMNLP 2024 · 被引用 5 次
- WavTokenizer: an Efficient Acoustic Discrete Codec Tokenizer for Audio Language ModelingShengpeng Ji, Ziyue Jiang, Wen Wang, Yifu Chen 等ICLR 2025
- Scaling Transformers for Low-Bitrate High-Quality Speech CodingJulian D. Parker, Anton Smirnov, Jordi Pons, CJ Carr 等ICLR 2025
