Unifying Uniform and Binary-coding Quantization for Accurate Compression of Large Language Models

How can we quantize large language models while preserving accuracy? Quantization is essential for deploying large language models (LLMs) efficiently. Binary-coding quantization (BCQ) and uniform quantization (UQ) are promising quantization schemes that have strong expressiveness and optimizability, respectively. However, neither scheme leverages both advantages. In this paper, we propose UniQuan F (Unified Quantization with Flexible Mapping), an accurate quantization method for LLMs. UniQuan F harnesses both strong expressiveness and optimizability by unifying the flexible mapping technique in UQ and BCQ's non-uniform quantization levels. We propose unified initialization, and local and periodic mapping techniques to optimize the parameters in UniQuan F precisely. After optimization, our unification theorem removes computational and memory overhead, allowing us to utilize the superior accuracy of UniQuan F without extra deployment costs induced by the unification. Experimental results demonstrate that UniQuan F outperforms existing UQ and BCQ methods, achieving up to 4.60% higher accuracy on GSM8K benchmark. (a) Uniform Quantization (UQ) (FlexRound, OmniQuant) Strong optimizability Steps Error Steep decrement Weak expressiveness (b) Binary-coding Quantization (BCQ) (ALTERNATING) Strong expressiveness Weak optimizability Error Steps Slow decrement (c) Unified Quantization (UniQuan) (UniQuan F