Lune

EMNLP Findings2020

Fully Quantized Transformer for Machine Translation

Gabriele Prato, Ella Charlaix, Mehdi Rezagholizadeh

Abstract

State-of-the-art neural machine translation methods employ massive amounts of parameters. Drastically reducing computational costs of such methods without affecting performance has been up to this point unsuccessful. To this end, we propose FullyQT: an allinclusive quantization strategy for the Transformer. To the best of our knowledge, we are the first to show that it is possible to avoid any loss in translation quality with a fully quantized Transformer. Indeed, compared to fullprecision, our 8-bit models score greater or equal BLEU on most tasks. Comparing ourselves to all previously proposed methods, we achieve state-of-the-art quantization results.

Ask Lune

Know this paper in minutes, not an afternoon.

Lune can read every page of it and the top-tier papers that cite it, and gives you its claims, its evidence and its limits, each one cited to the paper it comes from. No skimming, no AI slop.

17 top-tier papers have cited it since. Know what they found before your reviewers do.

Questions to start from

Lune can draw on

the full paper · the top-tier papers that cite it · related work

Get the answer

Free to start. No credit card required.

Lineage

Builds onCited byLeft to right by year; the more cited, the higher.

Cited by top-tier papers17

  1. Degree-Quant: Quantization-Aware Training for Graph Neural NetworksShyam Anil Tailor, Javier Fernández-Marqués, Nicholas Donald LaneICLR 2021 · 180 citations
  2. Scalable Vision Transformers with Hierarchical PoolingZizheng Pan, Bohan Zhuang, Jing Liu, Haoyu He et al.ICCV 2021 · 154 citations
  3. Softermax: Hardware/Software Co-Design of an Efficient Softmax for TransformersJacob R. Stevens, Rangharajan Venkatesan, Steve Dai, Brucek Khailany et al.DAC 2021 · 143 citations
  4. Sparse is Enough in Scaling TransformersSebastian Jaszczur, Aakanksha Chowdhery, Afroz Mohiuddin, Lukasz Kaiser et al.NeurIPS 2021 · 127 citations
  5. HeatViT: Hardware-Efficient Adaptive Token Pruning for Vision TransformersPeiyan Dong, Mengshu Sun, Alec Lu, Yanyue Xie et al.HPCA 2023 · 117 citations

Related papers

  1. Extremely Low Bit Transformer Quantization for On-Device Neural Machine TranslationInsoo Chung, Byeongwook Kim, Yoonjung Choi, Se Jung Kwon et al.EMNLP Findings 2020 · 23 citations
  2. Mr.BiQ: Post-Training Non-Uniform Quantization based on Minimizing the Reconstruction ErrorYongkweon Jeon, Chungman Lee, Eulrang Cho, Yeonju RoCVPR 2022 · 28 citations
  3. 8-bit Transformer Inference and Fine-tuning for Edge AcceleratorsJeffrey Yu, Kartik Prabhu, Yonatan Urman, Robert M. Radway et al.ASPLOS 2024 · 26 citations
  4. Binarized Neural Machine TranslationYichi Zhang, Ankush Garg, Yuan Cao, Lukasz Lew et al.NeurIPS 2023 · 21 citations
  5. Hypoformer: Hybrid Decomposition Transformer for Edge-friendly Neural Machine TranslationSunzhu Li, Peng Zhang, Guobing Gan, Xiuqing Lv et al.EMNLP 2022 · 3 citations
Dusk over the sea between two cliffs drawn in fine vertical lines