EMNLP Findings2023
SHARCS: Efficient Transformers Through Routing with Dynamic Width Sub-networks
Mohammadreza Salehi, Sachin Mehta, Aditya Kusupati, Ali Farhadi, Hannaneh Hajishirzi
Abstract
We introduce SHARCS for adaptive inference that takes into account the hardness of input samples. SHARCS can train a router on any transformer network, enabling the model to direct different samples to sub-networks with varying widths. Our experiments demonstrate that: (1) SHARCS outperforms or complements existing per-sample adaptive inference methods across various classification tasks in terms of accuracy vs. FLOPs; (2) SHARCS generalizes across different architectures and can be even applied to compressed and efficient transformer encoders to further improve their efficiency; (3) SHARCS can provide a 2× inference speed up at an insignificant drop in accuracy.
Ask Lune
Know this paper in minutes, not an afternoon.
Lune can read every page of it and the top-tier papers that cite it, and gives you its claims, its evidence and its limits, each one cited to the paper it comes from. No skimming, no AI slop.
3 top-tier papers have cited it since. Know what they found before your reviewers do.
Lune can draw on
the full paper · the top-tier papers that cite it · related work
Free to start. No credit card required.
Lineage
- TinyBERT: Distilling BERT for Natural Language Understanding cites MobileBERT: a Compact Task-Agnostic BERT for Resource-Limited Devices
- MobileBERT: a Compact Task-Agnostic BERT for Resource-Limited Devices cites TinyBERT: Distilling BERT for Natural Language Understanding
- MatFormer: Nested Transformer for Elastic Inference cites Language Models are Few-Shot Learners
- HydraViT: Stacking Heads for a Scalable ViT cites MatFormer: Nested Transformer for Elastic Inference
- Movement Pruning: Adaptive Sparsity by Fine-Tuning cites TinyBERT: Distilling BERT for Natural Language Understanding
- Movement Pruning: Adaptive Sparsity by Fine-Tuning cites Language Models are Few-Shot Learners
Cited by top-tier papers3
- MatFormer: Nested Transformer for Elastic InferenceDevvrit, Sneha Kudugunta, Aditya Kusupati, Tim Dettmers et al.NeurIPS 2024 · 97 citations
- HydraViT: Stacking Heads for a Scalable ViTJanek Haberer, Ali Hojjat, Olaf LandsiedelNeurIPS 2024 · 11 citations
- SoK: Efficiency Robustness of Dynamic Deep Learning SystemsRavishka Rathnasuriya, Tingxi Li, Zexin Xu, Zihe Song et al.USENIX Security 2025
Builds on16
- Language Models are Few-Shot LearnersTom B. Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah et al.NeurIPS 2020 · 64,255 citations
- TinyBERT: Distilling BERT for Natural Language UnderstandingXiaoqi Jiao, Yichun Yin, Lifeng Shang, Xin Jiang et al.EMNLP Findings 2020 · 1,787 citations
- BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and ComprehensionMike Lewis, Yinhan Liu, Naman Goyal, Marjan Ghazvininejad et al.ACL 2020 · 1,224 citations
- MobileBERT: a Compact Task-Agnostic BERT for Resource-Limited DevicesZhiqing Sun, Hongkun Yu, Xiaodan Song, Renjie Liu et al.ACL 2020 · 660 citations
- Movement Pruning: Adaptive Sparsity by Fine-TuningVictor Sanh, Thomas Wolf, Alexander M. RushNeurIPS 2020 · 656 citations
Related papers
- Adaptive Depth Networks with Skippable Sub-PathsWoochul Kang, Hyungseop LeeNeurIPS 2024 · 5 citations
- Adaptive Computation Modules: Granular Conditional Computation for Efficient InferenceBartosz Wójcik, Alessio Devoto, Karol Pustelnik, Pasquale Minervini et al.AAAI 2025 · 8 citations
- AdaPerceiver: Transformers with Adaptive Width, Depth, and TokensPurvish Jajal, Nicholas John Eliopoulos, Benjamin Shiue-Hal Chou, George K Thiruvathukal, Yung-Hsiang Lu, James C. DavisCVPR Findings 2026
- Faster Depth-Adaptive TransformersYijin Liu, Fandong Meng, Jie Zhou, Yufeng Chen et al.AAAI 2021 · 44 citations
- AdaBM: On-the-Fly Adaptive Bit Mapping for Image Super-ResolutionCheeun Hong, Kyoung Mu LeeCVPR 2024
