Convolutional Learnable-Group Weightless Neural Network
Qinhong Ma, Yulin Chen, Zhiwei Fan, Suzhen Wu, Bo Mao
Abstract
Weightless Neural Networks (WNNs) based on interconnected Lookup Tables (LUTs) have attracted attention for inference in extremely compact models, but achieving competitive accuracy under such tight resource budgets remains challenging. To address these issues, we introduce the Convolutional Learnable-Group Weightless Neural Network (CLGN). CLGN constructs convolutional layers using LUTs and incorporates a learnable GroupSum connection, thereby enhancing the accuracy of WNNs while maintaining low implementation resource consumption. Moreover, we propose a hierarchical training strategy to improve the training efficiency. We evaluate CLGN in two edge computing scenarios: (1) FPGA, where we evaluate accuracy, latency, throughput, power consumption, LUTs usage, and parameter size; and (2) Microprocessor, where we evaluate latency and memory usage. Compared with the state-of-the-art solutions, the proposed CLGN achieves superior accuracy while maintaining lower implementation resource consumption.
Ask about this paper
Your agent reads all of it.
Lune indexed this paper to the last equation, along with the top-tier papers that cite it. Ask a question and the answer quotes them.
Your agent calls
Luneget_paper_fulltext
Free to start. No credit card required.
Terminal
Install the CLIlune papers fulltext 4610408d-e7a7-435c-af0a-38e8dc175b7dBuilds on11
- Training Neural Networks with Fixed Sparse MasksYi-Lin Sung, Varun Nair, Colin RaffelNeurIPS 2021 · 295 citations
- Deep Differentiable Logic Gate NetworksFelix Petersen, Christian Borgelt, Hilde Kuehne, Oliver DeussenNeurIPS 2022 · 117 citations
- Adaptive Gradient Quantization for Data-Parallel SGDFartash Faghri, Iman Tabrizian, Ilia Markov, Dan Alistarh et al.NeurIPS 2020 · 108 citations
- DominoSearch: Find layer-wise fine-grained N: M sparse schemes from dense neural networksWei Sun, Aojun Zhou, Sander Stuijk, Rob G. J. Wijnhoven et al.NeurIPS 2021 · 67 citations
- Convolutional Differentiable Logic Gate NetworksFelix Petersen, Hilde Kuehne, Christian Borgelt, Julian Welzel et al.NeurIPS 2024 · 58 citations
Related papers
- Differentiable Weightless Neural NetworksAlan Tendler Leibel Bacellar, Zachary Susskind, Maurício Breternitz Jr., Eugene John et al.ICML 2024 · 34 citations
- Mix and Match: A Novel FPGA-Centric Deep Neural Network Quantization FrameworkSung-En Chang, Yanyu Li, Mengshu Sun, Runbin Shi et al.HPCA 2021 · 125 citations
- ShiftAddNet: A Hardware-Inspired Deep NetworkHaoran You, Xiaohan Chen, Yongan Zhang, Chaojian Li et al.NeurIPS 2020 · 99 citations
- MST-compression: Compressing and Accelerating Binary Neural Networks with Minimum Spanning TreeQuang Hieu Vo, Linh-Tam Tran, Sung-Ho Bae, Lok-Won Kim et al.ICCV 2023 · 2 citations
- A2Q: Accumulator-Aware Quantization with Guaranteed Overflow AvoidanceIan Colbert, Alessandro Pappalardo, Jakoba Petri-KoenigICCV 2023 · 19 citations
