Lune

DAC2023Top-tier venue

Map-and-Conquer: Energy-Efficient Mapping of Dynamic Neural Nets onto Heterogeneous MPSoCs

Halima Bouzidi, Mohanad Odema, Hamza Ouarnoughi, Smaïl Niar, Mohammad Abdullah Al Faruque

Abstract

Heterogeneous MPSoCs comprise diverse processing units of varying compute capabilities. To date, the mapping strategies of neural networks (NNs) onto such systems are yet to exploit the full potential of processing parallelism, made possible through both the intrinsic NNs’ structure and underlying hardware composition. In this paper, we propose a novel framework to effectively map NNs onto heterogeneous MPSoCs in a manner that enables them to leverage the underlying processing concurrency. Specifically, our approach identifies an optimal partitioning scheme of the NN along its ‘width’ dimension, which facilitates deployment of concurrent NN blocks onto different hardware computing units. Additionally, our approach contributes a novel scheme to deploy partitioned NNs onto the MPSoC as dynamic multi-exit networks for additional performance gains. Our experiments on a standard MPSoC platform have yielded dynamic mapping configurations that are 2.1x more energy-efficient than the GPU-only mapping while incurring 1.7x less latency than DLA-only mapping.

Ask Lune

Know this paper in minutes, not an afternoon.

Lune can read every page of it and the top-tier papers that cite it, and gives you its claims, its evidence and its limits, each one cited to the paper it comes from. No skimming, no AI slop.

1 top-tier paper has cited it since. Know what they found before your reviewers do.

Questions to start from

Lune can draw on

the full paper · the top-tier papers that cite it · related work

Get the answer

Free to start. No credit card required.

Lineage

  • MIA-Former: Efficient and Robust Vision Transformers via Multi-Grained Input-Adaptation cites DynamicViT: Efficient Vision Transformers with Dynamic Token Sparsification
Builds onCited byLeft to right by year; the more cited, the higher.

Cited by top-tier papers1

  1. SCAR: Scheduling Multi-Model AI Workloads on Heterogeneous Multi-Chiplet Module AcceleratorsMohanad Odema, Luke Chen, Hyoukjun Kwon, Mohammad Abdullah Al FaruqueMICRO 2024 · 11 citations

Builds on6

  1. DynamicViT: Efficient Vision Transformers with Dynamic Token SparsificationYongming Rao, Wenliang Zhao, Benlin Liu, Jiwen Lu et al.NeurIPS 2021 · 1,343 citations
  2. Visformer: The Vision-friendly TransformerZhengsu Chen, Lingxi Xie, Jianwei Niu, Xuefeng Liu et al.ICCV 2021 · 293 citations
  3. EdgeBERT: Sentence-Level Energy Optimizations for Latency-Aware Multi-Task NLP InferenceThierry Tambe, Coleman Hooper, Lillian Pentecost, Tianyu Jia et al.MICRO 2021 · 117 citations
  4. AxoNN: energy-aware execution of neural network inference on multi-accelerator heterogeneous SoCsIsmet Dagli, Alexander Cieslewicz, Jedidiah McClurg, Mehmet E. BelviranliDAC 2022 · 38 citations
  5. MIA-Former: Efficient and Robust Vision Transformers via Multi-Grained Input-AdaptationZhongzhi Yu, Yonggan Fu, Sicheng Li, Chaojian Li et al.AAAI 2022 · 20 citations

Related papers

  1. MARS: Exploiting Multi-Level Parallelism for DNN Workloads on Adaptive Multi-Accelerator SystemsGuan Shen, Jieru Zhao, Zeke Wang, Zhe Lin et al.DAC 2023 · 5 citations
  2. Accelerating DNN Inference with Heterogeneous Multi-DPU EnginesZelin Du, Wei Zhang, Zimeng Zhou, Zili Shao et al.DAC 2023 · 9 citations
  3. Memory and Computation Coordinated Mapping of DNNs onto Complex Heterogeneous SoCSize Zheng, Siyuan Chen, Yun LiangDAC 2023 · 10 citations
  4. H2H: heterogeneous model to heterogeneous system mapping with computation and communication awarenessXinyi Zhang, Cong Hao, Peipei Zhou, Alex K. Jones et al.DAC 2022 · 21 citations
  5. Shared Memory-contention-aware Concurrent DNN Execution for Diversely Heterogeneous System-on-ChipsIsmet Dagli, Mehmet E. BelviranliPPoPP 2024 · 18 citations
Dusk over the sea between two cliffs drawn in fine vertical lines