Provable Convergence Bounds for Hybrid Dynamical Sampling and Optimization
Matthew X. Burns, Qingyuan Hou, Michael C. Huang
摘要
Analog dynamical accelerators (DXs) are a growing sub-field in computer architecture research, offering order-of-magnitude gains in power efficiency and latency over traditional digital methods in several machine learning, optimization, and sampling tasks. However, limited-capacity accelerators require hybrid analog/digital algorithms to solve real-world problems, commonly using largeneighborhood local search (LNLS) frameworks. Unlike fully digital algorithms, hybrid LNLS has no non-asymptotic convergence guarantees and no principled hyperparameter selection schemes, particularly limiting cross-device training and inference. In this work, we provide non-asymptotic convergence guarantees for hybrid LNLS by reducing to block Langevin Diffusion (BLD) algorithms. Adapting tools from classical sampling theory, we prove exponential KL-divergence convergence for randomized and cyclic block selection strategies using ideal DXs. With finite device variation, we provide explicit bounds on the 2-Wasserstein bias in terms of step duration, noise strength, and function parameters. Our BLD model provides a key link between established theory and novel computing platforms, and our theoretical results provide a closed-form expression linking device variation, algorithm hyperparameters, and performance.
问问这篇 Paper
智能体会读完全文。
Lune 把这篇 Paper 索引到了每一个公式,引用它的顶会 Paper 也一样。你提问,回答直接引用原文。
它引用的顶会 Paper6
- BRIM: Bistable Resistively-Coupled Ising MachineRichard Afoakwa, Yiqiao Zhang, Uday Kumar Reddy Vengalam, Zeljko Ignjatovic 等HPCA 2021 · 被引用 57 次
- Energy-based learning algorithms for analog computing: a comparative studyBenjamin Scellier, Maxence Ernoult, Jack D. Kendall, Suhas KumarNeurIPS 2023 · 被引用 54 次
- Time-independent Generalization Bounds for SGLD in Non-convex SettingsTyler Farghly, Patrick RebeschiniNeurIPS 2021 · 被引用 30 次
- Increasing ising machine capacity with multi-chip architecturesAnshujit Sharma, Richard Afoakwa, Zeljko Ignjatovic, Michael C. HuangISCA 2022 · 被引用 28 次
- Supporting Energy-based Learning with an Ising Machine substrate: a Case Study on RBMUday Kumar Reddy Vengalam, Yongchao Liu, Tong Geng, Hui Wu 等MICRO 2023 · 被引用 8 次
相关 Paper
- A Dynamical System View of Langevin-Based Non-Convex SamplingMohammad Reza Karimi Jaghargh, Ya-Ping Hsieh, Andreas KrauseNeurIPS 2023 · 被引用 4 次
- Towards Exact Gradient-based Training on Analog In-memory ComputingZhaoxian Wu, Tayfun Gokmen, Malte J. Rasch, Tianyi ChenNeurIPS 2024 · 被引用 11 次
- Shifted Composition IV: Toward Ballistic Acceleration for Log-Concave SamplingJason M. Altschuler, Sinho Chewi, Matthew S. ZhangSTOC 2026 · 被引用 9 次
- Dimension-Independent Convergence of Underdamped Langevin Monte Carlo in KL DivergenceShiyuan Zhang, Qiwei Di, Xuheng Li, Quanquan GuICML 2026
- Improved Convergence Rate of Stochastic Gradient Langevin Dynamics with Variance Reduction and its Application to OptimizationYuri Kinoshita, Taiji SuzukiNeurIPS 2022 · 被引用 24 次
