Provable Convergence Bounds for Hybrid Dynamical Sampling and Optimization
Matthew X. Burns, Qingyuan Hou, Michael C. Huang
Abstract
Analog dynamical accelerators (DXs) are a growing sub-field in computer architecture research, offering order-of-magnitude gains in power efficiency and latency over traditional digital methods in several machine learning, optimization, and sampling tasks. However, limited-capacity accelerators require hybrid analog/digital algorithms to solve real-world problems, commonly using largeneighborhood local search (LNLS) frameworks. Unlike fully digital algorithms, hybrid LNLS has no non-asymptotic convergence guarantees and no principled hyperparameter selection schemes, particularly limiting cross-device training and inference. In this work, we provide non-asymptotic convergence guarantees for hybrid LNLS by reducing to block Langevin Diffusion (BLD) algorithms. Adapting tools from classical sampling theory, we prove exponential KL-divergence convergence for randomized and cyclic block selection strategies using ideal DXs. With finite device variation, we provide explicit bounds on the 2-Wasserstein bias in terms of step duration, noise strength, and function parameters. Our BLD model provides a key link between established theory and novel computing platforms, and our theoretical results provide a closed-form expression linking device variation, algorithm hyperparameters, and performance.
Ask about this paper
Your agent reads all of it.
Lune indexed this paper to the last equation, along with the top-tier papers that cite it. Ask a question and the answer quotes them.
Your agent calls
Luneget_paper_fulltext
Free to start. No credit card required.
Terminal
Install the CLIlune papers fulltext 76041864-1f7a-4fb8-a3ae-ec5a0f4fe3dcBuilds on6
- BRIM: Bistable Resistively-Coupled Ising MachineRichard Afoakwa, Yiqiao Zhang, Uday Kumar Reddy Vengalam, Zeljko Ignjatovic et al.HPCA 2021 · 57 citations
- Energy-based learning algorithms for analog computing: a comparative studyBenjamin Scellier, Maxence Ernoult, Jack D. Kendall, Suhas KumarNeurIPS 2023 · 54 citations
- Time-independent Generalization Bounds for SGLD in Non-convex SettingsTyler Farghly, Patrick RebeschiniNeurIPS 2021 · 30 citations
- Increasing ising machine capacity with multi-chip architecturesAnshujit Sharma, Richard Afoakwa, Zeljko Ignjatovic, Michael C. HuangISCA 2022 · 28 citations
- Supporting Energy-based Learning with an Ising Machine substrate: a Case Study on RBMUday Kumar Reddy Vengalam, Yongchao Liu, Tong Geng, Hui Wu et al.MICRO 2023 · 8 citations
Related papers
- A Dynamical System View of Langevin-Based Non-Convex SamplingMohammad Reza Karimi Jaghargh, Ya-Ping Hsieh, Andreas KrauseNeurIPS 2023 · 4 citations
- Towards Exact Gradient-based Training on Analog In-memory ComputingZhaoxian Wu, Tayfun Gokmen, Malte J. Rasch, Tianyi ChenNeurIPS 2024 · 11 citations
- Shifted Composition IV: Toward Ballistic Acceleration for Log-Concave SamplingJason M. Altschuler, Sinho Chewi, Matthew S. ZhangSTOC 2026 · 9 citations
- Dimension-Independent Convergence of Underdamped Langevin Monte Carlo in KL DivergenceShiyuan Zhang, Qiwei Di, Xuheng Li, Quanquan GuICML 2026
- Improved Convergence Rate of Stochastic Gradient Langevin Dynamics with Variance Reduction and its Application to OptimizationYuri Kinoshita, Taiji SuzukiNeurIPS 2022 · 24 citations
