Lune

NeurIPS2023Top-tier venue

Train 'n Trade: Foundations of Parameter Markets

Tzu-Heng Huang, Harit Vishwakarma, Frederic Sala

Abstract

Organizations typically train large models individually. This is costly and time-consuming, particularly for large-scale foundation models. Such vertical production is known to be suboptimal. Inspired by this economic insight, we ask whether it is possible to leverage others' expertise by trading the constituent parts in models, i.e., sets of weights, as if they were market commodities. While recent advances in aligning and interpolating models suggest that doing so may be possible, a number of fundamental questions must be answered to create viable parameter markets. In this work, we address these basic questions, propose a framework containing the infrastructure necessary for market operations to take place, study strategies for exchanging parameters, and offer means for agents to monetize parameters. Excitingly, compared to agents who train siloed models from scratch, we show that it is possible to mutually gain by using the market, even in competitive settings. This suggests that the notion of parameter markets may be a useful paradigm for improving large-scale model training in the future. 1

Ask Lune

Know this paper in minutes, not an afternoon.

Lune can read every page of it and the top-tier papers that cite it, and gives you its claims, its evidence and its limits, each one cited to the paper it comes from. No skimming, no AI slop.

4 top-tier papers have cited it since. Know what they found before your reviewers do.

Questions to start from

Lune can draw on

the full paper · the top-tier papers that cite it · related work

Get the answer

Free to start. No credit card required.

Lineage

  • The Role of Permutation Invariance in Linear Mode Connectivity of Neural Networks cites Optimizing Mode Connectivity via Neuron Alignment
  • DICE: Data Influence Cascade in Decentralized Learning cites Decentralized Training of Foundation Models in Heterogeneous Environments
Builds onCited byLeft to right by year; the more cited, the higher.

Cited by top-tier papers4

  1. Evaluating Sample Utility for Efficient Data Selection by Mimicking Model WeightsTzu-Heng Huang, Manjot Bilkhu, John Cooper, Frederic Sala et al.ICML 2026 · 3 citations
  2. Braess's Paradox of Generative AIBoaz Taitler, Omer Ben-PoratAAAI 2025 · 2 citations
  3. Learning Classifiers That Induce MarketsYonatan Sommer, Ivri Hikri, Lotan Amit, Nir RosenfeldICML 2025
  4. DICE: Data Influence Cascade in Decentralized LearningTongtian Zhu, Wenhao Li, Can Wang, Fengxiang HeICLR 2025

Builds on11

  1. The Role of Permutation Invariance in Linear Mode Connectivity of Neural NetworksRahim Entezari, Hanie Sedghi, Olga Saukh, Behnam NeyshaburICLR 2022 · 301 citations
  2. Robust anomaly detection and backdoor attack detection via differential privacyMin Du, Ruoxi Jia, Dawn SongICLR 2020 · 194 citations
  3. Linear Mode Connectivity in Multitask and Continual LearningSeyed-Iman Mirzadeh, Mehrdad Farajtabar, Dilan Görür, Razvan Pascanu et al.ICLR 2021 · 176 citations
  4. Decentralized Training of Foundation Models in Heterogeneous EnvironmentsBinhang Yuan, Yongjun He, Jared Davis, Tianyi Zhang et al.NeurIPS 2022 · 157 citations
  5. Optimizing Mode Connectivity via Neuron AlignmentN. Joseph Tatro, Pin-Yu Chen, Payel Das, Igor Melnyk et al.NeurIPS 2020 · 104 citations

Related papers

  1. Offline Training of Language Model Agents with Functions as Learnable WeightsShaokun Zhang, Jieyu Zhang, Jiale Liu, Linxin Song et al.ICML 2024 · 41 citations
  2. Decentralized Diffusion ModelsDavid McAllister, Matthew Tancik, Jiaming Song, Angjoo KanazawaCVPR 2025
  3. Why Keep Your Doubts to Yourself? Trading Visual Uncertainties among Vision-Language ModelsJusheng Zhang, Yijia Fan, Kaitong Cai, Jing Yang et al.ICLR 2026 · 6 citations
  4. Fairshare Data Pricing via Data Valuation for Large Language ModelsLuyang Zhang, Cathy Jiao, Beibei Li, Chenyan XiongNeurIPS 2025 · 11 citations
  5. Strategic Data Sharing between CompetitorsNikita Tsoy, Nikola KonstantinovNeurIPS 2023 · 12 citations
Dusk over the sea between two cliffs drawn in fine vertical lines