Lune

ACL Findings2025

Statistical inference on black-box generative models in the data kernel perspective space

Hayden S. Helm, Aranyak Acharyya, Youngser Park, Brandon Duderstadt, Carey E. Priebe

Abstract

Generative models are capable of producing human-expert level content across a variety of topics and domains. As the impact of generative models grows, it is necessary to develop statistical methods to understand collections of available models. These methods are particularly important in settings where the user may not have access to information related to a model's pre-training data, weights, or other relevant model-level covariates. In this paper we extend recent results on representations of black-box generative models to model-level statistical inference tasks. We demonstrate that the model-level representations are effective for multiple inference tasks.

Ask Lune

Know this paper in minutes, not an afternoon.

Lune can read every page of it and the top-tier papers that cite it, and gives you its claims, its evidence and its limits, each one cited to the paper it comes from. No skimming, no AI slop.

3 top-tier papers have cited it since. Know what they found before your reviewers do.

Questions to start from

Lune can draw on

the full paper · the top-tier papers that cite it · related work

Get the answer

Free to start. No credit card required.

Lineage

  • QLoRA: Efficient Finetuning of Quantized LLMs cites LoRA: Low-Rank Adaptation of Large Language Models
  • Query-efficient model evaluation using cached responses cites Merging Models with Fisher-Weighted Averaging
Builds onCited byLeft to right by year; the more cited, the higher.

Cited by top-tier papers3

  1. Detecting Perspective Shifts in Multi-Agent SystemsEric Bridgeford, Hayden HelmICML 2026 · 4 citations
  2. A Model of the Language ProcessBrandon Duderstadt, Hayden S. HelmACL 2026 · 1 citation
  3. Query-efficient model evaluation using cached responsesHayden Helm, Ben Johnson, Carey PriebeICML 2026

Builds on9

  1. Retrieval-Augmented Generation for Knowledge-Intensive NLP TasksPatrick Lewis, Ethan Perez, Aleksandra Piktus, Fabio Petroni et al.NeurIPS 2020 · 19,162 citations
  2. LoRA: Low-Rank Adaptation of Large Language ModelsEdward J. Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu et al.ICLR 2022 · 18,833 citations
  3. QLoRA: Efficient Finetuning of Quantized LLMsTim Dettmers, Artidoro Pagnoni, Ari Holtzman, Luke ZettlemoyerNeurIPS 2023 · 5,863 citations
  4. Merging Models with Fisher-Weighted AveragingMichael Matena, Colin RaffelNeurIPS 2022 · 741 citations
  5. RealToxicityPrompts: Evaluating Neural Toxic Degeneration in Language ModelsSamuel Gehman, Suchin Gururangan, Maarten Sap, Yejin Choi et al.EMNLP Findings 2020 · 692 citations

Related papers

  1. Generative Models as a Data Source for Multiview Representation LearningAli Jahanian, Xavier Puig, Yonglong Tian, Phillip IsolaICLR 2022 · 148 citations
  2. Label-Free Explainability for Unsupervised ModelsJonathan Crabbé, Mihaela van der SchaarICML 2022 · 24 citations
  3. Advances in Black-Box VI: Normalizing Flows, Importance Weighting, and OptimizationAbhinav Agrawal, Daniel Sheldon, Justin DomkeNeurIPS 2020 · 49 citations
  4. Adversarially-learned Inference via an Ensemble of Discrete Undirected Graphical ModelsAdarsh K. Jeewajee, Leslie Pack KaelblingNeurIPS 2020 · 1 citation
  5. Generative Modeling of Weights: Generalization or Memorization?Boya Zeng, Yida Yin, Zhiqiu Xu, Zhuang LiuCVPR 2026 · 12 citations
Dusk over the sea between two cliffs drawn in fine vertical lines