Lune

ICML2026Top-tier venue

iLoRA: Bayesian Low-Rank Adaptation with Latent Interaction Graphs for Microbiome Diagnosis

Yang Song, Yixuan Zhang, Lingfa Meng, Tongyuan Hu, Haizhou Shi, Hao Wang, Samir Bhatt, Hengguan Huang

Abstract

Parameter-efficient adaptation has made LLMs practical for domain prediction, but standard LoRA still relies on a static low-rank update and does not expose the latent interactions that often drive scientific labels. We introduce iLoRA. To our knowledge, it is the first Bayesian graph-conditioned LoRA framework. It infers a latent interaction graph from the input and uses it to generate input-conditioned LoRA updates. As a result, iLoRA learns prediction and latent interaction structure jointly, rather than training a predictor and applying interaction analysis only post hoc. We instantiate this idea for microbiome diagnosis, where disease state can depend on both species-level abundance and microbe–microbe cross-talk, and evaluate it in two complementary settings: interactive QA with human-annotated graphs, which tests latent structure recovery, and multi-cohort IBD diagnosis, which tests biomedical utility. Across both settings, iLoRA improves over strong LoRA and Bayesian adaptation baselines, recovers graphs aligned with human annotations and cohort-level microbiome associations, and provides calibrated uncertainty with moderate graph-branch overhead.

Ask Lune

Know this paper in minutes, not an afternoon.

Lune can read every page of it and the top-tier papers that cite it, and gives you its claims, its evidence and its limits, each one cited to the paper it comes from. No skimming, no AI slop.

Questions to start from

Lune can draw on

the full paper · related work

Get the answer

Free to start. No credit card required.

Lineage

  • QLoRA: Efficient Finetuning of Quantized LLMs cites The Power of Scale for Parameter-Efficient Prompt Tuning
  • QLoRA: Efficient Finetuning of Quantized LLMs cites Few-Shot Parameter-Efficient Fine-Tuning is Better and Cheaper than In-Context Learning
  • Bayesian Low-rank Adaptation for Large Language Models cites Few-Shot Parameter-Efficient Fine-Tuning is Better and Cheaper than In-Context Learning
  • Few-Shot Parameter-Efficient Fine-Tuning is Better and Cheaper than In-Context Learning cites The Power of Scale for Parameter-Efficient Prompt Tuning
Builds onCited byLeft to right by year; the more cited, the higher.

Builds on8

  1. QLoRA: Efficient Finetuning of Quantized LLMsTim Dettmers, Artidoro Pagnoni, Ari Holtzman, Luke ZettlemoyerNeurIPS 2023 · 5,863 citations
  2. Few-Shot Parameter-Efficient Fine-Tuning is Better and Cheaper than In-Context LearningHaokun Liu, Derek Tam, Mohammed Muqeeth, Jay Mohta et al.NeurIPS 2022 · 1,483 citations
  3. Bayesian Low-rank Adaptation for Large Language ModelsAdam X. Yang, Maxime Robeyns, Xi Wang, Laurence AitchisonICLR 2024 · 111 citations
  4. The Power of Scale for Parameter-Efficient Prompt TuningBrian Lester, Rami Al-Rfou, Noah ConstantEMNLP 2021 · 94 citations
  5. Deep Graph Random Process for Relational-Thinking-Based Speech RecognitionHengguan Huang, Fuzhao Xue, Hao Wang, Ye WangICML 2020 · 18 citations

Related papers

  1. Minimal Ranks, Maximum Confidence: Parameter-efficient Uncertainty Quantification for LoRAPatryk Marszalek, Klaudia Balazy, Jacek Tabor, Tomasz KusmierczykEMNLP Findings 2025 · 2 citations
  2. GraphLoRA: Structure-Aware Low-Rank Adaptation for Large Language Model RecommendationLin Mu, Guoji Wang, Li Ni, Lei Sang et al.ACL Findings 2026
  3. Bayesian-LoRA: Probabilistic Low-Rank Adaptation of Large Language ModelsMoule Lin, Shuhao Guan, Andrea Patane, David Gregg et al.ICML 2026
  4. Adaptive Utilization of Low-Rank Adaptation via Conditioned GatingGuang Yang, Changhao Guan, Chao Huang, Yufeng Chen et al.ICML 2026
  5. dLoRA: Dynamically Orchestrating Requests and Adapters for LoRA LLM ServingBingyang Wu, Ruidong Zhu, Zili Zhang, Peng Sun et al.OSDI 2024 · 79 citations
Dusk over the sea between two cliffs drawn in fine vertical lines