iLoRA: Bayesian Low-Rank Adaptation with Latent Interaction Graphs for Microbiome Diagnosis
Yang Song, Yixuan Zhang, Lingfa Meng, Tongyuan Hu, Haizhou Shi, Hao Wang, Samir Bhatt, Hengguan Huang
Abstract
Parameter-efficient adaptation has made LLMs practical for domain prediction, but standard LoRA still relies on a static low-rank update and does not expose the latent interactions that often drive scientific labels. We introduce iLoRA. To our knowledge, it is the first Bayesian graph-conditioned LoRA framework. It infers a latent interaction graph from the input and uses it to generate input-conditioned LoRA updates. As a result, iLoRA learns prediction and latent interaction structure jointly, rather than training a predictor and applying interaction analysis only post hoc. We instantiate this idea for microbiome diagnosis, where disease state can depend on both species-level abundance and microbe–microbe cross-talk, and evaluate it in two complementary settings: interactive QA with human-annotated graphs, which tests latent structure recovery, and multi-cohort IBD diagnosis, which tests biomedical utility. Across both settings, iLoRA improves over strong LoRA and Bayesian adaptation baselines, recovers graphs aligned with human annotations and cohort-level microbiome associations, and provides calibrated uncertainty with moderate graph-branch overhead.
Ask Lune
Know this paper in minutes, not an afternoon.
Lune can read every page of it and the top-tier papers that cite it, and gives you its claims, its evidence and its limits, each one cited to the paper it comes from. No skimming, no AI slop.
Lineage
- QLoRA: Efficient Finetuning of Quantized LLMs cites The Power of Scale for Parameter-Efficient Prompt Tuning
- QLoRA: Efficient Finetuning of Quantized LLMs cites Few-Shot Parameter-Efficient Fine-Tuning is Better and Cheaper than In-Context Learning
- Bayesian Low-rank Adaptation for Large Language Models cites Few-Shot Parameter-Efficient Fine-Tuning is Better and Cheaper than In-Context Learning
- Few-Shot Parameter-Efficient Fine-Tuning is Better and Cheaper than In-Context Learning cites The Power of Scale for Parameter-Efficient Prompt Tuning
Builds on8
- QLoRA: Efficient Finetuning of Quantized LLMsTim Dettmers, Artidoro Pagnoni, Ari Holtzman, Luke ZettlemoyerNeurIPS 2023 · 5,863 citations
- Few-Shot Parameter-Efficient Fine-Tuning is Better and Cheaper than In-Context LearningHaokun Liu, Derek Tam, Mohammed Muqeeth, Jay Mohta et al.NeurIPS 2022 · 1,483 citations
- Bayesian Low-rank Adaptation for Large Language ModelsAdam X. Yang, Maxime Robeyns, Xi Wang, Laurence AitchisonICLR 2024 · 111 citations
- The Power of Scale for Parameter-Efficient Prompt TuningBrian Lester, Rami Al-Rfou, Noah ConstantEMNLP 2021 · 94 citations
- Deep Graph Random Process for Relational-Thinking-Based Speech RecognitionHengguan Huang, Fuzhao Xue, Hao Wang, Ye WangICML 2020 · 18 citations
Related papers
- Minimal Ranks, Maximum Confidence: Parameter-efficient Uncertainty Quantification for LoRAPatryk Marszalek, Klaudia Balazy, Jacek Tabor, Tomasz KusmierczykEMNLP Findings 2025 · 2 citations
- GraphLoRA: Structure-Aware Low-Rank Adaptation for Large Language Model RecommendationLin Mu, Guoji Wang, Li Ni, Lei Sang et al.ACL Findings 2026
- Bayesian-LoRA: Probabilistic Low-Rank Adaptation of Large Language ModelsMoule Lin, Shuhao Guan, Andrea Patane, David Gregg et al.ICML 2026
- Adaptive Utilization of Low-Rank Adaptation via Conditioned GatingGuang Yang, Changhao Guan, Chao Huang, Yufeng Chen et al.ICML 2026
- dLoRA: Dynamically Orchestrating Requests and Adapters for LoRA LLM ServingBingyang Wu, Ruidong Zhu, Zili Zhang, Peng Sun et al.OSDI 2024 · 79 citations
