ACL Findings2025
Sci-LoRA: Mixture of Scientific LoRAs for Cross-Domain Lay Paraphrasing
Ming Cheng, Jiaying Gong, Hoda Eldardiry
Abstract
Lay paraphrasing aims to make scientific information accessible to audiences without technical backgrounds. However, most existing studies focus on a single domain, such as biomedicine. With the rise of interdisciplinary research, it is increasingly necessary to comprehend knowledge spanning multiple technical fields. To address this, we propose Sci-LoRA, a model that leverages a mixture of LoRAs finetuned on multiple scientific domains. In particular, Sci-LoRA dynamically generates and applies weights for each LoRA, enabling it to adjust the impact of different domains based on the input text, without requiring explicit domain labels. To balance domain-specific knowledge and generalization across various domains, Sci-LoRA integrates information at both the data and model levels. This dynamic fusion enhances the adaptability and performance across various domains. Experimental results across twelve domains on five public datasets show that Sci-LoRA significantly outperforms state-of-the-art large language models and demonstrates flexible generalization and adaptability in cross-domain lay paraphrasing.
Ask Lune
Know this paper in minutes, not an afternoon.
Lune can read every page of it and the top-tier papers that cite it, and gives you its claims, its evidence and its limits, each one cited to the paper it comes from. No skimming, no AI slop.
Lineage
Builds on16
- LoRA: Low-Rank Adaptation of Large Language ModelsEdward J. Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu et al.ICLR 2022 · 18,833 citations
- BERTScore: Evaluating Text Generation with BERTTianyi Zhang, Varsha Kishore, Felix Wu, Kilian Q. Weinberger et al.ICLR 2020 · 8,443 citations
- Automated Lay Language Summarization of Biomedical Scientific ReviewsYue Guo, Wei Qiu, Yizhong Wang, Trevor CohenAAAI 2021 · 100 citations
- Learning to Route Among Specialized Experts for Zero-Shot GeneralizationMohammed Muqeeth, Haokun Liu, Yufan Liu, Colin RaffelICML 2024 · 63 citations
- Making Science Simple: Corpora for the Lay Summarisation of Scientific LiteratureTomas Goldsack, Zhihao Zhang, Chenghua Lin, Carolina ScartonEMNLP 2022 · 38 citations
Related papers
- qa-FLoRA: Data-free query-adaptive Fusion of LoRAs for LLMsShreya Shukla, Aditya Sriram, Milinda Kuppur Narayanaswamy, Hiteshi JainAAAI 2026
- MLAS-LoRA: Language-Aware Parameters Detection and LoRA-Based Knowledge Transfer for Multilingual Machine TranslationTianyu Dong, Bo Li, Jinsong Liu, Shaolin Zhu et al.ACL 2025
- Magical: Medical Lay Language Generation via Semantic Invariance and Layperson-tailored AdaptationWeibin Liao, Tianlong Wang, Yinghao Zhu, Yasha Wang et al.NeurIPS 2025 · 7 citations
- LoraRetriever: Input-Aware LoRA Retrieval and Composition for Mixed Tasks in the WildZiyu Zhao, Leilei Gan, Guoyin Wang, Wangchunshu Zhou et al.ACL Findings 2024 · 10 citations
- Mixture-of-Subspaces in Low-Rank AdaptationTaiqiang Wu, Jiahao Wang, Zhe Zhao, Ngai WongEMNLP 2024 · 14 citations
