Lune

EMNLP Findings2025

Transfer-Aware Data Selection for Domain Adaptation in Text Retrieval

Linzhu Yu, Huan Li, Ke Chen, Lidan Shou

Abstract

Domain adaptation is widely adopted in text retrieval scenarios where large labeled data is unavailable. To improve model adaptability, existing methods try to expand more source datasets. However, we found from experiments that indiscriminately using a large amount of source data from various text tasks does not guarantee improved adaptability, but may negatively impact ranking model performance. To tackle this issue, we propose Trait, a framework that can effectively improve model adaptability by selecting beneficial data without evaluating all source data. Specifically, we first divide multiple source datasets into data chunks of the same size as the minimum selection unit to form the whole selection space. Then we devise an iterative process that includes Bayesian optimization-based selection and transfer-aware chunk evaluation to incrementally select beneficial chunks. To reduce unnecessary evaluation costs, we also design backtracking and pruning actions to adjust the selection subspace. Extensive experimental results show that Trait not only achieves average state-of-the-art for few-shot on nine target datasets by evaluating only 4% of BERRI source data, but is also highly competitive for zero-shot compared with LLM-based rankers.

Ask Lune

Know this paper in minutes, not an afternoon.

Lune can read every page of it and the top-tier papers that cite it, and gives you its claims, its evidence and its limits, each one cited to the paper it comes from. No skimming, no AI slop.

Questions to start from

Lune can draw on

the full paper · related work

Get the answer

Free to start. No credit card required.

Lineage

  • One Embedder, Any Task: Instruction-Finetuned Text Embeddings cites Large Dual Encoders Are Generalizable Retrievers
  • Is ChatGPT Good at Search? Investigating Large Language Models as Re-Ranking Agents cites Document Ranking with a Pretrained Sequence-to-Sequence Model
Builds onCited byLeft to right by year; the more cited, the higher.

Builds on11

  1. Document Ranking with a Pretrained Sequence-to-Sequence ModelRodrigo Nogueira, Zhiying Jiang, Ronak Pradeep, Jimmy LinEMNLP Findings 2020 · 469 citations
  2. Is ChatGPT Good at Search? Investigating Large Language Models as Re-Ranking AgentsWeiwei Sun, Lingyong Yan, Xinyu Ma, Shuaiqiang Wang et al.EMNLP 2023 · 182 citations
  3. Large Dual Encoders Are Generalizable RetrieversJianmo Ni, Chen Qu, Jing Lu, Zhuyun Dai et al.EMNLP 2022 · 145 citations
  4. One Embedder, Any Task: Instruction-Finetuned Text EmbeddingsHongjin Su, Weijia Shi, Jungo Kasai, Yizhong Wang et al.ACL Findings 2023 · 134 citations
  5. UL2: Unifying Language Learning ParadigmsYi Tay, Mostafa Dehghani, Vinh Q. Tran, Xavier Garcia et al.ICLR 2023 · 97 citations

Related papers

  1. Few-Shot Text Ranking with Meta Adapted Synthetic Weak SupervisionSi Sun, Yingzhuo Qian, Zhenghao Liu, Chenyan Xiong et al.ACL 2021
  2. Zero-Shot Dense Retrieval with Momentum Adversarial Domain Invariant RepresentationsJi Xin, Chenyan Xiong, Ashwin Srinivasan, Ankita Sharma et al.ACL Findings 2022 · 19 citations
  3. Boot and Switch: Alternating Distillation for Zero-Shot Dense RetrievalFan Jiang, Qiongkai Xu, Tom Drummond, Trevor CohnEMNLP Findings 2023
  4. Zero-Shot Task Adaptation with Relevant Feature InformationAtsutoshi Kumagai, Tomoharu Iwata, Yasuhiro FujiwaraAAAI 2024 · 1 citation
  5. Few-shot Prompting for Pairwise Ranking: An Effective Non-Parametric Retrieval ModelNilanjan Sinhababu, Andrew Parry, Debasis Ganguly, Debasis Samanta et al.EMNLP Findings 2024 · 7 citations
Dusk over the sea between two cliffs drawn in fine vertical lines