Lune

ACL Findings2022

Dual Context-Guided Continuous Prompt Tuning for Few-Shot Learning

Jie Zhou, Le Tian, Houjin Yu, Xiao Zhou, Hui Su, Jie Zhou

Abstract

Prompt-based paradigm has shown its competitive performance in many NLP tasks. However, its success heavily depends on prompt design, and the effectiveness varies upon the model and training data. In this paper, we propose a novel dual context-guided continuous prompt (DCCP) tuning method. To explore the rich contextual information in language structure and close the gap between discrete prompt tuning and continuous prompt tuning, DCCP introduces two auxiliary training objectives and constructs input in a pair-wise fashion. Experimental results demonstrate that our method is applicable to many NLP tasks, and can often outperform existing prompt tuning methods by a large margin in the few-shot setting.

Ask Lune

Know this paper in minutes, not an afternoon.

Lune can read every page of it and the top-tier papers that cite it, and gives you its claims, its evidence and its limits, each one cited to the paper it comes from. No skimming, no AI slop.

1 top-tier paper has cited it since. Know what they found before your reviewers do.

Questions to start from

Lune can draw on

the full paper · the top-tier papers that cite it · related work

Get the answer

Free to start. No credit card required.

Lineage

  • Differentiable Prompt Makes Pre-trained Language Models Better Few-shot Learners cites AutoPrompt: Eliciting Knowledge from Language Models with Automatically Generated Prompts
  • Differentiable Prompt Makes Pre-trained Language Models Better Few-shot Learners cites Language Models are Few-Shot Learners
  • Differentiable Prompt Makes Pre-trained Language Models Better Few-shot Learners cites True Few-Shot Learning with Language Models
  • Differentiable Prompt Makes Pre-trained Language Models Better Few-shot Learners cites Calibrate Before Use: Improving Few-shot Performance of Language Models
  • AutoPrompt: Eliciting Knowledge from Language Models with Automatically Generated Prompts cites Language Models are Few-Shot Learners
  • True Few-Shot Learning with Language Models cites AutoPrompt: Eliciting Knowledge from Language Models with Automatically Generated Prompts
  • True Few-Shot Learning with Language Models cites Language Models are Few-Shot Learners
  • True Few-Shot Learning with Language Models cites Calibrate Before Use: Improving Few-shot Performance of Language Models
  • Calibrate Before Use: Improving Few-shot Performance of Language Models cites Language Models are Few-Shot Learners
Builds onCited byLeft to right by year; the more cited, the higher.

Cited by top-tier papers1

  1. Meta-Semantics Augmented Few-Shot Relational LearningHan Wu, Jie YinEMNLP 2025

Builds on7

  1. Language Models are Few-Shot LearnersTom B. Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah et al.NeurIPS 2020 · 64,255 citations
  2. Calibrate Before Use: Improving Few-shot Performance of Language ModelsZihao Zhao, Eric Wallace, Shi Feng, Dan Klein et al.ICML 2021 · 1,843 citations
  3. AutoPrompt: Eliciting Knowledge from Language Models with Automatically Generated PromptsTaylor Shin, Yasaman Razeghi, Robert L. Logan IV, Eric Wallace et al.EMNLP 2020 · 1,162 citations
  4. True Few-Shot Learning with Language ModelsEthan Perez, Douwe Kiela, Kyunghyun ChoNeurIPS 2021 · 547 citations
  5. Differentiable Prompt Makes Pre-trained Language Models Better Few-shot LearnersNingyu Zhang, Luoqiu Li, Xiang Chen, Shumin Deng et al.ICLR 2022 · 205 citations

Related papers

  1. Adversarial Knowledge Stimulated Contrastive Prompting for Few-shot Language LearnersKai Zheng, Qingfeng Sun, Yaming Yang, Tengchao Lv et al.ACL Findings 2023 · 2 citations
  2. Don't Stop Pretraining? Make Prompt-based Fine-tuning Powerful LearnerZhengxiang Shi, Aldo LipaniNeurIPS 2023 · 36 citations
  3. Consistency-guided Prompt Learning for Vision-Language ModelsShuvendu Roy, Ali EtemadICLR 2024 · 102 citations
  4. Multitask Pre-training of Modular Prompt for Chinese Few-Shot LearningTianxiang Sun, Zhengfu He, Qin Zhu, Xipeng Qiu et al.ACL 2023 · 15 citations
  5. Enhancing Cross-lingual Prompting with Dual Prompt AugmentationMeng Zhou, Xin Li, Yue Jiang, Lidong BingACL Findings 2023 · 1 citation
Dusk over the sea between two cliffs drawn in fine vertical lines