Attribute Controlled Dialogue Prompting

Prompt-tuning has become an increasingly popular parameter-efficient method for adapting large pretrained language models to downstream tasks. However, both discrete prompting and continuous prompting assume fixed prompts for all data samples within a task, neglecting the fact that inputs vary greatly in some tasks such as open-domain dialogue generation. In this paper, we present a novel, instancespecific prompt-tuning algorithm for dialogue generation. Specifically, we generate prompts based on instance-level control code, rather than the conversation history, to explore their impact on controlled dialogue generation. Experiments on popular open-domain dialogue datasets, evaluated on both automated metrics and human evaluation, demonstrate that our method is superior to prompting baselines and comparable to fine-tuning with only 5%-6% of total parameters. * Work done during an internship at Huawei. overhead. We present results on both intent and persona controlled dialogue. 2 Related Work GPT-3 (Brown et al., 2020) introduces prompting, a method to steer a frozen PLM by transforming inputs into cloze-style phrases with task description and some task examples. Though it is memoryefficient since one single copy of the PLM can be shared across different tasks, the model's performance is largely restricted by the maximum conditional input length, the model size and manual guesswork for prompts (Zhao et al., 2021; Schick and Schütze, 2021a,b; Jiang et al., 2020) . Other works focus on automatically searching for better discrete prompts (Jiang et al., 2020; Shin et al., 2020; Gao et al., 2021; Ben-David et al., 2021) . Recently, there has been an increased interest in continuous prompts / prompt-tuning, which bridges the gap between prompting and fine-tuning, while remaining efficient during training (