PANDA: Persona Attributes Navigation for Detecting and Alleviating Overuse Problem in Large Language Models

In the persona-grounded dialogue (PGD) task, it is required not only to respond fluently, but also to ground the attributes according to the current conversation topic properly. However, due to their tendency to overly ground given attributes, LLMs often generate unnatural responses provoked by using attributes that deviate from the flow of the conversation or by exploiting too many attributes at once. We term this phenomenon the overuse problem of LLMs. Unfortunately, research devising precise criteria and frameworks to quantitatively verify LLMs’ overuse problem is obviously insufficient. To address this issue, we propose Persona Attributes Navigation for Detecting and Alleviating the overuse problem (PANDA) framework. PANDA is the first study to quantify the persona overuse problem of LLMs by establishing clear standards of the problem and verifying various LLMs based on them. Moreover, this framework navigates us into understanding persona attributes by introducing diverse and detailed dialogue topics that consider practical conversation situations. We provide insights related to LLMs’ persona attribute overuse problem through comprehensive verification and analysis with PANDA in the PGD task. Our code and resources can be found at http://github.com/jin62304/PANDA.