XMC-Agent : Dynamic Navigation over Scalable Hierarchical Index for Incremental Extreme Multi-label Classification

The eXtreme Multi-label Classification (XMC) aims to accurately assign a large number of labels to instances, presenting challenges in learning, managing, and predicting across a vast and rapidly growing set of labels. Traditional XMC methods, such as one-vs-all and treebased methods struggle with the increasing set of labels due to their static label assumptions, while embedding-based methods face difficulties with complex mapping relationships due to their late-interaction paradigm. In this paper, we propose a large language model (LLM) powered agent framework for extreme multilabel classification -XMC-AGENT, which can effectively learn, manage and predict an extremely large and dynamically increasing set of labels. Specifically, XMC-AGENT models the extreme multi-label classification task as a dynamic navigation problem, employing a scalable hierarchical label index to effectively manage the unified label space. Additionally, we design two algorithms to enhance the dynamic navigation capabilities of XMC-AGENT: a selfconstruction algorithm for building the scalable hierarchical index, and an iterative feedback learning algorithm for adjusting the agent to specific tasks. Experiments demonstrate that XMC-AGENT achieves the state-of-the-art performance on three datasets.