Experience Speaks Louder: Black-box Hard-label Adversarial Attack through Reinforcement Learning

Black-box hard-label adversarial attack on text can be used to evaluate the robustness of text classification models (TCMs), allowing for a more thorough examination of potential security flaws before deployment. Research on this problem is still in the embryonic stage and only a few methods are available. Existing adversarial attack methods often treat each adversarial text generation as an independent task, failing to leverage past adversarial experience to facilitate subsequent generation. Consequently, these methods suffer from high query costs (i.e., the number of times TCMs need to be accessed to generate adversarial text) and pose challenges in generating adversarial texts with high semantic similarity and low perturbation rates under a query-limited setup. To achieve query-efficient adversarial text generation, we propose EGAttack, an experience-guided black-box hard-label attack method based on reinforcement learning. Specifically, we models adversarial text generation as a sequential decision process and propose a new adversarial environment to capture the intricate relationships among word perturbations, text semantic, and the target model's outputs for agent learning. Meanwhile, we extend the classical reinforcement framework DDQN into adversarial DDQN (ADDQN) through designing a new adversarial agent and a new dual-objective constraint, driving EGAttack to learn from past adversarial experience while identifying the vulnerabilities of TCMs, thus generating high-quality adversarial texts. Experimental results across various target models and datasets demonstrate that EGAttack generates adversarial texts with higher semantic similarity, lower perturbation rates, and competitive attack success rate, making it an efficient and effective tool for evaluating the robustness of TCMs.