FigStep: Jailbreaking Large Vision-Language Models via Typographic Visual Prompts

Large Vision-Language Models (LVLMs) signify a groundbreaking paradigm shift within the Artificial Intelligence (AI) community, extending beyond the capabilities of Large Language Models (LLMs) by assimilating additional modalities (e.g., images). Despite this advancement, the safety of LVLMs remains adequately underexplored, with a potential overreliance on the safety assurances purported by their underlying LLMs. In this paper, we propose FigStep, a straightforward yet effective black-box jailbreak algorithm against LVLMs. Instead of feeding textual harmful instructions directly, FigStep converts the prohibited content into images through typography to bypass the safety alignment. The experimental results indicate that FigStep can achieve an average attack success rate of 82.50% on six promising open-source LVLMs. Not merely to demonstrate the efficacy of FigStep, we conduct comprehensive ablation studies and analyze the distribution of the semantic embeddings to uncover that the reason behind the success of FigStep is the deficiency of safety alignment for visual embeddings. Moreover, we compare FigStep with five text-only jailbreaks and four image-based jailbreaks to demonstrate the superiority of FigStep, i.e., negligible attack costs and better attack performance. Above all, our work reveals that current LVLMs are vulnerable to jailbreak attacks, which highlights the necessity of novel cross-modality safety alignment techniques. Our code and datasets are available at https://github.com/ThuCCSLab/FigStep . Content Warning: This paper contains harmful model responses. INTRODUCTION Large Vision-Language Models (LVLMs) are at the forefront of the recent transformative wave in Artificial Intelligence (AI) research. Unlike single-modal Large Language Models (LLMs) like Chat-GPT [32] , LVLMs can process queries with both visual and textual modalities. Noteworthy LVLMs like and LLaVA [25] have remarkable abilities, which could enhance end-user-oriented scenarios like image captioning for blind people [56] or recommendation systems for children [12] , where content safety is crucial. Typically, an LVLM consists of a visual module, a connector, and a textual module (see Figure 1 ). To be specific, the visual module is an