Lune

ICML2025Top-tier venue

Cannot See the Forest for the Trees: Invoking Heuristics and Biases to Elicit Irrational Choices of LLMs

Haoming Yang, Ke Ma, Xiaojun Jia, Yingfei Sun, Qianqian Xu, Qingming Huang

Abstract

Despite the remarkable performance of Large Language Models (LLMs), they remain vulnerable to jailbreak attacks, which can compromise their safety mechanisms. Existing studies often rely on brute-force optimization or manual design, failing to uncover potential risks in real-world scenarios. To address this, we propose a novel jailbreak attack framework, ICRT, inspired by heuristics and biases in human cognition. Leveraging the simplicity effect, we employ cognitive decomposition to reduce the complexity of malicious prompts. Simultaneously, relevance bias is utilized to reorganize prompts, enhancing semantic alignment and inducing harmful outputs effectively. Furthermore, we introduce a ranking-based harmfulness evaluation metric that surpasses the traditional binary success-or-failure paradigm by employing ranking aggregation methods such as Elo, HodgeRank, and Rank Centrality to comprehensively quantify the harmfulness of generated content. Experimental results show that our approach consistently bypasses mainstream LLMs' safety mechanisms and generates high-risk content.

Ask Lune

Know this paper in minutes, not an afternoon.

Lune can read every page of it and the top-tier papers that cite it, and gives you its claims, its evidence and its limits, each one cited to the paper it comes from. No skimming, no AI slop.

7 top-tier papers have cited it since. Know what they found before your reviewers do.

Questions to start from

Lune can draw on

the full paper · the top-tier papers that cite it · related work

Get the answer

Free to start. No credit card required.

Lineage

  • GPT-4 Is Too Smart To Be Safe: Stealthy Chat with LLMs via Cipher cites Training language models to follow instructions with human feedback
  • Fine-tuning Aligned Language Models Compromises Safety, Even When Users Do Not Intend To! cites Training language models to follow instructions with human feedback
  • Analogy-based Multi-Turn Jailbreak against Large Language Models cites AutoDAN: Generating Stealthy Jailbreak Prompts on Aligned Large Language Models
  • SoK: Robustness in Large Language Models against Jailbreak Attacks cites GPT-4 Is Too Smart To Be Safe: Stealthy Chat with LLMs via Cipher
  • SoK: Robustness in Large Language Models against Jailbreak Attacks cites Training language models to follow instructions with human feedback
  • SoK: Robustness in Large Language Models against Jailbreak Attacks cites AutoDAN: Generating Stealthy Jailbreak Prompts on Aligned Large Language Models
  • Training language models to follow instructions with human feedback cites Extracting Training Data from Large Language Models
  • AutoDAN: Generating Stealthy Jailbreak Prompts on Aligned Large Language Models cites GPT-4 Is Too Smart To Be Safe: Stealthy Chat with LLMs via Cipher
  • AutoDAN: Generating Stealthy Jailbreak Prompts on Aligned Large Language Models cites Training language models to follow instructions with human feedback
Builds onCited byLeft to right by year; the more cited, the higher.

Cited by top-tier papers7

  1. SeCon-RAG: A Two-Stage Semantic Filtering and Conflict-Free Framework for Trustworthy RAGXiaonan Si, Meilin Zhu, Simeng Qin, Lijia Yu et al.NeurIPS 2025 · 16 citations
  2. Analogy-based Multi-Turn Jailbreak against Large Language ModelsMengjie Wu, Yihao Huang, Zhenjun Lin, Kangjie Chen et al.NeurIPS 2025 · 9 citations
  3. Obscure but Effective: Classical Chinese Jailbreak Prompt Optimization via Bio-Inspired SearchXun Huang, Simeng Qin, Xiaoshuang Jia, Ranjie Duan et al.ICLR 2026 · 9 citations
  4. SoK: Robustness in Large Language Models against Jailbreak AttacksFeiyue Xu, Hongsheng Hu, Chaoxiang He, Sheng Hang et al.S&P 2026 · 4 citations
  5. PBI-Attack: Prior-Guided Bimodal Interactive Black-Box Jailbreak Attack for Toxicity MaximizationRuoxi Cheng, Yizhong Ding, Shuirong Cao, Ranjie Duan et al.EMNLP 2025 · 1 citation

Builds on20

  1. Training language models to follow instructions with human feedbackLong Ouyang, Jeffrey Wu, Xu Jiang, Diogo Almeida et al.NeurIPS 2022 · 24,707 citations
  2. Extracting Training Data from Large Language ModelsNicholas Carlini, Florian Tramèr, Eric Wallace, Matthew Jagielski et al.USENIX Security 2021 · 2,866 citations
  3. Fine-tuning Aligned Language Models Compromises Safety, Even When Users Do Not Intend To!Xiangyu Qi, Yi Zeng, Tinghao Xie, Pin-Yu Chen et al.ICLR 2024 · 1,104 citations
  4. AutoDAN: Generating Stealthy Jailbreak Prompts on Aligned Large Language ModelsXiaogeng Liu, Nan Xu, Muhao Chen, Chaowei XiaoICLR 2024 · 722 citations
  5. GPT-4 Is Too Smart To Be Safe: Stealthy Chat with LLMs via CipherYouliang Yuan, Wenxiang Jiao, Wenxuan Wang, Jen-tse Huang et al.ICLR 2024 · 441 citations

Related papers

  1. Dual Intention Escape: Penetrating and Toxic Jailbreak Attack against Large Language ModelsYanni Xue, Jiakai Wang, Zixin Yin, Yuqing Ma et al.WWW 2025 · 5 citations
  2. DrAttack: Prompt Decomposition and Reconstruction Makes Powerful LLMs JailbreakersXirui Li, Ruochen Wang, Minhao Cheng, Tianyi Zhou et al.EMNLP Findings 2024 · 27 citations
  3. Speak Easy: Eliciting Harmful Jailbreaks from LLMs with Simple InteractionsYik Siu Chan, Narutatsu Ri, Yuxin Xiao, Marzyeh GhassemiICML 2025
  4. Jailbreaking Attacks vs. Content Safety Filters: How Far Are We in the LLM Safety Arms Race?Yuan Xin, Dingfan Chen, Linyi Yang, Michael Backes et al.ACL Findings 2026
  5. Exploring Jailbreak Attacks on LLMs through Intent Concealment and DiversionTiehan Cui, Yanxu Mao, Peipei Liu, Congying Liu et al.ACL Findings 2025 · 3 citations
Dusk over the sea between two cliffs drawn in fine vertical lines