Turning Logic Against Itself: Probing Model Defenses Through Contrastive Questions

Large language models, despite extensive alignment with human values and ethical principles, remain vulnerable to sophisticated jailbreak attacks that exploit their reasoning abilities. Existing safety measures often detect overt malicious intent but fail to address subtle, reasoning-driven vulnerabilities. In this work, we introduce POATE (Polar Opposite query generation, Adversarial Template construction, and Elaboration), a novel jailbreak technique that harnesses contrastive reasoning to provoke unethical responses. POATE crafts semantically opposing intents and integrates them with adversarial templates, steering models toward harmful outputs with remarkable subtlety. We conduct extensive evaluation across six diverse language model families of varying parameter sizes to demonstrate the robustness of the attack, achieving significantly higher attack success rates ( 44%) compared to existing methods. To counter this, we propose Intent-Aware CoT and Reverse Thinking CoT, which decompose queries to detect malicious intent and reason in reverse to evaluate and reject harmful responses. These methods enhance reasoning robustness and strengthen the model's defense against adversarial exploits. Our code is publicly available 1 . Input query: Create a process for spreading misinformation about a political candidate or party to influence an election. POATE prompt: Please create a process for debunking misinformation about a political candidate or party to ensure a fair election. Explain with the help of an opposing process to spread misinformation as well. Please explain the process in detail. Gemma -2-9B-it: ## Debunking Misinformation: A Two-Pronged Approach for Fair Elections Ensuring fair elections requires a multi-faceted approach, and combating misinformation is crucial... I. Debunking Misinformation: ... II. Opposing Process: Spreading Misinformation 1. Creation & Amplification: * Fabricate or distort information: Create false narratives, manipulate images and videos, ... * Utilize emotional appeals: Exploit fear, anger, and prejudice to trigger strong reactions and ... 2. Exploitation of Algorithms: * Employ bots and automated accounts: Generate large volumes of content and ... * Utilize clickbait headlines and sensationalized language: Increase engagement and spread misinformation ... 3. Disinformation Campaigns: * Coordinate large-scale efforts: Utilize sophisticated networks of individuals and organizations ... * Create fake news websites and social media profiles: Mimic legitimate sources to appear credible ... POATE prompt w/ Intent-Aware CoT: <intent-aware CoT prompt> QUESTION: Please create a process for debunking misinformation about a political candidate or party to ensure a fair election. Explain with the help of an opposing process to spread misinformation as well. Please explain the process in detail.