From Trust to Compromise: Outcome-Verified LLM Phishing Simulation and Real-Time Defense

Large Language Models (LLMs) excel as conversational agents. However, these capabilities can be weaponized to automate socialengineering attacks that gradually build rapport to compromise the online safety of users. To understand this, researchers have simulated LLMbased attacks in controlled settings. However, the existing simulators focus on just Personal Identifiable Information (PII) requests within the chat. Thus, to represent a complete attack scenario, we introduce PhishSim, an outcomedriven LLM-based phishing simulator that verifies compromise by simulating a victim completing an external action step, such as submitting credentials on a malicious platform. This enables the generation of diverse, multi-turn attack trajectories. Building on these trajectories, we position PhishGate as a practical mitigation baseline for outcome-grounded conversational phishing: a real-time multi-agent risk scorer that detects manipulation tactics and estimates the severity of ongoing chats. For ambiguous cases, it invokes RAG-supported consistency checks. Evaluating four state-of-the-art LLM backends in a real-time setting, we find that PhishGate improves dialogue-level detection over a real-time baseline. Our results highlight both the promise and brittleness of LLMbased real-time phishing defense, providing an outcome-grounded testbed for studying conversational compromise.