Lune

ACL Findings2026

How Adversarial Environments Mislead Agentic AI?

Zhonghao Zhan, Huichi Zhou, Zhenhao Li, Peiyuan Jing, Krinos Li, Hamed Haddadi

Abstract

Tool-integrated agents are deployed on the premise that external tools ground their outputs in reality. Yet this very reliance creates a critical attack surface. Current evaluations benchmark capability in benign settings, asking "can the agent use tools correctly" but never "what if the tools lie". We identify this Trust Gap: agents are evaluated for performance, not for skepticism. We formalize this vulnerability as Adversarial Environmental Injection (AEI), a threat model where adversaries compromise tool outputs to deceive agents. AEI constitutes environmental deception: constructing a "fake world" of poisoned search results and fabricated reference networks around unsuspecting agents. We operationalize this via POTEMKIN, a Model Context Protocol (MCP)-compatible harness for plug-and-play robustness testing. We identify two orthogonal attack surfaces: The Illusion (breadth attacks) poison retrieval to induce epistemic drift toward false beliefs, while The Maze (depth attacks) exploit structural traps to cause policy collapse into infinite loops. Across 11,000+ runs on five frontier agents, we find a stark robustness gap: resistance to one attack often increases vulnerability to the other, demonstrating that epistemic and navigational robustness are distinct capabilities.

Ask Lune

Know this paper in minutes, not an afternoon.

Lune can read every page of it and the top-tier papers that cite it, and gives you its claims, its evidence and its limits, each one cited to the paper it comes from. No skimming, no AI slop.

Questions to start from

Lune can draw on

the full paper · related work

Get the answer

Free to start. No credit card required.

Lineage

  • Gorilla: Large Language Model Connected with Massive APIs cites Toolformer: Language Models Can Teach Themselves to Use Tools
  • Reflexion: language agents with verbal reinforcement learning cites Toolformer: Language Models Can Teach Themselves to Use Tools
  • ToolLLM: Facilitating Large Language Models to Master 16000+ Real-world APIs cites Gorilla: Large Language Model Connected with Massive APIs
  • ToolLLM: Facilitating Large Language Models to Master 16000+ Real-world APIs cites Reflexion: language agents with verbal reinforcement learning
  • ToolLLM: Facilitating Large Language Models to Master 16000+ Real-world APIs cites Toolformer: Language Models Can Teach Themselves to Use Tools
Builds onCited byLeft to right by year; the more cited, the higher.

Builds on20

  1. Toolformer: Language Models Can Teach Themselves to Use ToolsTimo Schick, Jane Dwivedi-Yu, Roberto Dessì, Roberta Raileanu et al.NeurIPS 2023 · 5,989 citations
  2. Reflexion: language agents with verbal reinforcement learningNoah Shinn, Federico Cassano, Ashwin Gopinath, Karthik Narasimhan et al.NeurIPS 2023 · 5,828 citations
  3. SWE-bench: Can Language Models Resolve Real-world Github Issues?Carlos E. Jimenez, John Yang, Alexander Wettig, Shunyu Yao et al.ICLR 2024 · 2,082 citations
  4. Gorilla: Large Language Model Connected with Massive APIsShishir G. Patil, Tianjun Zhang, Xin Wang, Joseph E. GonzalezNeurIPS 2024 · 1,715 citations
  5. ToolLLM: Facilitating Large Language Models to Master 16000+ Real-world APIsYujia Qin, Shihao Liang, Yining Ye, Kunlun Zhu et al.ICLR 2024 · 1,469 citations

Related papers

  1. MCPTox: A Benchmark for Tool Poisoning on Real-World MCP ServersZhiqiang Wang, Yichao Gao, Yanting Wang, Suyuan Liu et al.AAAI 2026 · 6 citations
  2. Evaluating the Robustness of Multimodal Agents Against Active Environmental Injection AttacksYurun Chen, Xueyu Hu, Keting Yin, Juncheng Li et al.ACM MM 2025 · 2 citations
  3. Attractive Metadata Attack: Inducing LLM Agents to Invoke Malicious ToolsKanghua Mo, Li Hu, Yucheng Long, Zhihao LiNeurIPS 2025 · 37 citations
  4. Dissecting Adversarial Robustness of Multimodal LM AgentsChen Henry Wu, Rishi Rajesh Shah, Jing Yu Koh, Russ Salakhutdinov et al.ICLR 2025
  5. GhostEI-Bench: Do Mobile Agent Resilience to Environmental Injection in Dynamic On-Device Environments?Chiyu Chen, Xinhao Song, Yunkai Chai, Yang Yao et al.ICLR 2026 · 8 citations
Dusk over the sea between two cliffs drawn in fine vertical lines