Concept-ROT: Poisoning Concepts in Large Language Models with Model Editing
Keltin Grimes, Marco Christiani, David Shriver, Marissa Catherine Connor
Abstract
Model editing methods modify specific behaviors of Large Language Models by altering a small, targeted set of network weights and require very little data and compute. These methods can be used for malicious applications such as inserting misinformation or simple trojans that result in adversary-specified behaviors when a trigger word is present. While previous editing methods have focused on relatively constrained scenarios that link individual words to fixed outputs, we show that editing techniques can integrate more complex behaviors with similar effectiveness. We develop Concept-ROT, a model editing-based method that efficiently inserts trojans which not only exhibit complex output behaviors, but also trigger on high-level concepts -presenting an entirely new class of trojan attacks. Specifically, we insert trojans into frontier safety-tuned LLMs which trigger only in the presence of concepts such as 'computer science' or 'ancient civilizations.' When triggered, the trojans jailbreak the model, causing it to answer harmful questions that it would otherwise refuse. Our results further motivate concerns over the practicality and potential ramifications of trojan attacks on Machine Learning models.
Ask Lune
Know this paper in minutes, not an afternoon.
Lune can read every page of it and the top-tier papers that cite it, and gives you its claims, its evidence and its limits, each one cited to the paper it comes from. No skimming, no AI slop.
5 top-tier papers have cited it since. Know what they found before your reviewers do.
Lune can draw on
the full paper · the top-tier papers that cite it · related work
Free to start. No credit card required.
Lineage
- Concept Concentration for Faithful Representation Intervention cites Locating and Editing Factual Associations in GPT
- Can Knowledge Editing Really Correct Hallucinations? cites LoRA: Low-Rank Adaptation of Large Language Models
- Can Knowledge Editing Really Correct Hallucinations? cites Locating and Editing Factual Associations in GPT
- TruthfulQA: Measuring How Models Mimic Human Falsehoods cites Measuring Massive Multitask Language Understanding
- ShadeEdit: A Utility-Preserving and Defense-Evasive Knowledge Manipulation Attack in Federated LLMs cites LoRA: Low-Rank Adaptation of Large Language Models
Cited by top-tier papers5
- Trusting What You Cannot See: Auditable Fine-Tuning and Inference for Proprietary AIHeng Jin, Chaoyu Zhang, Hexuan Yu, Shanghao Shi et al.USENIX Security 2026 · 4 citations
- VLMs can Aggregate Scattered Training PatchesZhanhui Zhou, Lingjie Chen, Chao Yang, Chaochao LuNeurIPS 2025
- Concept Concentration for Faithful Representation InterventionHongzheng Yang, Yongqiang Chen, Zeyu Qin, Tongliang Liu et al.ICML 2026
- Can Knowledge Editing Really Correct Hallucinations?Baixiang Huang, Canyu Chen, Xiongxiao Xu, Ali Payani et al.ICLR 2025
- ShadeEdit: A Utility-Preserving and Defense-Evasive Knowledge Manipulation Attack in Federated LLMsXu Zhang, Hangcheng Liu, Shangwei Guo, Shudong Zhang et al.AAAI 2026
Builds on26
- LoRA: Low-Rank Adaptation of Large Language ModelsEdward J. Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu et al.ICLR 2022 · 18,833 citations
- Measuring Massive Multitask Language UnderstandingDan Hendrycks, Collin Burns, Steven Basart, Andy Zou et al.ICLR 2021 · 7,905 citations
- Locating and Editing Factual Associations in GPTKevin Meng, David Bau, Alex Andonian, Yonatan BelinkovNeurIPS 2022 · 3,415 citations
- TruthfulQA: Measuring How Models Mimic Human FalsehoodsStephanie Lin, Jacob Hilton, Owain EvansACL 2022 · 3,228 citations
- WinoGrande: An Adversarial Winograd Schema Challenge at ScaleKeisuke Sakaguchi, Ronan Le Bras, Chandra Bhagavatula, Yejin ChoiAAAI 2020 · 3,037 citations
Related papers
- Compiling Activation Steering into Weights via Null-Space Constraints for Stealthy BackdoorsRui Yin, Tianxu Han, Naen Xu, Changjiang Li et al.ACL 2026
- Tracing and Reversing Edits in LLMsPaul Youssef, Zhixue Zhao, Christin Seifert, Jörg SchlöttererICLR 2026 · 7 citations
- Defending Large Language Models Against Jailbreak Attacks via Layer-specific EditingWei Zhao, Zhe Li, Yige Li, Ye Zhang et al.EMNLP Findings 2024 · 23 citations
- The Philosopher's Stone: Trojaning Plugins of Large Language ModelsTian Dong, Minhui Xue, Guoxing Chen, Rayne Holland et al.NDSS 2025
- SAME: Safety-Aware Model Editing Guided by Safety TransformationJiayi Wang, Shipeng Wang, Ji Wu, Jian SunACL 2026
