Lune

ICLR2023Top-tier venue

MACTA: A Multi-agent Reinforcement Learning Approach for Cache Timing Attacks and Detection

Jiaxun Cui, Xiaomeng Yang, Mulong Luo, Geunbae Lee, Peter Stone, Hsien-Hsin S. Lee, Benjamin Lee, G. Edward Suh, Wenjie Xiong, Yuandong Tian

Abstract

Security vulnerabilities in computer systems raise serious concerns as computers process an unprecedented amount of private and sensitive data today. Cachetiming attacks (CTA) pose an important practical threat as they can effectively breach many protection mechanisms in today's systems. However, the current detection techniques for cache timing attacks heavily rely on heuristics and expert knowledge, which can lead to brittleness and the inability to adapt to new attacks. To mitigate the CTA threat, we propose using MACTA, a multi-agent reinforcement learning (MARL) approach that leverages population-based training to train both attackers and detectors. Following best practices, we develop a realistic simulated MARL environment, MA-AUTOCAT, which enables training and evaluation of cache-timing attackers and detectors. Our empirical results suggest that MACTA is an effective solution without any manual input from security experts. MACTA detectors can generalize to a heuristic attack not exposed in training with a 97.8% detection rate and reduce the attack bandwidth of RL-based attackers by 20% on average. In the meantime, MACTA attackers are qualitatively more effective than other attacks studied, and the average evasion rate of MACTA attackers against an unseen state-of-the-art detector can reach up to 99%. Furthermore, we found that agents equipped with a Transformer encoder can learn effective policies in situations when agents with multi-layer perceptron encoders do not in this environment, suggesting the potential of Transformer structures in CTA problems.

Ask Lune

Know this paper in minutes, not an afternoon.

Lune can read every page of it and the top-tier papers that cite it, and gives you its claims, its evidence and its limits, each one cited to the paper it comes from. No skimming, no AI slop.

3 top-tier papers have cited it since. Know what they found before your reviewers do.

Questions to start from

Lune can draw on

the full paper · the top-tier papers that cite it · related work

Get the answer

Free to start. No credit card required.

Lineage

  • Meltdown: Reading Kernel Memory from User Space cites Spectre Attacks: Exploiting Speculative Execution
  • Minimum Coverage Sets for Training Robust Ad Hoc Teamwork Agents cites Collaborating with Humans without Human Data
Builds onCited byLeft to right by year; the more cited, the higher.

Cited by top-tier papers3

  1. Minimum Coverage Sets for Training Robust Ad Hoc Teamwork AgentsMuhammad Rahman, Jiaxun Cui, Peter StoneAAAI 2024 · 20 citations
  2. Lifelong Safety Alignment for Language ModelsHaoyu Wang, Yifei Zhao, Zeyu Qin, Chao Du et al.NeurIPS 2025 · 18 citations
  3. AttackGNN: Red-Teaming GNNs in Hardware Security Using Reinforcement LearningVasudev Gohil, Satwik Patnaik, Dileep Kalathil, Jeyavijayan RajendranUSENIX Security 2024 · 9 citations

Builds on19

  1. Spectre Attacks: Exploiting Speculative ExecutionPaul Kocher, Jann Horn, Anders Fogh, Daniel Genkin et al.S&P 2019 · 2,435 citations
  2. Meltdown: Reading Kernel Memory from User SpaceMoritz Lipp, Michael Schwarz, Daniel Gruss, Thomas Prescher et al.USENIX Security 2018 · 1,456 citations
  3. Stabilizing Transformers for Reinforcement LearningEmilio Parisotto, H. Francis Song, Jack W. Rae, Razvan Pascanu et al.ICML 2020 · 464 citations
  4. Mastering Complex Control in MOBA Games with Deep Reinforcement LearningDeheng Ye, Zhao Liu, Mingfei Sun, Bei Shi et al.AAAI 2020 · 395 citations
  5. Collaborating with Humans without Human DataDJ Strouse, Kevin R. McKee, Matt M. Botvinick, Edward Hughes et al.NeurIPS 2021 · 239 citations

Related papers

  1. AutoCAT: Reinforcement Learning for Automated Exploration of Cache-Timing AttacksMulong Luo, Wenjie Xiong, Geunbae Lee, Yueying Li et al.HPCA 2023 · 3 citations
  2. A Hard-Label Black-Box Evasion Attack against ML-based Malicious Traffic Detection SystemsZixuan Liu, Yi Zhao, Zhuotao Liu, Qi Li et al.NDSS 2026 · 3 citations
  3. DETERRENT: detecting trojans using reinforcement learningVasudev Gohil, Satwik Patnaik, Hao Guo, Dileep Kalathil et al.DAC 2022 · 26 citations
  4. Budget-Efficient Attacks and Robustness Training for Cooperative MARLJunyong Jiang, Xin Yuan, Longhe Lin, Songze Li et al.ICML 2026
  5. PolicyCleanse: Backdoor Detection and Mitigation for Competitive Reinforcement LearningJunfeng Guo, Ang Li, Lixu Wang, Cong LiuICCV 2023 · 27 citations
Dusk over the sea between two cliffs drawn in fine vertical lines