Invariance Makes LLM Unlearning Resilient Even to Unanticipated Downstream Fine-Tuning

We adapt IRM unlearning by replacing the ERM loss with an unlearning objective ℓ 𝑢 , while keeping the invariance regularization to resist downstream fine-tuning Here, 𝒟 𝑖 encodes the fine-tuning environment (e.g., GSM8K or AGNews), unrelated to unlearning. • The invariance regularization encourages 𝜽 to be robust to fine-tuning across all 𝒟 𝑖 .