Lune

Abstract

Large language models (LLMs) are deployed in a wide variety of user-facing applications. Typically, these deployments have some specific purpose, like answering questions grounded on documentation or acting as coding assistants, but they require general language understanding. In such deployments, LLMs should respond only to queries that align with the intended purpose and reject all other requests, such as generating poetry or answering questions about physics, a task we refer to as 'scoping'. We conduct a comprehensive empirical evaluation of various methods, ranging from prompting, fine-tuning to preference learning and the recently proposed general alignment technique known as Circuit Breakers (CB). Across three families of language models and a broad variety of tasks, we show that it is possible to scope language models. We examine scoping for multiple topics, and fine-grained topics. We ablate diversity of irrelevant queries, layer different techniques, conduct adversarial evaluations and more. Among other results, we find that when diverse examples of irrelevant queries are available, simple supervised fine-tuning produces the best results, but when such diversity is low, Circuit Breakers perform quite well. One can often get the benefits of both methods by layering them in succession. We intend our study to serve as a practitioner's guide to scoping LLMs.

Ask Lune

Know this paper in minutes, not an afternoon.

Lune can read every page of it and the top-tier papers that cite it, and gives you its claims, its evidence and its limits, each one cited to the paper it comes from. No skimming, no AI slop.

Questions to start from

Lune can draw on

the full paper · related work

Get the answer

Free to start. No credit card required.

Lineage

  • Direct Preference Optimization: Your Language Model is Secretly a Reward Model cites Training language models to follow instructions with human feedback
  • Finetuned Language Models are Zero-Shot Learners cites Training language models to follow instructions with human feedback
  • Training language models to follow instructions with human feedback cites Finetuned Language Models are Zero-Shot Learners
  • Jailbroken: How Does LLM Safety Training Fail? cites Training language models to follow instructions with human feedback
Builds onCited byLeft to right by year; the more cited, the higher.

Builds on18

  1. Training language models to follow instructions with human feedbackLong Ouyang, Jeffrey Wu, Xu Jiang, Diogo Almeida et al.NeurIPS 2022 · 24,707 citations
  2. LoRA: Low-Rank Adaptation of Large Language ModelsEdward J. Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu et al.ICLR 2022 · 18,833 citations
  3. Direct Preference Optimization: Your Language Model is Secretly a Reward ModelRafael Rafailov, Archit Sharma, Eric Mitchell, Christopher D. Manning et al.NeurIPS 2023 · 10,924 citations
  4. Finetuned Language Models are Zero-Shot LearnersJason Wei, Maarten Bosma, Vincent Y. Zhao, Kelvin Guu et al.ICLR 2022 · 4,966 citations
  5. Jailbroken: How Does LLM Safety Training Fail?Alexander Wei, Nika Haghtalab, Jacob SteinhardtNeurIPS 2023 · 2,230 citations

Related papers

  1. Steering Out-of-Distribution Generalization with Concept Ablation Fine-TuningHelena Casademunt, Caden Juang, Adam Karvonen, Samuel Marks et al.ICML 2026 · 32 citations
  2. Knowledgeable Preference Alignment for LLMs in Domain-specific Question AnsweringYichi Zhang, Zhuo Chen, Yin Fang, Yanxi Lu et al.ACL Findings 2024 · 24 citations
  3. REINFORCE Adversarial Attacks on Large Language Models: An Adaptive, Distributional, and Semantic ObjectiveSimon Geisler, Tom Wollschläger, M. H. I. Abdalla, Vincent Cohen-Addad et al.ICML 2025
  4. SHARP: Self-adaptive Harmful Category-aware Prompt Generation for Black-box JailbreakingYingjie Xue, Xingyou Xia, Jun Zhang, Yunbo Cao et al.ACL 2026
  5. DeAL: Decoding-time Alignment for Large Language ModelsJames Y. Huang, Sailik Sengupta, Daniele Bonadiman, Yi-An Lai et al.ACL 2025
Dusk over the sea between two cliffs drawn in fine vertical lines