Don't Walk the Line: Boundary Guidance for Filtered Generation

Generative models are increasingly paired with safety classifiers that filter harmful or undesirable outputs. A common strategy is to finetune the generator to reduce the probability of being filtered, but this can be suboptimal: it often pushes the model toward producing samples near the classifier's decision boundary, increasing both false positives and false negatives. We propose Boundary Guidance, a reinforcement learning fine-tuning method that explicitly steers generation away from the classifier's margin. On a benchmark of jailbreak, ambiguous, and long-context prompts, Boundary Guidance improves the safety while maintaining or improving the utility of outputs, as judged by LLM-asa-Judge evaluations. Comprehensive ablations across model scales and reward designs demonstrate the robustness of our approach. The code is available at .