Lune

ACL Findings2026

CLaS-Bench: A Cross-Lingual Alignment and Steering Benchmark

Daniil Gurgurov, Yusser Al Ghussin, Tanja Baeumel, Cheng-Ting Chou, Patrick Schramowski, Marius Mosbach, Josef van Genabith, Simon Ostermann

Abstract

Understanding and controlling the behavior of large language models (LLMs) is an increasingly important topic in multilingual NLP. Beyond prompting or fine-tuning, language steering, i.e., manipulating internal representations during inference, has emerged as a more efficient and interpretable technique for adapting models to a target language. Yet, no dedicated benchmarks or evaluation protocols exist to quantify the effectiveness of steering techniques. We introduce CLaS-Bench, a lightweight parallel-question benchmark for evaluating language-forcing behavior in LLMs across 32 languages, enabling systematic evaluation of multilingual steering methods. We evaluate a broad array of steering techniques, including residual-stream DiffMean interventions, probederived directions, language-specific neurons, PCA/LDA vectors, Sparse Autoencoders, and prompting baselines. Steering performance is measured along two axes: language control and semantic relevance, combined into a single harmonic-mean steering score. We find that across languages, the simple residual-based DiffMean method consistently outperforms all other methods. Moreover, a layer-wise analysis reveals that language-specific structure emerges predominantly in later layers and steering directions cluster based on language family. CLaS-Bench is the first standardized benchmark for multilingual steering, enabling both rigorous scientific analysis of language representations and practical evaluation of steering as a low-cost adaptation alternative.

Ask Lune

Know this paper in minutes, not an afternoon.

Lune can read every page of it and the top-tier papers that cite it, and gives you its claims, its evidence and its limits, each one cited to the paper it comes from. No skimming, no AI slop.

1 top-tier paper has cited it since. Know what they found before your reviewers do.

Questions to start from

Lune can draw on

the full paper · the top-tier papers that cite it · related work

Get the answer

Free to start. No credit card required.

Lineage

  • In-context Vectors: Making In Context Learning More Effective and Controllable Through Latent Space Steering cites Inference-Time Intervention: Eliciting Truthful Answers from a Language Model
  • From Weights to Activations: Is Steering the Next Frontier of Adaptation? cites Sparse Autoencoders Find Highly Interpretable Features in Language Models
  • From Weights to Activations: Is Steering the Next Frontier of Adaptation? cites In-context Vectors: Making In Context Learning More Effective and Controllable Through Latent Space Steering
  • From Weights to Activations: Is Steering the Next Frontier of Adaptation? cites Inference-Time Intervention: Eliciting Truthful Answers from a Language Model
  • From Weights to Activations: Is Steering the Next Frontier of Adaptation? cites How do Large Language Models Handle Multilingualism?
Builds onCited byLeft to right by year; the more cited, the higher.

Cited by top-tier papers1

  1. From Weights to Activations: Is Steering the Next Frontier of Adaptation?Simon Ostermann, Daniil Gurgurov, Tanja Baeumel, Michael A. Hedderich et al.ACL 2026 · 3 citations

Builds on14

  1. Inference-Time Intervention: Eliciting Truthful Answers from a Language ModelKenneth Li, Oam Patel, Fernanda B. Viégas, Hanspeter Pfister et al.NeurIPS 2023 · 1,549 citations
  2. Sparse Autoencoders Find Highly Interpretable Features in Language ModelsRobert Huben, Hoagy Cunningham, Logan Riggs Smith, Aidan Ewart et al.ICLR 2024 · 1,072 citations
  3. In-context Vectors: Making In Context Learning More Effective and Controllable Through Latent Space SteeringSheng Liu, Haotian Ye, Lei Xing, James Y. ZouICML 2024 · 244 citations
  4. How do Large Language Models Handle Multilingualism?Yiran Zhao, Wenxuan Zhang, Guizhen Chen, Kenji Kawaguchi et al.NeurIPS 2024 · 196 citations
  5. The State and Fate of Linguistic Diversity and Inclusion in the NLP WorldPratik Joshi, Sebastin Santy, Amar Budhiraja, Kalika Bali et al.ACL 2020 · 40 citations

Related papers

  1. AxBench: Steering LLMs? Even Simple Baselines Outperform Sparse AutoencodersZhengxuan Wu, Aryaman Arora, Atticus Geiger, Zheng Wang et al.ICML 2025
  2. STEER-BENCH: A Benchmark for Evaluating the Steerability of Large Language ModelsKai Chen, Zihao He, Taiwei Shi, Kristina LermanEMNLP 2025 · 1 citation
  3. ReCoVeR the Target Language: Language Steering without Sacrificing Task PerformanceHannah Sterz, Fabian David Schmidt, Goran Glavas, Ivan VulicEMNLP Findings 2025
  4. Improved Representation Steering for Language ModelsZhengxuan Wu, Qinan Yu, Aryaman Arora, Christopher D. Manning et al.NeurIPS 2025 · 22 citations
  5. Concept Heterogeneity-aware Representation SteeringLaziz Abdullaev, Noelle Y. L. Wong, Ryan Lee, Shiqi Jiang et al.ICML 2026
Dusk over the sea between two cliffs drawn in fine vertical lines