Controllable and Explainable Personality Sliders for LLMs at Inference Time
Florian Hoppe, David Khachaturov, Robert Mullins, Mark Huasong Meng
摘要
Aligning Large Language Models (LLMs) with specific personas typically relies on Supervised Fine-Tuning (SFT) or Reinforcement Learning from Human Feedback (RLHF); however, these methods are resource-intensive, requiring expensive data collection and distinct model training for each target personality. In this work, we propose a parameter-efficient framework for continuous, multi-dimensional personality control via inference-time activation steering. Our approach addresses the challenge of combining multiple interventions by iteratively retraining probes on the residual stream modified by prior traits, ensuring compatibility. Once established, these steering vectors function as modular, reusable primitives; users can instantly synthesize novel, complex personality profiles by simply adjusting steering coefficients (α) without any additional training. To support this, we introduce an automated pipeline that identifies optimal intervention layers via activation separation analysis and calibrates coefficients via hyperparameter optimization to maximize alignment while constraining perplexity. Empirical evaluations validate individual trait shifts using an LLM-as-a-judge framework and demonstrate, via the Big Five inventory, that our method effectively modulates the model's holistic personality profile without updating base model parameters.
问问这篇 Paper
智能体会读完全文。
Lune 把这篇 Paper 索引到了最后一个公式,引用它的顶会 Paper 也一样。你提问,回答直接引用原文。
它引用的顶会 Paper5
- Direct Preference Optimization: Your Language Model is Secretly a Reward ModelRafael Rafailov, Archit Sharma, Eric Mitchell, Christopher D. Manning 等NeurIPS 2023 · 被引用 10,924 次
- Separation and Concentration in Deep NetworksJohn Zarka, Florentin Guth, Stéphane MallatICLR 2021 · 被引用 36 次
- Editing models with task arithmeticGabriel Ilharco, Marco Túlio Ribeiro, Mitchell Wortsman, Ludwig Schmidt 等ICLR 2023 · 被引用 31 次
- Intrinsic Dimensionality Explains the Effectiveness of Language Model Fine-TuningArmen Aghajanyan, Sonal Gupta, Luke ZettlemoyerACL 2021
- BIG5-CHAT: Shaping LLM Personalities Through Training on Human-Grounded DataWenkai Li, Jiarui Liu, Andy Liu, Xuhui Zhou 等ACL 2025
相关 Paper
- Personality Vector: Modulating Personality of Large Language Models by Model MergingSeungjong Sun, Seo Yeon Baek, Jang Hyun KimEMNLP 2025 · 被引用 5 次
- PERSONA: Dynamic and Compositional Inference-Time Personality Control via Activation Vector AlgebraXiachong Feng, Liang Zhao, Weihong Zhong, Yichong Huang 等ICLR 2026 · 被引用 11 次
- COLD-Steer: Steering Large Language Models via In-Context One-step Learning DynamicsKartik Sharma, Rakshit S. TrivediICLR 2026 · 被引用 8 次
- Personalized Steering of Large Language Models: Versatile Steering Vectors Through Bi-directional Preference OptimizationYuanpu Cao, Tianrong Zhang, Bochuan Cao, Ziyi Yin 等NeurIPS 2024 · 被引用 135 次
- Steering at the Source: Style Modulation Heads for Robust Persona ControlYoshihiro Izawa, Gouki Minegishi, Koshi Eguchi, Sosuke Hosokawa 等ICML 2026 · 被引用 2 次
