Evaluating and Aligning Human Economic Risk Preferences in LLMs
Jiaxin Liu, Yixuan Tang, Yi Yang, Kar Yan Tam
Abstract
Large Language Models (LLMs) are increasingly used in decision-making scenarios that involve risk assessment, yet their alignment with human economic rationality remains unclear. In this study, we investigate whether LLMs exhibit risk preferences consistent with human expectations across different personas. Specifically, we propose an evaluation metric called Risk Disparity Score (RDS) and assess whether LLM-generated responses reflect appropriate levels of risk aversion or riskseeking behavior based on individual's persona. Our results reveal that while LLMs make reasonable decisions in simplified, personalized risk contexts, their performance declines in more complex economic decision-making tasks. To address this, we test whether current state-of-art alignment methods such as Direct Preference Optimization(DPO) and In Context Learning(ICL) can enhance LLM adherence to persona-specific risk preferences. We find DPO can improve the economic rationality of LLMs in loss-related parameters, offering a step toward more human-aligned AI decision-making.
Ask about this paper
Your agent reads all of it.
Lune indexed this paper to the last equation, along with the top-tier papers that cite it. Ask a question and the answer quotes them.
Your agent calls
Luneget_paper_fulltext
Free to start. No credit card required.
Terminal
Install the CLIlune papers fulltext 62e2ba9d-14ee-4cfd-a289-ac04fd659270Builds on4
- Language Models are Few-Shot LearnersTom B. Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah et al.NeurIPS 2020 · 64,255 citations
- Direct Preference Optimization: Your Language Model is Secretly a Reward ModelRafael Rafailov, Archit Sharma, Eric Mitchell, Christopher D. Manning et al.NeurIPS 2023 · 10,924 citations
- Second Thoughts are Best: Learning to Re-Align With Human Values from Text EditsRuibo Liu, Chenyan Jia, Ge Zhang, Ziyu Zhuang et al.NeurIPS 2022 · 46 citations
- EconAgent: Large Language Model-Empowered Agents for Simulating Macroeconomic ActivitiesNian Li, Chen Gao, Mingyu Li, Yong Li et al.ACL 2024 · 30 citations
Related papers
- Cal-DPO: Calibrated Direct Preference Optimization for Language Model AlignmentTeng Xiao, Yige Yuan, Huaisheng Zhu, Mingxiao Li et al.NeurIPS 2024 · 76 citations
- Decision-Making Behavior Evaluation Framework for LLMs under Uncertain ContextJingru Jia, Zehua Yuan, Junhao Pan, Paul McNamara et al.NeurIPS 2024 · 71 citations
- Risk-aware Direct Preference Optimization under Nested Risk MeasureLijun Zhang, Lin Li, Yajie Qi, Huizhong Song et al.NeurIPS 2025 · 3 citations
- Geometric-Averaged Preference Optimization for Soft Preference LabelsHiroki Furuta, Kuang-Huei Lee, Shixiang Shane Gu, Yutaka Matsuo et al.NeurIPS 2024 · 24 citations
- Noise, Adaptation, and Strategy: Assessing LLM Fidelity in Decision-MakingYuanjun Feng, Vivek Choudhary, Yash Raj ShresthaEMNLP 2025 · 2 citations
