ACL Findings2026
Same Claim, Different Judgment: Benchmarking Scenario-Induced Bias in Multilingual Financial Misinformation Detection
Zhiwei Liu, Yupeng Cao, Yuechen Jiang, Mohsinul Kabir, Polydoros Giannouris, Chen Xu, Ziyang Xu, Tianlei Zhu, Md. Tariquzzaman, Triantafillos Papadopoulos, Yan Wang, Lingfei Qian
Abstract
Large language models (LLMs) have been widely applied across various domains of finance. Since their training data are largely derived from human-authored corpora, LLMs may inherit a range of human biases. Behavioral biases can lead to instability and uncertainty in decision-making, particularly when processing financial information. However, existing research on LLM bias has mainly focused on direct questioning or simplified, generalpurpose settings, with limited consideration of the complex real-world financial environments and high-risk, context-sensitive, multilingual financial misinformation detection tasks (MFMD). In this work, we propose MFMD-Scen, a comprehensive benchmark for evaluating behavioral biases of LLMs in MFMD across diverse economic scenarios. In collaboration with financial experts, we construct three types of complex financial scenarios: (i) roleand personality-based, (ii) role-and regionbased, and (iii) role-based scenarios incorporating ethnicity and religious beliefs. We further develop a multilingual financial misinformation dataset covering English, Chinese, Greek, and Bengali. By integrating these scenarios with misinformation claims, MFMD-Scen enables a systematic evaluation of 22 mainstream LLMs. Our findings reveal that pronounced behavioral biases persist across both commercial and open-source models. This project is available at https://github.com/lzw108/FMD.
Ask Lune
Ask anything. The answer quotes the paper.
Lune reads all of it and the papers around it, then shows you the passages behind every answer.
Builds on1
Related papers
- All That Glisters Is Not Gold: A Benchmark for Reference-Free Counterfactual Financial Misinformation DetectionYuechen Jiang, Zhiwei Liu, Yupeng Cao, Yueru He et al.ACL 2026 · 9 citations
- Are Large Language Models Reliable Reviewers? A Benchmark for Error Detection in Financial DocumentsYing He, Zhouhong Gu, Zhecheng Hu, Yubo Zhou et al.ACL Findings 2026
- Fin-Bias: Comprehensive Evaluation for LLM Decision-Making under human bias in Finance DomainXiaoyu Hu, Jinman ZhaoACL Findings 2026 · 2 citations
- How does Misinformation Affect Large Language Model Behaviors and Preferences?Miao Peng, Nuo Chen, Jianheng Tang, Jia LiACL 2025 · 2 citations
- FinSafetyBench: Evaluating LLM Safety in Real-World Financial ScenariosYutao Hou, Yihan Jiang, Yuhan Xie, Jian Yang et al.ACL Findings 2026 · 2 citations
