Lune

ACL Findings2026

Same Claim, Different Judgment: Benchmarking Scenario-Induced Bias in Multilingual Financial Misinformation Detection

Zhiwei Liu, Yupeng Cao, Yuechen Jiang, Mohsinul Kabir, Polydoros Giannouris, Chen Xu, Ziyang Xu, Tianlei Zhu, Md. Tariquzzaman, Triantafillos Papadopoulos, Yan Wang, Lingfei Qian

Abstract

Large language models (LLMs) have been widely applied across various domains of finance. Since their training data are largely derived from human-authored corpora, LLMs may inherit a range of human biases. Behavioral biases can lead to instability and uncertainty in decision-making, particularly when processing financial information. However, existing research on LLM bias has mainly focused on direct questioning or simplified, generalpurpose settings, with limited consideration of the complex real-world financial environments and high-risk, context-sensitive, multilingual financial misinformation detection tasks (MFMD). In this work, we propose MFMD-Scen, a comprehensive benchmark for evaluating behavioral biases of LLMs in MFMD across diverse economic scenarios. In collaboration with financial experts, we construct three types of complex financial scenarios: (i) roleand personality-based, (ii) role-and regionbased, and (iii) role-based scenarios incorporating ethnicity and religious beliefs. We further develop a multilingual financial misinformation dataset covering English, Chinese, Greek, and Bengali. By integrating these scenarios with misinformation claims, MFMD-Scen enables a systematic evaluation of 22 mainstream LLMs. Our findings reveal that pronounced behavioral biases persist across both commercial and open-source models. This project is available at https://github.com/lzw108/FMD.

Ask Lune

Ask anything. The answer quotes the paper.

Lune reads all of it and the papers around it, then shows you the passages behind every answer.

Questions to start from

Lune answers from

the full paper · related work

Get the answer

Free to start. No credit card required.

Builds on1

  1. Systematic Biases in LLM Simulations of DebatesAmir Taubenfeld, Yaniv Dover, Roi Reichart, Ariel GoldsteinEMNLP 2024 · 35 citations

Related papers

  1. All That Glisters Is Not Gold: A Benchmark for Reference-Free Counterfactual Financial Misinformation DetectionYuechen Jiang, Zhiwei Liu, Yupeng Cao, Yueru He et al.ACL 2026 · 9 citations
  2. Are Large Language Models Reliable Reviewers? A Benchmark for Error Detection in Financial DocumentsYing He, Zhouhong Gu, Zhecheng Hu, Yubo Zhou et al.ACL Findings 2026
  3. Fin-Bias: Comprehensive Evaluation for LLM Decision-Making under human bias in Finance DomainXiaoyu Hu, Jinman ZhaoACL Findings 2026 · 2 citations
  4. How does Misinformation Affect Large Language Model Behaviors and Preferences?Miao Peng, Nuo Chen, Jianheng Tang, Jia LiACL 2025 · 2 citations
  5. FinSafetyBench: Evaluating LLM Safety in Real-World Financial ScenariosYutao Hou, Yihan Jiang, Yuhan Xie, Jian Yang et al.ACL Findings 2026 · 2 citations
Dusk over the sea between two cliffs drawn in fine vertical lines