SAHM: A Benchmark for Arabic Financial and Shari'ah-Compliant Reasoning
Rania Elbadry, Sarfraz Ahmad, Ahmed Heakl, Dani Bouch, Momina Ahsan, Muhra AlMahri, Marwa Elsaid Khalil, Yuxia Wang, Salem Lahlou, Sophia Ananiadou, Veselin Stoyanov, Jimin Huang
摘要
English financial NLP has advanced rapidly through benchmarks targeting earnings analysis, market sentiment, tabular reasoning, and financial question answering, yet Arabic financial NLP remains virtually nonexistent, despite 422 million speakers, 4-5 trillion Islamic finance industry requiring specialized Shari'ah compliance over instruments like sukuk, murabaha, and takaful. We introduce Sahm, the first Arabic financial benchmark spanning seven tasks: AAOIFI standards QA, fatwa-based QA/MCQ, accounting and business exams, financial sentiment analysis, extractive summarization, and event-cause reasoning, comprising 14,380 expert-verified instances from authentic regulatory, juristic, and corporate sources. Evaluating 20 LLMs, we find Arabic fluency does not imply financial reasoning: models achieving 91% on recognition tasks drop sharply on generation, and event-cause reasoning exposes the widest performance gap (1.89-9.84/10). We release the benchmark and dataset to support trustworthy Arabic financial assistants.
问问这篇 Paper
智能体会读完全文。
Lune 把这篇 Paper 索引到了最后一个公式,引用它的顶会 Paper 也一样。你提问,回答直接引用原文。
它引用的顶会 Paper6
- ConvFinQA: Exploring the Chain of Numerical Reasoning in Conversational Finance Question AnsweringZhiyu Chen, Shiyang Li, Charese Smiley, Zhiqiang Ma 等EMNLP 2022 · 被引用 57 次
- FinChain: A Symbolic Benchmark for Verifiable Chain-of-Thought Financial ReasoningZhuohan Xie, Daniil Orel, Rushil Thareja, Dhruv Sahnan 等ACL 2026 · 被引用 13 次
- KnowledgeFMath: A Knowledge-Intensive Math Reasoning Dataset in Finance DomainsYilun Zhao, Hongjun Liu, Yitao Long, Rui Zhang 等ACL 2024 · 被引用 10 次
- FinQA: A Dataset of Numerical Reasoning over Financial DataZhiyu Chen, Wenhu Chen, Charese Smiley, Sameena Shah 等EMNLP 2021 · 被引用 8 次
- ALLaM: Large Language Models for Arabic and EnglishM. Saiful Bari, Yazeed Alnumay, Norah A. Alzahrani, Nouf M. Alotaibi 等ICLR 2025 · 被引用 4 次
相关 Paper
- MultiFinBen: Benchmarking Large Language Models for Multilingual and Multimodal Financial ApplicationXueqing Peng, Lingfei Qian, Yan Wang, Ruoyu Xiang 等ACL 2026 · 被引用 6 次
- Plutus: Benchmarking Large Language Models in Low-Resource Greek FinanceXueqing Peng, Triantafillos Papadopoulos, Efstathia Soufleri, Polydoros Giannouris 等EMNLP 2025 · 被引用 2 次
- BizBench: A Quantitative Reasoning Benchmark for Business and FinanceMichael Krumdick, Rik Koncel-Kedziorski, Viet Dac Lai, Varshini Reddy 等ACL 2024 · 被引用 10 次
- EDINET-Bench: Evaluating LLMs on Complex Financial Tasks using Japanese Financial StatementsIssa Sugiura, Takashi Ishida, Taro Makino, Chieko Tazuke 等ICLR 2026 · 被引用 9 次
- FinMMR: Make Financial Numerical Reasoning More Multimodal, Comprehensive, and ChallengingZichen Tang, Haihong E, Jiacheng Liu, Zhongjun Yang 等ICCV 2025 · 被引用 1 次
