Let's Play Across Cultures: A Large Multilingual, Multicultural Benchmark for Assessing Language Models' Understanding of Sports
Punit Kumar Singh, Nishant Kumar, Akash Ghosh, Kunal Pasad, Khushi Soni, Manisha Jaishwal, Sriparna Saha, Syukron Abu Ishaq Alfarozi, Asres Temam Abagissa, Kitsuchart Pasupa, Haiqin Yang, José G. Moreno
Abstract
Language Models (LMs) are primarily evaluated on globally popular sports, often overlooking regional and indigenous sporting traditions. To address this gap, we introduce Cult-SportQA, a benchmark designed to assess LMs' understanding of traditional sports across 60 countries and 6 continents, encompassing four distinct cultural categories. The dataset features 33,000 multiple-choice questions (MCQs) across text and image modalities, each of which is categorized into three key types: historybased, rule-based, and scenario-based. To evaluate model performance, we employ zeroshot, few-shot, and chain-of-thought (CoT) prompting across a diverse set of Large Language Models (LLMs), Small Language Models (SLMs), and Multimodal Large Language Models (MLMs). By providing a comprehensive multilingual and multicultural sports benchmark, CultSportQA establishes a new standard for assessing AI's ability to understand and reason about traditional sports.
Ask Lune
Know this paper in minutes, not an afternoon.
Lune can read every page of it and the top-tier papers that cite it, and gives you its claims, its evidence and its limits, each one cited to the paper it comes from. No skimming, no AI slop.
1 top-tier paper has cited it since. Know what they found before your reviewers do.
Lune can draw on
the full paper · the top-tier papers that cite it · related work
Free to start. No credit card required.
Lineage
- CLINIC : Evaluating Multilingual Trustworthiness in Language Models for Healthcare cites Crosslingual Generalization through Multitask Finetuning
- Visual Instruction Tuning cites Language Models are Few-Shot Learners
- Visual Instruction Tuning cites Training language models to follow instructions with human feedback
- Training language models to follow instructions with human feedback cites Language Models are Few-Shot Learners
Cited by top-tier papers1
Builds on12
- Language Models are Few-Shot LearnersTom B. Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah et al.NeurIPS 2020 · 64,255 citations
- Training language models to follow instructions with human feedbackLong Ouyang, Jeffrey Wu, Xu Jiang, Diogo Almeida et al.NeurIPS 2022 · 24,707 citations
- Visual Instruction TuningHaotian Liu, Chunyuan Li, Qingyang Wu, Yong Jae LeeNeurIPS 2023 · 11,349 citations
- Crosslingual Generalization through Multitask FinetuningNiklas Muennighoff, Thomas Wang, Lintang Sutawika, Adam Roberts et al.ACL 2023 · 319 citations
- Visually Grounded Reasoning across Languages and CulturesFangyu Liu, Emanuele Bugliarello, Edoardo Maria Ponti, Siva Reddy et al.EMNLP 2021 · 87 citations
Related papers
- SPORTU: A Comprehensive Sports Understanding Benchmark for Multimodal Large Language ModelsHaotian Xia, Zhengbang Yang, Junbo Zou, Rhys Tracy et al.ICLR 2025
- Benchmarking Vision Language Models for Cultural UnderstandingShravan Nayak, Kanishk Jain, Rabiul Awal, Siva Reddy et al.EMNLP 2024 · 26 citations
- All Languages Matter: Evaluating LMMs on Culturally Diverse 100 LanguagesAshmal Vayani, Dinura Dissanayake, Hasindri Watawana, Noor Ahsan et al.CVPR 2025
- Seeing Culture: A Benchmark for Visual Reasoning and GroundingBurak Satar, Zhixin Ma, Patrick Amadeus Irawan, Wilfried A. Mulyawan et al.EMNLP 2025
- NativQA: Multilingual Culturally-Aligned Natural Query for LLMsMd. Arid Hasan, Maram Hasanain, Fatema Ahmad, Sahinur Rahman Laskar et al.ACL Findings 2025 · 3 citations
