Lune

ACL Findings2024

An Empirical Study on the Characteristics of Bias upon Context Length Variation for Bangla

Jayanta Sadhu, Ayan Antik Khan, Abhik Bhattacharjee, Rifat Shahriyar

Abstract

Pretrained language models inherently exhibit various social biases, prompting a crucial examination of their social impact across various linguistic contexts due to their widespread usage.Previous studies have provided numerous methods for intrinsic bias measurements, predominantly focused on high-resource languages.In this work, we aim to extend these investigations to Bangla, a low-resource language.Specifically, in this study, we (1) create a dataset for intrinsic gender bias measurement in Bangla, (2) discuss necessary adaptations to apply existing bias measurement methods for Bangla, and (3) examine the impact of context length variation on bias measurement, a factor that has been overlooked in previous studies.Through our experiments, we demonstrate a clear dependency of bias metrics on context length, highlighting the need for nuanced considerations in Bangla bias analysis.We consider our work as a stepping stone for bias measurement in the Bangla Language and make all of our resources publicly available to support future research.1 1 https://github.com/csebuetnlp/BanglaContextualBiasKey Take-away: Effect sizes converge to a definite value after a moderate amount of context length, but the differences in value are not drastic.Additionally, more context length ensures more statistically significant results.1504 -0 .0 2 -0 .0 1 0 .00 0 .0 1 Prior Bias Distribution -0.

Ask Lune

Know this paper in minutes, not an afternoon.

Lune can read every page of it and the top-tier papers that cite it, and gives you its claims, its evidence and its limits, each one cited to the paper it comes from. No skimming, no AI slop.

Questions to start from

Lune can draw on

the full paper · related work

Get the answer

Free to start. No credit card required.

Lineage

Builds onCited byLeft to right by year; the more cited, the higher.

Builds on4

  1. ELECTRA: Pre-training Text Encoders as Discriminators Rather Than GeneratorsKevin Clark, Minh-Thang Luong, Quoc V. Le, Christopher D. ManningICLR 2020 · 541 citations
  2. XL-Sum: Large-Scale Multilingual Abstractive Summarization for 44 LanguagesTahmid Hasan, Abhik Bhattacharjee, Md. Saiful Islam, Kazi Samin Mubasshir et al.ACL Findings 2021 · 228 citations
  3. The State and Fate of Linguistic Diversity and Inclusion in the NLP WorldPratik Joshi, Sebastin Santy, Amar Budhiraja, Kalika Bali et al.ACL 2020 · 40 citations
  4. Not Low-Resource Anymore: Aligner Ensembling, Batch Filtering, and New Datasets for Bengali-English Machine TranslationTahmid Hasan, Abhik Bhattacharjee, Kazi Samin, Masum Hasan et al.EMNLP 2020 · 7 citations

Related papers

  1. BanStereoSet: A Dataset to Measure Stereotypical Social Biases in LLMs for BanglaMahammed Kamruzzaman, Abdullah Al Monsur, Shrabon Kumar Das, Enamul Hassan et al.ACL Findings 2025 · 1 citation
  2. On Evaluating and Mitigating Gender Biases in Multilingual SettingsAniket Vashishtha, Kabir Ahuja, Sunayana SitaramACL Findings 2023 · 7 citations
  3. Intrinsic Bias Metrics Do Not Correlate with Application BiasSeraphina Goldfarb-Tarrant, Rebecca Marchant, Ricardo Muñoz Sánchez, Mugdha Pandya et al.ACL 2021
  4. What Do Compressed Multilingual Machine Translation Models Forget?Alireza Mohammadshahi, Vassilina Nikoulina, Alexandre Berard, Caroline Brun et al.EMNLP Findings 2022 · 7 citations
  5. BanglaByT5: Byte-Level Modelling for BanglaPramit Bhattacharyya, Arnab BhattacharyaEMNLP Findings 2025 · 3 citations
Dusk over the sea between two cliffs drawn in fine vertical lines