ACL Findings2024
Translation Deserves Better: Analyzing Translation Artifacts in Cross-lingual Visual Question Answering
ChaeHun Park, Koanho Lee, Hyesu Lim, Jaeseok Kim, Junmo Park, Yu-Jung Heo, Du-Seong Chang, Jaegul Choo
Abstract
Building a reliable visual question answering (VQA) system across different languages is a challenging problem, primarily due to the lack of abundant samples for training. To address this challenge, recent studies have employed machine translation systems for the cross-lingual VQA task. This involves translating the evaluation samples into a source language (usually English) and using monolingual models (i.e., translate-test). However, our analysis reveals that translated texts contain unique characteristics distinct from humanwritten ones, referred to as translation artifacts. We find that these artifacts can significantly affect the models, confirmed by extensive experiments across diverse models, languages, and translation processes. In light of this, we present a simple data augmentation strategy that can alleviate the adverse impacts of translation artifacts. Our code is available at https: //github.com/ddehun/VQA_translation.
Ask Lune
Know this paper in minutes, not an afternoon.
Lune can read every page of it and the top-tier papers that cite it, and gives you its claims, its evidence and its limits, each one cited to the paper it comes from. No skimming, no AI slop.
1 top-tier paper has cited it since. Know what they found before your reviewers do.
Lune can draw on
the full paper · the top-tier papers that cite it · related work
Free to start. No credit card required.
Lineage
- FLAVA: A Foundational Language And Vision Alignment Model cites VL-BERT: Pre-training of Generic Visual-Linguistic Representations
- FLAVA: A Foundational Language And Vision Alignment Model cites Learning Transferable Visual Models From Natural Language Supervision
- Generic Attention-model Explainability for Interpreting Bi-Modal and Encoder-Decoder Transformers cites Learning Transferable Visual Models From Natural Language Supervision
Cited by top-tier papers1
Builds on18
- Learning Transferable Visual Models From Natural Language SupervisionAlec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh et al.ICML 2021 · 47,906 citations
- VL-BERT: Pre-training of Generic Visual-Linguistic RepresentationsWeijie Su, Xizhou Zhu, Yue Cao, Bin Li et al.ICLR 2020 · 1,825 citations
- FLAVA: A Foundational Language And Vision Alignment ModelAmanpreet Singh, Ronghang Hu, Vedanuj Goswami, Guillaume Couairon et al.CVPR 2022 · 483 citations
- Generic Attention-model Explainability for Interpreting Bi-Modal and Encoder-Decoder TransformersHila Chefer, Shir Gur, Lior WolfICCV 2021 · 451 citations
- PaLI: A Jointly-Scaled Multilingual Language-Image ModelXi Chen, Xiao Wang, Soravit Changpinyo, A. J. Piergiovanni et al.ICLR 2023 · 194 citations
Related papers
- VQA-Augmented Machine Translation with Cross-Modal Contrastive LearningZhihui Zhang, Shiliang Sun, Jing Zhao, Tengfei Song et al.EMNLP Findings 2025
- Multilingual Transfer Learning for QA using Translation as Data AugmentationMihaela A. Bornea, Lin Pan, Sara Rosenthal, Radu Florian et al.AAAI 2021 · 45 citations
- Translation Artifacts in Cross-lingual Transfer LearningMikel Artetxe, Gorka Labaka, Eneko AgirreEMNLP 2020 · 68 citations
- Revisiting Machine Translation for Cross-lingual ClassificationMikel Artetxe, Vedanuj Goswami, Shruti Bhosale, Angela Fan et al.EMNLP 2023 · 10 citations
- MaXM: Towards Multilingual Visual Question AnsweringSoravit Changpinyo, Linting Xue, Michal Yarom, Ashish V. Thapliyal et al.EMNLP Findings 2023 · 20 citations
