EMNLP Findings2023
CCIM: Cross-modal Cross-lingual Interactive Image Translation
Cong Ma, Yaping Zhang, Mei Tu, Yang Zhao, Yu Zhou, Chengqing Zong
摘要
Text image machine translation (TIMT) which translates source language text images into target language texts has attracted intensive attention in recent years. Although the endto-end TIMT model directly generates target translation from encoded text image features with an efficient architecture, it lacks the recognized source language information resulting in a decrease in translation performance. In this paper, we propose a novel Cross-modal Cross-lingual Interactive Model (CCIM) to incorporate source language information by synchronously generating source language and target language results through an interactive attention mechanism between two language decoders. Extensive experimental results have shown the interactive decoder significantly outperforms end-to-end TIMT models and has faster decoding speed with smaller model size than cascade models. 1
问 Lune
几分钟读透这篇 Paper,不用再花一下午。
Lune 能读完全文,也能读引用它的顶会 Paper,讲清它的结论、证据和局限,每一点都注明出自哪篇 Paper。不用硬啃,也不靠猜。
脉络
它引用的 Paper5
- What Is Wrong With Scene Text Recognition Model Comparisons? Dataset and Model AnalysisJeonghun Baek, Geewook Kim, Junyeop Lee, Sungrae Park 等ICCV 2019 · 551 次引用
- Synchronous Speech Recognition and Speech-to-Text Translation with Interactive DecodingYuchen Liu, Jiajun Zhang, Hao Xiong, Long Zhou 等AAAI 2020 · 73 次引用
- Towards Fully Automated Manga TranslationRyota Hinami, Shonosuke Ishiwatari, Kazuhiko Yasuda, Yusuke MatsuiAAAI 2021 · 41 次引用
- Synchronous Interactive Decoding for Multilingual Neural Machine TranslationHao He, Qian Wang, Zhipeng Yu, Yang Zhao 等AAAI 2021 · 13 次引用
- Improving Multilingual Neural Machine Translation with Auxiliary Source LanguagesWeijia Xu, Yuwei Yin, Shuming Ma, Dongdong Zhang 等EMNLP Findings 2021 · 6 次引用
相关 Paper
- In-Image Neural Machine Translation with Segmented Pixel Sequence-to-Sequence ModelYanzhi Tian, Xiang Li, Zeming Liu, Yuhang Guo 等EMNLP Findings 2023 · 3 次引用
- Exploring Better Text Image Translation with Multimodal CodebookZhibin Lan, Jiawei Yu, Xiang Li, Wen Zhang 等ACL 2023 · 12 次引用
- PEIT: Bridging the Modality Gap with Pre-trained Models for End-to-End Image TranslationShaolin Zhu, Shangjie Li, Yikun Lei, Deyi XiongACL 2023 · 12 次引用
- VQA-Augmented Machine Translation with Cross-Modal Contrastive LearningZhihui Zhang, Shiliang Sun, Jing Zhao, Tengfei Song 等EMNLP Findings 2025
- InImageTrans: Multimodal LLM-based Text Image Machine TranslationFei Zuo, Kehai Chen, Yu Zhang, Zhengshan Xue 等ACL Findings 2025 · 1 次引用
