Lune

ACL Findings2025

A Query-Response Framework for Whole-Page Complex-Layout Document Image Translation with Relevant Regional Concentration

Zhiyang Zhang, Yaping Zhang, Yupu Liang, Zhiyuan Chen, Lu Xiang, Yang Zhao, Yu Zhou, Chengqing Zong

Abstract

Document Image Translation (DIT), which aims at translating documents in images from source language to the target, plays an important role in Document Intelligence. It requires a comprehensive understanding of document multi-modalities and a focused concentration on relevant textual regions during translation. However, most existing methods usually rely on the vanilla encoder-decoder paradigm, severely losing concentration on key regions that are especially crucial for complexlayout document translation. To tackle this issue, in this paper, we propose a new Query-Response DIT framework (QRDIT). QRDIT reformulates the DIT task into a parallel response/translation process of the multiple queries (i.e., relevant source texts), explicitly centralizing its focus toward the most relevant textual regions to ensure translation accuracy. A novel dynamic aggregation mechanism is also designed to enhance the text semantics in query features toward translation. Extensive experiments in four translation directions on three benchmarks demonstrate its state-of-theart performance, showing significant translation quality improvements toward whole-page complex-layout document images.

Ask Lune

Know this paper in minutes, not an afternoon.

Lune can read every page of it and the top-tier papers that cite it, and gives you its claims, its evidence and its limits, each one cited to the paper it comes from. No skimming, no AI slop.

2 top-tier papers have cited it since. Know what they found before your reviewers do.

Questions to start from

Lune can draw on

the full paper · the top-tier papers that cite it · related work

Get the answer

Free to start. No credit card required.

Lineage

  • LiLT: A Simple yet Effective Language-Independent Layout Transformer for Structured Document Understanding cites LayoutLM: Pre-training of Text and Layout for Document Image Understanding
  • LayoutLMv3: Pre-training for Document AI with Unified Text and Image Masking cites LiLT: A Simple yet Effective Language-Independent Layout Transformer for Structured Document Understanding
  • LayoutLMv3: Pre-training for Document AI with Unified Text and Image Masking cites LayoutLM: Pre-training of Text and Layout for Document Image Understanding
  • PRIM: Towards Practical In-Image Multilingual Machine Translation cites An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale
  • LayoutReader: Pre-training of Text and Layout for Reading Order Detection cites LayoutLM: Pre-training of Text and Layout for Document Image Understanding
Builds onCited byLeft to right by year; the more cited, the higher.

Cited by top-tier papers2

  1. PRIM: Towards Practical In-Image Multilingual Machine TranslationYanzhi Tian, Zeming Liu, Zhengyang Liu, Chong Feng et al.EMNLP 2025 · 4 citations
  2. DART: Disambiguation-Aware Reasoning for Video-guided Machine TranslationBoyu Guan, Chuang Han, Yang Zhao, Chengqing ZongACL 2026

Builds on12

  1. An Image is Worth 16x16 Words: Transformers for Image Recognition at ScaleAlexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn et al.ICLR 2021 · 21,477 citations
  2. LayoutLMv3: Pre-training for Document AI with Unified Text and Image MaskingYupan Huang, Tengchao Lv, Lei Cui, Yutong Lu et al.ACM MM 2022 · 606 citations
  3. LayoutLM: Pre-training of Text and Layout for Document Image UnderstandingYiheng Xu, Minghao Li, Lei Cui, Shaohan Huang et al.KDD 2020 · 575 citations
  4. LiLT: A Simple yet Effective Language-Independent Layout Transformer for Structured Document UnderstandingJiapeng Wang, Lianwen Jin, Kai DingACL 2022 · 188 citations
  5. LayoutReader: Pre-training of Text and Layout for Reading Order DetectionZilong Wang, Yiheng Xu, Lei Cui, Jingbo Shang et al.EMNLP 2021 · 51 citations

Related papers

  1. LayoutDIT: Layout-Aware End-to-End Document Image Translation with Multi-Step Conductive DecoderZhiyang Zhang, Yaping Zhang, Yupu Liang, Lu Xiang et al.EMNLP Findings 2023 · 9 citations
  2. Single-to-mix Modality Alignment with Multimodal Large Language Model for Document Image Machine TranslationYupu Liang, Yaping Zhang, Zhiyang Zhang, Yang Zhao et al.ACL 2025 · 6 citations
  3. DocLLM: A Layout-Aware Generative Language Model for Multimodal Document UnderstandingDongsheng Wang, Natraj Raman, Mathieu Sibue, Zhiqiang Ma et al.ACL 2024 · 37 citations
  4. DiT: Self-supervised Pre-training for Document Image TransformerJunlong Li, Yiheng Xu, Tengchao Lv, Lei Cui et al.ACM MM 2022 · 184 citations
  5. Attention Where It Matters: Rethinking Visual Document Understanding with Selective Region ConcentrationHaoyu Cao, Changcun Bao, Chaohu Liu, Huang Chen et al.ICCV 2023 · 21 citations
Dusk over the sea between two cliffs drawn in fine vertical lines