Lune

ACL Findings2026

ManCC: A Task-Anchored Benchmark for Manchu-Classical Chinese Cross-Lingual Modeling

Meiqi Wang, Xiaoxin Sun, Dongjie Wang, Ruixin Yu, Xiantao Heng, Shuo Wang, Zhen Huang, Peng Zhao, Suhua Wang, Minghao Yin

Abstract

Research in cross-lingual modeling for historical and extremely low-resource languages is hindered by the absence of standardized evaluation benchmarks. To address this, we present ManCC-the first task-anchored benchmark for Manchu-Classical Chinese translation. ManCC consists of a high-quality parallel corpus of 16,627 sentence pairs, derived from the Qing-dynasty historical text Manwen Laodang-Taizong, and a reproducible evaluation protocol that combines automatic metrics (BLEU and chrF) with a three-dimensional human assessment (fidelity, fluency, linguistic normativity). Through systematic evaluation across three model families (non-pretrained, multilingual pretrained, and large language models), we find that linguistic differences significantly influence performance, broader language coverage in multilingual pretraining facilitates lowresource transfer, and automatic metrics often fail to capture essential errors in historical translation-underscoring the necessity of human evaluation. ManCC not only provides foundational resources for Manchu-Classical Chinese translation but also establishes a diagnosable, reproducible platform for cross-lingual modeling of historical low-resource languages 1 .

Ask Lune

Know this paper in minutes, not an afternoon.

Lune can read every page of it and the top-tier papers that cite it, and gives you its claims, its evidence and its limits, each one cited to the paper it comes from. No skimming, no AI slop.

Questions to start from

Lune can draw on

the full paper · related work

Get the answer

Free to start. No credit card required.

Lineage

  • TopXGen: Topic-Diverse Parallel Data Generation for Low-Resource Machine Translation cites The State and Fate of Linguistic Diversity and Inclusion in the NLP World
  • Understanding In-Context Machine Translation for Low-Resource Languages: A Case Study on Manchu cites Hire a Linguist!: Learning Endangered Languages in LLMs with In-Context Linguistic Descriptions
Builds onCited byLeft to right by year; the more cited, the higher.

Builds on5

  1. The State and Fate of Linguistic Diversity and Inclusion in the NLP WorldPratik Joshi, Sebastin Santy, Amar Budhiraja, Kalika Bali et al.ACL 2020 · 40 citations
  2. Hire a Linguist!: Learning Endangered Languages in LLMs with In-Context Linguistic DescriptionsKexun Zhang, Yee Man Choi, Zhenqiao Song, Taiqi He et al.ACL Findings 2024 · 12 citations
  3. Understanding In-Context Machine Translation for Low-Resource Languages: A Case Study on ManchuRenhao Pei, Yihong Liu, Peiqin Lin, François Yvon et al.ACL 2025 · 11 citations
  4. TopXGen: Topic-Diverse Parallel Data Generation for Low-Resource Machine TranslationArmel Randy Zebaze, Benoît Sagot, Rachel BawdenEMNLP Findings 2025
  5. Benchmarking LLMs for Translating Classical Chinese Poetry: Evaluating Adequacy, Fluency, and EleganceAndong Chen, Lianzhang Lou, Kehai Chen, Xuefeng Bai et al.EMNLP 2025

Related papers

  1. MuBench: Assessment of Multilingual Capabilities of Large Language Models Across 61 LanguagesWenhan Han, Yifan Zhang, Zhixun Chen, Binbin Liu et al.ACL Findings 2026 · 1 citation
  2. MANBench: Is Your Multimodal Model Smarter than Human?Han Zhou, Qitong Xu, Yiheng Dong, Xin YangACL Findings 2025
  3. Enhancing Cross-lingual Transfer by Manifold MixupHuiyun Yang, Huadong Chen, Hao Zhou, Lei LiICLR 2022 · 49 citations
  4. MUG-Eval: A Proxy Evaluation Framework for Multilingual Generation Capabilities in Any LanguageSeyoung Song, Seogyeong Jeong, Eunsu Kim, Jiho Jin et al.EMNLP Findings 2025
  5. XTREME: A Massively Multilingual Multi-task Benchmark for Evaluating Cross-lingual GeneralisationJunjie Hu, Sebastian Ruder, Aditya Siddhant, Graham Neubig et al.ICML 2020 · 1,132 citations
Dusk over the sea between two cliffs drawn in fine vertical lines