EMNLP Findings2024
BookWorm: A Dataset for Character Description and Analysis
Argyrios Papoudakis, Mirella Lapata, Frank Keller
Abstract
Characters are at the heart of every story, driving the plot and engaging readers. In this study, we explore the understanding of characters in full-length books, which contain complex narratives and numerous interacting characters. We define two tasks: character description, which generates a brief factual profile, and character analysis, which offers an in-depth interpretation, including character development, personality, and social context. We introduce the BOOKWORM dataset, pairing books from the Gutenberg Project with human-written descriptions and analyses. Using this dataset, we evaluate state-of-the-art long-context models in zero-shot and fine-tuning settings, utilizing both retrieval-based and hierarchical processing for book-length inputs. Our findings show that retrieval-based approaches outperform hierarchical ones in both tasks. Additionally, finetuned models using coreference-based retrieval produce the most factual descriptions, as measured by fact-and entailment-based metrics. We hope our dataset, experiments, and analysis will inspire further research in character-based narrative understanding.
Ask Lune
Know this paper in minutes, not an afternoon.
Lune can read every page of it and the top-tier papers that cite it, and gives you its claims, its evidence and its limits, each one cited to the paper it comes from. No skimming, no AI slop.
2 top-tier papers have cited it since. Know what they found before your reviewers do.
Lune can draw on
the full paper · the top-tier papers that cite it · related work
Free to start. No credit card required.
Lineage
- LoRA: Low-Rank Adaptation of Large Language Models cites Deberta: decoding-Enhanced Bert with Disentangled Attention
- Big Bird: Transformers for Longer Sequences cites Reformer: The Efficient Transformer
- Deberta: decoding-Enhanced Bert with Disentangled Attention cites Reformer: The Efficient Transformer
Cited by top-tier papers2
Builds on18
- LoRA: Low-Rank Adaptation of Large Language ModelsEdward J. Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu et al.ICLR 2022 · 18,833 citations
- BERTScore: Evaluating Text Generation with BERTTianyi Zhang, Varsha Kishore, Felix Wu, Kilian Q. Weinberger et al.ICLR 2020 · 8,443 citations
- Deberta: decoding-Enhanced Bert with Disentangled AttentionPengcheng He, Xiaodong Liu, Jianfeng Gao, Weizhu ChenICLR 2021 · 3,729 citations
- Big Bird: Transformers for Longer SequencesManzil Zaheer, Guru Guruganesh, Kumar Avinava Dubey, Joshua Ainslie et al.NeurIPS 2020 · 3,159 citations
- Reformer: The Efficient TransformerNikita Kitaev, Lukasz Kaiser, Anselm LevskayaICLR 2020 · 2,878 citations
Related papers
- NarrativeXL: a Large-scale Dataset for Long-Term Memory ModelsArsenii Moskvichev, Ky-Vinh MaiEMNLP Findings 2023
- "Let Your Characters Tell Their Story": A Dataset for Character-Centric Narrative UnderstandingFaeze Brahman, Meng Huang, Oyvind Tafjord, Chao Zhao et al.EMNLP Findings 2021 · 20 citations
- Learning and Evaluating Character Representations in NovelsNaoya Inoue, Charuta Pethe, Allen Kim, Steven SkienaACL Findings 2022 · 5 citations
- Think Before you Write: QA-Guided Reasoning for Character Descriptions in BooksArgyrios Papoudakis, Mirella Lapata, Frank KellerACL Findings 2026
- Personality Understanding of Fictional Characters during Book ReadingMo Yu, Jiangnan Li, Shunyu Yao, Wenjie Pang et al.ACL 2023 · 3 citations
