EMNLP Findings2024
FactAlign: Long-form Factuality Alignment of Large Language Models
Chao-Wei Huang, Yun-Nung Chen
Abstract
Large language models have demonstrated significant potential as the next-generation information access engines. However, their reliability is hindered by issues of hallucination and generating non-factual content. This is particularly problematic in long-form responses, where assessing and ensuring factual accuracy is complex. In this paper, we address this gap by proposing FACTALIGN, a novel alignment framework designed to enhance the factuality of LLMs' long-form responses while maintaining their helpfulness. We introduce fKTO, a fine-grained, sentence-level alignment algorithm that extends the Kahneman-Tversky Optimization (KTO) alignment method. Leveraging recent advances in automatic factuality evaluation, FACTALIGN utilizes fine-grained factuality assessments to guide the alignment process.
Our experiments on open-domain prompts and information-seeking questions demonstrate that FACTALIGN significantly improves the factual accuracy of LLM responses while also improving their helpfulness. Further analyses identify that FACTALIGN is capable of training LLMs to provide more information without losing factual precision, thus improving the factual F1 score. 1
Ask Lune
Know this paper in minutes, not an afternoon.
Lune can read every page of it and the top-tier papers that cite it, and gives you its claims, its evidence and its limits, each one cited to the paper it comes from. No skimming, no AI slop.
6 top-tier papers have cited it since. Know what they found before your reviewers do.
Lune can draw on
the full paper · the top-tier papers that cite it · related work
Free to start. No credit card required.
Lineage
- Model Alignment as Prospect Theoretic Optimization cites Direct Preference Optimization: Your Language Model is Secretly a Reward Model
- Alleviating Hallucinations from Knowledge Misalignment in Large Language Models via Selective Abstention Learning cites Direct Preference Optimization: Your Language Model is Secretly a Reward Model
- Direct Preference Optimization: Your Language Model is Secretly a Reward Model cites Language Models are Few-Shot Learners
- Precise Information Control in Long-Form Text Generation cites Direct Preference Optimization: Your Language Model is Secretly a Reward Model
- Precise Information Control in Long-Form Text Generation cites Finetuned Language Models are Zero-Shot Learners
- Inference-Time Intervention: Eliciting Truthful Answers from a Language Model cites Language Models are Few-Shot Learners
- Finetuned Language Models are Zero-Shot Learners cites Language Models are Few-Shot Learners
Cited by top-tier papers6
- Knowledge Boundary of Large Language Models: A SurveyMoxin Li, Yong Zhao, Wenxuan Zhang, Shuaiyi Li et al.ACL 2025 · 33 citations
- LoVeC: Reinforcement Learning for Better Verbalized Confidence in Long-Form GenerationCaiqi Zhang, Xiaochen Zhu, Chengzu Li, Nigel Collier et al.ACL 2026 · 16 citations
- Precise Information Control in Long-Form Text GenerationJacqueline He, Howard Yen, Margaret Li, Shuyue Stella Li et al.NeurIPS 2025 · 8 citations
- Alleviating Hallucinations from Knowledge Misalignment in Large Language Models via Selective Abstention LearningLei Huang, Xiaocheng Feng, Weitao Ma, Yuchun Fan et al.ACL 2025
- Embedding Trust: Semantic Isotropy Predicts Nonfactuality in Long-Form Text GenerationDhrupad Bhardwaj, Julia Kempe, Tim G. J. RudnerICML 2026
Builds on22
- Language Models are Few-Shot LearnersTom B. Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah et al.NeurIPS 2020 · 64,255 citations
- Direct Preference Optimization: Your Language Model is Secretly a Reward ModelRafael Rafailov, Archit Sharma, Eric Mitchell, Christopher D. Manning et al.NeurIPS 2023 · 10,924 citations
- Finetuned Language Models are Zero-Shot LearnersJason Wei, Maarten Bosma, Vincent Y. Zhao, Kelvin Guu et al.ICLR 2022 · 4,966 citations
- Inference-Time Intervention: Eliciting Truthful Answers from a Language ModelKenneth Li, Oam Patel, Fernanda B. Viégas, Hanspeter Pfister et al.NeurIPS 2023 · 1,549 citations
- Model Alignment as Prospect Theoretic OptimizationKawin Ethayarajh, Winnie Xu, Niklas Muennighoff, Dan Jurafsky et al.ICML 2024 · 973 citations
Related papers
- FLAME : Factuality-Aware Alignment for Large Language ModelsSheng-Chieh Lin, Luyu Gao, Barlas Oguz, Wenhan Xiong et al.NeurIPS 2024 · 63 citations
- MR-ALIGN: Meta-Reasoning Informed Factuality Alignment for Large Reasoning ModelsXinming Wang, Jian Xu, Bin Yu, Sheng Lian et al.ACL Findings 2026
- Self-Alignment for Factuality: Mitigating Hallucinations in LLMs via Self-EvaluationXiaoying Zhang, Baolin Peng, Ye Tian, Jingyan Zhou et al.ACL 2024
- Mask-DPO: Generalizable Fine-grained Factuality Alignment of LLMsYuzhe Gu, Wenwei Zhang, Chengqi Lyu, Dahua Lin et al.ICLR 2025
- InFact: Informativeness Alignment for Improved LLM FactualityRoi Cohen, Russa Biswas, Gerard de MeloEMNLP Findings 2025
