ACL Findings2026
PrinciplismQA: A Philosophy-Grounded Approach to Assessing LLM-Human Clinical Medical Ethics Alignment
Chang Hong, Minghao Wu, Qingying Xiao, Yuchi Wang, Xiang Wan, Guangjun Yu, Benyou Wang, Yan Hu
Abstract
As medical LLMs transition to clinical deployment, assessing their ethical reasoning capability becomes critical. While achieving high accuracy on knowledge benchmarks, LLMs lack validated assessment for navigating ethical trade-offs in clinical decision-making where multiple valid solutions exist. Existing benchmarks lack systematic approaches to incorporate recognized philosophical frameworks and expert validation for ethical reasoning assessment. We introduce PRINCI-PLISMQA, a philosophy-grounded approach to assessing LLM clinical medical ethics alignment. Grounded in Principlism, our approach provides a systematic methodology for incorporating clinical ethics philosophy into LLM assessment design. PRINCIPLISMQA comprises 3,648 expert-validated questions spanning knowledge assessment and clinical reasoning. Our expert-calibrated pipeline enables reproducible evaluation and models ethical biases. Evaluating recent models reveals significant ethical reasoning gaps despite high knowledge accuracy, demonstrating that knowledgeoriented training does not ensure clinical ethical alignment. PRINCIPLISMQA provides a validated tool for assessing clinical AI deployment readiness.
Ask Lune
Know this paper in minutes, not an afternoon.
Lune can read every page of it and the top-tier papers that cite it, and gives you its claims, its evidence and its limits, each one cited to the paper it comes from. No skimming, no AI slop.
Related papers
- PsychEthicsBench: Evaluating Large Language Models Against Australian Mental Health EthicsYaling Shen, Stephanie Fong, Yiwen Jiang, Zimu Wang et al.ACL Findings 2026 · 1 citation
- Advancing Automated Ethical Profiling in SE: a Zero-Shot Evaluation of LLM ReasoningPatrizio Migliarini, Mashal Afzal Memon, Marco Autili, Paola InverardiASE 2025
- Ethical Reasoning over Moral Alignment: A Case and Framework for In-Context Ethical Policies in LLMsAbhinav Rao, Aditi Khandelwal, Kumar Tanmay, Utkarsh Agarwal et al.EMNLP Findings 2023 · 32 citations
- Evo-PI: Aligning Medical Reasoning via Evolving Principle-Guided SupervisionXianda Zheng, Huan Gao, Meng-Fen Chiang, Michael J. Witbrock et al.ACL 2026
- Structured Moral Reasoning in Language Models: A Value-Grounded Evaluation FrameworkMohna Chakraborty, Lu Wang, David JurgensEMNLP 2025
